Testing Accuracy and Repeatability of UAV Blocks Oriented with GNSS-Supported Aerial Triangulation

Abstract

UAV Photogrammetry today already enjoys a largely automated and efficient data processing pipeline. However, the goal of dispensing with Ground Control Points looks closer, as dual-frequency GNSS receivers are put on board. This paper reports on the accuracy in object space obtained by GNSS-supported orientation of four photogrammetric blocks, acquired by a senseFly eBee RTK and all flown according to the same flight plan at 80 m above ground over a test field. Differential corrections were sent to the eBee from a nearby ground station. Block orientation has been performed with three software packages: PhotoScan, Pix4D and MicMac. The influence on the checkpoint errors of the precision given to the projection centers has been studied: in most cases, values in Z are critical. Without GCP, the RTK solution consistently achieves a RMSE of about 2–3 cm on the horizontal coordinates of checkpoints. In elevation, the RMSE varies from flight to flight, from 2 to 10 cm. Using at least one GCP, with all packages and all test flights, the geocoding accuracy of GNSS-supported orientation is almost as good as that of a traditional GCP orientation in XY and only slightly worse in Z.

1. Introduction

1.1. The Impact of UAVs on Photogrammetry and Remote Sensing

Unmanned Aerial Vehicles (UAVs) are proving invaluable and exceptionally flexible carriers for sensors, superseding established technologies, enlarging the possibilities of existing ones and opening new fields of application, see [1,2] for two comprehensive reviews. Their impact has been strong in Remote Sensing (RS) with precision farming [3,4] being the main but not unique beneficiary [5,6]. Even stronger has been the influence on photogrammetry: perhaps a bit emphatically, the term UAV photogrammetry [7] has been coined. With fixed costs negligible compared to those of manned aircrafts and a Ground Sampling Density (GSD) in the cm range, UAVs are both economically and technically well suited to act as platform for photogrammetric surveys and are quickly building their place among surveying techniques [8]. Thanks to the effectiveness of Structure from Motion (SfM) and Dense Matching algorithms [9,10] in Digital Surface Model (DSM) production [11,12,13,14], the survey of stockpiles, open pit mines, earthworks, and corridors, in the past the preserve of terrestrial or airborne laser scanning (LS) or of manned airborne photogrammetry, is now increasingly carried out using UAVs [15,16,17,18]. Production and updating of large scale maps (including cadastral ones [19]) have also seen a shift from manned to unmanned aerial surveys, with demand for larger and larger areas to be covered. Use of UAVs in the survey of terrain movements, such as in landslides [20,21], glaciers [22,23] and geomorphology [24], is also expanding.

To provide a rough indication of the accuracy requirements of the above-mentioned applications is indeed quite difficult. It may range from 3–5 cm to 20–30 cm in horizontal coordinates as well as elevation, with the most demanding ones as far as elevation is concerned being survey of stockpiles, geomorphological studies (erosion and deposition) and earthworks. However, even stricter requirements might apply, e.g., in elevation, and demand for high precision surveys is likely to grow. With dense and accurate ground control, a horizontal accuracy of 0.5–1.0 GSD and a vertical one of 1.5–2 GSD should be achievable in DTM generation from UAV [25], though in practice a large range of values is reported [2]. Consistently and reliably delivering such accuracies [26] is nowadays one of the challenges in UAV photogrammetry.

Both the fixed and rotary wing versions of UAVs are improving in performance (more endurance and wind resistance), safety equipment (collision avoidance systems) and operational modes (parametrized flight plans). The regulations enacted by the flight authorities on the commercial use of drones are leading to higher product standards as well as to specialization of vehicles. On the other hand, they still are rather restrictive in the limits for visual line of sight operation, flying height above ground level (AGL) and flying over urban areas. However, some progress is being made. For the Italian Aviation Authority (Ente Nazionale per l’Aviazione Civile (ENAC)), specialized operations of UAVs with takeoff mass below 2 kg are considered not critical if the hardware meets certain conditions [27]. In USA a new version of Part 107 [28] for UAVs under 25 kg by the Federal Aviation Administration has been approved in June 2016: Notification to AirMan (NOTAM) is not required any more; there can be exceptions to AGL limits; and waivers are foreseen for operation from moving vehicles. A comprehensive discussion of the impact of regulations on UAVs can be found in [2,29].

The demand for quick delivery of products and largely automated procedures in surveying and RS fosters turnkey non-customizable systems, with tight integration between payload sensors and the on-board navigation system.

1.2. Trends in Block Georeferencing with UAV Photogrammetry

In manned aerial photogrammetry block orientation can be performed in different ways (see

Table 1. Block orientation methods in aerial photogrammetry.

Method	Observations	BBA	Block Deform. Control	Reliable Self-Calibration
Indirect Sensor Orientation (InSO)	- Image coordinates of tie points and GCP - Object coordinates of GCP	Yes	Yes	Yes
Direct Sensor Orientation (DSO)	- Camera station position and attitude	No	No	No
Integrated Sensor Orientation (ISO)	- Image coordinates of tie points and GCP - Camera station position and attitude - Object coordinates of (a few) GCP (opt.)	Yes	Yes (with GCP)	Only with some GCPs
GNSS-supported Aerial Triangulation (GNSS-AT)	- Image coordinates of tie points and GCP - Camera station position - Object coordinates of (a few) GCP (opt.)	Yes	Yes (with GCP)	Only with some GCPs

): by Indirect Sensor Orientation (InSO) with tie points and Ground Control Points (GCPs) [30] or by Direct Sensor Orientation (DSO) [2,31,32] where the exterior orientation parameters of each image are computed by processing the on-board GNSS and Inertial Measurement Unit (IMU) data so there is no need for tie points or GCPs.

In practice, Integrated Sensor Orientation (ISO) [33], a fusion of both techniques, is often used nowadays. In ISO, a Bundle Block Adjustment (BBA) is executed that includes as observations, besides the tie points, all exterior orientation data (camera positions and attitudes) as well as GCPs, if any.

Knowledge of the exterior orientation data can be exploited to improve the efficiency of tie point extraction, which in turn improves the relative orientation between images compare to DSO. If only GNSS data are available (or the IMU is not accurate enough), ISO degrades to GNSS-supported Aerial Triangulation (GNSS-AT) [34,35,36].

The drive towards DSO or ISO in UAV photogrammetry was apparent from that start because an on-board navigation system is already present. However, such systems, equipped with inexpensive single-frequency GNSS receivers and accelerations and rotation rates measured with Micro Electro Mechanical Systems (MEMS), are not accurate enough for DSO in the decimeter range [37,38]. Therefore, InSO is currently the norm in block orientation and control.

In most cases, targets are necessary, since outside urban areas it is difficult to find easy-to-measure and well defined natural features of appropriate size. The optimal distribution and the accuracy requirements of GCP in UAV blocks have not yet been fully assessed, though more and more studies are being devoted to the topic [12,26,39,40]. It is agreed upon that control information must be included as constraint in a BBA rather than just applying a rigid 3D similarity transformation from an arbitrary reference system after SfM.

GCPs are measured in most cases with GNNS surveys in RTK mode, either with respect to a nearby base station or with the differential corrections being sent by a Network of permanent GNSS stations (NRTK). In normal conditions, this technique should deliver horizontal coordinates accurate to 1–2 cm and elevations accurate to 2–3 cm, values often very close to the GSD of UAV images.

Dispensing with GCPs altogether would be a quite big step forward for both RS and photogrammetric surveys with UAVs, not only for the time savings in the field and in office, but also because sites where GCPs cannot be placed or measured easily or safely (e.g., in forests or mountain slopes) could also be surveyed.

1.3. Requirements for DSO, ISO or GNSS-AT

Using DSO or ISO with UAV blocks requires better-than-usual hardware components of the navigation system as well as a comprehensive system calibration. Moreover, camera calibration should be considered carefully, since block control moves from GCP to camera exterior orientation parameters: residual errors in interior orientation parameters are likely to affect the ground coordinates rather than being absorbed by exterior orientation parameters as in InSO [41].

1.3.1. Hardware and System Calibration

To compute the antenna trajectory with cm-level accuracy, GNSS-AT requires phase observations to be recorded by a dual-frequency receiver that can fix the integer ambiguity in about 1 min rather than the 5–10 min normally necessary to single-frequency receivers [42]. Although a few successful tests have also been performed with single-frequency receivers and rotary wing UAVs [43], such long fixing times make them hardly operational. As a radio link with the ground control station is available anyway, RTK positioning should be desirable, because the positioning quality can be checked in real time or just after the flight. However, recording raw GNSS data is also desirable as a backup in the case of radio link problems or to refine the camera station positions through kinematic data post-processing (PPK).

DSO requirements are the same as GNSS-AT on the GNSS side, while more accurate measurement of rotation rates are necessary on the IMU side. Rehak and Skaloud built a prototype system [44] equipped with a dual-frequency receiver and a so-called redundant IMU, where several MEMS are software-combined to improve their overall precision. After system calibration and processing in a Kalman Filter of the navigation data, a DSO solution with RMS errors on checkpoints ranging from 2 to 5 cm has been reported.

Finally, system calibration, i.e., the accurate synchronization between camera and navigation system, the determination of the exposure time of each image and of the lever arm (offset) between camera and the GNSS antenna phase centere [39,45], is required. In the case of DSO, the determination of the boresight angles between camera and IMU axes is also necessary. Two lever arm and camera calibration procedures for DSO and ISO with UAVs are presented in [46]: without using GCPs, bias in the estimation of interior orientation parameters (especially the principal distance) have been found. At least one GCP was found necessary to improve the reliability of results.

1.3.2. Camera Calibration

UAV camera calibration in InSO is discussed in several papers [2,39,40,47]. It is generally agreed that, unless a precalibration is executed on-site, just prior or just after the flight, a robust self-calibration procedure is better, due to inherent instability of the consumer cameras used in UAVs. Conditions for successful self-calibration are highly accurate and dense GCPs and, for rotary wings, the inclusion of oblique imaging [47,48]; for fixed wing, flying cross strips at a different elevation is advised [39,41] to reduce projective coupling between interior and exterior orientation parameters and to avoid deformations in the DSM.

In [39], self-calibration is used in GNSS-AT of a rectangular block with parallel and cross strips at a different elevation. Three cases are considered, where a basic configuration (GNSS data only, no GCP nor cross strips) is progressively strengthened first introducing cross strips at a different elevation and then introducing a varying number of GCPs. In the basic case, the RMSE in the horizontal plane is slightly larger than in elevation. If no cross strips are used, adding the GCPs does not bring real benefits, unless their number is large. The effect is felt mainly in elevation, with a 20% improvement in accuracy. Introducing the cross strips without GCP, the errors are overall smaller than in the best case with no cross, in XY as well as in elevation; adding the GCP improves marginally only the elevations.

Whether it is realistic to pursue the goal of a reliable and highly accurate camera calibration in GNSS-AT with UAVs images is not yet clear: in [49,50], different calibration methods led to quite different principal distance estimates in blocks with standard ground control. It should be noted that, as in most UAV blocks image scale ranges from 1:10.000 to 1:25.000, just one micrometer change in the principal distance would change the elevations by 1.0 cm to 2.5 cm in ISO, DSO and GNSS-AT.

The rationale for producing RTK-enabled UAVs is to avoid surveying any GCP or, if really necessary, to keep them to the minimum that delivers the same accuracy level as GCP-controlled blocks. GNSS kinematic surveys might suffer from false ambiguity fixing that cause systematic positioning errors (just think of airborne laser scanning, where despite top level GNSS/IMU integrated systems, the strips are still registered in elevation, to remove or highlight offsets). Whenever high accuracy and a good confidence level on the outcome are sought, it is therefore just common sense to provide control or check information on the ground.

1.4. Previous Work on the Accuracy of Block Orientation with RTK-Enabled UAVs

Still, relatively few papers, to the best of our knowledge, report on the empirical accuracy found for RTK-enabled UAVs. Most refer to prototype systems, while a couple examined the performance of commercial systems. In [51], a prototype system is presented on an OptoKopter platform, equipped with a Novatel Flexpak-G2L L1 receiver interfaced to a SLR Canon 550 D. The camera positions are determined with an estimated accuracy of 10–20 cm by post-processing GNSS observations; no details are given on the time needed to solve for the integer ambiguity. Data from a test flight were processed with PhotoScan, Pix4D and Bundler, checking the horizontal accuracy on orthomosaics on 22 checkpoints. Depending on the software package, the mean absolute error ranges from 11 to 76 cm.

In [43], a low cost system is presented, with an inexpensive Ublox LEA-6T L1 receiver, a comparatively expensive SONY RX1 camera and a DJI F550 exacopter. Though not much detail on the test block is provided, an accuracy on 10 checkpoints of about 2.4 cm in XY and 1.6 cm in elevation is reported.

In [39], besides camera calibration, a comparison between InSO and GNSS-AT is presented for a large block of 1900 images flown over a stockpile area of 0.66 km² with a MAVinci Sirius with a GSD of 2.7 cm. In the block adjustment, performed with PhotoScan, an a-priori standard deviation of 2 cm and 3 cm respectively is assigned to the camera horizontal and vertical coordinates; inaccurate camera position are highlighted. Overall, if cross strips are used, the horizontal and vertical RMSE on the checkpoints with GNSS-AT without GCPs (4.2 cm and 5.4 cm respectively) are slightly better than a traditional adjustment with GCPs (3.3 cm and 7.3 cm respectively). Without cross strips, the combined horizontal and vertical errors are equivalent, but elevation is still more accurate (6.7 cm against 8.1 cm) with GNSS-AT than with InSO. This is remarkable, since the opposite could be expected.

In [52] also a comparison between InSO and GNSS-AT is presented, with the latter performed in two ways: with an on-board low-grade receiver or a survey-grade receiver. Two senseFly eBees, one of them RTK-enabled, were used for image acquisition over a 0.48 km² gravel pit; the Postflight Terra 3D package was used for block adjustment and DSM and orthomosaic generation. The horizontal accuracy has been assessed measuring 17 targets on the orthoimages, while the vertical accuracy was verified on a larger set of 180 points by interpolation of the DSM elevation. A total horizontal error (RMSE) of 3.2 cm and 3.4 cm was found, respectively, with InSO and GNSS-AT. On the other hand, the vertical error of the DSM from GNSS-AT orientation was 8.9 cm, about three times greater than the DSM from InSO (3 cm). The authors therefore recommend using GCPs, especially for periodic surveys of morphological changes.

Two white papers from MAVinci [53] and senseFly [54] also report on accuracy tests: here we refer to validation of block orientation on checkpoints. In the MAVinci paper, dated April 2016, it is mentioned that the software package PhotoScan has a plugin, available to MAVinci users only, where camera positions are filtered after image matching. In the test site, performed on each unit delivered to customers, images are acquired over a flat area at about 90 m and with a GSD of 2.2 cm. The RMSE reported on six checkpoints is 4.3 cm in horizontal and 4.7 cm in elevation. The senseFly paper reports more articulated tests, carried out also in varying ambient conditions (wind and ambient light) and introducing errors in the 10% of the antenna positions. The BBA with camera self-calibration has been executed using Postflight Terra 3D software. In the test for maximum accuracy, achieved in ideal conditions from a flight with GSD of 2.5 cm from about 80 m with cross strips, a RMSE on 19 checkpoints of 2.6 cm in horizontal coordinates and 3.1 cm in elevation is reported.

The paper presents the results of a study on the accuracy and repeatability of UAV blocks collected and georeferenced with a senseFly eBee RTK. To this aim, four flights have been executed over a test field with 26 signalized known points, according to the same flight plan. Rather than trying to find the best accuracy achievable, the goal is to look at the accuracy gap, if any, between the same block oriented with InSO or with GNSS-AT.

The paper is structured as follows: in Section 2 (Materials and Methods) information about the test site, the UAV features, the reference network survey, and the characteristics of the different software packages used in the orientation of the blocks are provided. Then, the topics of the investigation are presented: the BBA strategy (using only GCP, using only GNSS camera data or a combination of the two), the influence of a-priori standard deviation assigned to camera position, GCPs and image coordinates in the BBA, the interaction between interior and exterior orientation with on-the-job camera calibration in the different software packages. Section 3 (Results) reports and discusses the most relevant results obtained from the experiments, with more details in Appendix A.

2. Materials and Methods

2.1. Test Site Description

Although this study is rather general in its purpose, not addressing specific requirements or application scenarios, some consideration on the fields where RTK-enabled UAVs can provide a significant breakthrough with respect to other survey techniques should be made, at least in order to establish the extent of the test area. In many countries, national UAV flight regulations limit the area that can be covered in a single mission: for example, in Italy, the National Commercial Aviation Authority mandates that the pilot maintains a line of sight with the UAV and the flying area is smaller than 500 × 500 m².

Though the potential benefits of dispensing with ground control network survey obviously apply to every UAV survey, such technology can be preeminently employed, in authors’ opinion, in cartographic update and monitoring of small areas. Studies in the scientific and technical literature on the costs of UAS cartographic surveys as a function of the area size are still missing. As the area increase so does the time necessary to complete the survey, due to the short operating time range of the majority of these devices (except, perhaps, the fuel-powered ones that are however less and less used in these applications). However, in this context, the real limit is so far represented by the ground survey costs: the small format of the digital cameras commonly used on board, their optical instability, the (sometimes) relevant lack of homogeneity of image coverage and overlaps, the low flying altitude and the subsequent small GSD requires a very dense GCP distribution to prevent block deformations.

For all these reasons, the survey was restricted to an area of about 500 × 400 m²; this size might well represent a case where map updating with UAVs can efficiently substitute a manned photogrammetric flight or the extent of the vast majority of monitoring applications (landslides, glacier displacement, construction site monitoring, etc.) the practitioners might encounter in their activity. The area of study (see Figure 1) covers part of the Campus of Parma University, for a total of about 200,000 m² and consists of parking lots, green areas, sporting facilities as well as buildings of various heights (from 6 to 35 m), with a mixture of countryside and suburbs.

Reference Network and GCP

To provide a ground control (and check) network to reliably evaluate the performance of GNSS-AT with the eBee RTK and compare the results with those obtained by traditional GCP-based orientation, 26 signalized targets were deployed and surveyed just before the UAV flights. As long as buildings, streets and parking lots in the area permitted it, the targets were uniformly distributed in the area. As the area includes buildings (up to 35 m high), some of the ground points were located on the top of such structures, so long as the operator could safely access them.

The GCP and the Checkpoints (CP) have been surveyed with two different receivers (a Leica 1200 and a Geomax Zenith 20) by repeating the point occupation at different times and by using three different kinematic GNNS techniques: in RTK mode and in post-processing mode with respect to a reference station set on a known point within the block area as well as in NRTK mode with respect to the Netgeo network, previously used to reference the known point with several static sessions. Each point has been surveyed on average 4 times, averaging the position or, in the case of a larger number of reoccupations, applying a trimmed mean. The average standard deviation of the repetitions on each point is 7 mm for the horizontal position and 11 mm for the elevation.

2.2. Data Acquisition

Data acquisition was performed with a fixed wing senseFly eBee RTK. The eBee RTK, made of expanded polypropylene foam with carbon structure and some composite parts, weighs approximately 0.73 kg and has a wingspan of 96 cm. The thrust is provided by an electric pusher propeller (160 W brushless DC motor). The system incorporates a compact camera Sony Cyber-Shot DSC-WX220 CMOS Exmor R sensor 18.2 Mp and 4.5 mm focal length (corresponding to a 35 mm equivalent focal length of 25.6 mm) fully triggered by the drone’s autopilot. The camera acquires regular nadiral image data in the visible spectrum (RGB camera with 4896 × 3672 pixels resolution) with exposure parameters set automatically.

The on-board receiver can process GPS and GLONASS L1 and L2 signals, with differential corrections sent from the master station via its flight control software and ground radio modem. Precise geo-tags are stored in the image metadata and in the flight log. A Geomax Zenith20 was used as master station, connected to ground station by a RS232 cable. The senseFly Emotion2 software has been used for flight control and RTK corrections transmission.

Flight planning has been carried out trying to achieve the highest precision allowed by the UAV platform, taking into account flying height and overlap, to better discriminate between the errors due to intrinsic block accuracy (e.g., image matching errors), and those caused by the orientation strategy (including camera center precision). The frame rate of the camera and the minimum flying speed of the UAV, however, impose some limitation on the base length and therefore the flying height: with a too small camera-to-object distance, the base length might be too small for the camera to shoot fast enough to guarantee the necessary forward overlap. Buildings in the area are up to 35 m high, so considering a conservative camera frame rate (0.5 Hz) and an average speed of 12.5 m/s, a flight height of 80 m was computed to allow a worst-case overlap of ca. 60%. At ground level, this corresponds to an overlap of about 85%; an 80% side lap, a rather common value in UAV flight plans, was used. The blocks consist of 12 strips, with a total number of about 150 images and a GSD of about 2.3 cm. In

Table 2. Flights environmental conditions, geometry of satellite sky coverage and specifications.

	Mean Wind Speed During Flight (m/s)	Wind Direction During Flight (° from North)	PDOP (Min–Max)	Images
RTK1	2.2	320°–40 °	2.4–2.9	151
RTK2	2.9	90°	1.8–2.0	152
RTK3	3.6–4.1	90°	1.7–1.8	144
RTK4	3.3–3.7	90°	1.7–1.9	149

, the flight conditions and the satellite constellation geometry described by the Position Dilution Of Precision (PDOP) of the four flights are summarized.

All four datasets were acquired on 29 August 2016, at regular intervals, between 12:00 a.m. and 6:15 p.m. In each flight, image collection took on average 11 min to be completed. The satellite constellation was always good, with PDOP values below 3 in the first flight, and below 2 in the others.

2.3. Photogrammetric Data Processing

The accuracy of GNSS-AT of UAV blocks depends critically on the BBA. Different software packages implement different optimization procedures, often using to some extent filtering algorithms to remove outliers and gross errors as well as different camera models and on-the-job calibration procedures. In GNSS-AT the relative weighting of the different observations: pixel coordinates of tie points, camera station positions and, if any are used, ground control object coordinates is of key importance. Basically, no commercial software package provides detailed insights on the routines used in all these stages, for comprehensible reasons of intellectual property protection. Very good open-source packages are available; however, to examine in depth the implemented algorithms, the user must invest a huge amount of time and resources to analyze the software code (which often is not commented extensively). In-house implementations allow full processing control and the highest level of routine customization; however, they lack sometimes the efficiency and ease-of-use of commercial packages.

In order to evaluate the influence on the results of the software package, data processing has been performed using an open-source distribution (MicMac) and two commercial packages (Agisoft PhotoScan and Pix4Dmapper). The BBA CALGE, an in-house development, has been used to collect specific information (such as correlations between interior and exterior orientation parameters) not provided by the above-mentioned commercial products.

2.3.1. MicMac

APERO and MicMac are two open source tools developed at IGN (Institut National de l’information Géographique et Forestière) that perform all the steps of a photogrammetric process, starting from SfM up to dense point clouds and orthophotos generation. APERO uses both the computer vision approach (SfM) for the estimation of an initial solution and photogrammetry for a rigorous compensation of the total error [55] through a BBA [56]. Block orientation with the navigation parameters recorded by the UAV during flight (ISO or GNSS-AT) is implemented.

The software provides statistical information on the data and allows detailed analysis of the photogrammetric processing results. Moreover, all the parameters and the results of the orientation and matching steps are stored in XML files which can be adapted whenever the user needs to change processing parameter settings. Incorporation of camera position and attitude in the BBA is available within an extended mathematical model, where estimation of the lever arm components and of the offset between actual and registered shooting time as well as camera trajectory interpolation are included.

2.3.2. Agisoft PhotoScan

PhotoScan (PS) is a commercial product, developed by Agisoft LLC Company. It has a very effective graphical user interface and, as MicMac, performs both block orientation and dense stereo matching using a multi-image approach. Although the software company spent a lot of efforts providing the user with more and more information on the processing results, and with several tools to analyze the matching outcome and camera calibration parameters, PhotoScan is still considered by many (not unreasonably, if compared to the customization obtainable with other software packages, e.g., MicMac) a sort of “black-box” software. All the photogrammetric process is performed with a high level of automation where the user can intervene on just few parameters: e.g., the maximum number of feature points extracted during SfM. The workflow is therefore extremely intuitive also for less experienced users. Due to commercial reasons very few information about the used algorithms are available: some details can be recovered from the PhotoScan User forum [57]. However, through a Python API and the several exporting options available, a more experienced user can fine-tune much more the procedures and integrate its efficient matching routines in more sophisticated processing pipeline, interacting with other software packages. As far as GNSS-AT or ISO are concerned, the standard deviation of each coordinate and attitude angle for each camera position can be set individually or by default values.

2.3.3. Pix4Dmapper Pro

Pix4Dmapper Pro by Pix4D [58] is a Swiss Computer Vision-based software developed at the École Polytechnique Fédérale de Lausanne (EPFL). Pix4D user-defined settings include the projection centers positioning accuracies and choice of the camera model. If the image geolocation information is stored in the image EXIF at each camera station, it is automatically loaded for the BBA together with the estimated position accuracy.

Though processing is automated, the user can set some options for the SfM, the BBA and the camera self-calibration. Tie points extraction is executed by feature matching with SIFT operator [59]. All the estimated parameters and the results of the orientation and matching processes are stored in the project output folder.

2.3.4. CALGE

CALGE [22,45] is a general purpose network adjustment software originally developed at Politecnico di Milano and now maintained and upgraded at the University of Parma. It can adjust data from topographic measurements (total stations and spirit leveling) and photogrammetric measurements (by bundles or by independent models) either separately or jointly. The Fraser camera model [60] is implemented for self-calibration. In addition to ground control points, block georeferencing and control can be provided by camera position and attitude obtained by GNSS/IMU pre-processed data. Each camera coordinate and orientation angle can be assigned a different precision or default values. Correlations between the interior orientation parameters of a camera, between the exterior orientation parameters of each camera station and between the exterior orientation parameters of each camera station and the camera interior orientation parameters are computed.

2.4. Description of the Tests

The general goal of the experiments is to evaluate the accuracy obtained on checkpoints with GNSS-AT and compare it with the accuracy of block orientation performed with GCPs only. The accuracy is evaluated computing the differences between the coordinates of the checkpoints (CP) estimated in BBA and those measured with a GNSS receiver. Besides the Root Mean Square Error (RMSE) of the differences, also the mean values and the standard deviations are computed.

The influence on the results of the software package used, particularly as far as the assignment of the standard deviations of the camera positions in GNSS-AT is concerned (Section 2.4.1), has been investigated. SfM has been performed separately in each software package, setting a high accuracy level in tie point extraction. To the contrary, in order to render the comparison among the three packages independent of the manual collection of image observations, the image coordinates of the 26 targets were measured with PhotoScan, imported in Pix4D and MicMac, and then used as GCP or CP according to the specific characteristics of the experiment.

To find out whether or under which conditions GNSS-AT delivers the same accuracy as InSO, three experiments were performed varying the block control configuration (Section 2.4.2).

In all cases, a self-calibrating BBA has been executed.

Table A1. Estimated camera calibration parameters in the BBA for each software package.

	Interior Orientation and Self-Calibration Parameters
PIX4D	Principal distance, Principal point coordinates, radial distortion parameters K1, K2, K3, tangential distortion parameters P1, P2
PhotoScan	Principal distance, Principal point coordinates, radial distortion parameters K1, K2, K3, tangential distortion parameters P1, P2
MicMac	Polynomial model (see MicMac manual [61])

in Appendix A reports the list of parameters that have been estimated: in PhotoScan and Pix4D the same set has been considered, while in MicMac a polynomial model has been used. The interaction between interior and exterior orientation parameters has been investigated to highlight the behavior differences of the three packages.

2.4.1. Influence of the Standard Deviation assigned to Projection Centers in the BBA

As already noted, in the BBA of UAV blocks with GNSS-AT an important issue is the assessment of the camera coordinate precisions at each shooting time, to define the weights of the pseudo-observations equations with respect to that of the collinearity equations on tie points. Processing of the GNSS observations also delivers an estimate of such precision; however, it is well known that they are normally quite optimistic and that perhaps only the ratios of the standard deviations reflect to some extent the different positioning quality of two camera stations. Failure to assign the appropriate weight might lead to bias in the solution.

Depending on the on-board software implementation, the nominal precision of the camera position can be encoded in the image EXIF header or saved in a separate file. For instance, the eBee RTK only stores this info in the image EXIF; interestingly, out of the three software packages, only Pix4D is able to recover this information from the eBee images. As far as the actual estimated precision of the RTK data processing for the 4 flights is concerned, the values recorded are fairly stable within each flight (with just a few mm of variation in all coordinates) while the mean values range from 1 cm to 1.5 cm in X and Y and from 1.8 cm to 2.5 cm in Z.

To gain some insight on how the standard deviations of camera stations are used in the BBA, in a first series of tests a systematic exploration of the block orientation accuracy with GNSS-AT has been carried out for each software package and each block. Using the default setting, the standard deviation of the camera coordinates in XY has been varied in the range from 1 cm to 20 cm, while the elevation standard deviation has been set twice as much as the horizontal one, to account for the gap in accuracy of GNSS positioning (i.e., from 2 to 40 cm). The standard deviation assigned to the tie points image coordinates has been kept fixed at 1 pixel, a value that turned out to be practically coincident with the average value of the mean reprojection error of the BBA with all packages and in all blocks.

In a second set of tests, only executed on PhotoScan using a Python script, both the standard deviations of the camera positions and of the tie points have been varied respectively in the (admittedly large) range from 1 to 20 cm and from 0.01 to 4.0 pixels.

For each adjustment of these two tests, the RMSE in the horizontal coordinates and in elevation have been computed: in these experiments, the coordinates of all the 26 known points have been used as CP.

2.4.2. eBee RTK Survey Accuracy Assessment

To find out whether or under which conditions GNSS-AT delivers the same accuracy as InSO, three experiments were performed varying the block control configuration. In the first one, referred to as 12GCP in the following, only GCP were used, fixing the coordinates of 12 points distributed in the survey area as in a standard InSO (see Figure 1). In the second one (RTK for short hereinafter), only the camera positions from the navigation data (without any GCP) were used. In the last one (RTK+1GCP), besides the camera positions, the coordinates of one GCP (GCP No. 13 in the middle of the block, see Figure 1) have also been fixed. This minimal, and practically quite acceptable, block control configuration has also been tested as remarks made in Section 1.3 and Section 1.4 suggest that camera self-calibration might be inaccurate, with bias affecting the ground coordinates, unless GCP are used.

All BBA have been executed assigning a standard deviation of 1 pixel to the tie points image coordinates, as mentioned in Section 2.4.1. As far as GCP are concerned, the empirical precision found from repeated measurements of GCP is in the order of 1 cm. However, a much lower standard deviation of 1 mm has been assigned to the coordinates of the GCPs as it has been found that, in PhotoScan, with 1 cm standard deviations, all GCP coordinates would otherwise show residuals much larger than expected (up to 10 cm). That is probably due to the overwhelming influence of the collinearity equations: since SfM extract huge number of tie points, the weight of the GCP coordinates needs strengthening to be effective. The BBAs for RTK and RTK+1GCP have been executed considering, in all three packages, camera position standard deviations between 1–2 cm in XY and 3–5 cm in elevation. This choice has been made based on the available information from the EXIF and the remarks in Section 2.4.1 (see also the discussion in Section 3.1).

In all of these experiments, since the reference solution is 12GCP, the coordinates of the remaining 14 known points have been used as CP in the computation of the RMSE.

2.4.3. Interaction between Interior and Exterior Orientation Parameters

To gain some insight on how the camera position data are treated in the BBA and affect the block orientation, attention should be paid not only to the CP ground coordinates, but also to the projection center positions and to the IO parameters. In all flights, the interior orientation parameters have been estimated with self-calibration; therefore, changes in the IO parameters, in particular to the principal distance, and in the camera heights in the three control configurations might highlight how the correlations between the interior and exterior orientation parameters influence the solution. As in most commercial packages, this information is usually not available; it has been retrieved using the CALGE BBA routine, which can output the correlation matrix for the estimated interior and exterior orientation parameters.

3. Results and Discussion

3.1. Influence of the Standard Deviation Assigned to the Observations

The results of varying the standard deviation of the camera coordinates on the accuracy of block orientation in the RTK configuration are shown in the graphs of Figure 2, where the RMSE for the horizontal coordinates and for the elevation has been computed on all the 26 available checkpoints. Note that the standard deviation value read on the abscissa is always the one assigned to the XY coordinates of a particular BBA, as well as for the plots referring to the Z coordinate.

The variety of the trends from program to program and from test flight to test flight is large. As the XY and Z graphs do not necessarily show their minimum at the same value of standard deviation, an optimization of the standard deviation should be performed for XY and Z coordinates, as GNSS measurements are less accurate in height than in latitude and longitude. Overall, with all packages the RMSE is barely affected by the precision assigned to the horizontal coordinates (especially in MicMac), though generally the larger the standard deviation the larger the error. With PhotoScan, however, in one case (the RTK1 flight) the RMSE range in XY is larger than 3 cm. The accuracy in elevation, to the contrary, is strongly affected by changes in the standard deviation assigned to the camera Z coordinate: the RMSE range is up to 8 cm with PhotoScan and Pix4D and much larger for MicMac. A common trend is hard to identify, even for the same package: in some cases, the RMSE shows a well-defined minimum, in some other it decreases with increasing standard deviation, in some other the opposite happens.

As far as the software packages are concerned, with PhotoScan the best results in Z are obtained for standard deviations of 2–4 cm in RTK1 and RTK4; then they deteriorate with larger standard deviations. In the other two cases, though there is no clear minimum, larger standard deviations produce smaller errors.

In Pix4D in three cases the Z error improves the larger the standard deviation; only in RTK4 there is a clear minimum at around 3 cm of standard deviation. In all cases an RMSE in Z of 4 cm or better is reached.

MicMac is almost insensitive to changes in the horizontal coordinate’s standard deviation in the whole range from 1 to 10 cm. In Z, three types of behavior are apparent. In RTK2 and RTK4, the error drops from 25 to 35 cm at 1 cm standard deviation to a clear minimum around 3.5 cm; then, the RMSE grows slowly and steadily (asymptotically) towards a constant error (of 8 and 5 cm respectively). In RTK1, the error decreases monotonically towards a constant error of 2 cm, while, in RTK3, there is a minimum at 1.5 cm and then an asymptotic increase to about 8 cm.

Looking for hints on how to operate practically, inner quality measurements such as the mean reprojection error are useless (or even wrong) guides in the search for such optimum. In principle the smaller the standard deviation assigned to the camera stations in the BBA, the more the solution will be constrained, so the reprojection error should increase. On the contrary, if the standard deviations are large, the contribution of camera station observations gets smaller, i.e., they will not affect the residuals of collinearity equations. In practice, however, the variations of the reprojection error will be very small, even if mm level standard deviations are assigned to the camera stations (see Table A1 in Appendix A). Simulations with CALGE show that only the weighted average of the residuals of the least squares adjustment, sigma zero, is affected to some extent as, unlike the reprojection error, it accounts for all observations. To the best of our knowledge, none of the software packages coming from Computer Vision provides this information, however.

If available, the estimates of camera precision provided by the RTK data processing should be used, perhaps with a penalty factor as they are generally too optimistic. For periodic surveys of targets, a sensitivity analysis is advisable, looking at the changes of the ground coordinates for different standard deviations.

As far as the four flights are considered, with a penalty factor of 2 applied to the nominal precision of the camera position encoded in the image EXIF (1 to 1.5 cm in X and Y and from 1.8 to 2.5 cm in Z), all of the software packages produce satisfactory, although not necessarily optimal, results in most cases.

The results of the second set of test on the accuracy of block orientation as a function of different standard deviations assigned to tie points and camera stations are summarized in Figure A1 in Appendix A, where the RMSE in elevation obtained by PhotoScan is plotted for RTK1 and RTK3 (notice that the curves in Figure 2 represent a section of the surface plotted). As long as the ratio between the two standard deviations is constant, the least squares solution should be unchanged and so the RMSE. Therefore, one would expect that the RMSE remain constant over an angle with vertex on the origin. This is indeed roughly the case for RTK1, where it seems that the tie point precision is less critical than the camera station precision as far as accuracy in concerned. However, with RTK3 the pattern looks quite different, as the best accuracy is found far from the “reasonable” set of tie point and camera station precision. The reason why this happens needs to be further investigated.

3.2. eBee RTK Survey Accuracy Assessment

The mean errors and the standard deviation of the errors, computed at the 14 CP for all blocks, configurations and software packages are reported in Appendix A in

Table A3. Mean values of the residuals in X, Y and Z coordinate on 14 checkpoints for the three BBA configurations: 12 GCP, RTK and RTK+1GCP.

MEAN X (m)				MEAN Y (m)			MEAN Z (m)
RTK 1
	12GCP	RTK	RTK+1GCP	12GCP	RTK	RTK+1GCP	12GCP	RTK	RTK+1GCP
Pix4D	0.001	−0.014	−0.011	0.007	−0.003	0.002	0.005	0.012	−0.002
PS	−0.004	0.016	0.014	−0.007	−0.001	0.002	−0.001	0.011	0.010
MicMac	0.003	−0.011	−0.010	0.004	0.002	0.001	0.003	−0.085	−0.004
RTK 2
Pix4D	0.002	−0.005	−0.006	0.003	0.008	−0.004	0.006	0.083	0.001
PS	−0.001	0.009	0.014	−0.003	−0.007	0.006	−0.003	−0.093	−0.002
MicMac	−0.003	−0.008	0.014	0.002	0.003	0.006	0.000	−0.063	−0.002
RTK 3
Pix4D	−0.002	−0.004	−0.006	0.004	0.004	−0.004	−0.011	0.069	0.001
PS	−0.001	0.004	0.002	−0.006	−0.005	0.009	−0.001	−0.026	0.024
MicMac	0.000	−0.002	−0.001	0.006	0.007	−0.001	0.004	0.035	−0.019
RTK 4
Pix4D	0.000	−0.008	0.008	0.005	0.004	0.004	−0.008	0.013	−0.026
PS	0.000	0.002	0.007	−0.005	0.000	0.002	0.004	0.004	−0.011
MicMac	0.004	−0.003	−0.002	0.003	0.000	−0.006	0.010	−0.039	0.000

and

Table A4. Standard deviations of the residuals in X, Y and Z coordinate on 14 checkpoints for the three configurations: 12GCP, RTK and RTK+1GCP.

σ X (m)				σ Y (m)			σ Z (m)
RTK 1
	12GCP	RTK	RTK+1GCP	12GCP	RTK	RTK+1GCP	12GCP	RTK	RTK+1GCP
Pix4D	0.012	0.010	0.014	0.007	0.014	0.020	0.021	0.017	0.029
PS	0.011	0.016	0.016	0.010	0.012	0.012	0.018	0.015	0.015
MicMac	0.007	0.012	0.013	0.008	0.011	0.011	0.020	0.012	0.012
RTK 2
Pix4D	0.010	0.024	0.024	0.008	0.009	0.009	0.025	0.016	0.016
PS	0.010	0.015	0.009	0.008	0.009	0.011	0.018	0.020	0.021
MicMac	0.007	0.025	0.027	0.008	0.011	0.012	0.020	0.029	0.028
RTK 3
Pix4D	0.015	0.011	0.024	0.008	0.007	0.009	0.020	0.020	0.016
PS	0.009	0.018	0.015	0.005	0.015	0.012	0.024	0.033	0.032
MicMac	0.010	0.010	0.013	0.007	0.008	0.008	0.019	0.026	0.037
RTK 4
Pix4D	0.014	0.019	0.012	0.007	0.009	0.009	0.023	0.022	0.021
PS	0.018	0.023	0.025	0.008	0.009	0.012	0.017	0.028	0.027
MicMac	0.009	0.018	0.024	0.009	0.005	0.007	0.029	0.026	0.030

. Figure 3 depicts the RMSE of each coordinate for all the combinations while

Table 3. Mean error in Z on the 14 checkpoints for the three BBA configurations: with GCP only (12GCP), with GNSS-AT only (RTK) and with GNSS-AT with the addition of 1 GCP (RTK+1GCP).

MEAN Z (m)
RTK 1				RTK 2
	12GCP	RTK	RTK+1GCP		12GCP	RTK	RTK+1GCP
Pix4D	0.005	0.012	−0.002	Pix4D	0.006	0.083	0.001
PS	−0.001	0.011	0.010	PS	−0.003	−0.093	−0.002
MicMac	0.003	−0.085	−0.004	MicMac	0.000	−0.063	−0.002
RTK 3				RTK 4
	12GCP	RTK	RTK+1GCP		12GCP	RTK	RTK+1GCP
Pix4D	−0.011	0.069	0.001	Pix4D	−0.008	0.013	−0.026
PS	−0.001	0.006	0.024	PS	0.004	0.004	−0.011
MicMac	0.004	0.035	−0.019	MicMac	0.010	−0.039	0.000

contains the mean error for the Z coordinate. In all flights and with all packages the RMSE obtained in the traditional block orientation by GCPs is in the range from 1.1 to 2.0 cm for the combined horizontal coordinates (equivalent to 0.5–0.8 GSD), while the elevations are about 1.5 times less accurate (from 0.7 to 1.3 GSD). There is little or no bias, since the mean error is always better than 1 cm in XY and never larger than 1.1 cm in Z (see

Table A2. Reprojection error in pixel for PhotoScan for the four flights as a function of the standard deviations σ (in cm) assigned to the camera station positions.

σ (cm)	0.3	1	5	15
RTK1	1.240	0.963	0.924	0.922
RTK2	0.957	0.890	0.875	0.874
RTK3	1.090	1.040	1.023	1.022
RTK4	1.000	0.951	0.927	0.925

).

Looking at the results of RTK, it is apparent that the accuracy of GNSS-assisted AT with the eBee RTK is on the one hand remarkable and often on the par with that obtained with ground control, especially in horizontal coordinates, where the combined RMSE in XY ranges from 0.6 to 1.2 GSD. On the other hand, comparing the 12GCP results with those of RTK, it is evident that there might be problems with heights: here the RMSE in Z ranges from 0.8 GSD (almost as good as the best 12GCP) to 4.0 GSD. In two cases (RTK1 and RTK4) both the XY and the Z accuracies of RTK are very satisfying, with average residuals no more than 5 mm larger than those of 12GCP. In other words, the RTK solution practically achieves the same accuracy found using GCP as far as the horizontal coordinates are concerned. In elevation, to the contrary, there are three cases with very good agreement (with RMSE better than 2.5 cm) and cases where the accuracy is worse, up to 9.5 cm in PhotoScan, 6.8 cm in MicMac and 8.5 cm in Pix4D, see Table 2. Here, a bias is apparent, since the mean error is markedly different from zero, with either positive or negative values for the same flight, depending on the software package.

Figure A2 in Appendix A shows the plots of the errors obtained with PhotoScan on all 26 available checkpoints for the RTK configuration and for all four flights. In the horizontal coordinates, the error spatial distribution does not show any global systematic pattern, though in three of the test flights, in the eastern side of the block, a residual component parallel to strip direction is apparent on most points. As far as the elevation is concerned, it is hard to find any similarity between the four flight patterns as well as any pattern at all within a particular flight.

As the error in elevation turned out to be higher than expected (in particular in RTK2 and RTK3), in RTK+1GCP an extended BBA has been computed, using as control information the projection center coordinates and those of GCP No. 13 (see Figure 1) located in the middle of the block. As shown in Table 3 and in the diagrams of Figure 3, the elevation accuracy improves considerably, down in the worst case to slightly more than 3 cm. Where the statistics in elevation were good already, such as for the Pix4D and PS solutions in RTK1 and RTK4, adding the GCP brings small changes (less than 1 cm) to all coordinates (some for the better, some for the worse). With the additional GCP, in the RTK1 block with MicMac a previous elevation RMSE of 8.5 cm is down to 1.4 cm. In other words, with at least a GCP the bias is removed or greatly reduced in all cases: the RMSE in Z now ranges from 0.6 to 1.6 GSD. As far as horizontal coordinates are concerned, the changes are very small, in the order of a few mm, except for RTK2 with PhotoScan and Pix4D, where they reach 1.2 cm; the combined RMSE in XY now ranges from 0.7 to 1.2 GSD.

MicMac is almost always the most accurate in XY in every configuration; PhotoScan is the best in elevation in the 12GCP and RTK cases, while in the RTK+1GCP case there is no clear winner.

The accuracy obtained on checkpoints is finally summarized in

Table 4. Best and worst RMSE among all software packages and flights, expressed in meters and GSD units, for each block control configuration.

	RMSE XY (m)			RMSE XY (GSD)
	12GCP	RTK	RTK+1GCP	12GCP	RTK	RTK+1GCP
Min	0.012	0.014	0.016	0.48	0.57	0.67
Max	0.020	0.029	0.029	0.85	1.19	1.21
	RMSE Z (m)			RMSE Z (GSD)
	12GCP	RTK	RTK+1GCP	12GCP	RTK	RTK+1GCP
Min	0.016	0.019	0.014	0.67	0.79	0.58
Max	0.030	0.095	0.039	1.25	3.96	1.63

, where the best and worst RMSE in each configuration are reported for all packages and all flights, expressed both in metres and in GSD units.

As stated before, the statistics in Figure 3 and Table A3 and Table A4 refer to the 14 checkpoints available. However, the accuracy of RTK and RTK+1GCP configurations has also been calculated on all available CP, i.e., 26 and 25, respectively. The changes to all the RMSE shown in Figure 3 are in both cases negligible, since they amount to just a few mm at most.

3.3. Interaction between Interior and Exterior Orientation Parameters

As far as the 12GCP configuration is concerned, the correlations between the principal distance and the camera heights, as can be expected, are very strong (on average 70% with maximum value 87%). This can produce restitution errors, if the scene has large differences in depth (e.g., high-rise buildings). Cross strips at different height would have limited the correlations, though the different image scale would probably limit the number of good tie-points between the sub-blocks. On the other hand, the analysis for RTK and RTK+1GCP shows much lower correlations, respectively, 5% (with 14% maxima) and 12% (with 33% maxima).

Table 5. Estimated camera principal distance in the three configurations for all software packages for the RTK2 flight.

RTK2—Estimated Principal Distance (mm)
	12 GCP	RTK	RTK+1GCP
Pix4D	4.5718	4.5771	4.5771
PhotoScan	4.5801	4.5853	4.5818
MicMac	4.6020	4.6134	4.6104

reports the estimated camera principal distance for the three configurations and for each software package in the RTK2 flight, where the bias in elevation was remarkable with all packages.

From Table 5 it can be noticed that the estimated values are remarkably different (by several micrometers) from package to package, though the standard deviation of the principal distance from the BBA varies between 0.2 and 1 µm depending on the flight and on the software. These differences might be due to some extent to the different camera models (see Table A4 in Appendix A) and the estimation procedures of the three software packages; they remark, however, the already mentioned unreliability of interior orientation parameters determined using self-calibration in nadiral blocks.

Moving the block control from ground to the projection centres (RTK) in the RTK2 block there is a sharp increase in principal distance (between 5 and 11 µm) with every software package. Adding 1 GCP, Pix4D keeps the principal distance unchanged, PhotoScan almost reverts to the 12GCP value and MicMac settles for a value in between.

The spatial plots of the camera location residuals (i.e., of the differences between the estimated and GNSS-measured position) of the RTK2 block are shown for RTK (Figure 4, left) and RTK+1GCP (Figure 4, right) configurations.

Table 6. Means of the residuals of the projection centers elevations for the three configurations in each flight.

MEAN DZ (m)
RTK1				RTK2
	12 GCP	RTK	RTK+1GCP		12 GCP	RTK	RTK+1GCP
Pix4D	0.024	0.000	0.046	Pix4D	−0.041	0.000	0.083
PhotoScan	0.025	0.000	0.000	PhotoScan	−0.017	0.000	0.007
MicMac	−0.137	0.15	−0.006	MicMac	0.010	0.256	0.154
RTK3				RTK4
	12 GCP	RTK	RTK+1GCP		12 GCP	RTK	RTK+1GCP
Pix4D	−0.025	0.000	0.079	Pix4D	−0.043	0.000	0.025
PhotoScan	−0.083	0.000	0.003	PhotoScan	−0.033	0.000	−0.001
MicMac	−0.028	0.06	0.167	MicMac	−0.146	0.122	0.034

reports the average of such residuals, in elevation only, for all blocks.

Cross examination of Table 3, Table 5 and Table 6 and of the projection center residual plots in Figure 4 highlights the different behaviors of the software packages when RTK-based georeferencing is constrained by the introduction of the GCP.

With RTK, in Pix4D the mean of the projection center coordinates is fixed; in the plot, even though some systematic displacements in elevation can be seen in two strips, the general pattern is random, with a balance of positive and negative signs in elevation. When the ground point is included in the BBA the projection centers are systematically shifted upwards; at the same time, as shown in Table 3, the principal distance is kept fixed.

On the contrary, the two plots of PhotoScan in RTK and RTK+1GCP differ very little and show no systematic patterns as far as the sign of the elevation changes is concerned. It is the principal distance which changes to adapt the block to the ground constraint of the single GCP. Considering the stereo normal case, if $z$ is the depth (vertical distance from camera centre), $c$ the principal distance and $m_b$ the image scale, a variation in principal distance produce a change in depth $\delta z=m_b\delta c$. Since the average image scale is about 1:18,000, a principal distance change of 1 µm is equivalent to a movement of camera height of 1.8 cm. Therefore, in RTK2, with a 3.5 µm change in principal distance, a variation $\delta z$ in depth of about 6.3 cm can be expected. The mean error change on the CP between RTK and RTK+1GCP is equal in PhotoScan to 7.3 cm, i.e., most of the 9.3 cm bias reported in Table 2.

Summarizing, it seems that the balance between the weights of the different observations (GCP and camera center constraints) and the interaction between interior and exterior orientation parameters have different patterns in the three packages: as can be seen in Table 6, the behavior found for the RTK2 flight also applies to the other blocks.

With MicMac, the behavior is different, because both principal distance and camera elevations are affected. In the RTK case, the RMSE in Z on CP is about 6 cm, with an estimated principal distance significantly higher than those calculated by PhotoScan and Pix4D. This excess in the estimated principal distance is compensated by an average upwards shift of the camera projection centers with a mean value of 26 cm. When one GCP is introduced, the average shift with respect to the measured positions is 15 cm and the principal distance change with respect to the 12GCP case (8.4 µm) is equivalent to a depth change of 15 cm.

4. Conclusions

An empirical study on the accuracy and repeatability of UAV block orientation by GNSS-AT, carried out using three different software packages on four repeated survey flights, has been performed.

An average horizontal RMSE of 2.2 cm has been obtained by GNSS-AT with all packages and blocks, compared to 1.5 cm with the traditional orientation with GCP; in elevation, the values are 5.5 cm and 2.1 cm, respectively: a dense ground control, therefore, still provides better accuracy. However, the results found are in line with the claims of the manufacturer and suggest that, as far as control of horizontal displacements is concerned, periodic photogrammetric surveys can rely on on-board RTK or PPK GNSS positioning to define a stable reference system to detect changes of well defined points exceeding 5 cm. As already found by other authors, in elevation, though, biases of several cm may arise, which can be controlled using at least one GCP. The location of the additional point and, when the conditions on site allow for it, the additional benefit of using more (e.g., three) well distributed points might be worth investigating. In our four test flights (see

Table A5. Statistics of the residuals in X, Y and Z coordinate on 13 checkpoints for the two BBA configurations: RTK+1GCP and RTK+3GCP.

	RTK+1GCP			RTK+3 GCP
RTK1
	X	Y	Z	X	Y	Z
Mean (m)	0.014	0.002	0.010	−0.001	−0.007	−0.003
σ (m)	0.016	0.012	0.015	0.011	0.009	0.010
RMSE (m)	0.021	0.012	0.018	0.011	0.012	0.011
RTK2
Mean (m)	0.014	0.006	−0.002	0.009	0.001	−0.002
σ (m)	0.009	0.011	0.021	0.008	0.009	0.019
RMSE (m)	0.017	0.013	0.021	0.012	0.009	0.019
RTK 3
Mean (m)	0.002	0.009	0.024	−0.002	−0.002	−0.011
σ (m)	0.015	0.012	0.032	0.014	0.006	0.027
RMSE (m)	0.015	0.015	0.039	0.014	0.006	0.029
RTK4
Mean (m)	0.007	0.002	−0.011	0.004	−0.009	0.007
σ (m)	0.025	0.012	0.027	0.022	0.009	0.029
RMSE (m)	0.026	0.012	0.029	0.022	0.013	0.030

in Appendix A), fixing three GCP along the block border slightly improved the results compared to just one in the middle.

It has been found that the influence of the standard deviations assigned to the camera projection centers on the horizontal accuracy is generally negligible, independently of the software package. In elevation, however, the picture is less clear, with different trends and differences in accuracy that can be relevant, if not in absolute terms at least in percentage. Currently, we do not have solutions to suggest for this problem, apart from maintaining that different standard deviations for XY and Z should be used, since this is inherent to satellite positioning. As far as the “best” standard deviations to assign are concerned, no indications can be provided, as this depends on the specific hardware and software, system calibration and satellite configuration.

Camera calibration in UAV blocks oriented with GNSS-AT was not meant to be the main topic of the paper. However, on-the-job calibration is likely in our opinion to be the best operational compromise, with the awareness that unless at least one GCP is provided, there might be biases in object coordinates, especially in elevation.

Three software packages have been used, not to find which one is the best, but to highlight that there may be differences in GNSS-AT implementation and therefore different outcomes. The analysis of the influence of the standard deviations and the residuals on the camera stations show that indeed the treatment of the observation weights among the three packages is different. As far as the horizontal coordinates are concerned, this does not seem to lead to remarkable differences; in elevation, though, this might not always be the case.

The flights took place under good satellite constellations. Severe obstructions might be found flying in a high mountain environment, where dispensing with GCP is most important, and this might deteriorate the positioning quality. Future work will therefore focus on the influence of satellite configuration on the accuracy of camera projection centers.