2. Materials and Methods
2.1. Test Site Description
The surveyed bridge belongs to a national road that crosses River Taro at (44°29′22″ N; 10°13′28″ E), a few kilometers west of the city of Parma (Italy). Including the ramps, the bridge is about 850 m long and about 8 m wide. The bridge is about 10.5 m higher than the riverbed, made of an alluvial plain with deposits of silt, sands and gravel, mostly dry at the survey period (September 2018). Both riversides and about one third of the northern bridge side are lined by high trees (see
Figure 1).
2.2. Reference Network and GCP
To provide an independent network to evaluate the georeferencing accuracy of GNSS-AT, a total of 60 signalized targets were deployed and surveyed: 31 over the bridge (road surface and parapet) and 29 distributed on the riverbed and on a factory service area on the river west bank.
The GCP and the CP coordinates were determined by TS measurements from a network of 12 stations. Moreover, 26 of the targets were surveyed twice in Network RTK mode with a Leica GS14 and a Geomax Zenith 35 Pro receiver, in order to provide double points for the connection of the GNSS network to the TS network.
The GNSS positions of the targets as well as of the higher flight camera stations (see
Section 2.3.), determined in the national reference frame ETRF2000(2008), were converted to a local cartesian reference system centered at mid bridge, with origin on the reference ellipsoid,
z axis along the ellipsoid normal and
y axis parallel to the north axis of the UTM 32N fuse of the ETRS89 datum.
The TS network was adjusted in this reference system, using as known points with accuracy of 1 cm in horizontal coordinates and 1.5 cm in elevation the GNSS positions expressed in the local reference system. The Root Mean Square (RMS) of the residuals on the GNSS coordinates turned out to be 1 cm for each horizontal coordinate and 1.2 cm for the elevation. From the network adjustment, the RMS of the estimated precisions of all network points in the local system turned out to be 8 mm in X,Y and 7 mm in Z.
2.3. Survey Flights
Even though, as mentioned in
Section 1.5, a single RTK-equipped multi-rotor could have acquired both the master and the auxiliary block, in our case we had to use two different UAVs. Indeed, we had available only a RTK-equipped fixed-wing senseFly eBee and a multi-rotor DJI Phantom4 Pro, not provided with the RTK option. The former was therefore used for the auxiliary flight while the latter was necessary for the high-resolution survey of the bridge (the master block), as only a multi-rotor could ensure adequate forward overlap for the low-elevation single strip over the bridge.
The eBee RTK is equipped with a 20 Mpx compact S.O.D.A. camera with 29 mm nominal focal length (35 mm equivalent). The RGB camera (resolution 5472 × 3648 pixels, pixel size 2.4 μm) acquires nadiral images with exposure parameters set automatically. The on-board receiver can process L1 and L2 GPS and GLONASS data and receive the differential corrections from the master station or from the control center of a CORS network via the flight control software and a ground radio modem. Camera positions are stored in the image metadata as geo-tags as well as in the flight log. A Geomax Zenith 35 Pro, set on a benchmark at the eastern bridge end, was used as master station.
The DJI Phantom4 Pro is equipped with a 20 Mpx CMOS sensor with 24 mm nominal focal length (35 mm equivalent). The FC6310 RGB camera (resolution 5472 × 3078 pixels, pixel size 2.5 μm) is mounted on a stabilized gimbal with controllable pitch range from −90 ° to +30 °. The single frequency on-board GNSS receiver is fit only for navigation purposes, not for accurate georeferencing.
Four flights were executed, two with the eBee and two with the Phantom4; the main parameters of each flight are shown in
Table 1. The high-resolution bridge survey flight is made of a single strip, flown manually with the Phantom4, with an average 80% forward overlap at ground level (see
Figure 2); while still under manual control and without landing, a second single strip was executed, at a higher flying height but with a slightly larger forward overlap (as a matter of fact, however, the overlap is far from constant in both strips). Finally, two flights were executed in automatic mode with the eBee, about half an hour apart from each other. The former is made of 11 strips, flown along the bridge direction (roughly in east–west direction), with forward and side overlap of about 50% and 70% respectively (covered area: about 740 m × 370 m). The latter consists of 7 strips, flown across the bridge, just east of bridge center, almost at the same height as the previous flight and with roughly the same overlaps (covered area: about 370 m × 220 m). The two eBee flights were combined in a single block (see
Figure 3) and will be referred to as the eBee block in the following. Due to proximity to Parma Airport, the activity was authorized by air traffic control and all flights were made under coordination of the control tower.
2.4. Photogrammetric Data Processing
Photogrammetric data processing was executed with the commercial package PhotoScan (PS) version 1.2.6, build 2834, by Agisoft LLC, St. Petersburg, Russia. Block orientation is performed with SfM algorithms in an arbitrary reference frame. A Helmert transformation is computed from the arbitrary reference to the object reference system, based on the GCP coordinates, or, in the case of GNSS-AT, on the camera center positions, loaded directly from the image geotags or from a file.
A global optimization stage is then executed that minimizes the sum of the reprojection error and of the GCP and/or the camera station coordinate residuals; camera parameters can be estimated by self-calibration or just applied if the camera has been pre-calibrated. Each GCP coordinate and each camera position can be assigned a specific a-priori precision; otherwise, default values can be assigned. In the tests, based also on previous experiences [
40] the following default standard deviations were assigned: 1 pixel to tie points image coordinates (automatically or manually extracted and matched), 5 mm to the GCP coordinates and 3 cm to the eBee camera station coordinates.
2.5. Test Description
The goal of the experiments is to find out whether GNSS-AT can be applied indirectly, i.e., also in cases where the UAV collecting the master block images is not RTK-equipped, or, due to site characteristics, the GNSS signal cannot be received reliably, or, finally, the master block is just made of a single strip. In such cases an auxiliary block, flown at higher elevation by a UAV with an on-board RTK GNSS receiver, could be employed to georeference the master block.
To this aim, the experiment foresaw several stages, comparing different eBee camera calibration parameter sets and flight combinations in terms of accuracy on the CP. Each stage is briefly described in the following, stating first its objectives.
O1. To evaluate the GNSS-AT orientation accuracy of the standalone eBee block with camera parameters obtained from on-site pre-calibration or from self-calibration.
Though the expected accuracy of RTK-equipped UAV blocks has already been reported by several studies [
40,
47] and can therefore be reasonably foreseen, it is still worth checking the eBee block accuracy prior to its combination with the Phantom4 strip, investigating the alternatives for calibration discussed in
Section 1.4. An on-site camera pre-calibration executed imaging a small test-field, located on the riverbed just north of the bridge (see
Figure 1). A subset of 18 eBee images, 9 from the east–west flight and 9 from the north–south flight (see
Figure 4), was selected for the calibration, in order to keep the effort commensurate to that of the survey. By keeping fixed all the test-field GCP or just one GCP at a time, four calibration parameter sets were estimated. Finally, four self-calibrating BBA were also performed fixing each time a single GCP, located at mid bridge, or at one of the bridge ends, or on the riverbed, to find out whether GNSS-AT orientation accuracy depends on the single GCP location.
These calibration parameter sets were then used in the GNSS-AT adjustments of the O4 and O5 stages.
O2. To assess the accuracy of the Phantom4 single strip at 30 m oriented with GCP.
A traditional GCP-based BBA of the Phantom4 single strip was performed fixing the coordinates of 17 GCP; 13 placed on both parapets along the bridge and 4 located on the riverbed in pairs, upstream and downstream with respect to the bridge. As a single strip is far from the ideal geometry for a reliable camera calibration, first self-calibration was applied in the joint adjustment of both the 30 m and 70 m Phantom4 flights and then used as pre-calibration dataset in the adjustment of the 30 m strip.
O3. To find out whether SfM effectively connects master and auxiliary blocks.
To this aim, an analysis of the number of tie points connections and their distribution across the master and the auxiliary blocks was performed.
O4. To assess the accuracy of the master–auxiliary joint adjustment using different camera calibration techniques, and compare it to the accuracy of the GCP-controlled master block.
To this aim, different joint adjustments of the eBee and of the Phantom4 30 m blocks were performed with GNSS-AT, with camera calibration parameters pre-calibrated of self-calibrated. Finally, the RMSE on CP were computed and compared to those obtained in O2.
O5. To find the best (minimum) auxiliary block configuration still ensuring the stability of the master block.
As flying the auxiliary block adds time and cost to the survey, it is worth searching for its minimal effective configuration. To this aim, the eBee and Phantom4 blocks were jointly processed, each time progressively removing from the former the longitudinal and/or the cross strips, and measuring the accuracy decrease on the CP. Strip removal was applied both to the whole eBee block and to the east–west eBee flight only (i.e., without cross strips). The longitudinal strips were removed in pairs, one from each side with respect to the bridge axis, from 10 to 2, i.e., progressively moving towards the quasi-singularity condition as the distance between the two flight lines farthest apart decreases.
In the full block case, all cross strips were maintained until only two longitudinal strips remained. Then the cross strips were first shortened from 10 to 6 images each and after progressively removed (from 7 to 4 to 2), always maintaining the first and last ones (i.e., the cross strips farthest apart from each other).
The accuracy of block orientation was evaluated by computing the differences between the coordinates of the CP estimated in every BBA and those estimated from the topographic network.
In the accuracy checks on single blocks (eBee block, Phantom4 strip) the targets were measured in all images. To the contrary, in the accuracy checks for the eBee and Phantom4 joint blocks, targets were measured only in the Phantom4 images, as the goal is to assess the accuracy of ground points from the indirectly-oriented master block. Any collimation of CP also in the eBee images would have, instead, bypassed the uncertainty propagation through the connection master–auxiliary established by the across-blocks tie points.
The above remarks imply that the number of CP varies according to the examined block configuration: 59 or 60 in the eBee block alone, 43 or 44 in the joint eBee and Phantom4 blocks, 27 in the GCP-controlled Phantom4 strip.
In
Table 2 a summary of the tests carried out is reported, where the test Id string is built on the following list of abbreviations:
eBeeprecal: the set of 18 images used for pre-calibration purposes, oriented with GNSS-AT and a variable number of GCP as control information;
eBee: the eBee block with camera stations from GNSS RTK positioning as control information;
eBeeOpt: the best (minimum) subset of the eBee block still ensuring block accuracy; camera stations are the only control information;
P4 30: the single strip flown with the Phantom4 at 30 m relative elevation;
+ k GCP: the adjusted block includes k GCP;
pre-c: the IO parameters are fixed to values obtained from an on-the-job pre-calibration;
self-c: the IO parameters are estimated by self-calibration in the BBA.
4. Discussion
As far as the accuracy of GNSS-AT block orientation is concerned, the RMSE on 59 CP of
Table 4 shows that the tie points horizontal coordinate accuracy of the eBee block, even if determined without a pre-calibrated camera and without any GCP (last row of
Table 4), is quite good. However, a significant bias may affect the elevations, unless a pre-calibrated camera is used or at least one GCP is used to strengthen the self-calibrating BBA. Both these findings agree with the results presented in previous studies on GNSS-AT by the authors [
11,
40] and by Zhou [
41] and Hugen [
47]; in Jozkow [
48] a large bias in elevation is also found in GNSS-AT self-calibration without GCP. On the other hand, Mian [
49] found that fixing a single GCP for system calibration purposes in the BBA of a UAV block with IMU and GNSS data did not prevent a bias of about 11 cm to remain in CP elevations. Finally, in [
50], increasing from 0 to 4, the number of GCP improved the RMSE in elevation by just more than 1 cm, from 6.7 to 5.4 cm and adding 14 more GCP the RMSE improved only to 5.1 cm. Though no specific bias estimate is provided in the paper, it can therefore reasonably be inferred that just a small amount of elevation bias was present in this case; moreover, the lack of improvements from 4 to 18 GCP shows the strength that precise GNSS-determined camera stations convey to the block, as also found by simulations by [
51].
As far as on-site pre-calibration is concerned, the tests performed on a small block using a single GCP or a sizeable number of GCPs (12) show variations by less than half pixel for the principal point and less than one pixel for the principal distance in the calibration parameters (
Table 3). Applying these different sets of parameters to the eBee block in the GNSS-AT BBA, the RMSE on the CP horizontal coordinates are fairly small, with differences from set to set smaller than 4 mm. In elevation the average error is 2.8 cm, while the differences are larger, up to 1.6 cm. They look related to the specific GCP rather than to the amount of GCP used in pre-calibration and are therefore likely to originate from the particular measurement error in the image or object coordinates of the GCP fixed.
A larger pre-calibration test-field and a stronger imaging geometry would likely deliver more stable IO parameters; however, two points must be stressed: with GNSS-AT a few (even a single) GCP seem sufficient for effective self-calibration, as control from the camera positions is spread all over the block. Though this point deserves deeper investigations by simulations, a less demanding imaging geometry might be therefore enough for GNSS-AT camera calibration, unlike GCP-based calibration where oblique imaging is recommendable [
20,
34]. In any case, as up-to-date camera parameters are necessary, only on-site (pre-)calibration represents a working alternative to self-calibration. However, there should be a balance between the time and effort required by the pre-calibration and that of the whole survey, when GNSS-AT is employed.
Zhou [
41] also applied GNSS-AT to corridor mapping, using a rotary-wing RTK-equipped UAV capable of oblique imaging. Therefore, in addition to camera calibration, lever arm calibration is also considered, to exploit the flexibility of image acquisition. In the experiment, made of three nadir strips 600 m long, they used a pre-calibration flight with strong geometry (a combination of nadir strips at different altitudes and of oblique imaging). Then they applied to the survey flight the same IO parameters configuration as we have done: with self-calibration (with or without a single GCP fixed) and with camera parameters fixed (again, with and without a single GCP fixed). While in horizontal coordinates the accuracy on CP stays the same, at about 3.3 cm with a GSD of 1 cm, in the self-calibration without GCP case, a 15 cm bias is found on elevations, as in our case. Fixing the GCP, the RMS is down to 1 cm. In the pre-calibrated case, using or not the single GCP does not change the RMS in elevation, which is slightly larger than that in the self-calibration case (2 cm). Our results are therefore very well in agreement with those in [
41], both with respect to the alternative between pre-calibration and self-calibration as well as for the need of a single CGP in case of self-calibration. The agreement applies also to accuracy on CP, as from
Table 6 and
Figure 7 it can be seen that our results are well comparable (slightly better in horizontal coordinates and slightly worse in elevation).
As this paper refers specifically to cases where GCP are difficult or impossible to place in the master block area, so preventing self-calibration with one or more GCP, the main finding of our experiment in this respect is that a small test-field, arranged in a convenient area near the survey site, seems sufficient for a camera pre-calibration that removes most, if not all, the bias in elevation from the GNSS-assisted survey flight.
Overall, the experiment shows that an auxiliary block, flown with a RTK-equipped UAV, can be successfully adjusted with the GNSS-AT technique to georeference and control a master block, flown at a lower elevation, without using GCP. The presented technique is likely to remain a special one, useful whenever the master block cannot be reliably georeferenced by GNSS-AT and surveying GCP is difficult or inconvenient. However, it should be noted that, as the RMSE on CP in
Table 6 shows, the error propagation from the auxiliary to the master block is not too unfavourable, as an accuracy loss compared to the eBee block between 30% and 50% is registered in horizontal coordinates and none in elevation. In a similarly difficult environment, [
33] proposed a new formulation of ISO, which foresees the tightly coupled integration of inertial, GNSS, and image observations. In their experiment they also consider the corridor case (actually made by three long strips, divided in two sections, one with good GNSS coverage and the other with no GNSS coverage). They show that, with the current poor IMU data quality in UAV, inertial navigation can hardly bridge GNSS gaps with the IMU data only (the elevation error grows very fast), while their method performs better, combining position and attitude data from IMU with SfM. However, for a strip of comparable length with our experiment, the RMS on CP is larger than 10 cm in horizontal coordinates and 6 cm in elevation. In comparison, the proposed approach is of course more demanding in operational terms but still more accurate even with a two-strip auxiliary block. Moreover, it works also for a single strip of (in principle) any length, as the stability is tied to the auxiliary block, while their method starts drifting anyway, if the GNSS-denied section of the flight is too long. Finally, no additional sensors and software are required, as GNSS-AT is today implemented in all major SfM packages for UAV image processing.
The number and spatial distribution of tie points common to master and auxiliary blocks is certainly a critical element for the successful transfer of georeference information and block deformation control. In this respect, the test setup presented in this paper is from the one hand a demanding case, as a long single strip (i.e., a weak configuration) was successfully oriented. On the other hand, as far as perspective differences between the two sets of images are concerned, the test area is not particularly demanding, as both the bridge and the riverbed are basically flat, while the viewing direction is nadiral for both cameras. Despite a relevant height difference of about 70 m between the two flights, the different camera focal lengths keep the ratio among the two GSD, which is the driving factor for image matching, at about 1:2.5. Further investigations on the limits of this ratio for master and auxiliary blocks might help to clarify how far can this technique be extended.
As in previous tests [
40], the position within the block of the single GCP used for self-calibration in GNSS-AT was found to be not relevant. However, this does not seem to be the case in the master–auxiliary joint block, where the GCP position affects the amount of elevation bias removed, with the most effective position being at mid-block. The reason for this difference is likely due to the weak single-strip geometry and therefore suggests using a pre-calibrated camera in the master–auxiliary case. No other study, to the best of our knowledge, reports on this point, however.
As far as the optimal configuration of the auxiliary block is concerned, our findings are obviously related to the specific case of the single strip, where both longitudinal and cross strip are necessary to ensure an accuracy not too far from a traditional adjustment.
More investigations are still necessary to assess the true applicability of the technique in a more demanding environment: think for instance of a narrow gorge, where the walls might be the main area of interest. In such cases, to ensure good connections between two blocks, both might need nadiral as well as oblique images. However, perspective and scale differences between the oblique images of the two flights might be too demanding for SfM to handle.
5. Conclusions
The results presented in the previous sections show that accurate georeferencing and control of a master block, even in the unfavourable case of a single strip, can be achieved by means of the joint adjustment with an auxiliary block, flown at a higher elevation with an RTK-equipped UAV. For a single strip about 700 m long, with 70 m height difference between master and auxiliary flight lines, a GSD of 0.9 cm and a strong auxiliary block, the accuracy verified on targets can be as good as 1.5 cm in each horizontal coordinates and less than 2.5 cm in elevation. With a weaker geometry of the auxiliary block the accuracy decreases, but still could be maintained below 4 cm for both coordinates.
The auxiliary block represents an obvious project overhead, that might be acceptable when cheaper or more efficient alternatives cannot be found. More investigations are necessary as far as its optimal configuration is concerned with different master blocks characteristics.
Another limit that the actual application of the technique on demanding environments may clarify is the maximum height difference between the master and auxiliary block, that in turn might depend on the ratio among the GSD of the two blocks.
However, on the one hand, the test results were obtained by a BBA with self-calibration without any GCP, as far as the horizontal coordinates accuracy is concerned. On the other hand, no significant bias could be found in the elevations using up-to-date camera calibration parameters. On-site pre-calibration with at least one GCP has proven to be adequate and on par or better than self-calibration, where one or more GCP are anyway necessary.
From the above remarks, it is clear that GCP are still necessary even with an RTK-equipped UAV. However, as more experimental evidence is gathered on the accuracy of the GNSS-supported GCP-free UAV photogrammetry, consistently confirming the high accuracy of horizontal coordinates and the limited amount of bias in elevation, a clearer picture of the future of this technique is emerging, particularly as far as medium-accuracy applications are concerned.
The strength conveyed to UAV blocks by the GNSS-determined camera projection centers is probably still underestimated. More investigations should be performed: from a theoretical standpoint, on residual calibration errors propagation; from a practical standpoint, on the requisites for an effective but affordable on-site calibration.
As we can expect a dramatic diffusion of RTK-equipped UAVs also to multi-copters, the proposed master–auxiliary technique is likely to remain a useful approach, to resort to in special cases. However, it shows the high degree of flexibility that RTK-endowed UAV and GNSS-AT offer today to the surveyor.