# Refining the Joint 3D Processing of Terrestrial and UAV Images Using Quality Measures

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### Paper Aim and Novelty

## 2. State of the Art

#### 2.1. Data Fusion

#### 2.2. Image Network Configurations

#### 2.3. Point Cloud Filtering

## 3. Proposed Methodology

#### 3.1. Work Premise and Motivation: Challenging Image Network Configurations

#### 3.2. Quality Parameters of 3D Tie Points

- (a)
- Re-projection error (or image coordinates residuals): it represents, in image space, the Euclidean distance between the measured point position and the back-projected position of the calculated 3D point. Even though low re-projection error values can suggest a high quality of the computed 3D point, this feature is not very significant when the point has been measured in few images.
- (b)
- Multiplicity (or image redundancy): this value indicates the number of images contributing to the 3D point calculation, i.e., the number of images where the point has been measured. Therefore, the multiplicity value refers to the excess of image observations with respect to the number of unknown 3D object coordinates, estimated within the adjustment step. High multiplicity values suggest greater reliability and precision of the computed 3D tie points, considering that multiple intersecting rays contribute to the point position check.
- (c)
- Maximum intersection angle: it refers to the maximum angle between intersecting rays contributing to the creation of a 3D point. Small intersection angles can negatively affect the adjustment procedure and reduce its reliability.
- (d)
- A-posteriori standard deviation of object coordinates (σ): from the covariance matrix of the least squares bundle adjustment, the standard deviations of all unknown parameters can be extracted. High standard deviation values can highlight 3D points with unsuitable object coordinates precision and problematic areas within the image network.

#### 3.3. Filtering Technique

#### 3.3.1. Single-Parameter Filtering: Tests and Issues

#### 3.3.2. Normalization and Linear Aggregation

- $L$ is the maximum value of the curve;
- $e$ is the Euler’s number;
- $x0$ is the x value of the sigmoid’s midpoint;
- $k$ is the steepness of the curve.

- ${A}_{v}$ is the overall aggregated quality score computed with the normalized quality parameters for each 3D tie point;
- ${V}_{re}$ is the normalized value of the re-projection error;
- ${V}_{mul}$ is the normalized value of the multiplicity;
- ${V}_{ang}$ is the normalized maximum angle of intersection;
- ${V}_{std}$ is the normalized value of the a-posteriori standard deviation;
- $w$ is a weight computed for each 3D tie point as in Equation (6):

#### 3.3.3. Filtering Threshold Identification—The Statistical Approach

#### 3.3.4. Thresholds Tests

- (1)
- Median values for each quality parameter, not weighting the aggregation function;
- (2)
- Median values for each quality parameter, weighting the aggregation function;
- (3)
- Median plus σ$MAD$, not weighting the aggregation function;
- (4)
- Median plus σ$MAD$, weighting the aggregation function.

## 4. Test and Results

#### 4.1. Experiments and Results

- (a)
- Perform the joint processing of UAV and terrestrial images in order to compute camera parameters, sparse and dense point clouds;
- (b)
- Compute quality parameters for each 3D tie point of the sparse point cloud (Section 3.2);
- (c)
- Normalize the computed quality values and their aggregation (Section 3.3.2), for assigning a quality score to each computed 3D tie point;
- (d)
- Analyze the statistical distribution of the computed quality parameters and identify suitable thresholds for each dataset (Section 3.3.3 and Section 3.3.4);
- (e)
- Filter those 3D tie points with an aggregated score higher than the selected threshold and generate a new set of filtered tie points;
- (f)
- Run a new bundle adjustment, refine the camera parameters and generate a new sparse point cloud;
- (g)
- Re-compute quality parameters on the filtered and refined cloud for evaluating the improvement of the inner quality parameters with respect to step (b);
- (h)
- Compute a new dense point cloud;
- (i)
- Employ external checks (e.g., 3D ground truth data) and noise estimation procedures to evaluate improvements of the newly generated dense point cloud with respect to the dense point cloud obtained from step (a).

#### 4.1.1. Modena Cathedral

#### 4.1.2. Nettuno Temple

#### 4.1.3. The World War I (WWI) Fortification of Mattarello (Trento, Italy)

#### 4.1.4. The ISPRS/EuroSDR Dortmund Benchmark

## 5. Discussion

- (a)
- the filtering method does not fully consider the tie point distribution in image space. Too aggressive filtering could prevent the orientation of some images during the orientation refinement. This issue is partially solved by adopting more relaxed thresholds, as presented in Section 3.3.4, and weighting the aggregation function.
- (b)
- the presented method has not been yet verified in the case of multi-temporal datasets.
- (c)
- the computation of the quality features and the filtering procedure are performed with an in-house developed tool (https://github.com/3DOM-FBK/Geometry) which has been so far tested only in combination with the exported outputs from the commercial software Agisoft Metashape [67]. Some issues about file format compatibility could arise while testing our filtering tool in combination with other open or commercial software.

## 6. Conclusions and Future Works

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Colomina, I.; Molina, P. Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS J. Photogramm. Remote Sens.
**2014**, 92, 79–97. [Google Scholar] [CrossRef] [Green Version] - Nex, F.; Remondino, F. UAV for 3D mapping applications: A review. Appl. Geomatics
**2013**, 6, 1–15. [Google Scholar] [CrossRef] - Hassanalian, M.; Abdelkefi, A. Classifications, applications, and design challenges of drones: A review. Prog. Aerosp. Sci.
**2017**, 91, 99–131. [Google Scholar] [CrossRef] - Granshaw, S.I. RPV, UAV, UAS, RPAS …or just drone? Photogramm. Rec.
**2018**, 33, 160–170. [Google Scholar] [CrossRef] - Anthony, D.; Elbaum, S.; Lorenz, A.; Detweiler, C. On crop height estimation with UAVs. In Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA, 14–18 September 2014. [Google Scholar]
- Giordan, D.; Hayakawa, Y.; Nex, F.; Remondino, F.; Tarolli, P. The use of remotely piloted aircraft systems (RPASs) for natural hazards monitoring and management. Nat. Hazards Earth Syst. Sci.
**2018**, 18, 1079–1096. [Google Scholar] [CrossRef] [Green Version] - Hildmann, H.; Kovacs, E. Using Unmanned Aerial Vehicles (UAVs) as Mobile Sensing Platforms (MSPs) for Disaster Response, Civil Security and Public Safety. Drones
**2019**, 3, 59. [Google Scholar] [CrossRef] [Green Version] - Iglesias, L.; De Santos-Berbel, C.; Pascual, V.; Castro, M. Using Small Unmanned Aerial Vehicle in 3D Modeling of Highways with Tree-Covered Roadsides to Estimate Sight Distance. Remote Sens.
**2019**, 11, 2625. [Google Scholar] [CrossRef] [Green Version] - Hein, D.; Kraft, T.; Brauchle, J.; Berger, R. Integrated UAV-Based Real-Time Mapping for Security Applications. ISPRS Int. J. Geo-Inf.
**2019**, 8, 219. [Google Scholar] [CrossRef] [Green Version] - Jeon, I.; Ham, S.; Cheon, J.; Klimkowska, A.M.; Kim, H.; Choi, K.; Lee, I. A real time drone mapping platform for marine surveillance. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, XLII-2/W, 385–391. [Google Scholar] [CrossRef] [Green Version] - Mandlburger, G.; Pfennigbauer, M.; Schwarz, R.; Flöry, S.; Nussbaumer, L. Concept and Performance Evaluation of a Novel UAV-Borne Topo-Bathymetric LiDAR Sensor. Remote Sens.
**2020**, 12, 986. [Google Scholar] [CrossRef] [Green Version] - Remondino, F.; Nocerino, E.; Toschi, I.; Menna, F. A critical review of automated photogrammetric processing of large datasets. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2017**, XLII-2/W, 591–599. [Google Scholar] [CrossRef] [Green Version] - Stathopoulou, E.-K.; Welponer, M.; Remondino, F. Open source image based 3D reconstruction pipelines: Review, comparison and evaluation. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, XLII-2/W, 331–338. [Google Scholar] [CrossRef] [Green Version] - Nocerino, E.; Menna, F.; Remondino, F.S.R. Accuracy and Block Deformation Analysis in Automatic UAV and Terrestrial Photogrammetry—Lesson Learnt. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2013**, II-5/W1, 203–208. [Google Scholar] [CrossRef] [Green Version] - Barazzetti, L. Network Design in Close-Range Photogrammetry with Short Baseline Images. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2017**, IV-2/W2, 17–23. [Google Scholar] [CrossRef] [Green Version] - Farella, E.M.; Torresani, A.; Remondino, F. Quality features for the integration of terrestrial and UAV images. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, XLII-2/W, 339–346. [Google Scholar] [CrossRef] [Green Version] - Farella, E.M.; Torresani, A.; Remondino, F. Sparse point cloud filtering based on covariance features. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, XLII-2/W, 465–472. [Google Scholar] [CrossRef] [Green Version] - Klein, L.A. Sensor and Data Fusion: A Tool for Information Assessment and Decision Making; SPIE: Bellingham, WA, USA, 2004. [Google Scholar]
- Bastonero, P.; Donadio, E.; Chiabrando, F.; Spanò, A. Fusion of 3D models derived from TLS and image-based techniques for CH enhanced documentation. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2014**, II–5, 73–80. [Google Scholar] [CrossRef] [Green Version] - Ramos, M.M.; Remondino, F. Data fusion in Cultural Heritage—A Review. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2015**, XL-5/W7, 359–363. [Google Scholar] [CrossRef] [Green Version] - Frohlich, R.; Gubo, S.; Lévai, A.; Kato, Z. 3D-2D Data Fusion in Cultural Heritage Applications. In Heritage Preservation; Springer: Singapore, 2018; pp. 111–130. [Google Scholar]
- Murtiyoso, A.; Grussenmeyer, P.; Suwardhi, D.; Awalludin, R. Multi-Scale and Multi-Sensor 3D Documentation of Heritage Complexes in Urban Areas. ISPRS Int. J. Geo-Inf.
**2018**, 7, 483. [Google Scholar] [CrossRef] [Green Version] - Noor, N.M.; Ibrahim, I.; Abdullah, A.; Abdullah, A.A.A. Information Fusion for Cultural Heritage Three-Dimensional Modeling of Malay Cities. ISPRS Int. J. Geo-Inf.
**2020**, 9, 177. [Google Scholar] [CrossRef] [Green Version] - Copa, L.; Poli, D.; Remondino, F. Fusion of Interferometric SAR and Photogrammetric Elevation Data. In Land Applications of Radar Remote Sensing; InTech: London, UK, 2014. [Google Scholar]
- Nguyen, H.; Cressie, N.; Braverman, A. Multivariate Spatial Data Fusion for Very Large Remote Sensing Datasets. Remote Sens.
**2017**, 9, 142. [Google Scholar] [CrossRef] [Green Version] - Xia, J.; Yokoya, N.; Iwasaki, A. Fusion of Hyperspectral and LiDAR Data with a Novel Ensemble Classifier. IEEE Geosci. Remote Sens. Lett.
**2018**, 15, 957–961. [Google Scholar] [CrossRef] - Wiemann, S.; Bernard, L. Spatial data fusion in Spatial Data Infrastructures using Linked Data. Int. J. Geogr. Inf. Sci.
**2015**, 30, 613–636. [Google Scholar] [CrossRef] - Lau, B.P.L.; Marakkalage, S.H.; Zhou, Y.; Hassan, N.U.; Yuen, C.; Zhang, M.; Tan, U.-X. A survey of data fusion in smart city applications. Inf. Fusion
**2019**, 52, 357–374. [Google Scholar] [CrossRef] - Fraser, C.S. Network design considerations for non-topographic photogrammetry. Photogramm. Eng. Remote Sens.
**1984**, 50, 1115–1126. [Google Scholar] - Mikhail, E.M.; Bethel, J.S.M.J.C. Introduction to Modern Photogrammetry; John Wiley & Sons: New York, NY, USA, 2001. [Google Scholar]
- Luhmann, T.; Robson, S.; Kyle, S.H.I. Close Range Photogrammetry. Principles, Techniques and Applications; Whittles Publishing: Dunbeath, UK, 2011; ISBN 978-184995-057-2. [Google Scholar]
- Hosseininaveh, A.; Serpico, M.; Robson, S.; Hess, M.; Boehm, J.; Pridden, I.; Amati, G. Automatic Image Selection in Photogrammetric Multi-View Stereo mMethods; Eurographics Assoc.: Goslar, Germany, 2012. [Google Scholar]
- Alsadik, B.; Gerke, M.; Vosselman, G. Automated camera network design for 3D modeling of cultural heritage objects. J. Cult. Herit.
**2013**, 14, 515–526. [Google Scholar] [CrossRef] - Ahmadabadian, A.H.; Robson, S.; Boehm, J.; Shortis, M. Stereo-imaging network design for precise and dense 3d reconstruction. Photogramm. Rec.
**2014**, 29, 317–336. [Google Scholar] [CrossRef] - Voltolini, F.; Remondino, F.; Pontin, M.G.L. Experiences and considerations in image-based-modeling of complex architectures. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2006**, XXXVI–5, 309–314. [Google Scholar] - El-Hakim, S.F.; Beraldin, J.A.B.F. Critical factors and configurations for practical image-based 3D modeling. In Proceedings of the 6th Conference Optical 3D Measurements Techniques, Zurich, Switzerland, 22–25 September 2003; pp. 159–167. [Google Scholar]
- Fraser, C.S.; Woods, A.; Brizzi, D. Hyper Redundancy for Accuracy Enhancement in Automated Close Range Photogrammetry. Photogramm. Rec.
**2005**, 20, 205–217. [Google Scholar] [CrossRef] - James, M.R.; Robson, S. Mitigating systematic error in topographic models derived from UAV and ground-based image networks. Earth Surf. Process. Landforms
**2014**, 39, 1413–1420. [Google Scholar] [CrossRef] [Green Version] - Nocerino, E.; Menna, F.; Remondino, F. Accuracy of typical photogrammetric networks in cultural heritage 3D modeling projects. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2014**, XL–5, 465–472. [Google Scholar] [CrossRef] [Green Version] - Dall’Asta, E.; Thoeni, K.; Santise, M.; Forlani, G.; Giacomini, A.; Roncella, R. Network Design and Quality Checks in Automatic Orientation of Close-Range Photogrammetric Blocks. Sensors
**2015**, 15, 7985–8008. [Google Scholar] [CrossRef] [PubMed] - Abate, D.; Murtiyoso, A. Bundle adjustment accuracy assessment of unordered aerial dataset collected through Kite platform. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2019**, 42, 1–8. [Google Scholar] [CrossRef] [Green Version] - Han, X.-F.; Jin, J.S.; Wang, M.-J.; Jiang, W.; Gao, L.; Xiao, L. A review of algorithms for filtering the 3D point cloud. Signal Process. Image Commun.
**2017**, 57, 103–112. [Google Scholar] [CrossRef] - Balta, H.; Velagic, J.; Bosschaerts, W.; De Cubber, G.; Siciliano, B. Fast Statistical Outlier Removal Based Method for Large 3D Point Clouds of Outdoor Environments. IFAC-Pap.
**2018**, 51, 348–353. [Google Scholar] [CrossRef] - Schall, O.; Belyaev, A.; Seidel, H.-P. Robust filtering of noisy scattered point data. In Proceedings of the Eurographics/IEEE VGTC Symposium Point-Based Graphics, Stony Brook, NY, USA, 21–22 June 2005. [Google Scholar]
- Narváez, E.A.; Narváez, N.E. Point cloud denoising using robust principal component analysis. In Proceedings of the First International Conference on Computer Graphics Theory and Applications, SciTePress-Science and and Technology Publications, Setubal, Portugal, 25–28 February 2006. [Google Scholar]
- Kalogerakis, E.; Nowrouzezahrai, D.; Simari, P.; Singh, K. Extracting lines of curvature from noisy point clouds. Comput. Des.
**2009**, 41, 282–292. [Google Scholar] [CrossRef] - Mallet, C.; Bretar, F.; Roux, M.; Soergel, U.; Heipke, C. Relevance assessment of full-waveform lidar data for urban area classification. ISPRS J. Photogramm. Remote Sens.
**2011**, 66, S71–S84. [Google Scholar] [CrossRef] - Kang, C.L.; Lu, T.N.; Zong, M.M.; Wang, F.; Cheng, Y. Point Cloud Smooth Sampling and Surface Reconstruction Based on Moving Least Squares. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2020**, XLII-3/W, 145–151. [Google Scholar] [CrossRef] [Green Version] - Barazzetti, L.; Remondino, F.; Scaioni, M. Extraction of Accurate Tie Points for Automated Pose Estimation of Close-Range Blocks. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2010**, 38-3A, 151–156. [Google Scholar] - Apollonio, F.I.; Ballabeni, A.; Gaiani, M.; Remondino, F. Evaluation of feature-based methods for automated network orientation. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2014**, XL–5, 47–54. [Google Scholar] [CrossRef] [Green Version] - Karel, W.; Ressl, C.; Pfeifer, N. Efficient Orientation and Calibration of Large Aerial Blocks of Multi-Camera Platforms. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2016**, XLI-B1, 199–204. [Google Scholar] [CrossRef] - Barba, S.; Barbarella, M.; Di Benedetto, A.; Fiani, M.; Gujski, L.; Limongiello, M. Accuracy Assessment of 3D Photogrammetric Models from an Unmanned Aerial Vehicle. Drones
**2019**, 3, 79. [Google Scholar] [CrossRef] [Green Version] - Calantropio, A.; Deseilligny, M.P.; Rinaudo, F.; Rupnik, E. Evaluation of photogrammetric block orientation using quality descriptors from statistically filtered tie points. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2018**, XLII–2, 185–191. [Google Scholar] [CrossRef] [Green Version] - Giang, N.T.; Muller, J.-M.; Rupnik, E.; Thom, C.; Pierrot-Deseilligny, M. Second Iteration of Photogrammetric Processing to Refine Image Orientation with Improved Tie-Points. Sensors
**2018**, 18, 2150. [Google Scholar] [CrossRef] [Green Version] - Wolff, K.; Kim, C.; Zimmer, H.; Schroers, C.; Botsch, M.; Sorkine-Hornung, O.; Sorkine-Hornung, A. Point Cloud Noise and Outlier Removal for Image-Based 3D Reconstruction. In Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 25–28 October 2016. [Google Scholar]
- Rakotosaona, M.-J.; La Barbera, V.; Guerrero, P.; Mitra, N.J.; Ovsjanikov, M. Pointcleannet: Learning to Denoise and Remove Outliers from Dense Point Clouds. Comput. Graph. Forum
**2019**, 39, 185–203. [Google Scholar] [CrossRef] [Green Version] - Roncella, R.; Re, C.; Forlani, G. Performance evaluation of a structure and motion strategy in architecture and Cultural Heritage. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2012**, XXXVIII-, 285–292. [Google Scholar] [CrossRef] [Green Version] - Han, J.; Kamber, M.; Pei, J. Advanced Pattern Mining. In Data Mining; Elsevier: Amsterdam, The Netherlands, 2012; pp. 279–325. [Google Scholar]
- Mauro, M.; Riemenschneider, H.; Van Gool, L.; Signoroni, A.; Leonardi, R. A unified framework for content-aware view selection and planning through view importance. In Proceedings of the Proceedings of the British Machine Vision Conference 2014, British Machine Vision Association, Nottingham, UK, 1–5 September 2014. [Google Scholar]
- Howell, D.C. Statistical Methods in Human Sciences; Wadsworth: New York, NY, USA, 1998. [Google Scholar]
- Miller, J. Reaction time analysis with outlier exclusion: Bias varies with sample size. Q. J. Exp. Psychol.
**1991**, 43, 907–912. [Google Scholar] [CrossRef] [PubMed] - Leys, C.; Ley, C.; Klein, O.; Bernard, P.; Licata, L. Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. J. Exp. Soc. Psychol.
**2013**, 49, 764–766. [Google Scholar] [CrossRef] [Green Version] - Rodriguez-Gonzálvez, P.; Garcia-Gago, J.; Gomez-Lahoz, J.; González-Aguilera, D. Confronting Passive and Active Sensors with Non-Gaussian Statistics. Sensors
**2014**, 14, 13759–13777. [Google Scholar] [CrossRef] [Green Version] - Toschi, I.; Rodriguez-Gonzálvez, P.; Remondino, F.; Minto, S.; Orlandini, S.F.A. Accuracy evaluation of a mobile mapping system with advanced statistical methods. In Proceedings of the 2015 3D Virtual Reconstruction and Visualization of Complex Architectures, Avila, Spain, 25–27 February 2015; 40, p. 245. [Google Scholar]
- Fiorillo, F.; Jiménez Fernández-Palacios, B.; Remondino, F.; Barba, S. 3D Surveying and modelling of the archaeological area of Paestum, Italy. Virtual Archaeol. Rev.
**2013**, 4, 55–60. [Google Scholar] [CrossRef] - Nex, F.; Gerke, M.; Remondino, F.; Przybilla, H.-J.; Bäumker, M.; Zurhorst, A. ISPRS Benchmark for multi-platform photogrammetry. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci.
**2015**, II-3/W4, 135–142. [Google Scholar] [CrossRef] [Green Version] - Agisoft LLC. Agisoft Metashape (Version 1.6.3); Agisoft LLC: Saint Petersburg, Russia, 2020. [Google Scholar]

**Figure 1.**The flowchart of the proposed pipeline. The standard photogrammetric workflow (left block) is enriched with the filtering procedure developed in Python (right block).

**Figure 2.**Several image network configurations in the Modena Cathedral subset: terrestrial-convergent (

**a**), terrestrial-parallel (

**b**), unmanned aerial vehicle (UAV) (

**c**) and terrestrial and UAV combined (

**d**).

**Figure 3.**Visualization of the a-posteriori standard deviation (σ) precisions computed on the original (not filtered) sparse point clouds: terrestrial-convergent (

**a**), terrestrial-parallel (

**b**), UAV (

**c**) and terrestrial and UAV combined (

**d**). σ values in (

**a**,

**b**) are visualized in the range [1–30 mm], for (

**c**,

**d**) in the range [1–100 mm].

**Figure 4.**A graphical representation of the considered quality features computed for each 3D tie point: (

**a**) re-projection error; (

**b**) multiplicity; (

**c**) intersection angle; (

**d**) a-posteriori standard deviation.

**Figure 5.**Some examples of the Modena Cathedral dataset: (

**a**) and (

**b**) terrestrial images; (

**c**) UAV-based image.

**Figure 6.**An example of quantile-quantile (Q-Q) plots for the unfiltered (

**a**) and filtered (

**b**) a-posteriori standard deviation values (mm) and related Skewness and Kurtosis values for the Modena Cathedral dataset. The quantiles of input sample (vertical axis) are plotted against the standard normal quantiles (horizontal axis). In the filtered case (

**b**), values approximate better to the straight line, assuming a more relevant normal behaviour.

**Figure 8.**Qualitative (visual) evaluation and comparisons of the dense point clouds derived from the standard photogrammetric workflow (

**a**,

**c**) and after the proposed filtering method (

**b**,

**d**). Less noisy data and more details are clearly visible in (

**b**,

**d**).

**Figure 10.**Selected five areas for cloud-to-cloud distance analyses between the laser scanning ground truth and the two photogrammetric clouds.

**Figure 11.**Qualitative evaluation and comparisons of the dense point clouds derived from the standard photogrammetric workflow (

**a**–

**c**)) and after the proposed filtering method (

**d**–

**f**).

**Figure 14.**Qualitative evaluation and comparisons of the dense point clouds derived from the standard photogrammetric workflow (

**a**–

**c**) and after the proposed filtering method (

**d**–

**f**) images. Less noisy data and more details are clearly visible in the results obtained with the proposed method.

**Figure 15.**Some examples of the terrestrial and UAV-based subset of the Dortmund benchmark: (

**a**) terrestrial image; (

**b**) UAV-based image.

**Figure 16.**Selected five areas for cloud-to-cloud distance analyses between the laser scanning ground truth and the two photogrammetric clouds.

**Figure 17.**Qualitative evaluation and comparisons of the dense point clouds derived from the standard photogrammetric workflow (

**a**,

**c**) and after the proposed filtering method (

**b**,

**d**).

**Table 1.**Considered image networks and variation of object coordinate precisions (in mm). In all datasets, the z-axis points upward.

Network | Numb. of Images | Numb. of 3D Tie Points | Average σ_{x}[mm] | Average σ_{y}[mm] | Average σ_{z}[mm] |
---|---|---|---|---|---|

1a | 15 | ≃104 K | 4.36 | 2.04 | 0.78 |

1b | 12 | ≃70 K | 1.63 | 7.02 | 3.52 |

1c | 16 | ≃149 K | 17.65 | 32.43 | 40.89 |

1d | 43 | ≃440 K | 12.76 | 17.14 | 14.63 |

**Table 2.**Improvements (+) and worsening (−) of median values of the considered quality feature when only one feature is used to filter the 3D tie points.

Variations of Single Quality Parameters | |||||
---|---|---|---|---|---|

Re-Proj. Error | Multiplicity | Inters. Angle | A-Post. Std. Dev. | ||

Employed feature for point filtering | Re-proj. Error | +52% | 0% | −24% | −47% |

Multiplicity | −10% | +50% | +67% | +2% | |

Int. Angle | +2% | +50% | +67% | +10% | |

A-Post. std. dev. | −8% | +33% | +35% | +11% |

**Table 3.**Average median improvement on quality parameters after applying different filtering thresholds.

Removed 3D Points | Re-Projection Error | Multiplicity | Inters. Angle | A-Post. St. Dev. | |
---|---|---|---|---|---|

1 | ~305 k (~75%) | +16% | +60% | +64% | +12% |

2 | ~289 k (~71%) | +16% | +50% | +61% | +15% |

3 | ~190 k (~47%) | +30% | +33% | +41% | +19% |

4 | ~167 k (41%) | +32% | +33% | +34% | +16% |

**Table 4.**Median, mean and standard deviation values for the quality features computed on the original (not filtered) sparse point cloud (~405,000 3D tie points).

Re-Projection Error (px) | Multiplicity | Intersection Angle (deg) | A-Post. Std. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 0.963 | 2 | 12.017 | 5.222 |

MEAN | 1.454 | 3.344 | 16.806 | 54.519 |

STD. DEV. | 1.446 | 2.762 | 16.742 | 244.976 |

**Table 5.**Median, mean and standard deviation values for the quality features values computed on the filtered sparse point cloud (ca 280,000 3D tie points).

Re-Proj. Error (px) | Multiplicity | Inter. Angle (deg) | A-Post. Std. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 0.827 (−14%) | 4 (+50%) | 31.048 (+61%) | 4.532 (−15%) |

MEAN | 1.008 (−44%) | 5.274 (+37%) | 33.915 (+50%) | 6.879 (>>−100%) |

STD. DEV. | 0.707 (−51%) | 3.564 (+22%) | 16.171 (−3%) | 7.395 (>>−100%) |

**Table 6.**RMSEs (Root Mean Square Errors) of plane fitting on five sub-areas for the dense point clouds derived from the original and filtered results.

Sub-Area | Original (mm) | Filtered (mm) | Variation |
---|---|---|---|

AREA 1 | 3.022 | 2.027 | (−33%) |

AREA 2 | 5.198 | 2.370 | (−54%) |

AREA 3 | 2.721 | 2.137 | (−21%) |

AREA 4 | 52.805 | 7.878 | (−85%) |

AREA 5 | 3.774 | 3.229 | (−14%) |

Average Variation | (~−41%) |

**Table 7.**Cloud-to-cloud distance analyses between the laser scanning and the photogrammetric point clouds derived from the original and filtered results.

Sub-Area | Original (mm) | Filtered (mm) | Variation | ||
---|---|---|---|---|---|

Mean | Std. Dev. | Mean | Std. Dev. | ||

AREA 1 | 9.767 | 17.762 | 6.443 | 19.089 | (−40%) |

AREA 2 | 13.327 | 31.877 | 10.452 | 33.685 | (−23%) |

AREA 3 | 29.906 | 51.526 | 25.044 | 41.812 | (−17%) |

AREA 4 | 37.972 | 81.564 | 32.390 | 73.520 | (−16%) |

AREA 5 | 41.344 | 60.513 | 37.883 | 58.847 | (−7%) |

Average Variation | (~−21%) |

**Table 8.**Median, mean and standard deviation values for quality feature values computed on the original (not filtered) sparse point cloud (~640,000 3D tie points).

Re-Projection Error (px) | Multiplicity | Intersection Angle (deg) | A-Post. Std. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 1.008 | 3 | 11.710 | 6.95 |

MEAN | 1.239 | 4.019 | 18.597 | 108.935 |

STD. DEV. | 0.881 | 3.529 | 19.712 | 2931.512 |

**Table 9.**Values of quality parameters computed on the filtered sparse point cloud (~187,000 3D tie points) and average variations of the metrics.

Re−Projection Error (px) | Multiplicity | Intersection Angle (deg) | A-Post. Std. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 0.773 (−23%) | 5 (+40%) | 34.538 (+66%) | 4.565 (−34%) |

MEAN | 0.899 (−27%) | 6.570 (+39%) | 38.413 (+52%) | 7.371 (>−100%) |

STD. DEV. | 0.532 (−40%) | 4.094 (+14%) | 20.324 (+3%) | 34.378 (>>−100%) |

**Table 10.**Check-point RMSEs in the original and filtered sparse point cloud and variation of the obtained values.

RMSExy (px) | RMSEx (mm) | RMSEy (mm) | RMSEy (mm) | RMSE (mm) | |
---|---|---|---|---|---|

Original | 0.351 | 10.728 | 15.905 | 18.886 | 26.028 |

Filtered | 0.319 | 8.235 | 5.326 | 11.102 | 16.332 |

Variation | ~−10% | ~−30% | >−100% | ~−70% | ~−59% |

**Table 11.**Cloud-to-cloud distance analyses on the original and filtered dense cloud and average variation of the mean values.

Sub-Area | Original (mm) | Filtered (mm) | Mean Variation | ||
---|---|---|---|---|---|

Mean | St. Deviation | Mean | St. Deviation | ||

AREA 1 | 59.394 | 92.244 | 52.529 | 86.107 | (~−10%) |

AREA 2 | 59.358 | 90.843 | 26.768 | 32.289 | (~−54%) |

AREA 3 | 49.3587 | 78.654 | 20.007 | 37.883 | (~−59%) |

AREA 4 | 60.956 | 98.024 | 36.479 | 76.630 | (~−41%) |

AREA 5 | 63.581 | 106.752 | 27.064 | 43.042 | (~−58%) |

Average Variation | (~−44%) |

**Table 12.**Median, mean and standard deviation values for the quality features values computed on the original (not filtered) sparse point cloud (~1.2 mil. 3D tie points).

Re-Projection Error (px) | Multiplicity | Intersection Angle (degree) | A-Post. Std. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 13.745 | 3 | 11.585 | 5.89 |

MEAN | 14.181 | 3.237 | 17.361 | 285.38 |

ST. DEV. | 11.229 | 1.712 | 17.506 | 409.94 |

**Table 13.**Values of the quality parameters computed on the filtered sparse point cloud and average variation of the results.

Re-Projection Error (px) | Multiplicity | Intersection Angle (degree) | A-Post. St. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 3.822 (−72%) | 4 (+25%) | 26.913 (+57%) | 4.956 (~−19%) |

MEAN | 5.008 (−65%) | 4.324 (+25%) | 29.991 (+42%) | 15.6908 (>−100%) |

ST. DEV. | 3.665 (−67%) | 1.765 (+3%) | 15.326 (−12%) | 274.57 (~−49%) |

**Table 14.**Cloud-to-cloud distance analysis on the original and filtered dense cloud and average variation of the mean values.

Sub-Area | Original Dense Cloud (mm) | Filtered Dense Cloud (mm) | Mean Variation | ||
---|---|---|---|---|---|

Mean | St. Dev. | Mean | St. Dev. | ||

AREA 1 | 63.694 | 35.645 | 20.548 | 17.854 | (~−67%) |

AREA 2 | 49.796 | 22.006 | 18.229 | 18.801 | (~−64%) |

AREA 3 | 100.720 | 52.869 | 52.584 | 46.421 | (~−48%) |

AREA 4 | 123.237 | 24.683 | 61.367 | 17.192 | (~−50%) |

AREA 5 | 52.432 | 33.733 | 40.717 | 18.122 | (~−21%) |

Average Variation | (~−50%) |

**Table 15.**Median, mean and standard deviation values for the quality parameter values computed on the original (not filtered) sparse point cloud (~315,000 3D tie points).

Re-Projection Error (px) | Multiplicity | Intersection Angle (degree) | A-Post. St. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 0.859 | 2 | 9.829 | 11.858 |

MEAN | 1.035 | 3.409 | 14.989 | 41.612 |

ST. DEV. | 0.775 | 2.458 | 14.833 | 239.027 |

**Table 16.**Values of the quality parameters computed on the filtered sparse point cloud and average variation of the results.

Re-Projection Error (px) | Multiplicity | Intersection Angle (degree) | A-Post. St. Dev. (mm) | |
---|---|---|---|---|

MEDIAN | 0.656 (−24%) | 3 (+33%) | 11.201 (+12%) | 6.938 (−71%) |

MEAN | 0.697 (−33%) | 3.478 (+2%) | 16.048 (+7%) | 21.270 (−96%) |

ST. DEV. | 0.347 (−55%) | 2.406 (−2%) | 14.756 (−1%) | 159.97 (−49%) |

**Table 17.**Results of the cloud-to-cloud distance analyses on the original and filtered dense clouds and average variation of the mean values.

Sub-Area | Original Dense Cloud (mm) | Filtered Dense Cloud (mm) | Mean Variation | ||
---|---|---|---|---|---|

Mean | Std. Dev. | Mean | Std. Dev. | ||

AREA 1 | 13.736 | 28.066 | 9.606 | 27.952 | (~−36%) |

AREA 2 | 19.090 | 41.204 | 9.703 | 33.258 | (~−53%) |

AREA 3 | 10.663 | 7.520 | 7.589 | 5.819 | (~−36%) |

AREA 4 | 9.391 | 5.100 | 2.698 | 4.255 | (~−67%) |

AREA 5 | 5.284 | 39.184 | 3.152 | 35.025 | (~−50%) |

Average Variation | (~−48%) |

**Table 18.**Checkpoint RMSEs in the original and filtered sparse point cloud and variation of the obtained values.

RMSExy (px) | RMSEx (mm) | RMSEy (mm) | RMSEy (mm) | RMSE | |
---|---|---|---|---|---|

Original | 0.420 | 8.92 | 8.82 | 9.51 | 15.74 |

Filtered | 0.338 | 6.17 | 4.60 | 8.12 | 11.18 |

Variation | ~−20% | ~−31% | ~−48% | ~−15% | ~−30% |

**Table 19.**Summary of average 3D reconstruction improvements in the considered four datasets verified with the available ground truth data.

Dataset | Plane Fitting | Cloud to Cloud Distance | Check Points RMSE |
---|---|---|---|

Modena Cathedral | (~41%) | (~21%) | - |

Nettuno temple | - | (~44%) | (~59%) |

WWI Fortification | - | (~50%) | - |

Dortmund Benchmark | - | (~48%) | (~30%) |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Farella, E.M.; Torresani, A.; Remondino, F.
Refining the Joint 3D Processing of Terrestrial and UAV Images Using Quality Measures. *Remote Sens.* **2020**, *12*, 2873.
https://doi.org/10.3390/rs12182873

**AMA Style**

Farella EM, Torresani A, Remondino F.
Refining the Joint 3D Processing of Terrestrial and UAV Images Using Quality Measures. *Remote Sensing*. 2020; 12(18):2873.
https://doi.org/10.3390/rs12182873

**Chicago/Turabian Style**

Farella, Elisa Mariarosaria, Alessandro Torresani, and Fabio Remondino.
2020. "Refining the Joint 3D Processing of Terrestrial and UAV Images Using Quality Measures" *Remote Sensing* 12, no. 18: 2873.
https://doi.org/10.3390/rs12182873