Designing and Evaluating a User-Oriented Calibration Field for the Target-Based Self-Calibration of Panoramic Terrestrial Laser Scanners

Abstract

Calibration of terrestrial laser scanners (TLSs) is one of the fundamental tasks for assuring the high measurement accuracy required by an increasing number of end-users. Nevertheless, the development of user-oriented calibration approaches is still an active topic of research. The calibration fields for the target-based self-calibration of TLSs described in the literature are based on the quasi-random distribution of a high number of targets, and they rely on heavy redundancy. This redundancy assures highly accurate calibration results, however, with the price of reduced efficiency. In contrast, this work follows the design, implementation, and validation of a user-oriented, cost-efficient calibration field intended for TLS calibration prior to measurement campaigns. Multiple goals and constraints are placed upon the design of the calibration field, such as comprehensive calibration for high-end panoramic TLSs considering all relevant mechanical misalignments, delivering stable and reusable calibration parameters, increasing calibration efficiency by minimizing calibration-field assembly, measurement acquisition and processing time through reducing the number of targets and scanner stations, as well as estimating calibration parameters with predefined quality criteria. The calibration field design was derived through a series of simulation experiments and it was compared with the current state of the art. The simulations indicate comparable calibration results, with eight times smaller number of targets (14 instead of 120). The implemented calibration field was tested on a range of instruments, successfully improving the measurement quality, both in situ and in the subsequent applications.

1. Introduction

Despite the enormous efforts of TLS manufacturers, fabrication procedures are not flawless. The remaining small misalignments of instruments’ mechanical components can cause systematic measurement errors, reducing the point cloud quality. Hence, these systematic errors need to be mathematically modeled. Additionally, the relations of the mechanical components can vary over time. Therefore, regular calibration is necessary to ensure the sufficient measurement quality needed for stringent engineering tasks, such as deformation monitoring [1] and reverse engineering [2].

To that end, multiple self-calibration approaches have been developed, assuring up-to-date calibration parameters (CPs) and successfully reducing the measurement uncertainty without requiring instruments of higher accuracy [3]. These approaches rely on different objects as calibration references, such as planes [4,5], cylinders [6], paraboloids [7], signalized target points [8,9,10,11,12,13,14], or automatically detected salient points [15,16], which are found or deliberately placed in the surroundings.

To achieve accurate and unbiased measurements, the calibration should be comprehensive, modeling all relevant mechanical misalignments [3,8,11,17]. Therefore, the measurement geometry needs to be sufficiently sensitive to allow separation and unbiased estimation of all CPs. This is challenging, mainly due to the high correlations between some of the CPs [18,19]. Such sensitive measurement geometry can only be achieved by an adequate distribution of reference objects in the instrument’s field-of-view and an adequate distribution and number of measurements. Several works addressed this problem, outlining the measurement geometry characteristics necessary for successful comprehensive target-based calibration [8,18,20]. However, for the remaining calibration approaches, similar guidelines are still missing. Some studies avoid this complexity by estimating only the most relevant subset of calibration parameters [9,18]. However, such a calibration approach is overly simplified, as it is known that the number of existing mechanical misalignments is substantially larger [3,8,21].

Therefore, we build upon the target-based TLS self-calibration as the best-studied calibration approach, which has successfully been used for comprehensive calibration in multiple experiments. Current implementations of the calibration fields for target-based calibration use high quantities of well-distributed targets, and they rely on this heavy redundancy to assure accurate and unbiased estimation of CPs. As a result, they lack efficiency due to complex calibration field assembly and long measurement acquisition and processing time, which was recognized as the main disadvantage of this approach in many instances [3,5,6,10,22,23,24,25,26]. This drawback hinders this approach to be well accepted as a standard user-oriented TLS calibration method.

The main goals of this study are:

• To design a user-oriented calibration field (in sense of cost-efficiency) that fulfills certain design criteria;
• To evaluate this calibration field based on several calibration attempts using a wide range of TLSs with a special focus on high-end instruments;
• To demonstrate the usability of the calibration parameters in situ and a posteriori.

Regarding the first goal, we want to increase efficiency by reducing the calibration field complexity. This is achieved by utilizing a small number of carefully selected target and scanner station positions, with the constraints of achieving the desired CP quality (desired precision, correlations, and reliability). The selected criteria are discussed in detail as the main guidance for the target reduction and the calibration field design. For comparison, a current typical calibration field consists of 120 well-distributed targets [3]. It should be noted that a few studies have already addressed the problem of calibration efficiency by reducing the number of targets [10,27]. However, they focused on estimating only a smaller subset of the CPs.

Regarding the second goal, the phrase “high-end denotes TLSs with high-quality components and measurement characteristics, such as two-face measurements, dynamic compensators for automatic leveling, low-measurement noise, and angular encoders with multiple reading heads, which reduces the amount of overall possible systematic errors. Examples of such devices are Leica ScanStation P-series and Zoller and Fröhlich Imager series instruments. We focus primarily on these devices, as they are most often used for highly demanding engineering tasks, and therefore, their successful calibration is of highest relevance. However, the calibration field designed herein can also be used to calibrate the instruments that do not fall into this category.

Regarding the third goal, we investigate the improvement in the point cloud uncertainty when the estimated calibration parameters are used. So far, this has only been investigated for in situ scans. Using the estimated parameters for scans of unrelated objects with different time offsets has never been documented in the literature before. The presented results give an impression of how much we can rely on such CPs for scans that are collected some time after the calibration.

This work is structured as follows. The review of the current state of the art and the aims of this study are presented in the current section. Section 2 gives an overview of the theoretical background (materials and methods) necessary to understand the conducted experiments. Section 3 presents the experimental results, the main experimental findings, and the corresponding discussion. Finally, Section 4 gives the main conclusions of our study.

2. Materials and Methods

This section firstly provides a brief summary of the theoretical basis for the TLS calibration (Section 2.1), and secondly it explains the main ideas behind the calibration field design (Section 2.2).

2.1. Basics of TLS Calibration

2.1.1. Calibration Parameters (CPs)

For the TLS calibration, it is necessary to transform the Cartesian coordinates back to spherical ones, mimicking the original observations (Equations (1)–(3)):

$$r_j^i=\sqrt{x_j^i{}^2+y_j^i{}^2+z_j^i{}^2}$$

(1)

$${\phi }_j^i=arctan\left(\frac{x_j^i}{y_j^i}\right)$$

(2)

$${\theta }_j^i=\arccos ( \frac{z_j^i}{r_j^i} )$$

(3)

where $r_j^i,{\phi }_j^i,{\theta }_j^i$ denote the range, horizontal, and vertical angle measurements; and $x_j^i$, $y_j^i, z_j^i$ are Cartesian coordinates of the i^th point measured from the j^th scanner station. To account for quadrants, the horizontal angles are corrected by adding 180° or 360° where necessary, and vertical angles are corrected by deducting the calculated value from 360° where necessary. This angle parameterization is adopted from [28], and it is just one of several parametrizations found in the literature [21,29]. This parameterization should be adapted with respect to the scanning mechanism of the used instrument [30].

From the variety of different CP functional models in the literature [8,21,31,32], we based our design on the functional model developed by the National Institute of Standards and Technology (NIST), which is the first one to describe the genuine geometrical relations between the mechanical misalignments and the TLS observations [14,21]. The latter parameterization was proven valid for the majority of panoramic TLSs in a large run-off experiment, where several major manufacturing companies declared it as functional [33]. This model is abbreviated herein because in the case of the high-end TLSs, the errors related to angular encoders can be safely neglected [28,34], which reduces the overall list of relevant CPs from 18 to 10:

$$∆r_j^i=x_2\sin ({\theta }_j^i) + x_{10}+v_{r_{j }^i}$$

(4)

$$∆{\phi }_j^i=\frac{x_{1z}}{r_{j }^i\tan ({\theta }_j^i)}+\frac{x_3}{r_j^i \sin ({\theta }_j^i)}+\frac{x_{5z}-x_7}{\tan ({\theta }_j^i)}+\frac{2x_6}{sin({\theta }_j^i)}+\frac{x_{1n}}{r_{j }^i}+v_{{\phi }_{j }^i}$$

(5)

$$∆{\theta }_j^i=\frac{(x_{1n}+x_2) \cos ({\theta }_{j }^i)}{r_{j }^i}+x_4+x_{5n}\cos ({\theta }_{j }^i)-\frac{x_{1z} sin({\theta }_{j }^i)}{r_{j }^i}- x_{5z} sin({\theta }_{j }^i)+v_{{\theta }_{j }^i}$$

(6)

where $∆r_j^i,∆{\phi }_j^i,∆{\theta }_j^i$ are measurement errors, which are further decomposed on $v_{r_{j }^i}$, $v_{{\phi }_{j }^i}, v_{{\theta }_{j }^i}$, the calibration adjustment residuals describing the random errors, and $x_{1n},x_2,\dots$, the calibration parameters describing the systematic errors (

Table 1. Relevant calibration parameters for high-end terrestrial laser scanners (TLSs).

CP	Description
x1n	Horizontal beam offset **
x1z	Vertical beam offset **
x2	Horizontal axis offset
x3	Mirror offset
x4	Vertical index offset
x5n	Horizontal beam tilt
x5z	Vertical beam tilt **
x6	Mirror tilt
x7	Horizontal axis error (tilt)
x10	Rangefinder offset *

* CPs not sensitive to two-face measurements. ** CPs partially sensitive to two-face measurements.

). Hence, this abbreviation simplifies the task of designing the calibration field. Similarly to total stations [35], high-end panoramic TLSs can utilize two-face measurements to estimate the majority of the CPs (Table 1) using two consecutive scans (front/back and back/front scan) from a single scanner station without a reference [21,28]. Other panoramic TLSs are restricted by the software limitations. This characteristic has a strong influence on the results of TLS calibration, and consequently, on the required measurement configuration (Section 3.1).

2.1.2. Functional Model

The algorithm of the target-based self-calibration is generally realized as the least-squares adjustment, and it is based on the functional model of rigid body movement or photogrammetric bundle adjustment:

$${\mathit{f}}_{\mathit{j}}^{\mathit{i}} = {\mathit{R}}_{(\mathit{k},\mathit{\varphi },\mathit{\omega }) }^{\mathit{i}}\mathit{x}\mathit{y}{\mathit{z}}_{\mathit{j}}^{\mathit{i}}+ {\mathit{T}}_{(\mathit{X},\mathit{Y},\mathit{Z})}^{\mathit{i}} - \mathit{X}\mathit{Y}{\mathit{Z}}_{\mathit{j}}^{\mathit{r}\mathit{e}\mathit{f}.} =\mathbf{0},$$

(7)

where i = 1,2…,s; j = 1,2…,p; s and p are total numbers of scanner stations and targets used in the experiment. The estimated parameters are separated in the rotation matrix ${\mathit{R}}_{(\mathit{k},\mathit{\varphi },\mathit{\omega }) }^{\mathit{i}}$, the translation vector ${\mathit{T}}_{(\mathit{X},\mathit{Y},\mathit{Z})}^{\mathit{i}}$ (jointly called exterior orientation parameters, EOPs), the vector of target coordinates in the reference coordinate system $\mathit{X}\mathit{Y}{\mathit{Z}}_{\mathit{j}}^{\mathit{r}\mathit{e}\mathit{f}.}$ (also called object points, OPs), and the calibration parameters (CPs).

The measurement vector from the scanner station i to the target j in the local coordinate system is equal to

$$\mathit{x}\mathit{y}{\mathit{z}}_j^i=\left[\begin{array}{c}{x}_j^i\\ y_j^i\\ z_j^i\end{array}\right]=\left[\begin{array}{c}(r_j^i+∆r_j^i)\sin ({\theta }_j^i+∆{\theta }_j^i)\sin ({\phi }_j^i+∆{\phi }_j^i)\\ (r_j^i+∆r_j^i)\sin ({\theta }_j^i+∆{\theta }_j^i)\cos ({\phi }_j^i+∆{\phi }_j^i)\\ (r_j^i+∆r_j^i)\cos ({\theta }_j^i+∆{\theta }_j^i)\end{array}\right]$$

(8)

in the case of the right-handed, y-headed, clockwise-rotating coordinate system. Further, the dynamic compensator observations are added as additional conditions (if available):

$${\mathsf{\omega }}^i={\omega }_{obs}^i+{\sigma }_{\omega }$$

(9)

$${\varphi }^i={\varphi }_{obs}^i+{\sigma }_{\varphi }$$

(10)

where ${\mathsf{\omega }}^i$ and ${\varphi }^i$ are true tilts with respect to the local horizon, ${\omega }_{obs}^i$ and ${\varphi }_{obs}^i$ are observed values, while ${\sigma }_{\omega }$ and ${\sigma }_{\varphi }$ are associated uncertainties from manufacturer specifications.

Herein, we transform the least-squares adjustment algorithm based on the strict Gauss–Helmert model (GHM) into the robust parameter estimator based on the Danish method. An adequate strategy for the detection and elimination of eventual outliers is necessary, as it is known that the target center estimation process can suffer from gross errors [9,17]. If no outliers are detected, the Danish method equals to the least-squares solution. This target-based self-calibration algorithm implementation is directly adopted from Reshetyuk (2009) [9], and readers are referred to the latter literature for more information.

2.1.3. Stochastic Model

Typically, as a starting point, the stochastic models are based on values provided in the manufacturers’ specifications given for a single point accuracy. Most often these are single scalar values separately provided for the horizontal angle, vertical angle, and range measurements. Such stochastic models are furtherly iteratively refined using the variance component estimation (VCE) based on the estimated measurement residuals of three observation groups (${\sigma }_{r\left[mm\right]},{\sigma }_{\phi \left[ ″ \right]}$, and ${\sigma }_{\theta \left[ ″ \right]}$) until the global test is accepted [36].

Designing a calibration field based on simulated measurements requires more accurate a priori knowledge about the measurement uncertainty. Hence, for our simulations, we used the empirically derived stochastic model (ESM) describing the uncertainty of the repeated target center estimation. Although it does not reflect the measurement uncertainty comprehensively, it comes closer to reality than relying on the values from manufacturers’ specifications [37]. As solving the optimization problem requires having one definite stochastic model, we focus on the combination of the particular instrument type (Leica ScanStation P-series) and the particular settings, target design, and target center estimation algorithm (more in Section 3.2.1). The empirical values for the latter stochastic model are directly adopted from [20]. A previous study [37] revealed that the corresponding standard deviations ${\sigma }_{r\left[mm\right]}, {\sigma }_{\phi \left[ ″ \right]}$, and ${\sigma }_{\theta \left[ ″ \right]}$ depend on the scanner-to-target distance within the investigated ranges from 1 to 100 meters. Hence, for the simulation purposes (Section 3.1), the covariance matrix of observations (Σ) is generated as a diagonal matrix, where the uncertainty of each observation is derived by sampling the empirical values from [20], respecting the scanner-to-target distances.

However, the final implementation of the calibration field in this study uses distances ranging from approximately 3 to 20 m, where the target center uncertainty is nearly constant in each measurement direction if the metric values are observed [37]. The standard deviations expressed for Euclidian distances in the direction of ranges (${\sigma }_{r\left[mm\right]}$), as well as horizontal and vertical angle measurements (${\sigma }_{\phi \left[ mm \right]}, {\sigma }_{\theta \left[ mm\right]}$), are nearly constant ($\le$0.1 mm). As the observations ${\phi }_j^i$ and ${\theta }_j^i$ in the calibration adjustment are angular values, the corresponding standard deviations of approximately 0.1 mm need to be transformed back to angular values, which are different for each scanner-to-target distance. Hence, the covariance matrix in the case of the empirical experiments (Section 3.2) is defined as:

$$\mathbf{\Sigma } = \left[\begin{array}{ccc}{\sigma }_{r \left[mm\right]}{}^2\mathit{I}& & \\ & {({\tan }^{-1}\frac{{\sigma }_{\phi \left[mm\right]}}{r_i})}^2\mathit{I}& \\ & & {({\tan }^{-1}\frac{{\sigma }_{\theta \left[mm\right]}}{r_i})}^2\mathit{I}\end{array}\right]$$

(11)

where the angular uncertainty is recalculated for each target individually based on the metric uncertainty of the observation groups ${\sigma }_{\phi \left[ mm \right]}$ and ${\sigma }_{\theta \left[ mm\right]}$, as well as the measurement range $r_i$. This way, we achieve a more general stochastic model that is applicable for various instruments, while still preserving the trend of constant metric uncertainty in each of the three measurement directions. In the empirical experiments, the VCE is used to adjust the Σ to correctly represent the measurement uncertainty of each individual instrument. The initial values for ${\sigma }_{r \left[mm\right]}$, ${\sigma }_{\phi \left[ mm \right]}$, and ${\sigma }_{\theta \left[ mm\right]}$ are safely defined in the first iteration as $3{\sigma }_{r \left[mm\right]}$ based on the value for range accuracy in the manufacturers’ specifications (to avoid an incorrect detection of valid measurements as outliers), and further refined by VCE until the global test is accepted.

2.2. TLS Calibration in the Context of Geodetic Networks

2.2.1. Optimization of Geodetic Networks

The task of designing a TLS calibration field can be described as the design of geodetic networks. It is an optimization problem that searches for a cost-efficient solution that fulfills imposed quality criteria, and is solved analytically (only partially) [38,39] or heuristically [40,41]. The network design is divided into four interrelated sub-problems, known as orders of the design [39]: the zero-order design (ZOD), or the problem of optimal datum; first-order design (FOD), or the problem of optimal network configuration (measurement geometry); second-order design (SOD), or the problem of optimal observation weights; and the third-order design (TOD), or the problem of densification of existing networks. Niemeier (2008) [42] states that there is no analytical solution for the complete design of geodetic networks.

For the target-based calibration, the datum choice has a negligible effect on the CPs [43]. Hence, the ZOD can be neglected and the datum can be arbitrarily chosen (we implemented the minimal constraint by fixing six EOPs [44]). Regarding the SOD, the observation weights are typically chosen to reflect the uncertainty of the measurement procedures and the equipment. In the case of the TLS self-calibration, the SOD is given for a particular scanner, scanner settings, and the target design (Section 2.1.3). As we aim to establish a new calibration field, the problem of TOD is not relevant. Hence, our aim is designing the TLS calibration field with a focus on the FOD problem.

Several publications have focused on the TLS-related FOD aiming at viewpoint planning (i.e., the optimal instrument positioning to achieve a certain point cloud quality). Different heuristic algorithms were utilized to find near-optimal solutions, such as simulated annealing, greedy search, genetic algorithm, particle swarm optimization, and ant colony optimization methods [45,46,47,48]. All studies reduced the problem to 2D due to the high numerical complexity. However, this is not possible in the case of the TLS calibration, as the vertical position of the targets is of high relevance [8,18]. Hence, we obtain the design solution based on simulations, using the least computationally expensive heuristic method, the man-made trial and error analysis. The number of possible solutions is reduced based on the results from [20].

Generally, the quality of the geodetic network is characterized by its precision, reliability, and cost [49]. Based on the previous work on the topic of TLS calibration, we adopt an additional criterion—the correlation of CPs. The exact adopted optimization goals or design criteria of the calibration field design are defined and discussed in the following sections.

2.2.2. Design Criteria

Efficiency

The efficiency is the most straightforward optimization goal we want to accomplish with the calibration field design. In short, as our calibration field is user-oriented, the overall workload and time necessary for the establishment and maintenance of the calibration field, as well as the measurement acquisition and processing, needs to be minimal. This is achieved by relying on the self-calibration technique (therefore avoiding costly and time-consuming establishment of reference values) and with use of the minimal number of targets $p$ and scans or scanner stations $s$:

$$s,p\to min$$

(12)

The latter is the only true optimization goal that we directly tried to minimize, while the further goals are rather considered as constraints, giving threshold values that should not be breached.

Precision of CPs

The general goal of the calibration is that after the calibration, the influence of remaining systematic errors (i.e., bias) should be small enough so as to be insignificant (can be considered as a part of the random noise). The required precision of the CPs strongly depends on the demands and circumstances of each individual user. Namely, the overall accumulated impact of all systematic errors depends on the size, shape, and position of the object of interest in the scanner’s field of view. The final magnitude of the random noise depends on multiple factors, such as measurement geometry, surface properties, measurement settings, as well as denoising and preprocessing strategies, and it is not homogenous. Finally, the necessary quality of the final point cloud depends on the individual task.

Thus, estimating the exact threshold value for the optimization function could be a subject of multiple separate studies in the future. Herein, we simplified this problem by bounding the CPs precision threshold to the measurement uncertainty. The adopted threshold is that the standard deviation of each CP needs to be equal to or smaller than the mean target center uncertainty of all observations in the calibration field:

$${\sigma }_x\le {\overline{\sigma }}_{r \left[mm\right]}, {\overline{\sigma }}_{\phi \left[ ″ /mm \right]}, {\overline{\sigma }}_{\theta \left[ ″/mm \right]}$$

(13)

To design the calibration field, we use the empirically derived values for Leica ScanStation P-series [20]. In this concrete case, the mean expected angular uncertainty (${\overline{\sigma }}_{\phi \left[ ″ \right]},{\overline{\sigma }}_{\theta \left[ ″ \right]})$ equals approximately 0.5″. Hence, we aim to estimate all tilt/angular CPs (Table 1) with the standard deviation lower than 0.5″. As the analogous metric uncertainties ${\overline{\sigma }}_{r\left[mm\right]}$, ${\overline{\sigma }}_{\phi \left[mm \right]}$, and ${\overline{\sigma }}_{\theta \left[ mm \right]}$ equal approximately 0.1 mm, we aim at estimating all offset (metric) CPs with accuracy lower than 0.1 mm.

We consider this criterion as feasible for several reasons. First, as a rule of a thumb, the target center uncertainty is an order of magnitude smaller than single point measurement noise. Hence, at least the standard deviation of each individual CP should be below the random noise. Second, expecting to achieve much higher CP precision is unrealistic, especially if we aim for cost-efficient calibration field design. As our simulations suggest (Section 3.1.2), even with 120 targets, the CP precision is not remarkably higher than the defined threshold. Finally, we decided to bind the threshold to the target center uncertainty, because even when the stochastic model is refined through the VCE (Section 2.1.3), this way the CP precision threshold is proportionally changed and will be fulfilled.

Correlations

Low correlations assure that if one of the CPs is incorrectly or imprecisely estimated, it will not influence the remaining parameters. Therefore, low correlations reduce the possibility of biased parameter estimates, they reduce the parameter coupling, they assure avoiding singularity in the calibration adjustment (caused by perfect coupling), and they can help identify insignificant parameters. Hence, as common practice for the camera self-calibration [36], multiple previous publications have aimed to achieve low correlations between individual TLS calibration parameters [9,11,18]. To date, there is no generally accepted threshold for sufficiently low correlation of the CPs. Therefore, in this work, we adopted a threshold where the correlations between all CPs should be smaller or equal to 0.8:

$${\rho }_x\le 0.8$$

(14)

The reason for such a high value is two-fold. First, in all our simulations we could not achieve notably lower correlations for such a high number of functionally similar CPs. Even in the keypoint- based calibration approach [15], with the high number of well-distributed observations, and in the case of using 120 targets (Section 3.1.2), the correlations were similarly high or higher. Hence, it is unreasonable to expect that the cost-efficient calibration would lead to even lower CP correlations. Second, higher correlation values do not necessarily negatively influence the calibration results. If each CP is estimated with high precision, higher correlation values will not adversely influence the CPs’ quality. Herein, we guarantee a high CP precision by fulfilling the precision threshold (Equation (12)).

Reliability

Multiple publications have pointed out that the TLS calibration must be robust to outliers, which are commonly present in the observations [3,9,17,50]. Therefore, the least-squares adjustment is often substituted with robust parameter estimators that can detect and process the outliers [9]. The possibility to correctly detect outliers strongly depends on the measurement configuration. Hence, it needs to be accounted for when designing the calibration field.

In this work, we incorporated the external reliability criterion for the calibration field design. The external reliability describes the network controllability within the parameter domain. It is defined by impact factors, which quantify the maximal impact of undetected outliers on the estimated parameters [51]:

$$\nabla x={({\mathit{A}}^{\mathit{T}}{\mathit{\Sigma }}^{-1}\mathit{A} )}^{-1}{\mathit{A}}^{\mathit{T}}{\mathit{\Sigma }}^{-1}(\mathit{B}{\left[0,\dots , \nabla i,\dots ,0\right]}^{\mathit{T}})$$

(15)

where $A$ and $B$ are the Jacobian matrices with respect to the estimated parameters and observations, while $\nabla i$ is the minimum detectable outlier [52].

The reliability analysis has already been introduced as part of the calibration field design for mobile laser scanning systems [53]. Herein, we define that the maximal impact of undetected outliers (impact factor) should be equal to or smaller than the mean target center uncertainty of all observations in the calibration field:

$$\nabla x\le {\overline{\sigma }}_{r\left[mm\right]},{\overline{\sigma }}_{\phi \left[ ″ /mm \right]},{\overline{\sigma }}_{\theta \left[ ″/mm \right]}$$

(16)

In other words, the impact of outliers on the CPs should not be higher than the measurement noise. In the previous publications, the reliability was not discussed as a relevant quality criterion for the CPs, because the high redundancy of targets and scans assured the high reliability by default (as will be presented in Section 3.1.2). However, for a cost-efficient solution with a small number of observations, the network reliability must be considered.

3. Results and Discussion

This section is separated into the simulation experiment, which is the basis of the calibration field design (Section 3.1), and the empirical experiment, which is the basis of the implemented calibration field evaluation (Section 3.2).

3.1. Simulation Experiments—Designing the Calibration Field

We derived the final calibration field design using the simulation-based trial and error analysis, where we aimed to fulfil all the optimization goals presented in Equations (12)–(14) and Equation (16). Additionally, we had the constraint that the calibration field needed to be implemented in the large machine hall we dedicated for future calibration experiments (Figure 1). We selected a large hall instead of commonly used office rooms with smaller dimensions for two reasons. First, it was shown by Lichti [18] that the correlations between CPs are lower with the higher maximum range. Second, in our previous work, it was shown that the angular target center precision is higher with higher ranges [37], which increases the sensitivity of the measurement configuration to estimate the CPs [20].

We applied the simulated CPs on all simulated observations according to Equations (4)–(6). The parameter values were chosen with realistic magnitudes based on the previously conducted empirical experiment (

Table 2. Simulation results of calibration of high-end TLSs with 14 targets (4 scans).

CP	${\mathit{x}}_{\mathit{t}\mathit{r}\mathit{u}\mathit{e}}$	$\widehat{\mathit{x}}$	${\mathit{\sigma }}_{\mathit{x}}$	${\mathit{\rho }}_{\mathit{x}(\mathit{m}\mathit{a}\mathit{x})}$	with	$\nabla \mathit{x}$
${\mathit{x}}_{10}$ [mm]	–2.00	–1.99	0.02	0.39	${\mathit{x}}_7$	0.02
${\mathit{x}}_2$ [mm]	–0.20	–0.21	0.01	–0.49	${\mathit{x}}_{5\mathit{n}}$	0.01
${\mathit{x}}_3$ [mm]	–0.20	–0.20	0.00	–0.67	${\mathit{x}}_6$	0.01
${\mathit{x}}_6$ [ “ ]	–8.00	–7.99	0.05	0.72	${\mathit{x}}_{5\mathit{z}}$	0.06
${\mathit{x}}_{1\mathit{z}}$ [mm]	–0.20	–0.20	0.01	0.72	${\mathit{x}}_6$	0.02
${\mathit{x}}_7$ [ “ ]	8.00	7.36	0.47	0.75	${\mathit{x}}_{5\mathit{z}}$	0.46
${\mathit{x}}_{5\mathit{z}}$ [ “ ]	–8.00	–8.58	0.39	0.75	${\mathit{x}}_7$	0.46
${\mathit{x}}_{1\mathit{n}}$ [mm]	–0.20	–0.20	0.01	–0.50	${\mathit{x}}_{5\mathit{n}}$	0.01
${\mathit{x}}_{5\mathit{n}}$ [ “ ]	–8.00	–7.39	0.29	–0.68	${\mathit{x}}_4$	0.35
${\mathit{x}}_4$ [ “ ]	–8.00	–8.06	0.07	–0.68	${\mathit{x}}_{5\mathit{n}}$	0.10

). Additionally, all observations were subject to synthetic random noise with magnitudes corresponding to the values adopted from [19]. The calibration parameters were estimated using the functional and stochastic model described in Section 2.1.2 and Section 2.1.3. Section 3.1.1 and Section 3.1.2 give the simulations results for the high-end panoramic TLSs, while Section 3.1.3 discusses adaptations necessary to fulfill the goal functions with the instruments not falling into this category. Separate solutions for these two TLS groups are necessary, as a lack of some measurement characteristics, such as two-face measurements, makes fulfilling the adopted design criteria more difficult (Section 2.1.1).

3.1.1. Simulated Design for High-End TLS

The final solution of our trial and error analysis is presented in Figure 2. Our educated guesses focused on combining the measurement configurations that are known to be sensitive towards estimating each individual CP [20]. We started the simulation trials with three targets and two-face scans from two scanner stations, which is a necessary minimum for the unambiguous estimation of the relevant CPs (Equations (4)–(6)) [20]. The number of targets was iteratively increased by 1 when aiming to improve the estimates of two-face sensitive CPs, or by 2 when aiming to improve the estimates of the remaining CPs. In the simulation process, we considered 85 different target locations and 4 different scanner stations in several hundred different set-ups, reaching the maximum network complexity when incorporating 24 targets. Finally, as the best solution, we derived the network of 14 targets measured from two scanner stations (S1 and S2) using two-face measurements (four scans), which successfully fulfilled all given optimization criteria. The coordinates of all targets and scanner stations are given in Appendix A.

The network is symmetric and it is realized by four repetitions of the minimal measurement geometry necessary for the comprehensive TLS calibration derived in [20]. In short, it is necessary to have a minimum of three targets measured from two stations using two-face measurements to estimate all relevant CPs (Figure 2, targets indicated with red frames). This measurement configuration is guaranteed to be highly sensitive, and therefore it is the backbone of the implemented calibration field. Four repetitions of this geometry were necessary to increase CP precision and reliability.

The instrument’s orientation around the standing axis needs to be differed by approximately 90° between the scanner stations, while the compensator observations need to be included in the calibration adjustment. Such a maneuver reduces the correlations and improves the standard deviation of some parameters [18]. Additionally, the targets between the scanner stations should allow observations from both sides. Further, we included 4 more targets perpendicular to the line connecting scanner stations S1 and S2 (Figure 2, targets indicated with blue frames). These targets have increased elevation angles at higher distances and they bring several advantages to our calibration field.

First, they form a highly sensitive measurement configuration for estimating the parameter x_5z, which is hard to estimate precisely due to the lack of two-face sensitivity [20]. Second, it improves the overall network geometry by increasing EOPs precision. Namely, a better volumetric distribution of targets increases the positioning accuracy similar to the Global Navigation Satellite System (GNSS), where geometric dilution of precision (GDOP) value improves with the larger volume of the pyramid closed, with four satellites and a receiver [54]. Improving the TLS positioning precision indirectly increases the CP quality (precision, correlations, and reliability). Third, it provides an additional set of targets at higher elevation angles, which boosts the accuracy of estimating the elevation-dependent CPs (majority), which is compromised due to the limited roof height in the hall [20]. Additionally, due to the longer ranges in comparison to the targets directly above stations S1 and S2 help to decouple the elevation-dependent tilt and offset parameters (e.g., x_1z and x_5z -x₇).

The results of the calibration simulations with the latter network geometry are given in Table 2. As can be seen, all of the CPs are estimated unbiasedly with the desired precision and reliability of 0.5″ for tilt (angular) CPs and 0.1 mm for offset (metric) CPs, while the correlations did not breach the threshold of 0.8. The CPs with the highest ${\sigma }_x$ and $\nabla x$ are the tilt parameters that rely on the targets with the higher elevation angles. These CPs are best estimated if the targets are placed directly above the instrument at distances of 10 m or more. However, the maximal distance in the given facility to such targets is approximately 6 m, making it suboptimal. This is an unavoidable phenomenon for the indoor calibration facilities due to the limited rooftop heights.

3.1.2. Comparison to Previous Calibration Fields

We further compared the derived calibration solution with the typical calibration field design to analyze the similarities and shortcomings of the cost-efficient calibration field concerning the usual implementations. Hence, we simulated a calibration experiment based on the typical calibration field derived by Chow et al. (2013) [3]. This calibration field consists of 120 targets measured with six scans taken from two scanner stations, where the instrument is rotated around the standing axis for approximately 120° after each scan. The calibration field is realized in a larger office room with dimensions of 14.5 × 11 ×3 m. We removed all observations with the incidence angles higher than 60° to avoid highly uncertain target center estimation, as it is a common practice in the literature [8]. The simulation was analogous to the previously explained one, and the results are presented in

Table 3. Simulation results for calibration of high-end TLSs with 120 targets (6 scans).

CP	${\mathit{x}}_{\mathit{t}\mathit{r}\mathit{u}\mathit{e}}$	$\widehat{\mathit{x}}$	${\mathit{\sigma }}_{\mathit{x}}$	${\mathit{\rho }}_{\mathit{x}(\mathit{m}\mathit{a}\mathit{x})}$	with	$\nabla \mathit{x}$
${\mathit{x}}_{10}$ [mm]	–2.00	–2.02	0.01	0.05	${\mathit{x}}_{1\mathit{n}}$	0.00
${\mathit{x}}_2$ [mm]	–0.20	–0.20	0.00	–0.28	${\mathit{x}}_{1\mathit{n}}$	0.00
${\mathit{x}}_3$ [mm]	–0.20	–0.20	0.00	–0.70	${\mathit{x}}_6$	0.00
${\mathit{x}}_6$ [ “ ]	–8.00	–7.97	0.03	–0.70	${\mathit{x}}_3$	0.03
${\mathit{x}}_{1\mathit{z}}$ [mm]	–0.20	–0.20	0.00	0.53	${\mathit{x}}_7$	0.01
${\mathit{x}}_7$ [ “ ]	8.00	8.07	0.46	0.72	${\mathit{x}}_{5\mathit{z}}$	0.29
${\mathit{x}}_{5\mathit{z}}$ [ “ ]	–8.00	–8.02	0.35	0.72	${\mathit{x}}_7$	0.09
${\mathit{x}}_{1\mathit{n}}$ [mm]	–0.20	–0.20	0.00	–0.62	${\mathit{x}}_{5\mathit{n}}$	0.00
${\mathit{x}}_{5\mathit{n}}$ [ “ ]	–8.00	–8.13	0.29	–0.62	${\mathit{x}}_{1\mathit{n}}$	0.27
${\mathit{x}}_4$ [ “ ]	–8.00	–7.92	0.06	–0.06	${\mathit{x}}_{5\mathit{z}}$	0.08

.

Comparing Table 2 and Table 3 reveals that having an eight times higher number of targets does not notably improve the calibration results. The highest correlations occur between the functionally similar CPs (e.g., x₃ vs. x₆ and x_5z vs. x₇, in the same range of approximately 70%). The precision of $x_7$ is practically identical, while the highest impact factors are only 0.1–0.2″ smaller in the case of 120 targets. To conclude, if sufficient attention is given to the measurement configuration, the number of targets can be remarkably reduced without notable losses in calibration parameter quality.

3.1.3. Simulation Results for Non-High-End TLS

Simulations for cases of instruments not falling into the category of high-end TLSs are realized by omitting compensator and two-face measurements and by increasing the measurement noise by a factor of 5. To still fulfill the desired optimization goals (now the goals are ${\sigma }_x \mathrm{and} \nabla x\le {2.5}^{″} \mathrm{or} 0.5 \mathrm{mm})$, we need to increase the number of scans from four to eight, while the measurement configuration is unchanged. The orientation of the instrument around the standing axis needs to be changed between each scan by approximately 90° on both scanner stations, and the reason for this is two-fold. First, this maneuver is known to reduce the parameter correlations [18]. Second, it mimics the two-face measurements of high-end TLSs when the instrument is coarsely rotated by 180° around its standing axis, which improves the CP quality (precision, correlations, and reliability). The simulation results for the latter conditions are presented in

Table 4. Simulation results of calibration of non-high-end TLSs with 14 targets (8 scans).

CP	${\mathit{x}}_{\mathit{t}\mathit{r}\mathit{u}\mathit{e}}$	$\widehat{\mathit{x}}$	${\mathit{\sigma }}_{\mathit{x}}$	${\mathit{\rho }}_{\mathit{x}(\mathit{m}\mathit{a}\mathit{x})}$	with	$\nabla \mathit{x}$
${\mathit{x}}_{10}$ [mm]	–2.00	–1.96	0.06	0.41	${\mathit{x}}_7$	0.06
${\mathit{x}}_2$ [mm]	–0.20	–0.22	0.03	–0.45	${\mathit{x}}_{5\mathit{n}}$	0.02
${\mathit{x}}_3$ [mm]	–0.20	–0.17	0.02	–0.61	${\mathit{x}}_6$	0.04
${\mathit{x}}_6$ [ “ ]	–8.00	–8.25	0.19	0.71	${\mathit{x}}_{1\mathit{z}}$	0.31
${\mathit{x}}_{1\mathit{z}}$ [mm]	–0.20	–0.19	0.03	0.71	${\mathit{x}}_6$	0.10
${\mathit{x}}_7$ [ “ ]	8.00	6.39	1.96	0.78	${\mathit{x}}_{5\mathit{z}}$	2.03
${\mathit{x}}_{5\mathit{z}}$ [ “ ]	–8.00	–10.70	1.69	0.78	${\mathit{x}}_7$	1.77
${\mathit{x}}_{1\mathit{n}}$ [mm]	–0.20	–0.16	0.03	–0.48	${\mathit{x}}_{5\mathit{n}}$	0.03
${\mathit{x}}_{5\mathit{n}}$ [ “ ]	–8.00	–9.04	1.18	–0.51	${\mathit{x}}_4$	1.06
${\mathit{x}}_4$ [ “ ]	–8.00	–7.41	0.33	–0.51	${\mathit{x}}_{5\mathit{n}}$	0.64

.

The realized CP quality is comparable to the high-end TLSs, with ${\sigma }_x$ and $\nabla x$ proportionally increased with the measurement noise, while ${\rho }_x$ remained similar. This solution is only one of a few possibilities that would lead to fulfilling the given optimization goals. We selected this one as the most feasible because it requires no extensive modifications in the measurement configuration. Although it may seem that the measurement time needs to be notably increased, with modern instruments this is only an increase of measurement time from approximately one to two hours.

3.2. Empirical Experiments—Evaluating the Calibration Field

3.2.1. Experiment setup

We calibrated a Leica ScanStation P20, Leica ScanStation P50, and Zoller and Fröhlich Imager 5016 (all high-end), as well as the Faro Focus3D X 130, Leica BLK360, and Leica HDS6100. All instruments were calibrated according to the measurement configurations presented in the latter section.

The simulated measurement configuration was implemented with centimeter-level accuracy for both targets and scanner station locations (deemed sufficient based on the simulations). The positioning was based on the known 3D model of the facility and the measurements with an electronic distance measurement device and a measuring tape. The implemented targets were planar black and white, 8-fold BOTA8 targets with dimensions of 30 × 30 cm, which guaranteed high precision target center estimation [55]. The targets had a semisphere mounting at the back side of the plate that was compatible with the laser tracker magnetic nests. Overall, 12 targets were installed on 12 magnetic nests in the facility.

The remaining two targets were placed on the wooden surveying tripods between the scanner stations. The latter targets were used to estimate the rangefinder offset (x₁₀), and therefore they needed to be observed from both sides. Hence, they had an additional printed target pattern on the back side and were mounted on typical surveying tribrachs. Additionally, they needed to allow an accurate transfer of the target center between the sides in the direction of range measurements (0.1 mm or better). Therefore, we used two-sided targets with a known thickness, which was measured with submillimeter-accurate calipers. To avoid an eventual bias of the CPs due to the unknown lateral shifts between the target patterns from both sides, we deweighted the angular measurements towards these targets in the calibration adjustment.

The instruments were placed on a heavy duty wooden tripod for TLSs and secured using the surveying tripod stabilization when possible. Focus^3D and BLK360 were not compatible, and hence they were placed on their original tripods without additional stabilization. During each measurement campaign, the atmospheric conditions were monitored. They all remained stable within this short measurement time. The measurement ranges spread from approximately 3 to 20 meters, and the vertical angles spread from 5° to 90° and from 270° to 355°. As all range measurements were relatively short, no atmospheric correction was applied. From each scanner station, two-face measurements were made and the compensator was turned on (when possible). Further details of each calibration are summarized in

Table 5. Details of each calibration experiment for six TLSs.

Instrument	P50	P20	Imager 5016	HDS6100	Focus3D	BLK360
Date	16 April 2019	8 August 2018	27 May 2019	7 August 2018	7 August 2018	7 August 2018
Duration	1 h	1 h	1 h	2 h	2 h	40 min
Scans #	4	4	4	4**	8	8
Observations #	168	168	168	168	336	336
Two-face m.	yes	yes	yes	yes	no	no
Compensator	yes	yes	yes	no	no	no
Resolution [mm @ 10 m]	1.6	1.6	1.6	1.6	1.6	5.0
${\mathit{\sigma }}_{\mathit{r}}$[mm]	1.2	1.0	1.0	2.0	2.0	4.0
${\mathit{\sigma }}_{\mathit{\phi } / \mathit{\theta }}$[ ″ ]	8.0	8.0	14.4	26.0	19.0	41.0
${\mathit{\sigma }}_{\mathit{c}\mathit{o}\mathit{m}\mathit{p}.}$[ ″ ]	1.5	1.5	14.4	-	-	-
Temperature*	17.9	28.0	16.8	31.2	33.1	33.1
Pressure*	996.2	991.5	990.1	989.0	988.3	988.3
Humidity*	42.5	54.0	75.0	31.5	27.5	27.5

Note: * Stable within: temp. +/− 0.5 °C, pressure +/− 1.5 hPa, humidity +/− 2.5%; ** 4 instead of 8 scans due to time limitations and availability of two-face measurements.

.

The target centers were estimated using the algorithm based on template matching described by Janßen et al. (2019) [55]. These observations were processed in the calibration adjustment as described in Section 2.1.2 and Section 2.1.3, with one modification. Namely, in all calibration cases, we observed an increased variance for the angular measurements of all highly elevated targets directly above scanner stations S1 and S2 (Figure 2). To account for this, we created respective separate observation groups, whose variances are refined separately through VCE, as described in Section 2.1.3. Therefore, the new covariance matrix adopted in the empirical experiment resembles the one given in Equation (11). However, it consists of five instead of three block matrices (one for ${\sigma }_r, {\sigma }_{\phi }, {\sigma }_{\phi }^{elevated},{\sigma }_{\theta }$, and ${\sigma }_{\theta }^{elevated}$ respectively). The number of outliers in the experiments was on average less than 1–2%, never breaching the value of 5% of observations.

3.2.2. Experiment Results for All TLSs

The estimated CPs together with the corresponding significances are presented in

Table 6. Estimated CPs for all 6 TLSs with the corresponding significance (sgf.).

CP	P50		P20		Imager		HDS6100		Focus3D		BLK360
	$\widehat{\mathit{x}}$	sgf.	$\widehat{\mathit{x}}$	sgf.	$\widehat{\mathit{x}}$	sgf.	$\widehat{\mathit{x}}$	sgf.	$\widehat{\mathit{x}}$	sgf.	$\widehat{\mathit{x}}$	sgf.
${\mathit{x}}_{10}$ [mm]	–0.03	N	0.58	Y	0.37	Y	0.11	N	0.23	Y	4.29	Y
${\mathit{x}}_2$ [mm]	–0.14	Y	–0.09	N	0.03	N	0.03	N	0.05	N	0.04	N
${\mathit{x}}_{1\mathit{z}}$ [mm]	0.14	N	–0.23	Y	0.59	Y	–0.01	N	0.87	N	1.37	Y
${\mathit{x}}_3$ [mm]	–0.03	N	0.16	Y	–0.15	N	–0.64	N	–0.28	N	–0.41	N
${\mathit{x}}_7$ [ “ ]	–33.50	Y	3.78	N	26.49	Y	–85.77	Y	122.14	Y	–16.97	N
${\mathit{x}}_6$ [ “ ]	3.41	Y	0.55	N	0.40	N	8.00	Y	10.28	Y	–19.98	Y
${\mathit{x}}_{1\mathit{n}}$ [mm]	0.01	N	0.08	N	–0.45	Y	–0.54	N	–0.17	N	0.05	N
${\mathit{x}}_4$ [ “ ]	4.51	Y	–6.70	Y	–6.44	Y	–8.13	Y	–57.31	Y	52.03	Y
${\mathit{x}}_{5\mathit{n}}$ [ “ ]	5.38	Y	–18.05	Y	12.74	N	57.07	Y	–2.51	N	–20.72	Y
${\mathit{x}}_{5\mathit{n}}$ [ “ ]	–16.78	Y	–2.53	N	2.66	N	–64.81	Y	36.08	Y	–3.49	N

. Since the CPs are correlated, they cannot be considered as independent values, and hence they need to be decorrelated before the statistical testing. Therefore, we applied the significance testing as described in [56], where the Cholesky decomposition is used to obtain uncorrelated parameter values. As can be appreciated in Table 6, all parameters are found to be significant in at least one of six cases. As expected, the tilt parameters are significant on a larger scale than the offset parameters. However, the x_1z is found to be significant in 3 out of 6 cases. Although not applying this CP would not notably reduce the point cloud quality, it would bias the estimate of functionally similar tilt parameters (x₇ and x_5z). This indicates that all parameters given in Equations (4)–(6) should be considered when designing a calibration field for a comprehensive calibration of TLSs.

The magnitudes of the estimated offset CPs are in the order of several tenths of a millimeter, while the spread is much higher in the case of the tilt CPs (from several arc seconds up to more than a hundred). It is observable that the values for high-end TLSs are generally a few times lower (up to an order of magnitude). These differences make evident the division on the high-end instruments and the ones not falling into that category.

Table 7. Observation residuals in the registration adjustment without applying CPs (no CPs), after applying CPs (CPs), and improvement in observation residuals (%).

	P50			P20			Imager 5016
	${\mathit{\sigma }}_{\mathit{r}} \left[\mathbf{mm}\right]$	${\mathit{\sigma }}_{\mathit{\phi }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{\theta } }$[ ″ ]	${\mathit{\sigma }}_{\mathit{r}} \left[\mathbf{mm}\right]$	${\mathit{\sigma }}_{\mathit{\phi }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{\theta }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{r}} \left[\mathbf{mm}\right]$	${\mathit{\sigma }}_{\mathit{\phi } }$[ ″ ]	${\mathit{\sigma }}_{\mathit{\theta } }$[ ″ ]
no CPs	0.15	93.73	18.36	0.28	26.14	8.79	0.12	32.07	4.82
CPs	0.06	24.72	1.86	0.25	13.39	4.47	0.06	17.73	3.40
%	56.76	73.62	89.87	10.87	48.79	49.22	48.98	44.70	29.50
	HDS6100			FOCUS 3D			BLK360
	${\mathit{\sigma }}_{\mathit{r}}$[mm]	${\mathit{\sigma }}_{\mathit{\phi }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{\theta }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{r}}$[mm]	${\mathit{\sigma }}_{\mathit{\phi }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{\theta }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{r} }$[mm]	${\mathit{\sigma }}_{\mathit{\phi }}$[ ″ ]	${\mathit{\sigma }}_{\mathit{\theta } }$[ ″ ]
no CPs	0.27	84.02	17.23	0.11	218.02	21.50	2.16	58.05	25.40
CPs	0.20	26.31	8.88	0.10	70.60	11.26	1.72	24.07	16.08
%	24.24	68.69	48.45	5.87	67.62	47.60	20.34	58.53	36.68

presents the standard deviation of the observation residuals in the registration adjustment with and without applying the estimated CPs, as well as the improvement achieved through the calibration. It is impossible to directly compare the success of our calibration field with the calibration fields used in previous calibration experiments due to different instruments, different distribution of observations, as well as different functional and stochastic models. However, the improvement in observation residuals is the most common evaluation of the calibration success in the literature [8,10,17,57]. Therefore, with a lack of better criterion, we rely on these values. Garcia and Lerma (2013) [17] analyzed and summarized the success of all TLS calibration experiments, presenting significant improvements of 36–48% for ranges, 30–80% for horizontal angles, and 31–74% for vertical angles. The values presented in Table 7 fit well in the latter intervals. The mean values are 28%, 60%, and 50% for ranges, horizontal, and vertical angles, respectively, where only range measurements fall marginally short in comparison to the previous experiments.

Furthermore, a number of observations in the previous target-based TLS calibration experiments fall within the range of 390–4680, with a mean of 2390. In this study, the number of observations is 168 or 336, depending on the scanner type (Table 5). This is a significant decrease of up to an order of magnitude. To conclude, the presented data demonstrate that we achieved comparable measurement improvements with a notably reduced number of observations. Hence, it can be concluded that the design of the cost-efficient calibration field based on the adopted optimization goals was successful, both for high-end and non-high-end TLSs.

3.2.3. Reproducibility of Results for the Leica ScanStation P50

We estimated the mean CP values and the empirical standard deviation based on the calibration of the Leica ScanStation P50, which was repeated four times (

Table 8. Leica ScanStation P50 calibration results for 4 repeated calibrations: estimated CPs for each realization (left), mean CP value and empirical standard deviation of all realizations (middle), mean CP standard deviations, and correlations and impact factors based on the covariance matrix after the adjustment (right).

CP	${\widehat{\mathit{x}}}_1$	${\widehat{\mathit{x}}}_2$	${\widehat{\mathit{x}}}_3$	${\widehat{\mathit{x}}}_4$	${\widehat{\mathit{x}}}_{\mathit{m}\mathit{e}\mathit{a}\mathit{n}}$	${\mathit{\sigma }}_{\mathit{E}(\mathit{x})}$	${\widehat{\mathit{\sigma }}}_{\mathit{x}}$	${\mathit{\rho }}_{\mathit{x}(\mathit{m}\mathit{a}\mathit{x})}$	$\nabla \mathit{x}$
${\mathit{x}}_{10}$[mm]	–0.16	0.03	0.01	–0.03	–0.04	0.09	0.06	0.37	0.10
${\mathit{x}}_2$[mm]	–0.12	–0.11	–0.12	–0.14	–0.12	0.01	0.02	–0.23	0.02
${\mathit{x}}_{1\mathit{z}}$[mm]	0.08	0.31	0.21	0.14	0.19	0.10	0.07	0.64	0.10
${\mathit{x}}_3$[mm]	–0.02	0	–0.06	–0.03	–0.03	0.03	0.05	–0.77	0.08
${\mathit{x}}_7$[ “ ]	–22.35	–25.38	–26.31	–33.5	–26.89	4.72	2.02	0.61	2.66
${\mathit{x}}_6$[ “ ]	4.19	3.59	4.11	3.41	3.83	0.38	0.31	–0.77	0.46
${\mathit{x}}_{1n}$[mm]	0.07	0.08	0.05	0.01	0.05	0.03	0.05	–0.64	0.07
${\mathit{x}}_4$[ “ ]	4.2	5.1	5.17	4.51	4.75	0.47	0.32	–0.70	0.64
${\mathit{x}}_{5n}$[ “ ]	4.5	4.67	5.62	5.38	5.04	0.54	1.48	–0.70	1.29
${\mathit{x}}_{5n}$[ “ ]	–12	–14.71	–14.33	–16.78	–14.46	1.96	1.07	0.18	0.86

). If we compare these empirical values with the ones derived from the calibration adjustment, we can observe the high level of congruence for most of the parameters (a few hundredths of a millimeter and a few tenths of an arc second). Additionally, we mostly achieved the goals set in the simulation experiments by estimating all offset parameters with a standard deviation of 0.1 mm and tilt parameters with a standard deviation of 0.5′.

The only parameters deviating from this are x₇ and x_5z, and the probable reason for that is two-fold. First, the estimation of these CPs relies on highly elevated targets directly above the instrument, for which we observed increased variance for angular measurements. Hence, to combat this increased variance and to achieve the initially planned CP accuracy in the future, we could increase the number of targets with high elevation angles. Second, if the calibration series for these two parameters are examined more carefully, we can notice an approximately linear trend of the CP values over time. Namely, both parameters decrease steadily with each calibration. This trend seems systematic and it cannot be adequately modeled with the variance propagation in the calibration adjustment. Therefore, in the future, we plan to repeat similar calibration series and further examine this phenomenon. Nevertheless, even with this discrepancy, the estimated CP standard deviations do not fall far away from the empirical values. Hence, they estimate the uncertainty of the estimated CPs well.

If we further analyze the impact factors, we can see that the estimated quantities are close to the values of standard deviations. Hence, we can conclude that our goal of keeping the maximal impact of undetected outliers in the same range as the measurement noise is achieved. Finally, the maximal correlations between the CPs remained under the threshold defined in Section 2.2.2. This conclusively confirms the successful implementation of the simulated calibration field design. A similar analysis for all instruments is omitted due to the limited instrument availability and time.

3.2.4. Point Cloud Quality Improvement

We additionally analyzed the degree to which applying the CPs improves the overall point cloud quality, both in situ and in later use. The CPs were applied twice. First, they were applied to the point clouds taken during the calibration to further investigate the effect of the up-to-date CPs. Second, the CPs were applied on the point clouds of unrelated measurement objects that were measured with different time offsets in comparison to the calibration.

To quantify the point cloud quality improvement, we analyzed the differences between the 1st and 2nd scans of the two-face measurements, before and after applying the CPs. The differences are computed using the M3C2 point cloud comparison algorithm [58]. These differences were computed in the direction of the local surface normals and were denoised based on the local point cloud statistics. As all external systematic errors act identically on both scans of two-face measurements, the only difference originates from the instrument’s mechanical misalignments and measurement noise. For example, the atmosphere-related systematic errors can be treated as constant for both consecutive scans, as there is no notable change in temperature or pressure in this short period of time. This leads to the cancelation of this systematic effect in the process of computing differences. Hence, for a perfectly calibrated instrument, the computed point cloud differences should have a central tendency of zero with low dispersion (lower than the measurement noise) [59]. Using this evaluation strategy limits us to the high-end TLSs allowing two-face measurements. Hence, we further analyze only these instruments, which are the main focus of this study anyway. It should be noted that this calibration evaluation strategy is not completely comprehensive (validates 9 out of 10 CPs). Namely, the influence of the estimated rangefinder offset (x₁₀) cannot be analyzed this way. Hence, for the validation of x₁₀, we rely on the improvement in observation residuals (Table 7).

Figure 3 presents the differences between the scans of two-face measurements of the machine hall (Figure 1) for the Leica ScanStation P50, Leica ScanStation P20, and Z+F Imager 5016 instruments before and after applying the CPs. The color scale is adjusted to highlight the differences before and after the calibration for each instrument individually. This is necessary due to uneven magnitudes of two-face differences. Hence, the results between the instruments are not directly comparable. In general, the green color represents the zero difference and it changes gradually until the defined threshold is breached (1 mm for P50 and Imager 5016, 2 mm for P20). The color depends on the direction of the differences (red for positive and blue for negative direction).

In the upper half of Figure 3, we can observe notable systematic trends in the computed differences with abrupt changes in signs between the parts measured in the front and in the back of the instrument. These differences originate from the two-face sensitive mechanical misalignments and indicate the degraded point cloud quality. In the lower half of the figure, we can observe that applying the estimated CPs notably reduced the differences and almost completely removed the systematic trends.

The corresponding histograms of the computed differences are given in Figure 4. It can be seen that applying the CPs notably improved the distributions of differences for all scanners (central tendency closer to zero and lower dispersion). Hence, our calibration successfully reduced the evident influence of systematic errors. The calculated point cloud differences are not necessarily normally distributed due to possible outliers, remaining systematic influences, and unknown influence of the M3C2 algorithm on the observations. Hence, following a few recent studies [60,61], besides the commonly used values of central tendency and dispersion (mean and standard deviation), we also calculated the robust measures of distribution—median and median absolute deviation (MAD). The standard deviation of the differences was reduced by the calibration process from 15% to 40%, while the MAD was reduced even more, from 43% to 74%. Additionally, the mean and median values were notably closer to zero for P50 (mean ranged from 0.22 to 0.06mm, median from 0.34 to 0.06mm) and Imager 5016 (mean ranged from 0.23 to –0.02, median from 0.22 to –0.01), further indicating the elimination of the systematic trends in measurements (for P20 values changed only marginally).

3.2.5. A Posteriori Use of Calibration Parameters

To test the a posteriori point cloud quality improvement, we measured a planar wall from the outer side of the machine hall with the P20 and P50 instruments, while we measured a building façade in the proximity of our institute with the Imager 5016 instrument. All measurements were several days apart from the calibration experiment, and in contrast to the point clouds presented in Figure 3, they were acquired outdoors. The measurements with the P20 instrument took place two days after the calibration, with the P50 instrument two weeks before the calibration, and with the Imager 5016 instrument approximately two months after the calibration. The point cloud differences and corresponding histograms are presented in Figure 5 and Figure 6, respectively. Again, the color scales were adjusted for each instrument individually (with the solid color thresholds at 5 mm for P50 and 1.5 mm for P20 and Imager 5016).

A few important observations should be pointed out. The first one is that applying the estimated CPs again succeeded in improving the point cloud quality (lower dispersion of the differences and central tendency closer to zero), even in the case of recently factory-calibrated instruments (the Leica ScanStation P50 and ZF Imager 5016 were calibrated a few months prior to the experiments). This indicates that the estimated CPs can be used a posteriori on unrelated objects to increase the measurement accuracy. To the authors’ knowledge, this is the first time in the literature that such results are presented for user calibration of TLSs.

The second observation is that the point cloud improvement is notably lower in comparison to the in situ case. For example, MAD is now reduced from a minimum of 28% to a maximum of 53%, which is evidently lower. The strong remaining systematic trend can still be observed in the case of Leica ScanStation P50 measurements, causing a bimodal distribution of the point cloud differences, even after the calibration. This systematic trend is likely caused by the insufficiently compensated influence of the two-face sensitive CPs, primarily the mirror tilt CP ($x_6$). The reason for this presumption is an abrupt change in the sign of the visible systematic trend in the point cloud differences (Figure 5, top), which appears on the location where the parts of the wall measured in the front and in the back of the instrument meet. This deterioration in the point cloud quality improvement is not completely surprising, as the measurements took place with different time offsets with respect to the calibration, and the temporal stability of the CPs was already questioned in the literature [8,17,57].

Finally, we can deduce that the estimated CPs can notably improve the point cloud quality, both in indoor and outdoor applications, where the improvement was higher in the case of the indoor scenes. However, this is likely caused by the temporal instability of the CPs rather than the different scenes. Namely, the derived CPs are based on the mechanical working principle of TLSs, and as such they are universally applicable (Section 2.1.1).

Therefore, it can be concluded that the implemented calibration field can deliver the CPs that improve the measurement quality, even in the a posteriori application. However, it is necessary to further investigate the stability of the CPs to use them in high-accuracy-demanding tasks, such as deformation monitoring. First, it is necessary to correctly quantify the uncertainty based on their temporal stability. Second, the physical causes for their instability should be investigated to mitigate these effects through the application of the eventual correction models or appropriate measurement strategies.

4. Conclusions

Within this study, we designed a cost-efficient TLS calibration field for the comprehensive target-based self-calibration of panoramic TLSs. The design was based on the discussed optimization goals and simulation experiments, and it was implemented in reality to empirically validate its feasibility. We successfully calibrated a wide range of TLSs, reaching in situ improvement in observation residuals comparable to previous studies (on average 28%, 60%, and 50% for ranges, horizontal, and vertical angles), with a notably reduced number of observations using only 14 targets. These results demonstrate that implementation of our workflow and the optimization goals can be used to derive further cost-efficient calibration fields.

In the calibration experiments, all of the adopted calibration parameters were found to be significant (at least once), stressing the importance of a comprehensive TLS calibration. Additionally, we demonstrated that the a posteriori estimated standard deviations represent the precision of the calibration parameters well in reality. We notably improved the point cloud quality of manufacturer-calibrated high-end TLSs, even with brand new instruments (Leica P50, Imager 5016). Thus, we can confirm that the implemented calibration field can indeed be used to obtain the reference solutions for further development of in situ calibration approaches.

Finally, we demonstrated a point cloud quality improvement based on a posteriori application of estimated calibration parameters on the measurements of an unrelated object. Our results suggest that in the future, it is necessary to further investigate the temporal stability of the estimated calibration parameters. Eventually, time-dependent correction models could account for this parameter variability. Alternatively, the combination of well-designed calibration fields, as introduced in this study, and in situ calibration strategies accounting for the temporal variability of some calibration parameters might lead to the most accurate laser scanning product.