Incremental Multi-Camera Extrinsic Calibration Method Based on PnP Integrating Weighted AprilTag Detections and Multi-View Triangulation

Abstract

Accurate extrinsic calibration of multi-camera systems is a central problem in three-dimensional computer vision, as errors in the relative positioning of sensors directly propagate into geometric distortions that critically degrade the quality of downstream applications. This paper proposes an incremental extrinsic camera parameter initialization method that improves upon the baseline iterative registration algorithm based on the Perspective-n-Point (PnP) problem. Unlike board-based calibration frameworks, the proposed approach operates on individually placed markers with no prior knowledge of their mutual positions, enabling recalibration without dedicated calibration sessions. The accuracy improvement is achieved through the introduction of heuristic weighting of fiducial marker detections using AprilTags, as well as the application of a multi-view triangulation algorithm for dynamic refinement of marker spatial coordinates at each stage of scene expansion. Theoretical analysis demonstrates that the incorporation of these mechanisms does not increase the overall asymptotic computational complexity of the complete calibration cycle (including the global optimization stage), despite the higher computational cost of the initialization stage itself. Empirical validation of the method is performed on both synthetic datasets with known ground-truth camera parameters and real-world capture data through the evaluation of geometric errors and their comparison with the baseline method. Experimental results, supplemented by an ablation study, indicate that the proposed algorithm achieves statistically significant improvements on synthetic data in more than 80% of cases, while on real data it is on average 85% more accurate in terms of reprojection error.

1. Introduction

Three-dimensional computer vision has become an integral component of a wide range of modern technologies, finding applications in areas such as autonomous driving, robotics, augmented and virtual reality, as well as industrial quality control [1,2,3]. The foundation for most three-dimensional computer vision systems consists of multi-camera setups [4], which enable richer spatial coverage of the surrounding environment compared to single cameras. Nearly all higher organisms, including humans, rely on binocular vision, as evolution has established this mechanism as the most effective means of perceiving three-dimensional environments.

The key factor determining the accuracy and reliability of any multi-camera system is the quality of its calibration. The calibration process involves estimating both the intrinsic parameters of each camera (intrinsics) and their relative spatial position and orientation (extrinsics) [4,5]. Intrinsics characterize the properties of the optical system and sensor, remaining invariant to camera placement in space. This enables their offline estimation during a preliminary setup stage and allows them to be treated as constant throughout a series of experiments. Although physical factors such as temperature fluctuations may introduce microscopic variations in the optical system, in practice these deviations are either negligibly small or successfully compensated by refinement methods applied during the computation pipeline of specific tasks. Extrinsic parameters, however, require strict recalibration upon any, even minor, change in the multi-camera rig configuration, as errors in the relative positioning of sensors directly propagate into geometric distortions and critically degrade the accuracy of the resulting 3D reconstruction or any other computer vision task.

This paper proposes an incremental initialization method for extrinsic camera calibration in an object-centric rig that utilizes AprilTag marker detection [6] in images, weighted PnP (Perspective-n-Point) solution techniques for pose estimation of objects with known geometry [5], and a multi-view triangulation algorithm for refining marker positions in space. Unlike board-based calibration frameworks such as Kalibr [7] or MC-Calib [8], which require a structured calibration object with known internal geometry, the proposed method places no constraints on the relative positions of the markers, recovering their 3D coordinates simultaneously with camera poses. This distinction has a direct practical consequence: board-based frameworks require a dedicated calibration session each time the camera configuration changes, with the calibration object actively repositioned through the shared field of view of all cameras. The proposed approach eliminates this overhead—once placed in the scene, the markers can be reused for recalibration at any time without additional capture sessions. The proposed method is compared against a baseline method that implements classical iterative calibration and relies solely on the PnP problem for estimating camera and marker poses. This comparison is conducted on both synthetic datasets and real-world capture data. To isolate the specific contribution of weighting and triangulation that distinguish the proposed method from the baseline, an ablation study is performed.

The principal contributions of this paper are as follows:

• An incremental initialization method for multi-camera extrinsic calibration is proposed, operating on arbitrarily placed AprilTag markers with unknown mutual positions and requiring no structured calibration object.
• A heuristic detection-quality weighting scheme for the PnP problem is introduced, with physical justification for each factor and empirical evidence of robustness to hyperparameter choice.
• A multi-view triangulation step is integrated directly into the incremental registration loop, enabling dynamic correction of marker positions and mitigation of geometric drift at each camera addition step.
• It is shown theoretically that the proposed modifications do not increase the asymptotic complexity of the complete calibration procedure, including the bundle adjustment stage.

The remainder of the paper is organized as follows. Section 2 reviews related work on target-based calibration, SfM-inspired incremental methods, and weighted pose estimation. Section 3 describes the baseline incremental calibration method. Section 4 covers the global optimization (bundle adjustment) stage that completes the calibration procedure. Section 5 introduces the proposed method, which extends the baseline with the two key modifications. Section 6 reports experimental results on synthetic and real-world data, and Section 7 presents an ablation study isolating the contribution of each component. Section 8 concludes the paper.

2. Related Work

Existing methods for extrinsic calibration of multi-camera systems fall into two main categories: methods that employ calibration targets with a priori known geometry [9], and targetless calibration methods that estimate camera parameters through analysis of natural features in the observed scene [10].

2.1. Target-Based Calibration Methods

One of the most widely adopted calibration approaches involves the use of planar regular patterns. In particular, Zhang’s method [11] reduces the calibration problem to establishing correspondences between known 3D coordinates of control points on the pattern (e.g., intersections of chessboard squares) and their 2D projections in images. Based on the established correspondences, extrinsic camera parameters are computed by solving the PnP problem. However, the application of regular patterns in multi-camera systems (especially in object-centric configurations) is hindered by the requirement to ensure simultaneous visibility of the calibration object across multiple cameras.

To address this problem, fiducial markers such as AprilTags [6,12] are actively employed. Each such marker constitutes a distinctive pattern on a square grid, in which a unique identifier is encoded using a binary code. The identifiers themselves are selected to maximize pairwise Hamming distances between codes of different markers, thereby ensuring reliable recognition and minimizing false positives. The works in [8,13,14] demonstrate methods for calibrating multi-camera systems using patterns composed of AprilTag markers arranged in a planar grid. Optimizing camera parameters using markers with known relative positions reduces orientation estimation errors compared to the use of single targets. Popular frameworks such as MC-Calib [8] and Kalibr [7] also employ calibration objects with marker grids for reliable initialization of extrinsics prior to launching global optimization. A common characteristic of all these approaches is the requirement for a structured calibration object whose internal geometry—the relative positions of its control points—is precisely known in advance. Consequently, frameworks such as Kalibr and MC-Calib are not applicable to scenarios in which markers are placed individually at arbitrary, a priori unknown positions—as is the case in object-centric rigs where fiducial markers are permanently affixed to stands in the scene and reused across recalibration sessions without requiring a calibration target to be repositioned in front of the cameras.

Recent studies continue to advance target-based multi-camera calibration for diverse setups, including large outdoor scenes, generic marker-based toolboxes, omnidirectional imaging systems, large capture volumes with bundle adjustment, RGB-D camera networks, and robust meta-board-based calibration frameworks [8,14,15,16,17,18].

2.2. Targetless Calibration and Incremental Registration

Alternative calibration methods draw upon algorithms from the Structure from Motion (SfM) domain [1,19,20] and eliminate the need for specialized calibration targets, operating solely on automatically extracted image features. In frameworks such as COLMAP [19], calibration is accomplished through extraction and matching of scene keypoints with subsequent iterative addition of new cameras to the visibility graph. The computation of parameters for each new camera is performed by solving the PnP problem with respect to already triangulated 3D scene points [19,20,21].

The fundamental problem of incremental methods is geometric drift. The estimation error of extrinsic parameters for the first camera pair propagates to all subsequent iterations, accumulating with each step. In closed-loop camera configurations, this leads to significant geometric misalignments upon loop closure. To mitigate these errors, global optimization is applied, which in the context of camera calibration is conventionally referred to as Bundle Adjustment (BA). However, such optimization is highly sensitive to the quality of the initial estimate [22,23,24]. Substantial drift accumulated during the initialization stage may cause the optimizer to converge to an incorrect local minimum or require a prohibitively large number of iterations [25].

2.3. Weighted Pose Estimation and Multi-View Triangulation

Classical algorithms for solving the PnP problem assume homoscedasticity of observations, that is, equal noise variance for all employed 2D points. In practice, the localization accuracy of marker corners or keypoints is non-uniform and depends on lens distortion, distance to the object, and viewing angle. To account for these factors, weighted optimization methods are employed. In [26], the minimized error function is weighted based on computed covariance matrices of 2D points, which improves localization accuracy. Similarly, the EPro-PnP method [27] utilizes probability distributions for pose prediction, where weight coefficients are computed by a neural network. However, rigorous computation of covariance matrices for detection errors or the use of learned predictive models substantially increases the computational cost of pose estimation.

To reduce geometric drift during incremental scene reconstruction, multi-view triangulation algorithms are also investigated. As demonstrated in [28], the accuracy of 3D point reconstruction increases with the addition of observations from new viewpoints. Integration of coordinate recalculation directly into the camera addition cycle enables dynamic correction of spatial structure. However, in many practical incremental pipelines, marker or point positions are initialized locally and are only refined later during bundle adjustment, if at all [19,20,21].

The method proposed in this paper belongs to the class of incremental PnP-based registration methods described in Section 2.2, adapted specifically to the object-centric rig calibration setting in which fiducial markers are permanently placed in the scene. Unlike the board-based frameworks discussed in Section 2.1—including Kalibr [7], MC-Calib [8], Meta-Calib [14], and CamOdoCal [29]—it imposes no prior constraints on marker placement and requires no structured calibration object. Unlike general-purpose SfM pipelines, its scope is confined to the initialization of extrinsic parameters of a known multi-camera rig, providing a high-quality starting point for subsequent bundle adjustment rather than attempting full scene reconstruction. Within this class, the standard PnP-based incremental registration algorithm—which underlies the initialization stages of both general SfM systems and board-based calibration frameworks—is the natural baseline for evaluating the proposed improvements.

3. Baseline Calibration Method

Since the problem of joint estimation of camera parameters and scene structure (Bundle Adjustment, BA) reduces to the minimization of a non-convex cost function in a high-dimensional space [23,24,25], applying classical optimization methods without a high-quality initial estimate leads to convergence to a local minimum with high probability [22,23,30]. In this regard, the standard approach to parameter initialization is the incremental registration strategy [20].

Incremental registration consists of sequentially solving a series of local geometric problems, where parameters of each new camera are estimated relative to the already reconstructed part of the scene. The key tools for such localization are algorithms for solving the PnP problem. Formally, the PnP problem involves finding the position and orientation of a camera $(\mathbf{R},\mathbf{t})$, where $\mathbf{R}\in {\mathbb{R}}^{3\times 3}$ is the rotation matrix and $\mathbf{t}\in {\mathbb{R}}^3$ is the translation vector, relative to a given coordinate system by minimizing the reprojection error between $A$ known 3D coordinates of scene points and their corresponding 2D projections on the image plane. In general form, the problem is formulated as a nonlinear optimization:

$$\left({\mathbf{R}}^{\mathit{*}},{\mathbf{t}}^{\mathit{*}}\right)=\underset{\left(\mathbf{R},\mathbf{t}\right)}{\mathrm{arg\; min}}{\displaystyle \sum _{i=1}^A}{‖{\mathbf{u}}_i-\pi \left(\mathbf{K},\mathbf{R},\mathbf{t},{\mathbf{x}}_i\right)‖}^2,$$

(1)

where ${\mathbf{u}}_i\in {\mathbb{R}}^2$ are the observed 2D coordinates of the $i$-th point on the image plane; ${\mathbf{x}}_i\in {\mathbb{R}}^3$ are the 3D coordinates of the $i$-th point in the world coordinate system; $\mathbf{K}\in {\mathbb{R}}^{3\times 3}$ is the camera intrinsic matrix; $\pi :{\mathbb{R}}^3\to {\mathbb{R}}^2$ is the function projecting a 3D point from the world coordinate system to the camera image plane; $(\mathbf{R},\mathbf{t})$ are the sought rotation matrix $\mathbf{R}\in {\mathbb{R}}^{3\times 3}$ and translation vector $\mathbf{t}\in {\mathbb{R}}^3$ required to transform a point from the world coordinate system to the camera coordinate system; $‖\cdot ‖$ denotes the Euclidean norm; $A$ is the total number of points; the expression under the summation represents the squared reprojection error for the $i$-th point.

Finding the transformation $(\mathbf{R},\mathbf{t})$ for a camera that minimizes the reprojection error is algebraically equivalent to finding the rotation and translation of a marker relative to a fixed camera, since any rigid transformation is invertible. Thus, solving the PnP problem enables both estimating camera poses relative to known 3D points and estimating point positions relative to known cameras.

In this work, the baseline calibration method under consideration is an iterative algorithm implementing the principle of sequential scene expansion. The calibration process begins with fixing the coordinate system of the first camera, after which new sensors and markers are added in a cyclic manner. The position and orientation of each subsequent camera are computed via PnP using already known 3D marker points, while the poses of newly detected markers are, in turn, estimated via PnP from cameras already placed in space. The baseline calibration method is presented in more detail in Algorithm 1.

Algorithm 1: Baseline extrinsic calibration method
Input:	$N$—number of cameras; $M$—number of markers; $\left\{{\mathcal{V}}_i | i\in \left[1,N\right]\right\}$—sets of markers indices visible from the i-th camera; $\left\{{\mathbf{u}}_{ij} | i\in \left[1,N\right],j\in {\mathcal{V}}_i\right\}$—detections of the j-th marker on the i-th camera image plane.
Output:	$\left\{{\mathbf{R}}_{c_i} | i\in \left[1,N\right]\right\}$—rotations of all cameras; $\left\{{\mathbf{t}}_{c_i} | i\in \left[1,N\right]\right\}$—translations of all cameras; $\left\{{\mathbf{R}}_{m_j} | j\in \left[1,M\right]\right\}$—rotations of all markers; $\left\{{\mathbf{t}}_{m_j} | j\in \left[1,M\right]\right\}$—translations of all markers.
1.	$\mathcal{U}⟵\left\{1,\dots ,N\right\}$	$⊳$ Set of uninitialized cameras indices
2.	$\mathcal{M}⟵\mathit{\varnothing }$	$⊳$ Set of positioned markers indices
3.	while $\mathcal{U}\ne \mathcal{\varnothing }$ do
4.	if $\left|\mathcal{U}\right|=N$ then
5.	$i⟵\underset{i\in \mathcal{U}}{\mathrm{arg\; max}}\left|{\mathcal{V}}_i\right|$	$⊳$ Pick camera with strict max observations
6.	${\mathbf{R}}_{c_i}⟵\mathbf{I}$	$⊳$ Initialize camera rotation as identity matrix
7.	${\mathbf{t}}_{c_i}⟵{\left(0,0,0\right)}^{\mathrm{T}}$	$⊳$ Initialize camera translation by zeros
8.	else
9.	$i⟵\underset{i}{\mathrm{arg\; max}}\left|{\mathcal{V}}_i\cap \mathcal{M}\right|$	$⊳$ Pick camera with max overlap with $\mathcal{M}$
10.	$\mathcal{R}⟵\left\{{\mathbf{R}}_{m_j} | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of visible markers rotations
11.	$\mathcal{T}⟵\left\{{\mathbf{t}}_{m_j} | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of visible markers translations
12.	$\mathcal{D}⟵\left\{{\mathbf{u}}_{ij} | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of visible markers detections
13.	${\mathbf{R}}_{c_i},{\mathbf{t}}_{c_i}⟵\mathrm{S}\mathrm{o}\mathrm{l}\mathrm{v}\mathrm{e}\mathrm{P}\mathrm{n}\mathrm{P}\left(\mathcal{R},\mathcal{T},\mathcal{D}\right)$	$⊳$ Estimate camera pose by solving PnP
14.	end if
15.	$\mathcal{U}⟵\mathcal{U}\setminus i$	$⊳$ Mark camera as initialized
16.	for each marker $j\in \left({\mathcal{V}}_i\setminus \mathcal{M}\right)$ do
17.	${\mathbf{R}}_{m_j},{\mathbf{t}}_{m_j}⟵\mathrm{S}\mathrm{o}\mathrm{l}\mathrm{v}\mathrm{e}\mathrm{P}\mathrm{n}\mathrm{P}\left({\mathbf{R}}_{c_i},{\mathbf{t}}_{c_i},{\mathbf{u}}_{ij}\right)$	$⊳$ Estimate marker pose by solving PnP
18.	$\mathcal{M}⟵\mathcal{M}\cup j$	$⊳$ Mark marker as initialized
19.	end for
20.	end while

However, the baseline method suffers from a substantial limitation–the problem of error accumulation. Since the estimation of position and orientation for each subsequent camera depends on the accuracy of marker position estimates at previous stages, errors are not compensated but rather accumulate along the chain of cameras. In closed-loop configurations, particularly in the object-centric systems considered in this work, this often leads to a significant discrepancy between the computed position of the last calibrated camera and its true position relative to the first.

Accumulated geometric inconsistencies can potentially be eliminated during global optimization, but BA is extremely sensitive to the quality of the initial estimate. An excessively coarse estimate of extrinsics from the baseline method may cause the optimization to converge to an incorrect local minimum. Furthermore, the computational complexity of BA grows nonlinearly with increasing numbers of cameras and markers, which becomes particularly apparent when the initial calibration is insufficiently accurate, forcing the optimization algorithm to perform a larger number of iterations to reach the optimum.

4. Global Optimization

Typically, regardless of the accuracy of the initial parameter estimate for the scene, the calibration process for multi-camera systems cannot avoid the stage of global optimization (BA). This is because sequential initialization algorithms, including the baseline method described earlier, minimize errors locally and do not account for global constraints in the visibility graph. Joint optimization of all scene parameters enables finding a solution corresponding to the maximum likelihood criterion, ensuring a statistically optimal result in the presence of Gaussian measurement noise [23].

Since both cameras and markers possess rotation and translation, the values of which are adjusted during global optimization, separate notation is required. Let ${\mathbf{R}}_{c_i}\in {\mathbb{R}}^{3\times 3}$, ${\mathbf{t}}_{c_i}\in {\mathbb{R}}^3$ denote the rotation and translation for the $i$-th camera, and ${\mathbf{R}}_{m_j}\in {\mathbb{R}}^{3\times 3}$, ${\mathbf{t}}_{m_j}\in {\mathbb{R}}^3$ denote the rotation and translation for the $j$-th marker. The optimized parameters for all cameras and all markers are packed into a single parameter vector $\mathsf{\theta }\in {\mathbb{R}}^S$ for launching BA, where $S=P\left(N-1+M\right)$ is the total number of parameters, which depends on the number of cameras $N$, the number of markers $M$, and the number of parameters $P$ for encoding rotation and translation. One camera is excluded from the vector $\mathsf{\theta }$ so that its position and orientation define the coordinate system and eliminate solution ambiguity. For representing rotations, the axis-angle vector parameterization is employed. This representation is the most compact parameterization of rotation, avoiding parameter redundancy inherent in rotation matrices or quaternions. Thus, 3 parameters are allocated for rotation and 3 parameters for translation for each camera and marker, yielding $P = 6$. The vector $\mathsf{\theta }$ is formed by concatenating the parameters of all cameras and markers: $\mathsf{\theta }={\left[{\widehat{\mathbf{R}}}_{c_1}^{\mathrm{T}},{\mathbf{t}}_{c_1}^{\mathrm{T}},\dots ,{\widehat{\mathbf{R}}}_{c_{N-1}}^{\mathrm{T}},{\mathbf{t}}_{c_{N-1}}^{\mathrm{T}},{\widehat{\mathbf{R}}}_{m_1}^{\mathrm{T}},{\mathbf{t}}_{m_1}^{\mathrm{T}},\dots ,{\widehat{\mathbf{R}}}_{m_M}^{\mathrm{T}},{\mathbf{t}}_{m_M}^{\mathrm{T}}\right]}^{\mathrm{T}}$, where the hat symbol $\widehat{\mathbf{R}}\in {\mathbb{R}}^3$ denotes the axis-angle representation of rotation.

The global optimization problem is formulated as follows:

$${\mathsf{\theta }}^{*}=\underset{\mathsf{\theta }}{\mathrm{arg\; min}}{\displaystyle \sum _{i=1}^N}\sum _{j\in {\mathcal{V}}_i}{‖{\mathbf{U}}_{ij}-\pi \left({\mathbf{K}}_i,{\mathbf{R}}_{c_i},{\mathbf{t}}_{c_i},{\mathbf{X}}_j\right)‖}^2,$$

(2)

where $\mathsf{\theta }\in {\mathbb{R}}^S$ is the vector of optimized parameters encoding rotations and translations for all cameras (${\mathbf{R}}_{c_i}\in {\mathbb{R}}^{3\times 3}$ and ${\mathbf{t}}_{c_i}\in {\mathbb{R}}^3$ for each $i$-th camera, respectively) and markers (coordinates ${\mathbf{X}}_j\in {\mathbb{R}}^{4\times 3}$ for the four corners of the $j$-th marker, obtained by applying rotation and translation to the template 3D representation of the marker ${\mathbf{X}}_{template}\in {\mathbb{R}}^{3\times 4}$); ${\mathbf{U}}_{ij}\in {\mathbb{R}}^{4\times 2}$ are the 2D coordinates of the detection of the four corners of the $j$-th marker on the image plane of the $i$-th camera; $\pi :{\mathbb{R}}^{4\times 3}\to {\mathbb{R}}^{4\times 2}$ is the function projecting the four corners of a marker from the 3D world coordinate system to the camera image plane; ${\mathcal{V}}_i$ is the set of indices of markers for which detections exist in the $i$-th camera; $‖\cdot ‖$ denotes the Euclidean norm computed along the second dimension, that is, implementing the mapping ${\mathbb{R}}^{4\times 2}\to {\mathbb{R}}^4$.

The expression under the summation in Equation (2) decomposes into 8 squared scalar quantities, since each detection of the $j$-th marker from the $i$-th camera involves 4 points (corners), indexed by variable $k\in \left[1,4\right]$, each having 2 coordinates indexed by variable $c\in \left\{x,y\right\}$:

$${‖{\mathbf{U}}_{ij}-\pi \left({\mathbf{K}}_i,{\mathbf{R}}_{c_i},{\mathbf{t}}_{c_i},{\mathbf{X}}_j\right)‖}^2={\displaystyle \sum _{k=1}^4}\sum _{\mathrm{c}\in \left\{x,y\right\}}{‖r_{ijkc}‖}^2,$$

(3)

Thus, the BA problem reduces to a nonlinear least squares problem, in which the sum of squared residuals ${‖r_{ijkc}‖}^2$ must be minimized, where $i\in \left[1,N\right]$ is the camera index, $j\in \left[1,M\right]$ is the marker index, $k\in \left[1,4\right]$ is the marker corner index, and $c\in \left\{x,y\right\}$ is one of the two dimensions on the camera image plane. The complete residual vector $\mathbf{r}$ has length $D={\displaystyle {\sum }_{i=1}^N}8\left|{\mathcal{V}}_i\right|$, encompassing all marker detections. Optionally, a robust loss function may be applied to the squared residuals to mitigate the influence of outliers, for example, the Huber loss [31].

Second-order iterative methods are traditionally employed to solve the BA problem, such as the Levenberg–Marquardt (LM) algorithm [32,33] or the more commonly used Trust Region Reflective (TRF) algorithm [34]. This choice is motivated by the specific structure of the objective function landscape in BA problems, which often exhibits an ill-conditioned Hessian matrix. Under such conditions, second-order methods that exploit curvature information can achieve quadratic convergence rates when provided with a sufficiently accurate initial estimate.

The computational complexity of a single iteration of such algorithms is determined by the cost of solving the normal equations of the form [23]:

$$\mathbf{H}∆\mathsf{\theta }=-{\mathbf{J}}^{\mathrm{T}}\mathbf{r},$$

(4)

where $\mathbf{H}\in {\mathbb{R}}^{S\times S}$ is the Hessian matrix of the sum of squared residuals objective function or its approximation (depending on the specific second-order optimization method); $\mathbf{J}\in {\mathbb{R}}^{D\times S}$ is the Jacobian matrix of the sum of squared residuals objective function; $∆\mathsf{\theta }$ is the sought parameter update for the current iteration.

The complexity of solving such a system comprises the cost of constructing and inverting the Hessian matrix $\mathbf{H}$. Inverting a matrix of size $S\times S$ in the general case amounts to $O\left(S^3\right)=O\left({\left(P\left(M+N\right)\right)}^3\right)$, or simply $O\left({\left(M+N\right)}^3\right)$, since $P$ is a constant. However, it can be observed that the rotation and translation parameters for each marker depend only on those cameras that observe it and are independent of one another. The same holds for cameras–no residual computation involves parameters from multiple cameras simultaneously. Therefore, the matrix $\mathbf{H}$ exhibits a specific sparsity pattern and has a block-diagonal structure. This property enables the application of the Schur complement method to the inversion of $\mathbf{H}$, which eliminates the marker parameters and reduces the problem to solving a system for cameras only [35], lowering the complexity to $O\left(M{N}^2\right)$ for constructing the complement and $O\left(N^3\right)$ for the final computation of ${\mathbf{H}}^{-1}$. Leveraging the same sparsity property, it can be established that constructing $\mathbf{H}$ costs $O\left(MN\right)$ in the worst case, when each camera observes every marker, i.e., $D=8MN$. Typically, the number of markers in the scene far exceeds the number of cameras, i.e., $M\gg N$, so $M{N}^2$ begins to dominate over $N^3$ and $MN$, and the per-iteration complexity ultimately reduces to $O\left(M{N}^2\right)$. The total runtime of BA is linear in the number of iterations $K$, which can reach hundreds for a poor initial estimate.

Thus, the asymptotic complexity of BA is linear in the number of markers and quadratic in the number of cameras, while the quality of initialization directly affects the number of iterations required to reach the optimum.

5. Proposed Calibration Method

The principal limitation of the baseline sequential initialization method described above is the problem of error accumulation, which leads to substantial geometric discrepancies, particularly in closed-loop configurations of object-centric systems. To address this limitation, we propose a modified method that incorporates mechanisms for improved robustness and drift correction directly into the incremental scene reconstruction process.

The proposed modification of the baseline incremental calibration method incorporates two key modifications aimed at improving accuracy and geometric consistency. First, a weighting scheme for observations is introduced in the PnP problem, where the contribution of each detection is determined by a heuristic reliability estimate. Second, the process is augmented with a multi-view triangulation stage, which is employed at each step of scene expansion to recompute the coordinates of all markers, thereby maintaining dynamic geometric consistency and preventing error accumulation.

5.1. Weighted PnP

Standard methods for solving the PnP problem, such as Efficient PnP (EPnP) [36] or iterative LM, minimize the sum of squared reprojection errors, assuming identical noise variance for all points. However, in real-world conditions, the quality of AprilTag marker detection varies significantly depending on observational conditions.

To account for this, a weighting function is introduced that assigns a confidence coefficient to each marker observation. Weights $w$ are incorporated into the summation of the PnP problem formulation (1), computed for each marker as follows:

$$w=\left({\displaystyle \frac{w_{len}}{{\gamma }_{len}}}\right)\cdot \left({1-\left(1-w_{sin}\right)}^{{\gamma }_{sin}}\right)·{\left(1-{\displaystyle \frac{w_{dist}}{{\gamma }_{diag}}}\right)}^{{\gamma }_{dist}},$$

(5)

where $w_{len}$ is the average side length of the detection quadrilateral (in pixels); $w_{sin}$ is the average of the sines of the angles of the detection quadrilateral; $w_{dist}$ is the Euclidean distance from the center of the detection quadrilateral to the principal point of the image; ${\gamma }_{diag}$ is the image diagonal length for normalization; the ratio ${\displaystyle \raisebox{1ex}{$w_{len}$}\!\left/ \!\raisebox{-1ex}{${\gamma }_{len}$}\right.}$ is clipped to $\left[0, 1\right]$, so any marker whose projected side length exceeds ${\gamma }_{len}$ pixels is considered fully reliable with respect to size and receives the maximum contribution from this term; and ${\gamma }_{len}$, ${\gamma }_{sin}$, ${\gamma }_{dist}$ are regularization hyperparameters determining the contribution of each factor.

The proposed heuristic weighting formula accounts for three geometric factors affecting detection accuracy:

• Projection size $w_{len}$. Markers closer to the camera occupy a larger area on the sensor, improving the localization accuracy of their corners.
• Viewing angle obliqueness $w_{sin}$. When a marker is observed at an oblique angle, its projection becomes distorted, increasing detection uncertainty.
• Radial position $w_{dist}$. Detections near the image edges (far from the principal point) are more susceptible to residual lens distortion.

The hyperparameter values used throughout all experiments are ${\gamma }_{len}=250$ px, ${\gamma }_{sin}=0.5$, and ${\gamma }_{dist}=2$. The value ${\gamma }_{len}=250$ px represents the projected marker side length at which detection is considered fully reliable; it corresponds to the expected maximum projected size of a marker at the typical operating distances of the experimental setup. The exponent ${\gamma }_{dist}=2$ is motivated by the Brown-Conrady distortion model: the dominant radial distortion term is cubic in the radial coordinate, so the rate at which corner localization error grows with distance from the principal point scales approximately quadratically, and the quadratic weight decay matches this order of nonlinearity. The exponent ${\gamma }_{sin}=0.5$ defines a square-root (convex) function that assigns relatively more weight to small deviations from frontal viewing—where even modest obliqueness introduces measurable perspective error in corner localization—while compressing differences among highly oblique detections that are already heavily downweighted. To assess the sensitivity to this choice, a grid search over ${\gamma }_{sin}\in \left\{0.05, 0.10, \dots , 1.00\right\}$ was conducted on 10 synthetic datasets, described in Section 6.1.1. A Kruskal-Wallis test found no statistically significant differences in either translation or rotation errors (calculated using Formulas (10) and (11) respectively) across the tested range. Since calibration accuracy is thus robust to the precise value of ${\gamma }_{sin}$, the value $0.5$ is adopted as a natural default—it is the simplest nonlinear choice in the range $\left(0, 1\right)$, introducing moderate convexity without collapsing to linear weighting (${\gamma }_{sin}=1.0$) or a near-threshold behavior (${\gamma }_{sin}=0.0$).

This weighting allows the PnP algorithm to minimize reprojection error non-uniformly across all visible markers, so that the optimization emphasis reflects detection reliability, thereby improving the stability of camera pose estimation.

5.2. Multiview Triangulation

A critical limitation of the baseline method is its single-shot estimation of marker positions: marker coordinates are fixed at the moment of first detection and never subsequently refined. This leads to error accumulation, as each marker inherits the error of the camera from which its position was estimated, compounds it with its own positioning inaccuracy, and propagates the combined error to the next camera added to the scene.

The proposed method eliminates this drawback by refining marker positions after each new camera is added. Specifically, multi-view triangulation is performed for each $j$-th marker from all already calibrated cameras that observe it. At the moment when the $i$-th camera is added, if the position of the $j$-th marker it observes is already known, this implies that the marker has been observed by at least two positioned cameras, including the $i$-th, thereby enabling triangulation. If the $j$-th marker is encountered for the first time, its position is estimated by solving the PnP problem, as in the baseline method. For computational efficiency, the following triangulation algorithm is proposed.

First, the 3D coordinate of the $k$-th corner of the $j$-th marker is independently computed such that rays cast from the optical centers of all cameras participating in the triangulation towards this point minimize the deviation from the rays directed toward the corresponding 2D detection coordinates on the image planes. Formally, this is achieved by solving an optimization problem minimizing the squared norms of the cross products of the ray pairs (i.e., the squared deviations from collinearity) in homogeneous coordinates:

$${\tilde{\mathbf{x}}}_{jk}^{*}=\underset{‖{\tilde{\mathbf{x}}}_{\mathit{jk}}‖=1}{\mathrm{arg\; min}}\sum _{i\in \left({\mathcal{O}}_j\cap \mathcal{P}\right)}{‖{\tilde{\mathbf{u}}}_{ijk}\times \left({\mathbf{P}}_i{\tilde{\mathbf{x}}}_{jk}\right)‖}^2,$$

(6)

where ${\tilde{\mathbf{x}}}_{jk}\in {\mathbb{R}}^4={\left(x_{jk},y_{jk},z_{jk},{\omega }_{jk}\right)}^{\mathrm{T}}$ are the homogeneous coordinates of the $k$-th corner of the $j$-th marker; ${\tilde{\mathbf{u}}}_{ijk}\in {\mathbb{R}}^3={\left(u_{ijk},v_{ijk},1\right)}^{\mathrm{T}}$ are the homogeneous coordinates of the detection of the $k$-th corner of the $j$-th marker on the image plane of the $i$-th camera; ${\mathbf{P}}_i\in {\mathbb{R}}^{3\times 4}$ is the projection matrix from the 3D world coordinate system to the image plane of the $i$-th camera; ${\mathcal{O}}_j$ is the set of indices of all cameras observing the $j$-th marker; $\mathcal{P}$ is the set of indices of all cameras already positioned at the current iteration; $‖\cdot ‖$ is the Euclidean norm; and $\times$ denotes the cross product.

Problem (6) represents a classical formulation for the Direct Linear Transformation (DLT) method [37]. The solution is obtained analytically and has a linear computational complexity of $O\left(N\right)$ in the worst case, where the marker is triangulated from all $N$ cameras simultaneously.

Since the geometry of the marker is known a priori, the next step estimates the rotation and translation that best align the template 3D representation of the marker ${\mathbf{X}}_{template}\in {\mathbb{R}}^{3\times 4}$ with the triangulated points. This problem is solved analytically using the Kabsch-Umeyama method [38] via the singular value decomposition (SVD) of the point covariance matrix, with a constant asymptotic complexity of $O\left(1\right)$.

This extension to the baseline algorithm ensures that the position of each marker is always consistent with all cameras that observed it. Consequently, error accumulation is mitigated as the camera chain grows.

More formally, the proposed method is described in Algorithm 2.

Algorithm 2: Proposed extrinsic calibration method
Input:	$N$—number of cameras; $M$—number of markers; $\left\{{\mathcal{V}}_i | i\in \left[1,N\right]\right\}$—sets of markers indices visible from the i-th camera; $\left\{{\mathcal{O}}_j | j\in \left[1,M\right]\right\}$—sets of cameras indices observing the j-th marker; $\left\{{\mathbf{u}}_{ij} | i\in \left[1,N\right],j\in {\mathcal{V}}_i\right\}$—detections of the j-th marker on the i-th camera image plane.
Output:	$\left\{{\mathbf{R}}_{c_i} | i\in \left[1,N\right]\right\}$—rotations of all cameras; $\left\{{\mathbf{t}}_{c_i} | i\in \left[1,N\right]\right\}$—translations of all cameras; $\left\{{\mathbf{R}}_{m_j} | j\in \left[1,M\right]\right\}$—rotations of all markers; $\left\{{\mathbf{t}}_{m_j} | j\in \left[1,M\right]\right\}$—translations of all markers.
1.	$\mathcal{U}⟵\left\{1,\dots ,N\right\}$	$⊳$ Set of uninitialized cameras indices
2.	$\mathcal{M}⟵\mathit{\varnothing }$	$⊳$ Set of positioned markers indices
3.	while $\mathcal{U}\ne \mathcal{\varnothing }$ do
4.	if $\left|\mathcal{U}\right|=N$ then
5.	$i⟵\underset{i\in \mathcal{U}}{\mathrm{arg\; max}}\left|{\mathcal{V}}_i\right|$	$⊳$ Pick camera with strict max observations
6.	${\mathbf{R}}_{c_i}⟵\mathbf{I}$	$⊳$ Initialize camera rotation as identity matrix
7.	${\mathbf{t}}_{c_i}⟵{\left(0,0,0\right)}^{\mathrm{T}}$	$⊳$ Initialize camera translation by zeros
8.	else
9.	$i⟵\underset{i}{\mathrm{arg\; max}}\left|{\mathcal{V}}_i\cap \mathcal{M}\right|$	$⊳$ Pick camera with strict max observations $\mathcal{M}$
10.	${\mathcal{R}}_m⟵\left\{{\mathbf{R}}_{m_j} | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of visible markers rotations
11.	${\mathcal{T}}_m⟵\left\{{\mathbf{t}}_{m_j} | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of visible markers translations
12.	${\mathcal{D}}_m⟵\left\{{\mathbf{u}}_{ij} | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of visible markers detections
13.	$\mathcal{W}⟵\left\{\mathrm{W}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\left({\mathbf{u}}_{ij}\right) | j\in \left({\mathcal{V}}_i\cap \mathcal{M}\right)\right\}$	$⊳$ Set of detections weights estimated by (5)
14.	${\mathbf{R}}_{c_i},{\mathbf{t}}_{c_i}⟵\mathrm{S}\mathrm{o}\mathrm{l}\mathrm{v}\mathrm{e}\mathrm{W}\mathrm{P}\mathrm{n}\mathrm{P}\left({\mathcal{R}}_m,{\mathcal{T}}_m,{\mathcal{D}}_m,\mathcal{W}\right)$	$⊳$ Estimate camera pose by solving weighted PnP
15.	end if
16.	$\mathcal{U}⟵\mathcal{U}\setminus i$	$⊳$ Mark camera as initialized
17.	for each marker $j\in {\mathcal{V}}_i$ do
18.	if $j\notin \mathcal{M}$ do
19.	${\mathbf{R}}_{m_j},{\mathbf{t}}_{m_j}⟵\mathrm{S}\mathrm{o}\mathrm{l}\mathrm{v}\mathrm{e}\mathrm{P}\mathrm{n}\mathrm{P}\left({\mathbf{R}}_{c_i},{\mathbf{t}}_{c_i},{\mathbf{u}}_{ij}\right)$	$⊳$ Estimate marker pose by solving PnP
20.	else
21.	${\mathcal{R}}_c⟵\left\{{\mathbf{R}}_{c_k} | k\in \left({\mathcal{O}}_j\setminus \mathcal{U}\right)\right\}$	$⊳$ Set of cameras rotations observing marker
22.	${\mathcal{T}}_c⟵\left\{{\mathbf{t}}_{c_k} | k\in \left({\mathcal{O}}_j\setminus \mathcal{U}\right)\right\}$	$⊳$ Set of cameras translations observing marker
23.	${\mathcal{D}}_c⟵\left\{{\mathbf{u}}_{kj} | k\in \left({\mathcal{O}}_j\setminus \mathcal{U}\right)\right\}$	$⊳$ Set of marker detections from all cameras
24.	${\mathbf{R}}_{m_j},{\mathbf{t}}_{m_j}⟵\mathrm{T}\mathrm{r}\mathrm{i}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{e}\left({\mathcal{R}}_c,{\mathcal{T}}_c,{\mathcal{D}}_c\right)$	$⊳$ Estimate marker pose by triangulation
25.	end if
26.	$\mathcal{M}⟵\mathcal{M}\cup j$	$⊳$ Mark marker as initialized
27.	end for
28.	end while

5.3. Method Complexity

The computational complexity of the baseline method for scene initialization is the sum of the complexities of solving PnP problems at each iteration. Since positioning each marker involves only a single camera, the complexity of this operation is independent of $M$ or $N$ and is $O\left(1\right)$. The total complexity of positioning all $M$ markers is $O\left(M\right)$. To position the $i$-th camera, the PnP problem is solved using all positioned markers visible from that camera. In the worst case, where the $i$-th camera observes all $M$ markers, solving the problem for a single camera has a complexity of $O\left(M\right)$. Thus, the complexity of the baseline initialization method is $O\left(M+MN\right)=O\left(MN\right)$.

However, as mentioned previously, the complete calibration process of a multi-camera system almost never proceeds without Bundle Adjustment (BA). The complexity of BA is estimated as $O\left(M{N}^2+N^3\right)$ or $O\left(M{N}^2\right)$ under the assumption that $M\gg N$, which clearly dominates the complexity of scene initialization by the baseline method.

The initialization method proposed in this paper introduces two heuristics to the baseline method:

• Weighted PnP. Computing the weights $w$ using heuristic Formula (5) is performed in $O\left(1\right)$, as it is independent of $N$, $M$, or any other variable quantities.
• Multi-view Triangulation. In the worst case, when adding the $i$-th camera, the positions of all $M$ markers are recalculated. As discussed in Section 5.2, the complexity of triangulating a single marker is $O\left(N\right)$; therefore, $O\left(MN\right)$ operations are performed for each new $i$-th camera. The resulting complexity added by the triangulation process to the baseline method is $O\left(M{N}^2\right)$, which does not exceed the complexity of a single BA iteration.

Thus, the complexity of the entire calibration process, comprising the proposed initialization method and BA, remains the same as that of the baseline method, amounting to $\left(M{N}^2+N^3\right)$ in the general case.

Incorporating the proposed scene initialization method does not alter the asymptotic complexity class of the entire calibration process. The primary advantage of the method lies in a substantial improvement in the quality of the initial estimate of scene parameters and the associated error covariance structure. This, in turn, leads to a reduction in the number of BA iterations required to achieve convergence, thereby reducing overall computational costs and enhancing calibration reliability in the presence of noisy data.

Empirical support for this claim is provided by the bundle adjustment convergence comparison in Section 7. The proposed initialization consistently reduces the number of BA iterations required to reach the stopping criterion, confirming that the additional cost incurred at the initialization stage is offset by faster convergence of the global optimization.

6. Experiments

To evaluate the effectiveness of the proposed initialization method, a series of experiments was conducted under both fully controlled conditions with known parameters of synthetically generated scenes and real-world capture scenarios typical of object-centric rigs. The experiments compare the baseline and proposed methods for initial camera calibration using various metrics appropriate to the nature of the data.

6.1. Synthetic Data

6.1.1. Generation

To conduct controlled and reproducible experiments, 40 unique synthetic datasets were generated. Each scene represents a multi-camera system comprising 40 virtual cameras and 80 AprilTag markers randomly distributed on the inner surface of a cylinder. This arrangement ensures substantial overlap between camera fields of view and is representative of standard object-centric configurations. All cameras were set to a resolution of 1920 × 1080 pixels. The camera field of view was randomly selected from the range of 80 to 120 degrees for each dataset. The height and radius of the cylinder were randomly chosen from the ranges $\left[1.4,1.8\right]$ meters and $\left[0.8,1.0\right]$ meters, respectively. Figure 1 presents an example of a generated dataset, where cameras are depicted as orange frustums with the optical center at the apex.

Images from the virtual cameras were generated by rendering the 3D scene. Since only AprilTag marker detections are relevant for this experiment, surface material properties and lighting were not simulated during rendering. Figure 2 shows an example of a render from one of the cameras.

To approximate real-world conditions, random noise was added to the ground-truth values of the intrinsic matrix ${\mathbf{K}}_i$ of each $i$-th camera immediately after rendering:

$${\mathbf{K}}_i^{*}=\left[\begin{array}{ccc}{f}_{x,i}+\mathrm{rand}\left(\beta \right)& 0& p_{x,i}+\mathrm{rand}\left(\beta \right)\\ 0& f_{y,i}+\mathrm{rand}\left(\beta \right)& p_{y,i}+\mathrm{rand}\left(\beta \right)\\ 0& 0& 1\end{array}\right],$$

(7)

where $f_{x,i}$ and $f_{y,i}$ are the true focal lengths of the $i$-th camera; $p_{x,i}$ and $p_{y,i}$ are the true coordinates of the principal point of the $i$-th camera; $\mathrm{rand}\left(\beta \right)$ is a random number selected from the range $\left[-\beta ;\beta \right]$; and $\beta$ is a parameter determining the noise amplitude (for the experiments, a value of 0.5% of the camera image diagonal length was used, i.e., $\beta \approx 11$ pixels).

The addition of noise simulates the errors incurred during preliminary intrinsic calibration, which is inherent in any real-world measurement process. Additionally, distortion $\mathbf{D}$ corresponding to the polynomial model for fisheye lenses was applied to all generated renders:

$$\mathbf{D}=\left[\begin{array}{c}\mathrm{rand}\left(2\cdot {10}^{-3}\right)\\ \mathrm{rand}\left(1\cdot {10}^{-3}\right)\\ \mathrm{rand}\left(5\cdot {10}^{-4}\right)\\ \mathrm{rand}\left(5\cdot {10}^{-4}\right)\end{array}\right],$$

(8)

where the values in the column vector $\mathbf{D}$ correspond to the four distortion coefficients of the Brown-Conrady model [10].

Adding slight distortion to the images allows for evaluating the robustness of the initialization methods to uncompensated optical distortions, which are always present in real cameras due to the physical limitations of optical systems.

6.1.2. Evaluation Metrics for Synthetic Data

In real-world conditions, ideal camera positions are unknown a priori; otherwise, calibration would be meaningless. Therefore, metrics that rely solely on 2D point locations on image planes and their corresponding 3D positions are employed to evaluate calibration quality. Such metrics include reprojection error, triangulation error (the distance between rays cast from different cameras toward the same 3D point) [39], photometric evaluation (when 3D reconstruction is performed during calibration, the similarity between rendered and real images can be assessed) [40], and others. However, this work utilizes synthetic images generated by virtual cameras with known precise intrinsic and extrinsic parameters; therefore, it is most appropriate to use metrics based on direct comparison between the ground-truth and calibrated parameters.

Among these evaluation approaches, geometric metrics such as reprojection error, angular error, and epipolar error directly quantify the consistency between the estimated camera model and the observed 2D measurements—precisely the quantity that calibration procedures optimize. Task-level metrics such as photometric similarity in 3D reconstruction are useful for assessing end-to-end system performance, but they are confounded by factors beyond calibration quality: photometric accuracy depends on scene geometry, surface material properties, and—in the case of learning-based methods [40]—on model capacity and training convergence. Isolating the contribution of calibration accuracy from these confounds requires controlled conditions that are not always practical to establish. The practical relevance of geometric calibration accuracy is nonetheless well-established: errors in extrinsic parameters propagate directly into the triangulation of 3D scene points [39], introduce geometric inconsistencies between camera pairs as characterised by epipolar geometry [4,41], and affect any downstream task that relies on the geometric correspondence between physical space and the image plane.

To evaluate the accuracy of the proposed method and compare it with the baseline method, metrics characterizing the errors in determining camera position and orientation were selected. It is important to note that extrinsic calibration algorithms recover the relative positioning of cameras; however, the resulting coordinate system may be translated and rotated relative to the ground truth coordinate system used for data generation. Nevertheless, since the physical size of the AprilTag markers is known a priori, the recovered 3D structure preserves absolute scale. This allows the two coordinate systems to be aligned for a valid comparison. Figure 3 presents an example of the ground truth camera system (green) and the estimated camera positions after calibration (red).

Alignment of the camera coordinate systems involves finding a rigid transformation, consisting of a rotation ${\mathbf{R}}_{align}\in {\mathbb{R}}^{3\times 3}$ and a translation ${\mathbf{t}}_{align}\in {\mathbb{R}}^3$, that best aligns the calibrated camera system with the ground truth system. To determine this transformation, the Kabsch-Umeyama algorithm [38] is employed. A separate scale factor estimation is not required, since the known physical marker dimensions were used during calibration. Formally, the alignment problem is formulated as follows:

$$\left({\mathbf{R}}_{align}^{*},{\mathbf{t}}_{align}^{*}\right)=\underset{\left({\mathbf{R}}_{\mathit{align}},{\mathbf{t}}_{\mathit{align}}\right)}{\mathrm{arg\; min}}{\displaystyle \sum _{i=1}^N}{‖{\mathbf{c}}_{gt,i}-\left({\mathbf{R}}_{align}{\mathbf{c}}_{est,i}+{\mathbf{t}}_{align}\right)‖}^2,$$

(9)

where ${\mathbf{c}}_{gt,i}\in {\mathbb{R}}^3$ are the coordinates of the optical center of the $i$-th camera in the ground truth system; ${\mathbf{c}}_{est,i}\in {\mathbb{R}}^3$ are the coordinates of the optical center of the $i$-th camera in the calibrated system; $N=40$ is the number of cameras in each synthetic dataset; and $‖\cdot ‖$ denotes the Euclidean norm.

In this context, the Kabsch-Umeyama algorithm provides an analytical solution to the least squares problem (9); therefore, it aligns the camera systems in the best possible way in terms of pairwise squared distances. The following metrics are then computed.

Translation error $E_{trans}$. It is computed as the Euclidean distance between the true position of the camera optical center and the aligned estimated position after calibration for the $i$-th camera:

$$E_{trans,i}=‖{\mathbf{c}}_{gt,i}-\left({\mathbf{R}}_{align}{\mathbf{c}}_{est,i}+{\mathbf{t}}_{align,i}\right)‖,$$

(10)

Rotation error ${\mathrm{E}}_{rot}$. It is computed as the angle required to align the true camera orientation with the aligned estimated orientation for the $i$-th camera:

$$E_{rot,i}=\arccos \left({\displaystyle \frac{\mathrm{trace}\left({\mathbf{R}}_{gt,i}{\left({\mathbf{R}}_{align}{\mathbf{R}}_{est,i}\right)}^{\mathrm{T}}\right)-1}{2}}\right),$$

(11)

where $\mathrm{trace}\left(\mathbf{M}\right)$ is the sum of the diagonal elements of a square matrix $\mathbf{M}$; ${\mathbf{R}}_{gt,i}\in {\mathbb{R}}^{3\times 3}$ is the rotation matrix transforming the global coordinate system to the coordinate system of the $i$-th ground truth camera; ${\mathbf{R}}_{est,i}\in {\mathbb{R}}^{3\times 3}$ is the rotation matrix transforming the global coordinate system to the coordinate system of the $i$-th camera estimated during calibration.

6.1.3. Results for Synthetic Data

Both the baseline and the proposed method were applied to all 40 synthetic datasets for the initial extrinsic calibration of cameras. For each case, both metrics $E_{trans}$ and $E_{rot}$ discussed earlier were computed. Figure 4 presents a boxplot where the y-axis represents the values of the $E_{trans}$ metric for all cameras across all datasets, and the x-axis indicates the dataset numbers. Figure 5 similarly illustrates the distribution of the $E_{rot}$ metric.

Both plots indicate that the proposed method yields consistently better results than the baseline method across both metrics.

To assess the statistical significance of the observed differences, the one-sided Wilcoxon test was applied with a significance level of $\alpha =0.05$. The results show that the proposed method significantly outperforms the baseline in 97.5% of the datasets for the $E_{trans}$ metric and in 85% of the datasets for the $E_{rot}$ metric. In the remaining cases, no statistically significant differences were observed.

Based on these results, it can be concluded that the proposed method for initial extrinsic camera calibration outperforms the baseline on synthetic datasets in terms of both translation and rotation metrics, yielding more accurate camera pose estimates.

6.2. Real Data

6.2.1. Multicamera Setup

A single GoPro Hero 13 Black camera was used for real-world data acquisition, configured for video recording at a resolution of 3840 × 2160. Twenty-nine AprilTag markers were placed on four vertical stands (six per stand), while the remaining five markers were distributed on the floor, as shown in Figure 6.

Subsequently, a camera pass around the setup was performed, and 20 frames were extracted from the video stream. Since the objective of the experiment is solely camera calibration using marker detections, the resulting dataset is equivalent to a simultaneous 20-camera capture, with the caveat that all cameras share the same intrinsics and distortion coefficients, as only a single physical camera was used. The intrinsics and distortion coefficients were pre-calibrated using Zhang’s classical algorithm [11] with a chessboard. The GoPro Hero 13 Black features a fisheye lens with strong distortion; however, since undistorted images and the corresponding camera parameters are used in the subsequent experiments, the pinhole camera model is assumed.

Two aspects of this setup warrant clarification. First, the sequential nature of the capture does not introduce any methodological limitation: the scene is static, and the method operates solely on 2D marker detections, which are independent of capture timing. The resulting dataset is therefore equivalent to a synchronous multi-camera capture for the purposes of calibration. Second, the fact that all virtual cameras share identical intrinsic parameters represents a special case of the general setting rather than a fundamental constraint: the method accepts independently calibrated intrinsic matrices ${\mathbf{K}}_i$ for each $i$-th camera and processes them separately throughout. The synthetic experiments, in which independent random noise is applied to the intrinsic parameters of each of the 40 cameras, already constitute an evaluation under heterogeneous intrinsics.

6.2.2. Evaluation Metrics for Real Data

Unlike synthetic data, real-world data lacks ground-truth camera positions. Therefore, calibration quality is assessed using metrics that evaluate the local and global geometric consistency of the scene.

One of the most well-known metrics is the reprojection error $E_{repr}$. This is a classical metric that is minimized both when solving the PnP problem (1) and during Bundle Adjustment (2). It is calculated as the Euclidean distance between the projection of the $k$-th corner of the $j$-th marker onto the image plane of the $i$-th camera and the detection of this corner:

$$E_{repr,ijk}=‖{\mathbf{u}}_{ijk}-\pi \left({\mathbf{K}}_i,{\mathbf{R}}_i,{\mathbf{t}}_i,{\mathbf{x}}_{jk}\right)‖,$$

(12)

where ${\mathbf{u}}_{ijk}\in {\mathbb{R}}^2$ is the detection of the $k$-th corner of the $j$-th marker on the image plane of the $i$-th camera; ${\mathbf{x}}_{jk}\in {\mathbb{R}}^3$ are the coordinates of the $k$-th corner of the $j$-th marker in the global coordinate system; $\pi \left({\mathbf{K}}_i,{\mathbf{R}}_i,{\mathbf{t}}_i,{\mathbf{x}}_{jk}\right)\in {\mathbb{R}}^2$ is the projection of point ${\mathbf{x}}_{jk}$ onto the image plane of the $i$-th camera; $‖\cdot ‖$ is the Euclidean norm.

Another well-known metric is the angular error $E_{ang}$. It is calculated as the angle between the ray back-projected from the pixel coordinates of the detection of the $k$-th corner of the $j$-th marker on the image plane of the $i$-th camera and the vector pointing from the optical center of the $i$-th camera to the actual 3D point of the $k$-th corner of that $j$-th marker:

$$E_{ang,ijk}=\arccos \left({\displaystyle \frac{{\mathbf{v}}_{det,ijk}\cdot {\mathbf{v}}_{est,ijk}}{‖{\mathbf{v}}_{det,ijk}‖‖{\mathbf{v}}_{est,ijk}‖}}\right),$$

(13)

where ${\mathbf{v}}_{det,ijk}\in {\mathbb{R}}^3={\mathbf{R}}_i^{\mathrm{T}}{\mathbf{K}}_i^{-1}{\tilde{\mathbf{u}}}_{ijk}$ is the ray originating from the optical center of the $i$-th camera towards the detection coordinates of the $k$-th corner of the $j$-th marker on the image plane, expressed in the world 3D coordinate system; ${\mathbf{R}}_i\in {\mathbb{R}}^{3\times 3}$ is the rotation matrix for transforming from the world coordinate system to the coordinate system of the $i$-th camera; ${\mathbf{K}}_i\in {\mathbb{R}}^{3\times 3}$ is the intrinsic matrix of the $i$-th camera; ${\tilde{\mathbf{u}}}_{ijk}\in {\mathbb{R}}^3$ are the homogeneous coordinates of the detection of the $k$-th corner of the $j$-th marker on the image plane of the $i$-th camera; ${\mathbf{v}}_{est,ijk}\in {\mathbb{R}}^3={\mathbf{X}}_{jk}+{\mathbf{R}}_i^{\mathrm{T}}{\mathbf{t}}_i$ is the ray originating from the optical center of the $i$-th camera towards the 3D coordinates ${\mathbf{X}}_{jk}\in {\mathbb{R}}^3$ of the $k$-th corner of the $j$-th marker in the world coordinate system; ${\mathbf{t}}_i$ is the translation vector for transforming from the world coordinate system to the coordinate system of the $i$-th camera; $‖\cdot ‖$ is the Euclidean norm.

Another widely known metric for evaluating the quality of calibration is the epipolar error $E_{epi}$. This metric evaluates the quality of pairwise camera poses using the averaged Sampson distance [42] for all point pairs, characterizing the deviation from satisfying the epipolar constraint:

$${\tilde{\mathbf{u}}}_{mjk}^{\mathrm{T}}{\mathbf{F}}_{nm}{\tilde{\mathbf{u}}}_{njk}=0,$$

(14)

where ${\tilde{\mathbf{u}}}_{njk}\in {\mathbb{R}}^3={\left(u_{njk},v_{njk},1\right)}^{\mathrm{T}}$ and ${\tilde{\mathbf{u}}}_{mjk}\in {\mathbb{R}}^3={\left(u_{mjk},v_{mjk},1\right)}^{\mathrm{T}}$ are the homogeneous coordinates of the detection of the $k$-th corner of the $j$-th marker on the image planes of the $n$-th and $m$-th cameras, respectively; ${\mathbf{F}}_{nm}$ is the fundamental matrix [4,41] encoding the epipolar geometry of the camera pair $n$ and $m$.

The error $E_{epi}$ is calculated for each pair of cameras $n$ and $m$:

$$E_{epi,nm}={\displaystyle \frac{1}{4\left|{\mathcal{V}}_n\cap {\mathcal{V}}_m\right|}}\sum _{j\in \left({\mathcal{V}}_n\cap {\mathcal{V}}_m\right)}{\displaystyle \sum _{k=1}^4}\sqrt{{\displaystyle \frac{{\left({\tilde{\mathbf{u}}}_{mjk}^{\mathrm{T}}{\mathbf{F}}_{nm}{\tilde{\mathbf{u}}}_{njk}\right)}^2}{l_{{njk}_1}^2+l_{{njk}_2}^2+l_{{mjk}_1}^2+l_{{mjk}_2}^2}}},$$

(15)

where ${\mathcal{V}}_n$ and ${\mathcal{V}}_m$ are the sets of indices of markers detected in the $n$-th and $m$-th cameras, respectively; ${\mathbf{l}}_{njk}\in {\mathbb{R}}^3={\mathbf{F}}_{nm}{\tilde{\mathbf{u}}}_{njk}={\left(l_{{njk}_1},l_{{njk}_2},l_{{njk}_3}\right)}^{\mathrm{T}}$ and ${\mathbf{l}}_{mjk}\in {\mathbb{R}}^3={\mathbf{F}}_{nm}^{\mathrm{T}}{\tilde{\mathbf{u}}}_{mjk}={\left(l_{{mjk}_1},l_{{mjk}_2},l_{{mjk}_3}\right)}^{\mathrm{T}}$ are the vectors corresponding to the epipolar lines on the image planes of the $n$-th and $m$-th cameras, respectively.

6.2.3. Results for Real Data

For the real-world data, both the baseline method and the proposed method were applied for the initial extrinsic camera calibration. It should be noted that for both methods, the iterative addition of cameras was performed in the same order, starting from camera 12.

Figure 7 presents a boxplot where the x-axis represents the camera indices $i=\overline{1,20}$ in the dataset, and the y-axis displays the values of the metric $E_{repr,ijk}$ for each $i$-th camera. These values correspond to all corners $k=\overline{1,4}$ of every marker $j\in {\mathcal{V}}_i$, where ${\mathcal{V}}_i$ is the set of indices of markers for which detections are available in the $i$-th camera.

As can be clearly seen from the plot in Figure 7, the proposed initial calibration method performs significantly better in terms of reprojection error. It is worth noting the case of camera 12, which initiated the iterative calibration process in both methods. This camera is the only instance where the baseline method outperformed the proposed method. Since the baseline algorithm does not update marker positions after their initial placement, the reprojection error for the first camera is essentially the direct result of solving the PnP problem over all markers visible to it. The proposed method, on the other hand, updates marker positions via triangulation at each iteration, thereby sacrificing local accuracy in favor of global consistency.

In a manner similar to Figure 7, Figure 8 presents a boxplot for all errors $E_{ang,ijk}$ on the real-world dataset.

As can be seen from Figure 8, the angular error exhibits results nearly identical to the previously discussed reprojection error. Compared to the baseline, the proposed method demonstrates superior results for all cameras except for camera 12, from which the iterative calibration originated.

Figure 9 presents a graphical representation of the matrix of pairwise epipolar errors $E_{epi,nm}$ for all camera pairs $n,m=\overline{1,20}$. Each camera pair is represented by a semicircle whose diameter is proportional to the error magnitude and whose color encodes the number of common points $4\left|{\mathcal{V}}_n\cap {\mathcal{V}}_m\right|$, where ${\mathcal{V}}_n$ and ${\mathcal{V}}_m$ are the sets of marker indices for which detections are available on the $n$-th and $m$-th cameras, respectively. The upper semicircle corresponds to the baseline method and the lower semicircle to the proposed method. A legend with color and size scales is provided at the top of the plot.

Notably, the most significant differences in errors between the two methods are predominantly observed for camera pairs with relatively few common points. This indicates that the proposed method achieves more stable relative positioning even for cameras whose fields of view barely overlap, unlike the baseline method. For camera 12, which served as the starting point for calibration in both cases, the proposed method exhibits the highest average errors, while the baseline method yields the lowest. This confirms the earlier observation about the special role of the starting camera.

Figure 10 presents a 3D representation of the entire scene comprising 20 cameras and 29 AprilTag markers, reconstructed using the baseline and the proposed initialization methods. The cameras are colored according to the mean reprojection error $E_{repr,i}={\displaystyle \frac{1}{4\left|{\mathcal{V}}_i\right|}}\sum _{j\in {\mathcal{V}}_i}{\displaystyle {\sum }_{k=1}^4}{E}_{repr,ijk}$ for each $i$-th camera, where ${\mathcal{V}}_i$ denotes the set of indices of markers detected by the $i$-th camera. For reference, the floor plane is indicated by a gray circle.

The 3D visualization illustrates how even minor camera displacement errors can substantially degrade calibration quality, as quantified by reprojection error. Despite the visually similar arrangement of markers and cameras in both cases, the baseline approach yields an average error exceeding 7 pixels, while the error of the proposed method remains below one.

Camera 12, from which the calibration process started in both methods, is indicated by a red arrow in the 3D visualization. It is easily identified in the baseline method by its bright green color, corresponding to the lowest reprojection error, as discussed above. A clear spatial pattern is also visible in the baseline method: cameras with minimal errors cluster together, as do those with maximal errors. Therefore, to confirm this pattern, we evaluate the correlation between the error magnitude of the $i$-th camera and its proximity to camera 12 for all $i=\overline{1,20}$. Euclidean distance in meters (16) and angular distance in radians (17) are used as proximity metrics:

$$d_{Euclidean,i}=‖{\mathbf{R}}_i^{\mathrm{T}}{\mathbf{t}}_i-{\mathbf{R}}_{12}^{\mathrm{T}}{\mathbf{t}}_{12}‖,$$

(16)

$$d_{angular,i}=\arccos \left({\displaystyle \frac{\mathrm{trace}\left({\mathbf{R}}_i{\mathbf{R}}_{12}^{\mathrm{T}}\right)-1}{2}}\right),$$

(17)

where ${\mathbf{R}}_i\in {\mathbb{R}}^{3\times 3}$ and ${\mathbf{t}}_i\in {\mathbb{R}}^3$ are the rotation and translation for transforming a 3D point from the world coordinate system to the $i$-th camera coordinate system (the extrinsics of the $i$-th camera); $-{\mathbf{R}}_i^{\mathrm{T}}{\mathbf{t}}_i\in {\mathbb{R}}^3$ are the coordinates of the optical center of the $i$-th camera in the world coordinate system; ${\mathbf{R}}_i{\mathbf{R}}_{12}^{\mathrm{T}}\in {\mathbb{R}}^{3\times 3}$ is the rotation matrix for transforming from the coordinate system of the 12th camera to the coordinate system of the $i$-th camera; $‖\cdot ‖$ denotes the Euclidean norm; and $\mathrm{trace}\left(\mathbf{M}\right)$ is the sum of the diagonal elements of a square matrix $\mathbf{M}$.

Table 1. The Pearson correlation coefficients.

Correlation	Baseline Method	Proposed Method
Between $E_{repr,i}$ and $d_{Euclidean,i}$	0.650	−0.632
Between $E_{repr,i}$ and $d_{angular,i}$	0.698	–

presents the Pearson correlation coefficients between the error magnitude of the $i$-th camera and its proximity to the 12th camera for all $i=\overline{1,20}$. Only statistically significant correlations are reported, at a significance level of $\alpha =0.05$.

The correlation coefficients indicate that the farther the $i$-th camera is from the starting camera, the worse its pose is estimated by the baseline algorithm, and the better it is estimated by the proposed method. This is presumably because the order of camera addition in both methods was determined by the number of commonly visible markers, so as to maximize the reliability of each subsequent camera pose estimate (Algorithms 1 and 2). Since all cameras were arranged in a circle, the minimal field-of-view overlap occurs for cameras on opposite sides of the scene. For the baseline method, this implies that the furthest camera accumulates the largest error, whereas for the proposed method, the farthest camera is positioned using markers whose coordinates have already been refined via triangulation from all previously added cameras, which yields greater reliability.

7. Ablation Study

Since the proposed method is a modification of the baseline achieved by adding two key extensions, an ablation study is performed to evaluate the contribution of each extension separately. To this end, two additional configurations of the proposed method were evaluated on the real data: one with PnP weighting disabled, and another with multi-view triangulation disabled.

For the comparison, the metric $E_{repr,ijk}$ was selected for all cameras $i=\overline{1,20}$, for all corners $k =\overline{1,4}$ of each marker $j\in {\mathcal{V}}_i$, where ${\mathcal{V}}_i$ is the set of indices of markers for which detections are available on the $i$-th camera. Figure 11 presents a comparison of error distributions for all aforementioned method variants in the form of a boxplot.

As can be seen from the figure, the primary contribution to the quality of the results is made by the multi-view triangulation of markers, which actively counteracts the sequential error accumulation characteristic of the baseline calibration method. PnP weighting, on the other hand, does not yield a comparable improvement on its own, but it reduces both the median and the bulk of the error distribution, albeit at the cost of slightly higher outlier errors. Importantly, the joint application of both extensions yields a significant improvement over their individual use.

Table 2. Reprojection error statistics for all ablation variants.

Method	Median (Pixels)	Mean (Pixels)
Baseline method	6.417	7.451
Proposed method (only weighted PnP)	5.828	7.244
Proposed method (only triangulation)	0.894	1.180
Proposed method (with both)	0.757	1.006

reports the exact median and mean reprojection errors for each method variant, providing a quantitative basis for the comparisons discussed below.

To assess the statistical significance of the observed differences, Wilcoxon signed-rank tests were performed on the metric $E_{repr,ijk}$ for all cameras $i=\overline{1,20}$, for all corners $k =\overline{1,4}$ of each marker $j\in {\mathcal{V}}_i$ visible from $i$-th camera for all pairwise ablation comparisons at a significance level of $\alpha =0.001$. Statistically significant results were obtained in all cases, including the weighted PnP component both in isolation and when added on top of the triangulation-only variant, confirming that the two extensions act as complementary rather than redundant mechanisms. As shown in Table 2, the weighted PnP scheme reduces the median reprojection error by approximately 9% relative to the baseline method, while the triangulation component reduces it by 86%. The full proposed method achieves a further 15% reduction in median error compared to the triangulation-only variant.

Furthermore, global optimization was executed for all versions of the proposed method as well as the baseline method, solving problem (2) using the Trust Region Reflective (TRF) method with a stopping criterion of $‖∆\mathsf{\theta }‖<{10}^{-6}$. Figure 12 presents plots of the objective function value versus the number of function evaluations. Insets magnify specific regions to facilitate comparison between the methods.

The plot confirms that the proposed method reduces the number of iterations required for bundle adjustment convergence, thereby lowering the overall computational cost of the calibration process. At the same time, it can be observed that the application of PnP weighting, compared to the baseline method, ultimately results in a lower (better) final value of the objective function, despite a slightly higher initial value.

The ablation results also demonstrate that the two components of the proposed method can be deployed independently. The weighted PnP scheme achieves a statistically significant improvement at $O(1)$ additional cost per iteration. The triangulation component provides the dominant accuracy gain at an initialization overhead of $O(NM)$, offset by a reduction in the number of subsequent bundle adjustment iterations required for convergence.

8. Conclusions and Future Work

This paper presents an incremental extrinsic camera parameter initialization method for multi-camera systems, which improves upon the baseline iterative registration algorithm based on solving the PnP problem. The key features distinguishing the proposed method from the baseline are the integration of heuristic weighting for AprilTag marker detections and the use of multi-view triangulation for the dynamic refinement of marker spatial coordinates at each step of scene expansion. Theoretical analysis demonstrates that the incorporation of these mechanisms does not increase the overall computational complexity of the calibration process, when global optimization is included, preserving the overall asymptotic complexity of $O\left(M{N}^2\right)$.

A series of experiments was conducted on synthetic and real-world data to evaluate the effectiveness of the proposed method. Synthetic data enables direct comparison between estimated and ground-truth camera parameters by aligning coordinate systems using the Kabsch-Umeyama algorithm. Validation on real-world data was performed using epipolar geometry metrics, the final reprojection error, and the angular error. The experimental results confirm that the proposed method yields more accurate camera localization and effectively mitigates geometric drift compared to the baseline. An ablation study isolating the contribution of each component reveals that multi-view triangulation is the primary factor in preventing geometric discrepancies. Furthermore, the weighted PnP scheme reduces the median positioning error and decreases the number of iterations required for the subsequent convergence of the global optimization (bundle adjustment).

Several promising directions for future research could further enhance the proposed calibration method. First, since the proposed method involves minimizing nonlinear functions at various stages, it would be worthwhile to investigate alternative optimization methods [18,43,44,45]. Modern population-based and heuristic algorithms could improve robustness to outliers and reduce the risk of convergence to local extrema.

Second, several extensions of the proposed weighting scheme merit investigation. A natural extension would be to integrate the proposed confidence metric directly into the bundle adjustment stage. Weighting the loss function during global optimization according to the accumulated heuristic detection confidence could allow the algorithm to penalize deviations of reliable observations more heavily, thereby further improving final calibration accuracy. Finally, the proposed weighting scheme could be used to refine the criterion for selecting the next camera during incremental addition, prioritizing configurations with the highest total observation reliability rather than simply the largest number of shared markers.