Next Article in Journal
Remote Sensing Image-Based Building Change Detection: A Case Study of the Qinling Mountains in China
Previous Article in Journal
A Lightweight Method for Road Defect Detection in UAV Remote Sensing Images with Complex Backgrounds and Cross-Scale Fusion
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Fast Intrinsic–Extrinsic Calibration for Pose-Only Structure-from-Motion

College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
*
Author to whom correspondence should be addressed.
Remote Sens. 2025, 17(13), 2247; https://doi.org/10.3390/rs17132247
Submission received: 20 May 2025 / Revised: 20 June 2025 / Accepted: 27 June 2025 / Published: 30 June 2025

Abstract

Structure-from-motion (SfM) is a foundational technology that facilitates 3D scene understanding and visual localization. However, bundle adjustment (BA)-based SfM is usually very time-consuming, especially when dealing with numerous unknown focal length cameras. To address these limitations, we proposed a novel SfM system based on pose-only adjustment (PA) for intrinsic and extrinsic joint optimization to accelerate computing. Firstly, we propose a base frame selection method based on depth uncertainty, which integrates the focal length and parallax angle under a multi-camera system to provide more stable depth estimation for subsequent optimization. We explicitly derive a global PA of joint intrinsic and extrinsic parameters to reduce the high dimensionality of the parameter space and deal with cameras with unknown focal lengths, improving the efficiency of optimization. Finally, a novel pose-only re-triangulation (PORT) mechanism is proposed for enhanced reconstruction completeness by recovering failed triangulations from incomplete point tracks. The proposed framework has been demonstrated to be both faster and comparable in accuracy to state-of-the-art SfM systems, as evidenced by public benchmarking and analysis of the visitor photo dataset.

1. Introduction

Structure-from-motion (SfM) demonstrates significant potential for geometry reconstruction and visual localization [1,2,3], while the efficient estimation of cameras with unknown focal lengths remains critically important in contemporary data-dense scenarios [2,4,5,6]. Bundle adjustment (BA), which involves the joint non-linear refinement of camera and point parameters, is a critical component of most SfM systems. However, it is computationally expensive, especially when dealing with the focal length and extrinsic camera parameters multiple times along with 3D point optimization [7]. As the size of image collections grows exponentially, the efficiency of optimization algorithms becomes increasingly critical in order to handle the computational burden. It is well known that BA often requires multiple iterations in incremental SfM [7], particularly as the amount of data increases, leading to a significant computational cost. Although a number of approaches have attempted to address the optimization efficiency problem [2,8], these approaches have not reduced the high dimensionality of the BA parameter space, and the optimization efficiency issues remain limiting for reconstruction. In comparison, the global SfM exhibits superior optimization efficiency by recovering all camera poses simultaneously. Nevertheless, this has resulted in challenges, including diminished robustness and reduced accuracy [9]. Consequently, the development of an optimization method that can enhance efficiency and reliability in unknown scenario is imperative.
A novel pose-only adjustment (PA) algorithm founded upon the pose-only imaging geometry has recently been proposed as a potential solution to the problem in this area [10,11]. In contradistinction to the BA, the PA attains a pose-only representation of 3D points by a pair of pose-only (PPO) constraints, which is an implicit representation of 3D points. It is noteworthy that the optimization process does not necessitate the incorporation of 3D points, as it is sufficient to optimize solely for the positional pose. This significantly reduces the dimensionality of the parameter space. As demonstrated in [12,13,14,15,16], SLAM experiments have further substantiated that PA optimization has the potential to replace BA optimization if efficiency improvements are needed. Interestingly, PA also mitigates the point-scattering phenomenon [10]. Unfortunately, these methods can only optimize the pose while ignoring the existence of a large number of unknown-focal-length cameras in real scenarios, and they are extremely dependent on the exact normalized coordinates, which are very sensitive to image noise and, thus, become inapplicable when the focal length is uncertain or varying. In real scenarios, the inevitable disturbances caused by motion inevitably affect the focal length of camera. The joint optimization of intrinsic and extrinsic parameters is, therefore, required to achieve the SfM task for real datasets.
To tackle the aforementioned challenges, this paper focuses on enhancing the efficiency of the SfM while improving its robustness to cameras of varying focal lengths from the perspective of the optimization process and without relying on any prior information. We explicitly derive a global PA for intrinsic and extrinsic joint optimization and propose a novel SfM system based on pose-only imaging geometry theory [10]. Specifically, we propose a base frame selection algorithm based on depth uncertainty, which takes into account the effect of camera focal length on depth estimation, improves the robustness of depth estimation, and facilitates the performance of the subsequent optimization process. Next, we adopt global PA instead of global BA to reduce the computational burden associated with the high-dimensional parameter space and enhance overall efficiency. Furthermore, we extend PA from extrinsic optimization only to joint intrinsic–extrinsic optimization, thus addressing the challenge of unknown camera parameters and improving robustness. Finally, we design a novel pose-only re-triangulation (PORT) module to continue trajectories of triangulation failure to improve reconstruction completeness. We evaluate our method using public 1DSfM [17], Cambridge Landmarks [18], and Mip-Nerf 360 [19] (MIP360), as well as self-collected multi-focal-length datasets. The results show that the system effectively improves efficiency while performing the high-precision reconstruction task, as illustrated in Figure 1. To further evaluate and demonstrate the capability of our SfM framework on challenging scenes, an unordered-lens SfM real-scene dataset for multiple focal lengths was collected.
The main contributions are threefold:
  • A depth uncertainty-based base frame selection method improves the robustness of depth estimation in scenes with different focal length combinations.
  • A new global pose-only SfM framework with combined focal length and extrinsic optimization improves reconstruction speed in complex scenes.
  • A novel PORT mechanism enhances reconstruction completeness by recovering failed triangulations from incomplete point tracks.
The paper is organized as follows. Section 2 introduces the related works on BA-based SfM and the PA optimization. In Section 3, the overall framework of the system is introduced, followed by an examination of the underlying frame selection principle based on depth uncertainty. The subsequent sections provide an explicit derivation process for PA-internal and external parametric optimization and a detailed delineation of the PORT process. Section 4 provides a detailed exposition of the performance of the proposed POMAP system with the optimization algorithm on both simulated and real datasets. In Section 5, the potential areas for enhancement of the current system are examined. The conclusions of the paper are summarized in Section 6.

2. Related Works

Optimization strategies have been shown to play a crucial role in SfM tasks [20,21]. In contrast, this paper focuses on the study of SfM optimization methods, combining traditional BA [7] and improved methods, while also exploring the more novel PA methods currently available.

2.1. Bundle Adjustment of SfM

Improving the speed of BA optimization must address three key issues [22]: high computational cost, limited efficiency, and susceptibility to input noise. The preconditioned conjugate gradient method was employed to accelerate BA solutions [23], while the sparse sparse BA (sSBA) algorithm proposed in [24] effectively addresses sparse BA problems through sparse linear subproblem formulations, substantially reducing computational complexity. Subsequent studies [2,4] developed adaptive Levenberg–Marquardt (LM) algorithms that dynamically select exact or truncated step sizes according to the problem scale, simultaneously reducing the memory footprint and computational costs during optimization. Additional enhancements include geometric estimation [25], batch or distributed processing [8,26], and re-triangulation (RT) [1,5], which collectively mitigate input noise effects, optimize memory efficiency, and improve system robustness. These approaches rely on iterative incremental SfM optimizations yet remain computationally intensive. In contrast, global SfM significantly reduces computational costs by recovering all camera poses simultaneously. However, it is more susceptible to input noise. To address the limitation of input, Moulon et al. [9] improved the robustness and precision of rotational averaging based on the trifocal tensor, while Zhuang et al. [27] focused on improving baseline insensitivity in translational averaging. GLOMAP [3] built on this by combining translational averaging and triangulation into a single global position step, improving efficiency while providing better-optimized initial values. Note that these works do not reduce the high dimensionality of the BA parameter space, and the issue of elevated memory utilization remains unresolved. Furthermore, hybrid SfM methodologies have been developed [28,29], including HSfM [28], which achieves an equilibrium between efficiency and robustness through global rotation estimation and incremental center estimation, although it relies on precise camera parameters.

2.2. Pose-Only Adjustment of SfM

The PPO constraint relationship decouples the camera pose from the three feature coordinates. This is actually an equivalent expression of the geometric constraints for dual-view imaging [11]. Building on this, Cai et al. [10] further extend the PPO problem to multiple views to obtain a depth-pose-only (DPO) constraint set for the 3D feature point. That is, the 3D feature point P W can be described in terms of the pose if the normalized coordinates are known. The DPO-based optimization problem is referred to as PA. Moreover, Cai et al. utilized PA in conjunction with the proposed linear global translation constraint (LiGT) [10] to verify the effectiveness of PA optimization in 3D reconstruction tasks. Building on the aforementioned theoretical framework, Ge et al. [12] pioneered the application of PA optimization in conjunction with preintegration merging theory within a VI-SLAM system. This approach enabled the attainment of an expeditious and lightweight SLAM, circumventing the challenges posed by the dimensional explosion and substantial memory consumption inherent to traditional BA. Furthermore, an in-depth analysis of the characteristics of PA optimization was conducted from the perspective of Hessian matrix sparsity, resulting in the generation of a more compact optimization matrix under a uniform optimization objective. Furthermore, a substantial body of research [13,14,15] has empirically demonstrated that PA can significantly improve optimization efficiency and is a potential alternative to BA in computationally intensive scenarios. However, it should be noted that the aforementioned PA optimization using solely extrinsic parameters is overly reliant on the precise normalized coordinates, and is vulnerable to focus noise and image noise. Moreover, it is incapable of adapting to scenarios where the focal length is unknown or in constant fluctuation.

3. Methods

3.1. Framework Overview

Our framework focuses on improving the efficiency and robustness of cameras with different focal lengths in the optimization process during SfM. As shown in Figure 2, the initialization phase involves feature extraction and matching, relative pose estimation, camera calibration, and global pose estimation. Feature extraction, matching, and pose estimation are implemented using well-established algorithms from existing frameworks [1], aiming at obtaining accurate matching relations, feature trajectories, initial camera parameters, and relative poses. For the problem of reconstruction failure due to mirroring, the doppelganger [30] can be used to filter out false matches. Subsequently, the global information with more accurate global rotation, global translation, and observing points is obtained through the rotation average [31] and global positioning [3]. In the optimization stage, base frames need to be selected before global PA to achieve an accurate implicit representation of 3D points. We propose the depth uncertainty for base frame selection. After global PA, PORT can be used further to improve the accuracy and completeness of the reconstruction. In addition, camera clustering [3] with larger error-point filtering can be combined to improve the robustness and accuracy of the system.

3.2. Depth Uncertainty-Based Base Frame Selection

The selection of base frames critically influences subsequent optimization. In real-world scenarios with multiple focal lengths, parallax-based selection becomes biased by ignoring focal parameters [32]. To mitigate this, depth uncertainty evaluation incorporating camera focal length enables robust base frame selection, enhancing optimization robustness in complex situations.
Let P j W be the current estimation of the 3D feature point corresponding to the pixel coordinate p b j or p f j in frame b or f, and the vector representing the center of light of camera b to the 3D point is l b j . R b f and t b f are the relative rotation and translation between frames b and f. I b = l b j / | | l b j | | , and α and β denote the angles formed by the two rays and the baseline t b f , shown in Figure 3; then,
l f j = l b j t b f ,
α = arccos ( I b · t b f | | t b f | | ) ,
β = arccos ( l f j · t b f | | l f j | | · | | t b f | | ) ,
where l f j denotes the vector from the optical center of frame f to P j W . Let f ^ be the camera focal length of frame f. Δ β , the angle spanning one pixel, can be added to β in order to compute parallax angle γ ; thus, by applying the law of sines, recover the norm of P j W :
Δ β = 2 tan 1 ( 1 2 f ^ ) ,
γ = π α β Δ β ,
| | l b j ^ | | = | | t b f | | sin ( β + Δ β ) sin γ ,
If the focal length in the x-axis f x differs from that in the y-axis f y , a virtual focal length f n e w can be introduced to simulate the angular change caused by moving 1 pixel along the polar line. The f n e w is calculated from the fundamental matrix F as
l e u = F p f j = [ a r , b r , c r ] T ,
θ = tan 1 ( a r b r ) ,
f n e w = cos 2 θ · f x 2 + s i n 2 θ · f y 2 ,
where l e u is the polar line in frame f, θ is the angle between l e u and the x-axis, and p f j is the j-th pixel point in frame f. a r , b r , and c r are solvable variables. Therefore, the depth uncertainty δ j 2 is computed as
δ j 2 = ( | | l b j ^ | | | | l b j | | ) 2 ,
The aforementioned equation can be resolved in order to ascertain the depth uncertainty. The set exhibiting the minimum depth uncertainty is then selected as the base frame for subsequent optimization.

3.3. Global Pose-Only Adjustment

Consider the j-th pixel coordinate in frame i as p i j , and P i j is the j-th normalized coordinate of the 3D feature point P j W in frame i. We use the DPO constraints to obtain the depth S b f of frame b, by which we can also get the pose-only description of the 3D feature point P j W :
P j W = H p o ( { R l , t l , P l } l = b , f ) = [ P f j ] × ( t f R f R b T t b ) [ P f j ] × R f R b T P b j P b j t b = λ b f ϕ b f P b j t b
where P l denotes the normalized coordinates from frames b or f, and [ P f j ] × is the antisymmetric matrix of P f j . R l , t l denotes the global rotation and translation from frames b or f. H p o ( · ) denotes the pose-only representation function of 3D points P j W . λ b f = [ P f j ] × ( t f R f R b T t b j ) , , and ϕ b f = [ P f j ] × R f R b T P b j . Then, the pixel point p i j can be estimated by transforming P b j from the normalization plane to frame i followed by reprojection. The progress shown in Figure 4 can be formulated as
p i j = H m ( H d ( H n ( H r ( H p o ( { R l , t l , P l } l = b , f ) , R i , t i ) ) , K i ) , M i )
where H m , H d , H n , and H r denotes the perspective projection functions, the distortion function, the normalization function, and the rigid transformation function. K i and M i , respectively, denote the distortion parameter vectors and the intrinsic parameter vectors including the focal length of the camera for the pending frame i.
The reprojection error can be also described as
e i j ( p i j , p ˜ i j ) = p i j p ˜ i j
where p ˜ i j denotes the measured values. Then, the optimization problem for m frames of individual images of n points can be described as
arg min R i , t i , K i , M i i = 1 . . . m i = 1 m j = 1 n e i j T e i j
In the above optimization problem, the optimization objectives are R i , t i , K i , and M i , which denote the rotation matrix, translation vectors, distortion parameters, and intrinsic parameters, respectively, excluding the 3D feature points. After obtaining the reprojection error, nonlinear optimization can be used to obtain the required parameters. In this paper, the Levenberg–Marquardt method is used to solve to obtain the update direction. Assuming J i = [ J R i , J t i , J K i , J M i ] is the Jacobian matrix of the frame i, where J R i , J t i , J K i , and J M i denote the Jacobian matrix of the rotation, the translation, the distortion parameters, and the intrinsic parameters, respectively; then, each part can be written in detail as follows:
J R i = e i j R i = e i j p i j p i j P i j D P i j D P i j P i j P i j C P i j C φ i , J t i = e i j t i = e i j p i j p i j P i j D P i j D P i j P i j P i j C P i j C t i , J K i = e i j K i = e i j p i j p i j P i j D P i j D K i , J M i = e i j M i = e i j p i j p i j M i .
where φ denotes the right rotation perturbation. p i j is the j-th pixel coordinate in the frame i. P i j D is the j-th distortion coordinate in the frame i. P i j is the j-th normalized image coordinate in the frame i, and P i j C is the j-th camera coordinate in the frame i. Next, a detailed derivation of the intrinsic Jacobian matrix for frame i is presented as an example. The derivation of the Jacobian matrix with respect to the extrinsics can be found in [12].
(1) Partial derivative of the normalized coordinates from the distortion coordinates: In reality, given the inherent imperfections of the camera component assembly and the possibility of lens positional fluctuations due to movement, it becomes imperative to incorporate radial and tangential distortions to accurately depict the aforementioned scenario. This paper aims to facilitate the calculation of the distortion transformation by providing a detailed description function H d ( · ) , and P i j D is the j-th distortion coordinate in the frame i which can be expressed in the following manner:
P i j = H n H r ( H p o ( { R l , t l , P l } l = b , f ) ) = λ b f R b i P b j + ϕ b f t b i ( 0 , 0 , 1 ) ( λ b f R b i P b j + ϕ b f t b i )
P i j D = H d ( P i j , K i ) = Δ P r a d + Δ P t a n + P i j
with
Δ P r a d = Δ x rad Δ y r a d = ( k 1 r 2 + k 2 r 4 + k 3 r 6 ) x i j y i j Δ P t a n = Δ x t a n Δ y t a n = 2 p 1 x i j y i j + p 2 ( r 2 + 2 x i j 2 ) 2 p 2 x i j y i j + p 1 ( r 2 + 2 y i j 2 ) r 2 = x i j 2 + y i j 2
where R b i = R i R b T and t b i = t i R b f t b . Δ P r a d denotes the error generated by radial distortion, Δ P t a n denotes the error generated by tangential distortion, and r is the distance from the normalized plane point to the center. K i = [ k 1 , k 2 , p 1 , p 2 , k 3 ] denotes the vector of distortion parameters; k 1 ,   k 2 , and k 3 is the radial distortion coefficient; and p 1 and p 2 denotes the tangential distortion coefficient. Therefore, the partial derivative of the distortion point to the normalized coordinates is
P i j D P i j = a 1 a 2 b 1 b 2
in which
a 1 = R d + 2 k 1 x 2 + 4 k 2 r 2 x 2 + 6 k 3 r 4 x 2 + 2 p 1 y + 6 p 2 x a 2 = 2 k 1 x y + 4 k 2 r 2 x y + 6 k 3 r 4 x y + 2 p 1 x + 2 p 2 y b 1 = 2 k 1 x y + 4 k 2 r 2 x y + 6 k 3 r 4 x y + 2 p 1 x + 2 p 2 y b 2 = R d + 2 k 1 y 2 + 4 k 2 r 2 y 2 + 6 k 3 r 4 y 2 + 2 p 2 x + 6 p 1 y R d = 1 + k 1 r 2 + k 2 r 4 + k 3 r 6
where ( x , y ) is shorthand for the normalized coordinates ( x i j , y i j ) . In this paper, only three parameters are used to represent the radial distortion parameters. This is because higher-order parameters often do not bring significant performance improvement in practical applications and increase the computational complexity, so three parameters are sufficient to describe the majority of cases.
Similarly, according to (17), the partial derivative of the distortion coordinates to the distortion parameters vectors is
P i j D K i = x r 2 x r 4 2 x y 3 x 2 + y 2 x r 6 y r 2 y r 4 3 y 2 + x 2 2 x y y r 6
(2) Partial derivative of the distortion coordinates from pixel coordinates: Assume that the camera model is a pinhole camera model, and its perspective projection and affine transformation is expressed as a function H m ( · ) . For a square detector unit, the skew coefficient can be defaulted to 0. Then, the intrinsic parameters of the camera include the focal length of the x-axis f x , the focal length of the y-axis f y , and the principal point ( u 0 , v 0 ) . Let the intrinsic vector M i = [ f x , f y , u 0 , v 0 ] ; then, the pixel point p i j can be described by the following equations:
p i j = H m ( P i j D , M i ) = f x x i j D + u 0 f y y i j D + v 0
where P i j D is ( x i j D , y i j D ) , and the partial derivative of the j-th pixel coordinates of frame i to the distortion coordinates can be described as
p i j P i j D = f x 0 0 f y
According to (20), the partial derivative of the pixel coordinates to the intrinsic reference vector can be obtained:
p i j M i = x i j D 0 1 0 0 y i j D 0 1
Thus, we can solve the Jacobian matrix of frame i and use the above method to solve the Jacobian matrix of frames b and f. Note that the optimization of the intrinsic parameters of frames b and f is slightly different from that of frame i, as it is hidden in the process of normalizing the points P b j and P f j to pixel coordinates.

3.4. Pose-Only Re-Triangulation

Re-triangulation is used to further improve the completeness of the reconstruction by continuing to track points that were not successfully triangulated earlier due to camera pose inaccuracies. Unlike traditional RT, which uses pre-BA re-triangulation and post-BA re-triangulation to minimize error accumulation, this paper uses the normalization error of PA as the error metric and adjusts it only for the extrinsic parameters; then, the optimization problem of PORT can be described as
arg min { R i , t i } i = 1 . . . m i m j n ( e i j ) T e i j
The process involves multiple iterations of triangulation, PA optimization, and filtering without changing the thresholds at the filtered points. PORT is a refinement module designed specifically for PA optimization. The traditional combination of bundle adjustment triangulation (BART) and PA may increase the optimization time due to the inconsistency of the optimization space, so PORT is used instead of BART in order to improve the consistency of the system, and secondly, the PORT module is richer in observation information due to the reintroduction of all the observation relationships, which is beneficial to the PA in choosing a more robust base frame. The advantage of using PORT is that it combines robustness while maintaining higher efficiency. The reason for optimizing only the extrinsic parameters is, firstly, to reduce the overfitting of the intrinsic parameters due to the error points. After the global initialization and global PA, the intrinsic parameters are already considered to be accurate. Refinement of the intrinsic parameters in the presence of a large number of uncertain 3D points may lead to the instability of the intrinsic parameters [33]. Secondly, we want the RT step to focus more on the accuracy and correctness of the 3D points, so only estimating the extrinsic parameters is better for visualizing the point characteristics to reduce point-scattering degrees. Finally, reducing the optimization parameters further improves the optimization efficiency. Since the RT process requires multiple iterations, fewer optimization parameters can significantly improve the optimization efficiency. After PORT, the reconstruction results can be obtained with richer information and a more complete structure.

4. Experiments

To demonstrate the optimization performance of our proposed POMAP, the experiments involved multiple datasets, including the publicly available Mip-NeRF360 dataset [19], Cambridge Landmarks dataset [18], and 1DSfM dataset [17], as well as the multi-focal-length (MFL) dataset captured and collected by our photography. These datasets cover a wide range of scenarios, ranging from scenes captured by a single camera in an ordered sequence to those captured by multiple cameras in an unordered manner, and from single object reconstruction to scene reconstruction. Specifically, we selected existing SOTA frameworks (COLMAP [1] and GLOMAP [3]) as benchmarks for comparison and performed extensive comparative analyses on the aforementioned datasets. Furthermore, we present ablations to study the behavior of the different components of our proposed system.
Metrics: We evaluate this in two downstream tasks, novel view synthesis and visual mapping and localization. For the Mip-NeRF360 dataset and our collection of multi-focal-length tourist data, both of which are small-scale object-centered reconstructions, this paper measures the quality of the reconstruction by rendering the image in the novel view synthesis that relies heavily on the accurate extrinsic and intrinsic parameters [34,35] where the quality of the rendered images measures the reconstruction. Specific image evaluation metrics include peak signal-to-noise ratio (PSNR), the structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The test method uses the popular 3DGS [34] algorithm. For evaluating the reconstruction quality of the Cambridge Landmarks dataset, the use of new viewpoint synthesis is very time consuming and computationally expensive, so we focus on the pose of the camera and directly compare each estimated pose with the real pose in the visual localization to judge the reconstruction effect. For the 1DSfM dataset, which is disorganized and features a large number of tourist photo datasets, we use the reprojection error and qualitative analysis, focusing on the comparison of solving the real and complex scenarios with optimization speed.
Implementation Details: For a fair comparison, the feature extraction and matching of COLMAP were consistently used as input to all methods in the novel view synthesis, while the HLOC tool [36] was used to ensure consistency in the visual localization. The running time of the experiments was recorded from the completion of feature matching, focusing on comparing the time taken for the optimization part. The hardware devices used in the experiments are a 13th Gen Intel(R) Core(TM) i9-13900HX CPU, and a DDR4 2200 MHz 64GB memory stick. The feature extraction and matching session was implemented using NVIDIA GeForce RTX 4060 Laptop GPU, and CUDA version 12.1. For both GLOMAP and COLMAP methods, the recommended default settings are used, and our method selects the g2o [37] and Ceres [38] optimization libraries in the optimization section and chooses a fixed parameters configuration based on the experimental conditions. Some of the experimental details are shown in the Table 1.

4.1. Simulation Test Performance

In order to evaluate the convergence and robustness of the algorithm, we conducted simulated data experiments and synthetic noise experiments.
Convergence test: The simulation data experiments use 200 pan-generated cameras with arbitrary viewing angles, each with its own independent internal datum, and we randomly add 10% or 20% noise to the focal length, principal points, and distortion parameters without adding noise to the extrinsic parameters and feature points. Then, we test the PA with intrinsic optimization only, with each noise experiment being performed five times and averaged. The results of the experiment are shown in Figure 5. The error in the PA optimization we use decreases quickly after the first iteration, but the non-linearity due to distortion does not converge until six or seven iterations later. It still retains a small error, but the number of iterations required is much smaller than for the BA.
Robustness test: Synthetic noise experiments are used on the MLF dataset to test the robustness of the proposed optimization method in the presence of image noise. We synthesize perfect-image observations by projecting COLMAP triangulations onto the real camera, and then, for each observation, we add random Gaussian noise to the reprojection. The results are shown in Table 2. For perfect image observations with 0px noise, we reliably converge to the true value, underlining the effectiveness of the optimization. As the noise level increases to extreme values, the AUC (less than 5 per cent error in camera parameters) score gradually decreases, indicating the high robustness of our proposed optimization method.

4.2. Experiments on Base Frame Selection

To validate the effectiveness of the depth uncertainty-based base frame selection method, we performed a comparison between the simulated dataset and the real dataset, examining the baseline method from [12] that includes maximum parallax angle (PA-A) and depth averaging options (PA-D). The results are shown in Table 3. It can be seen that our proposed base frame selection method has stronger anti-interference ability and robustness than other methods. When the image noise reaches 10 pixels, it still maintains high accuracy. The other two methods have a rapidly increasing error due to noise. We analyze that this is because cameras with different focal lengths have various sensitivities to noise, and that cameras with larger focal lengths occupy a smaller world scale per pixel when photographing the same object, and are, therefore, less affected by random image noise. Our method can select a group of views with the least interference from noise, thereby reducing the uncertainty of depth estimation to some extent. Notably, our method still maintains high accuracy in the real dataset. This indicates that our proposed base frame selection algorithm has strong robustness and can provide better information for subsequent optimization.

4.3. Novel View Synthesis

Mip-NeRF360 comprises seven object-centric scenes with high-resolution images captured sequentially by a single camera. Since these data do not provide accurate real parameters of the camera, we also evaluate the reconstruction results in terms of the quality of the rendered images in the novel view synthesis. As shown in Table 4, our method is superior in PNSR, SSIM, and LPIPS metrics compared to the two existing SOTA methods. In reconstruction speed, our proposed method is almost 1.5 times faster than GLOMAP and 3 times faster than COLMAP. Overall, in the object-centered reconstruction task, our proposed method is basically in line with the SOTA in terms of accuracy with a slight improvement, while there is a very significant speed improvement.
The multi-focal-lengths dataset comprises high-resolution data we collected from multiple unordered shooting scenes, each shot with at least four cameras of different focal lengths. This dataset is used mainly to test the robustness of the optimization algorithm, including indoor and outdoor situations. The results tested on the new perspective synthesis task are shown in Table 5 and Figure 6. Our proposed method is faster in achieving the same accuracy. Compared to the best available result, we observe an improvement of 0.36 dB in PSNR, which indicates that our proposed method can cope with the complexity of multi-focal recordings and achieve a good joint optimization of intrinsic and extrinsic parameters. Moreover, the reconstruction speed of the whole system is nearly 1.5 times that of GLOMAP and more than 3 times that of COLMAP, since no optimization of 3D points is required.

4.4. Visual Mapping and Localization

The Cambridge Landmarks dataset is a visual location dataset taken outdoors around Cambridge University. A total of 26 gigabytes of data were selected from five complex scenarios for testing. Since these data have a real pose, we used the percentage of the number of cameras at three different error thresholds as well as the average distance error and angular error to assess the reconstruction quality. The results are summarized in Table 6 and show that our method achieves better results in estimating poses. With the position error less than 50 cm and angle error less than 5°, the number of correct camera poses estimated by our method is significantly higher than that of other methods, reaching 73.73%. In terms of average positional accuracy, our proposed method improves the accuracy by 42.8% compared to COLMAP, while it improves the tie angle accuracy by 28.6%, which is the highest camera position estimation accuracy among the tested methods. From the above experiments, it can be seen that our proposed method can effectively improve the accuracy of visual localization tasks in outdoor areas and remain robust in complex situations. Figure 7 shows a comparison of trajectory predictions for St Mary’s Church.
The 1DSfM dataset is a widely adopted benchmark for evaluating SfM algorithms, which are characterized by collections of unordered tourist photographic images with varying focal lengths and complex real-world scene variations. This dataset has no truth values, and we analyze it qualitatively in terms of the reconstruction effect, like with Figure 8, focusing on the speed of each process in the reconstruction process, and the results are shown in Table 7. We compare in detail the computation time of each step of GLOMAP vs. POMAP. COLMAP is an incremental SfM, and, thus, it is not appropriate to compare it with the global is SfM on a case-by-case basis. Instead, we examine the overall time it takes. In Table 7, we start calculating the time after the matching phase, which includes the initialization, optimization, and RT phases. Since we use the same initialization method in [3], we focus on comparing the initialization and RT phases. From the results, it can be seen that POMAP is 24.16 times faster than COLMAP and 4.17 times faster than GLOMAP. In this particular instance, the global BA optimization accounts for a minimum of 50% of the overall process time, whereas POMAP with global PA is faster than GLOMAP with global BA, which is 55 times faster than BA optimization. This demonstrates that the proposed method can significantly enhance the optimization speed.

4.5. Ablation Study

In order to validate the effectiveness of the proposed system, an experimental study was conducted on the MFL dataset. This study involved the comparison of the quality of novel view synthesis for 11 different configurations under the same initialization conditions. These conditions included no optimization and an RT module (No optimization), the inclusion of PA with intrinsic optimization only (PA (intrinsic)), the inclusion of PA with extrinsic optimization only (PA (pose)), the optimization of complete PA (PA) and BA (BA), PA with intrinsic optimization only, or PA with extrinsic optimization only in combination with PORT PA optimization combined with PORT/BART, and BA optimization combined with PORT/BART. The results are shown in Table 8. Overall, the combination of PA intrinsic and extrinsic optimization with PORT is most effective when the camera parameters are unknown. From an RT perspective, PORT outperforms BART, achieving a 0.76 dB improvement in PSNR, as well as 3.2% and 3.5% improvements in SSIM and LPIPS, respectively. In terms of optimization, when the focal length is unknown, optimizing the intrinsic is essential. The PNSR obtained from PA with intrinsic optimization is 1.09 dB higher than that from PA with extrinsic optimization alone. At the same time, PA with intrinsic optimization alone is effective and improves the PNSR by 0.24 dB compared to no optimization. Notably, we find that using PA/BA in combination with BART/PORT increases computation time. We speculate this to be due to the variability in the optimization space between PA and BA.

5. Discussion

Our proposed POMAP achieves relatively good results in both novel view synthesis and visual localization tasks, with an accuracy almost comparable to that of current methods [1,3] as well as speed improvement, especially in reconstruction tasks facing large-scale datasets. It was evident that the proposed PA with joint intrinsic and extrinsic parameters optimization was capable of significantly reducing the time required for the optimization step. However, as the size of the input image increased, the time required for the feature extraction and matching part of the process became non-negligible. This may have a detrimental effect on the reconstruction speed gain brought about by the improvement of the optimization efficiency. Therefore, it would be advantageous to consider using the faster feature extraction and matching method to further enhance the reconstruction efficiency, for example, using methods such as Xfeat [39] with LightGlue [40] or Efficient LoFTR [41] with semi-dense feature matching.

6. Conclusions

This paper proposes a novel global SfM based on PA with joint intrinsic and extrinsic parameters optimization, addressing diverse scenarios while enhancing reconstruction speed and accuracy. Our global PA strategy jointly refines camera parameters, effectively handling challenges such as unknown-focal-length and multi-focal configurations. By avoiding 3D point optimization limitations, it significantly improves computational efficiency and system robustness. Additionally, our PORT mechanism enhances reconstruction quality and processing speed. Extensive experiments have shown that our framework is faster and achieves an accuracy similar to that of existing SfM baselines. For example, our PA optimization method is 55 times faster than the BA optimization method on the 1DSfM dataset, and the proposed system is 4 times faster than GLOMAP and 24 times faster than COLMAP. We believe this work significantly advances reconstruction under cameras with unknown focal lengths, benefiting downstream tasks such as novel view synthesis and visual localization.

Author Contributions

Conceptualization, X.T., Y.G. and Z.T.; Methodology, X.T. and Y.G.; Investigation, X.T. and Z.T.; Resources, M.L., X.C. and D.H.; Supervision, D.H.; Writing—original draft, X.T.; Writing—review and editing, Z.T. and X.C. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by The Science and Technology Innovation Program of Hunan Province (Grant No. 2024QK2006).

Data Availability Statement

The original contributions presented in the study are included in the article, and further inquiries can be directed to the corresponding author.

Acknowledgments

The authors of this paper are very grateful for the help provided by the following experts and personnel: Zhang Jinpu provided valuable comments on the structure of the paper for touching up and plotting, and Lian Xiangkai provided valuable information on the mathematical theory and plotting of the paper, which is hereby acknowledged.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Schönberger, J.L.; Frahm, J.M. Structure-from-Motion Revisited. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 4104–4113. [Google Scholar]
  2. Agarwal, S.; Snavely, N.; Simon, I.; Seitz, S.M.; Szeliski, R. Building Rome in a day. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision (ICCV), Kyoto, Japan, 29 September–2 October 2009; pp. 72–79. [Google Scholar]
  3. Pan, L.; Bar’ath, D.B.; Pollefeys, M.; Schönberger, J.L. Global Structure-from-Motion Revisited. In Proceedings of the European Conference on Computer Vision (ECCV), Milan, Italy, 29 September–4 October 2024. [Google Scholar]
  4. Agarwal, S.; Snavely, N.; Seitz, S.M.; Szeliski, R. Bundle Adjustment in the Large. In Proceedings of the European Conference on Computer Vision (ECCV), Heraklion, Greece, 5–11 September 2010. [Google Scholar]
  5. Wu, C. Towards Linear-Time Incremental Structure from Motion. In Proceedings of the International Conference on 3D Vision (3DV), Seattle, WA, USA, 29 June–1 July 2013; pp. 127–134. [Google Scholar]
  6. Zhu, S.; Zhang, R.; Zhou, L.; Shen, T.; Fang, T.; Tan, P.; Quan, L. Very Large-Scale Global SfM by Distributed Motion Averaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 4568–4577. [Google Scholar]
  7. Triggs, B.; Mclauchlan, P.F.; Hartley, R.I.; Fitzgibbon, A.W. Bundle adjustment—A modern synthesis. In Proceedings of the International Workshop on Vision Algorithms, Corfu, Greece, 21–22 September 2000. [Google Scholar]
  8. Cui, H.; Shen, S.; Gao, X.; Hu, Z. Batched Incremental Structure-from-Motion. In Proceedings of the International Conference on 3D Vision (3DV), Qingdao, China, 10–12 October 2017; pp. 205–214. [Google Scholar]
  9. Moulon, P.; Monasse, P.; Marlet, R. Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, Australia, 1–8 December 2013; pp. 3248–3255. [Google Scholar]
  10. Cai, Q.; Zhang, L.; Wu, Y.; Yu, W.; Hu, D. A Pose-Only Solution to Visual Reconstruction and Navigation. IEEE Trans. Pattern Analalysis Mach. Intell. (TPAMI) 2023, 45, 73–86. [Google Scholar] [CrossRef] [PubMed]
  11. Cai, Q.; Wu, Y.; Zhang, L.; Zhang, P. Equivalent Constraints for Two-View Geometry: Pose Solution/Pure Rotation Identification and 3D Reconstruction. Int. J. Comput. Vis. (IJCV) 2018, 127, 163–180. [Google Scholar] [CrossRef]
  12. Ge, Y.; Zhang, L.; Wu, Y.; Hu, D. PIPO-SLAM: Lightweight Visual-Inertial SLAM With Preintegration Merging Theory and Pose-Only Descriptions of Multiple View Geometry. IEEE Trans. Robot. (TRO) 2024, 40, 2046–2059. [Google Scholar] [CrossRef]
  13. Du, X.; Ji, C.; Zhang, L.; Luo, X.; Zhang, H.; Wang, M.; Wu, W.; Mao, J. SP-VIO: Robust and Efficient Filter-Based Visual Inertial Odometry with State Transformation Model and Pose-Only Visual Description. arXiv 2024, arXiv:2411.07551. [Google Scholar]
  14. Du, X.; Zhang, L.; Liu, R.; Wang, M.; Wu, W.; Mao, J. PO-MSCKF: An Efficient Visual-Inertial Odometry by Reconstructing the Multi-State Constrained Kalman Filter with the Pose-only Theory. arXiv 2024, arXiv:2407.01888. [Google Scholar]
  15. Tang, H.; Zhang, T.; Wang, L.; Wang, G.; Niu, X. PO-VINS: An Efficient and Robust Pose-Only Visual-Inertial State Estimator With LiDAR Enhancement. arXiv 2024, arXiv:2305.12644. [Google Scholar]
  16. Wang, L.; Tang, H.; Zhang, T.; Wang, Y.; Zhang, Q.; Niu, X. PO-KF: A Pose-Only Representation-Based Kalman Filter for Visual Inertial Odometry. IEEE Internet Things J. 2025, 12, 14856–14875. [Google Scholar] [CrossRef]
  17. Wilson, K.; Snavely, N. Robust Global Translations with 1DSfM. In Proceedings of the European Conference on Computer Vision (ECCV), Zurich, Switzerland, 6–12 September 2014. [Google Scholar]
  18. Kendall, A.; Grimes, M.; Cipolla, R. PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 2938–2946. [Google Scholar]
  19. Barron, J.T.; Mildenhall, B.; Verbin, D.; Srinivasan, P.P.; Hedman, P. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 5460–5469. [Google Scholar]
  20. Özyesil, O.; Voroninski, V.; Basri, R.; Singer, A. A survey of structure from motion. Acta Numer. 2017, 26, 305–364. [Google Scholar] [CrossRef]
  21. Jiang, S.; Jiang, C.; Jiang, W. Efficient structure from motion for large-scale UAV images: A review and a comparison of SfM tools. ISPRS J. Photogramm. Remote Sens. (JPRS) 2020, 167, 230–251. [Google Scholar] [CrossRef]
  22. Zhang, J.; Boutin, M.; Aliaga, D.G. Robust Bundle Adjustment for Structure from Motion. In Proceedings of the IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, 25–28 September 2006; pp. 2185–2188. [Google Scholar]
  23. Byröd, M.; Åström, K. Bundle Adjustment using Conjugate Gradients with Multiscale Preconditioning. In Proceedings of the British Machine Vision Conference (BMVC), London, UK, 7–10 September 2009. [Google Scholar]
  24. Konolige, K. Sparse Sparse Bundle Adjustment. In Proceedings of the British Machine Vision Conference (BMVC), Wales, UK, 30 August–2 September 2010. [Google Scholar]
  25. Sweeney, C.; Sattler, T.; Höllerer, T.; Turk, M.; Pollefeys, M. Optimizing the Viewing Graph for Structure-from-Motion. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; pp. 801–809. [Google Scholar]
  26. Zhang, R.; Zhu, S.; Fang, T.; Quan, L. Distributed Very Large Scale Bundle Adjustment by Global Camera Consensus. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 29–38. [Google Scholar]
  27. Zhuang, B.; Cheong, L.F.; Lee, G.H. Baseline Desensitizing in Translation Averaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–22 June 2018; pp. 4539–4547. [Google Scholar]
  28. Cui, H.; Gao, X.; Shen, S.; Hu, Z. HSfM: Hybrid Structure-from-Motion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 2393–2402. [Google Scholar]
  29. Chen, Y.; Yu, Z.; Song, S.; Yu, T.; Li, J.; Lee, G.H. AdaSfM: From Coarse Global to Fine Incremental Adaptive Structure from Motion. In Proceedings of the IEEE International Conference on Robotics & Automation (ICRA), London, UK, 29 May–2 June 2023; pp. 2054–2061. [Google Scholar]
  30. Cai, R.; Tung, J.; Wang, Q.; Averbuch-Elor, H.; Hariharan, B.; Snavely, N. Doppelgangers: Learning to Disambiguate Images of Similar Structures. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 2–3 October 2023; pp. 34–44. [Google Scholar]
  31. Chatterjee, A.; Govindu, V.M. Efficient and Robust Large-Scale Rotation Averaging. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, Australia, 1–8 December 2013; pp. 521–528. [Google Scholar]
  32. Pizzoli, M.; Forster, C.; Scaramuzza, D. REMODE: Probabilistic, monocular dense reconstruction in real time. In Proceedings of the IEEE International Conference on Robotics & Automation (ICRA), Hong Kong, China, 31 May–7 June 2014; pp. 2609–2616. [Google Scholar]
  33. Olsson, C.; Eriksson, A.; Hartley, R. Outlier removal using duality. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, 13–18 June 2010; pp. 1450–1457. [Google Scholar]
  34. Kerbl, B.; Kopanas, G.; Leimkühler, T.; Drettakis, G. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph. 2023, 42. [Google Scholar] [CrossRef]
  35. Zwicker, M.; Pfister, H.; van Baar, J.; Gross, M. EWA splatting. IEEE Trans. Vis. Comput. Graph. 2002, 8, 223–238. [Google Scholar] [CrossRef]
  36. Sarlin, P.E.; Cadena, C.; Siegwart, R.; Dymczyk, M. From Coarse to Fine: Robust Hierarchical Localization at Large Scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 15–20 June 2019. [Google Scholar]
  37. Kümmerle, R.; Grisetti, G.; Strasdat, H.; Konolige, K.; Burgard, W. G2o: A general framework for graph optimization. In Proceedings of the IEEE International Conference on Robotics & Automation (ICRA), Shanghai, China, 9–13 May 2011; pp. 3607–3613. [Google Scholar]
  38. Agarwal, S.; Mierle, K.; The Ceres Solver Team. Ceres Solver, version 2.2; Ceres: Boston, MA, USA, 2023; Apache License 2.0.
  39. Potje, G.; Cadar, F.; Araujo, A.; Martins, R.; Nascimento, E.R. XFeat: Accelerated Features for Lightweight Image Matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–22 June 2024; pp. 2682–2691. [Google Scholar]
  40. Lindenberger, P.; Sarlin, P.E.; Pollefeys, M. LightGlue: Local Feature Matching at Light Speed. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France, 1–6 October 2023; pp. 17581–17592. [Google Scholar]
  41. Wang, Y.; He, X.; Peng, S.; Tan, D.; Zhou, X. Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 16–22 June 2024; pp. 21666–21675. [Google Scholar]
Figure 1. The proposed POMAP achieves satisfactory accuracy by sparse reconstruction for pose estimation on 1DSfM dataset of tourist photos with camera sensors with different focal lengths.
Figure 1. The proposed POMAP achieves satisfactory accuracy by sparse reconstruction for pose estimation on 1DSfM dataset of tourist photos with camera sensors with different focal lengths.
Remotesensing 17 02247 g001
Figure 2. Pipeline of proposed POMAP system. Our work focuses on the optimization part, proposing a base frame selection method based on depth uncertainty, a global pose-only adjustment based on intrinsic and extrinsic, and a subsequent PORT module to improve the reconstruction integrity of the system.
Figure 2. Pipeline of proposed POMAP system. Our work focuses on the optimization part, proposing a base frame selection method based on depth uncertainty, a global pose-only adjustment based on intrinsic and extrinsic, and a subsequent PORT module to improve the reconstruction integrity of the system.
Remotesensing 17 02247 g002
Figure 3. Calculation depth uncertainty. O b and O f denote the camera optical centers corresponding to frames b and f, respectively. u, u is the projection of P j W in frames b and f, and e, e is the epipoles. The mode of l b j describes the depth information between the optical center of the camera at the currently predicted 3D point P j W . P j W ^ describes the 3D points that may be estimated after possessing a pixel of image noise, and the Euclidean distance between him and P j W can describe the possibility of depth δ j 2 .
Figure 3. Calculation depth uncertainty. O b and O f denote the camera optical centers corresponding to frames b and f, respectively. u, u is the projection of P j W in frames b and f, and e, e is the epipoles. The mode of l b j describes the depth information between the optical center of the camera at the currently predicted 3D point P j W . P j W ^ describes the 3D points that may be estimated after possessing a pixel of image noise, and the Euclidean distance between him and P j W can describe the possibility of depth δ j 2 .
Remotesensing 17 02247 g003
Figure 4. A simple demonstration of projection strategies. The depth S b f can be obtained from the depth constraint relationship between frames b and f. Then, P j W can be directly represented by the pose between frames b and f. This is then projected into camera coordinates of frame i and onto its pixel plane via affine and perspective projection. For clarity, the image plane is shown after the optical centre, although the actual calculations use the one before.
Figure 4. A simple demonstration of projection strategies. The depth S b f can be obtained from the depth constraint relationship between frames b and f. Then, P j W can be directly represented by the pose between frames b and f. This is then projected into camera coordinates of frame i and onto its pixel plane via affine and perspective projection. For clarity, the image plane is shown after the optical centre, although the actual calculations use the one before.
Remotesensing 17 02247 g004
Figure 5. Experiments on the simulation of different intrinsic noise.
Figure 5. Experiments on the simulation of different intrinsic noise.
Remotesensing 17 02247 g005
Figure 6. Qualitative results for novel view synthesis with 3DGS [34]. Visual comparisons across multiple scenes demonstrate that our method achieves superior camera parameters estimation. For example, the surface reconstruction of the Dashuifa site is closer to the real scene, and distant objects in Stump are clearly reconstructed, rather than being lost due to inaccurate parameters estimation as in GLOMAP and COLMAP.
Figure 6. Qualitative results for novel view synthesis with 3DGS [34]. Visual comparisons across multiple scenes demonstrate that our method achieves superior camera parameters estimation. For example, the surface reconstruction of the Dashuifa site is closer to the real scene, and distant objects in Stump are clearly reconstructed, rather than being lost due to inaccurate parameters estimation as in GLOMAP and COLMAP.
Remotesensing 17 02247 g006
Figure 7. Schematic representation of some of the results in the visual relocalization task. Our proposed method achieves better results in position estimation, with red representing the estimated trajectory and blue representing the true value.
Figure 7. Schematic representation of some of the results in the visual relocalization task. Our proposed method achieves better results in position estimation, with red representing the estimated trajectory and blue representing the true value.
Remotesensing 17 02247 g007
Figure 8. Example reconstructions from the proposed POMAP on 1DSfM datasets.
Figure 8. Example reconstructions from the proposed POMAP on 1DSfM datasets.
Remotesensing 17 02247 g008
Table 1. Optimization methods and filtering criteria.The italics indicate whether an optimisation or filtering method was used. The filtering method indicates that the method is used to remove some bad matching pairs.
Table 1. Optimization methods and filtering criteria.The italics indicate whether an optimisation or filtering method was used. The filtering method indicates that the method is used to remove some bad matching pairs.
MethodParametersMaximum IterationStopping Criteria
Optimization:
Global PA τ = 10 5 100 ρ = 0 or λ = NaN
PORT5 Δ Obs . points < 0.05 %
PA in PORT τ = 10 5 50 ρ = 0 or λ = NaN
Filtering Criteria:
Depth d 0.0
Angular θ 1 . 5
Reprojection ϵ 4.0 px
Table 2. Ablation on robustness of POMAP to different noise levels of points (%).
Table 2. Ablation on robustness of POMAP to different noise levels of points (%).
SceneNoise Level (Pixels)
0 1 2 4 8 16 32 64
Table100.0099.4999.4998.7597.8896.6094.4487.12
Book100.0099.4999.4998.5496.2994.2774.3644.44
Cap100.0089.3785.1492.3168.8658.0850.5114.13
Luffy100.0099.4999.3898.9097.5995.0488.8068.69
Teapot100.0099.4999.4999.3998.4396.6994.7874.44
Flower100.0099.4999.4399.1898.0496.5394.0884.95
Pavilion100.0087.3179.5088.5484.0177.3467.1045.68
Dashufa100.0083.3371.4664.4758.8761.5134.219.27
Average100.0094.6891.6792.5187.5075.7674.7953.59
Table 3. Performance comparison on simulated dataset (degrees/meter) and multi-focal-lengths dataset. As shown in the following tables, bolding indicates the best result for the indicator.
Table 3. Performance comparison on simulated dataset (degrees/meter) and multi-focal-lengths dataset. As shown in the following tables, bolding indicates the best result for the indicator.
MethodNoise Level (Pixel)MFL Dataset
0.0 1.5 3.0 5.0 10.0 SSIM PSNR LPIPS
PA-A0.33/0.010.97/0.054.97/0.109.11/0.1718.78/0.260.95634.780.111
PA-D0.36/0.021.69/0.100.91/0.111.75/0.145.26/0.170.94934.330.118
Ours0.25/0.020.36/0.030.35/0.030.68/0.050.69/0.030.95835.840.110
Table 4. Rendering performance on Mip-NeRF360. * indicates data from the authors of the original paper [3].
Table 4. Rendering performance on Mip-NeRF360. * indicates data from the authors of the original paper [3].
 PSNR (dB) ↑SSIM ↑LPIPS ↓Time (s) ↓
 COLMAP *GLOMAP *OursCOLMAP *GLOMAP *OursCOLMAPGLOMAPOursCOLMAPGLOMAPOurs
bicycle23.1523.1323.120.53200.53100.71680.36770.38390.3068127.183.256.8
bonsai29.6630.3630.480.89600.90400.93380.21930.21070.2004735.3341.2293.5
counter26.8126.7227.990.83700.83500.89720.22920.22880.2112736.9164.285.1
garden24.9824.9727.340.65300.65500.84440.18600.18900.1589289.2241.5162.3
kitchen29.2329.3529.840.85100.85500.92100.15030.15030.1388595.6467.3336.9
room29.1429.4131.070.87100.87600.91700.23560.23850.2170258.3122.076.4
stump23.9823.8125.310.60200.59500.75400.32860.32940.390157.740.034.6
Average26.7126.8227.880.74890.75010.85490.24520.24720.2319400.0208.5149.4
Table 5. Rendering performance on multi-focal-length dataset.
Table 5. Rendering performance on multi-focal-length dataset.
 PSNR (dB) ↑SSIM ↑LPIPS ↓Time (s) ↓
 COLMAPGLOMAPOursCOLMAPGLOMAPOursCOLMAPGLOMAPOursCOLMAPGLOMAPOurs
Table32.8532.9133.000.96990.97080.97060.05310.05200.05224.621.141.08
Book37.5137.0837.960.97110.97010.97180.11330.11480.111212.423.723.39
Cap42.6242.3443.970.99080.99020.99160.02060.02270.01764.446.183.86
Luffy37.4838.4238.800.97130.97420.97510.10220.09720.09339.242.931.93
Teapot   36.3436.5136.530.97230.97300.97270.07570.07480.07539.903.722.29
Flower39.4339.1839.430.96190.96160.96310.19200.19470.18837.562.931.88
Pavilion26.8125.9125.620.93190.92660.91900.08430.08450.090676.0847.1123.75
Dashufa32.8433.2533.510.95910.96090.96190.07890.07820.0773116.7032.3431.01
Average35.7435.7036.100.96600.96590.96570.09000.08990.088230.1212.518.65
Table 6. Comprehensive evaluation on Cambridge Landmarks.
Table 6. Comprehensive evaluation on Cambridge Landmarks.
SceneMethodLocalization (%)Pose Error
5 cm 5° 25 cm 2° 50 cm 5° Position (m) Angle (°) Time (s)
Kings CollegeGLOMAP3.7728.2848.690.5310.689394.9
COLMAP4.5131.7250.980.4880.40012,178.3
Ours4.8438.5272.380.3200.721263.5
Old HospitalGLOMAP0.344.8043.350.5611.335358.6
COLMAP2.5738.1068.270.3460.7326421.4
Ours0.226.9346.370.5231.288246.3
Shop FacadeGLOMAP27.2792.6498.700.0800.22853.50
COLMAP27.7195.2498.700.0820.254229.1
Ours28.1296.8798.880.0760.22342.15
St. Marys ChurchGLOMAP5.9949.9080.970.2510.268371.2
COLMAP3.6316.4821.251.2261.84615,049.9
Ours7.3351.4583.860.2410.259305.2
Great CourtGLOMAP0.7823.4350.980.4930.212506.6
COLMAP1.8320.5649.020.5100.54113,989.6
Ours1.8335.9067.170.3570.203486.5
AverageGLOMAP7.6339.8164.540.3830.546337.0
COLMAP8.0540.4257.640.5300.7559573.7
Ours8.4745.9373.730.3030.539268.7
Table 7. 3D Reconstruction performance comparison on 1DSFM dataset (time in seconds).
Table 7. 3D Reconstruction performance comparison on 1DSFM dataset (time in seconds).
SceneImagesCOLMAPGLOMAPOurs
Total Total BA BART Total PA PART
Alamo291512,935.71555.71070.8174.5224.034.1134.6
Ellis Island25878820.9722.9479.7119.2138.84.4117.6
Gendarmenmarkt146313,771.61577.2663.4719.2354.79.7288.3
Montreal Notre Dame229812,679.1584.1411.765.796.515.842.2
Notre Dame143035,619.41856.5926.9769.0661.075.6376.3
NYC Library25509215.1576.3419.791.1209.36.878.3
Piazza del Popolo237711,448.02780.81951.8179.1222.113.7115.1
Piccadilly734737,352.16961.25139.01559.12016.4194.61484.8
Roman Forum236512,242.41404.5666.2598.8453.327.8271.6
Tower of London15764709.0645.2453.446.1388.513.5293.3
Trafalgar Square15,97887,083.225,377.421,175.62369.86180.0182.75434.8
Union Square596112,205.61231.7850.6220.1108.86.172.3
Vienna Cathedral628824,791.92024.51232.9261.0414.132.4214.3
Yorkminster345215,032.94162.82601.0833.1861.270.0361.3
Average418521279.13675.82717.3571.8880.649.1663.2
Table 8. Ablation studies on multi-focal-lengths dataset.
Table 8. Ablation studies on multi-focal-lengths dataset.
MethodPSNR (dB) ↑SSIM ↑LPIPS ↓Time (s) ↓
No optimization33.150.93990.1441
PA (intrinsic)33.390.93810.16153.17
PA (pose)33.690.94040.14034.69
PA34.780.95020.12146.74
BA34.380.94200.122010.29
BA + BART34.750.94970.123212.51
PA + BART34.320.94880.124511.46
BA + PORT34.360.94870.122716.07
PA (intrinsic) + PORT34.540.94690.128912.27
PA (pose) + PORT34.380.94960.12508.75
POMAP (PA + PORT)35.080.95190.12018.65
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Tian, X.; Ge, Y.; Tan, Z.; Chen, X.; Li, M.; Hu, D. Fast Intrinsic–Extrinsic Calibration for Pose-Only Structure-from-Motion. Remote Sens. 2025, 17, 2247. https://doi.org/10.3390/rs17132247

AMA Style

Tian X, Ge Y, Tan Z, Chen X, Li M, Hu D. Fast Intrinsic–Extrinsic Calibration for Pose-Only Structure-from-Motion. Remote Sensing. 2025; 17(13):2247. https://doi.org/10.3390/rs17132247

Chicago/Turabian Style

Tian, Xiaoyang, Yangbing Ge, Zhen Tan, Xieyuanli Chen, Ming Li, and Dewen Hu. 2025. "Fast Intrinsic–Extrinsic Calibration for Pose-Only Structure-from-Motion" Remote Sensing 17, no. 13: 2247. https://doi.org/10.3390/rs17132247

APA Style

Tian, X., Ge, Y., Tan, Z., Chen, X., Li, M., & Hu, D. (2025). Fast Intrinsic–Extrinsic Calibration for Pose-Only Structure-from-Motion. Remote Sensing, 17(13), 2247. https://doi.org/10.3390/rs17132247

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop