RDIC-MSCKF: Risk–Direction-Decoupled and Innovation-Calibrated MSCKF for Stereo Visual-Inertial Odometry

Abstract

Filtering-based stereo visual-inertial odometry often assigns fixed or uniformly scaled covariance to tracks accepted by the front-end, although accepted observations can differ in terms of tracking reliability, local directional identifiability, and agreement with the batch-level innovation model. This issue is important for UAV-based multisensor inspection platforms, where pose estimates support autonomous flight, measurement registration, repeatable survey lines, and multisensor data fusion. This paper presents RDIC-MSCKF, a Risk–Direction-Decoupled and Innovation-Calibrated MSCKF, where innovation calibration denotes bounded empirical scaling within the visual update. RDIC-MSCKF maps robust tracking diagnostics to a bounded standard-deviation multiplier and uses a robust local photometric information matrix to add penalty-only anisotropic covariance along weak image directions. The resulting observation covariance is preserved during MSCKF landmark elimination through full projected-covariance whitening. In parallel with feature-block innovation gating, bounded minimum measurement-noise inflation is estimated from the pre-gate innovation population and applied through a Kalman-equivalent modal update. On ten evaluated EuRoC MAV sequences, RDIC-MSCKF obtains lower ATE RMSE than the S-MSCKF baseline on nine sequences; averaged over five runs per sequence, the mean RMSE decreases from 0.1869 m to 0.1236 m, corresponding to a 33.8% reduction, and the sequence-mean P90 error decreases by 32.0%. Runtime profiling on five representative EuRoC sequences gives a 26.32 ms mean and 37.33 ms P95 per-frame processing time for RDIC-MSCKF, with 0.01% of profiled frames above the 50 ms reference. Outdoor UAV flights with RTK reference trajectories further demonstrate lower Sim(2)-aligned horizontal RMSE than S-MSCKF on all three evaluated flights.

1. Introduction

Visual-inertial odometry (VIO) fuses visual and inertial measurements to provide continuous six-degree-of-freedom state estimation when GNSS is unavailable or degraded, and has become a key localization technique for UAVs, mobile robots, and autonomous inspection platforms. UAV-based multisensor inspection platforms further require reliable onboard pose estimates for autonomous flight, multiscale measurement registration, repeatable survey trajectories, and multisensor data fusion. Related international studies on UAV surveying, UAV monitoring, and robot–environment interaction illustrate the broader demand for dependable perception and motion estimation in machine systems [1,2,3]. Existing VIO methods can be broadly divided into optimization-based and filtering-based approaches. Optimization-based methods typically employ sliding-window optimization, fixed-lag smoothing, factor-graph formulations, or IMU preintegration to jointly refine multiple states and visual constraints [4,5,6,7,8,9,10,11]. These methods often achieve high estimation accuracy at the cost of increased computation and memory, which is caused by window size, feature number, and repeated relinearization. Filtering-based VIO instead follows a recursive prediction–update structure and maintains only a bounded set of historical states, providing bounded-state dimension, stable memory usage, and predictable update latency. These properties remain attractive for online systems with limited computational resources.

The Multi-State Constraint Kalman Filter (MSCKF) is a representative filtering-based VIO framework [12]. It maintains the inertial state together with a finite number of cloned camera poses and eliminates landmark positions by projecting multi-view visual residuals onto the left nullspace of the landmark Jacobian. Stereo MSCKF further exploits left–right observations to strengthen scale constraints and feature initialization [13]. Its recursive structure also makes the visual-measurement model directly determine how much information enters the posterior state and covariance. Mismodeled visual uncertainty can, therefore, affect not only the current update but also subsequent recursive estimation.

Conventional MSCKF front-ends rely on KLT tracking, epipolar constraints, RANSAC, and innovation gating to reject inconsistent measurements. However, accepted observations do not necessarily have the same statistical precision. Motion blur, rapid motion, vibration, illumination variation, partial occlusion, repetitive texture, and stereo matching disturbances may degrade a feature before any hard rejection criterion is triggered. Front-end screening, therefore, determines whether a measurement is usable, whereas the filter update must further determine how much reliable information that measurement should contribute.

Recent navigation studies have also emphasized the importance of evaluating measurement reliability and sensing availability under degraded conditions. Self-evaluation and mutual-evaluation strategies have been investigated to quantify navigation sensor reliability in multi-source systems [14], while consistency checking and signal-quality weighting have been used to mitigate degraded GNSS measurements under NLOS and multipath conditions [15]. Forecasting external sensing degradation, such as satellite signal blockage, further highlights the value of anticipating unreliable measurements before severe degradation affects the navigation solution [16]. These studies provide broader context for reliability-aware navigation measurement processing. Nevertheless, for stereo visual-inertial odometry, an explicit measurement-level uncertainty path from visual observation reliability to covariance-consistent filtering updates remains insufficiently explored.

Adaptive measurement-noise estimation has, consequently, been introduced into MSCKF-based estimators. Variational-Bayesian approaches estimate time-varying observation noise online; Gaussian–generalized-inverse-Gaussian formulations model nonstationary visual-measurement noise; CAMSCKF represents different noise characteristics through multivariate parameter clustering; and innovation-based adaptive MSCKF methods regulate visual information using observed innovation statistics. These studies have advanced MSCKF from fixed covariance assumptions toward dynamic measurement-uncertainty modeling. Nevertheless, three issues remain in terms of the complete visual update chain.

First, visual uncertainty is both observation-dependent and direction-dependent. Tracking quality and stereo consistency determine the overall reliability of a feature, while local image structure determines its identifiability along different directions. Edge-like and weak-texture patches suffer from the aperture problem, producing markedly different information content along the gradient-normal and edge-tangent directions. The Lucas–Kanade normal matrix, Shi–Tomasi criterion, and related local structure measures expose this anisotropy [17,18,19]. Scalar covariance adaptation can change the overall weight of a measurement, but cannot explicitly represent such directional differences.

Second, once heteroscedastic and anisotropic covariance is introduced, its structure must be preserved through MSCKF landmark elimination. After left-nullspace projection, the observation covariance is generally transformed into a full feature-block covariance with non-diagonal correlations. Approximating it by a scalar or diagonal form can destroy part of the directional uncertainty modeled at the observation level. The covariance treatment must, therefore, remain consistent through landmark elimination and whitening.

Third, feature-level acceptance does not guarantee batch-level innovation consistency. Multiple features may be simultaneously affected by common motion, imaging conditions, or prior-state errors, causing the innovation population of the current update to deviate from its nominal scale. Estimating the measurement-noise scale only from post-gating low-residual measurements can further introduce truncation bias. Batch-level information, therefore, requires calibration based on the pre-gate innovation population.

These issues reveal a continuous uncertainty-propagation problem in MSCKF: observation reliability and directional identifiability shape the local covariance, landmark elimination transforms this covariance at the feature-block level, and the final update is further affected by the statistical scale of the innovation population. Existing adaptive MSCKF methods have addressed time-varying noise estimation, nonstationary covariance modeling, clustered uncertainty, and innovation-based adaptation, but the coupling among observation-wise risk–direction uncertainty, covariance-preserving nullspace projection, and pre-gate batch-innovation calibration has not been systematically treated within a unified update framework.

To address this problem, we propose RDIC-MSCKF, a Risk–Direction-Decoupled and Innovation-Calibrated MSCKF for stereo visual-inertial odometry. RDIC-MSCKF preserves the standard MSCKF inertial propagation, camera-state cloning, and bounded recursive structure, while introducing an uncertainty-processing chain from the observation level to the feature-block and batch levels. Tracking diagnostics and local photometric information separately characterize dynamic tracking risk and directional identifiability to construct heteroscedastic and anisotropic observation covariance. The full projected covariance is then preserved and whitened after landmark elimination, and a bounded measurement-noise inflation factor is estimated from the pre-gate innovation population to calibrate the information strength of the current visual batch.

The main contributions are summarized as follows:

• We develop a risk–direction-decoupled stereo visual-measurement-covariance model that separately captures observation-dependent tracking reliability and local directional identifiability, yielding heteroscedastic and anisotropic covariance for accepted visual measurements.
• We establish a covariance-preserving landmark-elimination and whitening formulation for anisotropic MSCKF measurements. The full covariance after nullspace projection is explicitly retained, and its equivalence to weighted nullspace processing is derived to preserve the statistical meaning of observation-level uncertainty through feature elimination.
• We introduce a bounded batch-innovation-calibration method based on the pre-gate innovation population. The method determines a minimum measurement-noise inflation factor to regulate batch-level visual information and employs a modal update algebraically equivalent to the corresponding inflated-noise Kalman update.

RDIC-MSCKF is evaluated using the EuRoC MAV benchmark and real-world UAV flight data. The evaluation considers trajectory accuracy, tail error, innovation statistics, finite-window update spectra, computational performance, and component-wise ablation to assess both estimation performance and the behavior of the proposed uncertainty-processing mechanisms.

2. Related Work

2.1. Filtering-Based VIO and Adaptive MSCKF

Filtering-based VIO performs recursive state estimation with bounded memory and computational cost. The original MSCKF removes landmark states through multi-view constraints and updates a finite set of cloned camera poses [12], while stereo extensions exploit synchronized left–right observations to strengthen scale constraints and feature initialization [13]. Subsequent filtering-based systems have incorporated online calibration [20], robocentric formulations [21], direct photometric residuals [22], and consistency treatments based on first-estimates Jacobians, observability analysis, and invariant error-state design [23,24,25]. OpenVINS provides a representative modern implementation of this class of estimators [26].

Measurement uncertainty has received increasing attention in MSCKF-based estimation. Variational-Bayesian S-MSCKF adapts observation-noise statistics online [27], while the Gaussian–generalized-inverse-Gaussian adaptive MSCKF models nonstationary visual-measurement noise [28]. CAMSCKF further distinguishes different time-varying noise characteristics through measurement clustering and covariance estimation [29]. More recent statistically adaptive MSCKF formulations use innovation statistics to regulate the visual update according to observed residual behavior [30]. These studies establish adaptive measurement uncertainty as an important extension of the standard MSCKF measurement model.

RDIC-MSCKF extends this line of research toward observation-wise structured uncertainty. Its measurement model separates tracking-related reliability from local directional identifiability, producing heteroscedastic and anisotropic covariance before landmark elimination. The resulting covariance structure is then retained through the projected MSCKF feature block and further regulated using batch-level innovation statistics.

2.2. Feature Reliability and Directional Visual Uncertainty

Visual front-ends commonly combine KLT tracking [17], Tomasi–Kanade or Shi–Tomasi feature scoring [18,31], stereo matching, epipolar checks, and RANSAC to select geometrically reliable correspondences. Forward–backward tracking, descriptor confidence, learned matching scores, and feature-budget policies provide additional measures of correspondence quality and computational control [32,33,34,35,36]. These mechanisms mainly determine whether a feature should be retained or rejected.

For retained measurements, local image structure also determines the direction in which image displacement can be reliably estimated. The Lucas–Kanade normal matrix and related structure-tensor formulations describe this local directional information [17,18,19]. Edge-like patches typically exhibit a strong and a weak eigen-direction, reflecting the classical aperture problem. Similar photometric information matrices are widely used in direct visual estimation [22,37].

This directional structure provides more information than a scalar feature score. In RDIC-MSCKF, tracking diagnostics describe the overall reliability of an accepted observation, whereas the local photometric information matrix determines its directional uncertainty. The two sources are combined in the measurement covariance rather than collapsed into a single scalar confidence.

2.3. Covariance Propagation and Innovation Adaptation

In standard MSCKF formulations with isotropic measurement noise, an orthonormal landmark-nullspace basis preserves the scalar covariance form after projection. Once observation-dependent anisotropic covariance is introduced, however, the projected covariance is generally full. The covariance introduced at the image level must, therefore, be retained through landmark elimination and whitening to preserve its directional information in the final update.

Innovation-based adaptive estimation provides a complementary mechanism for regulating the overall update strength. Classical covariance matching, NIS-based monitoring, probabilistic data association, and covariance inflation use residual statistics or association likelihoods to detect mismatch between the assumed and observed stochastic models [38,39,40,41,42]. Robust estimation and robust SLAM formulations also reduce the effect of outliers through loss functions, switchable constraints, dynamic covariance scaling, or bundle-adjustment robustification [43,44,45,46,47]. In visual-inertial estimation, related strategies have been used to adapt measurement uncertainty under changing sensing conditions [48].

RDIC-MSCKF combines these two levels of uncertainty treatment. Observation-dependent anisotropic covariance is propagated through the complete MSCKF feature block, while the pre-gate innovation population provides an additional bounded calibration of the current visual batch. This establishes a continuous covariance path from individual observations to landmark-eliminated constraints and, finally, to the batch-level Kalman update.

Table 1 shows the progression from global and cluster-wise covariance adaptation to finer-grained statistical treatment of visual measurements. RDIC-MSCKF focuses on the covariance path across three levels: observation-wise risk and directional uncertainty, covariance-preserving landmark elimination, and batch-level innovation calibration.

Table 1. Comparison of representative adaptive MSCKF formulations.

Method	Adaptation Level	Heterogeneous Measurements	Directional Uncertainty	Covariance After Landmark Elimination	Innovation-Based Calibration
S-MSCKF [13]	Fixed	Not explicitly modeled	Not explicitly modeled	Isotropic form	Gating only
VB-SMSCKF [27]	Filter/update	Adaptive	Not image-direction based	Not explicitly emphasized	Adaptive estimation
GGIG-MSCKF [28]	Global	Adaptive	Not explicitly modeled	Isotropic model	Adaptive estimation
CAMSCKF [29]	Cluster-wise	Cluster-wise	Not image-direction based	Not explicitly emphasized	Innovation clustering
QA-MSCKF [30]	Global + row-wise	Projected residual level	Not explicitly modeled	Projected residual space	Statistical adaptation
RDIC-MSCKF	Observation + batch	Observation-wise	Yes	Full covariance retained	Pre-gate batch calibration

3. Methodology

3.1. Problem Formulation and Method Overview

Let $\delta \mathbf{x}\in {\mathbb{R}}^{n_x}$ denote the MSCKF error state, which includes the IMU error variables and the active camera-clone pose errors. For feature j observed by clone i, the normalized stereo measurement is

$${\mathbf{z}}_{j,i}={\left[\begin{array}{cccc}{u}_{j,i}^0& v_{j,i}^0& u_{j,i}^1& v_{j,i}^1\end{array}\right]}^{\top }\in {\mathbb{R}}^4.$$

(1)

Linearization gives

$${\mathbf{r}}_{j,i}={\mathbf{H}}_{x,j,i}\delta \mathbf{x}+{\mathbf{H}}_{f,j,i}\delta {\mathbf{p}}_j+{\mathit{ϵ}}_{j,i},{\mathit{ϵ}}_{j,i}\sim \mathcal{N}(\mathbf{0},{\mathbf{R}}_{j,i}),$$

(2)

where $\delta {\mathbf{p}}_j\in {\mathbb{R}}^3$ is the landmark-position error. A single stereo observation is four-dimensional. The inertial propagation model, state augmentation, and observability treatment follow the standard MSCKF design and are not changed by the proposed visual-uncertainty model.

RDIC-MSCKF modifies the visual update through three estimator interfaces. At the observation level, tracking reliability and local image-direction information define an observation-dependent stereo covariance ${\mathbf{R}}_{j,i}$. At the feature-block level, this heteroscedastic and anisotropic covariance is carried through landmark elimination by retaining the full projected covariance before whitening. At the batch level, a bounded empirical inflation is applied to the accepted whitened system using statistics collected before feature-level truncation. These interfaces form one coupled update chain within the recursive MSCKF estimator.

The resulting visual-update procedure is summarized in Algorithm 1.

Algorithm 1 Integrated RDIC-MSCKF Visual Update

1: Input: stereo images, IMU-propagated prior state and covariance, tracked stereo feature observations.   2: Output: updated state and covariance.   3: Run quality-aware stereo feature management: track and match features, rank grid candidates, retain reliable tracks, initialize new features, and enforce the feature budget.   4: Form risk and directional metadata from robust tracking diagnostics and local photometric information.   5: Assemble the stereo observation covariance from the tracking standard-deviation multiplier and the penalty-only directional covariance increment.   6: Construct multi-view stereo residuals and Jacobians for each feature using the prior state.   7: Eliminate landmark uncertainty with a complete left-nullspace projection.   8: Preserve and whiten the full projected covariance for each feature block.   9: Add each whitened feature-block statistic to the pre-gate batch population. 10: Send feature blocks that pass the feature-level innovation gate to the accepted-block stack. 11: Estimate the bounded minimum batch inflation from the pre-gate population. 12: If the accepted-block stack is nonempty, compress the accepted whitened system when required and apply the modal Kalman update using the accepted blocks and the estimated inflation; otherwise, skip the visual update for this frame.

3.2. Risk–Direction Decoupled Observation Uncertainty

Dynamic tracking risk. For each accepted feature candidate in the current frame, the front-end provides diagnostic errors for left/right temporal tracking, left–right temporal imbalance, selective forward–backward disagreement, stereo epipolar consistency, disparity jump, and lifetime. The frame index is omitted in this paragraph for readability; the resulting metadata is stored per observation and is later used as ${\sigma }_{j,i}^{\mathrm{trk}}$, ${\delta }_{j,i}^w$, and ${\delta }_{j,i}^s$ when feature tracks are stacked over clones. Let

$${\xi }_j={\left[\begin{array}{ccccc}{z}_{t,j}& z_{b,j}& z_{s,j}& z_{\mathrm{FB},j}& z_{d,j}\end{array}\right]}^{\top }$$

(3)

collect the robustly normalized temporal mean, temporal imbalance, stereo epipolar, forward–backward, and disparity-jump diagnostics. Each entry is computed from the current candidate population by

$$z_{m,j}={\left[{\displaystyle \frac{e_{m,j}-\mathrm{med}\left({\mathcal{E}}_m\right)}{\max (1.4826\mathrm{MAD}\left({\mathcal{E}}_m\right),{\sigma }_{m,\min })+\epsilon }}\right]}_{+},$$

(4)

where ${\mathcal{E}}_m$ is the frame-level sample set for diagnostic family m, ${\sigma }_{m,\min }$ is a stability floor, and ${[·]}_{+}$ retains only above-median degradation. Forward–backward error is evaluated selectively for high-suspicion tracks under a computation budget. When the selective forward–backward check is not executed, its risk term is assigned the implementation-defined neutral value. No additional FB-based penalty is introduced, and the missing check is not interpreted as a measured zero-tracking error.

Figure 1 illustrates the corresponding information flow and uncertainty path.

Figure 1. Integrated RDIC-MSCKF visual update chain. Stereo tracking produces visual observations and quality metadata. Tracking-risk and local photometric-information paths meet at the stereo observation covariance. Multi-view constraints are eliminated and whitened with the full projected covariance, after which the whitened blocks are split into accepted feature blocks and pre-gate batch statistics. The final update is performed by the bounded modal Kalman step. The left-to-right path shows how accepted stereo observations are converted into landmark-eliminated feature blocks, while the parallel uncertainty paths show how tracking reliability and directional image information define the covariance used before feature-level gating and batch-level calibration.

The tracking risk is represented as a quadratic robust energy

$${\rho }_j={\mathit{\xi }}_j^{\top }{\mathbf{W}}_r{\mathit{\xi }}_j,{\mathbf{W}}_r=\mathrm{diag}\left(1,{\displaystyle \frac{1}{2}},1,{\displaystyle \frac{3}{4}},{\displaystyle \frac{1}{2}}\right).$$

(5)

This energy is a monotone reliability score with design weights that encode the relative severity assigned to the diagnostic families. A lifetime tolerance

$$\vartheta \left({\mathsf{\ell }}_j\right)=1+0.25\log (1+{\mathsf{\ell }}_j)$$

(6)

where ${\mathsf{\ell }}_j$ denotes the current lifetime of track j. This factor prevents long-lived tracks from being penalized too aggressively for small transient deviations. The tracking quality and standard-deviation multiplier are

$$q_j^{\mathrm{trk}}=\exp [-{\rho }_j/\vartheta \left({\mathsf{\ell }}_j\right)],{\sigma }_j^{\mathrm{trk}}=1+4(1-q_j^{\mathrm{trk}}).$$

(7)

The multiplier ${\sigma }_j^{\mathrm{trk}}$ acts on standard deviation; it enters covariance as ${\left({\sigma }_j^{\mathrm{trk}}\right)}^2$. The back-end further clamps the received multiplier before squaring it, providing a bounded range for the covariance input.

Quality-aware feature management. The tracking quality $q_j^{\mathrm{trk}}$ forms the covariance multiplier and also regulates the visual constraint set before the back-end update. Within each image grid, features are ranked by a quality-aware score so that reliable and geometrically useful tracks are preferred when a cell contains more candidates than can be retained. Very low-quality tracks can be discarded even if they have not yet violated a hard geometric outlier test. Conversely, long-lived tracks with acceptable quality are allowed to survive grid competition because they provide multi-clone constraints and reduce unnecessary new feature initialization.

New feature detection follows the same quality-management policy. Grid cells with too few retained tracks are replenished by new detections, whereas cells already supported by high-quality tracks are not forced to add redundant features. When the total update row budget is approached, feature blocks are further ranked using projected information and innovation reliability so that the accepted stack remains computationally bounded. This policy controls the number, quality, and lifetime of feature blocks entering the MSCKF update, and, therefore, affects accuracy, finite-window spectra, and runtime.

Robust local photometric information. For a local patch ${\mathcal{P}}_j$ expressed in the previous-image coordinates, let ${\mathbf{u}}_j$ denote the tracked displacement from the previous image to the current image. The demeaned temporal photometric residual at pixel $\mathbf{p}$ is

$$e_{\mathbf{p}}=\left(I_i(\mathbf{p}+{\mathbf{u}}_j)-{\overline{I}}_i\right)-\left(I_{i-1}\left(\mathbf{p}\right)-{\overline{I}}_{i-1}\right).$$

(8)

Assuming local photometric noise variance ${\sigma }_I^2$, the unweighted Gauss–Newton information for small image displacement is proportional to the sum of gradient outer products. With Huber weight

$$w_{\mathbf{p}}=\left\{\begin{array}{cc}1,\hfill & |e_{\mathbf{p}}|\le {\delta }_H,\hfill \\ {\delta }_H/\left|e_{\mathbf{p}}\right|,\hfill & |e_{\mathbf{p}}|>{\delta }_H,\hfill \end{array}\right.$$

(9)

the robust observed-information approximation is

$${\mathbf{F}}_j={\displaystyle \frac{1}{{\sigma }_I^2}}\sum _{\mathbf{p}\in {\mathcal{P}}_j}{w}_{\mathbf{p}}{\mathbf{g}}_{\mathbf{p}}{\mathbf{g}}_{\mathbf{p}}^{\top }\in {\mathbb{R}}^{2\times 2},{\mathbf{g}}_{\mathbf{p}}=\nabla I_i(\mathbf{p}+{\mathbf{u}}_j).$$

(10)

Because the Huber weights depend on observed residuals, ${\mathbf{F}}_j$ is used as a weighted Gauss–Newton or robust observed-information matrix. Newly detected points use the current-image gradient version of the same approximation. The absolute photometric noise scale ${\sigma }_I^2$ acts as a common proportional factor in the local information matrix. The subsequent covariance penalty uses relative eigenvalue ratios with respect to the frame reference, so the FIM path provides a relative directional-identifiability cue for the retained observation.

Let

$${\mathbf{F}}_j={\mathbf{U}}_j^F\left[\begin{array}{cc}{\lambda }_j^{F,\min }& 0\\ 0& {\lambda }_j^{F,\max }\end{array}\right]{\left({\mathbf{U}}_j^F\right)}^{\top },{\mathbf{v}}_j^w={\mathbf{u}}_j^{F,\min },{\mathbf{v}}_j^s={\left[\begin{array}{cc}-v_{j,y}^w& v_{j,x}^w\end{array}\right]}^{\top }.$$

(11)

The columns of ${\mathbf{U}}_j^F$ are ordered by ascending eigenvalue, so ${\mathbf{v}}_j^w$ is the weakly identifiable image direction and ${\mathbf{v}}_j^s$ is the orthogonal strong direction in the pixel-gradient coordinate system. The sign of either eigenvector is immaterial because the covariance uses outer products. A small ${\lambda }_j^{F,\min }$ indicates aperture-like ambiguity. To avoid assigning super-baseline confidence to high-texture patches, the frame median

$${\lambda }_{\mathrm{ref}}^F=\mathrm{med}\{{\lambda }_j^{F,\min }:j\in {\mathcal{V}}_F\}$$

(12)

is computed from the valid local-information set ${\mathcal{V}}_F$ and clipped to a positive implementation range. This value is used only as a relative penalty reference. The directional increments are

$$\begin{array}{cccc}\hfill r_j^w& ={\left[\sqrt{{\lambda }_{\mathrm{ref}}^F/{\lambda }_j^{F,\min }}-1\right]}_{+},\hfill & \hfill r_j^s& ={\left[\sqrt{{\lambda }_{\mathrm{ref}}^F/{\lambda }_j^{F,\max }}-1\right]}_{+},\hfill \end{array}$$

(13)

$$\begin{array}{cccc}\hfill {\delta }_j^w& =\mathrm{clip}(1+{\alpha }_w{r}_j^w,1,{\delta }_{w,\max }),\hfill & \hfill {\delta }_j^s& =\min \{{\delta }_j^w,\mathrm{clip}(1+{\alpha }_s{r}_j^s,1,{\delta }_{s,\max })\}.\hfill \end{array}$$

(14)

Thus,

$$1\le {\delta }_j^s\le {\delta }_j^w.$$

(15)

Directional metadata is activated when the local information estimate passes reliability checks involving validity, track age, tracking quality, and stereo consistency.

Additive observation covariance. Let ${\sigma }_0^2$ be the baseline normalized-plane measurement variance. The visual residuals used by the MSCKF update are defined in normalized image coordinates, while the local photometric directions are obtained from calibrated image patches. In general, a pixel-plane covariance ${\mathbf{\Sigma }}_c^{\mathrm{px}}$ for camera c is mapped to the normalized plane by the intrinsic Jacobian

$${\mathbf{D}}_c=\left[\begin{array}{cc}1/f_{x,c}& 0\\ 0& 1/f_{y,c}\end{array}\right],{\mathbf{\Sigma }}_c^{\mathrm{norm}}={\mathbf{D}}_c{\mathbf{\Sigma }}_c^{\mathrm{px}}{\mathbf{D}}_c^{\top }.$$

(16)

For calibrated stereo cameras with near-square pixels, this diagonal normalization preserves local image directions to first order. The proposed covariance, therefore, uses the pixel-gradient eigenvectors as normalized-plane directional axes for the relative FIM penalty, while the baseline normalized variance ${\sigma }_0^2$ sets the measurement scale. With this coordinate convention, the two-dimensional covariance assigned to one image observation is

$${\mathbf{\Sigma }}_{j,i}^{2D}={\sigma }_0^2{\left({\sigma }_{j,i}^{\mathrm{trk}}\right)}^2{\mathbf{I}}_2+{\sigma }_0^2{\mathbf{D}}_{j,i}^F,$$

(17)

where

$${\mathbf{D}}_{j,i}^F=({\delta }_{j,i}^w-1){\mathbf{v}}_{j,i}^w{\left({\mathbf{v}}_{j,i}^w\right)}^{\top }+({\delta }_{j,i}^s-1){\mathbf{v}}_{j,i}^s{\left({\mathbf{v}}_{j,i}^s\right)}^{\top }.$$

(18)

The risk term expands the covariance isotropically according to dynamic reliability, whereas the photometric term adds direction-dependent variance associated with local identifiability loss. The design is additive rather than multiplicative, so the directional penalties are not scaled by tracking risk a second time. The stereo covariance uses the same two-dimensional block for the left and right observations,

$${\mathbf{R}}_{j,i}=\mathrm{blkdiag}({\mathbf{\Sigma }}_{j,i}^{2D},{\mathbf{\Sigma }}_{j,i}^{2D})\in {\mathbb{R}}^{4\times 4}.$$

(19)

This shared-quality construction applies the same calibrated directional covariance to the left and right observations, resulting in a block-diagonal stereo covariance.

Conservative properties. Because ${\sigma }_{j,i}^{\mathrm{trk}}\ge 1$ and ${\mathbf{D}}_{j,i}^F⪰\mathbf{0}$,

$${\mathbf{\Sigma }}_{j,i}^{2D}⪰{\sigma }_0^2{\left({\sigma }_{j,i}^{\mathrm{trk}}\right)}^2{\mathbf{I}}_2⪰{\sigma }_0^2{\mathbf{I}}_2.$$

(20)

This establishes the risk covariance lower bound. In addition, Equations (15) and (18) imply that the photometric term can only preserve or increase covariance relative to the risk-only baseline. It, therefore, reduces visual information in weakly identifiable directions but never rewards a feature with below-baseline measurement noise.

The main symbols used in the formulation are summarized in Table 2.

Table 2. Main symbols used in the non-experimental formulation.

Symbol	Dimension	Meaning
$\delta \mathbf{x}$	$n_x$	MSCKF error state
${\mathbf{z}}_{j,i}$	4	normalized stereo observation of feature j at clone i
${\mathit{ϵ}}_{j,i}$	4	stereo measurement noise
${\mathit{\xi }}_j$	5	robust tracking diagnostic vector
${\rho }_j$	scalar	tracking-risk energy from the robust diagnostic vector
${\mathsf{\ell }}_j$	frames	lifetime of feature track j
$\vartheta \left({\mathsf{\ell }}_j\right)$	scalar	lifetime-tolerance factor
$q_j^{\mathrm{trk}}$	scalar	tracking quality
${\sigma }_j^{\mathrm{trk}}$	scalar	tracking standard-deviation multiplier
${\mathbf{u}}_j$	2	tracked patch displacement used for temporal photometric information
${\mathcal{V}}_F$	set	valid local-information candidates used for the frame reference
${\mathbf{F}}_j$	$2\times 2$	robust local photometric information approximation
${\mathbf{v}}_j^w,{\mathbf{v}}_j^s$	2	weak and strong image directions used by the directional covariance penalty
${\lambda }_j^{F,\min },{\lambda }_j^{F,\max }$	scalar	photometric information eigenvalues
${\mathbf{\Sigma }}_{j,i}^{2D}$	$2\times 2$	single-image observation covariance
${\mathbf{R}}_{j,i}$	$4\times 4$	stereo observation covariance
${\tilde{\mathbf{r}}}_j,{\tilde{\mathbf{H}}}_j$	${\nu }_j$, ${\nu }_j\times n_x$	whitened feature-block residual and Jacobian
${\lambda }_{j,k}^B$	scalar	batch innovation prior eigenvalue
${\eta }_b$	scalar	batch quantile level
${\alpha }_{★}^2$	scalar	bounded batch measurement-noise variance inflation

The fixed parameter and implementation settings are listed in Table 3.

Table 3. Fixed parameter and implementation settings used by the integrated RDIC-MSCKF configuration. All evaluated sequences use the same settings without sequence-specific tuning. Entries marked as implementation settings are fixed in the program but are not interpreted as statistically optimal constants.

Quantity	Mathematical Symbol	Default/Range	Unit	Code Variable/Function	Design Basis
Temporal robust floor	${\sigma }_{t,\min }$	$1.0$	pixel-like	robustZ input	MAD stability floor
Balance robust floor	${\sigma }_{b,\min }$	$0.5$	pixel-like	robustZ input	MAD stability floor
FB robust floor	${\sigma }_{\mathrm{FB},\min }$	$0.5$	pixel	robustZ input	MAD stability floor
Stereo robust floor	${\sigma }_{s,\min }$	$0.5$	pixel-like	robustZ input	MAD stability floor
Disparity robust floor	${\sigma }_{d,\min }$	$1.0$	pixel	robustZ input	MAD stability floor
Risk weights	${\mathbf{W}}_r$	$\mathrm{diag}(1,0.5,1,0.75,0.5)$	dimensionless	computeRiskFromRobustZ	fixed diagnostic weighting
Lifetime tolerance gain	–	$0.25$	dimensionless	lifetimeTolerance	fixed lifetime moderation
Risk standard-deviation range	${\sigma }^{\mathrm{trk}}$	frontend $[1,5]$, backend $[1,6]$	dimensionless	qToSigmaMultiplier/ obs_var_scale	bounded covariance scaling
FIM reference eigenvalue	${\lambda }_{\mathrm{ref}}$	median clipped to $[5,5000]$	gradient2	adaptiveLambdaReference	relative frame reference
Weak-direction FIM penalty	${\delta }^w$	$[1,25]$	variance multiplier	fim_delta_para	penalty-only bound
Strong-direction FIM penalty	${\delta }^s$	$[1,\min (6,{\delta }^w)]$	variance multiplier	fim_delta_perp	preserve ${\delta }^s\le {\delta }^w$
Feature gate confidence	$p_g$	$0.95$	probability	chi_squared_test_table	nominal chi-square gate
Feature gate relaxation	–	$1.30$	dimensionless	gate_scale	chi-square threshold multiplier
Batch quantile	${\eta }_b$	$0.80$	probability	quantile_q	pre-gate batch statistic
Batch inflation bound	${\alpha }_{\max }^2$	$4.0$	whitened variance	alpha2_max	bounded empirical calibration
Minimum batch blocks	$N_{b,\min }$	6	feature blocks	min_blocks	activation threshold
Bisection tolerance	${ϵ}_{\alpha }$	${10}^{-3}$	whitened variance	alpha2_tol	search termination
Bisection iterations	$I_{\alpha }$	16	iterations	alpha2_max_iters	search cap
Selective FB budget	B	$\min (8,\max (3,\lfloor N_s/10\rfloor ))$	tracks/frame	computeFbBudget	$N_s$: suspicious tracks
Feature row budget	$M_{\max }$	1500	rows	MAX_ROWS	bounded update size

3.3. Covariance-Preserving MSCKF Feature Elimination

For a feature observed in $m_j$ stereo frames, the residuals are stacked as

$${\mathbf{r}}_j={\mathbf{H}}_{x,j}\delta \mathbf{x}+{\mathbf{H}}_{f,j}\delta {\mathbf{p}}_j+{\mathit{ϵ}}_j,{\mathit{ϵ}}_j\sim \mathcal{N}(\mathbf{0},{\mathbf{R}}_j),{\mathbf{R}}_j=\mathrm{blkdiag}({\mathbf{R}}_{j,1},\dots ,{\mathbf{R}}_{j,m_j}).$$

(21)

Here, ${\mathbf{r}}_j\in {\mathbb{R}}^{4m_j}$, ${\mathbf{H}}_{x,j}\in {\mathbb{R}}^{4m_j\times n_x}$, and ${\mathbf{H}}_{f,j}\in {\mathbb{R}}^{4m_j\times 3}$. Let $r_j^f=\mathrm{rank}\left({\mathbf{H}}_{f,j}\right)\le 3$ and choose

$${\mathbf{A}}_j\in {\mathbb{R}}^{4m_j\times (4m_j-r_j^f)},{\mathbf{A}}_j^{\top }{\mathbf{H}}_{f,j}=\mathbf{0},$$

(22)

with columns spanning the complete left nullspace of ${\mathbf{H}}_{f,j}$. The projected model is

$${\mathbf{r}}_j^p={\mathbf{A}}_j^{\top }{\mathbf{r}}_j,{\mathbf{H}}_j^p={\mathbf{A}}_j^{\top }{\mathbf{H}}_{x,j},{\mathbf{R}}_j^p={\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j.$$

(23)

Retaining ${\mathbf{R}}_j^p$ as a full covariance is essential because a diagonal-only treatment would discard correlations introduced by projection. If ${\mathbf{R}}_j^p={\mathbf{L}}_j{\mathbf{L}}_j^{\top }$ is nonsingular, whitening gives

$${\tilde{\mathbf{r}}}_j={\mathbf{L}}_j^{-1}{\mathbf{r}}_j^p,{\tilde{\mathbf{H}}}_j={\mathbf{L}}_j^{-1}{\mathbf{H}}_j^p,\mathrm{Cov}\left({\widetilde{\mathit{ϵ}}}_j\right)={\mathbf{I}}_{{\nu }_j},$$

(24)

where ${\nu }_j=4m_j-r_j^f$.

Proposition 1.

Projection with complete covariance whitening is equivalent, up to a nonsingular basis change, to whitening the original feature model first and then applying a weighted landmark-nullspace projection.

Proof. 

Define

$${\mathbf{C}}_j={\left({\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j\right)}^{-1/2}{\mathbf{A}}_j^{\top }.$$

(25)

Then,

$${\mathbf{C}}_j{\mathbf{H}}_{f,j}={\left({\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j\right)}^{-1/2}{\mathbf{A}}_j^{\top }{\mathbf{H}}_{f,j}=\mathbf{0}$$

(26)

and

$${\mathbf{C}}_j{\mathbf{R}}_j{\mathbf{C}}_j^{\top }={\left({\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j\right)}^{-1/2}\left({\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j\right){\left({\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j\right)}^{-1/2}={\mathbf{I}}_{{\nu }_j}.$$

(27)

Thus, ${\mathbf{C}}_j$ is a whitened operator whose rows annihilate landmark uncertainty. If the original model is first whitened by ${\mathbf{R}}_j^{-1/2}$, a weighted left-nullspace basis ${\mathbf{N}}_j$ satisfies ${\mathbf{N}}_j^{\top }{\mathbf{R}}_j^{-1/2}{\mathbf{H}}_{f,j}=0$. Both ${\mathbf{C}}_j$ and ${\mathbf{N}}_j^{\top }{\mathbf{R}}_j^{-1/2}$ have dimension ${\nu }_j$ and span the complete row space of the landmark-eliminating whitened constraints. Hence, there exists a nonsingular matrix ${\mathbf{T}}_j$ such that

$${\mathbf{C}}_j={\mathbf{T}}_j{\mathbf{N}}_j^{\top }{\mathbf{R}}_j^{-1/2}.$$

(28)

When both whitened constraint bases are chosen orthonormally, ${\mathbf{T}}_j$ is orthogonal. Under this orthogonal basis change, the state information matrix and information vector contributed by feature j are invariant:

$${\left({\mathbf{T}}_j{\tilde{\mathbf{H}}}_j\right)}^{\top }\left({\mathbf{T}}_j{\tilde{\mathbf{H}}}_j\right)={\tilde{\mathbf{H}}}_j^{\top }{\tilde{\mathbf{H}}}_j,{\left({\mathbf{T}}_j{\tilde{\mathbf{H}}}_j\right)}^{\top }\left({\mathbf{T}}_j{\tilde{\mathbf{r}}}_j\right)={\tilde{\mathbf{H}}}_j^{\top }{\tilde{\mathbf{r}}}_j.$$

(29)

For a nonsingular but non-orthogonal basis transformation, the same eliminated constraint space is represented with a correspondingly transformed covariance; the whitened orthonormal representation above is used to expose information equivalence. If ${\mathbf{H}}_{f,j}$ is rank deficient, $r_j^f$ reduces the dimension of the eliminated landmark column space and increases ${\nu }_j$ accordingly; the same argument holds after a rank-revealing basis is used for ${\mathbf{A}}_j$. This linear-algebra statement does not require a degenerate feature to be accepted: the actual update still requires stable whitening of ${\mathbf{R}}_j^p$ and subsequent innovation screening. If the numerical factorization of ${\mathbf{R}}_j^p$ is ill-conditioned, eigenvalue whitening and diagonal whitening are stability fallbacks rather than the theoretical main path. □

3.4. Bounded Batch-Innovation Calibration

Each whitened feature block follows

$${\tilde{\mathbf{r}}}_j={\tilde{\mathbf{H}}}_j\delta \mathbf{x}+{\widetilde{\mathit{ϵ}}}_j,{\widetilde{\mathit{ϵ}}}_j\sim \mathcal{N}(\mathbf{0},{\mathbf{I}}_{{\nu }_j}).$$

(30)

With prior covariance ${\mathbf{P}}^{-}$, the nominal feature-block innovation covariance and normalized innovation statistic are

$${\mathbf{S}}_j\left(1\right)={\tilde{\mathbf{H}}}_j{\mathbf{P}}^{-}{\tilde{\mathbf{H}}}_j^{\top }+\mathbf{I},{\gamma }_j\left(1\right)={\tilde{\mathbf{r}}}_j^{\top }{\mathbf{S}}_j{\left(1\right)}^{-1}{\tilde{\mathbf{r}}}_j,s_j\left(1\right)={\gamma }_j\left(1\right)/{\nu }_j.$$

(31)

Feature-level chi-square gates reject clearly inconsistent feature blocks. A secondary normalized-statistic screen is used for conservative lost-feature updates. Batch calibration is estimated from the pre-gate population for which the feature-block statistic is computable, avoiding a batch scale estimated only from the low-residual subset that remains after gating.

Let

$${\mathbf{B}}_j={\tilde{\mathbf{H}}}_j{\mathbf{P}}^{-}{\tilde{\mathbf{H}}}_j^{\top }={\mathbf{U}}_j^B{\mathbf{\Lambda }}_j^B{\left({\mathbf{U}}_j^B\right)}^{\top },{\mathbf{\Lambda }}_j^B=\mathrm{diag}({\lambda }_{j,1}^B,\dots ,{\lambda }_{j,{\nu }_j}^B),$$

(32)

and define modal residual coordinates ${\mathbf{y}}_j={\left({\mathbf{U}}_j^B\right)}^{\top }{\tilde{\mathbf{r}}}_j$. If the whitened measurement noise is inflated from $\mathbf{I}$ to ${\alpha }^2\mathbf{I}$, the feature statistic becomes

$$s_j\left({\alpha }^2\right)={\displaystyle \frac{1}{{\nu }_j}}{\displaystyle \sum _{k=1}^{{\nu }_j}}{\displaystyle \frac{y_{j,k}^2}{{\lambda }_{j,k}^B+{\alpha }^2}}.$$

(33)

The eigenvalue ${\lambda }_{j,k}^B$ represents the prior state uncertainty projected into innovation mode k, whereas $y_{j,k}^2$ is the observed innovation energy in that mode. The scalar ${\alpha }^2$ inflates only the measurement-noise term after the observation-level covariance has been applied; it does not change the propagated state covariance.

Let ${\eta }_b$ be the batch quantile level, and let ${\nu }_{\mathrm{med}}=\mathrm{med}\left\{{\nu }_j\right\}$ denote the median feature-block degrees of freedom, evaluated as the implementation’s integer median of block dimensions. The target normalized chi-square quantile is

$${\tau }_b={\chi }_{{\nu }_{\mathrm{med}}}^2\left({\eta }_b\right)/{\nu }_{\mathrm{med}}.$$

(34)

Here, ${\chi }_{{\nu }_{\mathrm{med}}}^2\left({\eta }_b\right)$ denotes the ${\eta }_b$ quantile of a chi-square distribution with ${\nu }_{\mathrm{med}}$ degrees of freedom. The bounded minimum inflation problem is

$$\begin{array}{cc}\hfill {\alpha }_{★}^2=\underset{{\alpha }^2}{\min }& {\alpha }^2\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathcal{Q}}_{{\eta }_b}\left({\alpha }^2\right)={\mathrm{Quantile}}_{{\eta }_b}{\left\{s_j\left({\alpha }^2\right)\right\}}_{j=1}^{N_b}\le {\tau }_b,\hfill \\ & 1\le {\alpha }^2\le {\alpha }_{\max }^2.\hfill \end{array}$$

(35)

For every feature block,

$${\displaystyle \frac{\partial s_j\left({\alpha }^2\right)}{\partial {\alpha }^2}}=-{\displaystyle \frac{1}{{\nu }_j}}{\displaystyle \sum _{k=1}^{{\nu }_j}}{\displaystyle \frac{y_{j,k}^2}{{({\lambda }_{j,k}^B+{\alpha }^2)}^2}}\le 0.$$

(36)

Therefore, ${\mathcal{Q}}_{{\eta }_b}\left({\alpha }^2\right)$ is nonincreasing, and bisection finds the smallest feasible inflation in the configured interval up to the numerical search tolerance. If the number of computable feature blocks is below a threshold, calibration is not activated. If the quantile condition already holds at ${\alpha }^2=1$, no inflation is applied. If the upper bound cannot satisfy the condition, the bounded upper value is used. The calibration is empirical because feature blocks share the same state prior and may have different dimensions; the median degree of freedom provides a practical batch reference within the recursive visual update.

3.5. Modal Update, Properties, and Complexity

After feature-level screening, all accepted whitened constraints are stacked as

$$\tilde{\mathbf{r}}=\tilde{\mathbf{H}}\delta \mathbf{x}+\widetilde{\mathit{ϵ}},\widetilde{\mathit{ϵ}}\sim \mathcal{N}(\mathbf{0},{\alpha }_{★}^2\mathbf{I}).$$

(37)

The standard Kalman form is

$${\mathbf{S}}_{\alpha }=\tilde{\mathbf{H}}{\mathbf{P}}^{-}{\tilde{\mathbf{H}}}^{\top }+{\alpha }_{★}^2\mathbf{I},{\mathbf{K}}_{\alpha }={\mathbf{P}}^{-}{\tilde{\mathbf{H}}}^{\top }{\mathbf{S}}_{\alpha }^{-1},\delta \widehat{\mathbf{x}}={\mathbf{K}}_{\alpha }\tilde{\mathbf{r}}.$$

(38)

Let ${\alpha }_{★}=\sqrt{{\alpha }_{★}^2}$, ${\mathbf{P}}^{-}={\mathbf{L}}_P{\mathbf{L}}_P^{\top }$, $\overline{\mathbf{H}}=\tilde{\mathbf{H}}/{\alpha }_{★}$, and $\overline{\mathbf{r}}=\tilde{\mathbf{r}}/{\alpha }_{★}$. Then, Equation (38) is equivalent to a unit-noise update defined by

$$\mathbf{G}=\overline{\mathbf{H}}{\mathbf{L}}_P=\mathbf{U}\mathbf{\Sigma }{\mathbf{V}}^{\top }.$$

(39)

Using the Woodbury identity and the thin SVD, the state correction is

$$\delta \widehat{\mathbf{x}}={\mathbf{L}}_P\mathbf{V}\mathrm{diag}\left({\displaystyle \frac{{\sigma }_k}{{\sigma }_k^2+1}}\right){\mathbf{U}}^{\top }\overline{\mathbf{r}}.$$

(40)

The posterior covariance is

$${\mathbf{P}}^{+}={\mathbf{L}}_P\left[\mathbf{I}-\mathbf{V}\mathrm{diag}\left({\displaystyle \frac{{\sigma }_k^2}{{\sigma }_k^2+1}}\right){\mathbf{V}}^{\top }\right]{\mathbf{L}}_P^{\top }.$$

(41)

Equations (40) and (41) are obtained by diagonalizing $\overline{\mathbf{H}}{\mathbf{P}}^{-}{\overline{\mathbf{H}}}^{\top }$ through $\mathbf{G}{\mathbf{G}}^{\top }$ and are algebraically identical to the inflated-noise Kalman update for the same linearized system. Standard and numerical fallback update paths can use Joseph-form covariance updates, whereas the inflated path above uses an equivalent square-root modal information contraction.

Since ${\alpha }_{★}^2\ge 1$,

$${\tilde{\mathbf{H}}}^{\top }{\left({\alpha }_{★}^2\mathbf{I}\right)}^{-1}\tilde{\mathbf{H}}⪯{\tilde{\mathbf{H}}}^{\top }\tilde{\mathbf{H}}.$$

(42)

For a fixed prior, fixed linearization point, and identical accepted constraint set, the posterior information matrix with inflated noise is no larger than the nominal one. Hence,

$${\mathbf{P}}^{+}\left({\alpha }_{★}^2\right)⪰{\mathbf{P}}^{+}\left(1\right).$$

(43)

For a fixed prior, linearization point, and accepted constraint set, this ordering states that batch inflation cannot increase visual-measurement information under the same linearized update model.

When the stacked whitened system has more rows than the state dimension, row compression is performed by QR factorization:

$$\tilde{\mathbf{H}}=\mathbf{Q}\left[\begin{array}{c}{\mathbf{H}}_c\\ \mathbf{0}\end{array}\right],{\mathbf{Q}}^{\top }\tilde{\mathbf{r}}=\left[\begin{array}{c}{\mathbf{r}}_c\\ {\mathbf{r}}_{\perp }\end{array}\right].$$

(44)

The residual term ${\mathbf{r}}_{\perp }$ is independent of the state increment and can be omitted from the update. If a configured row budget is exceeded, accepted features are ranked by a score that increases with projected prior information and decreases with large normalized innovation. This ranking is a computational budget policy.

Table 4 summarizes the asymptotic cost of the main update operations.

Table 4. Asymptotic cost of main update operations. N is the number of tracked candidates, B the selective forward–backward budget, L the number of pyramid levels, K the average KLT iterations, $A_p$ the patch area, $d_j=4m_j$ the raw stacked stereo residual dimension of feature j, $r_j=\mathrm{rank}\left({\mathbf{H}}_{f,j}\right)\le 3$, ${\nu }_j=d_j-r_j$ the projected feature-block dimension, $N_b$ the number of batch-statistic blocks, $I_{\alpha }$ the number of bisection iterations, M the accepted update rows, and $n_x$ the state dimension. Dense upper bounds are shown; practical MSCKF Jacobians retain clone sparsity and per-grid feature management is typically cheaper than global sorting.

Operation	Cost	Role
Stereo temporal KLT	$O\left(NKL{A}_p\right)$	baseline front-end tracking
Selective forward–backward check	$O\left(BKL{A}_p\right)$, $B\le N$	added diagnostic cost
Local information matrix	$O\left(N{A}_p\right)$	added patch-gradient accumulation
Quality-aware feature management	$O\left(N\log N\right)$ worst case, lower with per-grid sorting	feature retention and row-budget management
Nullspace basis for feature j	$O(d_j{r}_j^2+r_j^3)$, $r_j\le 3$	landmark elimination basis construction
Projection of residual/Jacobian	$O\left(d_j{\nu }_j{n}_x\right)$	construction of the projected MSCKF block
Projected covariance ${\mathbf{A}}_j^{\top }{\mathbf{R}}_j{\mathbf{A}}_j$	$O\left(d_j{\nu }_j^2\right)$ with block-diagonal ${\mathbf{R}}_j$	covariance preservation for anisotropic heteroscedastic blocks
Projected covariance whitening	$O\left({\nu }_j^3\right)$	LLT/eigendecomposition of the projected block covariance
Block innovation eigendecomposition	$O\left({\nu }_j^3\right)$	modal statistic for pre-gate batch calibration
Bisection for ${\alpha }_{★}^2$	$O\left(I_{\alpha }{N}_b\overline{\nu }\right)$	added bounded calibration
QR row compression	$O\left(M{n}_x^2\right)$ when $M>n_x$	standard computational compression
Modal Kalman update	$O(M{n}_x^2+n_x^3)$ dense upper bound	inflated-noise update; thin-SVD cost changes with the relation between M and $n_x$

4. Experiments

The experiments compare RDIC-MSCKF with a controlled S-MSCKF baseline under the same filtering structure, input data, calibration source, and evaluation protocol. The reported results characterize the final estimator configuration. The evaluation uses the public EuRoC MAV benchmark [49] to provide reproducible stereo visual-inertial sequences with industrial textures, illumination variation, and rapid motion. Additional real-world UAV flights provide platform-level validation of the spatial referencing capability required by UAV-based multisensor inspection platforms.

The controlled comparison keeps the underlying MSCKF state, IMU propagation, cloning, camera–IMU calibration source, EuRoC stereo/IMU streams, trajectory intervals, state-window settings, and evaluation scripts fixed for S-MSCKF and RDIC-MSCKF. Unless otherwise stated, trajectory errors are computed after rigid SE(3) fixed-scale alignment, and repeated runs are summarized at the sequence level. ROVIO and VINS-Mono are included as same-platform reference results because their front-ends and estimator architectures differ from those of the controlled S-MSCKF comparison. The runtime environment and timing scope are summarized in Table 5.

Table 5. Runtime profiling environment and timing scope.

Item	Setting
CPU and memory	Intel Core i9-13900HX CPU, 16 GB RAM
Operating system and ROS	Ubuntu 20.04, ROS Noetic
Build and libraries	Same ROS/catkin C++ workspace, compiler profile, OpenCV installation, and eigen installation for S-MSCKF and RDIC-MSCKF; no method-specific compiler options are applied
Threading and playback	Same estimator-node configuration, rosbag playback setting, input data, and calibration files for both estimators
Timing scope	Internal estimator timers include front-end image processing and feature-tracking callbacks, the back-end visual-inertial update, and timing/trajectory logging required for evaluation; external rosbag playback, ROS message transport latency, and offline trajectory alignment/error computation are excluded
Disabled during profiling	Visualization, diagnostic file output, and nonessential debug messages

4.1. EuRoC Accuracy Comparison

Table 6 reports the EuRoC benchmark ATE statistics. The S-MSCKF and RDIC-MSCKF entries are computed from repeated-run APE summaries under the baseline and RDIC configurations, respectively.

Table 6. ATE statistics on the EuRoC dataset (meters), reported as RMSE/STD/P90.

Seq.	RDIC-MSCKF	S-MSCKF	ROVIO	VINS-Mono
MH_01	0.09/0.03/0.13	0.17/0.07/0.25	0.33/0.17/0.43	0.19/0.07/0.25
MH_02	0.06/0.03/0.09	0.19/0.08/0.27	0.51/0.17/0.75	0.17/0.11/0.23
MH_03	0.17/0.07/0.27	0.25/0.10/0.36	0.34/0.13/0.45	0.20/0.11/0.33
MH_04	0.14/0.06/0.22	0.26/0.09/0.36	1.12/0.52/1.87	0.33/0.10/0.37
MH_05	0.25/0.10/0.38	0.35/0.12/0.49	0.98/0.42/1.23	0.34/0.08/0.37
V1_01	0.07/0.03/0.09	0.08/0.04/0.13	0.13/0.05/0.15	0.07/0.04/0.10
V1_02	0.10/0.05/0.13	0.16/0.07/0.23	0.15/0.05/0.20	0.10/0.03/0.11
V1_03	0.17/0.08/0.26	0.19/0.08/0.28	0.16/0.06/0.24	0.15/0.07/0.21
V2_01	0.09/0.04/0.14	0.07/0.04/0.13	0.27/0.14/0.42	0.09/0.05/0.15
V2_02	0.09/0.04/0.14	0.14/0.06/0.22	0.68/0.22/0.91	0.13/0.07/0.23

Bold values indicate the best value for each RMSE/STD/P90 submetric in a row.

The EuRoC sequences provide complementary accuracy conditions. The machine-hall sequences include larger spatial motion, industrial texture, illumination variation, and longer trajectories, which tend to expose errors associated with marginal feature tracks and heterogeneous measurement quality. The Vicon-room sequences are shorter and are captured in a more controlled indoor environment, but they still contain fast motion, viewpoint changes, and texture-dependent tracking difficulty. V2_03 is excluded because both filtering configurations failed to complete the sequence after persistent stereo-matching loss associated with severe left–right photometric inconsistency.

For RDIC-MSCKF and S-MSCKF, each RMSE, STD, and P90 entry is the arithmetic mean of the corresponding APE-summary metric over five repeated runs for that sequence. RMSE measures the average aligned trajectory error, the APE-summary STD measures dispersion of the aligned position error within the trajectory summaries, and P90 reflects the upper tail while being less sensitive than the maximum to rare events. This table statistic is distinct from the run-to-run standard deviation shown as error bars in Figure 2. The sequence-mean values quoted in the Abstract and Conclusions are computed from the same sequence-level arithmetic means.

Figure 2. EuRoC ATE-RMSE comparison between S-MSCKF and RDIC-MSCKF. Bars show the mean over five repeated runs for each sequence, and error bars show run-to-run standard deviation.

Table 6 shows that RDIC-MSCKF improves over the S-MSCKF baseline on most EuRoC sequences. The difference is especially clear in the machine-hall group, where RDIC-MSCKF reduces all three reported ATE statistics on MH_01–MH_04 and reduces RMSE on MH_05.

RDIC-MSCKF reduces RMSE on V1_01, V1_02, V1_03, and V2_02, whereas S-MSCKF remains better on V2_01. These results indicate that the benefit depends on sequence conditions. Covariance-aware information scaling is most useful when accepted visual measurements exhibit substantial covariance mismatch; on well-conditioned sequences, reduced visual information can slightly degrade accuracy. ROVIO and VINS-Mono provide architectural context, whereas the controlled comparison is between RDIC-MSCKF and S-MSCKF.

Figure 2 provides a sequence-wise visual summary of the repeated-run RMSE values for the two filtering variants. It complements Table 6 by showing whether the numerical differences are consistent across runs. Figure 3 gives a representative MH_02 trajectory-level comparison against ground truth and same-platform VIO references. The trajectory plot uses clearly separated trajectory lines to illustrate the path differences corresponding to the reported ATE values.

Figure 3. Trajectory comparison on the EuRoC MH_02 sequence.

Together, Table 6 and Figure 3 show that RDIC-MSCKF improves the S-MSCKF baseline on most evaluated EuRoC sequences, while the V2_01 result indicates sequence dependence. The following subsections examine update-spectrum, innovation-statistic, and runtime behavior.

4.2. Contextual Comparison with Adaptive MSCKF Methods

To place the improvement of RDIC-MSCKF in the context of recent adaptive MSCKF studies, we further compare the relative accuracy gains reported by representative methods with respect to their own MSCKF baselines. Direct comparison of absolute trajectory errors across different publications is not appropriate because the implementations, parameter settings, sequence subsets, initialization procedures, and evaluation protocols are not identical. We, therefore, use the within-study relative reduction

$$\eta ={\displaystyle \frac{{\overline{E}}_{\mathrm{base}}-{\overline{E}}_{\mathrm{method}}}{{\overline{E}}_{\mathrm{base}}}}\times 100\%,$$

(45)

where ${\overline{E}}_{\mathrm{base}}$ and ${\overline{E}}_{\mathrm{method}}$ denote the sequence-averaged position error reported for the baseline and the corresponding adaptive method, respectively.

Table 7 summarizes the results calculated from the numerical values reported in the original publications. VBU-SMSCKF [27] reduces the mean EuRoC RMSE from 0.1946 m to 0.1261 m over eight reported sequences, corresponding to a 35.2% reduction. The ANGIG-KF2 formulation [28] reduces the reported mean APE from 0.170 m to 0.120 m over eleven EuRoC sequences, corresponding to a 29.4% reduction. QA-MSCKF [30] reduces the mean RMSE from 0.278 m to 0.185 m over ten reported EuRoC sequences, giving a 33.5% reduction. Under the controlled evaluation protocol of the present study, RDIC-MSCKF reduces the sequence-average RMSE from 0.1869 m to 0.1236 m, corresponding to a 33.8% reduction.

Table 7. Within-study position-error reductions in representative adaptive MSCKF methods on EuRoC. Literature values were calculated from the numerical results reported in the corresponding publications.

Method	Metric	Baseline	Seq.	Baseline Mean (m)	Method Mean (m)	Relative Reduction
VBU-SMSCKF [27]	RMSE	S-MSCKF	8	0.1946	0.1261	35.2%
ANGIG-KF2 [28]	APE	Tra-MSCKF	11	0.170	0.120	29.4%
QA-MSCKF [30]	RMSE	S-MSCKF	10	0.278	0.185	33.5%
RDIC-MSCKF	RMSE	S-MSCKF	10	0.1869	0.1236	33.8%

Note: The relative reductions are calculated within each individual study using Equation (45). The absolute values should not be interpreted as a direct cross-paper ranking because the compared studies use different MSCKF implementations, sequence subsets, parameter settings, and trajectory-evaluation protocols. ANGIG-KF2 reports APE, whereas the other methods report RMSE. CAMSCKF is included in the methodological comparison in Table 1, but it is not included in this numerical contextual table because no directly compatible EuRoC aggregate value is used in this revision.

The within-study reductions show that adaptive measurement-uncertainty treatment produces substantial gains across different MSCKF formulations, with reported position-error reductions of approximately 29–35%. RDIC-MSCKF achieves an improvement of comparable magnitude while using a different uncertainty model: tracking-related reliability and local directional identifiability are represented at the observation level, the resulting anisotropic covariance is preserved through landmark elimination, and the remaining batch-level innovation mismatch is subsequently calibrated.

The sequence-wise behavior also provides useful context. VBU-SMSCKF improves all eight EuRoC sequences reported in its original comparison, ANGIG-KF2 reduces the reported APE on ten of eleven sequences, and QA-MSCKF improves RMSE on six of ten sequences. In the present controlled evaluation, RDIC-MSCKF improves RMSE on nine of the ten successfully evaluated EuRoC sequences. These results indicate that the proposed method achieves a within-study gain of comparable magnitude while maintaining relatively broad improvement across the evaluated sequences.

4.3. Component-Wise Ablation Study

To examine the cumulative effect of the proposed mechanisms, we conduct an ablation study on the ten successfully evaluated EuRoC sequences. All configurations use the same S-MSCKF state definition, inertial propagation, camera cloning, sensor calibration, and evaluation protocol. Starting from the S-MSCKF baseline A0, the proposed mechanisms are introduced sequentially along the uncertainty-processing chain, as summarized in Table 8.

Table 8. Configurations used in the component-wise ablation study.

Module	A0	A1	A2	A3	A4
S-MSCKF backbone	✓	✓	✓	✓	✓
Tracking-risk covariance	×	✓	✓	✓	✓
Directional covariance	×	×	✓	✓	✓
Full projected covariance	×	×	×	✓	✓
Batch-innovation calibration	×	×	×	×	✓
Quality-aware feature management	×	×	×	×	✓

A1 introduces observation-dependent tracking-risk covariance. A2 further incorporates directional covariance derived from local photometric information. A3 preserves the full projected covariance after landmark nullspace elimination, allowing the anisotropic observation uncertainty introduced in A2 to remain represented at the feature-block level. Finally, A4 introduces bounded batch-innovation calibration together with quality-aware feature management and corresponds to the complete RDIC-MSCKF configuration.

Table 9 reports the ATE-RMSE of A0–A4. The sequence-average RMSE decreases from 0.1869 m for the S-MSCKF baseline to 0.1236 m for the complete RDIC-MSCKF, corresponding to an overall reduction of 33.8%. The intermediate configurations show that the improvement arises progressively from observation-level uncertainty modeling, covariance propagation, and batch-level calibration.

Table 9. Component-wise ablation results on EuRoC. ATE-RMSE is reported in meters.

Sequence	A0	A1	A2	A3	A4
MH_01	0.170	0.128	0.135	0.088	0.092
MH_02	0.192	0.150	0.144	0.072	0.064
MH_03	0.247	0.231	0.173	0.204	0.172
MH_04	0.258	0.183	0.195	0.120	0.138
MH_05	0.348	0.251	0.247	0.229	0.254
V1_01	0.084	0.078	0.074	0.076	0.067
V1_02	0.161	0.141	0.132	0.120	0.097
V1_03	0.189	0.167	0.173	0.218	0.171
V2_01	0.074	0.071	0.075	0.091	0.091
V2_02	0.140	0.131	0.124	0.079	0.090
Average	0.1869	0.1531	0.1472	0.1297	0.1236

Bold values indicate the best value in each row.

The largest incremental gain is obtained in A1. Introducing tracking-risk covariance reduces the average RMSE from 0.1869 m to 0.1531 m and improves all ten evaluated sequences. This result indicates that feature tracks retained by the standard front-end still exhibit substantial observation-to-observation differences in measurement reliability. Modeling this variability directly in the measurement covariance, therefore, provides a strong correction to the fixed-noise assumption.

A2 further introduces directional uncertainty derived from local photometric information. The average RMSE decreases to 0.1472 m. Its effect is sequence dependent; for example, MH_03 improves from 0.231 m to 0.173 m, whereas several other sequences change only slightly. This behavior is consistent with the directional model, whose influence becomes more pronounced when accepted image patches contain weakly constrained directions. The result shows that observation-dependent scalar scaling alone does not capture all of the uncertainty structure present in visual measurements.

A3 preserves the full covariance generated after landmark nullspace projection. The average RMSE decreases from 0.1472 m to 0.1297 m. Pronounced gains are observed on MH_01, MH_02, MH_04, and V2_02; on MH_02; for example, the RMSE decreases from 0.144 m to 0.072 m. These results demonstrate that once anisotropic observation covariance is introduced, its projected correlation structure can materially affect the information carried by an MSCKF feature block. Preserving and whitening the full projected covariance is, therefore, important for propagating observation-level uncertainty through landmark elimination.

A4 adds the bounded batch-innovation calibration together with quality-aware feature management. The sequence-average RMSE is further reduced to 0.1236 m, with clear improvements on MH_02, MH_03, V1_01, V1_02, and V1_03. This final-stage gain reflects the combined effect of update-level innovation regulation and quality-aware control of the retained visual-constraint population. Because these mechanisms are activated jointly in A4, their individual contributions are not separated in this cumulative ablation.

Tail-Error Behavior

In addition to the average trajectory error, we evaluate the P90 error to examine the upper-tail behavior across the cumulative configurations of the trajectory-error distribution. Table 10 summarizes the sequence-average ATE-RMSE and P90 statistics for A0–A4.

Table 10. Aggregate accuracy and tail-error statistics of the ablation configurations.

Metric	A0	A1	A2	A3	A4
Mean ATE-RMSE (m)	0.1869	0.1531	0.1472	0.1297	0.1236
Mean P90 (m)	0.272	0.226	0.218	0.193	0.185
RMSE reduction (%)	–	18.1	21.2	30.6	33.8
P90 reduction (%)	–	16.9	19.9	29.0	32.0

Bold values indicate the best value in each row.

The P90 results follow the same overall trend as the average trajectory error. Introducing tracking-risk covariance in A1 reduces the mean P90 from 0.272 m to 0.226 m, corresponding to a 16.9% reduction relative to A0. This result indicates that observation-dependent reliability modeling suppresses a substantial portion of the large-error population in addition to improving the average trajectory accuracy.

Directional covariance in A2 further reduces the mean P90 to 0.218 m, giving a cumulative reduction of 19.9%. The additional improvement is moderate, consistent with the structure-dependent nature of directional uncertainty: its effect is concentrated in observations whose local image geometry contains weakly constrained directions.

Preserving the full projected covariance in A3 produces a more pronounced reduction in the upper error tail, decreasing the mean P90 to 0.193 m, or 29.0% below the baseline. Together with the corresponding RMSE reduction, this result indicates that retaining the correlation structure generated by anisotropic observations after landmark elimination affects both the typical trajectory error and its upper tail.

The complete RDIC-MSCKF configuration A4 achieves the lowest mean P90 of 0.185 m, corresponding to an overall reduction of 32.0% relative to A0. The smaller incremental change from A3 to A4 is consistent with the combined role of bounded batch calibration and quality-aware management: they regulate the update scale and retained visual-constraint population rather than uniformly modifying every visual update.

Overall, the RMSE and P90 results exhibit consistent cumulative trends. Tracking-risk covariance provides the largest initial gain, directional covariance offers structure-dependent refinement, full projected-covariance preservation substantially reduces both average and tail errors, and the complete A4 configuration provides the final bounded correction at the update level. The following NIS and NEES analyses further examine how these components affect the statistical behavior of the accepted visual updates.

4.4. Real-World UAV Trajectory Evaluation

To complement the public benchmark, we further evaluate the three VIO configurations on real-world UAV flights collected by the experimental platform shown in Figure 4. The flights provide an outdoor platform-level evaluation of stereo visual-inertial trajectory estimation under illumination variation, airframe vibration, and six-degree-of-freedom motion. The platform is equipped with a forward-facing Intel RealSense D435i stereo camera rigidly mounted on the airframe, and the D435i internal IMU provides the inertial measurements. The stereo image stream is recorded at a resolution of $848\times 480$ pixels and a frame rate of 20 Hz, while the IMU is recorded at 200 Hz. An RTK trajectory is recorded as the external reference for horizontal trajectory evaluation after Sim(2) alignment.

Figure 4. Real-world UAV platform used for the flight experiments. The platform carries a forward-facing Intel RealSense D435i stereo camera with its internal IMU; the recorded stereo image resolution is $848\times 480$ pixels at 20 Hz and the IMU rate is 200 Hz.

The onboard stereo-inertial data are processed by S-MSCKF, RDIC-MSCKF, and VINS-Mono, and the corresponding RTK trajectory is used as the horizontal reference. The RTK latitude, longitude, and altitude measurements are converted to a local ENU frame with the first valid RTK sample as the origin. For trajectory visualization and horizontal error reporting, each estimated trajectory is time-associated with the RTK sequence and aligned to the RTK ENU-XY path by a planar similarity transformation. Because Sim(2) alignment removes planar translation, heading, and scale, these metrics evaluate horizontal path shape rather than metric-scale accuracy.

Figure 5 and Table 11 show that the Sim(2)-aligned RDIC-MSCKF trajectories have smaller aligned horizontal path errors than the S-MSCKF baseline in all three flights. The aligned horizontal RMSE values of RDIC-MSCKF are 3.89 m, 8.34 m, and 2.87 m on UAV-1–UAV-3, respectively, whereas S-MSCKF obtains 17.80 m, 17.76 m, and 3.51 m. VINS-Mono provides an additional architectural reference with RMSE values of 10.53 m, 8.25 m, and 5.43 m. The second flight shows comparable RDIC-MSCKF and VINS-Mono aligned horizontal path error, while the first and third flights show clearer separation between RDIC-MSCKF and the reference methods. The UAV flights, therefore, provide additional outdoor evidence under illumination variation, airframe vibration, and six-degree-of-freedom platform motion.

Figure 5. Real-world UAV trajectory comparison on three flight sequences. The panels labeled UAV-1, UAV-2, and UAV-3 correspond to three UAV runs with RTK reference trajectories.

Table 11. Horizontal trajectory errors after Sim(2) alignment to the RTK reference (meters), reported as RMSE/Mean/P90 after time association and planar similarity alignment to the RTK ENU-XY reference.

Sequence	RDIC-MSCKF	S-MSCKF	VINS-Mono
UAV-1	3.89/3.25/6.10	17.80/16.03/25.68	10.53/9.21/18.87
UAV-2	8.34/7.67/13.17	17.76/15.82/25.36	8.25/7.00/13.98
UAV-3	2.87/2.60/4.45	3.51/3.08/5.34	5.43/5.16/8.01

Bold values indicate the best value for each RMSE/Mean/P90 submetric in a row.

4.5. Finite-Window Update-Spectrum Diagnostics

To examine how retained visual updates affect the numerical geometry of the filtered update, we evaluate finite-window effective-Jacobian spectra on five representative EuRoC sequences: MH_03, MH_04, MH_05, V1_02, and V1_03. This analysis complements ATE by characterizing the spectral structure of the constraints entering the bounded-state filter.

Following the source-resolved diagnostic protocol, visual updates are separated into remove updates formed by terminated feature tracks and prune updates triggered by camera-state sliding-window management. The diagnostic matrix is the effective whitened update Jacobian for the current finite camera-clone window, whose logged window size reaches 20 camera states in prune updates. Let its retained singular values be ${\left\{{\sigma }_k\right\}}_{k=1}^r$ after the diagnostic logger removes numerically ineffective values, with r recorded as ‘full_rank_count’. We define

$$p_k={\displaystyle \frac{{\sigma }_k}{{\sum }_{\mathsf{\ell }=1}^r{\sigma }_{\mathsf{\ell }}}},\mathrm{erank}=\exp \left(-{\displaystyle \sum _{k=1}^r}{p}_k\log p_k\right),{\mathrm{erank}}_{\mathrm{norm}}={\displaystyle \frac{\mathrm{erank}}{\min (M,n_x)}},$$

(46)

where M is the number of update rows and $n_x$ is the active error-state dimension. The condition-number diagnostic is

$$\kappa ={\displaystyle \frac{{\sigma }_{\max }}{{\sigma }_{\min }}},{\log }_{10}\kappa ={\log }_{10}\left({\displaystyle \frac{{\sigma }_{\max }}{{\sigma }_{\min }}}\right),$$

(47)

using the minimum retained singular value reported by the same logger. Higher normalized effective rank and lower ${\log }_{10}\kappa$ indicate a more balanced numerical update spectrum. The violin plots use sequence-balanced frame-level samples. For each method, source, metric, and sequence, 500 samples are drawn with a fixed random seed, using subsampling when more samples are available and sampling with replacement when fewer samples are available. The plotted summaries avoid domination by longer sequences; the equal-sequence median values reported below average the per-sequence medians over the five evaluated sequences.

Figure 6 shows that the two update sources behave differently. For remove updates, RDIC-MSCKF decreases the equal-sequence median normalized effective rank from 0.2393 to 0.1942 and increases the median ${\log }_{10}$ condition number from 4.1692 to 4.6372, indicating a less balanced spectrum. For prune updates, the normalized effective rank is nearly unchanged, moving from 0.01948 to 0.01973, while the lower condition number, 4.3176 versus 4.7569, indicates improved spectral balance. These finite-window spectra characterize the numerical geometry of accepted update blocks.

Figure 6. Source-resolved finite-window update-spectrum diagnostics for S-MSCKF and RDIC-MSCKF on MH_03, MH_04, MH_05, V1_02, and V1_03. Violin shapes show sequence-balanced frame-level distributions; thick and thin vertical segments denote interquartile and 10th–90th percentile ranges, respectively, and white markers denote medians. The upper row reports normalized effective rank, and the lower row reports ${\log }_{10}$ condition number.

4.6. Consistency and Numerical Behavior

4.6.1. Innovation and Covariance Diagnostics

To examine how the proposed components affect the statistical behavior of the visual update, we evaluate A0–A4 on five representative EuRoC sequences: MH_03, MH_04, MH_05, V1_02, and V1_03. Here, A0 denotes the standard S-MSCKF baseline; A1 introduces tracking-risk covariance; A2 further incorporates directional covariance; A3 retains the full projected covariance after landmark elimination; and A4 corresponds to the complete RDIC-MSCKF with bounded batch-innovation calibration and quality-aware feature management.

For each accepted visual update, the normalized innovation squared (NIS) is computed as

$${ϵ}_{\mathrm{NIS},k}={\mathbf{r}}_k^{\top }{\mathbf{S}}_k^{-1}{\mathbf{r}}_k,{\mathbf{S}}_k={\mathbf{H}}_k{\mathbf{P}}_k^{-}{\mathbf{H}}_k^{\top }+{\mathbf{R}}_k,$$

(48)

where ${\mathbf{r}}_k$, ${\mathbf{H}}_k$, ${\mathbf{P}}_k^{-}$, and ${\mathbf{R}}_k$ denote the stacked innovation, measurement Jacobian, prior covariance, and measurement covariance used in the corresponding update. Since the dimension of the stacked visual constraint varies with the number of retained feature blocks, the 95% NIS confidence bound is evaluated using the actual update dimension at each frame. The reported NIS statistics are computed from accepted updates and are, therefore, used as diagnostics of the innovation scale admitted by the filter.

To further examine the relationship between estimated uncertainty and translational error, we report an aligned position block-NEES,

$${ϵ}_{\mathrm{NEES},k}^p={\mathbf{e}}_{p,k}^{\top }{\left({\mathbf{P}}_{pp,k}^{\mathrm{ali}}\right)}^{-1}{\mathbf{e}}_{p,k},$$

(49)

where ${\mathbf{e}}_{p,k}$ is the position error after applying the rigid transformation used for ATE evaluation, and ${\mathbf{P}}_{pp,k}^{\mathrm{ali}}\in {\mathbb{R}}^{3\times 3}$ is the corresponding aligned position covariance block. Given the alignment rotation ${\mathbf{R}}_{\mathrm{ali}}$,

$${\mathbf{P}}_{pp,k}^{\mathrm{ali}}={\mathbf{R}}_{\mathrm{ali}}{\mathbf{P}}_{pp,k}{\mathbf{R}}_{\mathrm{ali}}^{\top }.$$

(50)

Because the alignment is estimated from the complete trajectory, this quantity is used as a translational covariance diagnostic rather than a strict online full-state NEES test.

For the position block-NEES, we report the median normalized by its dimension, the ratio between its 90th percentile and the 95% ${\chi }^2$ bound, and the fraction of samples lying within the corresponding 95% confidence region. A normalized median near one indicates agreement between the typical translational error scale and the estimated covariance scale. Values substantially below one indicate conservative covariance, whereas large upper-tail ratios indicate episodes in which the estimated covariance is small relative to the observed error.

Table 12 reveals a clear change in the accuracy–uncertainty relationship across the ablation sequence. The S-MSCKF baseline A0 achieves an inside95 ratio of 0.948 for the aligned position block, but its normalized median NEES is only 0.075. Its NIS P90/dim is likewise very small at 0.007. Together with an ATE-RMSE of 0.2406 m, these statistics indicate a strongly conservative covariance scale in the evaluated translational subspace.

Table 12. Accuracy and consistency-oriented diagnostics for five representative EuRoC sequences.

Method	ATE-RMSE Avg. (m)	Aligned Position NEES			NIS
		Med./3	P90 Ratio	inside95	P90/dim	inside95
A0	0.2406	0.075	0.528	0.948	0.007	1.000
A1	0.1946	2.284	16.372	0.599	0.574	0.985
A2	0.1840	1.464	28.804	0.705	0.541	0.990
A3	0.1782	1.176	6.666	0.742	0.473	0.995
A4 (RDIC-MSCKF)	0.1664	0.864	4.046	0.768	0.451	0.995

Notes: Bold values indicate the best value for each reported diagnostic metric. The ATE-RMSE column reports the arithmetic mean over the five sequences used in this diagnostic analysis (MH_03, MH_04, MH_05, V1_02, and V1_03), and, therefore, differs from the ten-sequence average reported in the main EuRoC and ablation results. Med./3 is the median aligned position NEES normalized by its three-dimensional position block. The P90 ratio is the 90th percentile of the position NEES divided by ${\chi }_{3,0.95}^2$. The NIS P90/dim value is the 90th percentile of NIS normalized by the corresponding visual-update dimension. inside95 denotes the fraction of samples within the associated 95% confidence region.

Introducing tracking-risk covariance in A1 reduces the ATE-RMSE from 0.2406 m to 0.1946 m. The normalized median position NEES increases to 2.284, showing that observation-dependent reliability modeling substantially changes the information assigned to accepted visual measurements. The accompanying increase in the upper NEES tail indicates that tracking-risk adaptation alone does not fully regulate the covariance behavior of difficult visual updates.

Directional covariance is introduced in A2 to account for local weak and strong image directions. The ATE-RMSE is further reduced to 0.1840 m and the normalized median position NEES decreases from 2.284 to 1.464. The P90 NEES ratio, however, remains large, indicating that directional uncertainty improves the typical covariance scale but does not by itself suppress all high-error tail events.

A3 retains the full feature-block covariance after landmark nullspace projection. This reduces the ATE-RMSE to 0.1782 m and lowers the normalized median position NEES to 1.176. More importantly, the P90 NEES ratio decreases from 28.804 for A2 to 6.666 for A3. The result indicates that preserving the covariance structure generated by anisotropic observations through landmark elimination improves the relationship between observation-level uncertainty modeling and the final MSCKF update.

The complete RDIC-MSCKF configuration A4 further introduces bounded batch-innovation calibration and quality-aware feature management. It achieves the lowest ATE-RMSE of 0.1664 m and a normalized median position NEES of 0.864, which is the closest to unity among the evaluated variants. The P90 NEES ratio is further reduced to 4.046. Although the position inside95 ratio of 0.768 remains below the nominal 95% level, the combined results show a substantially improved tradeoff between trajectory accuracy, typical covariance scale, and high-NEES tail behavior.

The NIS statistics provide a complementary view. A0 exhibits an extremely small normalized innovation scale, consistent with its conservative covariance behavior. A1 and A2 substantially increase the innovation scale as more informative observation-dependent covariance models are introduced. A3 and A4 moderately reduce the NIS upper-tail level while retaining much higher normalized innovation values than A0. Thus, the accuracy improvement of RDIC-MSCKF is not obtained by simply increasing the aggressiveness of the visual update; the later stages of the covariance-processing chain regulate the information introduced by the earlier observation-level adaptations.

Figure 7 further shows that the effects vary across sequences, with the complete RDIC-MSCKF generally reducing excessive position-NEES events while maintaining lower trajectory error. These results support the complementary roles of the proposed components: tracking-risk covariance adapts observation-level reliability, directional covariance captures local anisotropy, full projected covariance preserves this uncertainty through landmark elimination, and batch-innovation calibration regulates the remaining update-level mismatch.

Figure 7. Per-sequence innovation and covariance diagnostics for A0–A4 on representative EuRoC sequences. Each heatmap reports the raw statistic indicated by its color bar. The progression from A0 to RDIC-MSCKF illustrates how observation-level uncertainty modeling, projected-covariance preservation, and batch calibration affect the accuracy–uncertainty relationship of the filter.

The reported quantities should be interpreted as consistency-oriented diagnostics rather than a proof of full-state statistical consistency. In particular, the position block-NEES depends on trajectory alignment, while the NIS statistics are conditioned on accepted visual updates.

4.6.2. Representative Update-Level NIS Behavior

The preceding multi-sequence analysis summarizes the overall innovation and covariance behavior of A0–A4. To further examine how the innovation scale evolves during individual visual updates, we use the EuRoC MH_02 sequence as a representative case and compare the update-level NIS/DoF trajectories of S-MSCKF and RDIC-MSCKF.

For each accepted visual update, the NIS is computed according to Equation (48). Because the residual dimension varies with the number of retained constraints, we report the normalized quantity

$${\overline{\gamma }}_k={\displaystyle \frac{{ϵ}_{\mathrm{NIS},k}}{{\mathrm{DoF}}_k}},$$

(51)

where ${\mathrm{DoF}}_k$ denotes the dimension of the visual residual represented by the corresponding update log after landmark elimination, covariance whitening, feature-level gating, and row compression. The normalized NIS provides a dimension-independent indicator of the innovation scale admitted by each accepted update. Under an ideal locally linearized model with correctly specified innovation covariance, its expected scale is approximately one.

Figure 8 shows the raw update-level NIS/DoF values and their moving averages on MH_02. A moving-average window of 25 accepted updates is used to reveal the temporal trend, while the dashed horizontal line at $\overline{\gamma }=1$ indicates the nominal reference scale.

Figure 8. Representative update-level NIS/DoF behavior on EuRoC MH_02. Raw accepted-update statistics and their 25-update moving averages are shown for S-MSCKF and RDIC-MSCKF. The dashed line denotes the nominal reference $\mathrm{NIS}/\mathrm{DoF}=1$.

The S-MSCKF moving average remains below the unit reference for most of the sequence, consistent with the conservative innovation scale observed in the multi-sequence statistics. RDIC-MSCKF shifts the accepted-update NIS/DoF toward the nominal range over substantial portions of the trajectory, while retaining transient excursions in several intervals. These excursions indicate that the proposed uncertainty treatment does not force the innovation statistic toward a fixed target on every update; instead, the resulting scale reflects the interaction among observation-dependent covariance, projected feature-block covariance, and batch-level calibration.

This temporal behavior complements the aggregate results in Table 12. The baseline exhibits low normalized innovation values together with relatively large trajectory error, whereas RDIC-MSCKF admits a less conservative innovation scale while achieving lower ATE. The bounded batch calibration subsequently limits excessive visual information when the pre-gate innovation population indicates a batch-level scale mismatch.

The NIS/DoF curves should be interpreted as diagnostics of the accepted visual-update population rather than as a standalone proof of full filter consistency. In particular, the plotted statistics are conditioned on feature-level gating and other front-end selection operations.

4.7. Computational Efficiency Analysis

Runtime feasibility is essential for a filtering-based stereo VIO system because the estimator must process each stereo frame while maintaining bounded latency for downstream control and navigation. The runtime comparison on the same platform includes the added uncertainty calculations, quality-aware feature management, and covariance-aware update.

Runtime profiling is performed on five representative EuRoC sequences: MH_02, MH_03, MH_05, V1_02, and V1_03. The first 10 s of each sequence are excluded to reduce initialization and cache-warm-up effects. Front-end and back-end timing records are associated by nearest timestamps within a 12-ms tolerance. For each profiled stereo frame, the per-frame processing time is defined as the sum of the measured front-end and back-end processing times. This definition measures estimator processing demand; rosbag playback, ROS transport latency, and offline trajectory evaluation are excluded. Since EuRoC stereo images are provided at 20 Hz, 50 ms is used as the nominal per-frame processing reference. Aggregate statistics are first computed per sequence and then equally averaged over the five sequences.

Figure 9 shows the net cost of the two configurations. In the back-end, RDIC-MSCKF is slower than S-MSCKF, which is consistent with the additional covariance construction, projected whitening, feature-block statistic evaluation, and bounded batch calibration. In the front-end, RDIC-MSCKF is faster in the plotted sequences because quality-aware feature management changes the number, lifetime, and replenishment pattern of retained visual constraints.

Figure 9. Runtime breakdown of S-MSCKF and RDIC-MSCKF. Blue denotes S-MSCKF and red denotes RDIC-MSCKF. The reported timing includes both uncertainty calculations and changes in the retained visual-constraint population.

Table 13 summarizes the corresponding equal-sequence averages. The mean per-frame processing time decreases from 36.77 ms for S-MSCKF to 26.32 ms for RDIC-MSCKF, and the P95 value decreases from 61.69 ms to 37.33 ms. The front-end accounts for most of the reduction, decreasing from 30.24 ms to 18.56 ms on average, whereas the back-end mean increases from 6.44 ms to 7.76 ms. The lower total runtime reflects the combined effect of additional back-end computation and a reduced front-end feature-processing load.

Table 13. Equal-sequence average runtime statistics for five representative EuRoC sequences.

Metric	S-MSCKF	RDIC-MSCKF	Relative Change
Per-frame mean (ms)	36.77	26.32	$-28.4\%$
Per-frame P95 (ms)	61.69	37.33	$-39.5\%$
Front-end mean (ms)	30.24	18.56	$-38.6\%$
Back-end mean (ms)	6.44	7.76	$+20.5\%$
Coefficient of variation	0.387	0.254	$-34.4\%$
Frames over 50 ms (%)	18.58	0.01	$-99.9\%$

Bold values indicate the better value for each metric.

Figure 10 further reports the frame-by-frame timing behavior. The yellow dashed line provides a 50 ms reference corresponding to a 20 Hz stereo-frame budget. Faint curves show individual-frame measurements, which can contain occasional spikes, and solid curves show moving-average trends. The RDIC-MSCKF moving-average curves remain below the 50 ms reference on the plotted sequences, and Table 13 reports a 26.32 ms mean, a 37.33 ms P95, and 0.01% of profiled frames exceeding 50 ms. These statistics support real-time compatibility on the evaluated platform while still allowing isolated raw-frame spikes.

Figure 10. Per-frame processing time of S-MSCKF and RDIC-MSCKF. Blue denotes S-MSCKF, red denotes RDIC-MSCKF, and the yellow dashed line marks the 50 ms reference budget.

5. Discussion

5.1. Interpretation of the Integrated Uncertainty Model

RDIC-MSCKF treats accepted stereo tracks as a heterogeneous measurement set. Tracking diagnostics describe dynamic and matching reliability, including motion blur, illumination variation, partial occlusion, stereo imbalance, and disparity discontinuities. Local photometric information describes directional identifiability, especially near edges and weak-gradient regions where the aperture problem makes two-dimensional localization anisotropic. Batch calibration then acts after landmark elimination and whitening, where the relevant question is whether the current population of feature-block innovations is compatible with the assumed visual update model. These three levels address different sources of uncertainty while preserving the MSCKF propagation, cloning, and landmark-elimination structure.

Hard gates and RANSAC-type tests remain necessary for clearly inconsistent feature blocks. RDIC-MSCKF adds a covariance path for measurements that pass these tests but still differ in terms of reliability or directional precision. This differs from residual-only robust losses: tracking and photometric diagnostics are formed before the final Kalman update and are explicitly carried through the covariance algebra.

5.2. Accuracy and Information Scaling

The penalty-only FIM mapping keeps high-texture patches at the baseline noise level unless additional calibration evidence is available. This reduces the risk of overconfident visual updates, while also attenuating some useful constraints in favorable scenes. The V2_01 result is consistent with this trade-off: S-MSCKF remains slightly better when the fixed-covariance baseline is already well matched to the sequence conditions.

The bounded batch inflation has a related effect. Because ${\alpha }_{★}^2\ge 1$, the batch step reduces or preserves visual information for a fixed prior, linearization point, and accepted constraint set. Consequently, the method may reduce update strength when elevated batch statistics arise from prior mismatch or correlated observations rather than measurement noise alone.

5.3. Computational Behavior

The runtime results show a redistribution of computation. RDIC-MSCKF adds back-end operations for covariance construction, projected whitening, feature-block statistics, and modal updating. At the same time, quality-aware feature management changes the retained track population and reduces the front-end feature-processing load in the profiled sequences. The measured per-frame processing time, therefore, reflects both effects: a higher back-end mean and a lower front-end mean, with a lower total mean and P95 on the evaluated platform.

For UAV inspection platforms, this behavior is relevant because localization must share onboard computation with perception, payload processing, and task-management processes. A bounded-state estimator with predictable memory and update size is attractive when pose estimates are used for navigation, spatial registration of multisensor measurements, repeatable survey execution, and data fusion. The UAV flight experiments provide platform-level validation under outdoor motion, and future task-oriented tests will extend the evaluation to field campaigns with the complete multisensor payload.

5.4. Limitations

The uncertainty model is based on structured tracking-reliability and directional-information cues. The median/MAD normalization, bounded exponential mapping, and penalty-only FIM rule provide conservative covariance behavior, while the risk weights, lifetime tolerance, and photometric scale remain design choices. The local photometric matrix is derived from weighted Gauss–Newton or observed-information terms and leaves several image-measurement effects for future modeling, including rolling exposure, non-Lambertian appearance, local depth discontinuities, and subpixel tracking bias. The stereo covariance further uses one shared two-dimensional covariance block for the left and right camera observations.

The batch quantile criterion uses feature-block innovations that share the same prior state, often share image regions or motion conditions, and may be affected by common front-end error sources. The median feature-block degree of freedom provides a practical scalar reference for heterogeneous blocks, while the bounded search interval defines the maximum admissible inflation. Future work should calibrate the risk and FIM mappings from empirical stereo reprojection statistics, estimate separate left/right covariance terms when sufficient evidence is available, model feature-block dependence more explicitly, and evaluate the estimator in broader UAV-based multisensor inspection missions.

6. Conclusions

This paper presented RDIC-MSCKF, which incorporates observation-dependent risk and directional covariance, covariance-preserving landmark elimination, and bounded batch-innovation calibration into a stereo MSCKF visual update. The theoretical analysis shows that the resulting covariance operations preserve the bounded-state MSCKF propagation and clone structure while maintaining controlled visual information flow under the stated assumptions.

RDIC-MSCKF reduced ATE RMSE on nine of the ten evaluated EuRoC sequences. Averaged over five runs, the sequence-mean RMSE decreased from 0.1869 m to 0.1236 m, while the sequence-mean P90 error decreased by 32.0%. On five representative sequences, the method achieved a mean processing time of 26.32 ms and a P95 of 37.33 ms, supporting 20 Hz stereo processing on the evaluated platform. On MH_02, the moving-average NIS/DoF was closer to the nominal unit reference.

In three real-world UAV flights with RTK reference trajectories, RDIC-MSCKF obtained lower Sim(2)-aligned horizontal RMSE than the S-MSCKF baseline, with values of 3.89 m, 8.34 m, and 2.87 m on the three reported flights. These field trajectories provide real-world flight evidence under outdoor illumination, airframe vibration, and six-degree-of-freedom motion.

Future work will investigate the probability calibration of the observation model, independent left–right covariance estimation, correlated batch statistics, and task-oriented inspection flights with the complete multisensor payload.