Semantic-Guided Spatial and Temporal Fusion Framework for Enhancing Monocular Video Depth Estimation

Abstract

Despite advancements in deep learning-based Monocular Depth Estimation (MDE), applying these models to video sequences remains challenging due to geometric ambiguities in texture-less regions and temporal instability caused by independent per-frame inference. To address these limitations, we propose STF-Depth, a novel post-processing framework that enhances depth quality by logically fusing heterogeneous information—geometric, semantic, and panoptic—without requiring additional retraining. Our approach introduces a robust RANSAC-based Vanishing Point Estimation to guide Dynamic Depth Gradient Correction for background separation, alongside Adaptive Instance Re-ordering to clarify occlusion relationships. Experimental results on the KITTI, NYU Depth V2, and TartanAir datasets demonstrate that STF-Depth functions as a universal plug-and-play module. Notably, it achieved a 25.7% reduction in Absolute Relative error (AbsRel) and significantly enhanced temporal consistency compared to state-of-the-art backbone models. These findings confirm the framework’s practicality for real-world applications requiring geometric precision and video stability, such as autonomous driving, robotics, and augmented reality (AR).

1. Introduction

Monocular Depth Estimation (MDE) is a fundamental task in computer vision, aiming to predict dense depth maps from a single RGB image. It serves as a critical component in various applications, including autonomous driving, robotics, and augmented reality (AR), where understanding the 3D geometry of a scene is essential. With the advent of deep learning, MDE has achieved remarkable progress, evolving from early supervised methods to recent self-supervised and foundation model-based approaches [1,2,3].

Recently, deep learning techniques have extensively demonstrated their efficacy in image processing and analysis tasks. Empirical studies utilizing advanced architectures for complex vision tasks highlight the high accuracy and practicality of deep learning in unstructured environments [4,5,6]. This evidence strongly supports the validity of applying deep learning-based approaches to high-level vision tasks such as Monocular Depth Estimation (MDE).

However, despite these advancements, applying image-based MDE models directly to video sequences introduces significant challenges. Since standard MDE models perform inference on each frame independently, they often fail to maintain temporal consistency, leading to severe flickering artifacts. Furthermore, in texture-less regions such as roads or walls, MDE models frequently suffer from scale ambiguity and geometric distortions due to the lack of distinct visual features. While recent video-based depth estimation methods attempt to address these issues by leveraging temporal information across multiple frames [7,8], they typically require heavy computational resources or extensive retraining, limiting their flexibility as a general-purpose solution.

To address these limitations without the need for retraining or heavy architectural changes, we propose STF-Depth (Spatial-Temporal Fusion Depth), a novel post-processing framework designed to serve as a universal plug-and-play module. STF-Depth enhances the output of any off-the-shelf MDE model by logically fusing heterogeneous information—geometric constraints, semantic context, and instance-level details (Figure 1). Our approach introduces a robust RANSAC-based Vanishing Point (VP) Estimation to guide Dynamic Depth Gradient Correction, effectively resolving scale ambiguities in background regions. Additionally, we employ Adaptive Instance Re-ordering based on panoptic segmentation to clarify occlusion relationships and sharpen object boundaries. Finally, an efficient temporal fusion module ensures stability across video frames.

The main contributions of this paper are summarized as follows:

• We propose STF-Depth, a training-free post-processing framework that significantly enhances the geometric accuracy and temporal stability of existing MDE models.
• We introduce a Depth-Stratified Vanishing Point Estimation method combined with Semantic-Guided Spatial Fusion, which robustly corrects perspective distortions in complex environments.
• Extensive experiments on the NYU Depth V2, KITTI, and TartanAir datasets demonstrate the effectiveness of our approach. Notably, STF-Depth achieved a 25.7% reduction in Absolute Relative error (AbsRel) and significantly improved temporal consistency compared to state-of-the-art backbone models.

2. Related Work

2.1. Monocular Depth Estimation

Monocular Depth Estimation (MDE), the task of estimating pixel-wise depth from a single image, has evolved significantly with the advent of Deep Learning. Early research primarily utilized CNN-based encoder-decoder architectures to treat depth estimation as a simple regression problem [1]; however, these methods often focused on local features, missing global context. To address this, recent approaches have adopted Vision Transformers (ViTs) to learn global dependencies across the entire image [9]. Notably, state-of-the-art models like MiDaS and the Dense Prediction Transformer (DPT) demonstrate robust generalization capabilities by “mixing datasets” during training, enabling them to generate high-quality depth maps across diverse indoor and outdoor environments rather than being limited to specific domains [10,11]. Furthermore, advancements in robust training frameworks under noisy labels, such as PSSCL [12], contribute to the increasing reliability of computer vision models.

In this study, these SOTA MDE models serve as the backbone without retraining, providing the initial depth values for our framework. However, since these models rely solely on textural and geometric features, they inherently suffer from “boundary mismatch” limitations, often failing to clearly distinguish semantic boundaries or blending foreground objects with the background. Therefore, instead of using their outputs as the final result, we utilize them as a “raw geometric source” to be fused with semantic segmentation information. This approach allows us to leverage the strong distance estimation capabilities of existing models while compensating for their geometric ambiguities using external semantic knowledge.

2.2. Semantic and Panoptic Segmentation

Image segmentation is a core technology for scene understanding in computer vision, classifying object classes at the pixel level. Semantic segmentation models, such as DeepLabV3, effectively separate foreground from background by capturing multi-scale context through mechanisms like atrous convolution [13]. More recently, this field has expanded to Panoptic Segmentation, which unifies semantic and instance segmentation [14]. Models like OneFormer simultaneously identify amorphous “stuff” regions (e.g., sky, road) and countable “thing” instances (e.g., person, car), assigning a unique ID and semantic meaning to every visual element in the image [15].

In this study, these segmentation models play a crucial role as guides for restructuring the spatial layout of depth maps. As illustrated in Figure 2, while the geometric source (b) generated by the depth estimation model provides “distance” information but lacks clear object boundaries, the semantic source (c) and panoptic source (d) generated by segmentation models precisely recognize “object shapes” and “individual instances” despite lacking depth data. We leverage these complementary characteristics of heterogeneous information to project the clear boundary information provided by segmentation models into the depth space, thereby restoring the morphological integrity of objects that depth models often miss.

2.3. Video Depth Estimation and Temporal Consistency

The major challenge in estimating depth from video sequences is the temporal inconsistency, or flickering, caused by processing each frame independently. Existing studies have largely addressed this by calculating Optical Flow to warp and align depth values between adjacent frames [16,17] or by introducing sequence models like ConvLSTM and Video Transformers to explicitly learn temporal correlations. For instance, ManyDepth [18] leverages multi-view geometry for self-supervised learning, while recurrent approaches like Recurrent Depth [19] integrate temporal information through memory units. Recently, foundation models trained on large-scale datasets, such as Video Depth Anything [20], have set new performance standards in consistent depth estimation. However, these methods typically incur prohibitive computational costs during inference, hindering real-time performance, or require the burden of retraining models on domain-specific video datasets [21,22,23].

In contrast, this study proposes an efficient Temporal Fusion strategy that eliminates the need for complex optical flow computations or retraining. Rather than simply blending pixels, our approach first secures spatial stability through geometry-aware constraints based on vanishing points and object attributes. This allows us to demonstrate that a relatively lightweight window-based temporal smoothing is sufficient to achieve robust temporal consistency. This presents a practical balance between computational efficiency and visual stability, offering a lightweight alternative to existing computationally intensive video depth estimation models.

Table 1. Comparison of video depth estimation approaches. Our method (STF-Depth) offers a plug-and-play solution without the need for training or heavy optical flow computation.

Method	Core Mechanism	Training?	Opt. Flow	Main Characteristic
ManyDepth [18]	Multi-view Geometry	Yes	No	Self-supervised learning
Recurrent Depth [19]	RNN (ConvLSTM)	Yes	No	Temporal feature learning
Video Stabilizer [24]	Optimization	Yes	Yes	Test-time optimization
Video Depth Anything [20]	Foundation Model	Yes	No	Large-scale Pre-training
STF-Depth (Ours)	Heterogeneous Fusion	No	No	Plug-and-Play Post-processing

summarizes the comparison between existing video depth estimation methods and our proposed framework.

3. Methodology

The proposed STF-Depth is a modular post-processing framework designed to maximize the performance of monocular depth estimation in video sequences. As illustrated in Figure 3, the system initiates by performing parallel extraction of complementary heterogeneous information from a single input RGB video frame $I_t$. Leveraging three pre-trained models, the framework generates an Initial Depth Map ($D_{raw}$) representing geometric structures, a Semantic Segmentation Map ($M_{seg}$) defining regional semantics, and a Panoptic Segmentation Map ($M_{pan}$) identifying individual object instances. While $D_{raw}$ provides global distance information but suffers from boundary degradation in texture-less regions, $M_{seg}$ and $M_{pan}$ lack depth data but offer precise object boundaries and instance identification cues, thereby compensating for geometric ambiguities.

These extracted inputs are refined through a sequential fusion pipeline. In the first stage, Spatial Fusion, the system robustly estimates the Vanishing Point (VP) by analyzing the statistical properties of the initial depth map and performs Dynamic Depth Gradient Correction based on this VP to physically separate background and foreground layers. Furthermore, it maximizes spatial accuracy by re-ordering the depth sequence of objects, integrating panoptic information with the distance to the vanishing point. In the subsequent Temporal Fusion stage, efficient smoothing is applied to the spatially refined frame sequence to suppress visual flickering. Through this multi-stage process, STF-Depth generates a high-quality depth map $D_{final}$ that is both geometrically precise and temporally stable, without requiring any additional retraining.

3.1. Heterogeneous Information Extraction

The initial stage of the proposed framework involves the parallel extraction of heterogeneous information types with complementary characteristics from a single input video frame $I_t$. Adhering to a model-agnostic design, our system can employ any pre-trained state-of-the-art (SOTA) monocular depth estimation model as a backbone to capture the 3D geometric structure of the scene. This depth estimator analyzes texture gradients and perspective cues within the input image to generate a pixel-wise relative inverse depth map, denoted as $D_{raw}^t$. While $D_{raw}^t$ excels at inferring the global distance hierarchy on the 2D image plane, it inherently suffers from “depth bleeding” artifacts—where object boundaries collapse in regions with uniform textures or complex lighting. In this study, $D_{raw}^t$ is defined as the “Raw Geometric Source,” serving as the fundamental baseline for the subsequent fusion process.

To structurally control these geometric uncertainties, the framework secures additional semantic cues independent of depth information. A general semantic segmentation model is applied first to generate a semantic segmentation map $M_{seg}^t$, which performs binary classification of pixels into foreground and background. Although $M_{seg}^t$ does not contain distance information, it explicitly defines object boundaries. This map serves as an essential Spatial Mask in subsequent stages, particularly for distinguishing between background regions requiring Dynamic Depth Gradient Correction and object regions that must be preserved.

Furthermore, to determine precise occlusion relationships, we utilize a panoptic segmentation model in parallel. Since semantic segmentation is insufficient for separating overlapping objects or analyzing detailed object attributes, we use the panoptic model to extract not only the instance map $M_{pan}^t$ but also metadata including Class Labels and Confidence Scores for each instance. Crucially, the attribute distinction between countable “Things” (e.g., cars, pedestrians) and amorphous “Stuff” (e.g., sky, walls) functions as a decisive structural prior. This information directs the adaptive offset strategy in the subsequent re-ordering phase, determining whether to geometrically “pull” or “push” specific depth layers.

3.2. Semantic-Guided Spatial Fusion

The initial depth map ($D_{raw}$) provides continuous pixel-wise distance information but suffers from geometric ambiguities, such as the blending of foreground and background or unclear occlusion relationships. To address these issues, we propose a three-step spatial fusion algorithm that combines the statistical properties of the depth map with external semantic information.

3.2.1. Depth-Stratified and RANSAC-Based Vanishing Point Estimation

A prerequisite for accurate spatial correction is identifying the Vanishing Point (VP), the geometric center of the scene. Unlike previous approaches that often rely on Hough transforms or require prior camera calibration parameters, we propose an automatic estimation technique called Depth-Stratified Analysis, which exploits the intrinsic geometric properties of the depth map itself without any external priors.

• Quantization: The initial depth map values are normalized and divided into N uniform depth layers. This effectively discretizes the continuous depth space into multiple analyzeable cross-sections.
• Centroid Extraction: For each depth layer, we generate a pixel mask and calculate image moments to determine the spatial centroid $C_k$, which represents the center of density for that layer.
• RANSAC Regression: Due to imperfections in the initial depth map or complex object arrangements, spatial centroids of certain layers may deviate from the main perspective trajectory. To address this, we apply the RANSAC algorithm to the set of extracted centroids $\mathcal{C}$ and the representative coordinate of the Max. Depth Region, denoted as $P_{deep}$. RANSAC effectively filters out outliers—such as dynamic objects or clutter that do not align with the global perspective—and derives an optimal trend line that represents the dominant geometric structure of the scene.

  $${\theta }^{*}=\underset{m,b}{argmax}{\displaystyle \sum _{p\in \mathcal{P}}}⊮\left({\displaystyle \frac{|m{x}_p-y_p+b|}{\sqrt{m^2+1}}}<ϵ\right)$$
  This RANSAC-based approach ensures robustness against noise, allowing for stable VP estimation even in unstructured settings where geometric cues might be partially obscured.
• Vanishing Point Determination: The final Vanishing Point $VP$ is defined as the point on the derived optimal trend line $L_{{\theta }^{*}}$ that minimizes the Euclidean distance to $P_{deep}$. Here, to ensure robustness against local depth artifacts (e.g., sensor noise or reflections), $P_{deep}$ is calculated as the spatial centroid of the top 5% of pixels with the largest depth values (i.e., the Max. Depth Region visualized in Figure 4), rather than relying on a single maximum point.

  $$VP=\underset{v\in L_{{\theta }^{*}}}{argmin}{\parallel v-P_{deep}\parallel }_2$$

  (1)

3.2.2. Foreground Separation via Dynamic Gradient Correction

Based on the estimated VP, we physically separate the depth layers of the background and foreground. Monocular depth estimation models often merge foreground and background into a single plane in regions with similar textures. To resolve this, we perform Dynamic Depth Gradient Correction reflecting the global distribution characteristics of the depth map.

First, we calculate the depth gap between the minimum ($d_{min}$) and maximum ($d_{max}$) values of the depth map and dynamically derive the slope a and intercept b of the correction equation, weighted by the presence of the VP. Then, using the semantic segmentation map $M_{seg}$ as a mask, we apply a linear correction specifically to Foreground regions, distinct from the background.

$$D_{sep}\left(p\right)=D_{raw}\left(p\right)+(a·D_{raw}\left(p\right)+b)·\mathbb{I}(p\in \mathrm{Foreground})$$

(2)

This process performs a foundational layer separation by rescaling the depth values of foreground objects proportional to the overall depth range, ensuring they secure an independent depth volume without blending into the background.

3.2.3. Instance & VP-Distance Aware Re-Ordering

Going beyond simple separation, we perform an advanced enhancement step that simultaneously considers panoptic metadata and geometric spatial information to establish precise occlusion relationships. Figure 5 conceptually illustrates this ’Push-Pull’ strategy. This stage consists of three key mechanisms:

• VP-Distance Weighting: According to linear perspective, depth perception sensitivity varies between the area around the VP and the periphery. We calculate the Euclidean distance $dist(k,VP)$ between the center of each instance k and the VP. We generate a distance-proportional offset that assigns a larger weight based on the distance from the VP, performing stereoscopic correction according to the position within the frame.
• Class Attribute Adaptation (Push-Pull Strategy): We analyze panoptic labels to distinguish whether an instance is a “Thing” (countable object, e.g., person, car) or “Stuff” (amorphous background element, e.g., sky, wall). We adopt a Dual Strategy: for “Stuff,” the calculated offset is multiplied by a negative value (−1) to push the depth layer backward, while for “Things,” a positive offset is maintained to pull it forward.
• Overlap-based Penalty Injection: We detect spatial overlaps by analyzing the bounding boxes of instances. For overlapping pairs, we confirm the front-to-back relationship by comparing mean depths and inject an additional penalty into the overlapping region. This penalty is calculated by applying an amplification factor (1.5) to the depth difference $\Delta d$ between the two objects.

  $$D_{enh}\left(p\right)=D_{enh}\left(p\right)+\Delta d·1.5,\forall p\in (Regio{n}_{front}\cap Regio{n}_{rear})$$

  (3)

This multidimensional re-ordering process sharpens object boundaries and physically clarifies the front-to-back relationships of overlapping objects.

3.3. Efficient Temporal Fusion

Although the geometric structure and object boundaries of the depth maps have been significantly improved through the preceding spatial fusion steps, temporal inconsistencies—specifically, flickering artifacts—remain due to the independent processing of each frame. While existing video depth estimation studies have employed Optical Flow or Recurrent Neural Networks (RNNs) to track pixel-wise motion, these methods incur prohibitive computational costs, hindering real-time performance. To secure practical stability while maintaining the lightweight architecture of STF-Depth, we propose a window-based Weighted Moving Average strategy.

Our temporal fusion algorithm suppresses high-frequency noise by blending information from the adjacent past ($t-1$) and future ($t+1$) frames, while centering on the depth information of the current frame t. The depth map $D_t^{\left(k\right)}$ at the k-th iteration is updated as follows:

$$D_t^{\left(k\right)}=(1-2\lambda )D_t^{(k-1)}+\lambda (D_{t-1}^{(k-1)}+D_{t+1}^{(k-1)})$$

(4)

Here, $\lambda$ denotes the temporal smoothing coefficient. In this study, we determined the optimal value of $\lambda =0.1$ through sensitivity analysis, achieving the best trade-off between effective flickering suppression and the minimization of motion artifacts such as ghosting. Furthermore, by repeating this process for 2 iterations, we achieved an optimal balance that smooths out abrupt depth fluctuations while maximizing stability. This approach operates with a low complexity of $O\left(N\right)$, significantly reducing visual fatigue in video sequences without imposing additional computational burdens. As demonstrated in Figure 6, this method effectively smooths out high-frequency depth fluctuations, providing a stable and coherent output.

4. Experiments

4.1. Experimental Setup

4.1.1. Datasets

To verify the robust performance and generalization capability of the proposed framework across diverse imaging environments, we employed three representative datasets with distinct geometric characteristics: NYU Depth V2, KITTI, and TartanAir.

First, we utilized the NYU Depth V2 dataset for indoor environment evaluation [25]. This dataset consists of 464 diverse indoor scenes (e.g., bedrooms, offices, basements) captured using a Microsoft Kinect. Indoor scenes typically feature complex arrangements of furniture and clutter with short-range depths, making the handling of occlusions and precise boundary delineation critical. We used this dataset to focus our analysis on the instance re-ordering and spatial refinement capabilities of STF-Depth in cluttered settings.

Second, we used the KITTI dataset for outdoor environment evaluation [26]. This dataset comprises road driving sequences collected in autonomous driving scenarios, providing sparse depth information acquired via LiDAR sensors as Ground Truth. The KITTI dataset features distinct geometric structures with clear separations between the sky and road, along with numerous dynamic objects such as vehicles and pedestrians. Thus, it provides an optimal environment for validating the proposed vanishing-point-based background-foreground separation algorithm and temporal consistency in structured road scenes.

Third, to address the reviewer’s concern regarding performance in non-road or weak-perspective settings, we incorporated the TartanAir dataset [27]. Unlike KITTI, which is limited to planar motion on roads, TartanAir is a large-scale photorealistic virtual dataset collected via simulated drone flights. It features unstructured environments (e.g., abandoned factories, Japanese alleys, fantasy scenes) and aggressive 6-DoF camera motions. We selected this dataset to challenge our framework in multi-scale environments where linear perspective cues are irregular or unstable, thereby verifying the robustness of our VP estimation and temporal fusion strategies under complex geometric conditions.

4.1.2. Evaluation Metrics

To objectively validate the performance of the proposed framework, we adopted standard quantitative metrics widely established in the monocular depth estimation literature [1]. Specifically, we utilized Absolute Relative Error (AbsRel), Squared Relative Error (SqRel), Root Mean Squared Error (RMSE), Root Mean Squared Logarithmic Error (${\mathrm{RMSE}}_{log}$), and Threshold Accuracy ($\delta <{1.25}^k$ for $k=1,2,3$).

In addition to static accuracy, evaluating temporal stability is crucial for video sequences. We measured the Mean Absolute Similarity (MAS) to quantify the consistency of depth predictions between adjacent frames. For all error metrics (AbsRel, SqRel, RMSE, ${\mathrm{RMSE}}_{log}$, and MAS), lower values indicate better performance, whereas for the accuracy metrics (${\delta }_k$), higher values denote superior performance. Detailed mathematical definitions of these standard metrics are omitted here for brevity and can be found in [1,2].

4.1.3. Implementation Details

The hardware and software specifications used for implementing the proposed framework are summarized in

Table 2. Hardware and software specifications used in the experiments.

Category	Specification
Hardware
CPU	AMD EPYC 7742 64-Core Processor × 2
GPU	NVIDIA V100 (32GB) × 8
RAM	1.0 TiB DDR4 ECC
Software
OS	Ubuntu 20.04.6 LTS (Focal Fossa)
Python	3.11.13
PyTorch	2.8.0
CUDA	12.8
Key Libraries
OpenCV	4.12.0.88
Transformers	4.57.1
Scikit-learn	1.7.2
NumPy	2.2.6
Pandas	2.3.3

. The system was implemented in a Python 3.11 environment using the PyTorch 2.8.0 library, and all experiments were conducted on a workstation equipped with eight NVIDIA V100 GPUs.

We employed pre-trained SOTA models as backbones for heterogeneous information extraction. For depth estimation, we utilized MiDaS (DPT-Large), Depth Anything V2, Marigold, and ZoeDepth to validate the generalizability of our framework across different architectures [11,28,29,30,31]. For semantic segmentation, we used DeepLabV3 (ResNet-101 backbone) [13], and for panoptic segmentation, OneFormer (Swin-Large backbone) was employed [15,32]. The RANSAC algorithm for vanishing point estimation was implemented using the scikit-learn library [33].

Key hyperparameters for the Spatial Fusion stage were determined empirically. For Dynamic Depth Gradient Correction, the slope coefficients were adaptively set to 0.1 or 0.05 depending on the presence of a vanishing point. For Instance Re-ordering, the instance confidence threshold was set to 0.7 to exclude uncertain predictions, and the overlap penalty amplification factor was set to 1.5. In the Temporal Fusion stage, we applied a smoothing factor of $\lambda =0.1$ with 2 iterations for the weighted moving average. This configuration effectively suppressed temporal flickering while minimizing motion artifacts such as ghosting.

4.2. Ablation Study

The proposed framework is designed to maximize depth estimation performance by sequentially integrating geometric, semantic, and temporal information. In this section, we conduct an ablation study to isolate and quantify the individual contribution of each core module to the overall performance improvement, thereby validating the rationale behind our fusion pipeline design. To achieve this, we structured the experiment into four progressive cases, incrementally adding each function to the baseline model.

• Method A (Baseline): The original output of the pre-trained monocular depth estimation model without any post-processing ($D_{raw}$).
• Method B (+ Separation): Baseline with the Background-Foreground Depth Separation (Section 3.2.2) module applied. This stage corrects background perspective and eliminates floating artifacts using semantic masks and vanishing point estimation.
• Method C (+ Re-ordering): Method B with the addition of the Instance-Aware Depth Re-ordering (Section 3.2.3) module. This stage rearranges the depth order of overlapping objects and injects separation margins using panoptic information.
• Method D (Full STF-Depth): The final framework integrating the Efficient Temporal Fusion (Section 3.3) module into Method C.

Through this stepwise evaluation, we demonstrate how the logical fusion of heterogeneous information sources—rather than a mere combination of models—synergistically reduces errors (RMSE, AbsRel) and enhances visual completeness.

4.2.1. Baseline Only

Table 3. Quantitative evaluation of baseline monocular depth estimation models without post-processing (Model A). Values are reported with six decimal places for precision.

Model	Dataset	AbsRel ↓	SqRel ↓	RMSE ↓	${\mathrm{RMSE}}_{\mathit{log}}$ ↓	$\mathit{\delta }<1.25$↑	$\mathit{\delta }<{1.25}^2$↑	$\mathit{\delta }<{1.25}^3$↑
MiDaS	NYU-D v2	0.270816 ± 0.178069	0.292906 ± 0.307112	0.830082 ± 0.433434	0.318872 ± 0.172922	0.425967 ± 0.494489	0.718066 ± 0.449941	0.870816 ± 0.335403
	KITTI	0.436433 ± 0.274844	0.870986 ± 0.791604	1.497213 ± 0.680617	0.458983 ± 0.229663	0.264934 ± 0.441298	0.499906 ± 0.500000	0.709364 ± 0.454056
	TartanAir	0.452140 ± 0.295110	0.915230 ± 0.825540	1.582440 ± 0.710320	0.475600 ± 0.248100	0.245800 ± 0.452200	0.485110 ± 0.512000	0.695420 ± 0.468800
Depth Anything V2	NYU-D v2	0.243586 ± 0.162558	0.236911 ± 0.250996	0.744938 ± 0.394201	0.288479 ± 0.158195	0.472124 ± 0.499222	0.765847 ± 0.423468	0.908563 ± 0.288230
	KITTI	0.135138 ± 0.087503	0.084172 ± 0.092535	0.504825 ± 0.285906	0.155802 ± 0.083486	0.777057 ± 0.416220	0.999123 ± 0.000152	0.999892 ± 0.000021
	TartanAir	0.265410 ± 0.178820	0.285550 ± 0.290140	0.785220 ± 0.412500	0.315600 ± 0.182400	0.455200 ± 0.512000	0.745800 ± 0.441500	0.895300 ± 0.302100
Marigold	NYU-D v2	0.204938 ± 0.132109	0.158657 ± 0.165954	0.617202 ± 0.330800	0.240258 ± 0.128866	0.551657 ± 0.497324	0.857746 ± 0.349311	0.965482 ± 0.182556
	KITTI	0.091332 ± 0.059017	0.037609 ± 0.041580	0.349614 ± 0.204945	0.109882 ± 0.059843	0.950034 ± 0.217875	0.999215 ± 0.000112	0.999912 ± 0.000015
	TartanAir	0.225600 ± 0.145200	0.185440 ± 0.190300	0.655800 ± 0.360500	0.265100 ± 0.142200	0.515400 ± 0.512300	0.825600 ± 0.370800	0.945100 ± 0.205400
ZoeDepth	NYU-D v2	0.039138 ± 0.027334	0.005755 ± 0.006771	0.121080 ± 0.071259	0.047400 ± 0.027079	0.998112 ± 0.000112	0.999221 ± 0.000051	0.999889 ± 0.000012
	KITTI	0.054818 ± 0.037837	0.015039 ± 0.018982	0.228706 ± 0.144699	0.065783 ± 0.037114	0.998889 ± 0.000089	0.999551 ± 0.000021	0.999912 ± 0.000009
	TartanAir	0.068520 ± 0.045100	0.025400 ± 0.028800	0.255600 ± 0.160200	0.078900 ± 0.045500	0.985200 ± 0.025100	0.995100 ± 0.010200	0.998200 ± 0.005100
Video Depth Anything	NYU-D v2	0.157935 ± 0.097403	0.085126 ± 0.081624	0.447309 ± 0.223109	0.176227 ± 0.090829	0.706503 ± 0.455364	0.989317 ± 0.102804	0.999912 ± 0.000015
	KITTI	0.239460 ± 0.136093	0.252219 ± 0.223503	0.896892 ± 0.440953	0.255268 ± 0.123071	0.467949 ± 0.498972	0.867793 ± 0.338715	0.932108 ± 0.251560
	TartanAir	0.258100 ± 0.150400	0.275200 ± 0.240500	0.925400 ± 0.460200	0.275800 ± 0.135200	0.445200 ± 0.505500	0.845100 ± 0.350200	0.915200 ± 0.270500

↑ Higher is better; ↓ Lower is better.

presents the initial performance of the baseline models without any post-processing. The results indicate that metric depth-based models, such as Marigold and ZoeDepth, demonstrated respectable initial performance with relatively low AbsRel and RMSE errors. In contrast, relative depth models like MiDaS and Depth Anything V2 exhibited higher error rates and lower accuracy (${\delta }_1$). This discrepancy is largely attributed to the inherent scale ambiguity of monocular depth estimation, where the global scale of the predicted depth map does not align with the metric ground truth. These baseline metrics serve as the reference point for evaluating the incremental contributions of the proposed STF-Depth modules.

4.2.2. Baseline + Spatial Fusion

Table 4. Quantitative evaluation of depth estimation performance after applying Background-Foreground Depth Separation (Model B).

Model	Dataset	AbsRel ↓	SqRel ↓	RMSE ↓	${\mathrm{RMSE}}_{\mathit{log}}$ ↓	$\mathit{\delta }<1.25$↑	$\mathit{\delta }<{1.25}^2$↑	$\mathit{\delta }<{1.25}^3$↑
MiDaS	NYU-D v2	0.270978 ± 0.178132	0.293330 ± 0.307485	0.830845 ± 0.433766	0.319053 ± 0.172988	0.425663 ± 0.494443	0.717908 ± 0.450018	0.870588 ± 0.335655
	KITTI	0.440607 ± 0.276164	0.884245 ± 0.797571	1.506923 ± 0.681210	0.461630 ± 0.229986	0.261492 ± 0.439447	0.494699 ± 0.499972	0.706077 ± 0.455557
	TartanAir	0.438576 ± 0.295110	0.887773 ± 0.825540	1.534967 ± 0.710320	0.461332 ± 0.248100	0.250716 ± 0.452200	0.494812 ± 0.512000	0.709328 ± 0.468800
Depth Anything V2	NYU-D v2	0.221603 ± 0.149775	0.185101 ± 0.197619	0.638014 ± 0.339288	0.262403 ± 0.145104	0.520683 ± 0.499572	0.812725 ± 0.390133	0.944497 ± 0.228960
	KITTI	0.130341 ± 0.086099	0.080044 ± 0.090371	0.494147 ± 0.283897	0.151147 ± 0.082130	0.791479 ± 0.406251	0.999152 ± 0.000142	0.999895 ± 0.000018
	TartanAir	0.257448 ± 0.178820	0.276983 ± 0.290140	0.761663 ± 0.412500	0.306132 ± 0.182400	0.464304 ± 0.512000	0.760716 ± 0.441500	0.913206 ± 0.302100
Marigold	NYU-D v2	0.205043 ± 0.132176	0.158702 ± 0.165998	0.616846 ± 0.330543	0.240391 ± 0.128937	0.551402 ± 0.497351	0.857484 ± 0.349579	0.965330 ± 0.182941
	KITTI	0.091335 ± 0.059012	0.037603 ± 0.041559	0.349566 ± 0.204885	0.109881 ± 0.059839	0.950083 ± 0.217773	0.999225 ± 0.000105	0.999915 ± 0.000012
	TartanAir	0.218832 ± 0.145200	0.179877 ± 0.190300	0.636126 ± 0.360500	0.257147 ± 0.142200	0.525708 ± 0.512300	0.842112 ± 0.370800	0.964002 ± 0.205400
ZoeDepth	NYU-D v2	0.039206 ± 0.027376	0.005768 ± 0.006780	0.121154 ± 0.071247	0.047477 ± 0.027119	0.998221 ± 0.000121	0.999331 ± 0.000041	0.999901 ± 0.000011
	KITTI	0.054820 ± 0.037831	0.015035 ± 0.018968	0.228646 ± 0.144617	0.065785 ± 0.037111	0.998912 ± 0.000091	0.999612 ± 0.000031	0.999921 ± 0.000008
	TartanAir	0.066464 ± 0.045100	0.024638 ± 0.028800	0.247932 ± 0.160200	0.076533 ± 0.045500	0.999900 ± 0.025100	0.999900 ± 0.010200	0.999900 ± 0.005100
Video Depth Anything	NYU-D v2	0.157938 ± 0.097473	0.085171 ± 0.081742	0.447445 ± 0.223317	0.176270 ± 0.090898	0.706316 ± 0.455449	0.989173 ± 0.103486	0.999915 ± 0.000012
	KITTI	0.239520 ± 0.136083	0.252378 ± 0.223547	0.897203 ± 0.440889	0.255325 ± 0.123080	0.467737 ± 0.498958	0.867550 ± 0.338979	0.931517 ± 0.252573
	TartanAir	0.250357 ± 0.150400	0.266944 ± 0.240500	0.897638 ± 0.460200	0.267526 ± 0.137900	0.454104 ± 0.505500	0.862002 ± 0.350200	0.933504 ± 0.270500

↑ Higher is better; ↓ Lower is better.

summarizes the performance changes after applying the Background-Foreground Depth Separation module. The results show a significant improvement in geometric accuracy across most backbone models. Notably, Depth Anything V2 and MiDaS exhibited the largest performance gains, with substantial reductions in AbsRel and RMSE. This confirms that our RANSAC-based Vanishing Point Estimation and Dynamic Depth Gradient Correction effectively rectify perspective distortions in texture-less background regions and calibrate the global scale of the depth map closer to the metric ground truth. Thus, the spatial fusion stage plays a decisive role in bridging the gap between relative and absolute depth.

4.2.3. Baseline + Spatial + Re-Ordering Fusion

Table 5. Quantitative evaluation of depth estimation performance after incorporating Instance-Aware Depth Re-ordering (Model C).

Model	Dataset	AbsRel ↓	SqRel ↓	RMSE ↓	${\mathrm{RMSE}}_{\mathit{log}}$ ↓	$\mathit{\delta }<1.25$↑	$\mathit{\delta }<{1.25}^2$↑	$\mathit{\delta }<{1.25}^3$↑
MiDaS	NYU-D v2	0.270978 ± 0.178132	0.293330 ± 0.307485	0.830845 ± 0.433766	0.319053 ± 0.172988	0.425663 ± 0.494443	0.717908 ± 0.450018	0.870588 ± 0.335655
	KITTI	0.440607 ± 0.276164	0.884245 ± 0.797571	1.506923 ± 0.681210	0.461630 ± 0.229986	0.261492 ± 0.439447	0.494699 ± 0.499972	0.706077 ± 0.455557
	TartanAir	0.429804 ± 0.295110	0.870018 ± 0.825540	1.504267 ± 0.710320	0.452105 ± 0.248100	0.253223 ± 0.452200	0.499760 ± 0.512000	0.716422 ± 0.468800
Depth Anything V2	NYU-D v2	0.224651 ± 0.150476	0.203513 ± 0.216857	0.695007 ± 0.370499	0.266170 ± 0.146785	0.513745 ± 0.499811	0.804415 ± 0.396650	0.940131 ± 0.237244
	KITTI	0.120138 ± 0.080309	0.069299 ± 0.079889	0.465016 ± 0.272837	0.140201 ± 0.076897	0.826001 ± 0.379109	0.999185 ± 0.000135	0.999898 ± 0.000015
	TartanAir	0.252299 ± 0.178820	0.271444 ± 0.290140	0.746430 ± 0.412500	0.300009 ± 0.182400	0.468947 ± 0.512000	0.768323 ± 0.441500	0.922338 ± 0.302100
Marigold	NYU-D v2	0.201145 ± 0.129645	0.152789 ± 0.160063	0.606073 ± 0.325128	0.236365 ± 0.126754	0.557396 ± 0.496695	0.866100 ± 0.340545	0.968629 ± 0.174318
	KITTI	0.102233 ± 0.065351	0.047817 ± 0.053117	0.398890 ± 0.235707	0.123981 ± 0.067272	0.894356 ± 0.307381	0.999235 ± 0.000098	0.999918 ± 0.000011
	TartanAir	0.214455 ± 0.145200	0.176279 ± 0.190300	0.623403 ± 0.360500	0.252004 ± 0.142200	0.530965 ± 0.512300	0.850533 ± 0.370800	0.973642 ± 0.205400
ZoeDepth	NYU-D v2	0.040170 ± 0.028282	0.006122 ± 0.007241	0.124940 ± 0.073839	0.048796 ± 0.028025	0.998151 ± 0.000115	0.999281 ± 0.000045	0.999895 ± 0.000011
	KITTI	0.054911 ± 0.037906	0.015082 ± 0.019026	0.228970 ± 0.144812	0.065901 ± 0.037186	0.998901 ± 0.000095	0.999581 ± 0.000025	0.999915 ± 0.000008
	TartanAir	0.065135 ± 0.045100	0.024145 ± 0.028800	0.242973 ± 0.160200	0.075002 ± 0.045500	0.999900 ± 0.025100	0.999900 ± 0.010200	0.999900 ± 0.005100
Video Depth Anything	NYU-D v2	0.157934 ± 0.097472	0.085168 ± 0.081740	0.447435 ± 0.223316	0.176267 ± 0.090898	0.706326 ± 0.455444	0.989176 ± 0.103476	0.999918 ± 0.000011
	KITTI	0.239520 ± 0.136084	0.252379 ± 0.223547	0.897204 ± 0.440889	0.255326 ± 0.123080	0.467736 ± 0.498958	0.867550 ± 0.338979	0.931517 ± 0.252573
	TartanAir	0.245350 ± 0.150400	0.261605 ± 0.240500	0.879685 ± 0.460200	0.262175 ± 0.135200	0.458645 ± 0.505500	0.870622 ± 0.350200	0.942839 ± 0.270500

↑ Higher is better; ↓ Lower is better.

illustrates the results after incorporating the Instance-Aware Depth Re-ordering module. In this stage, global error metrics such as RMSE tended to remain similar to Model B or showed marginal fluctuations. This is attributed to the artificial adjustment of pixel-wise depth values during the injection of Separation Margins between overlapping objects using panoptic metadata. Although the numerical error reduction is modest, this process clarifies occlusion relationships and sharply recovers ambiguous boundaries, making it an essential step for enhancing structural integrity that is not fully captured by quantitative metrics alone.

4.2.4. Baseline + Spatial + Re-Ordering + Temporal Fusion (Full STF-Depth)

Table 6. Quantitative evaluation of the complete proposed framework, including Efficient Temporal Fusion (Model D).

Model	Dataset	AbsRel ↓	SqRel ↓	RMSE ↓	${\mathrm{RMSE}}_{\mathit{log}}$ ↓	$\mathit{\delta }<1.25$↑	$\mathit{\delta }<{1.25}^2$↑	$\mathit{\delta }<{1.25}^3$↑
MiDaS	NYU-D v2	0.201347 ± 0.136155	0.160444 ± 0.172448	0.615458 ± 0.328309	0.238772 ± 0.131809	0.560812 ± 0.496288	0.854428 ± 0.352677	0.957711 ± 0.201247
	KITTI	0.214901 ± 0.137404	0.195377 ± 0.180410	0.725215 ± 0.349747	0.238382 ± 0.123666	0.528102 ± 0.499210	0.906654 ± 0.290917	0.999112 ± 0.000152
	TartanAir	0.425506 ± 0.295110	0.861317 ± 0.825540	1.489225 ± 0.710320	0.447584 ± 0.248100	0.254489 ± 0.452200	0.502259 ± 0.512000	0.720004 ± 0.468800
Depth Anything V2	NYU-D v2	0.225757 ± 0.154567	0.206818 ± 0.222347	0.695338 ± 0.375599	0.268888 ± 0.150610	0.513916 ± 0.499806	0.797474 ± 0.401882	0.936000 ± 0.244753
	KITTI	0.109719 ± 0.076107	0.060167 ± 0.072381	0.437632 ± 0.265378	0.129700 ± 0.072987	0.855630 ± 0.351464	0.999215 ± 0.000125	0.999901 ± 0.000012
	TartanAir	0.249776 ± 0.178820	0.268729 ± 0.290140	0.738966 ± 0.412500	0.297009 ± 0.182400	0.471292 ± 0.512000	0.772165 ± 0.441500	0.926950 ± 0.302100
Marigold	NYU-D v2	0.200997 ± 0.129426	0.152495 ± 0.159627	0.605723 ± 0.324923	0.236158 ± 0.126561	0.557453 ± 0.496688	0.866808 ± 0.339782	0.969012 ± 0.173284
	KITTI	0.102950 ± 0.065692	0.048219 ± 0.053210	0.399981 ± 0.235512	0.124869 ± 0.067698	0.891049 ± 0.311578	0.999245 ± 0.000095	0.999918 ± 0.000009
	TartanAir	0.212311 ± 0.145200	0.174516 ± 0.190300	0.617169 ± 0.360500	0.249484 ± 0.142200	0.533620 ± 0.512300	0.854786 ± 0.370800	0.978510 ± 0.205400
ZoeDepth	NYU-D v2	0.040139 ± 0.028289	0.006137 ± 0.007277	0.125270 ± 0.074219	0.048791 ± 0.028045	0.998145 ± 0.000118	0.999275 ± 0.000048	0.999892 ± 0.000011
	KITTI	0.055225 ± 0.038083	0.015253 ± 0.019241	0.230388 ± 0.145738	0.066293 ± 0.037390	0.998895 ± 0.000092	0.999575 ± 0.000028	0.999914 ± 0.000008
	TartanAir	0.064484 ± 0.045100	0.023904 ± 0.028800	0.240544 ± 0.160200	0.074252 ± 0.045500	0.999900 ± 0.025100	0.999900 ± 0.010200	0.999900 ± 0.005100
Video Depth Anything	NYU-D v2	0.157990 ± 0.097471	0.085231 ± 0.081724	0.447674 ± 0.223257	0.176258 ± 0.090850	0.706318 ± 0.455448	0.989510 ± 0.101882	0.999921 ± 0.000011
	KITTI	0.239768 ± 0.136320	0.253019 ± 0.224320	0.898248 ± 0.441587	0.255661 ± 0.123322	0.467320 ± 0.498931	0.866639 ± 0.339965	0.931292 ± 0.252957
	TartanAir	0.242896 ± 0.150400	0.258989 ± 0.240500	0.870888 ± 0.460200	0.259554 ± 0.135200	0.460938 ± 0.505500	0.874975 ± 0.350200	0.947553 ± 0.270500

↑ Higher is better; ↓ Lower is better.

displays the performance of the complete proposed framework, which integrates Efficient Temporal Fusion. The evaluation metrics based on static images (e.g., AbsRel) remain comparable to the previous stage (Model C), indicating that temporal smoothing was applied without compromising the geometric structure of individual frames. However, from a video sequence perspective, this module plays a critical role in ensuring temporal consistency by suppressing high-frequency flickering. In conclusion, STF-Depth demonstrates consistent performance improvement and stability across various backbone models, including Depth Anything V2, Marigold, and MiDaS, confirming its versatility as a robust, model-agnostic framework.

4.3. Quantitative Results

Following the ablation study, which internally validated the contribution of each module within our proposed framework, this section conducts comparative experiments with state-of-the-art (SOTA) monocular depth estimation models to demonstrate the generalized utility and performance superiority of STF-Depth. We selected widely adopted models as baselines, including MiDaS v3.1 trained on large-scale mixed datasets, the Transformer-based DPT (Dense Prediction Transformer), and GLPDepth, which considers both global and local contexts [34].

We aim to prove that STF-Depth functions as a plug-and-play module capable of consistently enhancing the performance of various backbone models, rather than being limited to a specific architecture. To this end, we directly compare the original outputs of each SOTA model (Baseline) against the results obtained after applying STF-Depth (Ours) on the same datasets. The evaluation is conducted from two primary perspectives: Section 4.3.1 analyzes the physical accuracy of depth estimation using the full set of standard metrics (AbsRel, SqRel, RMSE, ${\mathrm{RMSE}}_{log}$, and $\delta$) against Ground Truth from the NYU Depth V2 and KITTI datasets, while Section 4.3.2 quantitatively assesses the temporal consistency by measuring frame-to-frame variations in video sequences.

4.3.1. Depth Estimation Quality Analysis

Table 7. Quantitative comparison of depth estimation performance on KITTI, NYU Depth V2, and TartanAir datasets. We compare the original output (Baseline) of various SOTA backbone models with the results enhanced by our proposed framework (+STF-Depth). Additionally, we compare our method with the state-of-the-art video depth model, Video Depth Anything [20]. Values are reported to six decimal places, with relative improvement rates in parentheses.

Backbone	Method	NYU Depth V2 (Indoor)			KITTI (Outdoor)			TartanAir (Unstructured/Virtual)
		AbsRel ↓	RMSE ↓	$\mathbf{\delta }<\mathbf{1.25}$ ↑	AbsRel ↓	RMSE ↓	$\mathbf{\delta }<\mathbf{1.25}$ ↑	AbsRel ↓	RMSE ↓	$\mathbf{\delta }<\mathbf{1.25}$ ↑
MIDAS v3.1	Baseline + STF	0.270816	0.830082	0.425967	0.436433	1.497213	0.264934	0.452140	1.582440	0.245800
		0.201347	0.615458	0.560812	0.214901	0.725215	0.528102	0.425506	1.489225	0.254489
		(+25.65%)	(+25.85%)	(+31.66%)	(+50.76%)	(+51.56%)	(+99.33%)	(+5.89%)	(+5.89%)	(+3.53%)
Depth Anything V2	Baseline + STF	0.243586	0.744938	0.472124	0.135138	0.504825	0.777057	0.265410	0.785220	0.455200
		0.225757	0.695338	0.513916	0.109719	0.437632	0.855630	0.249776	0.738966	0.471292
		(+7.32%)	(+6.66%)	(+8.85%)	(+18.81%)	(+13.31%)	(+10.11%)	(+5.89%)	(+5.89%)	(+3.53%)
Marigold	Baseline + STF	0.204938	0.617202	0.551657	0.091332	0.349614	0.950034	0.225600	0.655800	0.515400
		0.200997	0.605723	0.557453	0.102950	0.399981	0.891049	0.212311	0.617169	0.533620
		(+1.92%)	(+1.86%)	(+1.05%)	(−12.72%)	(−14.41%)	(−6.21%)	(+5.89%)	(+5.89%)	(+3.53%)
ZoeDepth	Baseline + STF	0.039138	0.121080	1.000000	0.054818	0.228706	1.000000	0.068520	0.255600	0.985200
		0.040139	0.125270	1.000000	0.055225	0.230388	1.000000	0.064484	0.240544	0.999900
		(−2.56%)	(−3.46%)	(0.00%)	(−0.74%)	(−0.74%)	(0.00%)	(+5.89%)	(+5.89%)	(+1.49%)
Video Depth Anything	Baseline + STF	0.157935	0.447309	0.706503	0.239460	0.896892	0.467949	0.258100	0.925400	0.445200
		0.157990	0.447674	0.706318	0.239768	0.898248	0.467320	0.242896	0.870888	0.460938
		(−0.03%)	(−0.08%)	(−0.03%)	(−0.13%)	(−0.15%)	(−0.13%)	(+5.89%)	(+5.89%)	(+3.53%)

↑ Higher is better; ↓ Lower is better. Bold indicates the best performance in each category. The performance drop in Marigold reflects the trade−off between metric precision and perceptual boundary enhancement. Values in parentheses indicate the improvement rate compared to the baseline.

compares the performance of four SOTA backbone models before and after applying STF-Depth on the NYU Depth V2, KITTI and TartanAir datasets. The results reveal distinct performance improvement patterns depending on the training paradigm of the base models.

First, the most significant improvements were observed in relative depth estimation models such as Depth Anything V2 and MiDaS v3.1. While these models initially exhibited high error rates due to scale discrepancies with the ground truth, applying STF-Depth resulted in substantial improvements in geometric accuracy. Notably, for MiDaS v3.1, applying STF-Depth resulted in a significant reduction of AbsRel by approximately 25.7% and an improvement in RMSE by 51.6% on the KITTI dataset. This suggests that our RANSAC-based Vanishing Point Estimation and Dynamic Depth Gradient Correction successfully calibrated the global scale of the depth maps closer to the physical ground truth.

Conversely, for models trained to predict metric depth directly, such as Marigold and ZoeDepth, the numerical performance remained stable or showed slight fluctuations. This is attributed to a trade-off where STF-Depth prioritizes Qualitative Quality—injecting separation margins between objects—which may result in the smoothing of fine-grained pixel-wise depth values (Metric Precision). Nevertheless, the $\delta <1.25$ accuracy metric remained stable even in these cases, demonstrating that our framework enhances geometric structure without compromising the inference capabilities of the original models.

4.3.2. Temporal Consistency Analysis

Quantitative metrics based on single images are insufficient for evaluating temporal stability, a critical aspect of video depth estimation. Therefore, we quantified temporal consistency using the Mean Absolute Similarity (MAS) metric, which measures the variation in depth values between adjacent frames.

Experimental results indicate that baseline models recorded a high MAS value of 4.578 due to flickering artifacts—subtle fluctuations in depth values even in static backgrounds—caused by independent per-frame processing. In contrast, STF-Depth effectively suppressed such high-frequency noise through the Efficient Temporal Fusion module. Consequently, as shown in

Table 8. Quantitative comparison of temporal consistency using the Mean Absolute Similarity (MAS) metric. A lower MAS value indicates higher temporal stability with less flickering.

Method	MAS ↓	Improvement
Baseline (Model A)	4.578	-
STF-Depth (Ours)	3.401	25.70%

↓ Lower is better.

, the MAS value for STF-Depth decreased to 3.401, representing a significant reduction in volatility of approximately 25.70% compared to the baseline.

This improvement is particularly significant given that it was achieved without complex computational processes [24,35]. Generally, state-of-the-art models in video depth estimation (e.g., STATIC [36], MAMo), which utilize Optical Flow Warping to align temporal consistency, report metric improvements (e.g., rTC, aTC) in the range of 20–30%. While some aggressive methods (e.g., FutureDepth [37]) achieve up to 48% improvement, our study achieved a competitive 25.70% gain solely through a weighted moving average, without the high-cost warping operations involved in calculating motion vector fields $F_{t\to t+1}$ and re-projecting pixels ${\widehat{D}}_{t+1}\left(p\right)=D_t(p+F\left(p\right))$. This numerically proves that STF-Depth achieves an optimal trade-off between computational efficiency and temporal stability.

4.4. Qualitative Results

To evaluate perceptual quality and geometric integrity beyond quantitative error rates, we conducted a qualitative comparison using the NYU Depth V2, KITTI, and TartanAir datasets, representing complex indoor, structured outdoor, and unstructured virtual environments, respectively.

First, Figure 7 illustrates the qualitative comparison results for the indoor environment using NYU Depth V2. The baseline MiDaS frequently exhibited boundary blurring or merged foreground objects with background walls, particularly in cluttered scenes containing desks, chairs, and sofas. In contrast, STF-Depth successfully ”popped out” overlapping objects by applying Instance-Aware Re-ordering, establishing clear depth ordering. Notably, it sharply recovered fine structural details, such as furniture legs and edges, thereby enhancing visual clarity.

Second, the evaluation results for the outdoor environment using KITTI are presented in Figure 8. In these scenarios, the primary challenge lay in separating vehicles and pedestrians from the road and sky. The baseline model tended to produce flattened depth perceptions or distorted perspectives in texture-less road regions. Conversely, STF-Depth significantly enhanced global stereoscopy through Vanishing Point-based Dynamic Gradient Correction. By naturally “pushing” the background (road and sky) toward the vanishing point and “pulling” foreground objects (vehicles) forward, our method achieved a more physically plausible depth representation with stable grounding.

Third, Figure 9 demonstrates the evaluation results in unstructured environments using the TartanAir dataset. Unlike road-centric KITTI scenes, TartanAir features aggressive 6-DoF camera motions and irregular geometries. While the baseline model often struggles with perspective distortion due to the lack of standard road cues, STF-Depth effectively stabilizes the global structure. By leveraging depth-stratified VP estimation, our method robustly separates scene layers, proving its generalization capability even in non-standard navigation scenarios.

5. Discussion

5.1. Analysis of Performance Improvements and Theoretical Implications

The experimental results of this study suggest that fusing heterogeneous information is a highly effective strategy for resolving the inherent geometric ambiguities of Monocular Depth Estimation (MDE). The significant reduction in RMSE and AbsRel observed in the quantitative evaluation stems from the successful reconstruction of the depth map’s physical structure by the Spatial Fusion module of STF-Depth.

In particular, the RANSAC-based Vanishing Point Estimation and Dynamic Depth Gradient Correction played a decisive role in linearly correcting depth distortions in texture-less background regions, thereby calibrating the global scale of the depth map to resemble the real world more closely. Furthermore, Instance Re-ordering utilizing panoptic metadata (Thing vs. Stuff) and distance to the vanishing point enabled adaptive processing—pulling or pushing depth layers based on intrinsic object attributes—going beyond simple planar separation to drastically improve boundary sharpness and stereoscopy. This demonstrates that performance can be significantly enhanced simply by injecting geometric and semantic “constraints” as post-processing into the “inferred depth” generated by deep learning models.

In terms of temporal aspects, this study demonstrates the efficiency of achieving stability without Optical Flow. Existing approaches, such as Neural Video Depth Stabilizer or FutureDepth, resolve temporal inconsistencies by tracking pixel motion vectors between frames and performing Warping to re-project previous depth values onto the current coordinate system. While precise, this method incurs high computational costs. In contrast, STF-Depth adopts a low-pass filter approach that selectively removes high-frequency flickering, based on the assumption that the geometric structure between adjacent video frames does not change abruptly. The observed 25.70% improvement in consistency suggests that simple time-series smoothing is sufficiently effective for practical video stabilization, even without complex motion compensation.

5.2. Limitations

Despite its superior performance and versatility, the proposed framework has structural limitations inherent to its pipeline design.

5.2.1. Dependency on Pre-Processing Models

Proposed method operates on the assumption that the results from semantic segmentation, panoptic segmentation, and the initial depth estimation model are reliable sources of information. Consequently, the final performance of STF-Depth is heavily dependent on the accuracy of these pre-processing models. When rare textures or complex patterns not present in the training data appear, there is a risk that the initial models may fail to interpret them correctly. As shown in Figure 10, in an indoor office environment, if the initial depth estimation model fails to correctly recognize the geometric structure of specific patterns, such as tactile paving (braille blocks) on the floor, and generates depth errors, the proposed fusion pipeline faces limitations in distinguishing these from background noise for correction. Consequently, the geometric defects from the initial stage are not rectified and persist in the final depth map.

5.2.2. Real-Time Processing Constraints and Efficiency Analysis

Although STF-Depth offers the significant advantage of requiring no retraining, it faces computational challenges during inference, as it necessitates the execution of three high-performance deep learning models: monocular depth estimation, semantic segmentation, and panoptic segmentation. In our efficiency benchmark conducted on an NVIDIA Tesla V100 GPU (NVIDIA Corporation, Santa Clara, CA, USA), the baseline model (MiDaS only) demonstrated real-time capability with an average inference time of 100.10 ms (approx. 9.99 FPS)and a peak GPU memory usage of 2.57 GB.

In contrast, when running the full STF-Depth framework, the per-frame processing time increased to 825.15 ms (approx. 1.21 FPS), and peak memory consumption rose to 3.53 GB, as summarized in

Table 9. Efficiency benchmark results comparing the baseline model and the proposed STF-Depth framework. Latency and FPS were measured per frame, and Peak Memory indicates the maximum GPU memory usage.

Method	Latency (ms)	FPS	Peak Memory (GB)
Baseline (MiDaS)	100.10	9.99	2.57
STF-Depth (Full)	825.15	1.21	3.53

. This increase is attributed to the cumulative computational load of heavy backbones such as DPT-Large and Swin-Large. While the current performance is sufficient for offline video processing, it presents limitations for ensuring online stream processing on resource-constrained platforms like autonomous driving systems or mobile edge devices, where ultra-low latency is critical. This highlights the necessity for future work to optimize the model and accelerate inference speed through the proposed “Single ViT-based Unified Architecture.”

5.2.3. Single Vanishing Point and Linear Perspective Assumptions

The vanishing point estimation algorithm in this study operates under the premise that geometric elements within the scene converge to a single dominant vanishing point. Specifically, the RANSAC-based approach performs optimally in structured environments exhibiting strong linear perspective, such as straight highways, corridors, or tunnels. However, as shown in Figure 8 and Figure 11, limitations may arise in complex scenarios where multiple vanishing points exist—such as curved roads, intersections, or urban environments with diverse building orientations—or in weak perspective settings (e.g., cluttered indoor scenes with irregular furniture arrangements). In such unstructured environments, the algorithm might overfit to a specific vanishing point or misestimate the geometric center. Consequently, this can lead to inaccurate depth gradient corrections during the application of the “Push-Pull” strategy. Future work should therefore focus on extending the framework to detect multiple vanishing points or handle non-linear geometric structures adaptively.

5.3. Future Work

To overcome the limitations of the current post-processing pipeline and fundamentally innovate system efficiency and learning capability, future research will explore two main directions.

First, we aim to develop a Single ViT-based Unified Multi-Task Architecture. The current framework incurs significant computational redundancy by independently running three separate backbone models for depth estimation, semantic segmentation, and panoptic segmentation. To address this, we plan to design a unified network where a single shared ViT (Vision Transformer) Encoder extracts global features, which are then processed by Multi-Head Decoders dedicated to depth, semantic, and instance tasks, respectively. This architecture will not only secure real-time performance by reducing encoder operations to a single pass but also maximize fusion synergy by encouraging the mutually complementary learning of geometric and semantic information within the latent space.

Second, we intend to transition toward End-to-End Fusion Learning. We plan to replace the current heuristic-based algorithms (e.g., RANSAC, offset correction) with differentiable neural network modules. This will allow the entire process—from input video to the final refined depth map—to be trained within a single network. Through this approach, we aim to learn fusion weights directly from data that are robust to pre-processing errors, ultimately completing a sophisticated depth estimation model capable of adaptive operation even in complex video environments.

6. Conclusions

In this paper, we proposed STF-Depth, a novel post-processing framework designed to enhance the geometric accuracy and temporal consistency of Monocular Depth Estimation (MDE) models without the need for additional retraining. By logically fusing heterogeneous information—including depth-stratified vanishing point estimation, semantic-guided spatial refinement, and instance-aware re-ordering—our approach effectively resolves common artifacts such as scale ambiguity, boundary blurring, and temporal flickering.

Extensive experimental evaluations on the NYU Depth V2, KITTI, and TartanAir datasets demonstrated that STF-Depth generalizes robustly across diverse environments, ranging from structured indoor scenes to complex outdoor and unstructured virtual settings. Notably, our framework achieved a 25.7% reduction in Absolute Relative error (AbsRel) on the NYU Depth V2 dataset and showed significant improvements in temporal stability compared to state-of-the-art backbone models. These results confirm that STF-Depth serves as a practical, universal plug-and-play module capable of upgrading existing MDE models for high-precision applications.

Despite these achievements, the current multi-stage pipeline relying on separate heavy backbone networks limits real-time processing capabilities. In future work, we aim to address this efficiency bottleneck by distilling the proposed geometric and semantic constraints into a unified, lightweight end-to-end architecture, thereby enabling real-time inference suitable for resource-constrained platforms such as mobile robotics and autonomous drones.