AuxDepthNet: Real-Time Monocular 3D Object Detection with Depth-Sensitive Features

Abstract

Monocular 3D object detection is a challenging task in autonomous systems due to the lack of explicit depth information in single-view images. Existing methods often depend on external depth estimators or expensive sensors, which increase computational complexity and complicate integration into existing systems. To overcome these limitations, we propose AuxDepthNet, an efficient framework for real-time monocular 3D object detection that eliminates the reliance on external depth maps or pre-trained depth models. AuxDepthNet introduces two key components: the Auxiliary Depth Feature (ADF) module, which implicitly learns depth-sensitive features to improve spatial reasoning and computational efficiency, and the Depth Position Mapping (DPM) module, which embeds depth positional information directly into the detection process to enable accurate object localization and 3D bounding box regression. Leveraging the DepthFusion Transformer (DFT) architecture, AuxDepthNet globally integrates visual and depth-sensitive features through depth-guided interactions, ensuring robust and efficient detection. Extensive experiments on the KITTI dataset show that AuxDepthNet achieves state-of-the-art performance, with ${AP}_{3D}$ scores of 24.72% (Easy), 18.63% (Moderate), and 15.31% (Hard), and ${AP}_{BEV}$ scores of 34.11% (Easy), 25.18% (Moderate), and 21.90% (Hard) at an IoU threshold of 0.7.

1. Introduction

Three-dimensional (3D) object detection [1,2,3,4,5] plays a critical role in autonomous driving and robotic perception by enabling machines to perceive and interact with their surroundings in a spatially aware manner. Traditionally, this task has relied on precise depth information from multiple sensors, such as LiDAR, stereo cameras, or depth sensors. While these systems provide high accuracy, they are often expensive and complex to deploy. In recent years, monocular 3D object detection has gained attention as a cost-effective alternative, requiring only a single RGB camera [6,7,8]. Despite its simplicity and accessibility, monocular 3D detection faces challenges due to the absence of explicit depth cues, making effective depth reasoning essential for improving accuracy and practicality.

Monocular 3D object detection methods can be broadly categorized into two approaches, as illustrated in Figure 1. The first category utilizes pre-trained depth estimation models to generate pseudo-depth maps, which are combined with LiDAR-based 3D detectors for object recognition and localization [9,10]. This approach, as exemplified by methods like Pseudo-LiDAR and F-Pointnet [5,11,12], achieves improved localization but suffers from inaccuracies in depth priors and high computational costs due to the reliance on external depth estimators, limiting their applicability in real-time scenarios.

The second category focuses on feature fusion, where features from monocular images and estimated depth maps are extracted and combined to enhance detection. Methods like $D^{4}LCN$ and $\mathrm{C}\mathrm{a}\mathrm{D}\mathrm{D}\mathrm{N}$ [13,14], as shown in Figure 1b, use specialized convolutional architectures to integrate visual and depth features. While these methods demonstrate promising results, their reliance on the quality of depth maps and the complexity of their architectures can hinder efficiency and robustness in dynamic environments.

To address these challenges, we propose AuxDepthNet, a novel framework for real-time monocular 3D object detection that avoids the need for external depth estimators or pre-generated depth maps. AuxDepthNet introduces two key modules: the Auxiliary Depth Feature Module (ADF) and the Depth Position Mapping Module (DPM), as depicted in Figure 1c. These modules enable implicit learning of depth-sensitive features and inject depth positional cues directly into the detection pipeline. Conventional CNN-based monocular 3D detection frameworks often struggle to capture global dependencies and integrate multi-source depth features. In contrast, Transformers leverage self-attention mechanisms to process image and depth information simultaneously [15,16], allowing for end-to-end 3D object detection within a streamlined pipeline [17]. Built upon the DepthFusion Transformer architecture, our method efficiently integrates contextual and depth-sensitive features, achieving superior detection performance with reduced computational costs [11,12].

Figure 1. Representative depth-assisted monocular 3D object detection frameworks. (a) Depth estimation methods use monocular inputs to construct pseudo-LiDAR data, enabling LiDAR-style 3D detectors [5,11,12]. (b) Fusion-based methods combine visual and depth features to improve object detection accuracy [13,14,18]. (c) Our AuxDepthNet leverages depth guidance during training to develop depth-sensitive features and performs end-to-end 3D object detection without requiring external depth estimators.

Figure 1. Representative depth-assisted monocular 3D object detection frameworks. (a) Depth estimation methods use monocular inputs to construct pseudo-LiDAR data, enabling LiDAR-style 3D detectors [5,11,12]. (b) Fusion-based methods combine visual and depth features to improve object detection accuracy [13,14,18]. (c) Our AuxDepthNet leverages depth guidance during training to develop depth-sensitive features and performs end-to-end 3D object detection without requiring external depth estimators.

The main contributions of this paper are as follows:

(1)

We propose AuxDepthNet, a novel framework for efficient, real-time monocular 3D object detection that eliminates reliance on external depth maps or estimators.

(2)

We design the Auxiliary Depth Feature Module (ADF) and Depth Position Mapping Module (DPM) to implicitly learn depth-sensitive features and encode depth positional cues into the detection process.

(3)

We provide a plug-and-play architecture that can be seamlessly integrated into other image-based detection frameworks to enhance their 3D reasoning capabilities.

2. Related Work

Monocular 3D object detection has gained significant attention for its cost-effectiveness compared to sensor-based approaches, such as LiDAR and stereo cameras. Existing methods can be broadly categorized into depth-based methods and Transformer-based methods, which are discussed below.

2.1. Monocular 3D Object Detection Methods

Image-based 3D object detection leverages monocular or stereo images to estimate depth and generate 3D proposals. Approaches like MONO3D [13], MLF [19], and Frustum PointNet [20] use depth maps or disparity estimates to compute 3D coordinates, often integrating them into 2D detection pipelines. However, poor depth representations in these methods limit the ability of convolutional networks to accurately localize objects, especially at greater distances, leading to performance gaps compared to LiDAR-based systems. Depth-based methods aim to bridge this gap by explicitly or implicitly leveraging depth information. One common approach is to estimate pseudo-depth maps from monocular images and utilize them for 3D object detection. For instance, Pseudo-LiDAR [5] and F-PointNet [12] rely on pre-trained depth models to generate point clouds, which are then processed with LiDAR-style 3D detectors. While these methods improve spatial reasoning, they suffer from depth estimation errors and increased computational costs due to reliance on external depth estimators.

Another line of work focuses on integrating monocular image features with estimated depth information in a feature fusion framework. Methods like D⁴LCN [13] and CaDDN [21] incorporate depth-sensitive features into the 2D detection pipeline to enhance accuracy. However, these approaches are constrained by the quality of the estimated depth maps and often rely on complex architectures, limiting their real-time applicability. In contrast, our proposed method avoids reliance on external depth estimators by directly learning depth-sensitive features through the Auxiliary Depth Feature (ADF) module. This design allows AuxDepthNet to achieve high accuracy while maintaining computational efficiency.

2.2. Transformer in Monocular 3D Object Detection

The recent success of Transformers in computer vision has inspired their application in monocular 3D object detection. Unlike convolutional neural networks (CNNs), Transformers capture global spatial dependencies using self-attention mechanisms, addressing the limitations of local feature extraction. For example, MonoDETR [22] employs a dual-encoder architecture and a depth-guided decoder to improve depth representation and object localization. Similarly, MonoPSTR [23] introduces scale-aware attention and position-coded queries to enhance detection precision and efficiency. Recently, diffusion models [24,25,26,27] have also demonstrated their potential in depth estimation by iteratively refining depth predictions, offering an alternative to traditional depth map generation methods. These models could complement Transformer-based architectures by providing robust depth priors for enhanced detection.

Despite their advancements, Transformer-based methods often rely on pre-computed depth maps or handcrafted priors to guide detection [17,28]. This dependence on external inputs can hinder adaptability to diverse scenarios. AuxDepthNet addresses this limitation by embedding depth reasoning directly into the network via the Auxiliary Depth Feature (ADF) and Depth Position Mapping (DPM) modules. These modules enable the model to implicitly learn depth-sensitive features, eliminating external dependencies and delivering robust, scalable performance.

3. Proposed Method

3.1. Overview

As illustrated in Figure 2, the AuxDepthNet architecture employs a multi-faceted feature representation to enhance monocular 3D object detection. Specifically, our framework integrates three key feature types: depth-sensitive, context-sensitive, and depth-guided features—each capturing critical information at different processing stages.

We adopt DLA-102x as the Backbone network, which takes an input image of size $3\times H\times W$ and produces multi-scale feature maps. The final output of the Backbone, denoted $F$, is a feature map with dimensions $C\times H^{\prime }\times W^{\prime }$, where $C$ is the number of channels and $H^{\prime },W^{\prime }$ may be smaller than the original input dimensions due to downsampling operations. These feature maps feed into the Auxiliary Depth Feature (ADF) module, which encodes depth-related cues through an additional “auxiliary” learning task.

This task eliminates the need for pre-computed depth maps by implicitly capturing depth prototypes, thereby strengthening depth-sensitive feature representation. In this context, “auxiliary” denotes an extra but complementary objective that supports the model’s primary 3D detection goal by providing more accurate depth information for improved spatial reasoning.

Meanwhile, context-sensitive features extracted by the Backbone are refined via a feature pyramid and the DepthFusion Transformer (DFT) to supply essential semantic and spatial context. In parallel, depth-guided features receive further enhancement from the Depth Position Mapping (DPM) module and positional encoding, embedding depth-related positional cues into the latent space. This combination of local and global spatial relationships, delivered at minimal computational cost, results in robust and efficient 2D and 3D object detection.

3.2. Depth-Sensitive Feature Enhancement

Current depth-assisted approaches face challenges in generalizing to varied datasets and environments, which hampers their adaptability. Moreover, their dependency on external depth sensors or estimators not only increases hardware requirements but also risks propagating inaccuracies into the detection pipeline [5,13,17,18]. To address these challenges, we propose the Auxiliary Depth Feature (ADF) module, which leverages auxiliary supervision during training to implicitly learn depth-sensitive features, eliminating the need for external depth maps or estimators.

Unlike previous methods [3,5,8], which rely on pre-computed depth maps or external estimators, the ADF module captures depth-sensitive features directly from the input feature map using auxiliary learning. This approach ensures scalability, improves generalization across datasets, and enables real-time applicability.

As illustrated in Figure 3, the Auxiliary Depth Feature (ADF) module operates in three stages to enhance depth-sensitive features. First, it generates initial depth-sensitive features through an auxiliary supervision task, which predicts a probability distribution over discretized depth bins for each pixel. Next, the module learns depth prototypes by leveraging spatial-depth attention mechanisms, aggregating features across spatial dimensions to capture meaningful depth relationships. Finally, these depth prototypes are projected back to refine the feature map, producing an enhanced representation that integrates depth-sensitive information and spatial context, thereby improving the accuracy and robustness of 3D object detection.

The ADF module is designed as a lightweight and efficient solution that enhances depth reasoning and spatial localization while maintaining low computational cost. The term ‘auxiliary’ refers to the fact that the module learns depth-sensitive features through an auxiliary supervision task. This task does not rely on pre-computed depth maps or external depth estimators, but instead enables the network to learn depth-related information directly from the image data during training.

The following sections detail each step of the ADF module and its role in improving depth representation for 3D object detection.

3.2.1. Backbone Network Architecture and Multi-Scale Feature Extraction

Figure 4 illustrates the changes in input image size and channel dimensions as the Backbone processes the input. We assume that the input image has a resolution of $1280\times 288\times 3$ ($height\times width\times channels$). In the first layer of the Backbone, the image undergoes an initial convolution operation, resulting in a feature map with a size of $640\times 144\times 16$. As the network progresses, the spatial dimensions of the feature map gradually decrease, while the number of channels increases. Specifically, at Level 0, the feature map retains the same size of $640\times 144\times 16$, but at Level 1, after downsampling, the feature map size reduces to $320\times 72\times 32$. Further down the network, the feature map dimensions continue to shrink while the channel depth continues to grow. At Level 2, the output is $160\times 36\times 128$, at Level 3 it is $80\times 18\times 256$, at Level 4 it is $40\times 9\times 512$, and finally, at Level 5, the feature map size becomes $20\times 5\times 1024$. This process demonstrates the gradual extraction of deeper features from the image through successive downsampling and convolution operations.

Additionally, the DLA-102× network includes a feature aggregation module, which fuses feature maps from different layers. Through this aggregation process, the final output feature map has a size of $320\times 72\times 256$, containing information from multiple scales that enhances the subsequent depth-sensitive feature extraction process.

Mathematically, let the input image $I$ have a size of $H\times W\times 3$; the Backbone network first applies convolution to produce an initial feature map $F_0$:

$$F_0=Backbone\left(I\right),\hspace{1em}{F}_0\in R^{C_0\times H_0\times W_0}$$

(1)

where $C_0$ is the number of channels in the feature map, and $H_0,W_0$ are the spatial dimensions after convolution and pooling. As the network deepens, each layer generates deeper feature representations through convolution operations and downsampling, gradually reducing the spatial dimensions and increasing the number of channels. After $\mathrm{n}$ layers, the feature map’s size and channels become the following:

$$F_n={Backbone}_n\left(F_{n-1}\right),\hspace{1em}{F}_n\in R^{C_n\times H_n\times W_n}$$

(2)

Finally, after the feature aggregation module, the resulting feature map $F_{aggregated}$ has the following dimensions:

$$F_{aggregated}\in R^{C_{aggregated}\times H_{aggregated}\times W_{aggregated}}$$

(3)

This aggregated feature map serves as the foundation for subsequent depth-sensitive feature extraction, providing rich visual information for depth estimation and object detection tasks. This input–output definition not only provides clear guidance for understanding the feature extraction process in the Backbone but also forms a solid foundation for extracting depth-sensitive features, ensuring the success of the subsequent tasks.

3.2.2. Extracting Foundational Depth-Sensitive Features

In the Auxiliary Depth Feature (ADF) module, foundational depth-sensitive features are generated using an auxiliary depth estimation task modeled as a sequential classification problem. Given the input feature map $F\in R^{C\times H\times W}$ from the Backbone, we apply two convolutional layers to predict the probability distribution of discretized depth bins $D_b\in R^{D\times H\times W}$, where $D_b$ denotes the number of depth categories. The predicted probability represents the confidence of each pixel belonging to a specific depth bin:

$$P_{i,j,d}={\displaystyle \frac{\mathit{exp}\left(z_{i,j,d}\right)}{{\sum }_{k=1}^{D_b}\mathit{exp}\left(z_{i,j,k}\right)}}.$$

(4)

where $z_{i,j,d}$ is the raw score for pixel $\left(i,j\right)$ in bin $n$.

To discretize continuous depth values into bins, we utilize Linear-Increasing Discretization (LID), which refines depth granularity for closer objects while allocating broader intervals for farther ones. $D$ represents the total number of discretized depth bins, and $i$ denotes the index of the corresponding depth bin. The depth binning strategy is defined as follows.

The discretization can be formulated as follows:

$$B_i=\left\{\begin{array}{c}{\displaystyle \frac{{\left(i+1\right)}^2}{D}}, if i<\sqrt{D}, \\ {\displaystyle \frac{\left(i+1\right)}{D}}, otherwise.\end{array}\right.$$

(5)

By applying depth-wise separable convolutions, the intermediate feature map $X\in R^{C\times H\times W}$ efficiently captures depth-sensitive features with reduced computational overhead. These features, further enhanced through attention mechanisms, provide a robust foundation for downstream 3D object detection tasks.

3.2.3. Depth-Sensitive Prototype Representation Module

This module aims to refine feature representations through depth prototype learning and enhancement. Starting from the initial depth-sensitive feature map $F_{init}\in R^{C\times H\times W}$, generated by the Backbone network, the module predicts the depth distribution $P_{depth}\in R^{D\times H\times W}$ for each pixel, where $C$ is the feature dimension, $H$ and $W$ are spatial dimensions, and $D$ represents the number of depth bins. We extract spatial features $F_{init}\left[i\right]$ from the input image and match them with their corresponding depth information using a trainable attention mechanism to obtain $Attention\left(i,d\right)$. This attention weight measures the confidence of pixel $\mathrm{i}$ at different depth ranges, allowing the network to adaptively emphasize relevant depth features without relying on manually designed priors.

The predicted probability represents the confidence of each pixel belonging to a specific depth bin, which is computed using softmax normalization. The ADF module employs D-Convolution (D-Conv) and Grouped Convolution (G-Conv) to refine depth-sensitive features. D-Conv expands the receptive field without increasing parameters by using dilation rates, while G-Conv reduces computational complexity by performing convolution in separate channel groups. Using the predicted depth distribution, the module estimates depth prototype representations $P\left[d\right]\in R^C$ for each depth bin $\mathrm{d}$ by aggregating features across all pixels:

$$P\left[d\right]=\sum _{i=1}^{N}Attention\left(i,d\right)\times F_{init}\left[i\right],d\in \{1,\dots ,D\}.$$

(6)

where $P\left[d\right]$, $N=H\times W$ represents the total number of spatial positions, and $Attention\left(i,d\right)$ denotes the attention weight for pixel $i$ and depth bin $d$, derived from the similarity between $F_{init}$ and $P_{depth}$. All pixels contributing strongly to depth $d$ are combined into a single global depth prototype $P\left[d\right]$. Intuitively, the model scans the entire image to collect spatial cues most representative of depth $d$. If certain pixels are more likely to be at depth $d$, they receive higher weights and thus dominate the resulting prototype.

3.2.4. Feature Enhancement with Depth Prototype

We redistribute the depth prototypes back to each pixel $i$ by weighting the prototypes $P\left[d\right]$ again with $Attention\left(i,d\right)$. In other words, each pixel gathers relevant cues from all depth levels based on its learned attention distribution. This re-projection step injects global depth context into local features, thereby strengthening the model’s spatial–depth reasoning capability.

The depth prototypes are then projected back to enhance the feature map, creating an updated representation:

$$F_{enhanced}=\sum _{d=1}^{D}Attention\left(i,d\right)\times P\left[d\right],i\in \{1,\dots ,N\}$$

(7)

Each pixel $i$ not only retains its local feature representation but also absorbs contextual information from the global depth prototype, allowing for enhanced depth reasoning and spatial consistency. Next, each pixel $i$ retrieves or reclaims information from every depth prototype $P\left[d\right]$ by applying the same attention distribution $Attention\left(i,d\right)$. As a result, each pixel’s final representation $F_{enhanced}\left[i\right]={\sum }_{d=1}^{D}Attention\left(i,d\right)\times P\left[d\right]$ incorporates both its original local features and the global depth cues distilled into $P\left[d\right]$. Physically, this second step “injects” scene-wide depth awareness back into each local position, reinforcing the model’s three-dimensional perception.

Finally, the enhanced feature map $F_{en}\in R^{C\times H\times W}$ undergoes a convolutional refinement to integrate depth-sensitive information. As a result, the enhanced depth feature is generated by combining the initial depth-sensitive feature with the reconstructed features through concatenation. This mechanism ensures that the depth-sensitive prototypes effectively capture spatial–depth relationships, significantly improving feature quality for 3D object detection tasks.

3.3. Depth Position Mapping and Transformer Integration

Inspired by the success of Transformer architectures in capturing long-range dependencies and modeling global relationships [29], we introduce the Depth Position Mapping (DPM) module as a key component of our framework. The DPM module is designed to address the challenges of integrating spatial and depth cues in monocular 3D object detection. Unlike traditional methods that rely solely on local visual features or predefined depth priors, DPM leverages a depth-guided mapping mechanism to explicitly encode positional depth information into the feature space. By embedding depth positions into learnable queries within an encoder–decoder structure, the module enables a precise alignment of spatial and depth representations. This design enhances the understanding of scene-level depth geometry while improving the quality of feature fusion for downstream 3D attribute prediction. The DPM module thus provides a robust and efficient solution for bridging the gap between depth estimation and object localization in monocular 3D object detection.

To further enhance these depth-sensitive features, we introduce the DepthFusion Transformer (DFT), which leverages self-attention to capture long-range spatial dependencies. By integrating local depth cues with a global spatial perspective, the model achieves more accurate depth reasoning, ultimately improving 3D object localization. This approach helps the model handle common challenges in real-world monocular 3D detection, such as occlusion and variable distances, by focusing on pertinent spatial regions and depth information. The encoder–decoder design within the DepthFusion Transformer (DFT) is pivotal in capturing long-range interactions between depth and spatial features. In the encoder, spatial features are processed via self-attention, while the decoder employs cross-attention to refine depth features against spatial context. This structure ensures the model can effectively capture global interactions throughout the entire image, boosting object localization even in scenarios with occlusion or diverse depth scales.

3.3.1. Transformer Encoder

The Transformer Encoder in the Depth Position Mapping (DPM) module plays a critical role in capturing global dependencies and refining feature representations by leveraging self-attention mechanisms. It enables the model to incorporate contextual information across the entire sequence, facilitating a deeper understanding of spatial and depth relationships. The encoder refines input features by employing a multi-head self-attention mechanism and a feed-forward neural network (FFN) [17,28]. Given the input feature tensor $X\in R^{N\times L\times C}$, where $N$ is the batch size, $\mathrm{L}$ is the sequence length, and $\mathrm{C}$ is the feature dimensionality, the encoder projects X into query $Q_f$, key $K_f$, and value $V_f$ matrices, where $Q_{\mathrm{f}}$, $K_{\mathrm{f}}$, $V_{\mathrm{f}}\in {\mathcal{R}}^{N\times L\times C}$. These are further divided into $H$ attention heads, with each head having a dimensionality of $D_k=C/H$. The attention weights are computed as follows:

$$Attention\left(Q_f,K_f\right)=Softmax\left({\displaystyle \frac{Q_f\times {K_f}^T}{\sqrt{D_k}}}\right).$$

(8)

enabling the encoder to capture long-range dependencies across the sequence. A linear attention mechanism enhances computational efficiency by transforming [29] $Q_{\mathrm{f}}$ and $K_{\mathrm{f}}$, using an ELU-based feature map. The output of the attention layer, $\mathrm{Z}\in {\mathrm{R}}^{\mathrm{N}\times \mathrm{L}\times \mathrm{C}}$, is combined with the input via a residual connection and normalized using layer normalization, $X_1=Norm\left(X+Z\right)$. This is followed by a two-layer FFN, $FFN\left(X\right)=ReLU\left(X\times W_1\right)\times W_2$, where $W_1$ and $W_2$ are learnable weight matrices. The FFN output is also added back to the input with a residual connection and normalized, $X_{ot}=Norm\left(X_1+FFN\left(X_1\right)\right)$. The final output, $X_{ot}\in R^{N\times L\times C}$, integrates depth-sensitive global context and long-range dependencies, providing refined features for downstream tasks.

3.3.2. Transformer Decoder

The Transformer Decoder is designed to integrate depth-sensitive features with contextual information, enabling robust representation refinement for downstream tasks [29]. By employing a multi-layer structure, each layer consists of self-attention, cross-attention, and a feed-forward network. Self-attention operates on the input queries to capture internal dependencies, while cross-attention aligns these queries with encoder outputs, ensuring the effective fusion of spatial and contextual features. Specifically, cross-attention uses depth-sensitive features as queries and leverages contextual embeddings as keys and values, allowing the decoder to focus on relevant spatial–depth relationships. Each layer applies residual connections and normalization to stabilize feature updates, while a feed-forward network further enhances the refined representations. Through iterative processing across multiple layers, the decoder progressively aligns and integrates depth-sensitive features and contextual cues, resulting in task-specific predictions with enhanced accuracy and robustness.

The computational complexity of the transformer is as follows. Let the input feature map have spatial dimensions $H\times W$, batch size $N$ and per-position embedding dimension C, and flattened sequence length $L=H\times W$. Each attention layer uses $h$ heads with per-head dimension $D=C/h$. For each attention computation, three linear projections contribute $3\times N\times L\times C^2$ multiply-accumulate operations (MACs), while the linear Attention mechanism avoids the explicit $L\times L$ similarity matrix and requires only $N\times \left(L\times C+C^2/h\right)$ MACs, which are negligible compared to the projections. An output projection adds another $N\times L\times C^2$, resulting in a dominant per-attention complexity of approximately $4\times N\times L\times C^2$. Each layer also includes a feed-forward network contributing $2\times N\times L\times C^2$. Thus, the encoder layer (one attention + FFN) requires $6\times N\times L\times C^2$ MACs, and the decoder layer (two attentions + FFN) requires $10\times N\times L\times C^2$. The total complexity of the transformer is therefore $16\times N\times L\times C^2$, which scales linearly with $L$. In contrast, standard softmax attention incurs $O\left(N{L}^{2}C\right)$ complexity due to the quadratic term in sequence length. This linear formulation yields substantial computational benefits for high-resolution inputs.

3.3.3. Depth Position Mapping (DPM) Module

The Depth Position Mapping (DPM) module embeds depth-related positional information to enhance the model’s understanding of spatial-depth relationships, as shown in Figure 5. Given the input features, $X\in R^{B\times N\times C}$, where $\mathrm{B}$ is the batch size, $N=H\times W$ is the number of spatial positions, and $C$ is the number of spatial positions, and $F\in R^{B\times C\times H\times W}$. Based on previously predicted depth bins $D_b=\left[d_1,\dots ,d_D\right]\in R^{D\times C}$, where $D$ is the number of depth bins and $d_i$ represents the learnable embedding for the $\mathrm{i}$-th depth category, the depth features are locally refined using a depth-wise separable convolution with a $3\times 3$ kernel. The operation is defined as follows:

$$F^{\prime }={Conv}_{3\times 3}\left(F\right)+F$$

(9)

where $Con{v}_{3\times 3}$ integrates local depth information while maintaining computational efficiency. The resulting depth-sensitive feature map $F^{\prime }$ is then flattened back to ${\mathrm{R}}^{\mathrm{B}\times \mathrm{N}\times \mathrm{C}}$, ensuring alignment between the depth positional mappings and the spatial features. By embedding depth-sensitive positional cues through $D_b$, the DPM module improves the model’s ability to capture 3D geometric structures, while preserving efficiency and scalability for downstream tasks.

3.4. Loss Function

The proposed framework adopts a single-stage detector design, which directly predicts object bounding boxes and class probabilities using pre-defined 2D-3D anchors. This architecture is optimized for efficient object detection and depth estimation in a single forward pass. To handle challenges such as class imbalance and bounding box regression, the framework incorporates Focal Loss [30,31] for classification and Smooth L1 Loss [32,33] for bounding box regression. These loss functions are combined with a custom depth loss to ensure robust performance across all tasks.

For the classification task, we use Focal Loss to address class imbalance by down-weighting easy examples and emphasizing hard ones. For each positive anchor $i$, let $p_i$ denote the predicted probability that the anchor belongs to the positive class, and let $p_i^{gt}\in \left\{\mathrm{0,1}\right\}$ be the ground-truth label. The Focal Loss for a single anchor is defined as follows:

$$FocalLoss\left(p_i,p_i^{gt}\right)=\left\{\begin{array}{c}-\alpha {\left(1-p_i\right)}^{\mathit{\gamma }\mathit{log}}{p}_i,if p_i^{gt}=1,\\ -\left(1-\alpha \right)p_i^{\mathit{\gamma }\mathit{log}}\left(1-p_i\right),if p_i^{gt}=0,\end{array}\right.$$

(10)

where $p_i$ is the predicted probability of anchor $i$ being positive, and $p_i^{gt}$ is the ground-truth label for anchor $i$ (either 0 or 1). $\mathrm{\alpha }$ is a balancing factor that controls the importance of positive versus negative examples, and $\gamma$ is a focusing parameter that reduces the loss contribution from easy examples and focuses on harder ones. The loss is averaged over all positive anchors:

$$L_{cls}={\displaystyle \frac{1}{N_{pos}}}\sum _{i=1}^{N_{pos}}FocalLoss\left(p_i,p_i^{gt}\right),$$

(11)

where ${\mathrm{N}}_{\mathrm{pos}}$ is the number of positive anchors.

For bounding box regression, we adopt Smooth L1 Loss. Let ${\delta }_i=b_i-b_i^{gt}$ be the difference between the predicted bounding box ${\mathrm{b}}_{\mathrm{i}}$ and the ground-truth bounding box $b_i^{gt}$, where $b_i$ and $b_i^{gt}$ represent the predicted and ground-truth bounding box parameters, respectively. The Smooth L1 Loss is defined as follows:

$$SmoothL1\left({\delta }_i\right)=\left\{\begin{array}{c}0.5\times {\displaystyle \frac{{\delta }_i^2}{\beta }},if \left|{\delta }_i\right|<\beta ,\\ \left|{\delta }_i\right|-0.5\times \beta ,if \left|{\delta }_i\right|\ge \beta ,\end{array}\right.$$

(12)

where $\beta$ is a threshold that controls the transition between L1 and L2 loss. This loss is less sensitive to outliers compared to L2 loss and is computed as follows:

$${\mathcal{L}}_{reg}={\displaystyle \frac{1}{N_{\mathit{pos}}}}\sum _{i=1}^{N_{\mathit{pos}}}SmoothL1\left(b_i,b_i^{gt}\right)$$

(13)

where $N_{pos}$ is the number of positive anchors, and ${\delta }_i=b_i-b_i^{gt}$ is the difference between the predicted and ground-truth bounding box parameters.

In addition to the classification and regression losses, we introduce a depth estimation loss to enable depth-sensitive feature learning. To do this, we discretize the continuous depth range into $D$ bins, and the model outputs a probability distribution over these bins for each pixel. Let $d_{i,b}^{gt}\in \left[\mathrm{0,1}\right]$ denote the ground-truth probability for depth bin $b$ at pixel $i$, and $d_{i,b}\in \left[\mathrm{0,1}\right]$ denote the predicted probability for that bin. The depth loss is then formulated as follows:

$$L_{depth}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\sum _{b=1}^D{w}_{i,b}{d}_{i,b}^{gt}\mathit{log}{d}_{i,b}$$

(14)

where $\mathrm{N}$ is the total number of valid pixels, and $w_{i,b}={\left(1-d_{i,b}^{gt}\right)}^{-\gamma }$ is a weighting factor that emphasizes harder-to-predict depth bins. The term ${\mathrm{w}}_{\mathrm{i},\mathrm{b}}$ follows the same principle as Focal Loss, where bins with higher uncertainty or difficulty are weighted more heavily. Depending on whether a one-hot or soft labeling strategy is used, $d_{i,b}^{gt}$ can be 1 for the true bin and 0 otherwise, or distributed across several bins in a soft fashion.

$$L_{total}=L_{cls}+{\lambda }_{reg }\times L_{reg}+{\lambda }_{depth }\times L_{depth}.$$

(15)

where ${\lambda }_{reg }$ and ${\lambda }_{depth }$ control the weights of the regression and depth losses, respectively. During training, anchors with an intersection-over-union (IoU) greater than 0.5 with the ground truth boxes are selected for optimization. By combining classification, regression, and depth estimation objectives, the proposed loss function ensures a balanced and effective training process. The use of Focal Loss and Smooth L1 Loss further enhances the robustness of the framework in handling challenging 3D object detection scenarios.

4. Experiment and Analysis

4.1. Dataset

The KITTI dataset is a widely used benchmark for autonomous driving research, providing labeled data for tasks such as 3D object detection, stereo vision, and SLAM. It includes 7481 training images and 7518 test images, captured with stereo cameras and LiDAR in real-world driving scenarios. KITTI evaluates 3D object detection across three categories (Car, Pedestrian, Cyclist) and three difficulty levels (Easy, Moderate, Hard), making it a standard for testing perception algorithms. Consistent with prior work [34], we partitioned the 7481 training images into 3712 training samples and 3769 validation samples. The detailed distribution of object instances in this split is summarized in

Table 1. Data distribution of KITTI Training and validation splits used for monocular 3D detection.

Dataset	Images	Car Instances	Pedestrian Instances	Cyclist Instances
Training	3712	14,357	2207	734
Validation	3769	14,385	2280	893

.

4.2. Valuation Metrics

The KITTI dataset evaluates the performance of 3D object detection and bird’s-eye view (BEV) detection using Average Precision (AP) as the primary metric. To ensure a more accurate evaluation, it adopts the $A{P}_{40}$ metric, which calculates the Average Precision across 40 evenly spaced recall positions, reducing potential bias from fewer recall points in the original AP metric.

Detection tasks are divided into three difficulty levels—Easy, Moderate, and Hard—based on object attributes such as occlusion, truncation, and size. The dataset includes three object categories: Car, Pedestrian, and Cyclist. The evaluation employs intersection over union (IoU) thresholds of 0.7 for the Car category and 0.5 for Pedestrian and Cyclist.

4.3. Implementation Details

The proposed AuxDepthNet framework was evaluated on the KITTI dataset, targeting the Car category. Input images were resized to a resolution of $1280\times 288$. The model was trained for 120 epochs with a batch size of 16, using the Adam optimizer with an initial learning rate of $1\times {10}^{-4}$ and no weight decay. A cosine annealing learning rate scheduler was employed, with the minimum learning rate set to $5\times {10}^{-6}$.

To enhance the robustness of the model, several data augmentation techniques were applied during training. These included photometric distortions (e.g., random adjustments to brightness, contrast, saturation, and hue), cropping the top 100 pixels of the image, random horizontal flipping with a probability of 0.5, resizing to the target resolution, and normalization using RGB mean values [0.485, 0.456, 0.406] and standard deviations [0.229, 0.224, 0.225].

For testing, augmentation was limited to cropping, resizing, and normalization without photometric distortions or flipping. The detection head employed an anchor-based design, with IoU thresholds of 0.5 for positive samples and 0.4 for negative samples. The loss function combined Focal Loss with Smooth L1 Loss to balance the classification and regression tasks. During inference, non-maximum suppression (NMS) was applied with an IoU threshold of 0.4, and predictions with a confidence score below 0.75 were discarded.

To illustrate the convergence behavior of the model during training, we plotted the loss value across iterations. As shown in Figure 6, the training loss consistently decreases and stabilizes, demonstrating the effective optimization and convergence of the proposed framework.

As illustrated in Figure 6, the horizontal axis represents the number of iterations during training, while the vertical axis denotes the loss value. The curve demonstrates that the loss decreases rapidly in the early training phase and gradually stabilizes as training progresses. This behavior indicates that the model effectively learns from the data and reaches convergence after approximately 15,000 iterations. The stable and low final loss confirms the robustness and reliability of the training procedure.

4.4. Comparison with State-of-the-Art Methods

To evaluate the effectiveness of AuxDepthNet, we compare its performance with state-of-the-art monocular 3D object detection methods on the KITTI 3D Object Detection Benchmark, as shown in

Table 2. Detection performance of Car category on the KITTI validation set; the best and second-best results are highlighted in red and blue, respectively.

Method	AP3D@IoU = 0.7			APBEV@IoU = 0.7
	Easy	Mod.	Hard	Easy	Mod.	Hard
DDMP-3D [18]	19.71	12.78	9.80	28.08	17.89	13.44
CaDDN [21]	19.17	13.41	11.46	27.94	18.91	17.19
DFRNet [35]	19.40	13.63	10.35	28.17	19.17	14.84
MonoEF [36]	21.29	13.87	11.71	29.03	19.70	17.26
MonoFlex [37]	19.94	13.89	12.07	28.23	19.75	16.89
GUPNet [38]	20.11	14.20	11.77	-	-	-
MonoDTR [39]	21.99	15.39	12.73	28.59	20.38	17.14
MonoJSG [40]	24.69	16.14	13.64	32.59	21.26	18.18
DEVIANT [41]	21.88	14.46	11.89	29.65	20.44	17.43
DCD [42]	23.94	17.38	15.32	32.55	21.50	18.25
DID-M3D [43]	24.40	16.29	13.75	32.95	22.76	19.83
Cube R-CNN [44]	23.59	15.01	12.56	31.70	21.20	18.43
MonoUNI [45]	24.75	16.73	13.49	33.28	23.05	19.39
MonoATT [46]	24.72	17.37	15.00	36.87	24.42	21.88
DPL [47]	24.19	16.67	13.83	33.16	22.12	18.74
DPM3D [48]	23.41	13.65	12.91	32.23	20.13	17.14
AuxDepthNet (Ours)	24.72	18.63	15.31	34.11	25.18	21.90

. The benchmark measures Average Precision (AP) under IoU 0.7 for both 3D detection ${AP}_{3D}$ and bird’s-eye view detection ${AP}_{BEV}$ across three levels: Easy, Moderate, and Hard.

From Table 2, AuxDepthNet achieves competitive results. For 3D detection, it obtains an AP of 24.72, 18.63, and 15.31 on Easy, Moderate, and Hard levels, respectively. Compared to MonoUNI with 24.75, 16.73, and 13.43, our method improves the Moderate and Hard scores by 1.90 and 1.88 points. Similarly, AuxDepthNet outperforms Cube R-CNN, which achieves 23.59, 15.01, and 12.65, across all three categories.

For BEV detection, AuxDepthNet achieves an AP of 34.11, 25.18, and 21.90. Compared to MonoATT, which achieves 36.87, 24.42, and 21.88, our method improves the Moderate score by 0.76 points while maintaining competitive performance on other levels.

Overall, AuxDepthNet surpasses MonoUNI, Cube R-CNN, and MonoATT, especially on Moderate and Hard levels. These results demonstrate the effectiveness of our Depth Position Mapping (DPM) module in capturing depth-sensitive features and enhancing 3D detection accuracy on the KITTI benchmark.

4.5. Ablation Studies and Analysis

4.5.1. Effectiveness of Each Proposed Components

To assess the impact of each component in the AuxDepthNet framework, we conducted ablation studies highlighting their contributions to overall performance. (a) The baseline utilizes only context-aware features for 3D object detection, without incorporating depth-sensitive modules. (b) Context-aware and depth-sensitive features were integrated using a convolutional concatenate operation as an alternative to the depth-sensitive transformer module. (c) Depth-sensitive features in the transformer were replaced with object queries similar to DETR, creating a baseline with a DETR-like transformer. (d) Depth-sensitive features were replaced with those extracted from depth images generated by a pretrained depth estimator (DORN), instead of being learned in an end-to-end manner.

The results in

Table 3. Analysis of AuxDepthNet components on the KITTI validation set (Car category).

Index	Ablation	AP3D@IoU = 0.7
		Easy	Mod.	Hard
(a)	Baseline	19.50	15.51	12.66
(b)	w/o depth prototype enhancement	23.91	18.27	15.21
(c)	Depth-sensitive feature → object query	20.25	16.15	13.89
(d)	Depth-sensitive feature → DORN [49]	24.27	17.15	13.84
(e)	AuxDepthNet (full model)	24.72	18.63	15.31

highlight several key observations: substituting object queries in the transformer with depth-sensitive features (c→e) significantly enhances depth representation, resulting in notable performance improvements. Furthermore, using features derived from a pretrained depth estimator (d) underperforms compared to the end-to-end approach in our full framework (e), emphasizing the benefits of directly learning depth-sensitive features. Our depth-sensitive transformer module (e) effectively combines context- and depth-sensitive features, outperforming the simpler convolutional concatenation method (b). Additionally, the inclusion of the depth prototype enhancement module (d→e) further refines detection accuracy, particularly under more challenging scenarios. Ultimately, the complete AuxDepthNet model (e) delivers marked advancements across all levels of difficulty relative to the baseline (a), demonstrating the combined contributions of the proposed modules to robust and precise 3D object detection.

4.5.2. The Impact of Different Backbones

To further evaluate the impact of different Backbone networks on the performance of 3D object detection for the Car category in the KITTI validation set, we conducted an ablation study comparing various backbones. Specifically, we tested DLA-102, DLA-102x2, DLA-60, DenseNet, ResNet, and our proposed AuxDepthNet (Ours). Among these, our framework employs DLA-102 as the default Backbone due to its balanced trade-off between efficiency and accuracy. The results demonstrate that while other backbones provide competitive performance, DLA-102 achieves superior precision, validating its suitability for our approach. The results are shown in

Table 4. Comparison of different backbones on KITTI validation set for Car category; the table presents the performance (${AP}_{3D}$ and ${AP}_{\mathrm{B}\mathrm{E}\mathrm{V}}$ @ IoU = 0.7) using various backbones, including our proposed AuxDepthNet.

Backbone	AP3D@IoU = 0.7			APBEV@IoU = 0.7
	Easy	Mod.	Hard	Easy	Mod.	Hard
DLA-102	24.40	18.55	15.30	34.02	25.11	21.24
DLA-102x2	24.59	18.56	15.25	34.01	25.13	21.32
DLA-60	22.85	17.37	14.33	32.14	23.64	19.92
DenseNet	24.24	18.04	15.14	33.01	24.69	21.48
ResNet	24.09	17.98	15.15	32.97	24.39	21.33
Ours	24.72	18.63	15.31	34.11	25.18	21.90

.

The ablation study highlights the impact of different backbones on KITTI’s Car category detection. DLA-102 achieves a strong balance between efficiency and accuracy, while DLA-102x2 slightly outperforms it, particularly in the Easy and Moderate settings. DenseNet and ResNet provide competitive results but fall short of DLA-based models, and DLA-60 demonstrates limited performance due to reduced capacity. Notably, our proposed AuxDepthNet achieves the highest accuracy, with an ${AP}_{3\mathrm{D}}$ of 18.63 (Moderate) and ${AP}_{BEV}$ of 25.18 (Moderate), showcasing its superior depth-sensitive feature extraction capabilities for monocular 3D detection. So, our AuxDepthNet Backbone outperforms conventional backbones, highlighting the effectiveness of its design in enhancing depth-sensitive feature representation.

4.5.3. The Impact of Different Dilation Rates in the ADF Module

To investigate how various dilation rates in the ADF module affect 3D object detection performance, we conducted ablation experiments on the validation set of the KITTI dataset, focusing on the Car category. Specifically, we set the dilation rates to standard (no dilated convolution), 2, 8, 16, or 4 (Ours), all under the same network architecture and training hyperparameters, and evaluated their ${AP}_{3D}$ (Average Precision in 3D) metrics. As shown in Figure 7, different dilation rate configurations yielded varying results across the Easy, Moderate, and Hard difficulty levels. Notably, a dilation rate of 4 achieves the highest detection accuracy, reaching 24.72%, 18.63%, and 15.31% on Easy, Moderate, and Hard, respectively—outperforming the other settings. These findings suggest that using a suitable dilation rate helps capture more effective multi-scale features, thereby significantly enhancing the accuracy of 3D object detection. Hence, we chose a dilation rate of 4 for all subsequent experiments to achieve optimal performance.

5. Visualization

Figure 8 presents qualitative examples from the KITTI validation set. Compared to the baseline model without depth-sensitive modules, AuxDepthNet predictions align much more closely with the ground truth, demonstrating the effectiveness of the proposed depth-sensitive modules in enhancing object localization accuracy.

6. Conclusions

In this study, we proposed AuxDepthNet, a framework for real-time monocular 3D object detection that eliminates the need for external depth maps or pre-trained depth models. By introducing the Auxiliary Depth Feature Module (ADF) and the Depth Position Mapping Module (DPM), AuxDepthNet effectively learned depth-sensitive features and integrated depth positional information, enhancing spatial reasoning with minimal computational cost. Built on the DepthFusion Transformer architecture, the framework demonstrated robust performance in object localization and 3D bounding box regression.

Although AuxDepthNet does not rely on any externally pre-computed depth maps or pre-trained depth estimation networks, it still requires dataset-specific information, such as camera calibration parameters and anchor box statistics derived from the KITTI dataset. These configurations are standard hyper-parameters that can be re-estimated for other datasets with appropriate 3D annotations and calibration data. Therefore, the framework remains general in its architecture and learning process, without depending on handcrafted priors unique to KITTI.

Despite its effectiveness, AuxDepthNet has limitations, including its focus on the KITTI dataset and potential challenges in generalizing to diverse environments. Future work will explore broader dataset adaptation, improve robustness under extreme conditions, and optimize computational efficiency for edge applications.