UDF-3D: Uncertainty-Driven Decision-Level Fusion for Camera–LiDAR 3D Object Detection

Abstract

Camera and LiDAR provide highly complementary information, and effective fusion of both modalities is desirable for 3D object detection. However, existing decision-level fusion methods mainly rely on the confidence of objects while neglecting the object uncertainty. To address this, we propose UDF-3D, an uncertainty-driven camera–LiDAR decision-level fusion method based on Dempster–Shafer evidence theory. First, object uncertainty is quantified by introducing the theory of subjective logic, where subjective opinions incorporate category belief masses and an uncertainty mass. Second, a cost matrix is designed for object matching, where each element is a weighted combination of geometric and semantic information from both sensors, and the weights are determined by the uncertainty parameters. Third, we construct a view-frustum constraint to re-evaluate unmatched objects, thereby reducing the false-negative rate. Finally, we design a novel evidence discounting factor within the Dempster–Shafer framework for matched objects, thereby mitigating cross-modal object conflicts during fusion and improving detection accuracy. Experiments on the KITTI dataset demonstrate that the proposed method outperforms existing decision-level fusion approaches, yielding improved detection accuracy.

1. Introduction

The two most widely used sensing modalities for object detection are RGB cameras and Light Detection and Ranging (LiDAR) [1]. These sensors are highly complementary in terms of information representation and perceptual capability. Cameras provide high-resolution images with rich texture and appearance cues, which are thus well-suited for object classification, 2D detection, and semantic segmentation. However, due to the lack of explicit depth information, it is difficult to obtain precise spatial location information from images. LiDAR directly measures sparse and irregular 3D point clouds, providing reliable geometric and range information and obtaining accurate characterization of the 3D scene structure [2]. Nevertheless, single-modality 3D detection methods still suffer from notable limitations. Image-based methods struggle to achieve high-precision 3D bounding box regression because depth and structural cues are missing. While LiDAR-based methods perform well for nearby and large objects (e.g., cars and trucks), they often miss distant or small objects (e.g., pedestrians and cyclists) due to extremely sparse points and may produce false positives when background structures exhibit geometries similar to target objects [3,4,5,6,7,8]. In addition, early approaches that project point clouds into view images or voxel grids inevitably lose fine-grained 3D details during quantization and projection, which constrains the upper bound of detection accuracy. Therefore, it is necessary to develop a fusion method to fully exploit the complementary information of camera images and LiDAR point clouds for robust object detection.

According to the fusion stage, existing fusion methods can be broadly categorized into data-level fusion, feature-level fusion, and decision-level fusion. Data-level and feature-level fusion align and interact multi-modal information at the perception front-end or intermediate network layers. In principle, they can well exploit cross-modal complementarity, but they typically rely on accurate spatial–temporal calibration, are sensitive to misalignment errors, and often introduce complex network designs with challenging training procedures [9]. Decision-level fusion associates and merges candidate boxes at the detection output level across modalities. It leverages mature single-modality detectors directly, resulting in simple system implementation and high engineering flexibility [10]. Consequently, a key advantage of decision-level fusion lies in its ability to fully exploit useful information from each modality, leading to more reliable final decisions.

Current camera–LiDAR decision-level fusion for 3D object detection still faces two key challenges. First, most decision fusion methods treat the class confidence scores produced by detectors as reliable probabilities, which lacks interpretability and makes it difficult to determine which modality should be trusted more under conflicts [11]. Second, the fusion of detection results itself often remains at a relatively shallow level. A typical practice is to perform target matching first, concatenate a few scalar features such as modality-specific scores and IoU, and then feed them into a lightweight scoring module or apply simple weighting to obtain a fused confidence score, while directly taking the matched 3D boxes as the final outputs. The core contribution of this work is UDF-3D, an uncertainty-driven decision-level fusion framework. The main innovations are summarized as follows.

(1) An uncertainty-driven object matching method is proposed for camera–LiDAR decision-level fusion. Unlike conventional decision-level matching strategies that rely only on bounding-box geometric overlap or classification confidence, this paper represents camera and LiDAR detection outputs as subjective opinions containing class belief masses and object uncertainty masses and uses the uncertainty to adaptively adjust the weights of geometric consistency and semantic consistency in the matching cost, thereby reducing invalid candidate correspondences and improving cross-modal matching accuracy.

(2) We propose a new camera–LiDAR decision-level fusion method, whose core consists of an improved evidence discounting fusion strategy based on Dempster–Shafer evidence theory and a frustum re-evaluation module. The improved evidence discounting fusion is used to alleviate cross-modal conflicts during the fusion process, thereby improving the accuracy of fusion-based detection. The frustum re-evaluation module targets the missed-detection problem by using unmatched 2D detections as semantic and geometric anchors to recover true objects that may have been suppressed by NMS from the pre-NMS LiDAR candidate set. This method combines the frustum constraint with the proposed evidence discounting fusion, forming a complete missed-object re-evaluation and conflict suppression process, thereby improving object recall and the reliability of fusion-based detection.

The remainder of this paper is organized as follows. Section 2 reviews related work on camera–LiDAR 3D object detection. Section 3 presents the proposed uncertainty-driven decision-level fusion framework, including object uncertainty quantification, uncertainty-weighted object matching, frustum re-evaluation, and uncertainty-discounted decision-level fusion. Section 4 reports experimental results and ablation studies on the public KITTI dataset to evaluate the effectiveness of the proposed method under the standard KITTI evaluation protocol. Section 5 concludes the paper and discusses future research directions.

2. Related Work

Camera–LiDAR fusion methods are commonly categorized into three paradigms according to the fusion stage: data-level, feature-level, and decision-level fusion.

Data-level fusion focuses on directly integrating multi-modal information at the raw-data stage to preserve observation fidelity as much as possible [12]. For example, PointPainting proposed by Vora et al. [13] projects LiDAR points onto the output of an image semantic segmentation network and assigns a class score to each point, thereby significantly increasing the information density of the original point cloud without altering the LiDAR backbone architecture. Similarly, Wen et al. [14] map RGB image features onto corresponding point clouds via a dedicated point-feature fusion module, achieving lightweight data-level fusion outside the main backbone network. Subsequent works such as PointAugmenting [15] and FusionPainting [16] further improve multi-modal detection performance by introducing cross-modal data augmentation or adaptive point painting strategies while keeping the LiDAR detector largely unchanged. Although these methods are advantageous in retaining information completeness, they are often highly sensitive to extrinsic calibration and temporal synchronization accuracy, adapt poorly to differences in sensor resolution and field of view (FoV), and may incur non-negligible overhead in computational complexity and system deployment.

Compared with data-level methods, feature-level fusion emphasizes deep interaction between multi-modal features at intermediate layers of neural networks. One representative early work is MV3D proposed by Chen et al. [17], which fuses multi-view features through dedicated subnetworks after multi-view data encoding, serving as a typical example of early feature-level fusion. EPNet++ [18] further introduces cascade bi-directional fusion between image and point-cloud features to strengthen multi-modal representation. DeepFusion [19] aligns and fuses camera features with deep LiDAR backbone features rather than performing “painting” only at the input point level, substantially improving the representation capacity and robustness of high-level multi-modal features. This indicates that cross-modal alignment and fusion in a more abstract feature space can yield better detection performance. Building on this line of work, BEVFusion [20] encodes LiDAR point clouds and multi-view images separately and projects them into a shared bird’s-eye-view (BEV) space, where multi-modal feature fusion is performed. This design preserves geometric structure while enhancing semantic density, achieving strong performance in multi-task and multi-sensor 3D perception settings. Chen et al. [21] further construct a dual-feature interaction module using Transformers, adopting a soft-fusion strategy for bidirectional deep interaction between LiDAR and camera features, and introduce an uncertainty-based 3D IoU metric to model the uncertainty induced by coupling among 3D attributes, thereby alleviating error propagation and improving accuracy and stability. Zhang et al. [22] propose the Adaptive Fusion Transformer (AFTR), which explicitly models spatial and temporal relationships among multi-sensor BEV features within the detection backbone. By leveraging adaptive spatial cross-attention and spatio-temporal self-attention, AFTR performs unified cross-modal and cross-time feature modeling and demonstrates stronger robustness in complex traffic scenarios. More recently, uncertainty-aware BEV fusion has also been explored. Xu et al. [23] propose UncertainBEV, an uncertainty-aware BEV fusion framework for roadside 3D object detection. By modeling the uncertainty of camera and LiDAR BEV features, UncertainBEV dynamically adjusts the fusion weights to alleviate cross-modal feature misalignment and improve BEV feature representation. While feature-level fusion methods can exploit the complementary strengths of different sensors through deep fusion, they are typically parameter-heavy and structurally complex, sensitive to data scale and training strategy, and may suffer from high training cost and overfitting risk.

Decision-level fusion combines independently generated outputs from modality-specific detectors to enhance overall robustness and flexibility. A typical pipeline runs LiDAR and camera detectors in parallel and then performs cross-modal matching and fusion at the candidate-box level. The CLOCs framework proposed by Pang et al. [24] jointly models candidate objects from cameras and LiDAR after any given 2D and 3D detectors and before non-maximum suppression (NMS). By learning geometric and semantic consistency, it produces fusion scores that significantly improve final 3D detection results without modifying the original detector architectures. Its subsequent version, Fast-CLOCs [25], further simplifies the candidate-level network to achieve near real-time, high-accuracy decision-level fusion. Shen et al. [26] propose a generalized optimization-based multi-modal fusion framework that formulates a late-fusion optimization problem using a bidirectional nearest-neighbor probabilistic model and 3D object tracking, improving accuracy and robustness without additional training. These studies suggest that decision-level fusion has inherent advantages in handling multi-modal heterogeneity, reusing existing detector architectures, and smoothly integrating new modalities, making it particularly suitable for incremental upgrades and extensions in engineering systems.

However, the performance ceiling of decision-level fusion is still largely constrained by the underlying single-modality detectors. A common issue is the overconfidence of single-modality outputs. For both LiDAR-based and image-based detectors, the final class confidence is often trained as a highly “polarized” point estimate: predictions deemed correct receive scores close to 1, whereas incorrect predictions receive very low scores. However, such scores do not effectively distinguish between two fundamentally different cases: well-evidenced certainty and miscalibrated overconfidence induced by model bias. In typical scenarios where cross-modal information conflicts (e.g., distant objects with extremely sparse point clouds, adverse illumination, or partial occlusion), two mutually contradictory yet highly confident detections can cause traditional fusion strategies to fail. Simple score weighting or purely geometry-based matching is often insufficient to produce reliable fusion results. This motivates decision-level fusion strategies that explicitly model uncertainty to improve the accuracy of subsequent fusion decisions.

3. Method

We propose UDF-3D, an uncertainty-driven decision-level fusion framework for camera–LiDAR 3D object detection, as illustrated in Figure 1. The framework involves three stages: (1) object uncertainty quantification, (2) uncertainty-weighted cross-modal object matching, and (3) frustum re-evaluation and uncertainty-discounted decision-level fusion.

3.1. Object Uncertainty Quantification

Inspired by evidential deep learning (EDL) [27] and its application to object-detection uncertainty quantification [28], we design an uncertainty quantification scheme that models the class probability vector with a Dirichlet distribution. The concentration parameters are constructed by mapping the pre-activation logits to non-negative evidence, and the belief mass and overall uncertainty mass are then derived under subjective logic [29]. For decision-level fusion, uncertainty provides additional reliability information beyond the original detection outputs. The fusion module operates at the detection-output level, where the inputs mainly include candidate boxes and class scores. These outputs indicate what each detector predicts, but they do not sufficiently describe whether the prediction itself is reliable. Therefore, we adopt a representation that can simultaneously describe class-level evidence support and overall detection uncertainty. The class belief masses provide a structured representation for measuring semantic consistency between camera and LiDAR detections, while the uncertainty mass serves as an explicit reliability indicator for adaptive matching and subsequent evidence fusion. Compared with probabilistic uncertainty estimation methods such as MC Dropout, Bayesian neural networks, or ensemble-based approaches, which usually require multiple forward passes, sampling, or model ensembles, the adopted representation can be obtained from a single forward pass of each detector with minimal additional computational cost. This makes it suitable for a lightweight decision-level fusion framework. Based on this representation, the detection outputs are converted into belief masses and uncertainty masses for subsequent matching and fusion. The detailed procedure is as follows.

Let K denote the number of categories, and let $\mathbf{z}=[z_1,z_2,\dots ,z_K]\in {\mathbb{R}}^K$ be the network’s pre-activation outputs. The non-negative evidence is obtained through a $\mathrm{softplus}$ mapping

$$\mathbf{e}=\mathrm{softplus}\left(\mathbf{z}\right)=[e_1,e_2,\dots ,e_K],e_i\ge 0,i=1,\dots ,K.$$

(1)

Assume that the class-probability vector $\mathbf{p}={[p_1,\dots ,p_K]}^{\top }$ lies on the K-simplex, i.e., $p_i\ge 0$ and ${\sum }_{i=1}^K{p}_i=1$, and that $\mathbf{p}$ follows a Dirichlet distribution:

$$\mathbf{p}\sim \mathrm{Dir}\left(\mathsf{\alpha }\right),\mathsf{\alpha }=[{\alpha }_1,\dots ,{\alpha }_K].$$

(2)

The Dirichlet density is given by

$$\mathrm{Dir}(\mathbf{p}\mid \mathsf{\alpha })={\displaystyle \frac{\Gamma \left({\sum }_{i=1}^K{\alpha }_i\right)}{{\prod }_{i=1}^K\Gamma \left({\alpha }_i\right)}}{\displaystyle \prod _{i=1}^K}{p}_i^{{\alpha }_i-1}.$$

(3)

Then, we model the class-probability vector with a Dirichlet distribution. Since a Dirichlet distribution is uniquely determined by its concentration parameters $\mathsf{\alpha }=[{\alpha }_1,{\alpha }_2,\dots ,{\alpha }_K]$, it suffices to construct $\mathsf{\alpha }$. In particular, we incorporate a base-rate prior a and a prior weight W to form

$${\alpha }_i=e_i+a_{i}W,i=1,\dots ,K.$$

(4)

In practice, we set the base rate to a uniform distribution $a_i={\displaystyle \frac{1}{K}}$ to avoid introducing any prior preference among classes, and choose the prior weight as $W=K$ so that the prior term $a_{i}W$ equals 1 for all classes. This simplifies the Dirichlet parameters to

$$\mathsf{\alpha }=\mathbf{e}+\mathbf{1}.$$

(5)

Let S denote the Dirichlet strength, $S={\sum }_{j=1}^K{\alpha }_j$. Following subjective logic, the belief mass assigned to class i is defined as

$$b_i={\displaystyle \frac{e_i}{S}},i=1,\dots ,K.$$

(6)

Accordingly, under the Dirichlet distribution, the expected probability that the target belongs to class i and the overall uncertainty mass are given by [29]

$$\mathbb{E}\left[p_i\right]={\displaystyle \frac{{\alpha }_i}{S}},u={\displaystyle \frac{K}{S}}.$$

(7)

Here, the belief mass $b_i$ represents the degree of evidence support assigned to class i, and its value is determined by the proportion of the evidence $e_i$ for this class in the total Dirichlet strength S. The uncertainty mass u represents the overall uncertainty that is not assigned to any specific class. When the total evidence is limited, S is smaller and $u=K/S$ becomes larger, indicating that the detection result lacks sufficient evidential support. The expected probability $\mathbb{E}\left[p_i\right]$ denotes the mean probability of class i under the Dirichlet distribution, which combines the belief mass of this class with the uncertainty distributed according to the base rate. It should be noted that $u+{\sum }_{i=1}^K{b}_i=1$, indicating the complete allocation of belief mass and uncertainty mass.

3.2. Uncertainty-Weighted Cross-Modal Object Matching

For decision-level fusion, given image and LiDAR data captured at the same instant, each modality predicts multiple distinct detection boxes. Consequently, it is necessary to establish a one-to-one correspondence between detection boxes across the two modalities. The detailed workflow of the proposed uncertainty-weighted cross-modal matching module is summarized in Algorithm 1. Considering the inherent correlation between target geometry and category across different modalities, we construct a similarity matrix based on geometric and semantic consistency and incorporate uncertainty information to assign weights to these two factors. The specific details are as follows.

We denote the set of camera detections and each camera detection as

$$\begin{array}{cc}\hfill {\mathcal{B}}^C& =\{B_1^C,B_2^C,\dots ,B_M^C\},\hfill \\ \hfill B_i^C& =\{[x_i^C,y_i^C,h_i^C,w_i^C],{\mathbf{b}}_i^C,u_i^C\}.\hfill \end{array}$$

(8)

where M is the number of camera detections, $(x_i^C,y_i^C)$ is the center of the 2D box, $h_i^C$ and $w_i^C$ are its height and width, ${\mathbf{b}}_i^C$ is the belief-mass vector, and $u_i^C$ is the uncertainty of the i-th camera detection. Similarly, the set of LiDAR detections and each LiDAR detection are denoted as

$$\begin{array}{cc}\hfill {\mathcal{B}}^L& =\{B_1^L,B_2^L,\dots ,B_N^L\},\hfill \\ \hfill B_j^L& =\{[x_j^L,y_j^L,z_j^L,l_j^L,w_j^L,h_j^L,{\theta }_j^L],{\mathbf{b}}_j^L,u_j^L\}.\hfill \end{array}$$

(9)

Here, N is the number of LiDAR detections, $(x_j^L,y_j^L,z_j^L)$ is the 3D box center in the LiDAR coordinate system, $l_j^L$, $w_j^L$, and $h_j^L$ denote the 3D box dimensions, ${\theta }_j^L$ is the heading angle, ${\mathbf{b}}_j^L$ is the belief-mass vector, and $u_j^L$ is the uncertainty of the j-th LiDAR detection. For geometric matching, the LiDAR 3D box is projected onto the image plane, and $({\tilde{x}}_j^L,{\tilde{y}}_j^L)$, ${\tilde{h}}_j^L$, and ${\tilde{w}}_j^L$ denote the center, height, and width of its projected 2D box.

Algorithm 1 Uncertainty-weighted cross-modal matching

Input: Camera detections ${\mathcal{B}}^C$, LiDAR detections ${\mathcal{B}}^L$.

Output: Matching result.

1: function  UWeightedMatching$({\mathcal{B}}^C,{\mathcal{B}}^L)$    2:       Project each LiDAR box $B_j^L$ onto the image plane    3:       for each pair $(B_i^C,B_j^L)$ do    4:             Compute normalized center distance $d_{ij}^{\mathrm{norm}}$    5:             if $d_{ij}^{\mathrm{norm}}\le {\tau }_d$ then    6:                  $S_{ij}\leftarrow w_g{\mathrm{IoU}}_{ij}+w_s{L}_{\mathrm{sem},ij}$    7:                  $C_{ij}\leftarrow 1-S_{ij}$    8:             else    9:                  $C_{ij}\leftarrow T$  10:           end if  11:     end for  12:     Matching result ← HungarianAssignment$\left(\mathbf{C}\right)$  13:     return Matching result  14: end function

Normalized center-distance gating To reduce computational burden, we first introduce normalized center-distance gating to filter out candidate pairs with a low matching probability. The normalized center distance is given by

$$d_{ij}^{\mathrm{norm}}=\sqrt{{\left({\displaystyle \frac{x_i^C-{\tilde{x}}_j^L}{{\overline{w}}_{ij}}}\right)}^2+{\left({\displaystyle \frac{y_i^C-{\tilde{y}}_j^L}{{\overline{h}}_{ij}}}\right)}^2}$$

(10)

Here, $(x_i^C,y_i^C)$ and $({\tilde{x}}_j^L,{\tilde{y}}_j^L)$ are the center coordinates of the i-th camera detection box and the projected 2D box of the j-th LiDAR detection, respectively. Their average height and width are calculated as ${\overline{h}}_{ij}=(h_i^C+{\tilde{h}}_j^L)/2$ and ${\overline{w}}_{ij}=(w_i^C+{\tilde{w}}_j^L)/2$, respectively. We use image detection boxes as the reference and form candidate pairs with neighboring LiDAR detection boxes. Based on the normalized center distance, the candidate pair set is defined as

$$\mathcal{G}=\left\{(i,j)\mid d_{ij}^{\mathrm{norm}}\le {\tau }_d\right\},$$

(11)

where ${\tau }_d$ is the normalized center-distance threshold. If the normalized center distance between two detection boxes is excessively large, the pair is deemed to lack matching potential and is excluded from the subsequent Hungarian matching algorithm. Only candidate pairs belonging to $\mathcal{G}$ are retained.

Geometric consistency In practice, the bounding boxes of the same object in the image and point cloud have consistent geometric boundaries after coordinate alignment. Therefore, we use the Intersection over Union (IoU) metric to measure geometric consistency, defined as

$${\mathrm{IoU}}_{ij}={\displaystyle \frac{o_j^L\cap o_i^C}{o_j^L\cup o_i^C}},$$

(12)

where $o_j^L$ and $o_i^C$ denote the 2D boxes corresponding to the j-th LiDAR detection (after 3D-to-2D projection) and the i-th camera detection, respectively.

However, geometric overlap alone may be insufficient for reliable matching because projected boxes, especially small boxes at long range, can be sensitive to localization and calibration errors.

Semantic consistency To complement the geometric relationship, we incorporate semantic consistency into the matching process. In this paper, we construct a semantic similarity measure using belief masses to reflect whether the two sensors provide consistent opinions about the same object. The belief masses derived from the evidence vector represent the opinion associated with each detection box, and the Bhattacharyya coefficient is adopted to compute the similarity between the belief-mass vectors of the two modalities. Accordingly, the semantic similarity is defined as

$$L_{\mathrm{sem},ij}={\displaystyle \sum _{k=1}^K}\sqrt{b_{j,k}^L{b}_{i,k}^C}$$

(13)

where $b_{j,k}^L$ and $b_{i,k}^C$ denote the belief masses of the j-th LiDAR detection and the i-th camera detection for class k, respectively, where K is the number of classes.

Uncertainty-Weighted Matching (U-weighted Matching) The proposed U-weighted Matching assigns adaptive weights to geometric and semantic consistency according to the uncertainty of each modality. Specifically, we construct the similarity matrix as

$$\mathbf{S}=\left[\begin{array}{cccc}{S}_{11}& S_{12}& \dots & S_{1N}\\ S_{21}& S_{22}& \dots & S_{2N}\\ ⋮& ⋮& \ddots & ⋮\\ S_{M1}& S_{M2}& \dots & S_{MN}\end{array}\right],$$

(14)

where each entry is computed by a weighted combination of geometric and semantic consistency:

$$S_{ij}=w_g{\mathrm{IoU}}_{ij}+w_s{L}_{\mathrm{sem},ij}.$$

(15)

Here, $w_g$ and $w_s$ denote the adaptive weights for the geometric term ${\mathrm{IoU}}_{ij}$ and the semantic term $L_{\mathrm{sem},ij}$, respectively. They are defined as

$$\left\{\begin{array}{c}{\displaystyle w_g=\frac{(1-u_j^L)exp\left(-\gamma {\left(\frac{d}{d_{max}}\right)}^2\right)}{1-u_i^C+(1-u_j^L)exp\left(-\gamma {\left(\frac{d}{d_{max}}\right)}^2\right)},}\hfill \\ w_s=1-w_g.\hfill \end{array}\right.$$

(16)

where $u_i^C$ and $u_j^L$ are the uncertainties of the i-th image detection box and the j-th point-cloud detection box, respectively. $d_{max}$ denotes the maximum sensing distance of the LiDAR, d is the Euclidean distance from the center of the point-cloud detection box to the LiDAR origin, and $\gamma$ is a decay factor that controls the attenuation rate.

The weight design in Equation (16) reflects the fact that geometric and semantic consistency have different levels of reliability under different conditions. For nearby LiDAR detections with low uncertainty, the projected boxes usually provide more reliable geometric relationships, and thus geometric consistency is assigned a larger weight. In contrast, for distant objects or detections with high uncertainty, the projected boxes are more likely to be affected by sparse point clouds, localization errors, and calibration errors. In such cases, relying only on geometric overlap may lead to incorrect matching, while the semantic consistency represented by class belief masses provides an additional class-consistency constraint. Therefore, the adaptive weights in Equation (16) allow the matching cost to dynamically adjust the contributions of geometric and semantic consistency according to detection reliability.

We then convert the similarity matrix into a cost matrix for Hungarian assignment. For the candidate pairs that pass the normalized center-distance gating, the matching cost is defined as $1-S_{ij}$. For the remaining invalid pairs, a large penalty is assigned:

$$C_{ij}=\left\{\begin{array}{cc}1-S_{ij},\hfill & (i,j)\in \mathcal{G},\hfill \\ T,\hfill & (i,j)\notin \mathcal{G},\hfill \end{array}\right.$$

(17)

where $\mathcal{G}$ is the candidate pair set defined by normalized center-distance gating, T is a sufficiently large constant representing the maximum assignment cost, and $S_{ij}$ is given by (15). Thus, the cost matrix is

$$\mathbf{C}=\left[\begin{array}{cccc}{C}_{11}& C_{12}& \dots & C_{1N}\\ C_{21}& C_{22}& \dots & C_{2N}\\ ⋮& ⋮& \ddots & ⋮\\ C_{M1}& C_{M2}& \dots & C_{MN}\end{array}\right].$$

(18)

Finally, the Hungarian algorithm is applied to $\mathbf{C}$ to obtain the one-to-one matching results. Assignments with the penalty cost T are regarded as invalid and are treated as unmatched detections.

3.3. Frustum Re-Evaluation for Unmatched Detections

After applying the uncertainty-weighted object matching described in Section 3.2, the detections are divided into matched pairs and unmatched detections. In practical applications, due to the distinct sensing characteristics of image and point-cloud detectors, the two modalities may produce inconsistent predictions. Therefore, enforcing one-to-one matching for all detection boxes is not always feasible. A subset of detections may remain unmatched, including 2D boxes without corresponding 3D counterparts and 3D boxes without corresponding 2D observations.

To handle these unmatched detections, we design a filtering criterion that jointly considers the maximum expected class probability and the corresponding uncertainty:

$$\left\{\begin{array}{c}{max}_i\mathbb{E}\left[p_i\right]\ge {\tau }_p,\hfill \\ u\le {\tau }_u,\hfill \end{array}\right.$$

(19)

where ${\tau }_p$ and ${\tau }_u$ are the thresholds for the expected probability and the uncertainty, respectively. For unmatched LiDAR detections, this criterion is used to decide whether the 3D box should be retained in the final output. For unmatched camera detections, the retained 2D boxes are not directly output as 3D detections. Instead, they are used as anchors for the subsequent frustum re-evaluation module.

For unmatched camera detections, the main difficulty is that they provide reliable 2D image boxes but do not contain explicit 3D box parameters. Directly discarding these detections may increase false negatives. This problem is more serious for distant pedestrians and cyclists. In these cases, LiDAR points are usually sparse, and the corresponding 3D candidates may receive low confidence scores or be suppressed by NMS. To address this problem, we design a frustum re-evaluation module. This module uses the retained 2D detection box as a semantic and geometric anchor to recover potential 3D candidates from the LiDAR candidate set.

Specifically, for each retained unmatched 2D box, we use the camera intrinsic parameters and the camera–LiDAR calibration matrix to back-project its image-plane region into 3D space and construct the corresponding view frustum. As shown in Figure 2, for a given 2D detection box, the camera optical center, i.e., the origin of the camera coordinate system, is connected to the four corners of the box on the image plane. In this way, four boundary rays are formed in 3D space. LiDAR points whose projections fall inside the 2D detection box are located within the spatial region enclosed by these four boundary rays. Since the unique depth of the object cannot be determined from the 2D box alone, the depth range is determined by the valid spatial range of the LiDAR point cloud. Therefore, the four projection rays and the valid depth range jointly form a 3D frustum region. This region represents the possible spatial range of the object corresponding to the 2D box and provides a geometric constraint for searching or filtering 3D candidates.

After the frustum is constructed, it is used to constrain the pre-NMS LiDAR candidate boxes, as illustrated in Figure 3. Instead of searching all LiDAR detections after NMS, we use the pre-NMS LiDAR candidate set as the recovery source. The reason is that NMS may suppress true objects when their confidence scores are low or when they overlap with higher-scoring but less accurate boxes. This situation is common for small, distant, or partially occluded objects. By revisiting the pre-NMS candidates under the frustum constraint, the proposed module can recover valid 3D boxes that have been removed before the final LiDAR outputs are generated.

To avoid introducing too many false positives from the large pre-NMS candidate set, we apply strict frustum-based geometric filtering. A LiDAR candidate is retained only when its projected 3D box is spatially consistent with the frustum generated by the 2D anchor. In practice, this step removes candidates that are geometrically incompatible with the image detection and keeps only a small number of possible 3D hypotheses. For the remaining candidates, we recompute the cross-modal similarity using the U-weighted Matching strategy described in Section 3.2. If the matching cost satisfies the acceptance threshold, the candidate with the lowest matching cost is selected as the recovered 3D counterpart of the 2D detection. This avoids forcibly matching a 2D detection with an unreliable 3D candidate when no valid candidate exists in the frustum.

Following the complete workflow in Figure 4, after re-association, the recovered 2D–3D pair is not directly added to the final detection set. Instead, it is further evaluated by the uncertainty-discounted fusion strategy described in Section 3.4. The fused belief mass and uncertainty are then used to decide whether the recovered object should be retained. Therefore, the frustum re-evaluation module does not improve recall by simply adding more proposals. It first performs constrained candidate recovery and then applies uncertainty-aware evidence verification. This design helps recover true objects suppressed by NMS while limiting the introduction of false positives.

3.4. Uncertainty-Discounted Decision-Level Fusion

After uncertainty-weighted matching and frustum-based re-evaluation, the associated camera–LiDAR detection pairs are fused to obtain the final detection decisions. Given an associated pair $(B_i^C,B_j^L)$, the two detectors can be regarded as two independent opinions about the same object. We aim to combine these opinions via the Dempster–Shafer evidence theory [30] to obtain a more reliable and interpretable fused result. However, directly applying the classical Dempster–Shafer combination rule may yield abnormal fusion results under conflicts, such as an extreme bias toward a single source. Therefore, we adopt a discounting-based approach [31] to mitigate the impact of conflicts while preserving the desirable properties of the Dempster–Shafer rule.

To handle potential conflicts during fusion, we adopt an adaptive discounting scheme. Specifically, we use the Jensen–Shannon (JS) divergence to measure the conflict degree between two detections:

$$D_{\mathrm{JS}}\left({\mathbf{p}}^C\parallel {\mathbf{p}}^L\right)={\displaystyle \frac{1}{2}}{D}_{\mathrm{KL}}\left({\mathbf{p}}^C\parallel \mathbf{m}\right)+{\displaystyle \frac{1}{2}}{D}_{\mathrm{KL}}\left({\mathbf{p}}^L\parallel \mathbf{m}\right),$$

(20)

where ${\mathbf{p}}^s=\left[{\displaystyle \frac{{\alpha }_1^s}{S^s}},{\displaystyle \frac{{\alpha }_2^s}{S^s}},\dots ,{\displaystyle \frac{{\alpha }_K^s}{S^s}}\right]$, $s\in \{C,L\}$ represents the expected probability vector. $\mathbf{m}={\displaystyle \frac{1}{2}}\left({\mathbf{p}}^C+{\mathbf{p}}^L\right)$, and the KL divergence is defined as

$$D_{\mathrm{KL}}\left({\mathbf{p}}^s\parallel \mathbf{m}\right)={\displaystyle \sum _{k=1}^K}{p}_k^{s}log{\displaystyle \frac{p_k^s}{m_k}}.$$

(21)

We normalize the JS divergence to obtain the conflict degree:

$$\mathrm{conf}={\displaystyle \frac{D_{\mathrm{JS}}\left({\mathbf{p}}^C\parallel {\mathbf{p}}^L\right)}{log2}}.$$

(22)

Then, we construct the correlation matrix:

$$\mathbf{R}=\left[\begin{array}{ccc}1& 1-\mathrm{conf}& 1-u^C\\ 1-\mathrm{conf}& 1& 1-u^L\\ 1-u^C& 1-u^L& 1\end{array}\right],$$

(23)

where $u^C$ and $u^L$ are the uncertainties of the camera and LiDAR detections, respectively. The correlation matrix jointly considers the conflict between the two class opinions and the reliability of each modality.

We compute the eigenvector corresponding to the maximum eigenvalue:

$$\mathbf{R}\mathsf{\beta }={\lambda }_{max}\mathsf{\beta },$$

(24)

where $\mathsf{\beta }={({\beta }_C,{\beta }_L,{\beta }_0)}^{\top }$. The eigenvector corresponding to the maximum eigenvalue is used to derive the relative evidence weights from the correlation matrix. If one modality has low uncertainty and agrees well with the other modality, its corresponding eigenvector component becomes larger, and more of its evidence is retained. Conversely, if one modality has high uncertainty or strong conflict, its component becomes smaller, and its evidence is discounted before Dempster–Shafer fusion. Based on these relative weights, the discounting coefficients are constructed as

$${\overline{\beta }}_s={\displaystyle \frac{{\beta }_s}{max\left(\mathsf{\beta }\right)}},s\in \{C,L\}.$$

(25)

Thus, the discounted evidence is given by

$${\widehat{e}}_i^s={\overline{\beta }}_s{e}_i^s.$$

(26)

After evidence discounting, the corresponding discounted belief masses and uncertainty are recomputed. Then, the discounted opinions are combined under the Dempster–Shafer rule to obtain the fused belief masses and uncertainty:

$$\left\{\begin{array}{cc}\hfill {\widehat{b}}_i^{\oplus }& ={\displaystyle \frac{{\widehat{b}}_i^C{\widehat{b}}_i^L+{\widehat{b}}_i^C{\widehat{u}}^L+{\widehat{b}}_i^L{\widehat{u}}^C}{1-\kappa }},\hfill \\ \hfill {\widehat{u}}^{\oplus }& ={\displaystyle \frac{{\widehat{u}}^L{\widehat{u}}^C}{1-\kappa }}.\hfill \end{array}\right.$$

(27)

where $\kappa ={\sum }_{i=1}^K{\sum }_{\begin{array}{c}j=1\\ j\ne i\end{array}}^K{\widehat{b}}_i^C{\widehat{b}}_j^L$ denotes the conflict mass between mutually exclusive class hypotheses. We take the maximum expected class probability after fusion as the final confidence score. In this way, the proposed uncertainty-discounted fusion strategy mitigates cross-modal object conflicts during fusion and improves the reliability of final detection decisions. Figure 5 illustrates a typical category-conflict case, where inconsistent camera and LiDAR class evidence is corrected after applying the discounting strategy before final Dempster–Shafer fusion.

4. Experiments

We conducted a comprehensive experimental evaluation of the proposed method on the KITTI [32] dataset. Following the standard KITTI evaluation protocol, we reported results on the three main categories, i.e., Car, Pedestrian, and Cyclist.

4.1. Dataset and Evaluation Metrics

KITTI dataset We performed experiments on the KITTI dataset, which is widely used in autonomous driving. KITTI consists of urban driving scenes and provides high-quality annotations for multiple perception tasks, including 3D object detection, optical flow estimation, and depth estimation. The data collection platform is equipped with a pair of high-resolution color cameras, a Velodyne 64-beam LiDAR, and a GPS/IMU system. The annotations are provided primarily from LiDAR point clouds, yielding accurate 3D bounding-box labels. For 3D object detection, KITTI contains 7481 labeled images. Following the standard split, 3712 images are used for training and 3769 images are used for validation. In addition, a test set whose ground-truth labels are withheld is reserved for online benchmark evaluation. The annotations cover multiple categories, including Car, Pedestrian, and Cyclist. For each category, the dataset further defines three difficulty levels—Easy, Moderate, and Hard—based on object size, occlusion, and truncation, enabling thorough evaluation under varying challenges.

Evaluation metrics We followed the official KITTI evaluation protocol for 3D object detection. The primary metric is Average Precision (AP). Specifically, for each object category, AP is computed under the three predefined difficulty levels (Easy, Moderate, and Hard). The evaluation is based on the 3D bounding-box Intersection over Union (IoU), with IoU thresholds set to 0.7 for Car and 0.5 for Pedestrian and Cyclist. We report AP computed with 40 recall positions (AP_R40) as the main result, and AP under the Moderate level is used as the key criterion for method ranking.

4.2. Detector Settings

To assess the generality and effectiveness of the proposed framework, we instantiated it on a set of widely adopted 3D detection baselines that represent different design paradigms. We adopted YOLOv8 [33] and RT-DETR [34] as 2D detectors due to their favorable accuracy–speed trade-offs and their broad use as high-performance detection models. We used SECOND [3], PointPillars [4], and PartA2 [5] as the LiDAR 3D detection baselines in the proposed fusion framework. All LiDAR baseline detectors, including SECOND, PointPillars, and PartA2, use models trained with the OpenPCDet framework and were evaluated under the same KITTI data split and evaluation protocol. When integrated with the proposed UDF-3D framework, their detector configurations were kept unchanged to ensure fair internal comparisons and ablation studies. These baselines are standard and representative open-source architectures in the community. Although they exhibit different performance levels on the KITTI leaderboard, they provide a solid testbed for evaluating the robustness and general applicability of our fusion framework. Experimental results show that the proposed method consistently improves the performance of these baseline 3D detectors.

4.3. Results and Analysis

This subsection provides a comprehensive evaluation of the proposed LiDAR–image decision-level fusion method on the KITTI validation set. We followed the official evaluation protocol and reported both 3D AP_R40 and BEV AP_R40 as the core metrics. Our analysis focuses on: (1) comparisons with LiDAR-only baselines to verify the effectiveness of the fusion strategy; (2) comparisons with existing multi-modal fusion methods, especially the classical decision-level fusion approach, CLOCs, to position the contribution and competitiveness of our method.

As shown in

Table 1. Comparison of 3D object detection between baseline and fusion methods on the KITTI validation set (3D AP_R40).

Method	Modality	mAP	Car			Pedestrian			Cyclist
		Mod.	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
SECOND (Baseline) [3]	L	66.50	90.55	81.61	78.60	55.94	51.15	46.17	80.96	66.74	62.78
SECOND + YOLOv8	L & I	71.87	92.15	83.32	80.41	66.42	60.19	55.65	87.14	72.11	65.49
SECOND + RT-DETR	L & I	72.29	91.59	83.06	80.17	65.20	61.16	55.18	82.66	72.64	68.33
PointPillars (Baseline) [4]	L	63.93	87.76	77.40	75.19	57.31	51.45	46.87	81.57	62.94	58.96
PointPillars + YOLOv8	L & I	71.11	88.83	80.01	77.11	67.36	61.20	56.73	86.15	72.11	67.63
PointPillars + RT-DETR	L & I	71.53	88.00	79.64	76.70	66.20	62.28	58.06	86.21	72.68	68.24
PartA2 (Baseline) [5]	L	70.89	92.45	82.88	80.64	66.84	59.69	54.58	90.35	70.10	66.97
PartA2 + YOLOv8	L & I	75.75	93.17	83.81	82.63	77.07	70.31	63.94	90.90	73.14	70.14
PartA2 + RT-DETR	L & I	76.69	92.06	83.07	82.62	76.34	70.45	65.69	91.92	76.55	72.32

Note: Green values indicate the highest relative to the baseline, and blue values indicate the overall best result.

and

Table 2. Comparison of BEV object detection between baseline and fusion methods on the KITTI validation set (BEV AP_R40).

Method	Modality	mAP	Car			Pedestrian			Cyclist
		Mod.	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
SECOND (Baseline)	L	71.42	92.42	88.55	87.65	60.74	56.57	52.13	86.04	69.15	66.90
SECOND + YOLOv8	L & I	78.19	93.41	90.04	89.68	72.01	65.91	61.49	89.78	78.61	74.04
SECOND + RT-DETR	L & I	78.93	93.22	90.16	89.88	71.34	67.44	63.28	88.19	79.18	74.66
PointPillars (Baseline)	L	70.14	92.04	88.06	86.67	61.58	56.01	52.01	85.27	66.34	62.35
PointPillars + YOLOv8	L & I	78.23	93.34	90.05	87.44	73.69	67.44	63.02	91.46	77.21	72.52
PointPillars + RT-DETR	L & I	78.93	93.01	89.99	87.40	72.91	68.94	64.61	92.03	77.87	75.57
PartA2 (Baseline)	L	76.05	93.55	89.38	87.13	70.50	64.10	59.17	91.93	74.66	70.61
PartA2 + YOLOv8	L & I	81.39	94.76	91.12	89.14	81.39	74.60	69.60	92.45	78.45	74.00
PartA2 + RT-DETR	L & I	82.87	94.88	91.55	89.74	80.64	76.05	71.34	94.62	81.02	76.74

Note: Green values indicate the highest relative to the baseline, and blue values indicate the overall best result.

, the proposed fusion framework achieves clear improvements over the original baselines across different LiDAR detectors, including SECOND, PointPillars, and PartA2, as well as different 2D detector settings. This indicates that the performance gain of the proposed method does not depend on a specific detector combination. In these two tables, green values indicate the best result within each detector group, and blue values indicate the best result in each column across the whole table. The category-wise results show that the improvements are more evident for Pedestrian and Cyclist. These categories usually have sparse point-cloud observations, smaller object scales, and more frequent occlusions, and therefore, camera semantic cues can provide stronger complementary information. For the Car category, although the LiDAR baselines already achieve relatively high detection accuracy, the proposed method still brings stable improvements. The qualitative results in Figure 6 further show that the proposed fusion strategy can reduce typical false positives and missed detections produced by the LiDAR-only baseline.

To position the proposed method among existing camera–LiDAR fusion frameworks, we compared UDF-3D with representative fusion methods, including data-level fusion methods, feature-level fusion methods, and the classical decision-level fusion method CLOCs. The reported CLOCs results correspond to its commonly used SECOND + Cascade-RCNN detector setting, where the SECOND branch also adopts the official SECOND pretrained weights provided by OpenPCDet. Thus, CLOCs is included as a reference under the same decision-level fusion paradigm. The results are shown in

Table 3. Comparison of 3D object detection between existing fusion methods and the proposed fusion methods on the KITTI validation set (3D AP_R40). The abbreviations correspond to SECOND (SD), YOLOv8 (YL), PartA2 (PA), and RT-DETR (RD).

Method	Modality	mAP	Car			Pedestrian			Cyclist
		Mod.	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
F-PointNet [35]	L & I	62.91	83.76	70.92	63.65	70.00	61.32	53.59	77.15	56.49	53.37
F-PointPillars [36]	L & I	71.32	88.90	79.28	78.07	66.11	61.89	56.91	87.54	72.78	66.07
PointFusion [12]	L & I	40.15	77.92	63.00	53.27	33.36	28.04	23.38	49.34	29.42	26.98
PointPainting [13]	L & I	70.34	88.38	77.74	76.76	69.38	61.67	54.58	85.21	71.62	66.98
EPNet++ [18]	L & I	70.80	92.51	83.17	82.27	73.77	65.42	59.13	86.23	63.82	60.02
3D-CVF [37]	L & I	-	89.67	79.88	78.47	-	-	-	-	-	-
CAT-Det [38]	L & I	73.54	90.12	81.46	79.15	74.08	66.35	58.92	87.64	72.82	68.20
CLOCs [24]	L & I	69.45	92.89	83.07	78.65	64.68	57.37	51.19	87.57	67.92	63.67
UDF-3D (SD+YL)	L & I	71.87	92.15	83.32	80.41	66.42	60.19	55.65	87.14	72.11	65.49
UDF-3D (PA+YL)	L & I	75.75	93.17	83.81	82.63	77.07	70.31	63.94	90.90	73.14	70.14
UDF-3D (PA+RD)	L & I	76.69	92.06	83.07	82.62	76.34	70.45	65.69	91.92	76.55	72.32

Note: Blue values indicate the overall best result.

and

Table 4. Comparison of BEV object detection between existing fusion methods and the proposed fusion methods on the KITTI validation set (BEV AP_R40). The abbreviations correspond to SECOND (SD), YOLOv8 (YL), PartA2 (PA), and RT-DETR (RD).

Method	Modality	mAP	Car			Pedestrian			Cyclist
		Mod.	Easy	Mod.	Hard	Easy	Mod.	Hard	Easy	Mod.	Hard
F-PointNet [35]	L & I	70.15	88.16	84.02	76.44	72.38	66.39	59.57	81.82	60.03	56.32
F-PointPillars [36]	L & I	78.04	90.20	89.43	88.77	72.17	67.89	63.46	88.58	76.79	74.80
PointFusion [12]	L & I	47.08	87.45	76.13	65.32	37.91	32.35	27.35	54.02	32.77	30.19
PointPainting [13]	L & I	75.80	90.19	87.64	86.71	72.65	66.06	61.24	86.33	73.69	70.17
EPNet++ [18]	L & I	75.36	95.98	89.08	88.86	78.23	72.09	66.17	86.25	64.91	61.30
3D-CVF [37]	L & I	-	93.52	89.56	82.45	-	-	-	-	-	-
CAT-Det [38]	L & I	70.45	92.59	90.07	85.82	57.13	48.78	45.56	85.35	72.51	65.55
CLOCs [24]	L & I	75.65	96.18	92.53	89.43	69.26	63.13	56.79	91.35	71.30	67.65
UDF-3D (SD+YL)	L & I	78.19	93.41	90.04	89.68	72.01	65.91	61.49	89.78	78.61	74.04
UDF-3D (PA+YL)	L & I	81.39	94.76	91.12	89.14	81.39	74.60	69.60	92.45	78.45	74.00
UDF-3D (PA+RD)	L & I	82.87	94.88	91.55	89.74	80.64	76.05	71.34	94.62	81.02	76.74

Note: Blue values indicate the overall best result.

. Overall, UDF-3D (PA+RD) achieves the highest Moderate mAP in both 3D and BEV detection, indicating that the proposed method has competitive overall performance.

It should be noted that some existing methods still show strong performance for the Car category. For example, in BEV detection, CLOCs obtains slightly higher AP than UDF-3D for Car under the Easy and Moderate settings. This suggests that existing decision-level fusion methods can still achieve good detection performance for large objects with relatively sufficient LiDAR observations. However, UDF-3D shows clearer advantages for the Pedestrian and Cyclist categories. These two categories usually have sparse point-cloud observations, smaller object scales, and stronger cross-modal complementarity, and therefore rely more on the effective cooperation between camera semantic information and LiDAR geometric information. These results indicate that the proposed method is more adaptable to challenging object categories.

From the perspective of overall Moderate mAP, UDF-3D outperforms CLOCs in both 3D and BEV detection, indicating that the proposed method does not merely achieve local improvements in a single category, but provides more stable overall detection performance. Different from CLOCs, UDF-3D does not require an additional trainable fusion network for candidate box pairs. Instead, it performs decision-level fusion through uncertainty modeling, candidate association, and uncertainty-discounted fusion. Therefore, the proposed framework can be conveniently incorporated into different combinations of 2D and 3D detectors, reducing the dependence on a specific fusion network structure and retraining process. Compared with other existing fusion methods, such as EPNet++ and CAT-Det, UDF-3D also achieves better overall Moderate mAP, further demonstrating that the proposed method improves overall detection performance while preserving the flexibility of decision-level fusion.

Although UDF-3D improves the overall detection performance, some challenging cases still deserve further discussion. As shown in Figure 7, distant objects with severe occlusion or truncation usually correspond to relatively sparse LiDAR observations. In such cases, the 2D detector may also fail to provide sufficient image cues, while the LiDAR branch provides weak geometric information, which can weaken the fusion effect of UDF-3D to some extent. Since the frustum re-evaluation module still relies on valid 2D observations and pre-NMS 3D candidates, its ability to further recover such objects may be limited when both cues are insufficient.

In summary, the proposed Dempster–Shafer evidence-theory-based fusion framework explicitly models and fuses the uncertainties of LiDAR and image detections, thereby significantly improving the overall performance of multi-modal 3D object detection. The experiments show that our method not only significantly surpasses the LiDAR-only baselines, but also achieves superior performance over the representative decision-level fusion method CLOCs under the same fusion paradigm, providing an effective solution for fusion perception in autonomous driving.

4.4. Ablation Study

To evaluate the contribution of each module to the proposed fusion framework, we conduct an ablation study on the KITTI validation set. PointPillars+YOLOv8 is used as the representative detector combination.

Table 5. Ablation study of the proposed fusion modules using PointPillars+YOLOv8 on the KITTI validation set.

U-Weighted Matching	Discounted Fusion	Frustum Re-Evaluation	3D APR40			BEV APR40
			Easy	Mod.	Hard	Easy	Mod.	Hard
✗	✗	✗	75.55	63.93	60.34	79.63	70.14	67.01
✓	✗	✗	76.99	66.18	62.50	82.01	73.65	70.18
✗	✓	✗	78.12	68.54	65.18	83.67	75.95	73.03
✓	✓	✗	80.42	70.72	66.74	85.86	77.86	74.02
✓	✗	✓	78.55	69.20	65.36	84.28	76.54	73.68
✓	✓	✓	80.78	71.11	67.16	86.16	78.23	74.33

reports the class-averaged 3D AP_R40 and BEV AP_R40 over Car, Pedestrian, and Cyclist under the Easy, Moderate, and Hard difficulty levels. The first row denotes the PointPillars single-modal baseline without any fusion module. The remaining rows show the results obtained by adding different combinations of the proposed modules based on the LiDAR and RGB branches.

The ablation results indicate that U-weighted Matching effectively improves the detection performance. This module establishes cross-modal associations between camera boxes and LiDAR boxes. It uses the richer semantic information from the RGB branch to constrain LiDAR candidates. In this way, false positives without image semantic support can be reduced. Compared with the LiDAR-only baseline, the overall AP is improved after this module is introduced. This indicates that the RGB branch can provide useful semantic information for LiDAR detection. The frustum re-evaluation module is mainly used to recover objects missed by the LiDAR branch. Pedestrian and Cyclist objects, as well as distant objects, usually have sparse LiDAR points. Therefore, LiDAR detectors may assign low confidence scores to them, or they may be suppressed during NMS. This module constructs frustums from unmatched camera detections. It then searches for potential 3D boxes from the pre-NMS LiDAR candidate set. Therefore, it can help recover some missed objects. The uncertainty-discounted fusion module further improves the reliability of the fusion decision. Simple score fusion can be affected by high-confidence but unreliable predictions. In contrast, the proposed uncertainty-discounted fusion module is based on Dempster–Shafer evidence theory. It adjusts the evidence according to uncertainty and cross-modal conflict. When the two modalities are consistent and have low uncertainty, the belief of the correct class is strengthened. When one modality has high uncertainty or semantic conflict, its influence is suppressed. Therefore, this module can reduce errors caused by conflicting or weak evidence. It also makes the fusion results more stable and contributes more to the improvement of accuracy.

To further illustrate the positive effect of uncertainty on detection accuracy, two variants are evaluated within the proposed framework: one does not model object uncertainty, while the other adopts the proposed uncertainty-based fusion method. The detection performance of these two settings is compared using AP curves under different 3D IoU evaluation thresholds, as shown in Figure 8. It can be observed that incorporating uncertainty information leads to higher AP values across most IoU thresholds, indicating that the proposed uncertainty-based fusion method improves the detection accuracy of the final results. The x-axis represents the 3D IoU threshold used for evaluation, while the y-axis represents the corresponding 3D AP_R40. We compared the setting that accounts for uncertainty (U) with the setting that does not (N-U). These two variants are not entirely identical in their matching and fusion formulas, as uncertainty information primarily serves the Dempster–Shafer fusion rule; however, they follow the same candidate association process: candidate filtering, constructing similarity using geometric and semantic consistency, and one-to-one assignment using the Hungarian algorithm. The Average Precision (AP) curves indicate that, for most Intersection over Union (IoU) thresholds and difficulty levels, U outperforms N-U in the detection of Car, Pedestrian, and Cyclist. As the evaluation IoU threshold increases, U’s advantage over N-U becomes increasingly pronounced. This indicates that uncertainty-aware fusion not only enhances detection performance under lenient evaluation thresholds but also maintains superior performance under stricter localization requirements. For Car, as the LiDAR baseline already achieves relatively high detection accuracy, the gap between U and N-U is limited at low IoU thresholds. However, at higher IoU thresholds, inaccurate matches and localization errors are penalized more severely. In this scenario, U more effectively suppresses unreliable fusion results, thereby demonstrating a more significant performance advantage. For Pedestrian and Cyclist, U’s contribution is more consistent. These two categories typically feature smaller object sizes and sparser point cloud observations, and are more susceptible to occlusion, distance and pose variations. Consequently, single-modal detectors are more likely to produce uncertain or erroneous predictions. Cameras provide textural and semantic information, while LiDAR provides spatial and geometric information. Uncertainty-aware fusion amplifies reliable evidence when both modalities are trustworthy, while suppressing unreliable evidence when one modality is uncertain. Consequently, for Pedestrian and Cyclist, U demonstrates more significant improvements compared to N-U. This further verifies the effectiveness of explicitly modeling uncertainty in decision-level fusion.

Overall, the combination of the three modules significantly improves detection performance. U-weighted Matching mainly improves cross-modal association and reduces unreliable matches. The frustum re-evaluation module mainly recovers missed detections from the pre-NMS LiDAR candidate set. The uncertainty-discounted fusion module mainly suppresses unreliable and conflicting detection results during the final decision fusion stage. The AP curves further show that the uncertainty-driven fusion method can maintain better performance under different IoU thresholds. When all three modules are enabled, the proposed method achieves the best performance. This verifies the effectiveness of the proposed uncertainty-driven decision-level fusion framework.

4.5. Sensitivity Analysis of the Decay Factor

To evaluate the sensitivity of the proposed method to the decay factor $\gamma$ in Equation (16), we conducted experiments using PointPillars+YOLOv8 on the KITTI validation set. As shown in

Table 6. Sensitivity analysis of the decay factor $\gamma$ using PointPillars+YOLOv8 on the KITTI validation set. The reported values are class-averaged 3D AP_R40 over Car, Pedestrian, and Cyclist.

$\mathsf{\gamma }$	3D Easy	3D Mod.	3D Hard
1.0	80.42	70.73	66.84
1.5	80.61	70.96	67.02
2.5	80.78	71.11	67.16
3.5	80.66	71.03	67.08
4.0	80.51	70.88	66.95

, the detection performance first improves and then slightly decreases as $\gamma$ increases. The best results are obtained when $\gamma =2.5$. A small $\gamma$ provides insufficient distance attenuation, whereas an excessively large $\gamma$ may over-reduce the contribution of geometric consistency for distant objects. The relatively small performance variation across the tested settings indicates that the proposed method is not highly sensitive to $\gamma$. Therefore, $\gamma =2.5$ is adopted as the default setting in all experiments.

4.6. Computational Complexity

To evaluate the efficiency and deployment potential of the proposed framework, we further report the computational complexity of UDF-3D after the ablation study. The measurements of our method are conducted on the KITTI validation set using 200 samples under Ubuntu 20.04 with an NVIDIA RTX 3060 GPU, PyTorch 2.1, and CUDA 11.8. As shown in

Table 7. Computational complexity comparison of UDF-3D and representative fusion methods on the KITTI validation set.

Group	Method/Module	Speed (ms)	GFLOPS	GPU Memory (MB)	FPS
UDF-3D modules	LiDAR branch (PointPillars)	24.44	62.82	248.07	–
	RGB branch (YOLOv8/RT-DETR)	19.30/64.24	79.07/251.75	150.90/203.40	–
	Uncertainty Quantification	0.59	0.00	0.00	–
	Frustum Re-evaluation	10.57	0.00	29.42	–
	Uncertainty-Discounted Fusion	0.16	0.00	0.00	–
End-to-end	UDF-3D	35.76/75.56	141.89/314.57	428.39/480.89	27.96/13.23
	EPNet++	169.79	357.85	5059.64	5.89
	PointPainting	271.77	1782.02	742.06	3.68

, the additional costs introduced by uncertainty quantification and uncertainty-discounted fusion are negligible, while the frustum re-evaluation module introduces only a limited overhead for recovering suppressed candidates. Since UDF-3D performs fusion at the decision level, the LiDAR and RGB detection branches can run in parallel. Therefore, the final latency is computed as the slower detection branch plus the subsequent uncertainty quantification, frustum re-evaluation, and uncertainty-discounted fusion time. For UDF-3D, the values are reported in the format of YOLOv8/RT-DETR; specifically, 35.76 ms corresponds to the YOLOv8 setting, whereas 75.56 ms corresponds to the RT-DETR setting.

The results show that UDF-3D maintains a lightweight and deployment-friendly fusion pipeline. This is mainly because it avoids dense feature-level interaction between image and point-cloud backbones and only performs matching and fusion on detection candidates. Although the RT-DETR-based setting increases the computation of the RGB branch, the overall latency remains moderate compared with the listed multi-modal fusion methods. The uncertainty quantification and uncertainty-discounted fusion modules together take less than 1 ms, indicating that the performance gain is mainly obtained from more reliable matching, conflict discounting, and frustum re-evaluation rather than from heavy computational modules.

The GPU memory results further show that the additional memory overhead of UDF-3D is limited under the current setting. Since the proposed framework does not maintain additional dense cross-modal feature maps, its extra memory mainly comes from candidate-level data structures, including the matching matrix, frustum candidates, and belief/uncertainty vectors. Therefore, the memory cost is more dependent on the number of retained candidates than on the spatial size of image, voxel, or BEV feature maps. For parallel execution, although the LiDAR and RGB branches can be executed independently, the decision-level fusion stage requires the outputs from both branches. As a result, the overall parallel efficiency is mainly bounded by the slower detection branch before the subsequent fusion process can be performed.

5. Conclusions

In this paper, we propose UDF-3D, an uncertainty-driven decision-level fusion framework for camera–LiDAR 3D object detection. The proposed framework represents detector outputs as subjective-logic opinions, where class-wise belief masses and object-level uncertainty are explicitly quantified from detector outputs. Based on the quantified uncertainty, an uncertainty-weighted matching strategy is developed to improve cross-modal object association by jointly considering geometric overlap and semantic consistency. For unmatched detections, a frustum re-evaluation mechanism is designed to recover valid 3D candidates that may be suppressed by NMS. For matched detections, an uncertainty-discounted Dempster–Shafer fusion rule is introduced to reduce the influence of conflicting evidence from different modalities on the fusion results. Ablation results further show that the three modules contribute to the framework from different aspects: uncertainty-weighted matching improves cross-modal association, frustum re-evaluation reduces missed detections, and uncertainty-discounted fusion reduces the influence of unreliable and conflicting detection results on the final decision. Experimental results on the KITTI validation set show that UDF-3D consistently improves different LiDAR-only baselines and achieves competitive performance compared with existing fusion methods. The improvements are especially evident for Pedestrian and Cyclist, indicating that the proposed framework can effectively exploit camera–LiDAR complementarity for challenging objects with sparse point-cloud observations. In addition, the computational complexity analysis shows that the proposed fusion modules introduce only limited additional overhead, which preserves the practical efficiency and real-time potential of the decision-level fusion framework. In future work, we will further validate the proposed framework on larger-scale and more complex datasets, such as nuScenes and Waymo, to more comprehensively evaluate its generalizability under different sensor configurations, data formats, and evaluation protocols.