SemABC: Semantic-Guided Adaptive Bias Calibration for Generative Zero-Shot Point Cloud Segmentation

Abstract

Due to the limited quantity and high cost of high-quality three-dimensional annotations, generalized zero-shot point cloud segmentation aims to transfer the knowledge of seen to unseen classes by leveraging semantic correlations to achieve generalization purposes. Existing generative point cloud semantic segmentation approaches rely on generators trained on seen classes to synthesize visual features for unseen classes in order to help the segmentation model gain the ability of generalization, but this often leads to a bias toward seen classes. To address this issue, we propose a semantic-guided adaptive bias calibration approach with a dual-branch network architecture. This network consists of a novel visual–semantic fusion branch alongside the primary segmentation branch to suppress the bias toward seen classes. Specifically, the visual–semantic branch exploits the visual–semantic relevance of the synthetic features of unseen classes to provide auxiliary predictions. Furthermore, we introduce an adaptive bias calibration module that dynamically integrates the predictions from both the main and auxiliary branches to achieve unbiased segmentation results. Extensive experiments conducted on standard benchmarks demonstrate that our approach significantly outperforms state-of-the-art methods on both seen and unseen classes, thereby validating the effectiveness of our approach.

1. Introduction

Among three-dimensional (3D) scene understanding tasks, point cloud segmentation plays a fundamental role in fields such as autonomous driving [1,2], robot navigation [3,4], and augmented reality [5]. However, due to the sparsity and unordered properties of 3D point clouds, as well as the high cost of acquiring high-quality annotations [6], the supervised learning methods for point clouds [7,8] face challenges in generalization to a wider range of scenarios. Since generalized zero-shot learning (GZSL) [9,10,11] enables knowledge transfer from seen to unseen classes without the need for labeled data during training, recently, researchers have been particularly interested in exploring the potential of these techniques to handle the prediction of both seen and unseen classes in point cloud segmentation tasks. GZSL can be divided into transductive approaches [12,13,14], which leverage data samples from unseen classes during training, and inductive approaches [15,16,17], which rely solely on the samples of seen classes during training. Given the difficulties of acquiring all kinds of unlabeled data samples in real-world scenarios, inductive GZSL is more suitable for varied practical applications.

Inductive GZSL methods for point cloud semantic segmentation, such as 3DGenZ [18] and SV-Seg [19], employ generative models [20,21] to create synthesized features based on semantic embeddings [22,23,24,25] of unseen classes. However, since the generator is trained exclusively on seen classes, the features it generates for unseen classes are inevitably biased toward the characteristics of seen classes. Consequently, the segmentor trained with these features tends to favor the seen classes. To address this issue, these approaches [18,19] typically reduce the bias toward seen classes by subtracting a predefined constant value from their predicted probability score for calibration purposes, which may affect the generalization and robustness of the model in open scenarios.

To address this problem, we propose a semantic-guided adaptive bias calibration approach with a dual-branch network architecture. This network comprises a primary segmentation branch (main branch) and a visual–semantic fusion branch (auxiliary branch). The primary segmentation branch encodes the input point cloud and predicts the segmentation probabilities, as other approaches do, while the visual–semantic fusion branch additionally explores the correlation between the visual features of the point clouds and the semantic embeddings of the categories to reduce the bias toward seen classes. Our motivation is that, since these synthesized features are generated based on category semantics of unseen classes, their relevance to semantics is inherently higher than that of the seen classes. Therefore, we propose to incorporate the spatial and channel attention mechanisms to exploit the visual–semantic correlation and align the two modalities via contrastive learning, as shown in the first row of Figure 1. In this way, the visual–semantic fusion branch is able to produce segmentation predictions in favor of unseen classes, which forms a complement to those from the primary segmentation branch.

With this dual-branch network architecture, we further propose an adaptive bias calibration module that dynamically adjusts the weights of the auxiliary predictions based on the prediction of the main branch, as shown in the bottom row of Figure 1. When the main branch exhibits high confidence with the predicted probability, we assign less weight to the auxiliary predictions to avoid introducing additional noise or making unnecessary corrections to high-confidence predictions. In contrast, when confidence is low, we increase the influence of the auxiliary predictions, leveraging different perspectives from the auxiliary branch to enhance the reliability of the overall prediction.

In real-world dynamic scenarios, our method can be applied to the identification of unknown obstacles in urban autonomous driving, such as identifying new types of vehicles not covered in the training set, such as electric scooters and delivery robots. Through the visual–semantic branch, these objects are associated with the “small mobile vehicle” semantic category to avoid misclassification as background. Additionally, it can be applied in construction site adaptation scenarios, where temporary roadblocks, cones, and other objects with variable shapes result in low confidence in the main branch. In such cases, our model gives more weight to the auxiliary branch, using the semantic association to improve segmentation performance.

Our main contributions are summarized as follows:

• We propose a dual-branch network where the auxiliary branch exploits visual–semantic correlations to complement the prediction of the main branch. It leverages the higher semantic relevance of the synthesized visual features of unseen classes to prevent bias toward the seen classes.
• We propose an adaptive bias calibration based on the confidence of segmentation predictions. This module dynamically integrates the predictions from the main and auxiliary branches, thus effectively suppressing the network’s bias towards seen classes.
• Extensive experiments show that our method significantly outperforms existing generative GZSL methods in terms of both segmentation accuracy and robustness on the benchmark datasets.

2. Related Works

2.1. Point Cloud Semantic Segmentation

As a core task of 3D visual understanding, point cloud semantic segmentation aims to endow each spatial point in the unstructured point cloud with an accurate semantic label. Many works design network architectures for point clouds, such as PointNet [26], PointNet++ [27], KPConv [28], and RandLA-Net [29], and utilize a fully supervised learning paradigm that depends on large-scale labeled datasets for training. However, the fine-grained annotation of point clouds requires enormous human resources, which significantly limits the application and advancement of these methods in practical scenarios.

2.2. Zero-Shot Point Cloud Semantic Segmentation

Although there have been significant advances in zero-shot learning for images [30,31], research on zero-shot segmentation for 3D point clouds is still underexplored. The pioneering work in this area was carried out by Cheraghian et al. [32], who adapted the two-dimensional (2D) Zero-Shot Learning (ZSL) framework to 3D point cloud data by aligning the PointNet features and semantic embeddings in the same space. Subsequently, they further expanded 3D ZSL through several studies, including introducing transductive learning strategies [33] and building geometric primitive transfer models [34]. However, these methods assume the availability of unlabeled samples from unseen categories, which greatly limits their applicability in real-world situations.

To eliminate reliance on samples from unseen categories during training, Michele et al. [18] proposed a generative framework that utilizes a generator to synthesize pseudo-features based on the semantic information of unseen classes, achieving breakthrough progress in 3D ZSL and GZSL. Building on this, subsequent work by SV-Seg [19] improved feature diversity by enhancing the multi-dimensional alignment of the visual–semantic space. Since these methods rely on data from seen classes to learn models for recognizing both seen and unseen data, they tend to exhibit bias toward seen classes.

By contrast, we propose a dual-branch network architecture that includes a visual–semantic fusion branch. This branch takes advantage of the higher semantic relevance of synthetic features for unseen classes compared to seen classes. By doing so, it allows us to obtain auxiliary predictions that are unbiased toward seen classes through visual–semantic alignment. This adaptive approach helps guide the calibration for semantic segmentation.

2.3. Calibrated Stacking

In generalized zero-shot learning, since the training data only include seen classes while the test data come from both seen and unseen classes, the test data from unseen classes are often misclassified as seen classes. To address this problem, Chao et al. [35] proposed the calibrated stacking method, which suppresses the bias toward seen classes by subtracting a predefined constant value. This has been widely used in both 2D [36] and 3D [18,19] GZSL approaches. However, this fixed-parameter calibration may weaken the model’s adaptability in open environments, and using homogenized calibration parameters for heterogeneous seen classes would reduce the model’s robustness. To address these limitations, we propose an adaptive bias calibration module that dynamically adjusts the weights of auxiliary predictions according to the confidence of the main branch’s prediction. The weighted auxiliary predictions are subsequently integrated into the main branch prediction, yielding the final prediction following adaptive bias calibration.

3. Methods

3.1. Overview

We propose a semantic-guided adaptive bias calibration approach for the zero-shot point cloud semantic segmentation problem. Our approach specifically addresses the problem that existing generative segmentation approaches [18,19] often misclassify between seen and unseen classes. As illustrated in Figure 2, we design a dual-branch network architecture. One branch is the primary segmentation branch, following existing work [19], which tends to produce segmentation results that can better distinguish different objects but often misclassifies between seen and unseen classes. The other branch is the visual–semantic fusion branch, which fuses the semantic embeddings with point cloud features through contrastive learning to identify potential features, especially for unseen classes. The outputs of the two branches are adaptively combined to calibrate the bias and produce accurate segmentation probabilities.

To formulate our problem, we establish an object category set C, which includes the seen $C^S$ and unseen classes $C^U$. Following the general setting of inductive generalized ZSL works, our training process takes all the object category embeddings for $C=\{C^S,C^U\}$ and only the point clouds of seen classes $C^S$ with their ground truth labels to train the network. After training, the network will encounter point clouds from either seen or unseen classes.

In the remainder of this section, we first introduce the network modules, including the primary segmentation branch (Section 3.2), the visual–semantic fusion branch (Section 3.3), and the adaptive bias calibration module (Section 3.4). Then, we describe the entire training strategy, which is a three-stage training process (Section 3.5).

3.2. Primary Segmentation Branch

We adopt the network architecture from [19] as the primary segmentation branch. Specifically, this branch consists of three modules: an encoder E, a segmentor S, and an auxiliary generator G. The encoder takes point clouds as input, where each point contains a 3D coordinate and its accompanying attributes provided by specific datasets, and outputs per-point features. The segmentor uses these features as input to predict segmentation $P_{main}$, i.e., the probability that the points belong to each class.

The auxiliary generator G is used only during the training process to assist in the inductive generalization to unseen classes. It takes a class embedding $t_u$ as input, which is the embedding of a given class label encoded by pre-trained Natural Language Processing (NLP) models, i.e., GloVe [37] + Word2Vec [38]. Note that $t_u$ is disturbed by random noise $z_u$ before being fed into the generator in order to introduce randomness and diversify the generated data. The generator outputs the features $\widetilde{F_u}$ of the given classes, which share the same dimensions as those of encoder E. In this way, the primary segmentation branch can produce the visual features $F_{s+u}$ for both the seen and unseen classes.

3.3. Visual–Semantic Fusion Branch

The proposed visual–semantic fusion branch aims to integrate the visual features $F_{s+u}$ with the category semantic embeddings T for a more accurate classification between the seen and unseen classes. The network architecture of the branch is illustrated in Figure 3. We utilize a contrastive learning strategy to train it. $F_{s+u}$ is the visual features produced by the primary segmentation branch, which consists of the seen features from the encoder E and unseen features synthesized by the generator G. T denotes the semantic embeddings of all the category labels C. We first separately project the visual features $F_{s+u}$ and semantic embeddings T to obtain the same dimensions along the feature channels, denoted as $\widehat{F}$ and $\widehat{T}$.

Then, we perform channel and spatial attention to fuse the visual features $\widehat{F}$ and semantic embeddings $\widehat{T}$. Specifically, channel attention calculates the importance of each channel of one modality, allowing the features of the other modality to concentrate on the relevant key channels. Meanwhile, spatial attention enhances significant regions and depresses irrelevant regions by analyzing the spatial correlation of the two modalities.

The channel attention is formulated as follows:

$$\begin{array}{c}\hfill Q_{tc}=\widehat{T}{W}_{q{t}_c},K_{tc}=\widehat{T}{W}_{k{t}_c},V_{fc}=\widehat{F}{W}_{v{f}_c},\\ \hfill F_c=\widehat{F}+Softmax({\displaystyle \frac{Q_{tc}^T{K}_{tc}}{\sqrt{D_K}}})V_{fc},\end{array}$$

(1)

$$\begin{array}{c}\hfill Q_{fc}=\widehat{F}{W}_{q{f}_c},K_{fc}=\widehat{F}{W}_{k{f}_c},V_{tc}=\widehat{T}{W}_{v{t}_c},\\ \hfill T_c=\widehat{T}+Softmax({\displaystyle \frac{Q_{fc}^T{K}_{fc}}{\sqrt{D_K}}})V_{tc},\end{array}$$

(2)

where $W_{q{t}_c}$, $W_{k{t}_c}$, and $W_{v{f}_c}$ are the learnable projection weights that map $\widehat{T}$ to the query ($Q_{tc}$) and key ($K_{tc}$) vectors and map $\widehat{F}$ to the value ($V_{fc}$) vectors. Similarly, $W_{q{f}_c}$, $W_{k{f}_c}$, and $W_{v{t}_c}$ project $\widehat{F}$ to the corresponding query ($Q_{fc}$) and key ($K_{fc}$) and project $\widehat{T}$ to the value ($V_{tc}$). $D_K$ is the dimension size of the projected vectors. In this way, we obtain the fused visual features $F_c$ and semantic features $T_c$ encoded with channel attention.

The spatial attention is computed as follows:

$$\begin{array}{c}\hfill Q_{fs}=F_c{W}_{q{f}_s},K_{ts}=T_c{W}_{k{t}_s},V_{ts}=T_c{W}_{v{t}_s},\\ \hfill F_s=F_c+Softmax({\displaystyle \frac{Q_{fs}{K}_{ts}^T}{\sqrt{D_K}}})V_{ts},\end{array}$$

(3)

$$\begin{array}{c}\hfill Q_{ts}=T_c{W}_{q{t}_s},K_{fs}=F_c{W}_{k{f}_s},V_{fs}=F_c{W}_{v{f}_s},\\ \hfill T_s=T_c+Softmax({\displaystyle \frac{Q_{ts}{K}_{fs}^T}{\sqrt{D_K}}})V_{fs},\end{array}$$

(4)

where $W_{q{f}_s}$, $W_{k{t}_s}$, and $W_{v{t}_s}$ denote the learnable weights that map $F_c$ to its corresponding query ($Q_{fs}$) and map $T_c$ to the key ($K_{ts}$) and value ($V_{ts}$) vectors, whereas $W_{q{t}_s}$, $W_{k{f}_s}$, and $W_{v{f}_s}$ similarly project $T_c$ and $F_c$ to the query ($Q_{ts}$), key ($K_{fs}$), and value ($V_{fs}$). $D_K$ is the dimension of the output vectors. After the spatial attention, we obtain the fused visual features $F_s$ and semantic features $T_s$.

Next, the features $F_s$ and $T_s$ are input into a multi-layer perceptron (MLP) and a following layer normalization to obtain the final fused features:

$$\begin{array}{c}\hfill F_{fusion}=LayerNorm(F_s+MLP(F_s)),\end{array}$$

(5)

$$\begin{array}{c}\hfill T_{fusion}=LayerNorm(T_s+MLP(T_s)).\end{array}$$

(6)

We compute the contrastive loss for the visual–semantic fusion branch. For a visual feature, the semantic feature corresponding to its category is regarded as a positive sample, while all other semantic features are considered negative. The contrastive loss $L_{aux}$ is defined as follows:

$$\begin{array}{c}\hfill P_{aux}=softmax(F_{fusion}^T{T}_{fusion}).\end{array}$$

(7)

$$\begin{array}{c}\hfill L_{aux}=-log{\displaystyle \frac{exp(F_{fusion}^c·T_{fusion}^c/\tau )}{\sum (exp(F_{fusion}^c·T_{fusion}^c/\tau )+exp(F_{fusion}^c·T_{fusion}^{\widehat{c}}/\tau ))}},\end{array}$$

(8)

where $P_{aux}$ represents the probability of each point belonging to each category. $\tau$ is the temperature parameter, c indicates the class associated with the visual features, and $\widehat{c}$ represents other classes different from c.

3.4. Adaptive Bias Calibration Module

We introduce an adaptive bias calibration method based on the confidence of the predicted segmentation probabilities to improve the model’s adaptability in open scenarios. That is, we estimate a confidence that indicates the likelihood of a successful prediction and use it as a weight to integrate the predictions of two branches. Intuitively, when the confidence of the primary segmentation branch is high, we assign a smaller weight to the prediction from the visual–semantic fusion branch, and conversely, when the confidence is low, we give more weight to the visual–semantic fusion branch’s prediction.

Specifically, we adopt the kurtosis measure to estimate the predicted confidence, denoted as $\kappa$ (see Equation (9)), which is a statistical measure that describes the shape of the data distribution. Kurtosis reflects the steepness or flatness of the distribution, particularly indicating the height of the peak in the probability density curve at the average value. A higher kurtosis indicates that the distribution has a more pronounced peak, representing a higher degree of prediction confidence. Given the predicted point-wise segmentation probabilities $P_{main}$, we adopt $\kappa$ to balance the effect of the two branches:

$$\begin{array}{c}\hfill \kappa =E{[{({\displaystyle \frac{P_{main}-\mu }{\sigma }})}^4]}^{\rho },\end{array}$$

(9)

$$\begin{array}{c}\hfill P=P_{main}+{\displaystyle \frac{\beta }{\kappa }}{P}_{aux},\end{array}$$

(10)

where $\mu$ and $\sigma$ are the mean and standard deviation of $P_{main}$, and $\rho$ is a parameter that modulates the strength of the kurtosis influence. $\beta$ is a hyperparameter that controls the confidence weight value.

3.5. Three-Stage Training Strategy

Next, a three-stage training strategy is introduced to progressively train the modules of our dual-branch network. The first stage trains the encoder E and segmentor $S_{seen}$ with the point clouds $P_s$ of seen classes c from the training set. We calculate the cross-entropy loss between the prediction and ground truth labels $Y_s$:

$$\begin{array}{c}\hfill L_s=-\sum _c{Y}_{s}log(S_{seen}(E(P_s))),\end{array}$$

(11)

The second stage freezes the encoder E and trains the generator G on the data of seen classes from the training set. The goal is to ensure that the synthesized features from G closely resemble the real features extracted by the frozen encoder E. The loss function for this stage is

$$\begin{array}{c}\hfill L_G=\sum _c(L_{align}+L_{con}+L_{MMD}+\alpha L_{cst}),\end{array}$$

(12)

where $L_{align}$ refers to the semantic–visual alignment loss, which computes the cosine distance between the semantic and visual features of the seen classes. $L_{con}$ represents the contrastive loss between the features of real and synthetic data for the seen classes. $L_{MMD}$ is the maximum mean deviation (MMD) loss to reduce the distributional mismatch between the synthesized features $\widetilde{F_s}$ and real features $F_s$ of seen classes. $L_{cst}$ denotes the semantic–visual consistency loss, which ensures that the distance distributions of seen and unseen classes are consistent in synthesized visual and semantic space by minimizing the cosine distance. $\alpha$ is a hyperparameter for loss balance.

The third stage freezes the encoder E and generator G to train the visual–semantic fusion branch and the final segmentor S. It leverages E to acquire real features of seen classes c and the G to produce synthesized features of unseen classes $c^{\prime }$. The loss for this stage is as follows:

$$\begin{array}{c}\hfill L_{all}=\lambda \times L_{seg}+(1-\lambda )\times L_{aux},\end{array}$$

(13)

where $\lambda$ is the hyperparameter to balance the two losses.

The segmentation loss $L_{seg}$ is defined as:

$$\begin{array}{c}\hfill L_{seg}=-\sum _c{Y}_{s}log(S(F_s))-\sum _{c^{\prime }}{Y}_{u}log(S(\widetilde{F_u})),\end{array}$$

(14)

where $Y_u$ denotes ground truth labels of unseen classes.

4. Experiments

4.1. Settings

Datasets. 

We conduct experiments on three public 3D semantic segmentation datasets: ScanNet [39], S3DIS [40], and SemanticKITTI [41], which follow the setting of SV-Seg [19]. We select four unseen categories for each dataset. For ScanNet, which includes 20 categories, the unseen classes are “desk”, “bookshelf”, “sofa”, and “toilet”. For S3DIS, with 13 categories, the unseen classes are “beam”, “column”, “window”, and “sofa”. For SemanticKITTI, with 19 categories, we set “motorcycle”, “truck”, “bicyclist”, and “traffic sign” as unseen classes.

Metrics. 

We employ the mean Intersection-over-Union (mIoU) to measure the accuracy of segmentation results and the Harmonic mean of mIoU (HmIoU) to report the overall performance on the combined seen and unseen classes. The metrics are defined as follows:

$$\begin{array}{c}\hfill mIoU={\displaystyle \frac{1}{k}}\sum _{i=1}^k{\displaystyle \frac{T{P}_i}{F{N}_i+F{P}_i+T{P}_i}},\end{array}$$

(15)

$$\begin{array}{c}\hfill HmIoU={\displaystyle \frac{2\times mIo{U}_{seen}\times mIo{U}_{unseen}}{mIo{U}_{seen}+mIo{U}_{unseen}}},\end{array}$$

(16)

where $T{P}_i$, $F{P}_i$, and $F{N}_i$ represent the counts of true positives, false positives, and false negatives for category $c_i$, respectively. k is the total number of categories.

Baselines. 

We compare the proposed approach with four baselines. Following 3DGenZ [18], we evaluated ZSLPC-Seg [32] and DeViSe-3DSeg [42], which are developed for ZSL classification and 2D segmentation, and are then adapted to solve the zero-shot point cloud segmentation problem. In addition, we compare with 3DGenZ [18] and SV-Seg [19], which are the state-of-the-art generative point cloud segmentation methods.

4.2. Implementation Details

To facilitate comparison with the baseline, we utilize the same encoder and generator described by SV-Seg [19]. The encoders for each dataset are different, i.e., FKAConv [43] for ScanNet, ConvPoint [44] for S3DIS, and KPConv [28] for SemanticKITTI. The generator uniformly employs the Generative Moment Matching Network (GMMN) [20]. All encoders are pre-trained on seen class data using parameters recommended in their related papers. When pre-training is complete, we train the generator using the frozen encoder to obtain features from the seen classes. We use $GloVe+Word2Vec$ to compute a $600D$ semantic embedding vector for each category label. The model is trained for a total of 20 epochs with the Adam optimizer. The learning rates are $0.0002$ for the generator, $0.007$ for the encoder, and $0.07$ for the segmentor. As for the visual–semantic fusion branch, we adopt the SGD optimizer with a setting of $0.0002$ as the initial learning rate. We use the same hyperparameters for all three datasets. Specifically, the parameter $\rho$ in Equation (9) is set to 1, the parameter $\beta$ in Equation (10) is set to $1.2$, and the parameter $\lambda$ in Equation (13) is set to $0.7$.

4.3. Comparisons

Our comparison against the state-of-the-art methods is presented in

Table 1. Experimental results are presented on three benchmark datasets. The first part introduces the upper limits of various supervision levels, while the second part demonstrates the performance of the baseline and our method on generalized 3D zero-shot semantic segmentation. In this context, $C^S$ represents visible classes, and $C^U$ represents unseen classes. ${\widehat{C}}^U$ represents the synthesized unseen data.

Method	Training Set		ScanNet				S3DIS				SemanticKITTI
	Encoder	Segmentor	mIoU			HmIoU	mIoU			HmIoU	mIoU			HmIoU
			${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$		${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$		${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$
Full supervision	$C^S\cup C^U$	$C^S\cup C^U$	43.3	51.9	45.1	47.2	74.0	50.0	66.6	59.6	59.4	50.3	57.5	54.5
Unseen classes only for segmentor	$C^S$	$C^S\cup C^U$	41.5	39.2	40.3	40.3	60.9	21.5	48.7	31.8	52.9	13.2	42.3	21.2
Supervision with seen classes	$C^S$	$C^S$	39.2	0.0	31.3	0.0	70.2	0.0	48.6	0.0	55.8	0.0	44.0	0.0
ZSLPC-Seg [32]	$C^S$	$C^U$	16.4	4.2	13.9	6.7	5.2	1.3	4.0	2.1	26.4	10.2	21.8	14.7
DeViSe-3DSeg [42]	$C^S$	$C^U$	12.8	3.0	10.9	4.8	3.6	1.4	3.0	2.0	42.9	4.2	27.6	7.5
3DGenZ [18]	$C^S$	$C^S\cup {\widehat{C}}^U$	32.8	7.7	27.8	12.5	53.1	7.3	39	12.9	41.4	10.8	35	17.1
SV-Seg [19]	$C^S$	$C^S\cup {\widehat{C}}^U$	34.5	14.3	30.4	20.2	58.9	9.7	43.8	16.7	46.4	12.8	39.4	20.1
Ours	$C^S$	$C^S\cup {\widehat{C}}^U$	34.9	14.9	30.9	20.9	68.1	10.4	50.3	18.0	49.5	15.2	42.3	23.3

. Compared to the best performance (from SV-Seg [19]), our approach achieves HmIoU improvements of 0.7%, 1.3%, and 3.2% on the ScanNet, S3DIS, and SemanticKITTI datasets, respectively. The mIoU improvements for unseen classes are 0.6%, 0.7%, and 2.4% for the three datasets, demonstrating that our approach effectively mitigates semantic bias and enhances generalization to unseen classes. In addition, the improvements for seen classes are 0.4%, 9.2%, and 3.1%. This indicates that fewer unseen classes are wrongly classified as seen ones, leading to an mIoU increase for seen classes.

We conduct a qualitative comparison with SV-Seg [19], which performs bias calibration by subtracting a pre-defined constant value from their predicted probability scores, on the three benchmark datasets, as shown in Figure 4. Consequently, for the unseen classes, such as “desk” in ScanNet, “beam” in S3DIS, and “motorcycle” in SemanticKITTI, our model outperforms SV-Seg [19], demonstrating its effectiveness in addressing the bias toward seen classes. Regarding the seen classes, such as “sink” of ScanNet, “chair” of S3DIS, and “poly” of SemanticKITTI, the baseline incorrectly classifies them as unseen classes: “toilet”, “beam/column”, and “truck”. In contrast, our model exhibits better accuracy in recognizing these seen class samples. These observations indicate that the fixed calibration strategy in SV-Seg [19] can negatively impact the accuracy for seen classes, while our adaptive bias calibration method proves to be more effective in this regard.

4.4. Ablation Studies

4.4.1. Module Effectiveness

Table 2. Effects of the VSFM and BCM on the ScanNet dataset.

Methods	VSFM	BCM	mIoU			HmIoU
			${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$
Fixed Calibration	✕	✕	34.6	14.2	30.5	20.1
w/o BCM	✓	✕	34.9	13.9	30.8	19.9
w/o VSFM	✕	✓	32.4	14.0	28.7	19.6
Ours	✓	✓	34.9	14.9	30.9	20.9

✕ symbolizes the absence of corresponding modules, ✓symbolizes the presence of corresponding modules.

evaluates the effectiveness of the proposed visual–semantic fusion branch and the adaptive bias calibration module, especially their collaborative mechanism. The first baseline, denoted as Fixed Calibration, directly adopts the prediction results of the existing generative segmentation network SV-Seg [19]. Note that here, deviation calibration is performed by subtracting the fixed calibration value. The second experiment, i.e., w/o BCM, employs our dual-branch network architecture to calibrate bias by averaging the predictions of the two branches, rather than using our adaptive calibration module. The third experiment, i.e., w/o VSFM, removes the core fusion module of the visual–semantic branch, directly calculates the similarity matrix between visual and semantic features as the output segmentation probability of this branch, and then applies our adaptive bias calibration scheme.

The experimental results reveal that the most significant improvement in the performance occurs when the VSFM and BCM modules work together, resulting in an increase of 0.8% in HmIoU. The absence of the VSFM module (w/o VSFM) leads to a more noticeable decline in the performance of seen classes, whereas the lack of BCM (w/o BCM) causes a more substantial decrease in the performance of unseen classes. These results suggest that the two proposed modules establish a bidirectional compensatory system with complementary characteristics: on one hand, the VSFM module employs the channel and spatial attention mechanism to facilitate dynamic interaction and fusion between visual–semantic features, effectively enhancing the spatial separability of visual features for both seen and unseen classes. On the other hand, the BCM module implements a confidence-based weight adjustment strategy that dynamically regulates the contribution of auxiliary predictions while adaptively incorporating these results into the predictions of the main branch.

4.4.2. Channel and Spatial Attention

To evaluate the effectiveness of channel and spatial attention mechanisms used in the VSFM, we conduct a quantitative analysis through an ablation study (see

Table 3. Ablation experiments of the VSFM on the ScanNet dataset, including Channel and Spatial Attention.

Channel	Spatial	mIoU			HmIoU
		${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$
✕	✕	32.8	13.9	29.0	19.5
✓	✕	32.0	13.8	28.3	19.3
✕	✓	31.8	14.4	28.4	19.9
✓	✓	34.9	14.9	30.9	20.9

✕ symbolizes the absence of corresponding modules, ✓symbolizes the presence of corresponding modules.

). The experimental results indicate that while each mechanism yields only slight improvements when used independently, their combination results in a 1.4% increase in the HmIoU metric on the ScanNet dataset. This underscores the cumulative enhancement effect of the dual attention mechanism: spatial attention improves the representation of critical areas by analyzing spatial relationships between modalities, while channel attention fosters a joint focus on key channels across them. The collaboration between these two mechanisms effectively balances the need for detailed recognition and generalization.

4.4.3. Prediction Confidence Estimation

To assess the impact of various prediction confidence estimation methods on segmentation performance, we compare five approaches based on the prediction probability distribution: Maximum Probability (Max), Entropy [45], Energy Score [46], Variance [47], and Kurtosis [48]. In addition, as a reference, SV-Seg [19], which employs pre-defined calibration values, is adopted as the baseline. As shown in

Table 4. The influence of common uncertainty algorithms on segmentation performance.

Method	ScanNet				S3DIS
	mIoU			HmIoU	mIoU			HmIoU
	${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$		${\mathit{C}}^{\mathit{S}}$	${\mathit{C}}^{\mathit{U}}$	$\mathit{All}$
Baseline	34.5	14.3	30.4	20.2	58.9	9.7	43.8	16.7
Max	34.8	13.3	30.5	19.3	67.7	9.8	49.9	17.1
Entropy	34.4	14.1	30.3	20.0	68.1	9.3	50.0	16.4
Energy	34.6	14.1	30.5	20.0	67.3	9.5	49.5	16.7
Variance	34.8	13.8	30.6	19.8	66.9	9.8	49.3	17.1
Kurtosis	34.9	14.9	30.9	20.9	68.1	10.4	50.3	18.0

, the kurtosis-based estimation method exhibits the most significant performance improvement on the ScanNet and S3DIS datasets, with the HmIoU metrics increasing by 0.7% and 1.3%, respectively. Moreover, all methods that utilize prediction confidence for adaptive bias calibration achieve segmentation performances that are close to or exceed the baseline in terms of mIoU for unseen classes.

4.4.4. Hyperparameters

To verify the impact of hyperparameter values on network performance, we conduct experiments on the S3DIS and ScanNet datasets for the hyperparameter $\beta$ used to control the confidence weight in Equation (10) and the hyperparameter $\lambda$ used to balance the two branches in Equation (13), as shown in Figure 5. Figure 5a demonstrates that on the S3DIS dataset, setting the hyperparameter $\beta$ to $1.2$ results in the optimal HmIoU value. Figure 5b reveals that on the ScanNet dataset, the best HmIoU value is achieved when the hyperparameter $\lambda$ is $0.7$. These findings indicate that during network training, the main segmentation branch plays a more important role than the auxiliary branch.

4.5. Limitations

Our semantic-guided adaptive bias calibration approach still has limitations. In the visual–semantic fusion branch, segmentation prediction is obtained by aligning visual and semantic data. However, it aligns point cloud features of the same category with a single semantic embedding, overlooking the complexity and diversity within these features. We will explore methods to achieve multi-scale visual and semantic alignment in the future, which could lead to more accurate segmentation results.

5. Discussion

We propose a semantic-guided adaptive bias calibration approach to alleviate the bias towards seen classes in generalized zero-shot point cloud segmentation. We explore the visual–semantic correlation through the spatial-channel attention mechanism and utilize visual–semantic alignment to effectively distinguish different category samples in the feature space and generate auxiliary predictions to counteract the bias. We adaptively adjust the fusion weights between the main prediction from the primary segmentation branch and the auxiliary prediction from the visual–semantic fusion branch, providing auxiliary guidance preferentially for low-confidence samples while suppressing unnecessary corrections to high-confidence predictions. The experiments on common benchmarks confirm that our approach has significantly enhanced zero-shot segmentation performance. Despite these advances, our method still faces challenges in aligning visual and semantic information. This method aligns point cloud features of the same class with a single semantic embedding, which, in practical applications, may lead to segmentation misjudgments due to differences in object morphology, similar to other methods. In the future, we plan to explore multi-scale alignment methods between visual and semantic information further to improve the performance of zero-shot point cloud semantic segmentation.