BAFNet: A Few-Shot Segmentation Algorithm Based on Two-Stage Backbone Network Adaptation Fine-Tuning via Meta-Learning

Abstract

The objective of few-shot segmentation is to segment novel categories given only a few annotated support images. Current FSS methods typically rely on pretrained backbone networks while often overlooking the inherent discrepancy between pretraining tasks and downstream segmentation tasks. This oversight renders the models susceptible to noise interference and hinders rapid generalization to novel categories. To address these limitations, we propose BAFNet, a novel few-shot segmentation algorithm based on two-stage backbone adaptive fine-tuning. Our approach incorporates a Feature Activation Adapter module into the backbone network, which operates through similarity feature enhancement and low-dimensional adaptive learning. Building upon this foundation, we develop an adapter-based fine-tuning strategy for the training phase that enhances the backbone network’s capacity for extracting category-relevant features while optimizing similarity representation of the extracted features. Additionally, we introduce a support set-driven, in-episode, online fine-tuning strategy for the testing phase, which leverages data augmentation to generate pseudo-query sets for supervised fine-tuning optimization. Through comprehensive quantitative and qualitative experiments conducted on PASCAL-5ⁱ, COCO-20ⁱ, and the industrial MT Defect Dataset, our results demonstrate that the proposed BAFNet model achieves state-of-the-art few-shot segmentation performance while utilizing the minimal number of trainable parameters. Our method obtains superior performance for both the mean intersection over union and foreground-background intersection over union evaluation metrics, exhibiting remarkable applicability for both general images in complex scenes and industrial defect segmentation under few-shot conditions.

1. Introduction

With the advancement of deep learning, deep neural networks (DNNs) based on convolutional neural networks (CNNs) and transformer architectures have demonstrated significant progress in semantic segmentation tasks [1,2]. However, training such networks requires substantial amounts of densely annotated data [3], which often proves difficult to obtain. While extensive annotated data and parameters provide substantial information to networks, they also constrain the networks to limited datasets, resulting in insufficient generalization capability. These models struggle to rapidly adapt to novel categories beyond their training distribution.

In contrast, humans can easily recognize new categories after observing only a few annotated examples, leveraging their powerful knowledge background. Inspired by this capability, numerous researchers have focused on enabling convolutional neural networks to recognize novel categories using limited annotated data through a process known as few-shot learning (FSL) [4,5,6,7,8,9,10].

As an application branch of few-shot learning in segmentation tasks, few-shot segmentation aims to segment target regions of unknown categories using only limited annotated data. Most existing FSS research follows the meta-learning paradigm [11,12,13,14,15,16,17,18,19,20,21].

The critical requirement for few-shot segmentation tasks under the meta-learning paradigm is that the segmentation categories during testing remain strictly unseen during training. Current methods primarily focus on enhancing the contextual modeling capability of downstream segmentation networks during training while overlooking the transfer process from training categories to testing categories. Prolonged training with training set data causes models to overemphasize training phase target categories, making them more susceptible to noise influence and hindering rapid generalization to novel categories.

Furthermore, to prevent overfitting in few-shot segmentation, using pretrained backbone networks with frozen parameters during training has become a standard paradigm. However, directly employing ImageNet pretrained backbone networks for feature extraction may not represent the optimal approach. Few-shot tasks require extracting the most relevant semantic cues from backbone networks while simultaneously minimizing the generation of irrelevant noise.

In contemporary research, fine-tuning backbone networks for downstream tasks has become a common practice in deep learning. Transfer learning approaches utilizing ImageNet pretrained weights have achieved initial breakthroughs in various visual tasks, exemplified by R-CNN for detection [22] and FCN for segmentation [1]. However, in few-shot segmentation tasks, applying standard fine-tuning methods to learn novel categories often leads to overfitting or catastrophic forgetting [23].

To address these limitations, parameter-efficient tuning techniques have emerged as effective solutions. Several mainstream PET approaches have been developed, including prompt-tuning, prefix-tuning, and LoRA methods. In computer vision domains, adapters and prompt tuning have demonstrated remarkable performance in transfer learning. The few-shot learning community has also witnessed numerous attempts at backbone network fine-tuning, yielding significant achievements. For instance, P-Transfer [24] employs neural architecture search to identify parameters requiring fine-tuning in few-shot classification tasks, though this method exhibits considerable complexity and challenges for direct application to few-shot segmentation.

Other works reference the prompt tuning paradigm from natural language processing [17], fine-tuning specific parameters within vision transformers. While these methods demonstrate effectiveness in transformer-based architectures, most few-shot segmentation models utilize CNN-based backbones, necessitating substantial adaptations for prompt-based approaches. Additionally, some methods [23] introduce highly constrained parameter subsets for fine-tuning yet remain suboptimal for few-shot segmentation tasks.

Distinct from these approaches, we design a backbone adaptive fine-tuning network for few-shot segmentation. Inspired by partial parameter fine-tuning strategies, we construct specialized adapter structures within the backbone network of few-shot segmentation models to capture category-relevant information. We fine-tune these adapter parameter sets with two primary objectives: extracting the most relevant semantic cues from the backbone network and facilitating category adaptation from training to testing datasets.

To address the aforementioned challenges, we propose a novel, two-stage fine-tuning framework based on adapters, as illustrated in Figure 1. We reconsider the frozen backbone paradigm during training and introduce a novel mechanism: integrating adapter structures containing minimal trainable parameters within the backbone network. This approach enables enhanced model generalization when learning new tasks. For addressing category discrepancy between training and testing datasets, we employ the same network architecture for online fine-tuning during testing episodes.

Figure 1. Comparison of conventional FSS versus the proposed fine-tuning approach.

Our contributions are summarized as follows:

• We propose a Feature Activation Adapter module founded on similarity feature enhancement and low-dimensional adaptive learning. By integrating FAA modules into the backbone network during both training and testing phases, we enhance the network’s capacity for extracting category-relevant features. This approach optimizes similarity representation in feature extraction while reducing intra-class discrepancies caused by the mismatch between prior biases learned from general datasets and the spatial consistency requirements of few-shot segmentation tasks.
• We develop an adapter-based fine-tuning strategy for the training phase. Through the incorporation of feature activation adapters into pretrained backbone networks, we enable superior generalization capability when learning new tasks, thereby achieving more accurate correlation modeling between support and query features.
• We introduce a data augmentation-based fine-tuning strategy for the testing phase. This methodology facilitates online fine-tuning of model parameters through pseudo-query sample generation, effectively addressing the novel class adaptation challenge in few-shot segmentation tasks.
• We conduct comprehensive quantitative and qualitative experiments on three benchmark datasets: PASCAL-5ⁱ, COCO-20ⁱ, and the industrial MT Defect Dataset. The experimental results demonstrate that our BAFNet model achieves state-of-the-art few-shot segmentation performance. Notably, our method exhibits remarkable applicability across diverse scenarios, including general images in complex environments and industrial defect segmentation.

2. Related Works

In this section, we review the relevant literature on few-shot learning, meta-learning, and few-shot segmentation. We focus our discussion on key methodological developments and current research trends in these domains.

2.1. Few-Shot Learning and Meta Learning

With the continuous advancement of deep learning technologies, few-shot learning has achieved remarkable progress across various computer vision tasks, including image classification [25], object detection [26], and semantic segmentation [27]. Most few-shot learning approaches follow a two-phase meta-learning paradigm comprising meta-training and meta-testing stages [28]. Meta-learning, often conceptualized as “learning-to-learn” [29], aims to endow models with problem-solving capabilities by emulating human and animal learning paradigms. Unlike traditional artificial intelligence methods that employ fixed learning algorithms, meta-learning focuses on improving the learning algorithm itself through accumulated experience from multiple learning episodes. This approach offers several advantages, including enhanced data utilization efficiency, improved computational efficiency, and the ability to rapidly adapt to novel, unseen tasks by leveraging prior knowledge. Particularly in multi-task scenarios, meta-learning can extract substantial knowledge from a series of tasks to enhance learning of new tasks within the same series. Meta-learning holds promise for addressing several fundamental challenges in contemporary deep learning, such as improving data efficiency, mitigating data scarcity, facilitating knowledge transfer, and advancing unsupervised learning.

Model-agnostic meta-learning (MAML) [28] represents one of the most prominent meta-learning algorithms. Its core concept involves learning model parameters that enable rapid adaptation to new tasks, with this fast adaptation capability acquired through meta-training across multiple tasks. The MAML framework is architecture-agnostic, making it broadly applicable to various machine learning tasks, including image recognition and natural language processing. Building upon MAML, numerous meta-learning algorithms have been developed to enhance meta-learning capabilities, such as LEO [30], Reptile [31], and Meta-SGD [32].

Episodic training [33] serves as the cornerstone of meta-learning in few-shot learning contexts. This approach partitions datasets into numerous small groups (episodes) to simulate few-shot scenarios, where each episode constitutes an independent few-shot learning task. Models acquire few-shot recognition capabilities by learning general knowledge across these tasks. Based on episodic training, few-shot learning methods can be further categorized into three primary approaches: (1) metric-based methods [34,35], (2) optimization-based methods [28,36], and (3) data augmentation-based methods [37,38].

Metric-based methods represent the most widely adopted approach in few-shot learning, focusing on classification or regression through learning and comparing similarity relationships between data samples. The core principle involves learning an appropriate distance metric function that accurately captures similarities and differences between samples. Data augmentation-based methods enhance data quantity and diversity through various transformations and operations on original data, thereby alleviating data acquisition challenges and improving model robustness and generalization. Our approach integrates both metric-based and data augmentation-based methodologies, accomplishing episodic training while preventing overfitting through feature adaptive metric learning and data augmentation.

2.2. Few-Shot Segmentation

As a specific application of few-shot learning in segmentation tasks, few-shot segmentation aims to partition target regions of unknown categories using limited annotated data. The pioneering OSLSM work [39] established the fundamental paradigm in this field. This paradigm closely relates to metric-based few-shot learning, employing separate conditional and segmentation branches to extract feature representations from support and query images, respectively, for segmentation completion. Subsequently, researchers introduced masked average pooling [40] to generate more representative prototype vectors, thereby enabling more accurate measurement of query features. In recent years, methodologies that concatenate expanded prototype vectors with query features and complete segmentation through decoders [11,15,18,19,41] have gradually replaced traditional cosine distance measurement approaches. Furthermore, innovations including prototype alignment modules [12], prior masks, feature enhancement modules [15], base learners [11], background mining and suppression modules [17], multi-similarity modules, and attention mechanisms [18] have all contributed significantly to the advancement of few-shot segmentation.

The few-shot image semantic segmentation task possesses substantial research value due to its strong generalization capability and minimal requirement for labeled data. Meta-learning approaches remain the predominant framework within the field, where models segment query images under the guidance of support sets. However, several challenges continue to impede the development of few-shot segmentation algorithms.

The intra-class variance problem represents one critical challenge requiring resolution. This issue arises from significant appearance variations (such as shape, texture, illumination, and occlusion) among samples within the same category. Under few-shot conditions, models struggle to capture correlations between support and query sets amid diverse feature variations, leading to suboptimal segmentation results.

The backbone network prior bias constitutes another crucial problem demanding attention. This bias refers to the inherent prejudices present in pretrained backbone networks, where features learned from large-scale datasets tend to favor high-frequency textures or local patterns, creating discrepancies with the semantic abstraction and spatial consistency requirements of few-shot segmentation tasks. Pretrained features oriented toward general object recognition prove difficult to adapt to fine-grained pixel-level tasks in few-shot segmentation, manifesting as insufficient edge sensitivity. High-dimensional features extracted by backbone networks may contain segmentation-irrelevant information, interfering with feature matching under few-shot conditions. Traditional backbone network designs lack optimization for the meta-learning paradigm of few-shot segmentation, resulting in poor compatibility between feature embedding spaces and prototype matching.

To address intra-class variance, where targets of the same category exhibit discrete feature distributions due to variations in pose, occlusion, and illumination, we propose a Feature Activation Adapter module. During feature generation, we insert the FAA module between various layers of the backbone network to mitigate the impact of intra-class variations.

Regarding backbone network prior bias, existing few-shot image semantic segmentation methods typically employ frozen pretrained backbone networks. However, features learned from general datasets like ImageNet tend to prioritize global semantic discrimination, creating misalignment with the spatial consistency required for few-shot segmentation tasks. Inspired by the backbone fine-tuning approach in the SVF network model framework [42], we insert the Feature Activation Adapter module between backbone network layers during feature generation. This design activates relevant regions in intermediate layer features to alleviate backbone network prior bias, achieving few-shot task-driven feature adaptation.

Consequently, we propose BAFNet, a few-shot semantic segmentation algorithm constructed through multi-stage fine-tuning strategies during both the training and testing phases.

To provide a clearer structural comparison with representative recent methods, we summarize the key differences in Table 1, including the core mechanism, backbone adaptation strategy, and principal limitation.

Table 1. Comparison of representative few-shot segmentation methods in related works.

Method	Core Mechanism	Backbone Adaptation	Main Limitation
DPCN [43]	Dynamic prototype convolution	Frozen backbone	Limited adaptation to novel-category distribution shift
BAM [11]	Base learner with anti-classifier design	Frozen backbone	Backbone prior bias remains insufficiently corrected
MCE [44]	Masked cross-image feature encoding	Frozen backbone	Strong cross-image interaction but limited parameter-efficient adaptation
HDMNet [45]	Hierarchical dense correlation distillation	Frozen backbone	Higher structural complexity and still limited test-time adaptation
SAGNN [46]	Scale-aware graph reasoning	Frozen backbone	Graph modeling overhead with weak direct backbone correction
HSNet [47]	Hypercorrelation matching	Frozen backbone	High matching complexity and sensitivity to feature noise
SCCAN [48]	Self-calibrated cross-attention	Frozen backbone	Attention enhancement without explicit backbone fine-tuning
BAFNet (Ours)	Similarity activation + feature adapter + two-stage fine-tuning	Parameter-efficient adapter tuning (train and test)	Requires episode-level test-time adaptation

3. Proposed Method

In this section, we present the architectural framework and operational mechanism of the proposed BAFNet model.

3.1. Overview

Under extremely limited data conditions, model overfitting represents a critical challenge. Most existing few-shot segmentation methods adopt the paradigm of frozen pretrained backbone networks to prevent overfitting issues. While this strategy has achieved certain success, completely freezing ImageNet pretrained backbone networks for few-shot image segmentation may hinder our ability to extract the most relevant semantic cues from support and query images. This limitation arises because the backbone network has learned irrelevant categories during upstream training, introducing noise influenced by these unrelated categories into the feature extraction process.

To address this problem, we modify and improve upon the completely frozen pretrained backbone network paradigm. According to previous few-shot fine-tuning studies, fine-tuning all parameters of the backbone network fails to yield satisfactory results. Inspired by the SVF fine-tuning algorithm [42], which employs singular value decomposition for backbone network fine-tuning, we identify a viable solution: restricting the number of learnable parameters in the backbone network to suit few-shot scenarios.

As illustrated in Figure 2, we adopt a unified two-stage adapter fine-tuning pipeline instead of fully freezing the backbone. In meta-training, adapter parameters and decoder parameters are jointly optimized using episodic training data to improve support–query correlation modeling. In meta-testing, only the adapter parameters are updated within each episode, and support-set augmentation is used to construct pseudo-query pairs for supervised adaptation without violating the test-time label constraints. This design preserves parameter efficiency while improving adaptation from pretraining classes to novel categories.

Figure 2. Two-stage backbone fine-tuning for few-shot segmentation.

3.2. Feature Activation Adapter Module

We propose the Feature Activation Adapter (FAA) module, as illustrated in Figure 3 and Figure 4. For query features $F_q$ and support features $F_s$ from the backbone network, we derive the category prototype P using $F_s$ and the support mask $M_s$. We then process the support features $F_s$, support mask $M_s$, and prototype P through our proposed adapter module. Through feature enhancement and low-dimensional adaptive learning, we obtain the activated category-specific features $F_s^{*}$ and $F_q^{*}$. Subsequently, we add $F_s^{*}$ and $F_q^{*}$ to the original features $F_s$ and $F_q$, respectively, and feed them to the next encoder layer to achieve fine-tuning of the backbone network. We update only the parameters within the adapter while keeping the pretrained parameters in the backbone network frozen, enabling the base segmentation model to effectively adapt to new categories.

Figure 3. Feature Activation Adapter module structure diagram.

Figure 4. Diagram of the feature activation adapter structure.

Figure 3 illustrates the structure of the Feature Activation Adapter module, which comprises three key components: prototype generation, feature activation, and low-dimensional adaptive learning. The prototype generation component extracts category prototype vectors from intermediate layer features. The feature activation component enhances category-relevant regions in the feature maps based on information from the prototype vectors. The low-dimensional adaptive learning module introduces learnable parameters to capture correlation information between dual-branch features. The entire learning process follows a dimension reduction and then dimension restoration pattern; dimension reduction filters high-dimensional noise from the backbone network, while dimension restoration recovers the channel dimensions of feature maps for integration back into the backbone network.

For clarity, we explicitly define all symbols used in the FAA formulation. Let $F_s\in {\mathbb{R}}^{c\times h\times w}$ and $F_q\in {\mathbb{R}}^{c\times h\times w}$ denote support and query features, respectively, $M_s\in {\mathbb{R}}^{h\times w}$ denote the support mask (downsampled to the feature resolution), c denote the channel dimension, and h and w denote the spatial dimensions.

Finally, we inject the adapter-enhanced feature into the encoder using a residual update:

$$F^{\mathrm{out}}=F+\beta ·F^{*}$$

(1)

where F is the original backbone feature, $F^{*}$ is the adapter output, and $\beta$ is a learnable scaling factor controlling the injection strength.

Figure 4 details the specific architecture of the FAA module. For prototype generation, we take the support features $F_s\in {\mathbb{R}}^{c\times h\times w}$ and corresponding masks $M_s\in {\mathbb{R}}^{h\times w}$ as inputs (where c denotes the feature dimension and h and w represent the spatial dimensions, with $M_s$ downsampled to match the feature resolution). The model filters background vectors through spatial multiplication between support features $F_s$ and masks $M_s$ and then applies average pooling to foreground feature vectors to obtain the current layer’s category prototype $P\in {\mathbb{R}}^c$. This process is mathematically defined as follows:

$$P=\mathrm{Mean}(F_s\odot M_s)$$

(2)

where ⊙ denotes the Hadamard (element-wise) product and $\mathrm{Mean}(·)$ computes the spatial average only over non-zero mask positions, yielding a global foreground prototype vector.

With the prototype P established, we proceed to feature activation to capture the most category-relevant features. As shown in Figure 4, given a class prototype P, we activate category-related feature regions to enhance foreground-background discriminability. First, we compute the similarity map between $F_s$ and P:

$$\mathrm{sim}\text{-}\mathrm{map}(x,y)={\displaystyle \frac{F(x,y)·P}{\left|\right|F(x,y)\left|\right|·\left|\right|P\left|\right|}}$$

(3)

where $F(x,y)\in {\mathbb{R}}^c$ is the feature vector at spatial location $(x,y)$, · denotes the vector dot product, and $\left|\right|·\left|\right|$ denotes the $L_2$ norm. Therefore, $\mathrm{sim}\text{-}\mathrm{map}(x,y)\in [-1,1]$ measures cosine similarity between the local features and the class prototype.

Using this similarity activation matrix, category-enhanced intermediate features are obtained by

$$F^{\mathrm{mid}}=F\odot \mathrm{sim-map}$$

(4)

where sim-map is broadcast along the channel dimension before element-wise multiplication. Note that both support and query features are activated with the same prototype in an episode; the equations above use support features for notation simplicity.

We emphasize that both query features $F_q$ and support features $F_s$ undergo the same activation process using the identical prototype P. The previous description used $F_s$ as an illustrative example.

Following the aforementioned processing, we obtain the initially activated features $F^{\mathrm{mid}}$. However, the model’s capacity for encoding task-specific information remains insufficient at this stage. Additionally, a distribution discrepancy exists between $F^{\mathrm{mid}}$ and the original features F, and this discrepancy amplifies with increasing network depth. To address these challenges, our adapter module incorporates a low-dimensional learnable component that indirectly adjusts low-dimensional information within the backbone network while filtering high-dimensional noise. Given the limited training samples available, this module contains substantially fewer parameters compared with the entire model, thereby mitigating overfitting risks.

Specifically, the module consists of a down-projection linear layer (parameters $W_{\mathrm{down}}\in {\mathbb{R}}^{d\times r}$) for feature dimension compression and an up-projection linear layer (parameters $W_{\mathrm{up}}\in {\mathbb{R}}^{r\times d}$) for feature dimension restoration, where r denotes the hidden dimension:

$$F^{*}=\mathrm{ReLU}(F^{\mathrm{mid}}·W_{\mathrm{down}})·W_{\mathrm{up}}$$

(5)

3.3. Adapter-Based Fine-Tuning for Training Phase

Although the paradigm of freezing pretrained backbone models has achieved considerable success, directly applying parameters pretrained for image classification to few-shot image segmentation fails to adequately address the prior bias problem.

Our algorithm addresses this limitation by inserting Feature Activation Adapter modules into the pretrained backbone network and fine-tuning a small subset of parameters within these adapters. This approach prevents overfitting while enabling the model to achieve better generalization when learning new tasks and to more accurately capture correlations between the support and query features. As illustrated in Figure 5, our method fine-tunes the parameters of the Feature Activation Adapter module while keeping the original backbone network parameters frozen. This design, referred to as adapter fine-tuning, serves two primary functions: preserving rich semantic cues from the pretrained backbone and facilitating learning for novel class segmentation tasks within the adapter, where support set information guides the segmentation of query targets.

Figure 5. Training phase framework diagram.

Following established research paradigms, we employ multi-scale attention pooling to extract prototypes from support features after the backbone network. We then concatenate the obtained prototypes with query features and finally generate prediction results through a decoder. The MAP process is formulated as follows:

$$\mathrm{Prototype}=\mathrm{Mean}(F_s\odot M_s)$$

(6)

where ⊙ denotes spatial multiplication and Mean represents average pooling. This equation indicates that we compute the average of the feature vectors only at non-zero positions of $M_s$. Subsequently, we expand the prototype to match the dimensions of the query features, facilitating subsequent concatenation operations.

During the training phase of few-shot segmentation, we utilize a substantial number of episodes as training data. Therefore, we also train the decoder parameters in the downstream network to adapt to the image features extracted by the fine-tuned backbone network.

3.4. Test-Phase Fine-Tuning via Data Augmentation

In conventional few-shot segmentation frameworks, models typically perform direct inference with frozen parameters during the testing phase without undergoing fine-tuning. However, this approach often yields suboptimal performance due to domain discrepancies between novel classes and base classes, combined with intra-class diversity. This constitutes the fundamental novel class adaptation challenge in few-shot segmentation.

To address this limitation, we propose a test-phase adaptive fine-tuning strategy based on support set data augmentation. This approach enables online optimization of model parameters through the generation of pseudo-query samples while strictly adhering to the task independence assumption of few-shot learning, thereby eliminating data leakage risks.

During the testing phase, each episode comprises randomly sampled support and query sets. To prevent cross-task information leakage, the model must complete training and inference independently within each task, prohibiting cross-task parameter updates or data reuse. Furthermore, test-phase fine-tuning faces an additional challenge: under extremely scarce data conditions, query set labels remain strictly unavailable. While some existing methods attempt to generate query pseudo-labels based on prediction confidence from current algorithms, this approach suffers from a critical limitation where fine-tuning outcomes becomes heavily dependent on the pseudo-label quality.

To overcome this constraint, our algorithm introduces a data augmentation-based pseudo-query sample generation strategy during testing. We apply data augmentation techniques to image-label pairs from the support set, thereby creating supervisory signals for the fine-tuning process.

As illustrated in Figure 6, our proposed test-phase fine-tuning strategy comprises two main components. First, we perform in-episode online fine-tuning using the augmented pseudo-query set. Second, we utilize the fine-tuned model to generate predictions for genuine query images and subsequently compute segmentation metrics.

Figure 6. Test-phase fine-tuning framework diagram.

We base our fine-tuning strategy design on the assumption that the two branches of the few-shot image segmentation algorithm exhibit certain symmetry. Specifically, we hypothesize that the neural network demonstrates consistent prediction patterns for images belonging to the same category. This theoretical foundation enables us to utilize the support set to generate labeled pseudo-query sets for fine-tuning purposes (see Algorithm 1).

Algorithm 1 Test-phase fine-tuning procedure

Require:   Support image $I_s$, support mask $M_s$, query image $I_q$, fine-tuning iterations N Ensure:   Segmentation mask $M_q$ for query image $I_q$, and compute segmentation metric IoU   1: for each episode $(I_s,I_q,M_s,M_q)$ do   2:    Step 1: Generate pseudo-query image $I_q^{*}$ and its label mask $M_q^{*}$ from $I_s$ and $M_s$ using data augmentation methods   3:    for $i=1$ to N do   4:      Predict segmentation mask $M_q^{\prime *}$ for pseudo-query image $I_q^{*}$ using test-phase fine-tuned model   5:      Compute cross-entropy loss ${\mathcal{L}}_{ce}$ between prediction $M_q^{\prime *}$ and pseudo-query label $M_q^{*}$   6:      Compute gradients and optimize trainable parameters using SGD optimizer   7:    end for   8:    Step 2: Predict segmentation mask $M_q$ for genuine query image using optimized model   9:    Compute segmentation metric IoU for this episode 10: end for

To validate the feasibility of this assumption, we visualize the prediction outcomes from both branches of the few-shot network. Figure 7 illustrates the segmentation results for both support and query images using the same model. The left three columns demonstrate that when mis-segmented regions appear in the support segmentation, similar segmentation errors manifest in the query segmentation. The right three columns reveal that under-segmented regions in the support segmentation correspond to incomplete segmentation areas in the query segmentation.

Figure 7. Comparative visualization of dual-branch outputs in few-shot segmentation model.

These findings provide empirical evidence supporting the existence of symmetry between the two branches of our algorithm. Consequently, we implement an indirect, supervised online fine-tuning approach during the testing phase that leverages the support set.

3.5. Loss Function

We employ the cross-entropy loss as the primary loss function for model training. The theoretical advantage of this loss function lies in its intrinsic relationship with the Kullback–Leibler divergence, enabling effective quantification of information discrepancy between the model’s predicted distribution and the true conditional distribution. For segmentation tasks involving C semantic categories, the mathematical formulation of the cross-entropy loss is expressed as follows:

$$L_{ce}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{c=1}^C}{y}_{i,c}log\left(p_{i,c}\right)$$

(7)

where N represents the total number of pixel samples, $y_{i,c}$ denotes the ground-truth label indicating whether the ith pixel belongs to category c, and $p_{i,c}$ represents the model’s estimated probability for that category. Through gradient backpropagation via the chain rule, this loss function drives iterative updates of network parameters toward the true distribution direction.

Compared with traditional loss functions such as the mean squared error, cross-entropy loss demonstrates superior gradient stability in class-imbalanced scenarios. Its penalty for incorrect predictions increases logarithmically with probability deviation, effectively preventing the model from converging to local optima. For pixel-level prediction tasks, this loss function achieves a balance between fine-grained supervision and global optimization by computing the loss values independently for each pixel and then averaging them globally.

Notably, the synergistic design with the Softmax activation function ensures the output probabilities satisfy the normalization constraints while maintaining smoothness of the loss landscape. This optimization paradigm based on entropy minimization essentially transforms the segmentation problem into a conditional probability density estimation process in the feature space, providing theoretical guarantees for supervised learning in few-shot semantic segmentation scenarios.

4. Experiments and Analysis

In this section, we evaluate the performance of the proposed BAFNet framework through comprehensive experiments and comparative analysis.

4.1. Set-Up

We begin by introducing the benchmark datasets and evaluation metrics employed in our experimental framework.

4.1.1. Datasets

Following established practices in the few-shot segmentation literature, we evaluated BAFNet’s performance on two standard benchmark datasets: ${\mathrm{PASCAL-5}}^i$ [49] and ${\mathrm{COCO-20}}^i$ [50]. ${\mathrm{PASCAL-5}}^i$ is constructed from PASCAL VOC 2012, with additional annotations from SDS. For cross-validation purposes, the 20 categories in ${\mathrm{PASCAL-5}}^i$ were divided into four folds, each containing 5 categories. ${\mathrm{COCO-20}}^i$, built upon MS COCO, presents a more challenging dataset with 80 categories similarly partitioned into four folds, each comprising 20 categories. For each fold in both datasets, we sampled 1000 support-query image pairs, following the methodologies of MSANet [18], BAM [11], and PFENet [15] for fine-tuning and validation.

Furthermore, to verify our method’s generalization capability for industrial defect segmentation, we conducted extended few-shot defect segmentation experiments on the Magnetic Tile Defect Dataset [51]. This meticulously constructed, well-annotated industrial defect detection benchmark dataset simulates real assembly line manufacturing processes by capturing images of given magnetic tiles under various lighting conditions. The dataset contains 1344 images with cropped regions of interest for magnetic tile defects. It encompasses four defect categories: blowhole, crack, fray, and uneven. Each defect image has corresponding pixel-level annotations in the form of binary segmentation masks, where defective pixels are labeled white (255) and background regions (normal areas) are labeled black (0), as illustrated in Figure 8.

Figure 8. Examples of defect images and masks in MT dataset.

We partitioned the dataset into three folds according to sample distribution, with the specific category assignments for query and support sets detailed in Table 2.

Table 2. Category settings for query and support in three folds of MT Defect Dataset.

Class	Support	Query
Fold0	Crack, Blowhole, Fray, Uneven	Break
Fold1	Blowhole, Break, Fray	Crack, Uneven
Fold2	Blowhole, Crack, Break, Uneven	Fray

4.1.2. Evaluation Metrics

Following the established paradigm in few-shot semantic segmentation, where each training and testing episode contains only one target category, we treated all pixels belonging to the target as the foreground and the remaining pixels as the background. Consequently, we employed the mean intersection over union (mIoU) and foreground-background intersection over union (FB-IoU) as the most appropriate performance evaluation metrics.

The intersection over union (IoU) calculates the ratio between the intersection and union of the predicted and ground-truth regions for a specific category. In few-shot segmentation tasks, we primarily compute the IoU for foreground categories. The segmentation IoU for a single image can be derived from four essential elements in the prediction results, expressed by

$$\mathrm{IoU}={\displaystyle \frac{\mathrm{Interaction}}{\mathrm{Union}}}={\displaystyle \frac{\mathrm{TP}}{\mathrm{FN}+\mathrm{FP}+\mathrm{TP}}}$$

(8)

where true positive (TP) represents pixel regions correctly predicted as the foreground, reflecting the model’s precision in capturing target features; true negative (TN) represents dual correct identification of background regions, demonstrating the model’s effectiveness in suppressing background information; false positive (FP) represents pixels incorrectly predicted as the foreground; and false negative (FN) represents foreground pixels that remain unsegmented. The mean intersection over union (mIoU) represents the average of the IoU results across all foreground categories. Values approaching one indicate superior segmentation performance, calculated as follows:

$$\mathrm{mIoU}={\displaystyle \frac{1}{n+1}}{\displaystyle \sum _{i=0}^n}{\mathrm{IoU}}_i$$

(9)

where n denotes the total number of categories in the fold.

The foreground-background intersection over union (FB-IoU) computes the average between the foreground IoU and background IoU. Similarly, values closer to one indicate stronger foreground-background discrimination capability and prediction masks more closely aligned with the ground-truth masks. The calculation formula is

$$\mathrm{FB-IoU}={\displaystyle \frac{1}{2}}({\mathrm{IoU}}_{fg}+{\mathrm{IoU}}_{bg})$$

(10)

where ${\mathrm{IoU}}_{fg}$ and ${\mathrm{IoU}}_{bg}$ represent the foreground and background IoU results, respectively.

While the mIoU provides a global evaluation considering segmentation results across different foreground categories and demonstrates robust performance against class imbalance, the FB-IoU only distinguishes between the foreground and background without accounting for inter-class variations. Moreover, since background pixels typically dominate the majority, even classifying all pixels as background pixels can yield favorable FB-IoU results. Therefore, we primarily adopted the mIoU as the main performance metric for few-shot segmentation tasks, with the FB-IoU serving as a secondary evaluation indicator for comparing different algorithms.

4.1.3. Implementation Details

We conducted our experiments using the PyTorch deep learning framework with the Python 3.8 interpreter. The computational infrastructure employed a server platform comprising 4 NVIDIA RTX 3090 GPUs for parallel computing, an i9-12900KF CPU, and 64 GB of DDR5 memory, ensuring efficient and stable large-scale tensor operations. To ensure comparability with state-of-the-art mainstream algorithms, we adopted VGG-16 and ResNet-50 as backbone network benchmarks, both initialized with ImageNet pretrained weights and maintained through transfer learning strategies that froze the pretrained parameters to preserve generalization advantages in feature representation.

For the training strategy, we employed the stochastic gradient descent optimizer for parameter space exploration, with the momentum coefficient set to 0.9 and weight decay coefficient configured to be 0.0001. We executed 200 training epochs for the PASCAL-5ⁱ and MT Defect Dataset, while implementing a 50-epoch training strategy for COCO-20ⁱ due to its larger data scale. The batch size was set to 8 during training, while the single-sample inference mode was adopted during testing to simulate real-world few-shot segmentation requirements. All experimental procedures were modularly implemented through PyCharm IDE.

To optimize the learning rate adjustment throughout training, we integrated two distinct strategies: linear warm-up and polynomial decay. Polynomial decay reduces the learning rate by multiplying by $(1-{\displaystyle \frac{{curren{t}_{iter}}^{power}}{ma{x}_{iter}}})$, where the power equals 0.9. During the initial training phase (first half of the first epoch), the learning rate linearly increased from $bas{e}_{lr}\times 0.1$ to $bas{e}_{lr}$, where $bas{e}_{lr}$ represents the initial learning rate (set to 0.05). Subsequently, the learning rate adjustment strategy switched to polynomial decay, gradually decreasing from the initial learning rate toward zero.

For the test fine-tuning phase, we set the learning rate to $1.25\times {10}^{-4}$ to emulate the learning rate during later training stages. The data augmentation methods employed during testing primarily included random rotation, Gaussian blur, and horizontal flipping.

To improve reproducibility and provide a clearer summary of the optimization configuration, we additionally report the key hyperparameter settings in Table 3.

Table 3. Hyperparameter settings used in this work.

Item	Value/Strategy
Optimizer	SGD
Momentum	0.9
Weight decay	$1\times {10}^{-4}$
Initial learning rate ($bas{e}_{lr}$)	0.05
LR warm-up	Linear, from $0.1\times bas{e}_{lr}$ to $bas{e}_{lr}$
Warm-up duration	First half of the first epoch
LR decay	Polynomial decay ($power=0.9$)
Meta-train epochs (PASCAL-5i)	200
Meta-train epochs (COCO-20i)	50
Meta-train epochs (MT)	200
Batch size (train)	8
Inference mode (test)	Single-sample inference
Test-time fine-tuning learning rate	$1.25\times {10}^{-4}$
Test-time augmentation	Random rotation, Gaussian blur, horizontal flipping

4.2. Comparative Analysis with State-of-the-Art Methods

Table 4 presents the segmentation performance of our proposed BAFNet model on the PASCAL-5ⁱ dataset. Our approach builds upon the BAM framework by incorporating multi-stage adaptive fine-tuning, and we compare its performance against other advanced methods. In the table, boldface indicates the best performance, while underlining denotes the second-best performance.

Table 4. Quantitative experimental mIoU (%) results on the ${\mathrm{PASCAL-5}}^i$ dataset.

Backbone	Method	1-Shot					5-Shot
		Fold0	Fold1	Fold2	Fold3	mIoU	Fold0	Fold1	Fold2	Fold3	mIoU
VGG16	DPCN [43]	58.90	65.70	59.60	54.00	59.70	64.90	69.00	64.10	58.60	64.10
	BAM [11]	63.18	70.77	66.14	57.53	64.41	67.36	73.05	70.61	64.00	68.76
	MCE [44]	65.90	69.50	65.10	56.30	62.87	65.63	72.82	69.67	64.70	68.21
	HDMNet [45]	64.80	71.40	67.70	56.40	65.10	68.10	73.10	71.80	64.00	69.30
	Ours	65.13	71.61	67.53	58.45	65.68	69.44	73.92	72.16	65.03	70.14
ResNet50	HSNet [47]	64.30	70.70	60.30	60.50	64.00	70.30	73.20	67.40	67.10	69.90
	SAGNN [46]	64.70	69.60	57.00	57.20	62.10	64.90	70.00	57.00	59.30	62.80
	BAM [11]	68.97	73.59	67.55	61.13	67.81	70.59	75.05	70.79	67.20	70.91
	MCE [44]	65.30	71.22	66.17	61.04	65.93	69.23	73.68	70.45	66.76	70.03
	SCCAN [48]	70.90	73.90	66.80	61.70	68.30	73.10	76.40	70.30	66.10	71.50
	RiFeNet [52]	68.40	73.50	67.10	59.40	67.10	70.00	74.70	69.40	64.20	69.60
	HDMNet [45]	71.00	75.40	68.90	62.10	69.40	71.30	76.20	71.30	68.50	71.80
	Ours	71.13	75.21	68.95	62.15	69.36	72.76	76.74	71.65	68.28	72.36

Table 4 shows consistent gains of BAFNet across both backbones and shot settings. With VGG16, BAFNet improved over BAM by +1.27% (one-shot) and +1.38% (five-shot), and remained competitive with or superior to HDMNet (+0.58% in one-shot and +0.84% in five-shot). With ResNet50, BAFNet outperformed BAM by +1.55% (one-shot) and +1.45% (five-shot) and achieved comparable one-shot performance to HDMNet while surpassing it by +0.56% in five-shot performance. These results confirm that, with minimal additional trainable parameters, adapter-based fine-tuning improved support-guided query segmentation in a stable and backbone-agnostic manner.

Table 5 presents the mIoU metrics of our proposed BAFNet model on the COCO-20ⁱ dataset. When employing the VGG16 backbone network, BAFNet achieved a 2.02% mIoU improvement over the BAM baseline in the one-shot setting. In the five-shot scenario, our method demonstrated performance gains ranging from 0.02% to 10.69% (specifically, +0.02% over SANet and +2.05% over BAM). These results indicate that our algorithm provided significant improvements compared with the baseline model under the VGG16 configuration, while achieving comparable performance to advanced methods in the five-shot setting.

Table 5. Quantitative experimental mIoU (%) results on the ${\mathrm{COCO-20}}^i$ dataset.

Backbone	Method	1-Shot					5-Shot
		Fold0	Fold1	Fold2	Fold3	mIoU	Fold0	Fold1	Fold2	Fold3	mIoU
VGG16	SAGNN [46]	35.00	40.50	37.60	36.00	37.30	37.20	45.20	40.40	40.00	40.70
	BAM [11]	38.96	47.04	46.41	41.57	43.50	47.02	52.62	48.59	49.11	49.34
	HPA [53]	40.00	44.19	42.16	41.96	42.08	46.22	51.36	48.61	46.30	48.12
	SANet [54]	41.13	50.80	47.19	44.62	45.94	47.22	57.35	51.56	49.35	51.37
	Ours	41.10	49.01	48.37	43.58	45.52	49.41	54.53	50.41	51.19	51.39
ResNet50	HSNet [47]	36.30	43.10	38.70	38.70	39.20	43.30	51.30	48.20	45.00	46.90
	DPCN [43]	42.00	47.00	43.20	39.70	43.00	46.00	54.90	50.80	47.40	49.80
	BAM [11]	43.41	50.59	47.49	43.42	46.23	49.26	54.20	51.63	49.55	51.16
	FECANet [55]	38.50	44.60	42.60	40.70	41.60	44.60	51.50	48.40	45.80	47.60
	RiFeNet [52]	39.10	47.20	44.60	45.40	44.10	44.30	52.40	49.30	48.40	48.60
	DCPNet [21]	43.00	48.60	45.40	44.80	45.50	47.00	54.70	51.70	50.00	50.90
	MIANet [56]	42.49	52.95	47.77	47.42	47.66	45.84	58.18	51.29	51.90	51.65
	Ours	45.39	52.21	48.74	45.65	48.00	51.43	55.77	53.17	51.23	52.90

When utilizing the ResNet50 backbone network, BAFNet showed mIoU improvements between 0.58% and 8.8% in the one-shot setting (specifically, +0.58% over MIANet and +1.77% over BAM), demonstrating noticeable enhancement compared with the state-of-the-art MIANet method. In the five-shot scenario, BAFNet achieved performance gains ranging from 1.25% to 6% (specifically, +1.25% over MIANet and +1.74% over BAM), indicating substantial improvements over both the advanced MIANet algorithm and the baseline BAM method.

Table 6 presents the mIoU metrics of our proposed BAFNet model on the MT dataset. When employing the VGG16 backbone network, BAFNet achieved a 4.69% mIoU improvement over the BAM baseline in the one-shot setting. In the five-shot scenario, our method demonstrated a 4.82% performance gain. These results indicate that our algorithm provided significant improvements compared with the baseline method under the VGG16 configuration.

Table 6. Quantitative experimental mIoU (%) results on the MT dataset.

Backbone	Method	1-Shot				5-Shot
		Fold0	Fold1	Fold2	mIoU	Fold0	Fold1	Fold2	mIoU
VGG16	PFENet [57]	42.13	46.84	43.22	44.06	45.83	54.92	50.87	47.23
	BAM [11]	47.19	50.89	45.13	47.74	48.92	52.64	47.62	49.73
	Ours	52.51	54.93	49.85	52.43	54.11	57.42	52.11	54.55
ResNet50	PFENet [57]	45.21	49.23	46.63	47.02	47.53	52.51	48.76	49.53
	BAM [11]	51.11	54.47	48.65	51.41	53.73	56.32	50.44	53.36
	Ours	55.98	59.03	53.45	56.15	59.12	62.31	55.22	58.88

When utilizing the ResNet50 backbone network, BAFNet showed mIoU improvements ranging from 4.74% to 9.13% in the one-shot setting. In the five-shot scenario, our method achieved performance gains between 5.52% and 9.35%. These experimental outcomes demonstrate that our approach maintained excellent adaptability for industrial defect detection and segmentation tasks.

Table 4, Table 5 and Table 6 demonstrate that our BAFNet model achieved excellent segmentation performance across all three datasets, employing both VGG16 and ResNet50 backbone networks under both the one-shot and five-shot settings. Although the COCO-20ⁱ dataset presented greater challenges due to its more complex images and required the adapter to learn more intricate information, our proposed BAFNet model maintained robust and stable performance in few-shot semantic segmentation tasks. Compared with the baseline BAM model, we observed significant improvements while also achieving enhanced mIoU metrics relative to advanced models including MIANet and SANet. Our BAFNet model demonstrated outstanding segmentation capability in both complex scenarios and industrial defect segmentation applications.

Table 7 presents the comparative experimental results of FB-IoU performance for our BAFNet model on the PASCAL-5ⁱ, COCO-20ⁱ, and MT datasets. Our proposed method demonstrated substantial advantages compared with current mainstream few-shot segmentation approaches. On the PASCAL-5ⁱ dataset with the VGG16 backbone, BAFNet achieved FB-IoU improvements of 0.41–4.18% in the one-shot setting (specifically, +0.41% over EFTNet and +1.43% over BAM), with performance gains of 0.38–4.17% in the five-shot scenario (specifically, +0.38% over EFTNet and +1.27% over BAM). When utilizing the ResNet50 backbone, we observed FB-IoU improvements of 0.29–6.76% in the one-shot experiments (specifically, +0.29% over EFTNet and +1.56% over BAM) while maintaining performance advantages of 0.27–5.53% under five-shot conditions (specifically, +0.27% over EFTNet and +1.55% over BAM).

Table 7. FB-IoU performance comparison experiment.

Dataset	Backbone	Method	FB-IoU
			1-Shot	5-Shot
${\mathrm{PASCAL-5}}^i$	VGG16	BAM [11]	77.26	81.10
		MCE [44]	74.51	78.20
		EFTNet [58]	78.28	81.99
		Ours	78.69	82.37
	ResNet50	BAM [11]	79.71	82.18
		MCE [44]	74.51	78.20
		EFTNet [58]	80.98	83.46
		Ours	81.27	83.73
${\mathrm{COCO-20}}^i$	VGG16	EFTNet [58]	72.05	74.71
		Ours	72.32	75.24
	ResNet50	EFTNet [58]	73.30	76.38
		Ours	73.56	77.89
MT	VGG16	PFENet [57]	71.03	72.82
		BAM [32]	72.73	73.62
		Ours	75.27	77.45
	ResNet50	PFENet [57]	73.24	74.32
		BAM [11]	76.72	77.37
		Ours	78.76	79.65

On the COCO-20ⁱ dataset with the VGG16 backbone, our algorithm achieved comparable performance to the advanced methods in both the one-shot and five-shot settings. With the ResNet50 backbone, BAFNet demonstrated a comparable FB-IoU to EFTNet in the one-shot setting while achieving a 1.51% improvement over EFTNet in the five-shot condition. For the MT dataset with both the VGG16 and ResNet50 backbones, our method showed significant performance enhancements compared with the baseline methods in both the one-shot and five-shot scenarios.

Using the FB-IoU as an evaluation metric in Table 7 provides supplementary assessment of the model generalization capability. The BAFNet model, which incorporates Feature Activation Adapter modules and implements multi-stage fine-tuning strategies based on the baseline BAM framework, achieved state-of-the-art results under the FB-IoU evaluation metric. This validates the effectiveness of our proposed method and its constituent modules. Across all three datasets, our BAFNet model demonstrated superior segmentation performance.

4.3. Learning Curve Analysis on PASCAL and COCO

To provide a clearer view of the optimization behavior, we visualized the training and validation curves on both PASCAL-5ⁱ and COCO-20ⁱ, including the PASCAL-loss, PASCAL-mIoU, COCO-loss, and COCO-mIoU, as shown in Figure 9.

Figure 9. Training and validation learning curves on PASCAL-5ⁱ and COCO-20ⁱ, including PASCAL-loss, PASCAL-mIoU, COCO-loss, and COCO-mIoU.

Based on the loss curves, both datasets showed a clear downward trend as training proceeded, indicating that the optimization process converged stably. The validation loss remained generally higher than the training loss, which is expected in few-shot settings due to the class split between meta-train and meta-test episodes. Importantly, no persistent divergence was observed, suggesting that the proposed adapter-based strategy maintained good optimization stability while avoiding severe overfitting.

Based on the mIoU curves, both the training and validation mIoU increased progressively and then entered a mild plateau stage, which is consistent with normal convergence behavior. Compared with PASCAL-5ⁱ, COCO-20ⁱ exhibited relatively larger fluctuations during optimization, reflecting its higher scene complexity and broader intra-class variations. Nevertheless, the overall upward trend of the COCO-mIoU validates that BAFNet can still learn transferable support-query matching patterns under more challenging conditions.

Overall, these four curves provide complementary evidence that BAFNet achieved stable optimization dynamics, a controlled generalization gap, and consistent performance gains across datasets of different difficulty levels.

4.4. Qualitative Results and Visualization

We now present qualitative comparisons between BAFNet and other baseline approaches through visual analysis of the segmentation results.

4.4.1. Experimental Visualization Across Multiple Episodes

To visually demonstrate the effectiveness of our method, we conducted visualization experiments across multiple episodes on testing categories. Figure 10 presents the corresponding segmentation masks for the target categories generated by different approaches. In addition to comparing the segmentation results with the baseline methods, we also evaluated the outcomes after test-phase fine-tuning to comprehensively assess the effectiveness of our proposed model.

Figure 10. Visualization of qualitative experimental results.

As shown in Figure 10, the first row displays support images with their ground-truth masks highlighted in blue. The second row shows query images with their corresponding ground-truth masks highlighted in green. The third row presents the segmentation results from the baseline method. The fourth and fifth rows display our method’s segmentation results before and after test-phase fine-tuning, respectively, highlighted in red.

In the first column, the bottle category faced significant interference from the monitor displayed in the query image. Other methods clearly succumbed to this interference, failing to correctly capture the bottle in the corner while mistakenly focusing on the image shown on the monitor. Our method after training-phase fine-tuning successfully segmented the bottle in the query image, though it incorrectly segmented the bottle displayed on the monitor as foreground. Subsequent test-phase fine-tuning enabled the model to exclude interference from the monitor and correctly segment the target.

In the fourth column, the cow category query image contained distracting objects in the background. The baseline method in the third row was clearly affected by this interference, mistakenly segmenting the distracting objects as foreground. Our method after training-phase fine-tuning only optimized the segmentation edges while still being affected by similar objects. Finally, test-phase fine-tuning enhanced the model’s recognition capability for the cow category, successfully eliminating interference from distracting objects and achieving correct segmentation results.

Overall, our method demonstrates that the first stage effectively enhances the model’s ability to capture support-query similarities compared with the baseline model, while the second stage successfully focuses on novel categories, exhibiting superior generalization capability to new classes.

4.4.2. Feature Map Activation Visualization

To demonstrate the effectiveness of our proposed method, we conducted feature map visualization experiments in this study. As shown in Figure 11, the three rows in the sequence represent the support set images, baseline model, and our proposed method, while the four columns display the visualization results for different category feature maps. The entire backbone network comprised five layers with multiple downsampling operations, making the visualization results of the deep-layer feature maps less interpretable. Therefore, we selected the output features from the second layer for visualization.

Figure 11. Visualization of activation feature maps.

Our experimental results indicate that, compared with the feature maps generated by the baseline model, the feature maps processed through our feature activation method demonstrated significant activation in the target foreground regions. This enhanced activation enabled subsequent networks to focus more effectively on the target category features during feature extraction.

4.4.3. Query Prototype t-SNE Visualization

We validated the effectiveness of our proposed method through prototype space visualization analysis. Using t-SNE technology to perform dimensionality reduction and visualization mapping of query prototypes from the decoder output, we systematically compared the differences in the prototype distribution characteristics between the baseline model and our improved approach.

Figure 12 presents the comparative results, where the three columns represent the backbone network, baseline model, and our method, while the four rows correspond to the four folds from Fold0 to Fold3, with each fold containing five categories of foreground targets for segmentation.

Figure 12. Visualization of t-SNE results for query prototypes.

The experimental results demonstrate that, compared with the baseline model, prototypes of the same category became more compact in our approach. Our method generated more spatially discriminative prototypes that could more clearly distinguish between prototypes of the same category, as evidenced by the red and blue points in Fold2 and the red and green points in Fold3.

The similarity-based Feature Activation Adapter mechanism we constructed improved the structural characteristics of the prototype space; inter-class feature distances were significantly expanded, while intra-class distributions became more compact, forming well-bounded clustering effects. Compared with the inter-class overlap and intra-class dispersion observed in the baseline model, our improved solution achieved prototype representation reconstruction and discriminative enhancement through the synergistic action of the similarity metric module and low-dimensional adaptive learning module.

This optimization effect demonstrated consistency across multiple test folds, and this compact reconstruction of the prototype space effectively enhanced the segmentation performance of the few-shot segmentation model.

4.4.4. Heat Map Visualization

We employed gradient-weighted class activation mapping (Grad-CAM) [59] to visualize the attention regions of our model. By integrating gradient information of the target categories with feature maps from convolutional layers, we generated heat maps that highlighted the most critical regions for classification decisions. As shown in Figure 13, the left three columns correspond to class activation maps for natural and indoor scene images, while the right three columns display activation maps for three industrial defect images. Our observations demonstrate that the proposed method achieved superior class activation focus on the target segmentation regions across all scenarios.

Figure 13. Visual comparison of CAM attention maps for natural scenes, indoor scenes, and industrial defect images.

As evidenced by the visualization results in Figure 13, the proposed two-stage fine-tuning strategy with adapters enabled our method to not only capture target objects from unseen categories more accurately while significantly mitigating the negative impact of irrelevant background information but also successfully identify target instances when the baseline model failed to focus on critical regions. These capabilities allowed our approach to maintain better focus on novel category targets, thereby enhancing the few-shot semantic segmentation performance metrics.

5. Discussion

In this section, we discuss the effectiveness of BAFNet through comprehensive analysis.

5.1. Ablation Study

We designed systematic ablation experiments to quantitatively evaluate the contributions of key modules in BAFNet, specifically focusing on meta-training phase fine-tuning and meta-testing phase fine-tuning. Our experiments were conducted under the one-shot setting on the PASCAL-5ⁱ dataset. Table 8 presents the IoU metrics for each fold and the final mIoU. The complete model (after multi-stage fine-tuning) achieved an mIoU of 65.68%, representing a 1.27% improvement over the baseline model (BAM model before fine-tuning). The training phase backbone fine-tuning strategy alone contributed a 0.87% performance gain (64.41% → 65.28%), while the testing phase fine-tuning provided an additional 0.4% improvement (65.28% → 65.68%).

Table 8. Key module ablation study results.

Training FT	Testing FT	Fold0	Fold1	Fold2	Fold3	mIoU
–	–	63.18	70.77	66.14	57.53	64.41
✓	–	64.27	71.45	67.15	58.26	65.28
✓	✓	65.13	71.61	67.53	58.45	65.68

5.2. Analysis of Learnable Parameters in Feature Adapter

To investigate the impact of internal layer dimensions in the Feature Activation Adapter module on algorithm performance, we conducted ablation experiments under the one-shot learning paradigm using controlled variable methods. We set the hidden layer dimension of the learning module to r and performed experiments with different values of r, using the mIoU as the quantitative evaluation metric. As shown in Figure 14, the model achieved peak performance of 65.12% when $r=c/8$, where c represents the channel number of activated features in the adapter, demonstrating the effectiveness of the feature enhancement module. The performance curve exhibited non-monotonic characteristics. When $r<c/8$, excessively small dimensions led to insufficient model expressiveness, while when $r>c/8$, each additional single layer caused performance degradation due to overfitting and catastrophic forgetting of the risks associated with excessively large fine-tuning feature dimensions.

Figure 14. Adapter key parameter analysis results.

5.3. Analysis of Test-Phase Fine-Tuning Parameters

To examine the impact of the test-phase fine-tuning parameters on algorithm performance, we conducted experiments under the one-shot setting, investigating the effects of fine-tuning iterations and the number of pseudo-query sets. As illustrated in Figure 15, excessive fine-tuning iterations led to significant performance degradation, as overly scarce fine-tuning samples within episodes increased the likelihood of overfitting. Similarly, while different augmentation strategies provided some variation, generating excessive numbers of pseudo-query sets still carried overfitting risks. Our experiments achieved optimal performance of 65.68% with five fine-tuning iterations and two augmented pseudo-query images. Excessive fine-tuning yielded negative returns, increasing the time consumption per iteration while decreasing the segmentation metrics.

Figure 15. Test-phase fine-tuning strategy parameter ablation study.

6. Analysis of Trainable Parameters

We conducted a comprehensive comparison between full fine-tuning and adapter-based fine-tuning on the BAM framework, with experiments designed under the one-shot setting on the PASCAL-5ⁱ dataset. The comparison thoroughly examined both the number of trainable parameters and the corresponding performance, as detailed in Table 9.

Table 9. Comparison of trainable parameters and one-shot segmentation metrics.

Method	Params	Fold0	Fold1	Fold2	Fold3	mIoU
BAM+fine-tuning	19.62 M	61.62	69.95	65.34	55.49	63.10
BAM [11]	4.42 M	63.18	70.77	66.14	57.53	64.41
HDMNet [45]	8.72 M	64.80	71.40	67.70	56.40	65.10
Base+adapter (meta-train)	2.02 M	64.27	71.45	67.15	58.26	65.28
Base+adapter (train+test) (BAFNet)	0.21 M	65.13	71.61	67.53	58.45	65.68

The results demonstrate that our method achieved performance comparable to the advanced HDMNet approach during the meta-training phase while utilizing only a minimal number of parameters. After test-phase fine-tuning, the performance further improved by 0.4%. Full fine-tuning, while consuming substantial computational resources, introduced overfitting and catastrophic forgetting issues that compromised the effective original information in the backbone network, ultimately degrading segmentation performance.

Our proposed BAFNet model achieved state-of-the-art few-shot segmentation performance while maintaining the most efficient parameter utilization, effectively balancing parameter efficiency with high performance.

Computational Efficiency Analysis. To further substantiate the claims on efficiency, we additionally report the wall-clock runtime and GPU memory usage of representative settings on the BAM framework. As shown in Table 10, our adapter-based meta-training introduced only a marginal training-time overhead compared with BAM (13 h → 13.5 h) while achieving better segmentation performance. Moreover, the episode-level meta-test adaptation required only 1 h in total and used substantially lower GPU memory than full fine-tuning.

Table 10. Runtime and GPU memory usage comparison.

Method	Runtime	GPU Memory
BAM+fine-tuning	–	>12,000 MB
BAM	13 h	4900 MB
Base+adapter (meta-train)	13.5 h	6400 MB
Base+adapter (meta-test)	1 h	1500 MB

These results, together with the trainable-parameter comparison in Table 9, indicate that BAFNet provides a favorable trade-off among accuracy, parameter efficiency, and practical deployment cost.

Limitations and Future Directions. Despite the strong empirical performance, BAFNet still has several limitations. First, the method may be sensitive to noisy or weakly representative prototypes in challenging episodes. When the support foreground is ambiguous or contaminated by background interference, prototype quality can degrade and may weaken support–query matching reliability. One promising direction is to improve prototype representativeness by introducing more informative prototype construction strategies, such as cluster-based prototype aggregation and joint foreground–background prototype modeling.

Second, pseudo-query-based test-time fine-tuning may increase class shift risk in some cases. This risk mainly stems from the extremely limited query context and the current augmentation strategy, which may not always generate sufficiently diverse and semantically faithful pseudo-query samples. Future work will investigate stronger data generation mechanisms (e.g., diffusion-based augmentation) and more robust adaptation constraints to reduce overfitting and improve stability under distribution shift.

7. Conclusions

This paper systematically investigated the limitations of traditional few-shot semantic segmentation algorithms in fine-tuning pretrained backbone networks. To address these challenges, we proposed a novel few-shot image segmentation algorithm that incorporates parameter-efficient Feature Activation Adapter structures into the backbone network. This approach achieved performance improvements while introducing only minimal additional learnable parameters.

Our method not only adjusts adapter parameters during the meta-training phase to enhance the model’s understanding of support-query correlations and segmentation target recognition but also employs a data augmentation-based approach during meta-testing to generate pseudo-query sets. This enables supervised targeted fine-tuning for learning semantic information of novel categories.

Through extensive experiments on PASCAL-5ⁱ, COCO-20ⁱ, and industrial defect datasets, the results demonstrate that our BAFNet model achieved state-of-the-art segmentation performance while maintaining the most efficient parameter utilization. Compared with mainstream few-shot segmentation methods, our model exhibited superior capabilities in feature similarity learning and novel class generalization.

Ablation studies validated the effectiveness of each model component, confirming the importance of Feature Activation Adapter modules for enhanced feature understanding. Furthermore, experimental results indicate that thanks to the adapter fine-tuning characteristics, our algorithm generated more concentrated prototypes that effectively bridged the semantic relationship between support and query features, leading to more accurate query target segmentation. The proposed method demonstrated excellent few-shot segmentation performance for both general images in complex scenes and industrial defect segmentation scenarios.

From a broader perspective, the parameter-efficient and staged adaptation paradigm represented by BAFNet opens a promising direction for few-shot segmentation. Nevertheless, several limitations remain. The current data augmentation strategy is still relatively simple; the number of online test-time fine-tuning iterations and the learning rate are manually specified hyperparameters; and the FAA module design and insertion positions are relatively fixed. Future research should extend beyond incremental refinement of existing components by relaxing the assumptions of static images and closed-world settings, extending to multimodal inputs, sequential data, and open environments, and integrating more deeply with the rapidly evolving foundation-model ecosystem. We believe these directions can further advance few-shot learning toward more general, efficient, and robust real-world deployment.