Instance Segmentation in Autonomous Log Grasping Using EfficientViT-SAM MP-Former

Abstract

Segmenting individual timber logs in robotic grasping scenarios poses significant challenges due to cluttered arrangements, overlapping geometries, and visually uniform textures, requiring instance segmentation models that balance accuracy and computational efficiency. In this work, we study the integration of the EfficientViT-SAM backbone into the MP-Former framework to analyze its impact on segmentation accuracy, inference speed, and cross-dataset generalization in autonomous forestry applications. Our contributions are threefold: (1) we benchmark Mask2Former and MP-Former with different variants of Swin Transformer as backbones on the TimberSeg 1.0 dataset, (2) we study the use of the EfficientViT-SAM-XL architecture as an alternative encoder backbone to analyze its impact on inference speed and segmentation accuracy, and (3) we use an In-house dataset as a hold-out test set, comprising 113 images and 923 annotations in the annotated subset and 50 images in the unannotated subset, for evaluating model generalization under real-world deployment scenarios. On the TimberSeg 1.0 dataset, our top-performing model, EfficientViT-SAM-XL1 MP-Former, achieves an mAP of 61.05, outperforming the Swin-B Mask2Former of the TimberSeg 1.0 paper by +3.52 mAP, while running at 12 FPS (+3.53 FPS gain). When tested on our In-house dataset, the model attains an mAP of 67.06. Notably, it matches the memory efficiency of TimberSeg’s strongest baseline, despite having nearly double the number of parameters, demonstrating its practical viability for robotic applications in forestry environments.

1. Introduction

As forestry operations push toward autonomous machinery, there is a need for accurate and efficient log-handling workflows. Initial works [1,2] explored grasp prediction in cluttered scenes using deep learning. Ref. [3] presented a fully automated log-grasping system using a visual grasp detection network. Recent efforts [4,5,6] have worked on the instance segmentation of wood logs for autonomous loading. Ref. [7] presented a fully autonomous log-loading system using a Mask2Former-based perception stack trained on the TimberSeg 1.0 dataset [4], reporting that segmentation errors were the dominant cause of failed grasps in outdoor deployments, underscoring the need for more accurate and efficient instance segmentation backbones for forestry robotics. In this work, we focus on instance segmentation, as it enables precise delineation of individual logs, which is essential for downstream grasp planning in complex, overlapping log arrangements commonly encountered in autonomous forestry operations.

The release of the Segment Anything Model (SAM) [8] marked a major shift in segmentation research, offering a foundation model trained at scale for universal segmentation tasks. SAM’s encoder has since been repurposed via fine-tuning for diverse downstream tasks across domains such as medical imaging [9], remote sensing [10], and robotic grasping [11]. To make this capability accessible in real-world settings, efficient variants like EfficientSAM [12] and EfficientViT-SAM [13] have emerged, distilling SAM into faster, lightweight models. These models enable us to leverage SAM’s large-scale pretraining while reducing inference costs, offering a stronger alternative to conventional encoders pretrained on MS-COCO [14].

Several SAM variants, including SAM, EfficientViT-SAM, etc., have done evaluations on the TimberSeg 1.0 [4] dataset in a zero-shot setting, using prompts such as points or bounding boxes. While remarkable, zero-shot SAM applications require external inputs or detectors, introducing manual effort and added complexity. In contrast, we treat log segmentation as a conventional instance segmentation task, enabling us to directly fine-tune SAM-based encoders within a task-specific model. This eliminates the need for prompt engineering or auxiliary modules and allows the model to learn end-to-end from training data, making it more practical for real-world forestry automation systems. For this purpose, we extend [4]’s work by making incremental changes to their Swin-B Mask2Former [15] model. We began by surveying recent works on Mask2Former [15], which led us to the MP-Former [16] architecture. Then, we looked for efficient versions of SAM to use as an encoder instead of Swin Transformer [17], for which we chose EfficientViT-SAM. Our target of instance segmentation is only the top-lying logs that can be picked by a crane. As [5] is not publicly available and [6] has annotations of all logs, we only consider TimberSeg 1.0 [4] for this work along with our own In-house dataset as a hold-out test set.

The main contributions of this work are as follows:

• We benchmark Mask2Former and MP-Former with multiple Swin Transformer backbones on the TimberSeg 1.0 dataset for log instance segmentation, analyzing accuracy, recall, and inference speed in the context of autonomous log grasping.
• We integrate the EfficientViT-SAM-XL as an encoder within the MP-Former architecture for end-to-end fine-tuning. To enable compatibility with the MP-Former pixel decoder, we introduce lightweight upsampling layers to align EfficientViT-SAM feature resolutions and study transposed convolution and transposed CoordConv as alternative designs.
• We evaluate model generalization on a real-world In-house test set collected under operational deployment conditions.

2. Materials and Methods

2.1. Datasets Used

2.1.1. TimberSeg 1.0 Dataset

We use the TimberSeg 1.0 [4] dataset for training and validating all models. Ref. [4] employed a 5-fold cross-validation strategy, which we follow as well. Because the official fold assignments were not provided, we generated our own five splits using the same KFold function as in the official TimberSeg repository, implemented with scikit-learn and a fixed random seed of 42. Unlike the original code, where the folds are generated dynamically at each run, we precomputed the splits once and kept them fixed across all experiments to ensure full reproducibility. TimberSeg 1.0 contains 220 images and 2474 instances (26 instances were automatically filtered out by Detectron2 [18] for being too small). Given in

Table 1. Training and validation data distribution of TimberSeg 1.0 [4] across the folds used in our experiments.

Fold	Train Images	Train Instances	Val Images	Val Instances
Fold0	176	1919	44	555
Fold1	176	1970	44	504
Fold2	176	2005	44	469
Fold3	176	2026	44	448
Fold4	176	1976	44	498

is the data distribution of images and annotations across our 5-fold (train:val::80:20) splits.

The dataset was collected across four operational environments: forest, lumberyard, roadside and trailer. All images were recorded from the perspective of a log loader operator. Given in Figure 1 are some sample image-annotation pairs from the dataset.

2.1.2. In-House Dataset

To evaluate model performance under real deployment conditions, our project partners provided 113 images, which we manually annotated using the labelme [19] annotation tool, resulting in a total of 923 instance annotations. We refer to this as the ’In-house Annotated dataset’ and use it exclusively for testing to assess model generalization. Due to contractual agreements and data protection regulations, example images from this dataset cannot be publicly shown. In addition, we collected 50 unannotated images from [1,2,3] and from our project partners. This ’In-house Unannotated dataset’ is used for qualitative evaluation of model behavior under diverse viewpoints and lighting conditions. Representative samples that have been explicitly approved by the project partners for disclosure are shown in Figure 2. Although these images can be visualized in this paper, the dataset itself cannot be publicly released due to contractual agreements and data protection regulations.

2.2. Model Architecture Overview

Figure 3 shows our EfficientViT-SAM MP-Former architecture, which integrates an EfficientViT-SAM-XL [13] backbone into the MP-Former [16] segmentation framework. MP-Former improves upon Mask2Former [15] by resolving prediction inconsistencies across Transformer decoder layers via mask-piloted training. It introduces denoising queries based on noisy ground-truth masks to stabilize predictions, while standard matching queries use predicted masks and bipartite matching. This dual-query setup boosts segmentation accuracy and training speed without increasing inference-time computational cost.

2.2.1. Backbone

We use EfficientViT-SAM-XL [13] (XL0 and XL1) as the backbone, which builds on SAM [8] by replacing its ViT [20] encoder with EfficientViT [21]. It uses ReLU-based linear attention [22] to reduce complexity from quadratic to linear and incorporates MBConv [23] and Fused-MBConv [24] blocks. We exclude EfficientViT-SAM-L and experiment with only XL variants. For readability, we will refer to EfficientViT-SAM-XL0 as EVS-XL0 and EfficientViT-SAM-XL1 as EVS-XL1. EVS-XL consists of Stages 0–5 (Figure 4), where Stage 0 is Conv + ResBlock [25], Stage 1–3 have Fused-MBConv [24] and Stage 4–5 have MBConv [23] + EfficientViT module [21]. Linear attention based on ReLU is defined in Equation (1). Given an input $x\in {\mathbb{R}}^{N\times f}$, we compute the query, key, and value projections as $Q=x{W}_Q$, $K=x{W}_K$, and $V=x{W}_V$, where $W_Q,W_K,W_V\in {\mathbb{R}}^{f\times d}$ are learnable linear projection matrices. The i-th output feature $O_i$ can be written as:

$$\begin{array}{cc}\hfill O_i& ={\displaystyle \frac{ReLU\left(Q_i\right)\left({\sum }_{j=1}^{N}ReLU{\left(K_j\right)}^T{V}_j\right)}{ReLU\left(Q_i\right)\left({\sum }_{j=1}^{N}ReLU{\left(K_j\right)}^T\right)}}.\hfill \end{array}$$

(1)

As $\left({\sum }_{j=1}^N\mathrm{ReLU}{\left(K_j\right)}^T{V}_j\right)\in {\mathbb{R}}^{d\times d}$ and $\left({\sum }_{j=1}^N\mathrm{ReLU}{\left(K_j\right)}^T\right)\in {\mathbb{R}}^{d\times 1}$ are computed once and reused for each query, the overall attention operation requires only $\mathcal{O}\left(N\right)$ computational cost and $\mathcal{O}\left(N\right)$ memory.

Figure 3 illustrates the integration of the EVS-XL encoder (Figure 4) into the MP-Former segmentation framework. We extract features from Stages 2–5 (out of Stages 0–5, shown as color-coded blocks in the backbone) to feed into the Pixel Decoder. This stage selection mirrors the setup used for Swin Transformer backbones. However, Swin-B/S/T generates multi-scale feature maps at $(1/4,1/8,1/16,1/32)$ input resolution, while EVS-XL yields outputs at $(1/8,1/16,1/32,1/64)$. To ensure compatibility with MP-Former’s Pixel Decoder, we insert upsampling layers to match the expected resolutions of $(1/4,1/8,1/16,1/32)$ (as shown in the “Hierarchical encoder features” block in Figure 3). This ensures the Pixel Decoder configuration remains identical across Swin and EVS-XL variants. To assess upsampling strategies, we perform an ablation comparing transposed convolution [26] (EVS-XL-TC) and transposed CoordConv [27] (EVS-XL-TCC). CoordConv explicitly encodes spatial coordinates and has shown benefits for localization even in single-layer form [28]. This design allows us to evaluate whether improved spatial encoding during upsampling enhances performance in the Pixel Decoder and Transformer Decoder, particularly for instance segmentation involving densely packed timber logs.

2.2.2. Pixel Decoder

The Pixel Decoder receives the hierarchical feature representations produced by the backbone and progressively upsamples these low-resolution features to generate high-resolution, per-pixel embeddings. In Mask2Former, the authors conducted an ablation study comparing several decoder designs, including Feature Pyramid Network (FPN) [29], Semantic Feature Pyramid Network (Semantic FPN) [30], Feature-aligned Pyramid Network (FaPN) [31], Bi-directional Feature Pyramid Network (BiFPN) [32] and the Multi-Scale Deformable Attention Transformer (MSDeformAttn) [33]. Among these, MSDeformAttn achieved the best performance. Consequently, both our work and [4] adopt MSDeformAttn as the Pixel Decoder. Concretely, as shown in Figure 3, the decoder applies multiple MSDeformAttn layers to feature maps at resolutions of $1/8$, $1/16$, and $1/32$ of the input image, after which the final $1/8$ feature map is upsampled to produce $1/4$ resolution per-pixel embeddings.

2.2.3. Transformer Decoder

The Transformer Decoder [34] takes as input the per-pixel features generated by the Pixel Decoder and processes a set of learnable object queries to predict segmentation masks. Unlike conventional decoders that rely on standard cross-attention over the full image, Mask2Former employs masked cross-attention, which restricts each query’s attention to the predicted foreground regions rather than the entire feature map. This design improves both training efficiency and segmentation accuracy, especially for small objects and fine-grained structures. The decoder further integrates several architectural optimizations, including reordering the self-attention and masked cross-attention operations, using learnable queries that act as region-level proposals, and eliminating dropout to reduce computational overhead. Together, these modifications enhance both the efficiency and the predictive performance of the segmentation model. Final binary mask predictions are obtained by combining the learned object queries with the per-pixel embeddings.

Building upon this decoder design, MP-Former further refines Mask2Former by addressing the issue of inconsistent mask predictions across the successive Transformer Decoder layers, which can lead to inefficient use of object queries and degraded segmentation quality. To mitigate this, MP-Former introduces a mask-piloted (MP) training strategy that supplements the standard learnable queries with additional ground-truth (GT) guided denoising queries. During training, these MP queries employ noised GT masks within the masked cross-attention mechanism instead of relying on the predicted masks from the preceding layer. Through this process, the denoising queries learn to reconstruct clean GT masks and, in turn, implicitly guide the remaining queries toward more consistent predictions across layers. The original learnable queries are treated as matching queries, which use predicted masks for masked attention and are assigned to GT masks via bipartite matching for loss computation. In contrast, the MP queries are directly paired with their corresponding GT masks for supervision. This training paradigm significantly improves segmentation accuracy while also accelerating convergence, without introducing any additional computational cost during inference, thereby preserving the efficiency of the base Mask2Former framework.

2.3. Loss Function

Both Mask2Former and MP-Former rely on the same underlying loss formulation. The distinction is that MP-Former computes two loss branches: one tied to the mask-pilot module, where each prediction is directly paired with its corresponding ground-truth (GT) instance, and another from the standard matching stage, where predictions are associated with ground-truth masks via bipartite matching.

Mask2Former follows the point-sampling strategy introduced in PointRend [35] and Implicit PointRend [36]. For the matching stage, it randomly draws a shared set of K points from all predicted and GT masks to build the cost matrix used in bipartite matching. For the final loss computation, it applies importance sampling [35], selecting separate point sets for each predicted-GT pair. Consistent with the official implementation, we use K = 12,544, i.e., $112\times 112$ points. Restricting loss computation to these sampled points yields a roughly $3\times$ reduction in GPU memory compared to using full-resolution masks.

Auxiliary losses are applied to both the Transformer Decoder layers and the learnable query embeddings. The overall objective, ${\mathcal{L}}_{\mathrm{final}}$, combines a mask loss ${\mathcal{L}}_{\mathrm{mask}}$ and a classification loss ${\mathcal{L}}_{\mathrm{class}}$, with predefined weights. The mask loss itself is a weighted sum of the binary cross-entropy term ${\mathcal{L}}_{\mathrm{bce}}$ and the Dice loss ${\mathcal{L}}_{\mathrm{dice}}$ [37]. The full formulation of the final loss is shown below:

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{final}}& ={\mathcal{L}}_{\mathrm{mask}}+{\lambda }_{\mathrm{class}}{\mathcal{L}}_{\mathrm{class}},\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{final}}& ={\lambda }_{\mathrm{bce}}{\mathcal{L}}_{\mathrm{bce}}+{\lambda }_{\mathrm{dice}}{\mathcal{L}}_{\mathrm{dice}}+{\lambda }_{\mathrm{class}}{\mathcal{L}}_{\mathrm{class}}.\hfill \end{array}$$

(3)

We adopt the same weighting coefficients as the original implementations [4,15], setting ${\lambda }_{\mathrm{bce}}=5$ and ${\lambda }_{\mathrm{dice}}=5$ and ${\lambda }_{\mathrm{class}}=2$.

The binary cross-entropy component ${\mathcal{L}}_{\mathrm{bce}}$ is defined as follows:

$${\mathcal{L}}_{\mathrm{bce}}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left(y_i·log\left({\widehat{y}}_i\right)+(1-y_i)·log(1-{\widehat{y}}_i)\right)$$

(4)

where N denotes the number of pixels, $y_i$ is the GT label for pixel i (0 or 1), and ${\widehat{y}}_i$ is the predicted probability that pixel i belongs to class 1.

The Dice loss ${\mathcal{L}}_{\mathrm{dice}}$ is expressed as:

$${\mathcal{L}}_{\mathrm{dice}}=1-{\displaystyle \frac{2\times |Y\cap \widehat{Y}|}{\left|Y\right|+|\widehat{Y}|}}$$

(5)

where Y denotes the GT binary mask and $\widehat{Y}$ is the corresponding predicted mask.

The classification component, implemented as a cross-entropy loss, is defined as:

$${\mathcal{L}}_{\mathrm{class}}=-\sum _{c=1}^C{y}_{o,c}log\left(p_{o,c}\right)$$

(6)

where C is the total number of classes, $y_{o,c}$ is the binary indicator specifying whether class c is the true label for observation o, and $p_{o,c}$ is the model’s predicted probability that observation o belongs to class c.

2.4. Fine-Tuning Strategy

We initially fine-tune MP-Former with EVS-XL encoders pre-trained on SA-1B and randomly initialized decoders, but it fails to generalize due to TimberSeg 1.0’s small size. We then initialize decoder weights from Swin-B Mask2Former pre-trained on MS-COCO [14] and fine-tune the full model, as freezing the encoder degraded performance.

2.5. Implementation Details

We build on the Detectron2 library [18] in PyTorch [38], using official implementations of Mask2Former [15], MP-Former [16], and TimberSeg 1.0 [4]. All models are trained for 8000 iterations with a learning rate $1\times {10}^{-4}$, using AdamW [39], same as used in [4] while for batch size, we used 16 instead of 8. Unless stated otherwise, hyperparameters match those in TimberSeg 1.0. Training was done in Vienna Scientific Cluster VSC-5 with 2 nodes × 2 NVIDIA A40 GPUs (46 GB; NVIDIA Corporation, Santa Clara, CA, USA) and AMD EPYC 7252 CPUs (Advanced Micro Devices, Santa Clara, CA, USA). Inference and FPS measurements were done on a NVIDIA RTX 3080 Ti (12 GB; NVIDIA Corporation, Santa Clara, CA, USA) and Intel i7-12700K (Intel Corporation, Santa Clara, CA, USA) system, with FPS averaged over validation set using batch size 1 and single-scale inference. The models with Swin variant backbones use their corresponding official MS-COCO pre-trained weights.

To prevent overfitting in experiments with EVS-XL, where the upsampling layers are trained from scratch, we use heavy augmentations using the Albumentations [40] library, as given in Listing 1 and shown in Figure 5. For benchmarking on our In-house Annotated and Unannotated datasets, we take the average weight of five folds of the particular model. By averaging the weights, the resulting model is less likely to be influenced by the idiosyncrasies of any single fold. This improved the model’s ability to generalize to new, unseen data. To evaluate performance, we use mean average precision (AP), mean average precision at IoU 0.5 (AP50), mean average recall (AR) and inference frames per second as used in [4].

Listing 1. Albumentations augmentation pipeline.

import albumentations as A

transforms = A. Compose ([  A. Sequential ([

A. SomeOf ([

A. OneOf ([

A. MotionBlur (),

A. MedianBlur (),

A. Blur (),

A. AdvancedBlur (),

A. Defocus (),

A. GaussianBlur (),

A. GlassBlur (),

], p = 1.0),

A. OneOf ([

A. CLAHE (clip_limit=2),

A. RandomBrightnessContrast (),

A. HueSaturationValue (),

A. RandomGamma (),

A. Emboss (),

A. Equalize (),

A. RGBShift (),

A. Sharpen (),

A. RandomToneCurve (),

], p = 1.0),

A. OneOf ([

A. RandomFog (),

A. RandomRain (),

A. RandomShadow (),

A. RandomSunFlare (),

A. Spatter (),

], p = 1.0),

A. OneOf ([

A. GaussNoise (),

A. ISONoise (),

A. ImageCompression (),

A. ChromaticAberration (),

A. ColorJitter (),

A. Downscale (),

A. MultiplicativeNoise (),

], p = 1.0),

], n = 2, p = 1.0),

A. PixelDropout (dropout_prob = 0.2, p = 0.5),

], p = 0.5)

])

3. Results and Discussion

In this section, we present ablation experiments of all models on the TimberSeg 1.0 dataset (

Table 2. Performance comparison of all models on TimberSeg 1.0 [4]. † denotes Mask2Former [15] architecture, ⋄ denotes MP-Former [16] architecture, and △ denotes results from [4]. EVS is short for EfficientViT-SAM [13], TC for Transposed Conv [26], and TCC for Transposed CoordConv [27]. Best performances have been marked in bold. AP, AP50, AR report the mean ± std of 5 folds.

Backbone	AP	AP50	AR	FPS	Backbone Params	Total Params
EVS-XL1-TC (SA-1B) ⋄	61.05$\pm 1.88$	84.09$\pm 2.17$	69.59$\pm 1.63$	11.99	189.55M	209.53M
EVS-XL1-TCC (SA-1B) ⋄	61.06$\pm 1.42$	84.27$\pm 2.26$	69.53$\pm 0.83$	11.183	189.65M	209.63M
EVS-XL0-TC (SA-1B) ⋄	59.31$\pm 2.19$	82.68$\pm 2.16$	67.86$\pm 2.20$	13.6	118.91M	138.89M
EVS-XL0-TCC (SA-1B) ⋄	59.28$\pm 2.02$	83.42$\pm 2.25$	67.89$\pm 1.43$	12.61	119.0M	138.98M
Swin-B (IN22k) †,△ [4]	57.53$\pm 3.37$	84.28$\pm 2.44$	65.16$\pm 3.40$	8.47	86.88M	106.86M
Swin-B (IN22k) †	60.37$\pm 1.58$	85.42$\pm 2.34$	67.85$\pm 1.08$	7.76	86.88M	106.86M
Swin-B (IN22k) ⋄	61.1$\pm 1.47$	84.89$\pm 2.03$	69.42$\pm 0.69$	7.76	86.88M	106.86M
Swin-S (IN1k) †	58.10$\pm 1.88$	83.33$\pm 2.42$	65.97$\pm 1.28$	10.17	48.84M	68.69M
Swin-S (IN1k) ⋄	58.73$\pm 2.07$	83.29$\pm 2.34$	67.5$\pm 1.64$	10.17	48.84M	68.69M
Swin-T (IN1k) †	56.88$\pm 1.89$	82.75$\pm 1.88$	65.52$\pm 1.48$	12.65	27.52M	47.38M
Swin-T (IN1k) ⋄	57.47$\pm 1.91$	82.41$\pm 2.03$	67.12$\pm 1.52$	12.65	27.52M	47.38M

) and evaluate the average weights of 5 runs on our In-house Annotated dataset (

Table 3. Performance comparison of models on our In-house Annotated dataset. † means Mask2Former architecture, ⋄ means MP-Former architecture. EVS is short for EfficientViT-SAM, TCC for Transposed CoordConv, TC for Transposed Conv. Best performances have been marked in bold.

Backbone	AP	AP50	AR	FPS
EVS-XL1-TC (SA-1B) ⋄	67.06	86.09	74.16	11.99
EVS-XL1-TCC (SA-1B) ⋄	65.81	85.37	74.19	11.183
EVS-XL0-TC (SA-1B) ⋄	59.22	77.17	74.78	13.6
EVS-XL0-TCC (SA-1B) ⋄	66.16	85.90	74.22	12.61
Swin-B (IN22k) †	65.21	86.26	70.43	7.76
Swin-B (IN22k) ⋄	64.43	85.40	72.00	7.76
Swin-S (IN1k) †	63.25	84.65	70.42	10.17
Swin-S (IN1k) ⋄	61.00	81.36	71.09	10.17
Swin-T (IN1k) †	61.36	83.77	70.69	12.65
Swin-T (IN1k) ⋄	60.65	83.24	69.82	12.65

), measuring instance segmentation performance on AP, AP50, and AR along with FPS and model complexity (backbone and total parameter counts). Our ablation studies compare two EVS-XL variants with two types of upsampling layers: Transposed Conv [26] (TC) and Transposed CoordConv [27] (TCC), and three Swin-Transformer scales (B, S, T), using either Mask2Former (†) or MP-Former (⋄) architecture. In our experiments on TimberSeg 1.0, our fold 1 split almost always had the lowest AP, so for this dataset, we only provide visualizations of fold 1 with the fold 1 model weights. For our In-house Annotated and Unannotated datasets, all results are with averaged weights as mentioned in Section 2.5.

3.1. Benchmark on TimberSeg Dataset

Table 2 shows the ablation study of our models on five folds of the TimberSeg 1.0 dataset. The EVS-XL1-TCC model attains the second highest AP of $61.06\pm 1.42$, AP50 of $84.27\pm 2.26$, and AR of $69.53\pm 0.83$, while running at 11.18 FPS with 189.65M backbone (209.63M total) parameters. For EVS-XL0, TC scores slightly better AP than TCC but lower AP50 and AR. In the Swin variants, we see the MP-Former architecture score higher AP and AR than their Mask2Former counterparts. Compared to the Swin variants, the EVS-XL variants achieve better precision-throughput balance, even though they have a higher number of parameters. Interestingly, our Swin-S baselines outperform SwinB-Mask2Former of [4], which we attribute to our bigger batch size.

3.2. Benchmark on Our In-House Annotated Dataset

On our In-house Annotated dataset (Table 3), EVS-XL1-TC leads in AP with 67.06 AP, 86.09 AP50, and 74.16 AR at 11.99 FPS, confirming robust generalization to real-world scenes. We observe that TC performs better than TCC for EVS-XL1 and conversely for EVS-XL0. Notably, the Swin MP-Former variants, despite their top performance on TimberSeg 1.0, experience a relative drop on the In-house data, suggesting that the mask-piloted training strategy may induce a degree of dataset-specific overfitting. Because MP-Former introduces denoising queries directly guided by ground-truth masks during training, it can encourage the decoder to specialize more strongly to the spatial statistics and annotation style of the training dataset, which may reduce robustness when evaluated on scenes with different clutter patterns, viewpoints, or sensor characteristics. In contrast, the EVS-XL MP-Former variants maintain strong performance on the In-house dataset. We hypothesize that this improved robustness is related to the much broader pretraining of EfficientViT-SAM on the SA-1B corpus and to its convolutional and linear-attention inductive biases, which provide a more generic representation and may reduce the tendency of the decoder to over-specialize during mask-piloted training. A direct comparison between EVS backbones with and without mask-piloted training is left for future work. In addition, future work will explore stronger regularization strategies, such as increased data augmentation, query dropout, or reduced reliance on denoising queries, to further mitigate dataset-specific overfitting in mask-piloted training.

3.3. Effect of Key Design Factors

Based on the ablation results in Table 2 and Table 3, three design factors primarily influence model performance: (i) the segmentation framework (Mask2Former vs. MP-Former), (ii) the backbone architecture and pretraining (Swin vs. EfficientViT-SAM, and backbone scale), and (iii) the upsampling strategy used to adapt EfficientViT-SAM features to the pixel decoder. Since these factors were not varied in a full factorial design, the following analysis relies on controlled model pairs and aims to identify consistent empirical patterns rather than establish strict causal effects.

3.3.1. Effect of Segmentation Framework

For all Swin backbones, replacing Mask2Former with MP-Former yields a consistent improvement on TimberSeg 1.0. For Swin-B, AP increases from $60.37$ to $61.10$ (+0.73); for Swin-S, from $58.10$ to $58.73$ (+0.63); and for Swin-T, from $56.88$ to $57.47$ (+0.59). Similar gains are observed in AR. On the In-house dataset, however, MP-Former does not consistently outperform Mask2Former for Swin backbones: for Swin-B, AP decreases from $65.21$ to $64.43$, and for Swin-S, from $63.25$ to $61.00$. This indicates that mask-piloted training primarily improves in-domain accuracy on TimberSeg but may reduce robustness under dataset shift for Swin-based models due to potential overfitting.

3.3.2. Effect of Upsampling Strategy

The choice between transposed convolution (TC) and transposed CoordConv (TCC) has a negligible and inconsistent effect on TimberSeg performance. For EVS-XL1, AP changes from $61.05$ to $61.06$ (+0.01), and for EVS-XL0, from $59.31$ to $59.28$ ($-0.03$). On the In-house dataset, the relative ordering reverses between EVS-XL1 (TC better than TCC) and EVS-XL0 (TCC better than TC). All differences remain within approximately $\pm 1.3$ AP, indicating that the upsampling design has a second-order influence compared to backbone and framework choices.

3.3.3. Effect of Backbone Architecture and Pretraining

The impact of backbone choice differs markedly between in-domain performance and cross-dataset generalization. On TimberSeg 1.0, the best Swin-B MP-Former model ($61.10$ AP) and the best EVS-XL1 MP-Former model ($61.06$ AP) achieve nearly identical AP, indicating that backbone choice has a limited effect on peak in-domain accuracy when the framework is fixed. In contrast, on the In-house dataset, EVS-XL1-TC improves AP from $64.43$ (Swin-B MP-Former) to $67.06$ (+2.63), and also outperforms all Swin variants across both frameworks. This suggests that large-scale SA-1B pretraining and the convolutional and linear-attention inductive biases of EfficientViT-SAM primarily improve robustness under dataset shift rather than in-domain accuracy.

3.3.4. Summary of Effects

Across both datasets, the segmentation framework mainly governs in-domain accuracy on TimberSeg (+0.6–0.7 AP for Swin backbones), the upsampling strategy has a negligible and inconsistent effect, and the backbone architecture and pretraining dominate cross-dataset generalization (+2.6 AP on the In-house dataset). Given the coupled nature of the ablation design, these conclusions should be understood as robust trends supported by multiple controlled comparisons rather than as isolated causal attributions.

3.4. Visual Comparison

Figure 6 shows results of instance segmentation on our fold1 validation set of TimberSeg 1.0. Figure 7 shows results on our In-house Unannotated dataset. As we can see in Figure 6, in the third sample, EVS-XL1-TC and EVS-XL0-TCC are able to detect logs partially occluded by foliage and not annotated in the ground truth. In Figure 7, samples 4 to 6, we can see all the models have very good performance at night under very low-light conditions as well. Overall, the models do not exhibit distinct visual differences in segmentation in terms of false positives or false negatives, apart from the EVS-XL models having more refined edge predictions and less bleeding to neighboring overlapping logs. This is expected, as all models are trained with identical supervision on the same dataset and optimized for the same objective. In several examples, false detections and missed instances remain present across all backbones, indicating that the failure modes are shared by the architecture family rather than specific to the proposed encoder. Consequently, the main benefits of the EVS-XL backbones are reflected more clearly in the quantitative metrics (AP and AR) and in improved robustness across the two datasets, rather than in striking visual differences in isolated examples. Further improving the discrimination of hard false positives and resolving extreme instance ambiguities in dense clutter remains an open problem and will likely require additional geometric cues, multi-view information, or explicit reasoning about log topology beyond monocular RGB input.

3.5. Inference Speed

From Table 2 and Table 3, it is evident that the integrated EfficientViT-SAM MP-Former models achieve a substantially better accuracy-throughput trade-off than the Swin-based baselines. The EVS-XL1-TC model, which delivers the highest AP on the In-house Annotated dataset, runs at 11.99 FPS, a $54.51\%$ speedup over our Swin-B baseline implementations while achieving comparable AP and AR on the TimberSeg 1.0 dataset. The lighter EVS-XL0-TCC variant further increases throughput to 12.61 FPS, making it almost as fast as Swin-T baselines in spite of having $2.93$ times more parameters while still maintaining higher accuracy.

Despite the EVS-XL backbones having significantly more parameters than Swin-B (up to 189.55M backbone parameters vs. 86.88M for Swin-B), their inference speed remains comparable because the dominant runtime cost is not parameter count but the type of operations used in the backbone. Swin Transformer relies on window-based self-attention, whose computational cost grows quadratically with the number of spatial tokens within each window and becomes increasingly expensive at high resolution. In contrast, EfficientViT-SAM relies primarily on convolutional blocks and ReLU-based linear attention, whose complexity scales linearly with the number of tokens. As a result, despite having nearly twice the number of parameters, EVS-XL achieves similar or higher FPS because its operations are more hardware-efficient and memory-friendly at the input resolutions used in our experiments. In practice, all EVS-XL models comfortably fit within the 12 GB memory budget of the RTX 3080 Ti used for benchmarking, indicating that the increased parameter count does not impose a memory bottleneck in the targeted robotic deployment setting. This confirms that parameter count alone is not a reliable proxy for inference speed or practical deployability. Moreover, the MP-Former architecture introduces no additional inference-time overhead compared to Mask2Former, since the mask-piloted queries are used exclusively during training.

Overall, these results demonstrate that the integrated EVS-XL MP-Former models are well suited for near real-time robotic grasping pipelines on commodity GPU hardware, providing a strong balance between accuracy and throughput under our experimental setting.

3.6. Limitations and Future Directions

Although the integrated EfficientViT-SAM MP-Former improves quantitative performance and generalization, the qualitative visual differences between models remain limited, particularly in terms of hard false positives and false negatives in dense log piles. As shown in Figure 6 and Figure 7, all methods exhibit similar failure cases in extreme clutter and heavy occlusion, suggesting that current monocular RGB-based instance segmentation is approaching a performance ceiling for this setting. Addressing these limitations will likely require incorporating additional geometric information such as depth or multi-view cues, explicit modeling of log topology, or temporal consistency across frames. Exploring such directions is an important avenue for future work toward safer and more reliable robotic grasping in highly cluttered forestry environments.

In addition, although the integrated EVS-XL models fit within the memory budget of our target hardware, exploring model compression techniques such as structured pruning or post-training quantization is an important direction for future work to further reduce memory footprint and enable deployment on more resource-constrained robotic platforms. We leave a systematic study of compression and accuracy–efficiency trade-offs to future work, as the focus of this paper is on architectural benchmarking and backbone selection rather than deployment-specific optimization.

Finally, this work restricts its empirical study to the Mask2Former/MP-Former family in order to enable controlled architectural comparisons and reproducible ablations. A broader comparison against recent end-to-end instance segmentation pipelines and foundation-model-based systems, such as YOLO26 [41], RF-DETR [42], DINOv3 [43] or newer generations of SAM-based models (e.g., SAM2 [44]), remains an important direction for future work. Such a study would require a dedicated benchmarking effort that carefully controls for differences in training protocols, data scale, and real-time deployment constraints, and is beyond the scope of the present work, whose focus is on understanding the impact of backbone choice and pretraining within a fixed segmentation framework.

4. Conclusions

In this paper, we proposed the integration of EfficientViT-SAM-XL encoders into MP-Former and were successful in getting both AP and FPS improvements on TimberSeg 1.0. Furthermore, the integrated models achieved very good results on our In-house Annotated and Unannotated logs datasets. Additionally, we presented an ablation study on MP-Former and Mask2Former, highlighting MP-Former’s superior performance at the risk of potential overfitting. While our In-house dataset consists of a log-piles-to-truck-loading scenario, further evaluations on data having more varied views and weather conditions need to be done in future works.