RE-XswinUnet: Rotary Positional Encoding and Inter-Slice Contextual Connections for Multi-Organ Segmentation

Abstract

Medical image segmentation has been a central research focus in deep learning, but methods based on convolutions have limitations in modeling the long-range validity of images. To overcome this issue, hybrid CNN-Transformer architectures have been explored, with SwinUNet being a classic approach. However, SwinUNet still faces challenges such as insufficient modeling of relative position information, limited feature fusion capabilities in skip connections, and the loss of translational invariance caused by Patch Merging. To overcome these limitations, the architecture RE-XswinUnet is presented as a novel solution for medical image segmentation. In our design, relative position biases are replaced with rotary position embedding to enhance the model’s ability to extract detailed information. During the decoding stage, XskipNet is designed to improve cross-scale feature fusion and learning capabilities. Additionally, an SCAR Block downsampling module is incorporated to preserve translational invariance more effectively. The experimental results demonstrate that RE-XswinUnet achieves improvements of 2.65% and 0.95% in Dice coefficients on the Synapse multi-organ and ACDC datasets, respectively, validating its superiority in medical image segmentation tasks.

1. Introduction

With the continuous advancement of medical imaging technologies, medical image segmentation has been established as an indispensable component of computer-aided diagnostic systems [1]. Accurate and robust segmentation results are essential for assisting clinicians in lesion localization, lesion assessment, and preoperative planning. These improvements enhance both the efficiency and accuracy of diagnosis and treatment [2,3,4,5]. Since the success of AlexNet on ImageNet [6], convolutional neural networks (CNNs) have become the dominant paradigm in computer vision and have driven notable progress in medical image analysis [7,8].

CNN-based architectures, particularly U-Net [9], have been recognized as highly effective for medical image segmentation owing to their symmetric encoder–decoder design and skip connections [1]. Inspired by U-Net, numerous modifications have been proposed to strengthen feature extraction capabilities and adapt to diverse tasks [10,11,12]. For example, V-Net [13] integrates residual learning into 3D segmentation, making it particularly suitable for structural preservation in volumetric tasks. Three-dimensional U-Net [14] has been adapted to handle volumetric data, capturing inter-slice contextual information to improve segmentation in CT and MRI scans. Attention U-Net [15] has incorporated an attention gating mechanism to emphasize small organs or tumor boundaries, proving effective in noisy or low-contrast environments. UNet++ [16,17], by employing nested and dense skip connections, has been shown to mitigate semantic discrepancies between the encoder and decoder, thereby integrating high-level and low-level features more effectively [8,18]. nnU-Net [19], through its adaptive setup, has consistently achieved top-tier performance in multiple segmentation benchmarks. Moreover, ResUNet [20] and DenseUNet [21] have introduced residual and dense connection structures, respectively, which enhance gradient propagation and feature reuse. Despite their demonstrated effectiveness, CNN-based approaches are still limited by the fixed receptive field of convolutional operations, limiting their capacity to capture long-range dependencies, particularly for complex organ structures and morphological variations [5,7,8,16].

To overcome this limitation, Transformer architectures [22,23] have been introduced into computer vision due to their ability to model global dependencies. Methods such as Vision Transformers (ViTs) and DeiT [24] have demonstrated the feasibility of applying Transformers to visual tasks. ViT divides images into fixed-size patches and treats each patch as a token input to the self-attention module [18,25]. However, the pure Transformer architecture relies on large-scale training data and lacks spatial inductive biases, resulting in limited ability to model fine-grained structures and hindering its direct application to medical image segmentation [26,27]. More recently, considerable research has focused on hybrid CNN-Transformer architectures [28,29,30]. These models typically use dual-branch or embedded designs that integrate Transformer modules into a CNN backbone [31]. TransUNet [16], a representative hybrid architecture, incorporates ViTs modules between the encoder and decoder, employing self-attention to capture global semantic information and thereby improve segmentation accuracy [8,16]. The Segmentation Transformer (SETR) [32] utilizes a pure Transformer encoder to extract global features for semantic segmentation, combined with multi-scale reconstruction modules for precise prediction. CoTr (CNN-Transformer Hybrid) [33] adopts a parallel branch design in which CNNs and Transformers extract local and global features separately, while an attention fusion module enables information interaction. TransBTS [34] focuses on 3D medical images by embedding a Transformer into the bottleneck layer of a 3D U-Net to enhance volumetric structural continuity. UNetr [35] employs a pure Transformer encoder with a U-Net-style decoder to maintain global modeling capabilities while restoring spatial details. SwinUNet [36] adopts a U-Net backbone augmented with the hierarchical Swin Transformer [37], enabling the integration of windowed attention mechanisms and contextual feature modeling, and has become a prominent baseline in medical image segmentation research [18,38]. Nevertheless, despite their impressive results, most of these dual-branch methods have a limitation. They rely on direct connections at specific scales during encoding and decoding. This design limits their ability to effectively extract multi-scale information [17].

The Transformer architecture relies on self-attention to model relationships among all elements in a sequence, but it inherently lacks positional information. Unlike text, images exhibit strong spatial structures in which pixel layout is crucial for interpretation and segmentation [39,40]. Vision Transformers (ViTs) therefore typically incorporate explicit positional encoding mechanisms. Common strategies include absolute position encoding (APE) and relative position encoding (RPE) [41,42]. APE, widely used in early Transformers (e.g., ViTs), lacks translational invariance, is sensitive to input size, and generalizes poorly. RPE instead models relative spatial relationships between elements, yielding stronger generalization ability [43]. SwinUNet employs relative position bias (RPB), where learnable bias weights are introduced for each positional pair within a local window to adjust the attention score matrix. However, because RPB is embedded after the multiplication of query (Q) and key (K), relative positional information cannot directly participate in the computation of attention weights [42,44,45].

In summary, despite the advantages of existing methods and the trend towards hybrid models, key challenges remain unresolved. These include: (1) the ineffective participation of local RPE in attention calculations, which limits complex structure modeling; (2) overly simplistic skip connections that lack multi-scale semantic interaction; and (3) downsampling operations that disrupt spatial structures and impair translational invariance. To address these limitations, an enhanced SwinUNet-based architecture, RE-XSwinUNet, is proposed. First, RoPE is introduced to replace the traditional relative position bias (RPB), thereby enhancing the Transformer’s spatial modeling and detail perception capabilities. Second, designing a multi-scale skip connections block, XskipNet, to achieve rich semantic fusion. Third, an SCAR Block downsampling module is designed to enhance feature structural preservation and cross-block interaction capabilities.

The main contributions of this work are summarized as follows:

(1)

We integrate RoPE into SwinUNet, verifying the feasibility of RoPE in medical image segmentation and enhancing detail modeling capabilities.

(2)

A plug-and-play multi-scale skip connections block, XskipNet, is designed to enhance semantic information fusion capabilities.

(3)

We design SCAR Block, a novel downsampling module that preserves structural information while improving translational invariance.

(4)

Extensive experiments on the Synapse and ACDC datasets demonstrate competitive performance, and ablation experiments confirm the efficacy and generalizability for every component.

The paper is organized as follows: Section 2 provides a detailed introduction to the overall architecture and key module design of RE-XSwinUNet. Section 3 presents evaluation and visualization analysis on the Synapse dataset and the ACDC dataset. Section 4 provides a detailed discussion of each module’s contribution. Section 5 concludes with a summary and analysis of this paper.

2. Materials and Methods

In this section, we will provide a detailed account of the RE-XSwinUNet network architecture and the design of its key modules. The methodology is divided into four parts: the overall network architecture, Rotary Position Embedding (RoPE), the multi-scale skip connections block (XskipNet), and the structure-preserving downsampling module (SCAR Block). Each module is designed to address specific limitations of SwinUNet, with improvements justified from both theoretical and practical perspectives.

2.1. Overall Architecture

The overall architecture of the proposed RE-XSwinUNet is illustrated in Figure 1. The encoder–decoder structure of SwinUNet is retained. In the encoder, Rotary Position Embedding (RoPE) is introduced during the window multi-head self-attention stage to replace the original relative position bias (RPB), thereby enhancing the modeling of local and global details. During downsampling, the SCAR Block is employed to preserve structural information, improve translational invariance, and facilitate cross-block information interaction. In the decoder, multi-scale feature fusion is achieved using XskipNet skip connections, promoting the integration of semantic information across scales and improving the model’s generalization capability.

Specifically, an input image $I\in {ℝ}^{H\times W\times 3}$ is first processed through a patch embedding layer using a convolutional kernel of size, a stride of 4, and 96 output channels, producing a feature map $F_0\in {ℝ}^{{\displaystyle \frac{H}{4}}\times {\displaystyle \frac{W}{4}}\times C}$. The feature map $F_0$ is first encoded by a SwinUNet Transformer Block with RoPE. It is then downsampled by the SCAR Block to generate four multi-scale features:

$$\left\{F_1\in {\displaystyle \frac{W}{4}}\times {\displaystyle \frac{H}{4}}\times 96,F_2\in {\displaystyle \frac{W}{8}}\times {\displaystyle \frac{H}{8}}\times 192,F_3\in {\displaystyle \frac{W}{16}}\times {\displaystyle \frac{H}{16}}\times 384,F_4\in {\displaystyle \frac{W}{32}}\times {\displaystyle \frac{H}{32}}\times 968\right\}$$

The decoder gradually resumes spatial resolution through Patch Expanding and RoPE-SwinUNet Transformer Block. At each scale of the decoder, four multi-scale features generated by the encoder are cross-fused through XskipNet. Finally, a segmentation probability map is generated through a fully connected layer.

2.2. Rotational Position Embedding

Rotary Position Embedding (RoPE) is a method of encoding relative position information into the Transformer architecture [39,42,43]. Despite its prevalence in natural language processing tasks like text classification and machine translation, its application in computer vision remains limited. In Swin Transformer, elative position bias (RPB) is added to the attention score. Let $X_{i=1}^n=\left[x_1,x_2,\dots ,x_n\right]$ represent $n$ word embedding vectors. The query ($Q$, key ($K$), and value ($V$) matrices are derived by applying linear transformations to the input representations, as expressed in Equation (1):

$$Q=W_q•X_{i=1}^n,K=W_k•X_{i=1}^n,V=W_v•X_{i=1}^n$$

(1)

where $W_q$, $W_k$, and $W_v$ represent self-attention weights. The query at position $m$ and the key at position $n$ are represented by $Q_{\mathrm{m}}$ and $K_n$, respectively. The attention score can then be obtained as shown in Equation (2):

$$A\left(Q_m,K_n\right)=soft\mathit{max}({\displaystyle \frac{Q_n^T•K_n}{\sqrt{d}}})$$

(2)

where $d$ is the dimension of the key vector per head, and $Q_m^T$ denotes the transpose of $Q_m$. The RPB is obtained by adding the relative position bias B to Equation (2), as shown in Equation (3):

$$A\left(Q_m,K_n\right)=soft\mathit{max}({\displaystyle \frac{Q_n^T•K_n}{\sqrt{d}}})+bias$$

(3)

However, this bias is applied after the inner product, which limits its influence on the similarity between queries and keys. RoPE uses Euler’s Equation ($e^{j\phi }$) to embed relative position information directly into $Q_{\mathrm{m}}$ and $K_n$. The RoPE transformation is defined in Equation (4):

$$Q_{\mathrm{m}}^{\prime }=Q_{\mathrm{m}}{e}^{jm\phi },K_n^{\prime }=K_n{e}^{jm\phi }$$

(4)

Substituting Equation (4) into Equation (2) yields the attention scores for the Rotary Position Embedding, as shown in Equation (5):

$$A^{\prime }\left(Q_m,K_n,n-m\right)=soft\mathit{max}({\displaystyle \frac{\mathrm{Re}\left[Q_n^T•K_n•e^{j\left(n-m\right)\phi }\right]}{\sqrt{d}}})$$

(5)

where $\mathrm{Re}\left[•\right]$ denotes the real part, and the relative position $n-m$ is inherently captured in the inner product. The output of the attention mechanism is given by Equation (6):

$$f\left(x_m,m\right)=R\left(m\right)W_{\left\{q,k\right\}}{x}_m$$

(6)

where $x_m$ is the mth element of the attention output to be calculated, $R\left(m\right)$ is the Rotary Position Embedding matrix, and $W_{\left\{q,k\right\}}$ is the attention weight matrix. In actual calculations, it is composed of the following matrices, as shown in Equation (7):

$$R\left(m\right)=\left(\begin{array}{ccccccc}\cos m{\phi }_1& -\sin m{\phi }_1& 0& 0& \cdots & 0& 0\\ \sin m{\phi }_1& \cos m{\phi }_1& 0& 0& \cdots & 0& 0\\ 0& 0& \cos m{\phi }_2& -\sin m{\phi }_2& \cdots & 0& 0\\ 0& 0& \sin m{\phi }_2& \cos m{\phi }_2& \cdots & 0& 0\\ ⋮& ⋮& ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& 0& 0& \cdots & \cos m{\phi }_{d/2}& -\sin m{\phi }_{d/2}\\ 0& 0& 0& 0& \cdots & \sin m{\phi }_{d/2}& \cos m{\phi }_{d/2}\end{array}\right)$$

(7)

Note that $R\left(m\right)$ is an orthogonal matrix. Going one step further, we can obtain the product of $Q_{\mathrm{m}}$ and $K_n$, as shown in Equation (8):

$$Q_m^T{K}_n={\left\{R\left(m\right)W_q{x}_m\right\}}^T•\left\{R\left(n\right)W_k{x}_m\right\}=x_m^T{W}_q^{T}R\left(n-m\right)W_k{x}_n$$

(8)

Figure 2 shows the specific implementation block diagram of RoPE in this paper. Before computing the windowed self-attention, the relative position information is fused into the phase angle of the Euler Equation. This is achieved by rotating the keys and values vectors using the complex exponential function (Euler’s formula), after which the attention scores are computed. This approach effectively captures the similarity between keys and values, thereby augmenting the model’s capacity for relative position representation and detailed information extraction.

2.3. XskipNet Block

In the baseline SwinUNet, skip connections only concatenate features from encoder and decoder at the same resolution. This approach has two serious issues: First, concatenation at specific dimensions makes it difficult to fuse multi-scale information, reducing the model’s ability to capture details and semantic information; Second, direct concatenation may cause conflicts between shallow (detail-rich) and deep (semantic-rich) features.

To address these issues, the XskipNet block is proposed (Figure 3). Through multi-scale cross-fusion and attention-guided mechanisms within each skip connections, the model effectively merges detailed shallow features with rich semantic information from deeper layers. This integration significantly improves the accuracy of segmenting fine edges and small organ structures. Specifically, for a target scale of the skip connections $S_t$ (e.g., scale ${\displaystyle \frac{1}{8}}$), the corresponding encoder feature $E_t$ is used. In addition, the encoder feature $E_{t-1}$ is resized to the target scale $E_t$ using bilinear interpolation. Meanwhile, the channel count is modified using $1\times 1$ to align with the number of channels in $E_t$, yielding multi-scale features that share a uniform channel count. The multi-scale features are concatenated with the original scale features $E_t$ in the channel dimension to obtain the multi-scale fusion features $E_t^{\prime }$, which are then fed into Convolutional Block Attention Module (CBAM) for spatial and channel attention weighting [16,46,47]. The reduction ratio for channel attention is set to 16, and spatial attention uses $7\times 7$ convolution to generate spatial weight maps. The corresponding scale features in the decoder are enhanced through residual addition of the attention-weighted features, thereby achieving feature fusion.

For target scale ${\displaystyle \frac{1}{8}}$, its multi-scale fusion feature $E_t^{\prime }$ is shown in Equation (9):

$$E_t^{\prime }=CBAM\left(Concat\left(E_t,Conv\left(Bilinear\left(E_{t-1}\right)\right)\right)\right)$$

(9)

Among them, $E_t$ is the ${\displaystyle \frac{1}{8}}$-scale feature of the encoder, $E_{t-1}$ is the ${\displaystyle \frac{1}{4}}$-scale feature of the encoder, $Bilinear\left(•\right)$ represents bilinear downsampling, $Conv\left(•\right)$ is the convolution layer for adjusting channels, and $CBAM\left(•\right)$ represents the weighting operation.

Furthermore, the decoder’s multi-scale fusion feature $F_t$ is shown in Equation (10):

$$F_t=Concat\left(E_t^{\prime }+S_t\right)$$

(10)

Similarly, the fusion feature $E_{t+1}^{\prime }$ of target scale ${\displaystyle \frac{1}{16}}$ can be obtained as shown in Equation (11):

$$E_{t+1}^{\prime }=CBAM\left(Concat\left(E_{t+1},Conv\left(Bilinear\left(E_t^{\prime }\right)\right)\right)\right)$$

(11)

The fusion feature $F_{t+1}$ of the decoder’s target scale ${\displaystyle \frac{1}{16}}$ is shown in Equation (12):

$$F_{t+1}=Concat\left(E_{t+1}^{\prime }+S_{t+1}\right)$$

(12)

The XskipNet block, which is designed to combine information between shallow details and deep semantics effectively, improves the accuracy of edge and small organ segmentation. At the same time, it maintains multi-scale consistency and feature complementarity, reducing conflicts and redundant information introduced by direct splicing.

2.4. SCAR Block

In Swin Transformer, Patch Merging is responsible for reducing the sampling rate while increasing the number of channels [48,49,50]. The input feature map $x\in {ℝ}^{B\times H\times W\times C}$ is first subsampled by selecting one pixel from every two pixels along both spatial dimensions. Subsequently, the chosen pixels are amalgamated along the channel axis, thereby engendering the resultant transformed feature map $x_1\in {ℝ}^{B\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}\times 4C}$. Finally, a fully connected layer is employed to calibrate the channel count from 4C to 2C, yielding the output $f(x)\in {ℝ}^{B\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}\times 2C}$. The Equation for Patch Merging is shown in Equation (13):

$$f(x)=MLP\left(Norm\left(M_{merge}\left(x\right)\right)\right)$$

(13)

Among them, $M_{merge}\left(•\right)$ is the checkerboard sampling and stitching operation, $Norm\left(•\right)$ is the normalization layer, and $MLP\left(•\right)$ is the fully connected layer responsible for channel compression. This method limits the exchange of non-local information across blocks. When changing the number of channels through a fully connected neural network, the spatial structure of the feature map is altered, thereby destroying translational invariance. In order to address this issue, we redesigned a downsample module called the SCAR Block.

Figure 4 shows the SCAR Block network architecture design. This module performs downsampling without destroying the original feature map spatial structure. The module uses standard 2D convolution for downsampling and introduces the Squeeze Excitation (SE) [51] attention mechanism to highlight important channel features and suppress irrelevant ones. Concurrently, the residual connections are incorporated within the module. This design choice effectively mitigates the vanishing gradient problem prevalent in deep networks, thereby enhancing the stability of the network throughout the training process.

Specifically, the main branch first passes through an $3\times 3$ convolution (stride = 2) and a normalization layer to transform the input feature map $B\times C\times H\times W$ into $B\times 2C\times {\displaystyle \frac{H}{2}}\times {\displaystyle \frac{W}{2}}$. The output is then fed into the Squeeze Excitation module to obtain a weight vector of size $B\times 2C\times 1\times 1$, which is then multiplied by the input feature map along the channels to perform channel weighting. The residual branch uses $1\times 1$ convolution (stride = 2) and normalization to perform downsampling and channel number adjustment. The outputs of the two branches are summed and subsequently subjected to a ReLU activation function. This procedure diminishes the spatial resolution and concurrently accentuates the model’s proficiency in extracting salient features at different scales during the downsampling phase.

3. Results

3.1. Datasets

In this study, the Synapse multi-organ segmentation dataset [52] and the Automated Cardiac Diagnosis Challenge (ACDC) dataset [53] were utilized. The Synapse dataset originates from a subset of the Multi-Atlas Labeling Beyond the Cranial Vault (BTCV) Challenge. The ACDC dataset is the official dataset for the Automated Cardiac Diagnosis Challenge held during MICCAI 2017. The manual segmentation labels (i.e., ground truth) for both datasets were provided by the organizers and meticulously annotated by experienced radiologists following standardized clinical protocols. These annotations are recognized as industry benchmarks and have been extensively used in numerous prior studies [16,36] to enable fair and consistent comparisons of segmentation algorithms.

We used the Synapse multi-organ CT dataset, which comprises abdominal contrast-enhanced CT scans from 30 patients. This dataset contains a total of 3779 axial slices. Each scan includes 85–198 slices with an in-plane resolution of 512 × 512 pixels. To ensure consistency with prior work [54], we resampled all slices to a resolution of 224 × 224 pixels. Eight abdominal structures were annotated in this dataset, including the aorta, gallbladder, left kidney, right kidney, liver, pancreas, spleen, and stomach. Following common practice, the dataset was randomly divided into 18 cases for training and 12 cases for validation, resulting in a training set of 2212 axial slice images [16].

We evaluated the RE-XswinUNet architecture using the ACDC dataset, which contains cine cardiac MR images from 100 patients captured during breath-hold. The dataset provides manual annotations for the left ventricle (LV), right ventricle (RV), and myocardium (MYO). Each short-axis series extends from the left ventricular base to the apex. The images have a slice thickness of 5–8 mm and an in-plane resolution of 0.83–1.75 mm²/pixel. Following the ACDC protocol, we randomly split the data into 70 patients (1930 slices) for training, 10 for validation, and 20 for testing. In addition, we employed the same baseline methods as reported in the original challenge to ensure a fair performance comparison.

Both the Synapse and ACDC experiments represent internal validation, as their training and test datasets are derived from the same source. While this setup allows for fair comparisons with established benchmarks, it may lead to performance estimates that do not fully generalize to external data collected from different clinical environments.

3.2. Implementation Details

The implementation of this study was conducted utilizing Python 3.8 in conjunction with the PyTorch 1.8.0. The input CT or MR images resolution was set to 224 × 224, and the patch size to 16 × 16. All experiments were performed on a NVIDIA A800 GPU. Each training cycle comprised 500 steps, utilizing a batch size of 16. We adopted the AdamW optimizer, setting the learning rate at 1 × 10⁻⁴ to reduce overfitting. For fair comparison, The loss function is adopted from the loss function used in the baseline model SwinUNet. The publicly available pre-trained weights of TransUNet were used to initialize RE-XSwinUNet.

3.3. Evaluation Indicators

This study used the average Dice similarity coefficient (DSC) and average Hausdorff distance (HD95) as evaluation indicators.

In the field of image segmentation, model performance is often evaluated by Dice similarity coefficient, which measures the degree of overlap between the actual segmentation results and the predicted segmentation. This coefficient ranges in value from 0 to 1. When the value is closer to 1, it is indicative of superior segmentation performance. Its calculation formula can be expressed in Equation (14):

$$DSC(P,T)={\displaystyle \frac{2\left|P\cap T\right|}{\left|P\right|+\left|T\right|}}$$

(14)

where $P$ denotes the segmentation label, and $T$ denotes the prediction result.

The Hausdorff distance, as a measurement criterion, is used to assess the distance between the segmentation result and the labels. It is particularly sensitive to the segmentation boundaries. A smaller HD95 indicates higher similarity, while a larger value indicates greater discrepancy. In this study, the Hausdorff distance was utilized to assess model performance by comparing the actual labels with the segmentation results. The mathematical expression for the Hausdorff distance is presented in Equation (15):

$$HD(S_p,S_{\mathrm{t}})=\max \left\{\underset{p\in S_p}{\max }\left\{\underset{t\in S_t}{\min } d\left(p,t\right)\right\},\underset{t\in S_t}{\max }\left\{\underset{p\in S_p}{\min } d\left|\left(p,t\right)\right|\right\}\right\}$$

(15)

where $S_{\mathrm{t}}$ denotes the set of points representing the predicted results, $S_p$ denotes the set of points representing the segmentation labels, and $\underset{t\in S_t}{\min }d\left(p,t\right)$ denotes the minimum Euclidean distance from $p$ to $S_{\mathrm{t}}$.

3.4. Overall Performance on Synapse Dataset

The comparative performance of the proposed RE-XswinUNet architecture is presented in

Table 1. Comparison results of different models on the Synapse dataset (average DSC score (%), average HD (mm), and DSC score for eight organ).

Method	DSC	HD	Aorta	Gall.	Kidney(L)	Kidney(R)	Liver	Pancreas	Spleen	Stomach
V-Net [13,16]	68.81	-	75.34	51.87	77.10	80.75	87.84	40.05	80.56	56.98
DARR [16,54]	69.77	-	74.74	53.77	72.31	73.24	94.08	54.18	89.90	45.96
U-Net [9,16]	74.68	36.87	87.74	63.66	80.60	78.19	93.74	56.90	85.87	74.16
AttnUNet [15,16]	75.57	36.97	55.92	63.91	79.20	72.71	93.56	49.37	87.19	74.95
R50-ViT [16,55]	71.29	32.87	73.73	55.13	75.80	72.20	91.51	45.99	81.99	73.95
TransUNet [16]	77.48	31.69	87.23	63.13	81.87	77.02	94.08	55.86	85.08	75.62
CoT-TransUNet [36]	78.24	23.75	88.69	62.56	88.33	76.91	94.57	55.23	86.35	78.28
MT-UNet [56]	78.59	26.59	87.92	64.99	81.47	77.29	93.06	59.46	87.75	76.81
SwinUNet [36]	79.13	21.55	85.47	66.53	83.28	79.61	94.29	56.58	90.66	76.60
RE-XSwinUNet	81.78	18.83	87.99	68.19	83.76	81.43	94.55	65.11	91.72	81.52

. The model is evaluated against state-of-the-art methods, including SwinUNet, TransUNet, and UNet, on the Synapse dataset. The results indicate that the RE-XswinUNet architecture achieves a significant improvement in performance, with an DSC of 81.78% and a reduced HD of 18.83 mm. Relative to the SwinUNet baseline, the DSC was increased by 2.65%, while the HD was reduced by 2.72 mm. Furthermore, higher DSC were consistently obtained across all organs, with particularly pronounced improvements observed in small organs (e.g., pancreas and gallbladder). These results demonstrate that the RE-XswinUNet architecture can effectively extract more detailed anatomical information.

Figure 5 presents representative visualization results on the Synapse dataset, comparing our method with other approaches. The segmentation outcomes of the stomach (first row) and spleen (second row) indicate that the UNet model is prone to over-segmentation, a phenomenon that may be attributed to the inherent dependence of convolutional neural network architectures on local contextual information. When the pancreas is considered (first and third rows), the proposed model produces more precise boundaries and achieves higher segmentation accuracy than SwinUNet. This improvement can be ascribed to the integration of RoPE and XskipNet, as discussed in Section 4. In contrast, for the left kidney, right kidney, and stomach, the performance gain of RE-XSwinUNet over SwinUNet and TransUNet is less substantial, which may be explained by the clearer structures and well-defined boundaries of these larger organs that can already be effectively segmented by conventional architectures. Overall, the proposed model yields moderate improvements for large organs, while considerable enhancements are obtained for smaller and more structurally complex organs.

3.5. Overall Performance on ACDC Dataset

To validate the generalization capability of the RE-XSwinUNet architecture on other datasets, we trained and evaluated it on the ACDC dataset for automated cardiac diagnosis. The experimental evaluation results are shown in

Table 2. Comparison results of different models on the ACDC dataset (average Dice score (%)).

Methods	DSC	RV	Myo	LV
R50 U-Net [16,57]	87.55	87.10	80.63	94.92
R50 Att-UNet [36,57]	86.75	87.58	79.20	93.47
R50 ViT [36,57]	87.57	86.07	81.88	94.75
TransUNet [16,57]	89.71	88.86	84.53	95.73
SwinUNet [36,57]	90.00	88.55	85.62	95.83
RE-XSwinUNet	90.95	91.24	85.52	96.08

, where the DSC of RE-XSwinUNet reaches 90.95%, outperforming baseline models such as TransUNet and SwinUNet in terms of performance. The improvements are particularly noticeable in the right ventricle and left ventricle, primarily due to the ability of RoPE and the XskipNet module proposed in this paper to model detailed information.

3.6. Model Complexity and Efficiency Analysis

To assess the practicality of RE-XSwinUNet, we examined its model complexity against leading baseline methods by comparing parameter counts and estimated memory usage. As summarized in

Table 3. Comparison of Model Complexity.

Methods	Total Params (M)	Estimated Total Size (MB)
UNet	31.037	567.90
TransUNet	108.596	841.84
SwinUNet	37.169	408.04
RE-XSwinUNet	40.241	423.98

, RE-XSwinUNet has 40.241 million parameters, which is considerably lower than TransUNet (108.596 million) but moderately higher than both SwinUNet (37.169 million) and UNet (31.037 million).

RE-XSwinUNet exhibits notable parameter efficiency relative to TransUNet, attaining higher segmentation accuracy while utilizing fewer than 40% of the parameters. In comparison with SwinUNet, the proposed model incurs only a marginal parameter increase of around 3 million. This slight growth in model size leads to substantial gains in segmentation performance, especially in the delineation of small anatomical structures. With an estimated total memory footprint of 423.98 MB, RE-XSwinUNet remains compatible with common clinical computational environments, supporting its practical deployability.

4. Discussion

4.1. Ablation Studies

To evaluate the contributions of the proposed modules individually and in combination, and to address potential difficulties in component selection, we conducted a comprehensive ablation study. We integrated the RoPE, XskipNet, and SCAR block modules into the SwinUNet baseline model and report their individual and combined performance on the Synapse dataset in

Table 4. Evaluation results of each module on the Synapse dataset (average Dice score (%)).

Block	DSC	Aorta	Gallb.	Kidney (L)	Kidney (R)	Liver	Pancreas	Spleen	Stomach
SwinUNet [36]	79.13	85.47	66.53	83.28	79.61	94.29	56.58	90.66	76.60
RoPE	80.38	86.32	67.78	82.76	78.50	94.12	63.43	91.19	78.92
XskipNet	79.97	85.55	67.83	82.37	78.14	94.58	62.26	91.72	77.28
SCAR Block	80.12	86.82	66.34	84.47	81.76	94.38	56.10	91.79	79.32
RoPE+XskipNet	81.15	87.25	68.05	83.25	80.35	94.52	64.85	91.45	80.28
RoPE+SCAR Block	80.95	87.42	67.45	84.12	81.25	94.45	62.75	91.68	80.65
XskipNet+SCAR Block	80.65	86.95	67.28	84.35	81.08	94.62	60.85	91.85	79.95
RE-XSwinUNet (All)	81.78	87.99	68.19	83.76	81.43	94.55	65.11	91.72	81.52

. Additionally, a thorough theoretical investigation was conducted to explain the principles behind the performance gains of these three modules, further demonstrating the feasibility of the RE-XSwinUNet architecture.

RoPE. Following the introduction of RoPE, the average Dice coefficient was elevated to 80.38%, representing an increase of 1.25% compared with the original baseline model, SwinUNet. In particular, the segmentation of small organs exhibited even more pronounced improvement. For example, the segmentation accuracy of the pancreas was enhanced from 56.58% to 63.43%, corresponding to an improvement of 6.85%. We attribute the effectiveness of RoPE primarily to its long-range attenuation characteristic. As the relative position increases, the inner product of $Q_{\mathrm{m}}$ and $K_n$ in Equation (8) gradually decreases, enhancing the model’s generalization ability and suppressing excessive focus on long-range dependencies. For Equation (8), when $n=m$, Equation (16) can be obtained:

$$R\left(n-m\right)=R\left(0\right)=R_m^T•R_m=E$$

(16)

where E represents the identity matrix. Further derivation yields Equation (17):

$$Q_m^T{K}_n=x_m^T{W}_q^T{W}_k{x}_m$$

(17)

At this point, the dot product of the attention vector does not decay. When $n\ne m$, as $\left|n-m\right|$ increases, the direction of the vector shifts due to rotation, and its dot product equals the projection of the angle, becoming Equation (18):

$$\left|Q_m^T{K}_n\right|=\left|x_m^T{W}_q^{T}R\left(n-m\right)W_k{x}_n\right|\le ‖W_q{x}_m‖•‖W_k{x}_n‖\cos {\theta }_{n-m}$$

(18)

where $‖•‖$ represents the norm of the matrix, and ${\theta }_{n-m}$ represents the angle between two vectors. This angle increases with $\left|\mathit{n}-\mathit{m}\right|$, which results in a decrease in $\mathit{cos}{\mathit{\theta }}_{\mathit{n}-\mathit{m}}$.

This distance-dependent attenuation characteristic prevents interference from tokens at a distance, thereby improving the model’s ability to model local details and relationships. This results in the most significant improvement in performance on the pancreas. At the same time, RoPE can control attention distribution more smoothly and structurally, which helps improve robustness and generalizability.

XskipNet. The design of the XskipNet block resulted in an increase in the DSC from 79.13% to 79.97%. Significant enhancements in the segmentation accuracy of several key organs, including the aorta, liver, and kidneys, were observed. For larger organs, such as the stomach and spleen, relatively minor improvements were noted. In contrast, considerable progress was achieved for smaller organs, like the pancreas and gallbladder. The segmentation accuracy of the pancreas and gallbladder was found to have increased by 5.56% and 1.3%, respectively. These findings thereby confirm the module’s optimization capability for organs that are highly sensitive to detailed information.

The effectiveness of XskipNet is primarily attributed to its multi-scale feature fusion and attention-guided mechanism. Small organs (e.g., pancreas and gallbladder) occupy only a small proportion of medical images and possess complex boundaries, making them prone to detail loss during downsampling. By fusing shallow-layer high-resolution features, XskipNet mitigates detail loss during downsampling and consequently improves edge segmentation accuracy. Furthermore, since small organs are susceptible to interference from surrounding tissues, the spatial attention mechanism of CBAM enables the model to focus on the organ region and suppress background noise. For large organs, which exhibit clearer structures and occupy a larger proportion, traditional skip connections are generally sufficient, and thus the advantages of XskipNet are less significant.

SCAR Block. After the original patch merging was replaced, structural preservation and translational invariance of the model were enhanced. The Dice score on the aorta increased from 85.47% to 86.82%, while the Dice coefficients on the left kidney and right kidney reached 84.47% and 81.76%, respectively. Significant improvements were achieved in the segmentation of large organs (e.g., liver, kidneys, and aorta). These improvements are not coincidental; they are a direct result of the SCAR Block’s ability to maintain structural integrity and enrich semantic information during downsampling.

In the SCAR Block, convolutional downsampling prevents the disruption of spatial structure caused by traditional patch merging, thereby preserving the overall geometric consistency of large organs. The SE module adaptively recalibrates channel-wise feature responses, allowing the model to focus on stable and discriminative semantic patterns within large organs. Meanwhile, residual connections ensure stable transmission of deep features, alleviating information decay under long-range dependencies.

Module Combinations. To thoroughly evaluate the synergistic effects among innovative modules, we systematically conducted modular combination ablation experiments on the SwinUNet baseline model. Among all dual-module configurations, the “RoPE + XskipNet” combination demonstrated the highest performance with a DSC of 81.15%. Specifically, pancreatic segmentation accuracy improved from the baseline 56.58% to 63.43% after incorporating the RoPE module, and further increased to 64.85% when combined with XskipNet. This result demonstrates that integrating RoP’s relative positional awareness with XskipNet’s cross-scale detail fusion mechanism enables more effective capture of small organ fine structures and boundary features.

Among other combinations, “RoPE+SCAR Block” demonstrated outstanding performance in aortic (87.42%) and renal segmentation through synergistic positional encoding and structural preservation. While “XskipNet+SCAR Block” achieved the best liver segmentation (94.62%), its pancreatic segmentation (60.85%) significantly lagged behind the former due to lacking positional encoding support. Overall, all dual-module combinations significantly outperformed single-module approaches, with the fusion of detail modeling and multi-scale features contributing most substantially to the overall performance enhancement.

4.2. Significance Verification Based on t-Test

To verify whether the performance improvement stems from random fluctuations, this paper conducted five independent replicate experiments on the Synapse dataset. Dice score data were generated using five distinct random seeds (Seed1234, Seed2341, Seed3412, Seed4123, Seed5432), followed by independent samples t-tests. Following standard practice in medical image segmentation, a significance level of α = 0.05 was set. Figure 6 presents the average DSC scores and their standard deviations across the five experiments for different methods. Statistical analysis revealed that RE-XSwinUNet achieved a 2.28% average improvement in Dice score compared to SwinUNet (p = 0.0003 < α, t(8) = 5.99). The 95% confidence interval for this performance gain was [1.37%, 3.19%], which did not include zero, confirming that the observed enhancement exceeded the range of random fluctuation. These results statistically validate the reliability of RE-XSwinUNet’s performance improvement, dispel concerns about the robustness of the enhancement approach, and establish its statistical significance.

4.3. Limitations and Future Work

Although RE-XSwinUNet demonstrates outstanding segmentation performance across both CT and MRI modalities, its clinical application should be carefully evaluated within the practical context of actual imaging modality selection. In clinical practice, particularly under stringent radiation protection regulations such as the European EURATOM Directive 2013/59, the inherent radiation risks associated with CT must be fully considered when selecting it for selective soft tissue assessment. For multiple abdominal organs covered in this study (e.g., pancreas, gallbladder, and liver), magnetic resonance imaging is generally considered the more appropriate imaging modality. The primary objective of this study is to propose a robust and universal segmentation architecture. To this end, we validated the model’s performance on both CT (Synapse) and MRI (ACDC) datasets to demonstrate its cross-modal adaptability and modality-agnostic characteristics. However, further exploring the model’s generalization capabilities in MRI-based multi-organ abdominal segmentation remains a highly promising research direction warranting future investigation. Additionally, to ensure fair comparison with prior studies like those on TransUNet and SwinUNet, this work only completed internal validation and has not yet been evaluated on external test sets. The model’s generalization performance requires more comprehensive validation in subsequent research.

5. Conclusions

In this study, we propose a SwinUNet architecture with rotational position encoding and cross-slice context connection, named RE-XSwinUNet, which is specifically designed to achieve the conducting of accurate and robust multi-organ segmentation tasks. The framework addresses key limitations of existing SwinUNet-based approaches through three major innovations. First, the integration of Rotary Position Embedding (RoPE) enables effective modeling of relative spatial relationships, thereby enhancing the SwinUNet’s capacity to capture fine-grained anatomical structures. Second, the proposed XskipNet module facilitates multi-scale, attention-guided feature fusion between the encoder and decoder, leading to substantial improvements in the segmentation accuracy of small organs with complex boundaries. Third, the structure-preserving SCAR block replaces conventional patch merging, ensuring translational invariance and promoting inter-block information flow during downsampling. Superior performance of RE-XswinUnet was demonstrated on the Synapse and ACDC datasets, surpassing numerous state-of-the-art network models. This study is expected to advance the development of multi-organ segmentation techniques. Future work will focus on exploring the generalization performance of RE-XswinUnet across various imaging modalities and in external validation.