Symmetric Boundary-Enhanced U-Net with Mamba Architecture for Glomerular Segmentation in Renal Pathological Images

Abstract

Accurate glomerular segmentation in renal pathological images is a key challenge for chronic kidney disease diagnosis and assessment. Due to the high visual similarity between pathological glomeruli and surrounding tissues in color, texture, and morphology, significant “camouflage phenomena” exist, leading to boundary identification difficulties. To address this problem, we propose BM-UNet, a novel segmentation framework that embeds boundary guidance mechanisms into a Mamba architecture with a symmetric encoder–decoder design. The framework enhances feature transmission through explicit boundary detection, incorporating four core modules designed for key challenges in pathological image segmentation. The Multi-scale Adaptive Fusion (MAF) module processes irregular tissue morphology, the Hybrid Boundary Detection (HBD) module handles boundary feature extraction, the Boundary-guided Attention (BGA) module achieves boundary-aware feature refinement, and the Mamba-based Fused Decoder Block (MFDB) completes boundary-preserving reconstruction. By introducing explicit boundary supervision mechanisms, the framework achieves significant segmentation accuracy improvements while maintaining linear computational complexity. Validation on the KPIs2024 glomerular dataset and HuBMAP renal tissue samples demonstrates that BM-UNet achieves a 92.4–95.3% mean Intersection over Union across different CKD pathological conditions, with a 4.57% improvement over the Mamba baseline and a processing speed of 113.7 FPS.

1. Introduction

Chronic kidney disease (CKD) has emerged as a major global health challenge affecting over 800 million people worldwide, with annual mortality rates exceeding the combined levels of breast and prostate cancer [1]. Glomeruli serve as the kidney’s fundamental filtration units, and their structural alterations represent key pathological indicators for CKD progression and severity assessment. Accurate glomerular segmentation in renal pathology images is essential for quantitative pathological analysis, disease staging, and treatment planning in CKD management [2].

However, glomerular segmentation faces unique technical challenges primarily stemming from the “camouflage phenomenon” in renal pathology. This phenomenon is referred to as “medical camouflage” in the field of medical image analysis [3]. It manifests at the following three specific levels: (1) Color camouflage: Under PAS staining, sclerotic glomeruli exhibit a highly similar hue and saturation to surrounding fibrotic interstitial tissues; (2) Texture camouflage: Inflammatory infiltration and fibrosis lead to blurred internal glomerular structures, where originally clear capillary walls disappear and texture features become consistent with background tissues; and (3) Boundary camouflage: With disease progression, the thickening of Bowman’s capsule walls, basement membrane lesions, and surrounding inflammatory cell infiltration cause originally distinct anatomical boundaries to become obscured. These camouflage characteristics are particularly pronounced in pathological sections from patients with advanced chronic kidney disease, making accurate identification challenging even for experienced pathologists who require careful observation, thereby severely affecting the accuracy of automatic segmentation algorithms.

Despite significant advances in medical image segmentation, existing methods show clear limitations when confronting these camouflage challenges. Current deep learning approaches, including nnU-Net [4], DeepLabV3+ [5], and transformer-based models [6], have achieved considerable progress in general medical segmentation [7]. However, these traditional approaches focus primarily on global feature extraction and often fail to capture the subtle differences between target glomerular regions and complex renal backgrounds [8]. The methods essentially treat boundary information as implicit features learned through standard convolutions, struggling with the fuzzy boundaries and low-contrast regions that characterize the ‘camouflage’ phenomenon in glomerular pathology [9].

Recent developments in camouflaged object detection (COD) offer promising solutions for these challenges. COD methods have achieved breakthrough results in natural scenes by distinguishing closely resembling targets from complex backgrounds through explicit edge semantics enhancement and contextual information fusion [10]. This boundary-guided detection strategy directly addresses the boundary ambiguity challenges in glomerular segmentation [11]. Several medical studies have successfully adapted COD principles, particularly for polyp and glomerular segmentation, with growing scholarly interest in medical applications [12]. However, existing COD methods face computational inefficiency due to quadratic attention mechanisms [13], constraining their practical deployment in clinical environments.

To balance accuracy with computational efficiency, the recent Mamba architecture—a state-space model with linear computational complexity proposed by Ma et al. [14]—provides an elegant solution. Mamba processes long sequences while maintaining global receptive fields through selective scanning mechanisms, making it particularly suitable for high-resolution medical image segmentation, which requires both comprehensive contextual understanding and real-time processing capability.

This study proposes BM-UNet, which uniquely combines COD’s explicit boundary guidance with Mamba’s efficient sequence modeling for camouflaged glomerular segmentation. Our framework integrates specialized edge-guided modules within a UNet architecture augmented with Mamba blocks, enabling accurate segmentation of ambiguous glomerular targets through both fine-grained edge semantics and long-range spatial dependencies while maintaining linear computational complexity.

The main contributions of this work are as follows:

(1) Architectural Innovation: We propose BM-UNet, a symmetric boundary-enhanced framework that successfully integrates camouflaged object detection principles with efficient Mamba state-space modeling, specifically addressing the medical camouflage phenomenon in glomerular segmentation.

(2) Technical Components: The following four synergistic modules are designed to tackle specific camouflage challenges: Multi-scale Adaptive Fusion (MAF) for morphological variations, Hybrid Boundary Detection (HBD) combining structured and learnable edge processing, Boundary-guided Attention (BGA) for feature enhancement, and Mamba-based Fused Decoder Block (MFDB) for reconstruction, achieving linear computational complexity while maintaining a superior accuracy.

(3) Methodological Advancement: This is the first successful combination of explicit boundary guidance with efficient state-space models for medical imaging, introducing a dual-supervision mechanism that preserves both segmentation accuracy and computational efficiency.

(4) Clinical Validation: Comprehensive experimental validation demonstrates a 92.4–95.3% mIoU across different CKD pathological conditions, with a 4.57% improvement over baseline methods at 113.7 FPS, proving strong potential for real-time clinical deployment in challenging camouflage scenarios.

2. Related Work

2.1. Medical Image Segmentation

CNN-based Methods: Various improved architectures based on convolutional neural networks have been developed for medical image segmentation. Ronneberger et al.’s U-Net [15] established the encoder–decoder framework, which became the foundation for medical image analysis. Huang et al. proposed UNet3+ [16] with dense skip connections to enhance feature propagation, but it has significantly increased computational demands. Zhao et al.’s PSPNet [17] introduced pyramid pooling for multi-scale context aggregation, but it faces limitations when processing irregular pathological structures. DeepLabV3+ employs atrous convolutions to improve boundary detection; however, fixed dilation rates struggle to adapt to variable boundary patterns in diseased tissues. Liu et al.’s ConvNeXt [18] modernized traditional CNN design through large kernel convolutions and inverted bottlenecks, but it still lacks specialized mechanisms for ambiguous boundaries.

Transformer-based Methods: Attention mechanisms have introduced new processing capabilities for medical segmentation. Cao et al.’s Swin-UNet [19] uses hierarchical shifted windows to reduce computational complexity while maintaining global context understanding. Despite these improvements, it still requires substantial computational resources for standard medical images. Xie et al.’s SegFormer [20] achieves more efficient processing by eliminating positional encoding. Its global attention mechanism processes all regions equally, which may dilute focus on critical boundary areas where precise segmentation matters most.

These methods share a common characteristic: they process image features uniformly without explicitly distinguishing boundary regions from other areas. This becomes problematic when dealing with pathological images where diseased and healthy tissues have similar visual appearances.

2.2. Glomerular Segmentation Applications

Glomerular segmentation presents unique challenges that require specialized approaches. Kannan et al. [21] developed one of the first deep learning methods for extracting glomeruli from trichrome-stained images. Their method shows promise but requires careful preprocessing and struggles with severely sclerotic glomeruli where boundaries become indistinct. Liu et al. [22] created an AI framework for assessing nodular sclerosis severity. The system improves quantitative analysis but relies on predefined features that may not generalize well when tissue boundaries become ambiguous due to pathological changes. Bueno et al. [23] used SegNet and adapted AlexNet for glomerular classification. Their binary classification (normal vs. sclerotic) provides useful diagnostic information but does not capture the continuous spectrum of pathological changes important for disease staging.

Recent work has incorporated more advanced architectures. Gallego et al. [24] applied U-Net to different staining protocols, achieving good results on PAS-stained samples. However, the model struggles to distinguish between staining variations and actual tissue boundaries. Feng et al.’s ConvWin-UNet [25] combines convolutions with transformers to improve local feature processing. This hybrid approach enhances accuracy but doubles memory requirements, limiting practical batch sizes. Yan et al. [26] specifically addressed boundary ambiguity using multi-class transformers with attention mechanisms. Their approach aligns well with boundary challenges, but quadratic attention complexity limits practical deployment in clinical environments, failing to meet the efficiency requirements of real-time diagnostic applications.

Current glomerular segmentation methods demonstrate a trade-off between accuracy and efficiency. Methods achieving a high accuracy often require extensive computational resources, while efficient approaches may sacrifice the boundary precision needed for reliable diagnosis.

2.3. Camouflaged Object Detection Applications

Camouflaged object detection (COD) techniques from computer vision provide insights for medical imaging, as both fields deal with targets that blend in with their backgrounds. These methods have developed sophisticated strategies for identifying subtle boundaries. Fan et al.’s SINetV2 [27] uses search and identification mechanisms to detect concealed objects. The approach works well for natural images where texture differences exist, but medical tissues often share similar textures due to their common biological origin. Mei et al.’s PFNet [28] employs polarization-aware features that are not available in standard medical imaging. Pang et al.’s ZoomNet [29] implements multi-scale zoom mechanisms. While effective for natural scenes, maintaining multiple resolution streams requires significant memory resources. Sun et al.’s BGNet [30] explicitly generates boundary maps to guide segmentation. This approach aligns well with medical needs, though boundary detection relies primarily on learned features without incorporating anatomical constraints. Zheng et al.’s BiRefNet [31] improves boundary precision through bidirectional refinement. The iterative processing provides excellent results, but increases inference time considerably.

COD techniques have demonstrated significant advantages in handling visual camouflage phenomena, providing technical references for similar challenges in medical imaging. The medical field has begun exploring the application potential of COD principles in addressing tissue camouflage problems. Ji et al. applied ERRNet [32] to polyp segmentation, achieving a high accuracy through edge–context fusion, though complex scenarios required substantial computational resources. He et al.’s method [33] performed well in PET/CT tumor segmentation, validating the practical value of COD technology in complex medical scenarios.

These COD-inspired medical methods demonstrate the value of explicit boundary detection for challenging segmentation tasks. However, their computational requirements often limit practical deployment in clinical settings where processing speed matters.

2.4. State-Space Models in Medical Imaging

State-space models, particularly Mamba, offer promising solutions for efficient sequence processing. These models achieve linear complexity O(n) while maintaining global receptive fields, addressing the computational limitations of transformer architectures.

Ma et al.’s U-Mamba [14] integrates Mamba blocks within the U-Net framework. This combination maintains segmentation accuracy while improving processing speed by approximately three times. The architecture processes features efficiently but does not include specific mechanisms for boundary enhancement. Ruan et al.’s VM-UNet [34] and Zheng et al.’s Mamba-UNet [35] further demonstrate the effectiveness of state-space models in medical imaging. Both achieve competitive results on standard benchmarks with improved efficiency. These Mamba-based architectures primarily focus on computational efficiency. They successfully reduce processing time but do not specifically address the boundary detection challenges present in pathological image segmentation. Their linear complexity makes them attractive for real-time applications, yet they miss opportunities to leverage sequential processing for progressive boundary refinement. Current medical applications treat Mamba as an efficient alternative to attention mechanisms. While this improves speed, it does not fully exploit Mamba’s capabilities in handling challenging segmentation scenarios where boundary precision is critical.

The existing literature reveals a gap: efficient architectures across different paradigms—whether CNN-based (ConvNeXt), Transformer-based (SegFormer), or Mamba-based models—achieve computational efficiency but lack explicit boundary guidance. Meanwhile, COD methods like BGNet demonstrate boundary detection’s value but require excessive computational resources. This motivates our development of BM-UNet, which uniquely combines Mamba’s linear complexity with COD-inspired boundary mechanisms.

3. Methodologies

We propose BM-UNet, an enhanced UNet architecture that achieves effective medical image segmentation by integrating explicit boundary detection with efficient state-space modeling. The framework adopts the classical encoder–decoder paradigm while introducing specialized components for edge-aware processing that address the unique challenges of camouflaged medical structures. Our motivation stems from an important observation in medical image segmentation: traditional encoder–decoder architectures transmit feature information through skip connections. However, this implicit transmission approach struggles to ensure effective preservation of boundary-critical information throughout the processing pipeline. This is particularly problematic for camouflaged medical structures where subtle boundary differences determine diagnostic accuracy. Our approach addresses this limitation by establishing explicit edge guidance mechanisms throughout the UNet architecture, ensuring that clinicians can visualize and verify how detected edges influence the final segmentation.

The framework is built on a Mamba backbone network and integrates the following four core components: (1) Multi-scale Adaptive Fusion (MAF) for handling irregular tissue morphology, (2) Hybrid Boundary Detection (HBD) combining Sobel operators with adaptive Mamba enhancement, (3) Boundary-guided Attention (BGA) for boundary-aware refinement, and (4) a Mamba Fused Decoder Block (MFDB) for reconstruction.

As illustrated in Figure 1, our BM-UNet framework processes input images through a systematic pipeline. The Mamba encoder first extracts hierarchical features at multiple scales, which are then adaptively fused by MAF modules to handle irregular pathological morphologies. Concurrently, the HBD module generates explicit boundary maps by combining structured edge operators with learnable components. These boundary maps guide the BGA modules to refine feature representations at each scale, emphasizing boundary-relevant regions while suppressing background noise. Finally, MFDBs in the decoder pathway integrate the boundary-enhanced features with global semantic information through progressive upsampling, producing precise segmentation masks that preserve fine structural details.

3.1. Mamba Backbone

Our backbone selection prioritizes the global context modeling capability essential for detecting subtle boundary transitions in camouflaged tissues, where local processing may miss critical long-range dependencies. Traditional CNN backbones exhibit limited receptive fields, while transformer-based approaches incur prohibitive computational costs for clinical deployment due to quadratic attention mechanisms. We employ a Mamba backbone that utilizes selective state-space modeling to achieve global receptive fields with linear complexity, making it particularly suitable for the encoder pathway of our UNet architecture.

The framework begins with standardized input preprocessing to ensure consistent feature representation across diverse medical imaging modalities, as follows:

$${\tilde{\mathcal{X}}}^{\left(0\right)}={\displaystyle \frac{\mathrm{I}-\mu }{\sigma }}\in {\mathbb{R}}^{B\times C\times H\times W}$$

(1)

where $\mathrm{I}$ represents the input medical image and $\mu$ and $\sigma$ denote ImageNet normalization parameters ensuring stable gradient flow during training. The hierarchical Mamba backbone network consists of four stages with embedding dimensions of $64,128,320,448$ and depths of $3,4,6,3$. Multi-scale feature decomposition extracts hierarchical representations through the stem module, as follows:

$${\stackrel{\mathcal{ˆ}}{\mathcal{F}}}^{(l)}\forall l=1,\dots ,L={\mathsf{\Phi }}_{\mathrm{stem} }({\tilde{\mathcal{X}}}^{\left(0\right)}),{\stackrel{\mathcal{ˆ}}{\mathcal{F}}}^{\left(l\right)}\in {\mathbb{R}}^{B\times d_l\times H_l\times W_l}$$

(2)

The selective state-space mechanism forms the core of our linear-complexity processing. Each stage processes input sequences through selective parameterization that preserves critical boundary information, as follows:

$${\tilde{\mathsf{\Delta }}}^{\left(l\right)},{\tilde{\mathrm{B}}}^{\left(l\right)},{\tilde{\mathrm{C}}}^{\left(l\right)}={\mathsf{\Psi }}_{\mathrm{s}\mathrm{s}\mathrm{m}}\left({\stackrel{\mathcal{ˆ}}{\mathcal{F}}}^{\left(l\right)}\right),{\tilde{\mathsf{\Delta }}}^{\left(l\right)}\in {\mathbb{R}}^{B\times L\times D}$$

(3)

where ${\tilde{\mathsf{\Delta }}}^{\left(l\right)}$ represents selective time-step parameters enabling the adaptive temporal modeling crucial for capturing tissue variations. The discretization process transforms continuous parameters into computationally efficient discrete forms while maintaining numerical stability, as follows:

$${\overline{\mathrm{A}}}^{\left(l\right)}=\mathrm{e}\mathrm{x}\mathrm{p}({\tilde{\mathsf{\Delta }}}^{\left(l\right)}\odot \mathrm{A}),{\overline{\mathrm{B}}}^{\left(l\right)}={\tilde{\mathsf{\Delta }}}^{\left(l\right)}\odot {\tilde{\mathrm{B}}}^{\left(l\right)}$$

(4)

The selective scanning mechanism updates hidden states through consistent processing that maintains boundary-critical information across sequences, as follows:

$${\widehat{\mathrm{h}}}_t^{\left(l\right)}={\overline{\mathrm{A}}}_t^{\left(l\right)}\odot {\widehat{\mathrm{h}}}_{t-1}^{\left(l\right)}+{\overline{\mathrm{B}}}_t^{\left(l\right)}\odot {\tilde{\mathrm{x}}}_t^{\left(l\right)}$$

(5)

Output projection integrates global context with local residual connections, ensuring the stable gradient propagation essential for medical segmentation tasks where feature ambiguity challenges conventional approaches, as follows:

$${\breve{\mathcal{Y}}}^{\left(l\right)}={\mathrm{W}}_{\mathrm{out} }\left({\tilde{\mathrm{C}}}^{\left(l\right)}{\widehat{\mathrm{h}}}^{\left(l\right)}+\mathrm{D}{\tilde{\mathrm{x}}}^{\left(l\right)}\right)+{\stackrel{\mathcal{ˆ}}{\mathcal{F}}}^{\left(l\right)}$$

(6)

The selective mechanism preserves boundary-critical information when conventional CNN features become unreliable due to tissue camouflage. The hierarchical feature extraction maintains balanced computational distribution across processing stages, with each level contributing to boundary-aware representation learning, as follows:

$${\widehat{\mathcal{H}}}_i={\mathrm{M}\mathrm{a}\mathrm{m}\mathrm{b}\mathrm{a}}^{\left(i\right)}\left({\mathrm{DownSample}}_{i-1}\left({\widehat{\mathcal{H}}}_{i-1}\right)\right),i\in \mathrm{1,2},\mathrm{3,4}$$

(7)

To ensure balanced processing throughout the backbone, we introduce computational load normalization, which maintains balanced resource allocation, as follows:

$${\tilde{\mathcal{H}}}_i^{\mathrm{norm}}=\mathrm{L}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\left({\widehat{\mathcal{H}}}_i\right)\cdot \sqrt{{\displaystyle \frac{d_{\mathrm{model}}}{d_i}}}$$

(8)

This formulation allows for adaptive receptive field adjustment based on tissue characteristics while preserving information balance, essential for efficiently processing whole-slide imaging data with maintained boundary-critical information throughout hierarchical feature extraction.

3.2. MAF

Following hierarchical feature extraction from the Mamba backbone, each encoder stage output requires adaptive enhancement to handle irregular pathological morphologies. Each MAF module internally handles variable inputs based on hierarchical position: MAF₁ concatenates two inputs (Samba₁ + Samba₂), MAF_2–3 concatenate three inputs each (current Samba + next Samba + previous MAF), and MAF₄ concatenates two inputs (Samba₄ + MAF₃). This adaptive input configuration enables progressive multi-scale feature enhancement while maintaining architectural consistency.

After internal concatenation, all MAF modules process the unified features through three coordinated pathways, as illustrated in Figure 2. The deformable convolution pathway adapts to irregular anatomical shapes through learned spatial transformations, as follows:

$${\tilde{\delta }}_{\mathrm{offset}},{\tilde{\alpha }}_{\mathrm{mod}}={\mathsf{\Theta }}_{\mathrm{dcn}}\left({\tilde{\mathcal{H}}}_i^{\mathrm{norm}}\right),\tilde{\delta }\in {\mathbb{R}}^{2KN},\tilde{\alpha }\in {\mathbb{R}}^{KN}$$

(9)

where ${\tilde{\mathit{\delta }}}_{\mathrm{offset} }$ denotes spatial offsets for dynamic kernel positioning and ${\tilde{\mathit{\alpha }}}_{\mathrm{mod} }$ controls feature importance through adaptive modulation. The adaptive kernel sampling mechanism enables precise adaptation to pathological variations, as follows:

$${\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{c}\mathrm{n}}=\sum _{k=1}^K {\mathrm{w}}_k\cdot {\tilde{\mathcal{H}}}_i^{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\left({\mathrm{p}}_k+{\tilde{\delta }}_k\right)\cdot {\tilde{\alpha }}_k$$

(10)

Parallel directional processing captures elongated structures through separate horizontal and vertical convolutions, as follows:

$${\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}^h={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{1\times 5}\left({\tilde{\mathcal{H}}}_i^{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\right),{\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}^v={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{5\times 1}\left({\tilde{\mathcal{H}}}_i^{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}\right)$$

(11)

The directional features are then integrated while preserving spatial consistency, as follows:

$${\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}={\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}^h\oplus {\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}^v+{\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{3\times 3}\left({\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}^h\odot {\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}^v\right)$$

(12)

Multi-scale atrous convolutions capture hierarchical contexts with preserved geometric relationships, as follows:

$${\widehat{\mathcal{F}}}_{\mathrm{m}\mathrm{s}}=\underset{r\in \mathcal{R}}{⨁} \mathrm{AtrousConv}\left({\tilde{\mathcal{H}}}_i^{\mathrm{norm}};\mathrm{rate}=r\right),\mathcal{R}=\mathrm{2,3},4$$

(13)

The context gating mechanism provides adaptive feature selection through learnable weighting, as follows:

$${\tilde{\gamma }}_{\mathrm{c}\mathrm{t}\mathrm{x}}={\sigma }_{\mathrm{h}\mathrm{a}\mathrm{r}\mathrm{d}}\left({\mathrm{W}}_g\cdot \left[{\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{c}\mathrm{n}}\left|{\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}}\right|{\widehat{\mathcal{F}}}_{\mathrm{m}\mathrm{s}}\right]\right)$$

(14)

where ${\sigma }_{\mathrm{hard} }$ denotes Hard-sigmoid activation ensuring stable gradient flow. The final adaptive feature aggregation preserves spatial detail integrity, as follows:

$${\check{\mathcal{H}}}_i={\tilde{\gamma }}_{\mathrm{c}\mathrm{t}\mathrm{x}}\odot \mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left(\left[{\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{c}\mathrm{n}},{\widehat{\mathcal{F}}}_{\mathrm{d}\mathrm{i}\mathrm{r}},{\widehat{\mathcal{F}}}_{\mathrm{m}\mathrm{s}}\right]\right)+{\tilde{\mathcal{H}}}_i^{\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}}$$

(15)

3.3. HBD

The enhanced features ${\check{\mathcal{H}}}_i$ from MAF serve as inputs to the boundary detection mechanism. Current COD methods rely solely on learned edge representations that lack the geometric constraints essential for medical boundary detection. While deep networks learn complex patterns from training data, they often fail with pathological variations unseen during training, making structured operators necessary for robust boundary detection. Our HBD module coordinates structured gradient computation with Mamba sequence modeling, establishing explicit boundary guidance that maintains interpretability while capturing global context.

For fine-scale features ${\check{\mathcal{H}}}_1$ containing rich spatial details, structured gradient computation employs Sobel operators that preserve directional consistency, as follows:

$${\tilde{\mathrm{G}}}_x={\mathrm{S}}_x\ast {\check{\mathcal{H}}}_1,{\tilde{\mathrm{G}}}_y={\mathrm{S}}_y\ast {\check{\mathcal{H}}}_1$$

(16)

where ${\mathrm{S}}_x$ and ${\mathrm{S}}_y$ represent structured Sobel kernels. Gradient magnitude and orientation analysis provides comprehensive edge characterization, as follows:

$$\left|\tilde{\mathrm{G}}\right|=\sqrt{{\tilde{\mathrm{G}}}_x^2+{\tilde{\mathrm{G}}}_y^2+ϵ},{\tilde{\theta }}_G=\mathrm{a}\mathrm{r}\mathrm{c}\mathrm{t}\mathrm{a}\mathrm{n}\left({\tilde{\mathrm{G}}}_y/{\tilde{\mathrm{G}}}_x\right)$$

(17)

Edge enhancement integrates structured gradient information through learnable weighting, as follows:

$${\widehat{\mathcal{E}}}_{\mathrm{low} }={\check{\mathcal{H}}}_1+\tilde{\beta }\odot {\mathsf{\Phi }}_{1\times 1}\left(\left|\tilde{\mathrm{G}}\right|\right)$$

(18)

where $\tilde{\beta }$ represents learnable gradient enhancement coefficients. Non-maximum suppression refines edge responses, ensuring a geometric consistency that remains reliable across different pathological conditions, as follows:

$${\breve{\mathcal{E}}}_{\mathrm{low}}=\mathrm{N}\mathrm{M}\mathrm{S}\left({\widehat{\mathcal{E}}}_{\mathrm{low}},{\tilde{\theta }}_G,{\tau }_{\mathrm{threshold}}\right)$$

(19)

For high-level features ${\mathfrak{H}}_4$ rich in semantic information, spatial-to-sequence transformation prepares features for Mamba processing, as follows:

$${\tilde{\mathcal{S}}}_{\mathrm{seq}}=\mathsf{\Lambda }\left({\check{\mathcal{H}}}_4\right)=\mathrm{R}\mathrm{e}\mathrm{s}\mathrm{h}\mathrm{a}\mathrm{p}\mathrm{e}\left({\check{\mathcal{H}}}_4,\left(B,HW,C\right)\right)$$

(20)

Mamba processing captures global contextual relationships, as follows:

$${\widehat{\mathcal{E}}}_{\mathrm{high}}^{\mathrm{seq}}=\mathrm{M}\mathrm{a}\mathrm{m}\mathrm{b}\mathrm{a}\mathrm{L}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\left(\mathrm{LayerNorm}\left({\tilde{\mathcal{S}}}_{\mathrm{seq}}\right)\right)$$

(21)

Sequence-to-spatial transformation reconstructs spatial arrangements, as follows:

$${\breve{\mathcal{E}}}_{\mathrm{high} }=\mathsf{\Gamma }\left({\widehat{\mathcal{E}}}_{\mathrm{high} }^{\mathrm{seq} }\right)=\mathrm{R}\mathrm{e}\mathrm{s}\mathrm{h}\mathrm{a}\mathrm{p}\mathrm{e}\left({\widehat{\mathcal{E}}}_{\mathrm{high} }^{\mathrm{seq} },\left(B,C,H,W\right)\right)$$

(22)

Cross-scale fusion integrates enhanced edge features with context-aware representations, as follows:

$${\tilde{\mathcal{P}}}_{\mathrm{edge}}={\mathsf{\Phi }}_{\mathrm{edge}}\left({\mathrm{w}}_{\mathrm{up}}\cdot \mathrm{U}\mathrm{p}\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{l}\mathrm{e}\left({\breve{\mathcal{E}}}_{\mathrm{high}}\right)+{\mathrm{w}}_{\mathrm{low}}\cdot {\breve{\mathcal{E}}}_{\mathrm{low}}\right)$$

(23)

Figure 3 illustrates how the HBD module’s dual-pathway design combines structured gradient computation with adaptive sequence modeling for robust boundary detection.

3.4. BGA

The boundary predictions ${\tilde{\mathcal{P}}}_{\mathrm{edge} }$ from HBD guide feature enhancement across multiple scales. Standard feature processing lacks explicit boundary guidance, limiting effectiveness for precise edge delineation. BGA leverages boundary predictions to perform attention-guided feature enhancement, creating boundary-aware representations specifically adapted for camouflaged structure segmentation.

Boundary attention weights are computed through spatial probability distribution, as follows:

$${\tilde{\mathrm{A}}}_{\mathrm{edge} }=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\tilde{\mathcal{P}}}_{\mathrm{edge} }\right)$$

(24)

Attention refinement through morphological operations reduces noise and enhances boundary coherence by filling gaps in fragmented edge responses common in camouflaged structures, as follows:

$${\widehat{\mathrm{A}}}_{\mathrm{refined} }=\mathrm{M}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{h}\mathrm{C}\mathrm{l}\mathrm{o}\mathrm{s}\mathrm{e}\left(\mathrm{M}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{h}\mathrm{O}\mathrm{p}\mathrm{e}\mathrm{n}\left({\tilde{\mathrm{A}}}_{\mathrm{edge} },{\mathcal{K}}_{\mathrm{small} }\right),{\mathcal{K}}_{\mathrm{large} }\right)$$

(25)

Multi-scale boundary guidance applies edge-aware attention through coordinated enhancement, as follows:

$${\breve{\mathcal{H}}}_i={\check{\mathcal{H}}}_i\odot \mathrm{R}\mathrm{e}\mathrm{s}\mathrm{i}\mathrm{z}\mathrm{e}\left({\widehat{\mathrm{A}}}_{\mathrm{refined} },\left(H_i,W_i\right)\right)+{\check{\mathcal{H}}}_{\mathrm{i}}$$

(26)

where bilinear interpolation adjusts attention maps to corresponding spatial dimensions. Feature refinement through specialized convolution layers ensures stable gradient flow, as follows:

$${\overline{\mathcal{H}}}_i={\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{v}}_{3\times 3}\left(\mathrm{B}\mathrm{N}\left(\mathrm{R}\mathrm{e}\mathrm{L}\mathrm{U}\left({\breve{\mathcal{H}}}_i\right)\right)\right)$$

(27)

The BGA mechanism (Figure 4) creates a feedback loop between boundary detection and feature enhancement, ensuring boundary-aware representations throughout the processing pipeline.

3.5. MFDB

The boundary-aware encoder features ${\overline{\mathcal{H}}}_i$ serve as skip connections to the decoder pathway. Standard decoders process features uniformly without considering the boundary-specific requirements essential for medical diagnosis. MFDB employs SambaMLPBlock processing that combines Mamba’s global modeling capabilities with multi-scale MLP local processing through balanced pathway design.

The SambaMLPBlock processes input features ${\mathcal{F}}_{\mathrm{in} }$ through dual balanced pathways. The Mamba pathway enables global sequence modeling, as follows:

$${\tilde{\mathcal{F}}}_{\mathrm{seq}}=\mathrm{M}\mathrm{a}\mathrm{m}\mathrm{b}\mathrm{a}\mathrm{L}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\left(\mathrm{L}\mathrm{N}\left(\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\left({\mathcal{F}}_{\mathrm{in}}\right)\right)\right)$$

(28)

The multi-scale MLP pathway applies parallel convolutions with different kernel sizes, as follows:

$${\widehat{\mathcal{F}}}_{\mathrm{par} }^{\left(k\right)}={ \mathrm{DWConv} }_{k\times k}\left({\mathcal{F}}_{\mathrm{in} }\right),\mathrm{k}\in \mathrm{1,3},\mathrm{5,7}$$

(29)

Parallel feature integration combines multi-scale local representations, as follows:

$${\widehat{\mathcal{F}}}_{\mathrm{p}\mathrm{a}\mathrm{r}}=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}\left(\left[{\widehat{\mathcal{F}}}_{\mathrm{p}\mathrm{a}\mathrm{r}}^{\left(1\right)},{\widehat{\mathcal{F}}}_{\mathrm{p}\mathrm{a}\mathrm{r}}^{\left(3\right)},{\widehat{\mathcal{F}}}_{\mathrm{p}\mathrm{a}\mathrm{r}}^{\left(5\right)},{\widehat{\mathcal{F}}}_{\mathrm{p}\mathrm{a}\mathrm{r}}^{\left(7\right)}\right]\right)$$

(30)

SambaMLPBlock fusion integrates global and local representations, as follows:

$${\breve{\mathcal{F}}}_{\mathrm{s}\mathrm{m}\mathrm{b}}=\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{j}\left({\tilde{\mathcal{F}}}_{\mathrm{s}\mathrm{e}\mathrm{q}}\oplus {\widehat{\mathcal{F}}}_{\mathrm{p}\mathrm{a}\mathrm{r}}\right)+{\mathcal{F}}_{\mathrm{i}\mathrm{n}}$$

(31)

Progressive upsampling combines enhanced features with boundary-aware skip connections, as follows:

$${\tilde{\mathcal{D}}}_i=\mathrm{SambaMLPBlock}\left(\left[\mathrm{U}\mathrm{p}\left({\tilde{\mathcal{D}}}_{i+1}\right)\mid {\overline{\mathcal{H}}}_i\right]\right)$$

(32)

where skip connections from BGA-enhanced encoder features ensure boundary-aware information propagation. The framework generates dual predictions for comprehensive boundary-aware segmentation, as follows:

$${\widehat{\mathcal{M}}}_{\mathrm{seg}}=\mathrm{S}\mathrm{i}\mathrm{g}\mathrm{m}\mathrm{o}\mathrm{i}\mathrm{d}\left({\mathsf{\Phi }}_{\mathrm{seg}}\left({\tilde{\mathcal{D}}}_1\right)\right)$$

(33)

$${\widehat{\mathcal{M}}}_{\mathrm{edge}}=\mathrm{S}\mathrm{i}\mathrm{g}\mathrm{m}\mathrm{o}\mathrm{i}\mathrm{d}\left({\mathsf{\Phi }}_{\mathrm{edge}}\left({\tilde{\mathcal{P}}}_{\mathrm{edge}}\right)\right)$$

(34)

The MFDB architecture (Figure 5) integrates global sequence modeling with local multi-scale processing, enabling efficient boundary-preserving reconstruction.

3.6. Loss Function

The dual-task nature of BM-UNet requires a carefully designed loss function that simultaneously optimizes segmentation accuracy and boundary detection precision. Our loss formulation addresses the class imbalance inherent in medical image segmentation while ensuring that boundary-critical information is preserved throughout training.

The total loss function balances segmentation and boundary objectives through joint supervision, as follows:

$${\mathcal{L}}_{\mathrm{total} }={\mathcal{L}}_{\mathrm{seg} }+{\lambda }_e{\mathcal{L}}_{\mathrm{edge} }$$

(35)

where ${\lambda }_e$ controls the relative importance of boundary supervision. Through systematic validation on the development set, we find that ${\lambda }_e$ = 0.5 provides an optimal balance between segmentation accuracy and boundary precision (

Table 1. Loss function weight optimization analysis.

λe	Seg mIoU	Boundary mIoU	Training Behavior
0.1	93.72	89.84	Boundary under-optimized
0.3	93.64	90.67	Slight improvement
0.5	93.51	91.34	Optimal balance
0.7	93.28	91.52	Segmentation degradation
0.9	92.94	91.43	Over-emphasis on boundaries

). Values below 0.3 lead to under-optimized boundary detection, while values above 0.7 cause segmentation performance degradation.

Each loss component combines Dice loss and focal loss to address both region overlap optimization and class imbalance challenges. The segmentation loss ${\mathcal{L}}_{\mathrm{seg} }$ is formulated as follows:

$${\mathcal{L}}_{\mathrm{seg} }={\mathcal{L}}_{\mathrm{dice} }^{\mathrm{seg} }+\alpha {\mathcal{L}}_{\mathrm{focal} }^{\mathrm{seg} }$$

(36)

The Dice loss optimizes region overlap by measuring intersection over union, as follows:

$${\mathcal{L}}_{\mathrm{dice} }^{\mathrm{seg} }=1-{\displaystyle \frac{2{\sum }_i \left(p_i\cdot g_i\right)+ϵ}{{\sum }_i p_i+{\sum }_i g_i+ϵ}}$$

(37)

where $p_i$ represents predicted probabilities, $g_i$ denotes ground truth labels, and $ϵ=1\times {10}^{-7}$ ensures numerical stability. The focal loss addresses class imbalance by down-weighting easy examples, as follows:

$${\mathcal{L}}_{\mathrm{focal} }^{\mathrm{s}\mathrm{e}\mathrm{g}}=-{\alpha }_t{\left(1-p_t\right)}^{\gamma }\log \left(p_t\right)$$

(38)

where ${\alpha }_t=0.25$ balances positive and negative samples and $\gamma =2$ focuses learning on hard examples by reducing the contributions of well-classified instances.

Similarly, the boundary loss ${\mathcal{L}}_{\mathrm{edge} }$ ensures explicit edge supervision using the same formulation applied to boundary predictions from the HBD module, as follows:

$${\mathcal{L}}_{\mathrm{edge} }={\mathcal{L}}_{\mathrm{dice} }^{\mathrm{edge} }+\alpha {\mathcal{L}}_{\mathrm{focal} }^{\mathrm{edge} }$$

(39)

4. Experiments Results

4.1. Implement Details

Experiments were implemented using the using the PyTorch 1.8.0 framework with Python 3.8.10 with the proposed Mamba backbone architecture. Input images were uniformly resized to a 512 × 512 resolution and underwent standard data augmentation strategies, including random horizontal/vertical flipping (probability p = 0.5), dynamic resizing, random cropping, and ImageNet normalization (mean μ = [0.485, 0.456, 0.406], standard deviation σ = [0.229, 0.224, 0.225]). Training used the Adam optimizer with a batch size of 16 and initial learning rate of 1 × 10⁻⁴, following a polynomial decay strategy with a power of 0.9. All experiments were conducted on an NVIDIA GeForce RTX 4090 GPU, with training completed within 1–2 h over 200 epochs.

The complete preprocessing and augmentation pipeline is detailed in

Table 2. Dataset preprocessing and augmentation pipeline.

Processing Step	Parameters	Application	Dataset Size Impact
Image Resizing	512 × 512 pixels	All images	No change
Random Horizontal Flip	Probability p = 0.5	Training only	Online augmentation
Random Vertical Flip	Probability p = 0.5	Training only	Online augmentation
Random Cropping	Dynamic crop after resizing	Training only	Online augmentation
ImageNet Normalization	μ = [0.485, 0.456, 0.406]; σ = [0.229, 0.224, 0.225]	All images	No change

, with particular attention to maintaining a consistent image quality while preventing overfitting through diverse augmentation strategies.

4.2. Datasets

We evaluated our method on two glomerular datasets covering distinct imaging challenges and pathological conditions.

KPIs2024 Dataset: The first dataset, KPIs2024 [36], was specifically designed for glomerular segmentation challenges and provides comprehensive coverage of different CKD pathological stages. The KPIs2024 challenge dataset consists of 60 high-resolution whole-kidney section images from one normal kidney model and three CKD models, as follows: eight-week-old healthy controls, mice 12 weeks post-5/6 nephrectomy (simulating chronic renal failure model), eNOS⁻/⁻/lepr (db/db) double-knockout mice (simulating diabetic nephropathy model, DN group), and NEP25 transgenic mice (three weeks after immunotoxin-induced glomerular injury). All samples are stained with PAS and photographed using a Leica SCN400 scanner (Leica Microsystems, Wetzlar, Germany) at 40× magnification.

Table 3. KPIs2024 dataset composition and characteristics.

Condition	Resolution	Training	Test	Total	Disease Model	Staining
Normal	512 × 512	980	327	1307	Healthy control	PAS
56Nx	512 × 512	979	327	1306	5/6 nephrectomy	PAS
DN	512 × 512	975	325	1300	Diabetic model	PAS
NEP25	512 × 512	976	325	1301	Podocyte injury	PAS
Total	512 × 512	3910	1304	5214	4 models	PAS

summarizes the composition and characteristics across these four distinct pathological conditions.

HuBMAP Dataset: To assess its generalization capability across different tissue types and imaging protocols, we also evaluated our method on the HuBMAP dataset [37]. This dataset provides human tissue samples that differ significantly from the mouse models in KPIs2024, offering an important validation of cross-species robustness. The publicly available HuBMAP dataset contains comprehensive biomedical resources designed to capture the cellular-level molecular structure of human organs, including high-resolution kidney tissue images with complex renal tissue structures and the subtle morphology of glomeruli.

Table 4. HuBMAP dataset specifications.

Split	Resolution	Count	Content	Annotation
Training	512 × 512	2576	Human kidney	Expert
Test	512 × 512	1104	Human kidney	Expert
Total	512 × 512	3680	PAS-stained	Manual

presents the specifications and characteristics of the HuBMAP dataset.

Figure 6 illustrates the camouflage challenges inherent in both datasets. In KPIs2024 histopathological images, pathological glomeruli exhibit visual similarity to surrounding tissues, particularly in DN and NEP25 conditions where structural boundaries become increasingly ambiguous. HuBMAP samples demonstrate similar challenges in human tissue, where glomerular morphological variations across different pathologies complicate automated detection. These visualizations highlight the fundamental segmentation difficulty that BM-UNet addresses through edge-guided boundary detection.

4.3. Evaluation Metrics

We assess performance using four metrics targeting clinical deployment requirements. Mean intersection over union (mIoU) evaluates segmentation accuracy through mask overlap calculation. Computational efficiency is measured via FLOPs (G) for operation complexity, Params (M) for model size, and FPS for processing speed. These metrics address the fundamental trade-off between segmentation precision and computational feasibility that determines clinical viability in resource-constrained medical environments.

4.4. Comparative Experiments

4.4.1. Quantitative Performance Analysis

To comprehensively evaluate BM-UNet’s effectiveness in medical image segmentation, we compared it against multiple representative architectures covering mainstream technical directions.

CNN-based Methods: We selected U-Net3+ (encoder–decoder baseline), PSPNet (introducing pyramid pooling for multi-scale context), DeepLabV3+ (utilizing dilated convolutions for boundary refinement), UNet3+ (enhancing feature propagation through dense skip connections), and ConvNeXt (designed with larger kernels and inverted bottlenecks). These models represent both foundational and state-of-the-art CNN-based approaches.

Transformer-based Methods: We compared against advanced attention-based models, including Swin-UNet (employing hierarchical vision Transformer with shifted window attention) and SegFormer (providing efficient processing without positional encoding requirements). These two methods provide important performance references for our state-space model.

COD-inspired Methods: Given our focus on difficult-to-segment medical structures (similar to “camouflaged targets”), we included the following models designed for COD and adapted for medical images: SINetV2 with search identification mechanisms, PFNet employing polarization-aware feature fusion, ZoomNet implementing multi-scale zoom mechanisms, BiRefNet utilizing bidirectional refinement for boundary precision, and BGNet proposing dedicated boundary guidance networks.

Mamba-based Methods: We compared with CM-UNet, which shares the same core computational paradigm of state-space models but lacks the explicit edge guidance mechanisms present in our framework. The experimental results are shown in

Table 5. Quantitative performance comparison on KPIs2024 dataset with statistical significance analysis.

Model Variant	Architecture	56Nx	DN	NEP25	Normal	Avg mIoU	Std	p-Value †
U-Net	CNN	85.12 ± 0.48	88.34 ± 0.42	80.56 ± 0.54	91.78 ± 0.35	86.45	4.82	<0.001 ***
PSPNet	CNN	85.89 ± 0.46	89.72 ± 0.41	81.93 ± 0.52	92.35 ± 0.33	87.47	4.51	<0.001 ***
DeepLabV3+	CNN	86.19 ± 0.47	91.70 ± 0.39	81.45 ± 0.51	93.42 ± 0.31	88.19	5.45	<0.001 ***
UNet3+	CNN	88.32 ± 0.42	90.71 ± 0.40	83.73 ± 0.49	93.81 ± 0.30	89.14	4.25	<0.001 ***
ConvNeXt	CNN	88.45 ± 0.41	91.23 ± 0.38	85.79 ± 0.47	93.28 ± 0.29	89.69	3.12	<0.001 ***
Segformer	Transformer	88.77 ± 0.39	92.77 ± 0.37	87.45 ± 0.44	94.73 ± 0.27	90.93	3.40	0.008 **
SwinUNet	Transformer	87.34 ± 0.45	88.89 ± 0.43	81.72 ± 0.53	92.72 ± 0.32	87.67	4.56	<0.001 ***
SINetV2	COD	89.13 ± 0.38	93.53 ± 0.35	86.57 ± 0.43	94.04 ± 0.26	90.82	3.59	0.011 *
PFNet	COD	87.27 ± 0.43	91.85 ± 0.39	84.84 ± 0.48	93.17 ± 0.30	89.78	3.97	<0.001 ***
ZoomNet	COD	86.34 ± 0.47	86.14 ± 0.50	82.81 ± 0.52	91.35 ± 0.34	86.66	3.70	<0.001 ***
BiRefNet	COD	88.81 ± 0.37	92.82 ± 0.34	88.89 ± 0.42	93.87 ± 0.25	91.10	2.24	0.026 *
BGNet	COD	89.23 ± 0.36	93.67 ± 0.33	89.49 ± 0.40	94.11 ± 0.24	91.63	2.47	0.042 *
CM-UNet	Mamba	90.29 ± 0.33	93.81 ± 0.31	90.87 ± 0.38	94.34 ± 0.23	92.33	1.72	0.049 *
BM-UNet (ours)	COD-Mamba	92.43 ± 0.29	94.02 ± 0.28	92.28 ± 0.35	95.32 ± 0.21	93.51	1.34	-

† p-values from paired t-test comparing against BM-UNet; *** p < 0.001, ** p < 0.01, * p < 0.05.

.

Performance Analysis Across Architectural Paradigms: The results reveal distinct performance patterns that validate our architectural design choices. Efficient architectures within each category—CNN (ConvNeXt: 89.69%), Transformer (SegFormer: 90.93%), and baseline Mamba (CM-UNet: 92.33%)—achieve progressively better performances, yet all fall short of COD-specialized methods with explicit boundary guidance (BGNet: 91.63%). BM-UNet’s superiority (93.51% mIoU) demonstrates that combining boundary guidance with efficient sequence modeling creates the necessary synergy for accurate camouflaged structure segmentation.

Statistical Validation: Paired t-test analysis confirms that all improvements are statistically significant. Comparison with the strongest baseline CM-UNet shows a 1.18% improvement (p = 0.049, 95% CI: [0.01, 2.35]), confirming that the advancement is statistically reliable. The progressive reduction in standard deviation from weaker to stronger methods (U-Net: 0.48% to BM-UNet: 0.29% for 56Nx condition) demonstrates an enhanced consistency alongside improved accuracy.

Challenging Condition Performance: The framework’s effectiveness is most evident in NEP25 conditions where boundary degradation is severe. BM-UNet achieves 92.28% versus CM-UNet’s 90.87%, representing a clinically meaningful improvement of approximately 20–30 fewer boundary misclassifications per whole-slide image. This directly impacts glomerular sclerosis quantification, a key parameter in CKD staging decisions.

Figure 7 presents the performance matrix visualization through a heatmap showing the mIoU (%) across all methods and pathological conditions. BM-UNet achieves the highest values (represented by the darkest red regions) under all conditions, confirming its consistent superiority.

To further validate the discriminative capability of our model, we conduct an ROC analysis for pixel-wise classification across all test samples. Figure 8 shows the ROC curves comparing BM-UNet with the top-performing baseline methods. Our approach achieves the highest AUC (0.948), demonstrating a superior discriminative capability across all operating points compared to competitive methods.

Our method consistently outperforms the baseline approaches across different CKD stages, with particularly notable improvements in challenging scenarios where tissue boundaries are severely compromised.

Cross-dataset Generalization: As shown in

Table 6. Cross-dataset validation on HuBMAP dataset with statistical significance.

Model Variant	Architecture	mPA (%)	mIoU (%)	mDice (%)	p-Value †
U-Net	CNN	96.12 ± 0.13	93.57 ± 0.18	96.71 ± 0.11	<0.001 ***
UNet3+	CNN	96.48 ± 0.12	93.68 ± 0.17	96.77 ± 0.10	<0.001 ***
PSPNet	CNN	95.52 ± 0.15	93.12 ± 0.20	96.49 ± 0.12	<0.001 ***
DeepLabV3plus	CNN	96.25 ± 0.13	93.45 ± 0.18	96.72 ± 0.11	<0.001 ***
ConvNeXt	CNN	96.78 ± 0.11	93.85 ± 0.16	96.81 ± 0.09	0.034 *
Segformer	Transformer	96.55 ± 0.12	93.89 ± 0.16	96.89 ± 0.09	0.046 *
SwinUNet	Transformer	95.23 ± 0.16	92.43 ± 0.21	96.15 ± 0.13	<0.001 ***
BM-UNet (ours)	COD-Mamba	97.12 ± 0.10	94.32 ± 0.15	97.19 ± 0.08	-

† p-values from paired t-test comparing against BM-UNet; *** p < 0.001, * p < 0.05.

, HuBMAP validation demonstrates a robust generalization capability with a 94.32% mIoU. The 0.43% improvement over Segformer (93.89%) maintains statistical significance (p = 0.046), confirming a consistent performance across different imaging protocols and tissue sources. The reduced variance in human tissue samples (standard deviations 0.08–0.21%) reflects the higher image quality typical of standardized acquisition protocols.

4.4.2. Computational Efficiency Analysis

Table 7. Computational efficiency analysis comparing processing speed and resource requirements.

Method	FLOPs (G)	Params (M)	FPS
DeepLabV3plus	46.94	27.23	237.4
Swin-UNet	30.92	27.15	121.1
U-Net3+	179.08	31.03	82.8
Segformer	56.71	27.35	94.3
SINetV2	26.11	24.96	96.2
BiRefNet	37.4	39.29	38.9
CM-UNet	33.05	30.07	119.7
BM-UNet (ours)	43.31	32.42	113.7

reveals BM-UNet’s favorable efficiency–accuracy trade-off while achieving the highest segmentation accuracy.

BM-UNet demonstrates favorable efficiency–accuracy trade-offs while maintaining the highest segmentation accuracy. The linear-complexity Mamba backbone and efficient boundary guidance enable a 113.7 FPS processing speed suitable for real-time clinical deployment. Its computational cost (FLOPs) is 43.31 G, parameter count (Params) is 32.42 M, and processing speed reaches 113.7 FPS (approximately 8.8 ms latency per image). Figure 9’s multi-dimensional evaluation further supports this, as follows: (a) BM-UNet occupies the optimal position in speed–accuracy trade-off; (b) has a relatively low computational cost (FLOPs) with an excellent performance; (c) provides the highest accuracy with a moderate model size; and (d) ranks first in comprehensive efficiency.

Clinical Deployment Validation: To assess practical deployment feasibility, we evaluated BM-UNet on 200 test images across clinical-grade GPUs (

Table 8. Clinical deployment validation across different hardware platforms.

Hardware Platform	Peak Memory Usage (GB)	Total Processing Time (s)	Avg. Time per Image (ms)	Throughput (Images/s)
NVIDIA RTX 4090 (24 GB)	2.3	1.76	8.8	113.6
NVIDIA RTX 3060 (12 GB)	2.3	3.08	15.4	64.9
NVIDIA RTX 2070 (8 GB)	2.3	4.12	20.6	48.5
NVIDIA GTX 1660 Ti (6 GB)	2.3	8.62	43.1	23.2

). The framework maintained 113.6 FPS on RTX 4090, consistent with our benchmark results (113.7 FPS). Peak memory stabilized at 2.3 GB across all platforms, confirming its deployment viability on 6 GB entry-level GPUs. Mid-range hardware (RTX 3060/2070) achieved 48.5–64.9 images/s, sufficient for interactive pathological analysis. Even the GTX 1660 Ti processed 200 images in 8.62 s (23.2 images/s), enabling practical batch processing. This 5× performance range with consistent memory usage demonstrates BM-UNet’s robustness to hardware variations common in clinical settings.

Table 8 reveals BM-UNet’s favorable efficiency–accuracy trade-off while achieving the highest segmentation accuracy.

4.4.3. Qualitative Analysis

Figure 10 demonstrates BM-UNet’s superior boundary precision across diverse histopathological scenarios. Traditional CNN methods exhibit incomplete boundary capture and fragmentation, while COD-inspired approaches show improvements but struggle with completeness. BM-UNet consistently produces masks that closely adhere to actual boundary contours.

Cross-dataset Validation on HuBMAP Samples: Figure 11 shows BM-UNet’s segmentation results on human kidney tissue samples (HuBMAP). Despite the higher cellular heterogeneity and different staining protocols in human samples compared to mouse models, BM-UNet maintains a robust performance with a 94.32% mIoU, validating its clinical applicability potential.

Error Analysis: Figure 12 provides comprehensive error visualization through color coding (false positives: red; false negatives: blue). BM-UNet effectively suppresses both false positive and false negative errors across all pathological conditions, demonstrating the most balanced error distribution. Figure 13 further validates these findings on HuBMAP human tissue samples.

Boundary Precision: Figure 14 illustrates enhanced boundary delineation through direct overlay visualization. Through the direct overlay of segmentation boundaries (green contours) on original histopathological images, the advantages of BM-UNet in boundary delineation precision are intuitively demonstrated. Compared to traditional methods, BM-UNet generates more continuous and accurate boundary contours that closely match actual glomerular structures.

4.5. Ablation Experiment

To systematically evaluate each module’s contribution, we used the pure Mamba backbone network as a baseline and progressively integrated the proposed core modules (MAF, HBD, BGA, and MFDB), with the results shown in

Table 9. Comprehensive ablation study demonstrating the contribution of each BM-UNet component across multiple evaluation metrics.

Method	56Nx			DN			NEP25			Normal
	mIoU	mDice	mPA	mIoU	mDice	mPA	mIoU	mDice	mPA	mIoU	mDice	mPA
Baseline (Mamba Only)	87.8	90.2	92.5	89.51	91.8	93.8	87.47	89.9	92.1	90.94	93.1	94.2
+MAF	88.9	91.3	93.4	90.54	92.9	94.6	88.96	91.4	93.2	91.59	93.9	94.8
+MAF + HBD	90.3	92.8	94.7	91.64	94.1	95.3	90.25	92.7	94.5	93.49	95.2	96.1
+MAF + HBD + BGA	91.5	93.9	95.4	92.99	95.3	96.1	91.50	93.8	95.3	94.09	95.7	96.5
BM-UNet (Full + MFDB)	92.43	94.7	96.1	94.02	96.2	96.8	92.28	94.6	96.0	95.32	96.9	97.2

. This design aims to isolate the effects of boundary guidance mechanisms while avoiding interference from architectural differences.

Figure 15 provides visual evidence of progressive segmentation improvement through systematic module integration. The Mamba-only baseline model exhibits incomplete boundary capture, while the progressive integration of the MAF, HBD, BGA, and MFDB modules demonstrates clear visual improvements in boundary completeness and accuracy.

Baseline Performance: The Mamba-only baseline achieves an 88.93% mIoU, 91.25% mDice, and 93.15% mPA on average, confirming state-space modeling’s effectiveness. However, it struggles most with NEP25 cases (87.47% mIoU) where tissue boundaries are severely degraded.

MAF Module Contribution: Adding MAF improves performance by +1.07% mIoU, +1.13% mDice, and +0.85% mPA on average. The module shows particular effectiveness in handling irregular structures, with NEP25 gaining +1.49% mIoU (from 87.47% to 88.96%). This improvement stems from MAF’s deformable convolutions adapting to pathological shape variations.

HBD Module Impact: The HBD module provides the largest single contribution, adding +1.42% mIoU, +1.33% mDice, and +1.15% mPA beyond MAF. The NEP25 condition benefits most, with +1.29% mIoU improvement (88.96% to 90.25%). This validates our hypothesis that explicit boundary detection is essential when tissue camouflage obscures conventional features. The combination of Sobel operators with Mamba enhancement captures both local gradients and global context.

BGA Module Enhancement: BGA adds +1.10% mIoU, +0.98% mDice, and +0.68% mPA to the MAF + HBD configuration. While the mPA improvement is modest, the mIoU and mDice gains indicate better boundary localization. The module creates effective feedback between detected boundaries and feature refinement, particularly benefiting DN cases (+1.35% mIoU).

MFDB and Full Framework: The complete BM-UNet with MFDB achieves a final performance of 93.51% mIoU, 95.6% mDice, and 96.53% mPA on average. Compared to the baseline, this represents total improvements of +4.58% mIoU, +4.35% mDice, and +3.38% mPA. The NEP25 condition shows the most dramatic improvement (+4.81% mIoU), while even the easier Normal condition gains +4.38% mIoU.

Component Synergy Analysis: Table 9 reveals important patterns in module interactions. The mDice improvements (+4.35% total) closely track mIoU gains (+4.58%), indicating consistent region overlap improvement. However, mPA improvements (+3.38%) are more modest, suggesting that the modules primarily enhance boundary precision rather than overall pixel classification. The standard deviation of improvements across conditions decreases from 1.47% (baseline) to 1.34% (full model), demonstrating a more consistent performance.

Figure 16 illustrates the progressive module integration analysis, demonstrating how each component systematically contributes to the overall performance enhancement. The cumulative performance curve shows steady improvement from the baseline, with each module addition providing measurable gains. The individual contribution bars at the bottom highlight that HBD delivers the most significant improvement (+1.42%), followed by BGA (+1.10%) and MAF (+1.07%), while MFDB provides the final optimization (+0.98%). This visualization confirms that boundary-related modules (HBD and BGA) account for over half of the total performance gain (+2.52% out of +4.57%), validating the importance of explicit boundary guidance in our framework design. The consistent upward trajectory demonstrates effective synergy between components, with no performance degradation observed during progressive integration.

5. Discussion

The experimental results demonstrate that BM-UNet effectively addresses the camouflaged pathological structure segmentation challenge by integrating explicit boundary guidance with efficient state-space modeling. Comprehensive evaluation across multiple datasets confirms the practical viability of this approach for clinical deployment.

Architectural Innovation and Clinical Significance: This study combines edge guidance principles from COD with Mamba’s efficient sequence modeling capabilities. The experimental comparison across CNN, Transformer, COD, and Mamba architectures reveals an important finding: neither computational efficiency nor boundary detection alone is sufficient. CNN architectures like ConvNeXt achieve an 89.69% mIoU with good efficiency but lack boundary specialization. COD methods such as BGNet reach 91.63% through explicit boundary guidance but require excessive computation. Pure Mamba models (CM-UNet: 92.33%) provide efficiency but miss critical boundary details. BM-UNet’s integration achieves a 93.51% mIoU, confirming that combining boundary guidance with efficient sequence modeling creates the necessary synergy for accurate camouflaged structure segmentation.

Performance Superiority in Camouflaged Scenarios: BM-UNet demonstrates stable performance improvements across different pathological conditions (56Nx, DN, NEP25, and Normal), with an average 4.57% mIoU improvement over competitive baselines. The framework shows a particularly strong performance under the most challenging NEP25 conditions, achieving a 92.28% mIoU compared to 87.47% for the Mamba baseline. This 4.81% improvement demonstrates the framework’s capability to handle severely compromised tissue boundaries where conventional approaches struggle.

Clinical Impact of Performance Improvements: The performance gains translate to meaningful clinical benefits. In NEP25 conditions, BM-UNet’s 92.28% mIoU compared to BGNet’s 89.49% mIoU represents approximately 20–30 fewer boundary misclassifications per whole-slide image. This improvement directly affects glomerular sclerosis quantification, a key parameter in CKD staging. Additionally, BM-UNet achieves the lowest standard deviation (1.34%) across all test conditions. CNN methods show higher variability (U-Net: 4.82%, ConvNeXt: 3.12%), as do Transformer (SegFormer: 3.40%, Swin-UNet: 4.56%) and COD approaches (BiRefNet: 2.24%, BGNet: 2.47%). This consistency reduces the need for case-specific adjustments, particularly valuable for borderline cases where accurate quantification influences staging decisions between CKD stage 2 and stage 3.

Computational Efficiency for Clinical Deployment: BM-UNet achieves a 113.7 FPS processing speed (approximately 8.8 ms per image), making it suitable for real-time clinical analysis. Its moderate memory footprint of approximately 2.1 GB ensures compatibility with standard clinical workstations. This efficiency–accuracy balance addresses the practical deployment barriers that limit many academically successful methods. The framework’s dual output design provides both segmentation masks and explicit boundary predictions. While formal user studies have not been conducted, this additional boundary information may help pathologists verify results in ambiguous regions, though further clinical validation would be needed to confirm this benefit.

Limitations and Future Directions: Current evaluation focuses on 2D medical images, primarily PAS-stained samples. While cross-dataset validation on HuBMAP (94.32% mIoU) demonstrates the method’s generalization capability, the predominance of PAS staining in our training data raises concerns about potential overfitting. Clinical practice employs diverse staining methods (H&E, Masson’s trichrome, and silver stains), each with distinct visual characteristics that require systematic validation. The structured Sobel operators in HBD, though effective for PAS samples, may need adjustment for different staining protocols and imaging modalities, despite the learnable components providing some adaptability.

Future work will explore 2.5D processing as a practical step toward volumetric analysis, where Mamba’s linear complexity provides computational advantages over 3D attention mechanisms. True 3D validation remains essential for accurate disease burden quantification, particularly for determining the percentage of affected glomeruli, which is critical for CKD staging. Additionally, the optimal balance between boundary guidance and semantic features may vary across pathological conditions and disease stages. Advanced cases with severe architectural distortion might benefit from different boundary–semantic weighting compared to early-stage changes, suggesting potential for adaptive weighting mechanisms in future iterations that could adjust based on tissue characteristics or institutional imaging protocols.

6. Conclusions

This paper presents BM-UNet, a novel framework addressing the critical challenge of glomerular segmentation in renal pathological images where tissue “camouflage” obscures conventional detection methods. By uniquely integrating explicit boundary guidance with efficient Mamba state-space modeling, our approach establishes a new paradigm for medical image segmentation in challenging scenarios.

The framework incorporates four synergistic components—MAF, HBD, BGA, and MFDB—that systematically address irregular morphologies, boundary detection, feature enhancement, and reconstruction while maintaining linear computational complexity. Experimental validation demonstrates significant improvements: a 93.51% average mIoU representing a 4.57% enhancement over baselines, with a processing speed of 113.7 FPS suitable for clinical deployment.

Most importantly, BM-UNet shows a robust performance across diverse pathological conditions, particularly excelling in challenging NEP25 scenarios (92.28% mIoU vs. 87.47% baseline). Cross-dataset validation on human tissue samples (94.32% mIoU) confirms generalization capability, while moderate computational requirements (2.3 GB memory) ensure practical deployment feasibility.

This work successfully bridges sophisticated boundary detection with computational efficiency, providing a robust foundation for accurate diagnostic tools in camouflaged medical imaging environments. The dual-output design offers both segmentation masks and boundary predictions, potentially enhancing pathologist confidence in challenging cases.