Cross-Conditioned Spectral Diffusion Fusion for Symmetry-Aware Mirror Segmentation

Abstract

Mirror segmentation aims to identify mirror pixels from a single RGB image, yet remains challenging because mirrors provide weak intrinsic texture cues and their appearance is dominated by scene-dependent reflections under varying illumination and viewpoints. While recent models improve performance by leveraging contextual contrast, symmetry priors, frequency/spectral cues, or additional modalities (e.g., depth), many cross-cue or symmetry-aware designs still rely on direct spatial-domain fusion, such as concatenation, addition, or attention. Such fusion can amplify reflection-induced high-frequency variations and lead to leakage, shape distortion, and unstable boundaries. In this paper, we propose a symmetry-aware mirror segmentation framework that stabilizes cross-branch interaction via a frequency-domain cross-conditioned fusion mechanism. We build a dual-path Siamese encoder using the original image and its horizontally flipped counterpart, and introduce Heat Conduction Operator-based Cross Fusion (HCOCF), which performs heat-conduction-inspired spectral attenuation in the DCT domain. Unlike conventional fusion, HCOCF generates a nonnegative cross-conditioned attenuation coefficient map from the opposite branch and applies it to the DCT coefficient grid of the target branch. This produces a DCT-domain attenuation mask that controls the spectral refinement strength of each target feature stream, enabling global context propagation while suppressing unstable reflection-induced high-frequency responses without aggressive direct feature mixing. For multi-scale decoding, we adapt the cross-scale decoder of the baseline symmetry-aware architecture by replacing simple addition with conditional feature aggregation, which refines the HCOCF-enhanced features and improves boundary recovery. Extensive experiments on MSD, PMD, and RGBD-Mirror demonstrate competitive performance against representative supervised mirror segmentation methods. In particular, our RGB-only model achieves 88.47% IoU on MSD and 73.72% IoU on PMD, and remains competitive on RGBD-Mirror without using depth input.

1. Introduction

Mirror segmentation aims to accurately identify mirror pixels in an input image. Despite the distinctive physical property that mirrors reflect the surrounding scene, mirror surfaces themselves often provide weak and unstable intrinsic texture cues, making the problem highly ambiguous. A mirror’s appearance can change drastically with the background, material, illumination, and viewing angle; even the same mirror may resemble entirely different objects when the reflected content changes. Such uncertainty can cause false recognition and geometric failure in downstream vision systems, including object detection/tracking, indoor robotics, 3D reconstruction, and neural rendering (e.g., Gaussian Splatting), making reliable mirror segmentation practically important [1,2].

Recent deep learning approaches improve performance using either a single RGB image or auxiliary signals such as depth. However, many methods tend to exploit dataset-specific contextual and shape priors (e.g., bathroom mirrors, closet doors, framed layouts, or typical placements) rather than explicitly modeling the reflective nature of mirrors.

Depth information can provide useful geometric cues for reducing reflection ambiguity, although its effectiveness depends on the quality of the acquired depth and the way it is integrated with RGB features. In addition, RGB-D approaches require an additional depth sensor and corresponding data acquisition. Therefore, developing an effective RGB-only model remains valuable, particularly when depth input is unavailable or undesirable.

Existing methods can be broadly categorized into (i) RGB-based methods that learn contextual contrast or inside–outside relations between mirror and non-mirror regions, and (ii) methods that leverage additional modalities to mitigate reflection ambiguity. Yang et al. proposed MirrorNet, which exploits multi-scale contextual contrast cues but may fail when contrast becomes weak, such as when the content inside and outside the mirror is visually similar [1]. Lin et al. introduced PMDNet, which progressively combines relation cues and explicit boundary learning; nevertheless, complex reflection-induced local variations in texture and illumination may still lead to errors [3]. Guan et al. modeled semantic associations between mirrors and surrounding objects [4], and Tan et al. introduced pixel-level cues based on visual chirality [5].

Mei et al. proposed PDNet for RGB-D mirror segmentation using ToF depth discontinuities, demonstrating that depth can provide complementary geometric cues, although RGB-D approaches require additional sensing and data acquisition [6].

Recent frequency- and spectral-aware methods further indicate that reflection-induced ambiguity can be reduced by exploiting spatial-frequency or spectral representations. For example, DPRNet integrates dual-domain spatial/frequency perception [7], CSFwinformer learns cross-space-frequency representations [8], and S²MD exploits multi-directional similarity together with spectral saliency enhancement [9]. These studies show that frequency/spectral cues are useful for suppressing reflection ambiguity and improving mirror localization. Nevertheless, existing frequency/spectral designs primarily strengthen feature representations or saliency responses, while the fusion mechanism between symmetry-related branches remains less explicitly controlled.

Despite this progress, the fusion mechanism itself remains a key limitation. Many existing methods perform cross-cue or cross-branch interaction through direct spatial-domain fusion, such as concatenation, addition, or attention. In mirror scenes, reflection patterns and illumination changes frequently appear as strong high-frequency fluctuations; direct feature mixing may amplify these unstable patterns, producing boundary noise, leakage, and shape distortion. Therefore, beyond introducing stronger contextual, symmetry, or spectral cues, it is important to explicitly stabilize how complementary branches exchange information. Figure 1 illustrates these issues: MirrorNet [1] and PMDNet [3] may over-predict into non-mirror regions or collapse boundaries, and SATNet [10] can still exhibit leakage or distorted shapes even with symmetry cues. This suggests that simply adding contextual, symmetry, or spectral cues is not sufficient; instead, the feature exchange mechanism itself should be explicitly stabilized.

Figure 1. Qualitative comparison of challenging cases. Context/contrast-based methods (MirrorNet [1] and PMDNet [3]) often over- or under-segment when reflection patterns dominate, while symmetry-aware SATNet [10] still exhibits leakage or shape distortion under strong high-frequency reflections. Our method produces cleaner masks closer to the ground truth by stabilizing cross-branch fusion via frequency-domain, cross-conditioned spectral diffusion.

Since mirror reflections often form symmetry-like correspondences, a natural starting point is a Siamese dual-branch encoder that processes an image and its horizontally flipped counterpart. SATNet [10] adopts such a dual-encoder structure and learns symmetry-aware interactions via attention-based fusion. In this work, we focus on stabilizing the exchange between the original and symmetry-aligned branches. Specifically, we use the opposite branch to generate a nonnegative attenuation coefficient map, which parameterizes the DCT-domain attenuation mask of the target branch. In this way, cross-branch information is exchanged through spectral control rather than direct feature injection. Inspired by this line of work, we redesign the fusion mechanism to: (i) strengthen cross-branch correlations in a way that is less sensitive to unstable local reflection patterns, and (ii) integrate mirror cues across scales more robustly for accurate boundary recovery.

Specifically, we propose Heat Conduction Operator-based Cross Fusion (HCOCF), which explicitly controls cross-branch interaction in the frequency domain.

HCOCF applies heat-conduction-inspired spectral attenuation in the DCT domain. The attenuation coefficients are generated from the opposite branch in the spatial feature domain and are then applied to the target branch’s DCT coefficient grid. Instead of directly mixing features, the conditioning branch modulates channel-frequency attenuation, encouraging global context propagation while suppressing unstable reflection-induced high-frequency responses.

We adopt the DCT because it provides a fully real-valued spectral representation, and its cosine basis corresponds to an even spatial extension that is consistent with reflective (Neumann-type) boundary treatment commonly used for diffusion on rectangular domains. This formulation is closely related to adaptive thermal diffusivity in heat-conduction-based vision modeling [11], but extends it to cross-branch fusion, where one branch determines how the other branch should be spectrally refined. We further incorporate gMLP-based conditioning for global dependency modeling [12] and a cross-branch squeeze-and-excitation-style recalibration (CrossSE) to stabilize channel-wise interactions [13]. Moreover, mirror boundaries depend on fine structural cues, and single-scale fusion is insufficient for precise reconstruction. For multi-scale decoding, we retain the three-branch decoder topology of the baseline symmetry-aware architecture and adapt it to the HCOCF-refined features. Specifically, the original addition-based interaction is replaced with conditional feature aggregation, followed by delayed aggregation of the main, symmetry-aligned, and mixed representations. From an information fusion perspective, the proposed framework goes beyond conventional feature aggregation. Instead of injecting raw features from one branch into another, HCOCF performs cross-conditioned spectral attenuation, in which one branch generates an attenuation coefficient map that controls the DCT-domain refinement of the other branch. This results in a more interpretable and reflection-robust fusion process, making the proposed design well suited for mirror segmentation.

Our main contributions are summarized as follows:

• We propose HCOCF, a heat-conduction-inspired cross-conditioned spectral attenuation module for symmetry-aware mirror segmentation. HCOCF generates a nonnegative attenuation coefficient map from the opposite branch and applies it to the target branch’s DCT coefficient grid, enabling cross-branch interaction through spectral attenuation rather than direct feature injection.
• We construct an RGB-only symmetry-aware mirror segmentation framework by integrating HCOCF into a SATNet-derived dual-path architecture. For multi-scale decoding, the baseline three-branch decoder is adapted with conditional feature aggregation to better refine the HCOCF-enhanced features.
• We conduct extensive quantitative and qualitative experiments on MSD, PMD, and RGBD-Mirror, demonstrating competitive performance against representative supervised mirror-specific methods.

The remainder of this paper is organized as follows. Section 2 reviews related work. Section 3 presents the proposed method in detail. Section 4 reports experimental results, including quantitative and qualitative comparisons and ablation studies. Section 5 concludes the paper.

2. Related Work

We review prior work from three perspectives: (i) mirror segmentation, (ii) symmetry-aware mirror understanding, and (iii) diffusion/heat-conduction-based global modeling.

Although the term “mirror detection” is widely used in prior literature, many of these works address pixel-wise mask prediction. For consistency, we therefore use the term “mirror segmentation” throughout this paper when referring to the pixel-wise prediction task, while retaining “mirror detection” only when it is part of an official paper title, method or dataset name, or terminology adopted from the cited work.

2.1. Mirror Segmentation

Because mirrors lack stable intrinsic cues and their appearance is dominated by reflections, early deep models focused on contrast or discontinuity between mirror interiors and surrounding regions. MirrorNet exploits multi-scale contextual contrast and introduced the large-scale MSD dataset [1], but may fail when contrast becomes weak. PMDNet progressively learns mirror/non-mirror relations and improves boundaries via relation cues and explicit edge-focused designs [3], yet complex reflection-induced local variations can still degrade robustness.

Mirror-like distractors such as windows, paintings, and door frames motivate leveraging semantic relations with surrounding objects. SANet models semantic associations to improve localization under weak contrast [4]. Efficiency-oriented designs such as HetNet explore heterogeneous cues for better speed/accuracy trade-offs [14]. Recent work also investigates frequency-domain cues: DPRNet integrates spatial/frequency dual-domain information, and CSFwinformer learns cross-space-frequency representations with window-based Transformers [7,8]. These results suggest that suppressing reflection-induced high-frequency fluctuations and enforcing global structure are beneficial for mirror segmentation. More recently, Shao et al. proposed S²MD, which combines a multi-directional similarity perception module (MSPM) and a spectral saliency enhancement decoder module (SSEDM) [9].

Depth can reduce ambiguity, and PDNet exploits ToF depth discontinuities as mirror cues [6]. Depth therefore provides potentially useful complementary information for mirror segmentation, although its effectiveness depends on the sensing conditions and the integration strategy, and RGB-D methods additionally require depth acquisition. These considerations motivate complementary investigation of RGB-only solutions that can capture mirror-specific structural regularities without requiring an additional modality.

Beyond mirror-specific architectures, several mainstream Transformer-based segmentation architectures developed in recent years have demonstrated strong performance across general dense-prediction tasks. SegFormer employs a hierarchical Transformer encoder together with a lightweight decoder to capture multi-scale contextual information, whereas Mask2Former formulates segmentation through masked attention and supports semantic, instance, and panoptic segmentation  [15,16]. These architectures demonstrate the strong adaptability of Transformer-based representations to general segmentation tasks. However, they are not specifically designed to model the reflection-dependent ambiguity or symmetry-related correspondence characteristic of mirror scenes. To broaden the evaluation beyond mirror-specific approaches, we therefore include controlled experiments with SegFormer-B2 and Mask2Former (Swin-T) in Section 4.3.

2.2. Symmetry-Aware Approaches

Mirror reflections often manifest as horizontally flipped correspondences or loose symmetry. VCNet [5] introduces visual chirality as a pixel-level cue related to left-right inversion. SATNet [10] processes original and flipped images via a dual-path encoder and learns symmetry-aware interactions using attention. However, many symmetry-aware methods still rely on direct spatial-domain fusion, which can amplify unstable high-frequency reflection patterns and blur the distinction between shared mirror structure and branch-specific appearance fluctuations.

Our approach retains the dual-path backbone but replaces direct spatial-domain feature mixing with a cross-conditioned spectral attenuation operator, in which one branch generates an attenuation coefficient map that controls the DCT-domain refinement of the other branch. This design yields a more stable and interpretable form of cross-branch interaction for mirror segmentation.

2.3. Heat Conduction/Diffusion-Based Global Modeling

Diffusion processes provide global propagation and high-frequency attenuation simultaneously, making them attractive for modeling long-range dependencies. In classical scale-space theory, the heat equation describes progressive smoothing over scale, while in the frequency domain its solution induces stronger attenuation for higher frequencies [17,18]. This property makes diffusion-based modeling particularly appealing for visual tasks in which global structure should be preserved while unstable local fluctuations should be suppressed [19].

Recent work by Wang et al. introduced the Heat Conduction Operator (HCO) and demonstrated that heat-conduction-inspired smoothing can be efficiently incorporated into modern vision models through a DCT/IDCT-based formulation with adaptive thermal diffusivity [11]. Compared with self-attention, this formulation provides an efficient way to obtain global interactions while retaining an explicit spectral interpretation. Unlike earlier diffusion-style operators that mainly refine a single feature stream, HCO shows that thermal diffusivity can be learned in the spectral domain as a dense channel-frequency quantity rather than as a single scalar.

Our method is closely related to vHeat [11], but differs in how the DCT-domain attenuation is conditioned. In vHeat, learnable frequency-side representations are used to estimate adaptive thermal diffusivity for single-stream refinement. By contrast, our CrossHCO uses the opposite-branch spatial feature as the conditioning signal to generate a nonnegative attenuation coefficient map. This map is then applied to the DCT coefficient grid of the target branch to construct a cross-conditioned attenuation mask. In this way, we extend heat-conduction-inspired DCT-domain attenuation from single-stream refinement to cross-branch controlled fusion, which is more suitable for symmetry-aware mirror segmentation.

Beyond image-based diffusion modeling, physics-inspired thermal diffusion has also been explored in other segmentation domains. For example, HSBNet formulates local neighborhood encoding in point clouds as an anisotropic thermal diffusion process and combines it with a dual-branch fusion architecture for boundary-aware segmentation [20]. While this line of work focuses on local geometric modeling in 3D point clouds, our method instead addresses symmetry-aware mirror segmentation in 2D images through cross-conditioned spectral diffusion in the DCT domain.

3. Method

We present the overall architecture, the proposed HCOCF module, and the adapted cross-scale decoder. Given that mirror reflections often resemble left-right inverted correspondences, we adopt a Siamese dual-path encoder of SATNet [10] that processes an image and its horizontally flipped counterpart. Compared to the prior symmetry-aware baseline [10], the main methodological change is HCOCF, which stabilizes cross-branch interaction via cross-conditioned spectral attenuation. For decoding, we adapt the baseline three-branch cross-scale decoder by replacing addition-based interaction with conditional feature aggregation.

3.1. Overall Architecture

Figure 2 presents the overall architecture of the proposed network.

Figure 2. Overall architecture of the proposed network. The left part shows the dual-path Siamese encoder, the middle part illustrates HCOCF at high-level stages, and the right part presents the decoder. At the coarsest stage, the decoder is initialized by a three-branch layout without top-down conditioning, while at the upper decoder stages, CSF produces three branch-specific refined features, which are subsequently enhanced by CCL and merged by the projection block to form the decoder representation. Colors distinguish the network modules and feature streams, as specified in the legend.

Given an input image $I\in {\mathbb{R}}^{3\times H\times W}$, we first generate its horizontally flipped counterpart $I_f={\mathcal{F}}_{flip}\left(I\right)$. For notational simplicity, the batch dimension is omitted consistently throughout the Section 3.1. The original and flipped images are then processed by a weight-sharing backbone (Swin-S Transformer [21]) to extract multi-scale features:

$${\left\{F_{main}^l\right\}}_{l=1}^4,{\left\{F_{flip}^l\right\}}_{l=1}^4.$$

(1)

Following Swin’s patch embedding and hierarchical downsampling, the feature shape at stage l is

$$F_{\{·\}}^l\in {\mathbb{R}}^{2^{l-1}C\times {\displaystyle \frac{H}{2^{l+1}}}\times {\displaystyle \frac{W}{2^{l+1}}}},l\in \{1,2,3,4\}.$$

(2)

To align the flipped-branch features with the main branch, we horizontally flip them back at each stage:

$$F_{sym}^l={\mathcal{F}}_{flip}\left(F_{flip}^l\right).$$

(3)

This alignment ensures that $F_{main}^l$ and $F_{sym}^l$ correspond spatially at the same coordinates.

For shallow stages ($l=1,2$), the features mainly encode local texture and boundary information, so the aligned pairs $(F_{main}^l,F_{sym}^l)$ are directly forwarded to the decoder through skip connections. For deeper stages ($l=3,4$), the features contain stronger semantic content and more stable reflection–real-scene correlations. Therefore, we apply HCOCF to obtain refined cross-branch representations:

$$({\overline{F}}_{main}^l,{\overline{F}}_{sym}^l)=\mathrm{HCOCF}(F_{main}^l,F_{sym}^l),l\in \{3,4\}.$$

(4)

For $l=1,2$, we simply set

$${\overline{F}}_{main}^l=F_{main}^l,{\overline{F}}_{sym}^l=F_{sym}^l.$$

At the decoder, the high-level HCOCF-refined features are progressively propagated to shallower stages through a three-branch cross-scale refinement structure. Following the baseline symmetry-aware decoder [10], the main, symmetry-aligned, and mixed representations are maintained separately and aggregated after conditional refinement. At the coarsest stage ($l=4$), there is no incoming top-down decoder feature. Thus, the decoder is initialized by directly forming three branch-specific outputs from $({\overline{F}}_{main}^4,{\overline{F}}_{sym}^4)$:

$$G_{main}^4={\overline{F}}_{main}^4,G_{sym}^4={\overline{F}}_{sym}^4,$$

(5)

and

$$G_{mix}^4={\varphi }_{mix}\left(\mathrm{Concat}\left({\overline{F}}_{main}^4,{\overline{F}}_{sym}^4\right)\right),$$

(6)

where

$${\varphi }_{mix}\left(X\right)=\delta \left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left({\mathrm{Conv}}_{1\times 1}\left(X\right)\right)\right)\right),$$

(7)

and $\delta (·)$ denotes ReLU. These three branch-specific outputs are then processed by CCL blocks and merged by a projection block:

$$F_{dec}^4=\mathrm{Proj}\left(\mathrm{Concat}\left(\mathrm{CCL}\left(G_{main}^4\right),\mathrm{CCL}\left(G_{mix}^4\right),\mathrm{CCL}\left(G_{sym}^4\right)\right)\right).$$

(8)

The resulting coarse decoder feature $F_{dec}^4$ is propagated upward to initialize the top-down decoding process. Note that we use the Context-Contrasted Local (CCL) block [22], which has been widely used for mirror segmentation to emphasize local responses relative to contextual responses [1].

For decoder stages $l\in \{1,2,3\}$, the decoder refinement block takes the current-scale main-branch feature, the symmetry-aligned feature, and the top-down decoder feature from the deeper stage, and outputs three branch-specific refined representations:

$$(G_{main}^l,G_{mix}^l,G_{sym}^l)=\mathrm{CSF}\left({\overline{F}}_{main}^l,{\overline{F}}_{sym}^l,\mathrm{Up}\left(F_{dec}^{l+1}\right)\right),$$

(9)

where $\mathrm{Up}(·)$ denotes bilinear upsampling. For compact notation, we denote the decoder refinement block as $\mathrm{CSF}(·)$, and its detailed relation to the baseline decoder is described in Section 3.3. Each branch output is then processed by a CCL block [22], and the resulting three boundary-enhanced features are fused by a projection block:

$$F_{dec}^l=\mathrm{Proj}\left(\mathrm{Concat}\left(\mathrm{CCL}\left(G_{main}^l\right),\mathrm{CCL}\left(G_{mix}^l\right),\mathrm{CCL}\left(G_{sym}^l\right)\right)\right),l\in \{1,2,3\}.$$

(10)

To facilitate optimization, we generate intermediate predictions from multiple decoder stages and apply deep supervision. Each stage output is transformed into a binary segmentation map through a $1\times 1$ convolution, and the final prediction is taken from the finest-resolution output (Output 1). Figure 3 illustrates the internal structure of HCOCF.

Figure 3. Structure of the proposed Heat Conduction Operator-based Cross Fusion (HCOCF). Each branch generates a nonnegative attenuation coefficient map for the opposite branch via CrossHCO. The coefficient map is applied to the target branch’s DCT coefficient grid to form a cross-conditioned spectral attenuation mask, followed by cross-branch channel recalibration via CrossSE. Purple/red and blue denote the main- and symmetry-branch features, respectively; orange denotes CrossHCO, cyan denotes CrossSE, and gray denotes basic processing operations.

3.2. HCO-Based Cross Fusion (HCOCF)

Mirror regions exhibit a strong yet unstable relationship between the real scene and its reflected counterpart. Although the two branches share globally correlated structure, their local responses can differ significantly due to reflection-dependent texture, illumination, and viewpoint changes. As a result, directly mixing the two branch features in the spatial domain—e.g., via concatenation, addition, or attention—may amplify branch-specific high-frequency fluctuations instead of reinforcing their shared structure  [10,23]. To address this issue, we propose HCO-based Cross Fusion (HCOCF), a fusion module that performs cross-conditioned spectral diffusion between the main and symmetry-aligned branches.

Our design is motivated by the observation that diffusion processes naturally provide two desirable properties for mirror segmentation: (i) global propagation of coherent structural information and (ii) attenuation of unstable high-frequency responses. Recent heat-conduction-based vision models have shown that such diffusion can be implemented efficiently in the DCT domain by learning an adaptive thermal diffusivity tensor over channel-frequency locations [11]. In vision, HCO was proposed as a plug-and-play operator that models long-range patch correlations through adaptive thermal diffusion, providing global receptive fields with lower computational overhead than self-attention-based mechanisms [11]. Inspired by this formulation, we extend HCO from a single-stream refinement operator to a cross-branch fusion operator. In the proposed CrossHCO, one branch generates a nonnegative attenuation coefficient map for the other branch, and this map is used to construct a DCT-domain attenuation mask for the target feature. Therefore, cross-branch information is exchanged by controlling spectral attenuation rather than by directly injecting spatial features.

Specifically, HCOCF consists of two components: (i) CrossHCO, which performs DCT-domain spectral attenuation on one branch conditioned on the opposite branch, and (ii) CrossSE, which further recalibrates channel responses using a joint representation of the two diffused branches. Compared with direct feature fusion, this design exchanges information in a more indirect and stable manner [24,25]: the conditioning branch does not overwrite the target feature itself, but instead generates an attenuation coefficient map that controls how strongly the DCT coefficients of the target branch are preserved or suppressed. This is particularly beneficial in mirror segmentation, where globally consistent mirror structure should be preserved while local reflection artifacts should be suppressed.

3.2.1. Background: Heat Conduction Operator

The proposed fusion module builds upon the Heat Conduction Operator (HCO) of Wang et al. [11]. HCO is inspired by the physical process of heat conduction and can be interpreted as a diffusion operator that spreads information globally while suppressing high-frequency components. This behavior is closely related to classical isotropic diffusion and Gaussian scale-space theory [17,18], where smoothing is governed by the heat equation.

Let $u(x,y,t)$ denote a continuous signal evolving over diffusion time t. The 2D heat equation is

$${\displaystyle \frac{\partial u(x,y,t)}{\partial t}}=k{\nabla }^{2}u(x,y,t),$$

(11)

where $k>0$ is the diffusion coefficient and ${\nabla }^2$ is the Laplacian operator [26]. Given the initial condition $u(x,y,0)=f(x,y)$, the signal gradually diffuses over time. In the Fourier domain, the solution takes the form

$$\tilde{u}({\omega }_x,{\omega }_y,t)=\tilde{f}({\omega }_x,{\omega }_y)exp\left(-k({\omega }_x^2+{\omega }_y^2)t\right),$$

(12)

which shows that high-frequency components are attenuated more strongly than low-frequency ones.

For deep feature maps, HCO can be implemented efficiently in the discrete cosine transform (DCT) domain [11]. Compared with FFT-based implementations, DCT is real-valued, numerically stable, and well suited for image-like signals due to its energy compaction property [27]. Given an input feature map $F\in {\mathbb{R}}^{C\times H\times W}$, we apply channel-wise 2D-DCT:

$$F^{\mathcal{D}}=\mathrm{DCT}2\mathrm{D}\left(F\right)\in {\mathbb{R}}^{C\times H\times W}.$$

(13)

Importantly, the spectral weighting used in our method is not newly hand-designed. Following the original HCO/vHeat formulation [11], we retain the squared-frequency dependence originating from the heat-conduction equation. For a DCT coefficient grid of spatial resolution $H\times W$, the discrete DCT angular frequencies are defined as

$${\omega }_u={\displaystyle \frac{\pi u}{H}},{\omega }_v={\displaystyle \frac{\pi v}{W}},$$

(14)

where $u=0,\dots ,H-1$ and $v=0,\dots ,W-1$ denote the DCT frequency-bin indices along the height and width dimensions, respectively, rather than spatial coordinates. Accordingly, u indexes the vertical frequency axis and v indexes the horizontal frequency axis of the DCT coefficient grid. The corresponding spectral weighting term is

$$\mathsf{\Lambda }(u,v)={\omega }_u^2+{\omega }_v^2={\left({\displaystyle \frac{\pi u}{H}}\right)}^2+{\left({\displaystyle \frac{\pi v}{W}}\right)}^2.$$

(15)

Thus, $\mathsf{\Lambda }(u,v)$ is a fixed, nonnegative, and non-learnable quadratic frequency term directly inherited from the original HCO formulation, rather than an additional spectral weighting function designed specifically for our mirror segmentation framework. In particular, $\mathsf{\Lambda }(0,0)=0$, while its value increases quadratically as either DCT frequency index increases.

While the classical heat equation is often written with a scalar diffusion coefficient, the adaptive HCO formulation of vHeat predicts frequency-dependent thermal diffusivity for DCT-domain feature refinement [11]. In our cross-conditioned design, however, the conditioning branch generates a nonnegative attenuation coefficient map that is used to parameterize the DCT-domain attenuation mask of the target branch. We denote this coefficient map as

$$A\in {\mathbb{R}}^{C\times H\times W},$$

(16)

where $q\in \{1,\dots ,C\}$ denotes the channel index, and each entry corresponds to a channel and a coefficient-grid location of the target DCT representation. Unlike the learned attenuation coefficient A, the spectral weighting $\mathsf{\Lambda }(u,v)$ in Equation (15) is fixed by the DCT frequency grid, shared across channels, and inherited from the original HCO formulation. The DCT-domain attenuation mask is then defined as

$$M(q,u,v)=exp\left(-A(q,u,v)\mathsf{\Lambda }(u,v)\right),$$

(17)

and the attenuated feature is reconstructed by inverse DCT:

$$F^{\mathrm{HCO}}=\mathrm{IDCT}2\mathrm{D}\left(F^{\mathcal{D}}\odot M\right).$$

(18)

This formulation separates the fixed squared-frequency prior $\mathsf{\Lambda }(u,v)$ inherited from HCO from the adaptive attenuation coefficient $A(q,u,v)$. Consequently, different channels and frequency locations can undergo different levels of diffusion, which is especially desirable in mirror segmentation, where stable structural cues and unstable reflection artifacts coexist in the same feature tensor. In HCOCF, our contribution lies in generating this adaptive attenuation coefficient using information from the opposite symmetry branch, while the functional form of $\mathsf{\Lambda }(u,v)$ remains unchanged from HCO.

3.2.2. Cross-Conditioned HCO (CrossHCO)

Instead of directly mixing two branch features, CrossHCO lets the conditioning branch control the DCT-domain attenuation of the target branch. Let $F_t$ denote the target feature to be refined and $F_c$ denote the conditioning feature from the opposite branch.

Specifically, we flatten the spatial dimensions of the conditioning feature into a token sequence:

$$Z_c=\mathrm{Reshape}\left({\mathrm{DWConv}}_{3\times 3}\left(F_c\right)\right)\in {\mathbb{R}}^{N\times C},N=HW.$$

(19)

The $H\times W$ spatial grid is flattened in row-major order, with the width dimension varying fastest. Specifically, for a spatial location $(h,w)$, where $h=0,\dots ,H-1$ and $w=0,\dots ,W-1$, the corresponding token index is

$$n=hW+w,n=0,\dots ,HW-1.$$

(20)

Thus, consecutive token indices first traverse the width dimension and then proceed to the next row. The same ordering is used for both the main and symmetry branches.

Here, ${\mathrm{DWConv}}_{3\times 3}$ denotes a depthwise $3\times 3$ convolution used to inject lightweight local spatial context before global conditioning and DCT-domain attenuation. We then apply a gMLP block [12] to encode long-range dependencies:

$$H_c=\mathrm{gMLP}\left(Z_c\right)\in {\mathbb{R}}^{N\times C}.$$

(21)

The gMLP output is reshaped back to feature-map form,

$${\tilde{F}}_c=\mathrm{Reshape}\left(H_c\right)\in {\mathbb{R}}^{C\times H\times W}.$$

(22)

The inverse reshape follows exactly the inverse of the row-major ordering defined in Equation (20), ensuring one-to-one spatial correspondence between each token and its original $(h,w)$ location. A nonnegative attenuation coefficient map is then generated by a pointwise projection:

$$A_{t\leftarrow c}=\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}\left({\tilde{F}}_c\right)\right)\in {\mathbb{R}}^{C\times H\times W}.$$

(23)

Here, the $1\times 1$ convolution performs learnable channel mixing on the gMLP-encoded conditioning feature, and ReLU enforces a nonnegative attenuation constraint. It should be noted that $A_{t\leftarrow c}$ is generated from spatial-domain conditioning features. Therefore, we interpret it as a cross-conditioned attenuation coefficient map, rather than a diffusivity tensor directly predicted in the spectral domain. Its $H\times W$ layout is used to parameterize the DCT coefficient grid of the target branch for element-wise spectral attenuation.

It is important to distinguish the roles of $A_{t\leftarrow c}(q,u,v)$ and $\mathsf{\Lambda }(u,v)$, where $q\in \{1,\dots ,C\}$ denotes the channel index. The latter is the fixed quadratic frequency weighting inherited from the original HCO formulation, whereas $A_{t\leftarrow c}$ is learned from the opposite branch and adaptively determines how strongly this fixed spectral prior is applied to each channel and DCT coefficient location.

Once $A_{t\leftarrow c}$ is obtained, the target feature $F_t$ is transformed into the DCT domain:

$$F_t^{\mathcal{D}}=\mathrm{DCT}2\mathrm{D}\left({\mathrm{DWConv}}_{3\times 3}\left(F_t\right)\right)\in {\mathbb{R}}^{C\times H\times W}.$$

(24)

The cross-conditioned attenuation mask is then constructed as

$$M_{t\leftarrow c}(q,u,v)=exp\left(-A_{t\leftarrow c}(q,u,v)\mathsf{\Lambda }(u,v)\right),$$

(25)

where $\mathsf{\Lambda }(u,v)$ is the fixed, non-learnable quadratic spectral weighting term defined in Equation (15) and inherited from the original HCO formulation.

The refined target feature is recovered by

$$F_t^{\mathrm{HCO}}=\mathrm{IDCT}2\mathrm{D}\left(F_t^{\mathcal{D}}\odot M_{t\leftarrow c}\right).$$

(26)

Within HCOCF, this directional operation is applied bidirectionally:

$$\begin{array}{c}\hfill {\widehat{F}}_{main}=\mathrm{CrossHCO}(F_{main}\mid F_{sym}),\\ \hfill {\widehat{F}}_{sym}=\mathrm{CrossHCO}(F_{sym}\mid F_{main}).\end{array}$$

(27)

As a result, each branch is refined by a DCT-domain attenuation mask generated under the guidance of the opposite branch. Globally consistent structures shared by both branches are encouraged, whereas branch-specific reflection artifacts or illumination-induced perturbations that are prominent in only one branch are relatively suppressed.

Conceptually, CrossHCO can be understood as indirect fusion via spectral attenuation control. Unlike concatenation, addition, or attention-based direct feature mixing, the conditioning branch does not inject raw features into the target branch. Instead, it generates an attenuation coefficient map that determines how strongly the DCT coefficients of the target branch are preserved or suppressed. This makes the interaction less sensitive to unstable local appearance variations and better aligned with the structural regularities of mirror reflection. Therefore, CrossHCO does not introduce a new hand-designed frequency-weighting function; rather, it retains the squared-frequency prior of HCO and extends the original formulation by determining the attenuation strength through cross-branch conditioning.

3.2.3. Cross-Branch Channel Recalibration (CrossSE)

CrossHCO mainly addresses spatial/global interaction through spectral diffusion. However, in mirror scenes, the reliability of feature channels may also differ across branches due to reflection distortion, illumination variation, or background clutter. To complement CrossHCO, we introduce CrossSE, a channel recalibration module inspired by the squeeze-and-excitation (SE) mechanism [13], but adapted to dual-branch interaction [28].

We first concatenate the two diffused branch features:

$$F_{cat}=\mathrm{Concat}({\widehat{F}}_{main},{\widehat{F}}_{sym})\in {\mathbb{R}}^{2C\times H\times W}.$$

(28)

A lightweight mixing block then aggregates their joint channel statistics:

$$F_{mix}=\delta \left(\mathrm{BN}\left(\mathrm{Conv}\left(F_{cat}\right)\right)\right),$$

(29)

where $\delta (·)$ denotes ReLU and $\mathrm{Conv}(·)$ is implemented as a $1\times 1$ convolution. Global average pooling produces a compact channel descriptor:

$$z=\mathrm{GAP}\left(F_{mix}\right).$$

(30)

We then use an MLP to predict $2C$ channel gates:

$$g=\sigma \left(\mathrm{MLP}\left(z\right)\right)\in {\mathbb{R}}^{2C},[g_{main},g_{sym}]=\mathrm{Split}\left(g\right),g_{main},g_{sym}\in {\mathbb{R}}^{C\times 1\times 1}.$$

(31)

Here, $\sigma (·)$ denotes the sigmoid function. The $2C$-dimensional gate vector is split evenly into two branch-specific channel gates, each of which is reshaped to ${\mathbb{R}}^{C\times 1\times 1}$. The singleton spatial dimensions are broadcast over the corresponding $H\times W$ feature map when the gate is applied.

The recalibrated outputs are given by

$${\overline{F}}_{main}=g_{main}\odot {\widehat{F}}_{main},{\overline{F}}_{sym}=g_{sym}\odot {\widehat{F}}_{sym}.$$

(32)

Thus, the multiplication in Equation (32) is performed channel-wise, with each scalar channel gate shared across all spatial locations of the corresponding branch feature. Unlike the original SE block, which predicts channel importance from a single feature stream [13], CrossSE estimates two branch-specific gates from a joint representation of both branches. This allows the importance of one branch’s channels to be determined in light of the other branch, thereby improving robustness under asymmetric or unreliable responses. Figure 4 illustrates the adapted cross-scale decoder and its CCFF refinement block.

Figure 4. Structure of the cross-scale decoder adapted with conditional feature aggregation. Following the three-branch topology of the baseline symmetry-aware decoder, main, symmetry-aligned, and mixed features are refined under a shared top-down condition before delayed aggregation. The main adaptation replaces the original addition-based interaction with CCFF-based conditional refinement [29]. Purple, blue, and green denote the main-branch, symmetry-branch, and previous-decoder features, respectively; orange denotes convolution, green blocks denote CCFF, and gray/blue-gray blocks denote normalization and residual refinement.

3.3. Cross-Scale Decoding with Conditional Feature Aggregation

The decoder must combine coarse semantic information from deeper layers with fine boundary cues from shallower layers. In mirror scenes, unstable local reflection patterns can corrupt naive feature aggregation, and premature mixing of heterogeneous representations may blur the distinction between structure and appearance. To decode the HCOCF-refined features, we follow the three-branch cross-scale refinement topology of SATNet’s decoder [10], which maintains main, symmetry-aligned, and mixed representations. In this paper, we refer to this topology as a CFDM-style decoder [10].

Compared with the original addition-based interaction, we replace simple addition with conditional feature aggregation. Specifically, the top-down decoder feature is used as a shared condition to refine the main, symmetry-aligned, and mixed features before delayed aggregation. This design preserves the overall decoder topology of the baseline while providing a stronger aggregation mechanism for the HCOCF-refined features.

We implement the conditional feature aggregation using a lightweight conditional cross-scale feature fusion block, denoted as CCFF [29]. For compact notation, the resulting CFDM-style decoder adapted with CCFF-based aggregation is denoted as Cross-Scale Fusion (CSF). CSF is applied only to decoder stages $l\in \{1,2,3\}$, where a top-down decoder feature from the deeper stage is available. At such a stage, CSF takes three inputs:

$${\overline{F}}_{main}^l,{\overline{F}}_{sym}^l,F_{dec}^{l+1},$$

(33)

where ${\overline{F}}_{main}^l$ is the current-scale main-branch feature, ${\overline{F}}_{sym}^l$ is the corresponding symmetry-aligned feature, and $F_{dec}^{l+1}$ is the decoder feature from the deeper stage. We first upsample the previous decoder feature:

$$F_{prev}^l=\mathrm{Up}\left(F_{dec}^{l+1}\right),$$

(34)

so that all features are aligned at the same spatial resolution.

Following the mixed-branch design of the CFDM decoder [10], CSF employs an additional mixed branch to explicitly capture the shared intra-scale structure between the main and symmetry features. Specifically, this mixed representation is obtained by concatenating the main and symmetry features, followed by a lightweight transformation block:

$$F_{mix}^l={\varphi }_{mix}\left(\mathrm{Concat}\left({\overline{F}}_{main}^l,{\overline{F}}_{sym}^l\right)\right),$$

(35)

where

$${\varphi }_{mix}\left(X\right)=\delta \left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left({\mathrm{Conv}}_{1\times 1}\left(X\right)\right)\right)\right),$$

(36)

and $\delta (·)$ denotes ReLU. This branch summarizes the common structural information shared by the main and symmetry-aligned features.

The upsampled top-down decoder feature is transformed into the same feature space:

$${\overline{F}}_{prev}^l={\varphi }_{prev}\left(F_{prev}^l\right),$$

(37)

where

$${\varphi }_{prev}\left(X\right)=\delta \left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left({\mathrm{Conv}}_{1\times 1}\left(X\right)\right)\right)\right).$$

(38)

This embedded top-down feature acts as a shared condition for all branch-specific refinements within the current stage.

CSF refines three current-scale representations independently under the shared condition ${\overline{F}}_{prev}^l$:

$$\begin{array}{cc}\hfill G_{main}^l& =\mathrm{CCFF}\left({\overline{F}}_{main}^l,{\overline{F}}_{prev}^l\right),\hfill \\ \hfill G_{mix}^l& =\mathrm{CCFF}\left(F_{mix}^l,{\overline{F}}_{prev}^l\right),\hfill \\ \hfill G_{sym}^l& =\mathrm{CCFF}\left({\overline{F}}_{sym}^l,{\overline{F}}_{prev}^l\right).\hfill \end{array}$$

(39)

This separate-refinement design avoids forcing heterogeneous representations to mix prematurely.

Inspired by the efficient hybrid encoder and lightweight fusion design used in RT-DETR [29], we adopt a compact CCFF block for conditional cross-scale refinement. Given a branch feature X and the condition feature P, CCFF is defined as

$$U=\mathrm{Concat}(X,P),$$

(40)

$$\mathrm{CCFF}(X,P)=\underset{\mathrm{shortcut}\mathrm{projection}}{\underbrace{{\mathrm{Conv}}_{1\times 1}^{\left(s\right)}\left(U\right)}}+\underset{\mathrm{residual}\mathrm{refinement}}{\underbrace{{\mathrm{RepBlock}}^{\times R}\left({\mathrm{Conv}}_{1\times 1}^{\left(r\right)}\left(U\right)\right)}},$$

(41)

where R is the number of RepBlock repetitions. If a RepVGG-style implementation is used, this design can be understood as a lightweight re-parameterizable residual refinement block [30]. This conditional refinement replaces the simple addition used in the original decoder [10], while the three-branch cross-scale topology is retained.

After CSF produces the three branch-specific outputs, each output is processed by a Context-Contrasted Local (CCL) block [22] to enhance boundary-sensitive responses. The three CCL-enhanced features are then merged by a projection block to form the decoder feature as in Equation (10). In our implementation, the local branch in CCL uses dilation 1, while the contextual branch uses dilations $\{8,6,4,2\}$, following the decoder design of SATNet [10].

This design preserves the overall decoder topology of the baseline while providing a stronger aggregation mechanism for the HCOCF-refined features. In contrast to the original CFDM [10], which mainly relies on direct addition-based aggregation, our method strengthens the skip connection to complement the representation of each branch feature before delayed aggregation at the decoder stage. This design helps maintain a more stable balance between coarse region-level semantics and fine boundary cues, which is desirable for mirror segmentation, where both structural and boundary cues are important.

3.4. Loss Function

We apply deep supervision at four decoder outputs. Let the i-th output logit be $S_i\in {\mathbb{R}}^{2\times H_i\times W_i}$, and let the ground-truth mask be $G\in {\{0,1\}}^{H\times W}$. Since the decoder outputs have different spatial resolutions, we first upsample the logits to the ground-truth resolution:

$${\tilde{S}}_i=\mathrm{Up}\left(S_i\right),{\tilde{S}}_i\in {\mathbb{R}}^{2\times H\times W}.$$

(42)

The corresponding probability map can then be obtained as

$${\tilde{P}}_i=\mathrm{Softmax}\left({\tilde{S}}_i\right),{\tilde{P}}_i\in {[0,1]}^{2\times H\times W}.$$

(43)

Note that we index outputs from coarse to fine: $i=1$ is the coarsest output and $i=4$ is the finest output (Output 1), which is used for final inference.

We compute pixel-wise cross-entropy at each scale from the upsampled softmax scores and combine them with weights ${\lambda }_i$:

$${\mathcal{L}}_{total}={\displaystyle \sum _{i=1}^4}{\lambda }_i{\mathcal{L}}_{ce}({\tilde{P}}_i,G),$$

(44)

where we set $\lambda =[1.25,1.25,1.0,1.5]$ to emphasize the finest prediction while maintaining sufficient supervision for coarse outputs [10].

4. Experiments

4.1. Experimental Setup

4.1.1. Datasets and Metrics

We evaluate on three widely used benchmarks: the (1) Mirror Segmentation Dataset (MSD) [1], (2) the Progressive Mirror Dataset (PMD) [3], and (3) RGBD-Mirror [6] for comparisons with depth-based methods. We follow the official train/test splits: MSD has 3063/955 images, PMD has 5096/571 images, and RGBD-Mirror has 2000/1049 images.

We report IoU, $F_{\beta }$, and MAE:

$$\mathrm{IoU}={\displaystyle \frac{|P\cap G|}{|P\cup G|}},$$

(45)

$$F_{\beta }={\displaystyle \frac{(1+{\beta }^2)·\mathrm{Precision}·\mathrm{Recall}}{{\beta }^2·\mathrm{Precision}+\mathrm{Recall}}},$$

(46)

$$\mathrm{MAE}={\displaystyle \frac{1}{HW}}{\displaystyle \sum _{i=1}^H}{\displaystyle \sum _{j=1}^W}\left|P(i,j)-G(i,j)\right|,$$

(47)

where we set ${\beta }^2=0.3$ to emphasize precision.

4.1.2. Implementation Details

We implement the model in PyTorch 2.7.0+cu128 and use Swin-S [21], initialized with ImageNet-1K pretraining [31], as the backbone. The original image and its horizontally flipped counterpart are processed by a weight-sharing Siamese encoder. Table 1 summarizes the training, preprocessing, and data augmentation settings used in our experiments.

Table 1. Training, preprocessing, and data augmentation settings used in our experiments.

Item	Setting
Model and preprocessing
Backbone initialization	Swin-S pretrained on ImageNet-1K
Input size and interpolation	$512\times 512$ for training and inference; bilinear interpolation for images and nearest-neighbor interpolation for masks
Normalization	Mean $[0.485,0.456,0.406]$; standard deviation $[0.229,0.224,0.225]$
Optimization
Optimizer	AdamW, ${\beta }_1=0.9$, ${\beta }_2=0.999$, weight decay $=0.01$
Learning rate	Initial learning rate $3\times {10}^{-4}$; polynomial schedule with power $=1$
Training duration	100 epochs with a batch size of 8
Training objective	Multi-scale cross-entropy with deep supervision; output weights $\lambda =[1.25,1.25,1.0,1.5]$
Data augmentation
Random crop	Crop height and width are independently sampled to retain approximately 75–100% of the original dimensions; the same crop is applied to the image and mask
Horizontal flip	Applied with probability $0.5$
Brightness/contrast	Applied with probability $0.5$; contrast factor $\alpha \sim \mathcal{U}(0.8,1.2)$ and brightness offset $\beta \sim 255\mathcal{U}(-0.2,0.2)$
Gaussian blur	Applied with probability $0.3$; kernel size selected from $3\times 3$ or $5\times 5$
Gaussian noise	Applied with probability $0.3$; $n\sim \mathcal{N}(0,5^2)$ on the $[0,255]$ intensity scale

During training, the same random crop and horizontal flip are first applied to each image–mask pair, followed by brightness/contrast adjustment, Gaussian blur, and Gaussian noise applied only to the image. After augmentation, images and masks are resized to $512\times 512$ using bilinear and nearest-neighbor interpolation, respectively, and the images are normalized using ImageNet statistics. At inference, stochastic data augmentation is disabled, and only resizing and normalization are applied.

All experiments were conducted on an NVIDIA RTX 5090 GPU. Training took approximately 2 min per epoch, and inference ran at approximately 45 FPS under the same evaluation setting.

4.2. Quantitative Comparison

We compare our method with representative mirror segmentation approaches covering diverse design principles, including contextual-contrast learning, relation modeling, semantic association, symmetry-aware reasoning, frequency-aware modeling, and RGB-D-based segmentation. Specifically, the compared methods include MirrorNet [1], PMDNet [3], SANet [4], HetNet [14], VCNet [5], SATNet [10], CSFwinformer-S/B [8], DPRNet [7], and S²MD [9]. For RGBD-Mirror, we additionally report RGB-D variants of PDNet and SANet when depth input is used [4,6]. All results are reported under the official dataset splits and evaluation metrics; when available, we use the officially reported or released results of competing methods. Results on MSD/PMD are reported in Table 2, and results on RGBD-Mirror are reported in Table 3.

Table 2. Quantitative comparison on MSD and PMD datasets. Higher IoU and $F_{\beta }$, and lower MAE, indicate better performance. The best and second-best results are shown in bold and underlined, respectively.

Method	MSD			PMD
	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
MirrorNet	78.88	0.856	0.066	58.51	0.741	0.043
PMDNet	81.54	0.892	0.047	66.05	0.792	0.032
SANet	79.85	0.879	0.054	66.84	0.837	0.032
HetNet	82.80	0.906	0.043	69.00	0.814	0.029
VCNet	80.08	0.898	0.044	64.02	0.815	0.028
SATNet	85.41	0.922	0.033	69.38	0.847	0.025
CSFwinformer-S	82.13	0.895	0.046	69.84	0.849	0.024
CSFwinformer-B	82.08	0.896	0.045	70.05	0.838	0.024
DPRNet	86.60	0.888	0.033	72.10	0.766	0.026
S2MD	87.11	0.936	0.032	69.77	0.846	0.024
Ours	88.47	0.944	0.029	73.72	0.876	0.022

Table 3. Quantitative comparison on RGBD-Mirror dataset. “w/Depth” indicates whether depth is used as input. Higher IoU and $F_{\beta }$, and lower MAE, indicate better performance. The best results, including ties, are shown in bold, and the second-best results, including ties when applicable, are underlined.

Method	w/Depth	RGBD-Mirror
		IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
PDNet		73.57	–	0.053
PDNet	✓	77.77	0.878	0.041
SANet		74.99	0.873	0.048
SANet	✓	78.43	0.834	0.041
VCNet		73.01	0.849	0.052
SATNet		78.42	0.906	0.031
CSFwinformer-B		78.66	0.900	0.031
DPRNet		76.10	0.811	0.047
S2MD		78.60	0.904	0.030
Ours		80.87	0.919	0.030

To further broaden the comparison beyond mirror-specific methods, we also evaluate representative mainstream Transformer-based segmentation architectures under a controlled training protocol, as described in Section 4.3.

4.2.1. Results on MSD and PMD Datasets

As shown in Table 2, our method achieves the best performance among the compared supervised mirror-specific methods on both MSD and PMD across all three metrics. On MSD, our model obtains 88.47% IoU, $F_{\beta }=0.944$, and MAE = 0.029. Compared with the strongest competing results on this dataset, our method improves IoU by 1.36 percentage points, increases $F_{\beta }$ by 0.008, and reduces MAE by 0.003. These results indicate that the proposed cross-conditioned spectral diffusion effectively suppresses reflection-induced fluctuations while preserving mirror-relevant global structure.

On PMD, our method achieves 73.72% IoU, $F_{\beta }=0.876$, and MAE = 0.022. The strongest competing scores for IoU, $F_{\beta }$, and MAE are 72.10%, 0.849, and 0.024, respectively. Thus, our method improves IoU by 1.62 percentage points, increases $F_{\beta }$ by 0.027, and reduces MAE by 0.002.

The larger IoU gain on PMD suggests that the proposed framework is particularly effective on challenging mirror scenes, where weak contrast, ambiguous mirror/non-mirror relations, and complex reflection patterns make feature integration difficult.

Compared with SATNet [10], which also exploits a symmetry-aware dual-branch structure, our complete model improves IoU by 3.06 percentage points on MSD and 4.34 percentage points on PMD. These gains support the effectiveness of the proposed architectural modifications for symmetry-aware mirror segmentation. The component-wise ablation analysis further separates the contributions of HCOCF and the adapted decoder. In addition, the adapted decoder supports accurate boundary recovery by integrating main-branch, symmetry-aligned, and top-down features through conditional multi-scale aggregation. Overall, the results validate the effectiveness of combining HCOCF for robust high-level branch interaction with the adapted decoder for boundary-sensitive decoding.

4.2.2. Results on RGBD-Mirror Dataset

Table 3 includes both RGB-only and depth-assisted methods. Our RGB-only model achieves 80.87% IoU, $F_{\beta }=0.919$, and MAE = 0.030, obtaining the best IoU and $F_{\beta }$ and an MAE tied for the best among the representative methods reported in the table. Its IoU is 3.10 and 2.44 percentage points higher than the reported results of depth-assisted PDNet [6] and SANet [4], respectively. Because these results are literature-reported point estimates rather than repeated measurements under a matched statistical protocol, these margins should not be interpreted as evidence of statistically significant superiority over the corresponding RGB-D methods.

To directly examine the contribution of depth within our own framework, we additionally construct a depth-assisted HCOCF variant using the depth encoder adopted from PDNet [6]. The depth encoder operates as a separate parallel branch and extracts multi-scale depth features at all encoder stages. The original RGB-based HCOCF processing is retained unchanged, and the depth features are not used to condition the HCOCF attenuation coefficients. Instead, after HCOCF processing, the depth feature at each stage is concatenated with the corresponding HCOCF-refined RGB representation and projected through a $1\times 1$ convolution before being passed to the decoder. Thus, this variant evaluates the complementary contribution of depth without altering the proposed cross-conditioned spectral attenuation mechanism.

As shown in Table 4, incorporating depth improves IoU from 80.87% to 83.06%, corresponding to an increase of 2.19 percentage points. $F_{\beta }$ also increases from 0.919 to 0.930, while MAE decreases from 0.030 to 0.029. Therefore, in this controlled comparison, depth provides complementary information to HCOCF rather than degrading the RGB-only model.

Table 4. Effect of incorporating depth information into HCOCF on RGBD-Mirror. The HCOCF+Depth variant uses a separate parallel depth encoder, while the RGB-based HCOCF processing remains unchanged. Higher IoU and $F_{\beta }$ and lower MAE indicate better performance; bold denotes the best result.

Method	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
HCOCF (RGB only)	80.87	0.919	0.030
HCOCF + Depth	83.06	0.930	0.029

These results indicate that depth information should not be regarded as inherently unreliable or detrimental for mirror segmentation. Instead, it can provide useful complementary cues when appropriately integrated. At the same time, the RGB-only HCOCF model achieves strong performance without requiring an additional depth input, which remains advantageous when depth sensing is unavailable or undesirable. The comparison between HCOCF and HCOCF+Depth is a controlled single-run ablation; therefore, we use it to characterize the observed effect of adding depth rather than as statistical evidence that depth is universally beneficial or detrimental in reflective scenes.

4.3. Comparison with General-Purpose Transformer-Based Segmentation Models

To broaden the comparative analysis beyond mirror-specific methods and include representative mainstream segmentation architectures, we additionally compare HCOCF with two established Transformer-based models: SegFormer-B2  [15] and Mask2Former with a Swin-T backbone  [16]. Unlike the literature-reported results in the main benchmark comparison, these two models were retrained on MSD under a controlled experimental protocol.

For this comparison, SegFormer-B2, Mask2Former (Swin-T), and HCOCF use the same MSD train/test split, input resolution, data augmentation, optimizer, learning-rate schedule, number of training epochs, batch size, and evaluation protocol. For all three models, only the backbone is initialized with ImageNet-1K pretrained weights. The architecture-specific model structures are otherwise retained without modification.

As shown in Table 5, SegFormer-B2 achieves an IoU of 84.98%, an $F_{\beta }$ of 0.930, and an MAE of 0.041, while Mask2Former (Swin-T) achieves 80.63%, 0.921, and 0.051, respectively. Under the same experimental protocol, HCOCF achieves an IoU of 88.47%, an $F_{\beta }$ of 0.944, and an MAE of 0.029. Compared with SegFormer-B2, HCOCF improves IoU by 3.49 percentage points and reduces MAE by 0.012. Compared with Mask2Former (Swin-T), HCOCF improves IoU by 7.84 percentage points and reduces MAE by 0.022.

Table 5. Controlled comparison with general-purpose Transformer-based segmentation models on MSD. All models are trained and evaluated under the same experimental protocol. Higher values (↑) are better for IoU and $F_{\beta }$; lower values (↓) are better for MAE. Bold denotes the best result for each metric.

Method	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
SegFormer-B2	84.98	0.930	0.041
Mask2Former (Swin-T)	80.63	0.921	0.051
HCOCF (Swin-S)	88.47	0.944	0.029

These results show that the tested general-purpose Transformer-based segmentation architectures do not match the performance of HCOCF on MSD under the same training and evaluation protocol. This comparison supports the usefulness of explicitly incorporating mirror-specific structural cues, including symmetry-aware observations and cross-conditioned spectral attenuation, rather than relying solely on a general-purpose segmentation architecture. Nevertheless, the compared methods employ different model designs and architecture-specific backbones; therefore, this experiment should be interpreted as a controlled task-level comparison rather than as an isolation of every individual architectural factor.

4.4. Qualitative Comparison

Figure 5 shows qualitative comparisons with representative prior methods on MSD and PMD. Columns correspond to (a) input image, (b)–(j) prior methods, (k) ours, and (l) ground truth. Rows 1–3 are from MSD and rows 4–6 are from PMD, representing: (1) low contrast (mirror resembles surroundings), (2) low contrast, (3) occlusion, (4) loose symmetry, (5) low contrast, and (6) low contrast.

Figure 5. Qualitative results with representative prior methods on MSD and PMD datasets. (a) Image. (b) MirrorNet. (c) PMDNet. (d) HetNet. (e) VCNet. (f) SANet. (g) SATNet. (h) CSFwinformer-S. (i) CSFwinformer-B. (j) DPRNet. (k) Ours. (l) GT. Rows 1–3 show MSD cases and rows 4–6 show PMD cases. The examples include low-contrast scenes, occlusion, and loose-symmetry cases. Our method produces masks closer to the ground truth with less leakage and fewer boundary errors.

In low-contrast cases (rows 1–2 and 5–6), prior methods often exhibit leakage, missing regions, holes, or noisy boundaries due to weak mirror/non-mirror contrast. Our method suppresses reflection-driven high-frequency fluctuations via HCOCF and enforces stronger global consistency, producing masks closer to the ground truth. In the occlusion case (row 3), many methods break the mask near the occluder or expand into the foreground, whereas our model better preserves region continuity and boundary quality by combining HCOCF with the adapted multi-scale decoder. In the loose-symmetry case (row 4), symmetry cues alone can be unreliable; our cross-conditioned diffusion avoids overly rigid correspondence assumptions and remains robust under imperfect alignment.

Figure 6 shows qualitative comparisons with representative RGB and RGB-D methods on RGBD-Mirror. (a) is the input RGB image, (b) is the depth map, (c)–(h) are competing methods, (i) is ours, and (j) is the ground truth. Depth can provide useful complementary geometric cues, while challenging conditions such as occlusion, small mirror regions, and low contrast remain difficult for both RGB and RGB-D mirror segmentation methods. In row 1, occlusion makes the mirror boundary ambiguous; our method preserves boundaries more stably. In row 2, a small mirror region and an occluder produce extremely weak cues; our method avoids false positives while preserving the small mirror region. In row 3, low contrast leads to over-segmentation in several competing methods, whereas our prediction more closely follows the ground-truth region. In row 4, two mirrors of different sizes are present. Most competing methods miss the smaller mirror, whereas our method successfully segments both mirror regions, consistent with HCOCF’s global stabilization and the adapted decoder’s multi-scale boundary recovery.

Figure 6. Qualitative results on RGBD-Mirror dataset. (a) Image. (b) Depth. (c) PDNet. (d) SANet. (e) VCNet. (f) SATNet. (g) CSFwinformer-B. (h) DPRNet. (i) Ours. (j) GT. The examples include ambiguous boundaries under occlusion, small mirrors, low-contrast scenes, and multiple-mirror cases. Despite using only RGB input, our method produces accurate mirror masks across these challenging examples without requiring depth at inference.

4.5. Ablation Study

We analyze the contribution of the main architectural components on both MSD and PMD to examine whether the component-wise trends remain consistent across independent RGB mirror segmentation benchmarks. Table 6 reports the effects of CrossHCO, the complete HCOCF module with CrossSE, and the CSF decoder adaptation on both datasets. Unless otherwise specified, the remaining controlled ablation experiments in this section are conducted on MSD. We further examine whether the effectiveness of HCOCF depends specifically on the DCT representation by comparing it with a controlled Fourier-domain variant using 2D rFFT/irFFT. Different HCO conditioning strategies are compared to isolate the contribution of cross-branch conditioning. In addition, Section 4.5.5 provides a DCT-domain analysis of representative addition, concatenation, and HCOCF fusion behaviors. The placement of HCOCF across scales is also analyzed. Unless otherwise specified, HCOCF is applied at stages 3 and 4, where high-level semantics and stable symmetry correlations are most prominent.

Table 6. Component-wise ablation results on MSD and PMD. The same component configurations are evaluated on both datasets. Within each dataset, all variants are trained under the same training conditions, and the retrained SATNet serves as the controlled baseline. HCOCF_3,4 denotes HCOCF applied at stages 3 and 4 and consists of CrossHCO and CrossSE. Higher IoU and $F_{\beta }$ and lower MAE indicate better performance; bold highlights the full model’s best or tied-best values.

Method	MSD			PMD
	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
SATNet (retrained baseline)	86.35	0.929	0.034	70.94	0.862	0.024
+ CrossHCO only	86.67	0.941	0.032	72.04	0.870	0.024
+ HCOCF3,4 (CrossHCO + CrossSE)	87.60	0.944	0.030	72.62	0.875	0.023
+ CSF (decoder adaptation)	87.77	0.944	0.032	72.97	0.867	0.024
Ours (HCOCF3,4 + CSF)	88.47	0.944	0.029	73.72	0.876	0.022

4.5.1. Effect of the Proposed Modules

Table 6 reports controlled component-wise ablations on both MSD and PMD. To avoid confounding architectural gains with training-recipe differences, we retrain the SATNet baseline under the same training conditions as the corresponding ablation variants within each dataset. Therefore, all module-level gains are measured relative to the retrained SATNet baseline for the corresponding dataset. The component-wise ablations on both datasets follow the controlled single-run setting, while training variability across random seeds is analyzed separately on MSD in Section 4.5.9.

On MSD, adding CrossHCO alone improves the retrained baseline by increasing IoU from 86.35% to 86.67%, increasing $F_{\beta }$ from 0.929 to 0.941, and reducing MAE from 0.034 to 0.032. When CrossSE is further added to form HCOCF_3,4, IoU increases to 87.60% and MAE decreases to 0.030. The CSF decoder adaptation independently improves IoU from 86.35% to 87.77%. Finally, the complete model, HCOCF_3,4 + CSF, achieves 88.47% IoU, $F_{\beta }=0.944$, and an MAE of 0.029. Compared with the retrained SATNet baseline, this corresponds to an improvement of 2.12 percentage points in IoU and a reduction of 0.005 in MAE.

A similar overall component-wise trend is observed on PMD. The retrained SATNet baseline achieves 70.94% IoU, $F_{\beta }=0.862$, and an MAE of 0.024. Adding CrossHCO increases IoU to 72.04% and $F_{\beta }$ to 0.870. Adding CrossSE to form HCOCF_3,4 further improves the results to 72.62% IoU, $F_{\beta }=0.875$, and an MAE of 0.023. The CSF decoder adaptation independently increases IoU to 72.97%. Finally, combining HCOCF_3,4 and CSF achieves the best overall PMD performance, with 73.72% IoU, $F_{\beta }=0.876$, and an MAE of 0.022. Relative to the retrained SATNet baseline, the complete model improves IoU by 2.78 percentage points, increases $F_{\beta }$ by 0.014, and reduces MAE by 0.002.

Taken together, the MSD and PMD results show that the overall component-wise trend is reproduced across the two RGB mirror segmentation benchmarks. CrossHCO improves IoU over the retrained baseline on both datasets, CrossSE further strengthens HCOCF, and the CSF decoder provides an independent improvement in segmentation performance. Their combination achieves the best overall result on both datasets, including the highest IoU and lowest MAE. These results provide cross-dataset evidence that the observed architectural benefits are not specific to MSD.

Figure 7 visualizes how HCOCF changes feature responses at stages 3 and 4. The response map is obtained by channel-wise averaging and min-max normalization of the feature magnitude. Before HCOCF at stage 3, activations can be scattered across background regions, while after HCOCF, both branches exhibit more concentrated responses around mirror structures, indicating earlier formation of mirror-aware localization. At stage 4, HCOCF further sharpens feature responses around true mirror regions and suppresses irrelevant background activations, supporting more precise segmentation.

Figure 7. Feature response visualization before and after HCOCF. (a) Input. (b,c) Stage 3 before/after HCOCF. (d,e) Stage 4 before/after HCOCF. In (b–e), the left panel is the main branch and the right panel is the symmetry-aligned branch. Note that, in each figure, the right panel is a flipped response map of the symmetry feature. HCOCF suppresses dispersed background responses and concentrates both branches on mirror-relevant structures, especially at high-level stages.

Figure 8 qualitatively compares predictions as modules are added. The baseline may confuse strong reflected objects with non-mirror regions or introduce boundary artifacts. CSF improves boundary recovery but may still retain reflection-induced noise. HCOCF substantially suppresses reflection-driven false responses via cross-conditioned spectral diffusion, and the full model further refines boundaries through multi-scale integration, producing outputs closest to the ground truth.

Figure 8. Visual ablation of HCOCF and the decoder adaptation. HCOCF suppresses reflection-induced false responses, while the decoder adaptation further refines mirror boundaries.

The ablation results also show that HCOCF benefits from a stronger decoder adaptation. HCOCF mainly improves representation stability by suppressing reflection-driven fluctuations and aligning the two branches at a structural level, whereas the adapted decoder mainly supports boundary-sensitive reconstruction by integrating coarse semantic cues and fine details across scales. This explains why each module individually improves the retrained baseline, while their combination yields the best overall result. In other words, HCOCF improves the quality of the features being propagated, and the adapted decoder improves how these features are converted into accurate mirror masks.

4.5.2. Comparison of HCOCF with Other Fusion Methods

To verify the effectiveness of the proposed HCOCF, we conducted comparative experiments with various fusion methods for combining the main and symmetry features. The compared methods include addition and concatenation, which are commonly used feature aggregation strategies, as well as an attention-based cross-attention method. The purpose of this experiment is to examine whether HCOCF is effective not only compared with simple feature aggregation methods, but also compared with an attention-based fusion method that explicitly models interactions between features.

In this experiment, the addition- and concatenation-based methods first combine the main and symmetry features into a single fused feature, which is then duplicated and fed into the two inputs of the decoder. This design is adopted because the proposed decoder is structured to receive two feature inputs at each stage. Specifically, addition generates a fused feature through element-wise summation, whereas concatenation combines the two features along the channel dimension and then applies a convolution layer to align the channel dimension. For both addition and concatenation, the resulting fused feature is duplicated and used as the decoder input.

For the attention-based comparison, we referred to the SAAM structure proposed in SATNet [10]. SAAM performs cross-attention by using each feature as a query, while the key and value are generated from the concatenation of the two features followed by a convolution layer for channel reduction. In addition, SAAM applies Efficient Channel Attention (ECA) [32] to the attention-enhanced feature. In this experiment, the cross-attention method is implemented based on the SAAM structure, but the ECA module is removed. This allows us to evaluate the effect of attention-based interaction between the main and symmetry features while excluding the influence of channel attention.

For a fair comparison, all fusion variants in Table 7 use the same CSF decoder and the same training settings; only the high-level branch interaction module at stages 3 and 4 is replaced. As shown in Table 7, addition and concatenation achieve IoU scores of 86.77% and 86.22%, respectively, with MAE values of 0.033 and 0.034. These results indicate that simply adding the two features or concatenating them along the channel dimension is insufficient to fully exploit the complementary information of the two branches. The cross-attention method achieves an IoU of 87.43% and an MAE of 0.033, showing better performance than the simple fusion methods. However, it still performs worse than the proposed HCOCF.

Table 7. Ablation study of different fusion methods on MSD. HCOCF achieves the best IoU and MAE among simple fusion strategies and attention-based fusion modules. For a fair comparison, all fusion variants use the same CSF decoder and the same training settings. Higher IoU and $F_{\beta }$ and lower MAE indicate better performance; bold and underline denote the best and second-best results, respectively.

Fusion Method	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
Add	86.77	0.943	0.033
Concat	86.22	0.940	0.034
Cross-attention	87.43	0.942	0.033
SAAM [10]	87.77	0.944	0.032
HCOCF (ours)	88.47	0.944	0.029

In contrast, the proposed HCOCF achieves the best performance among all compared methods, with an IoU of 88.47% and an MAE of 0.029. In particular, HCOCF improves the IoU by 1.04 percentage points over cross-attention and reduces the MAE from 0.033 to 0.029. This result demonstrates that cross-branch interaction based on the heat conduction operator can more effectively exploit the complementary relationship between the main and symmetry feature streams than conventional attention-based feature interaction.

Therefore, the results in Table 7 demonstrate that HCOCF models the interaction between the main and symmetry features more effectively than both simple fusion strategies and attention-based fusion. These results further validate the effectiveness of HCOCF as the core fusion module of the proposed model.

4.5.3. Comparison of DCT- and Fourier-Domain Implementations

To examine whether the performance of HCOCF depends specifically on the DCT representation, we implemented a controlled Fourier-domain variant using 2D Fourier transforms for real-valued inputs (rFFT/irFFT). The surrounding architecture, training settings, and evaluation protocol were kept unchanged, while only the transform-specific frequency-domain operations were adapted to the rFFT representation.

Because 2D rFFT represents a real-valued feature using a one-sided complex spectrum, the predicted attenuation map was adapted to preserve consistency with the Hermitian structure of the corresponding full Fourier spectrum. The attenuation predictor first generates a full-resolution map

$$A_{t\leftarrow c}^{\mathrm{raw}}\in {\mathbb{R}}^{C\times H\times W},$$

(48)

and the one-sided attenuation map applied to the rFFT coefficients is

$$A_{t\leftarrow c}^{\mathrm{rFFT}}\in {\mathbb{R}}^{C\times H\times (\lfloor W/2\rfloor +1)}.$$

(49)

For each channel and retained Fourier-frequency location, we construct

$$A_{t\leftarrow c}^{\mathrm{rFFT}}(q,k_y,k_x)={\displaystyle \frac{1}{2}}\left[A_{t\leftarrow c}^{\mathrm{raw}}(q,k_y,k_x)+A_{t\leftarrow c}^{\mathrm{raw}}(q,{\overline{k}}_y,{\overline{k}}_x)\right],$$

(50)

where $q\in \{1,\dots ,C\}$ denotes the channel index, $k_y\in \{0,\dots ,H-1\}$ denotes the vertical frequency index, and $k_x\in \{0,\dots ,\lfloor W/2\rfloor \}$ denotes the horizontal frequency index retained by the one-sided rFFT representation. The conjugate-frequency indices on the periodic full Fourier grid are

$${\overline{k}}_y=(-k_y)modH,{\overline{k}}_x=(-k_x)modW.$$

(51)

The raw attenuation map has a full $H\times W$ layout corresponding to the full Fourier coefficient grid, allowing the conjugate-frequency location to be accessed before restricting the map to the one-sided rFFT domain.

For the Fourier variant, the spectral weighting is defined on the discrete Fourier frequency grid. With unit sample spacing, we define

$$f_y\left(k_y\right)=fftfreq\left(H\right)\left[k_y\right],f_x\left(k_x\right)=rfftfreq\left(W\right)\left[k_x\right],$$

(52)

and the corresponding angular frequencies as

$${\tilde{\omega }}_y\left(k_y\right)=2\pi f_y\left(k_y\right),{\tilde{\omega }}_x\left(k_x\right)=2\pi f_x\left(k_x\right).$$

(53)

The Fourier-domain spectral weighting is then defined as

$${\mathsf{\Lambda }}_{\mathrm{FFT}}(k_y,k_x)=\alpha \left[{\tilde{\omega }}_y^2\left(k_y\right)+{\tilde{\omega }}_x^2\left(k_x\right)\right],$$

(54)

where

$$\alpha ={\displaystyle \frac{{\left[\pi (H-1)/H\right]}^2+{\left[\pi (W-1)/W\right]}^2}{{\displaystyle \underset{k_y,k_x}{max}\left[{\tilde{\omega }}_y^2\left(k_y\right)+{\tilde{\omega }}_x^2\left(k_x\right)\right]}}}.$$

(55)

The factor $\alpha$ is determined analytically from the two discrete frequency grids and is neither learned nor tuned as a hyperparameter. It is used only for this controlled Fourier-domain comparison to match the maximum quadratic spectral weight of the Fourier variant to that of the DCT variant, thereby avoiding an artificial difference in the overall attenuation scale. Thus, both variants use a quadratic radial frequency dependence with matched maximum spectral weighting, while differing in their transform basis and discrete frequency grid.

Using this weighting, the Fourier-domain attenuation mask is defined as

$$M_{t\leftarrow c}^{\mathrm{rFFT}}(q,k_y,k_x)=exp\left(-A_{t\leftarrow c}^{\mathrm{rFFT}}(q,k_y,k_x){\mathsf{\Lambda }}_{\mathrm{FFT}}(k_y,k_x)\right).$$

(56)

Therefore, the Fourier variant retains the same heat-kernel-inspired attenuation form as the DCT-based HCOCF while adapting the attenuation map and spectral weighting to the rFFT frequency representation.

As shown in Table 8, the Fourier (rFFT)-based variant achieves an IoU of 88.39%, an $F_{\beta }$ of 0.947, and an MAE of 0.030, whereas the DCT-based HCOCF achieves 88.47%, 0.944, and 0.029, respectively. Compared with the DCT-based model, the Fourier variant achieves an $F_{\beta }$ that is higher by 0.003. In contrast, the DCT-based model achieves an IoU higher by 0.08 percentage points and an MAE lower by 0.001.

Table 8. Comparison of Fourier (rFFT)-based and DCT-based HCOCF variants on MSD. The surrounding architecture, training settings, and evaluation protocol are kept unchanged. Higher IoU and $F_{\beta }$ and lower MAE indicate better performance; bold denotes the best result.

Method	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
Fourier (rFFT)-based HCOCF	88.39	0.947	0.030
DCT-based HCOCF	88.47	0.944	0.029

These results indicate a modest metric-dependent trade-off rather than a consistent performance advantage for either frequency-domain representation. Therefore, the effectiveness of HCOCF cannot be attributed simply to the use of the DCT. We retain the DCT in the final model because it provides a fully real-valued spectral representation without the explicit conjugate-frequency consistency handling required by the rFFT formulation. In addition, its cosine basis corresponds to an even spatial extension and is consistent with reflective (Neumann-type) boundary treatment commonly used for diffusion on rectangular domains. Together with its slightly better IoU and MAE, these properties make the DCT a suitable spectral representation for the proposed HCOCF.

4.5.4. Effect of Cross-Branch Conditioning

To isolate the contribution of cross-branch conditioning from the underlying HCO operation, we conduct an additional controlled comparison among three variants, as shown in Table 9. The surrounding dual-branch architecture, decoder, and training and evaluation settings are kept unchanged, while only the HCO formulation and the source of conditioning information are varied.

Table 9. Comparison of HCO conditioning strategies within the same dual-branch architecture on MSD. The surrounding architecture, decoder, and training settings are kept unchanged across all variants. Higher IoU and $F_{\beta }$ and lower MAE indicate better performance; bold denotes the best result.

Method	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
Dual-branch HCO	86.53	0.940	0.034
Self-conditioned HCO	87.03	0.947	0.034
HCOCF (ours)	88.47	0.944	0.029

Dual-branch HCO is not a reproduction of the complete vHeat network of Wang et al. [11]. Rather, it is a controlled variant within our dual-branch framework in which the original HCO operation is applied independently to the main and symmetry-aligned branches without cross-branch conditioning. Self-conditioned HCO retains the attenuation-map generation mechanism used in CrossHCO, but generates the attenuation coefficient map for each target branch using features from the same branch. In contrast, HCOCF generates the attenuation coefficient map from the opposite symmetry-related branch. Therefore, the three variants respectively evaluate independent HCO refinement, same-branch conditioning, and the proposed cross-branch conditioning strategy.

As shown in Table 9, Dual-branch HCO achieves an IoU of 86.53%, an $F_{\beta }$ of 0.940, and an MAE of 0.034. Self-conditioned HCO improves the IoU to 87.03% and $F_{\beta }$ to 0.947, while maintaining an MAE of 0.034. The proposed HCOCF achieves the highest IoU of 88.47% and the lowest MAE of 0.029. Compared with Dual-branch HCO, HCOCF improves IoU by 1.94 percentage points, increases $F_{\beta }$ by 0.004, and reduces MAE by 0.005. Compared with Self-conditioned HCO, HCOCF further improves IoU by 1.44 percentage points and reduces MAE by 0.005, although Self-conditioned HCO achieves a slightly higher $F_{\beta }$ by 0.003.

These results show that the performance gain is not obtained simply by applying HCO independently to both branches. The improvement from Dual-branch HCO to HCOCF indicates that allowing the opposite branch to control spectral attenuation provides additional benefit over a direct dual-branch application of the original HCO. Moreover, the improvement over Self-conditioned HCO indicates that the source of conditioning information itself is important: symmetry-related opposite-branch features provide more effective guidance for region-level spectral refinement than using the target branch alone. Although cross-conditioning does not yield the best value for every metric, it provides the best overall balance of IoU and MAE and is therefore adopted in the final HCOCF design.

4.5.5. DCT-Domain Analysis of Fusion Behavior

To provide direct spectral evidence for the motivation of HCOCF, we further analyze the DCT-domain feature responses before and after branch interaction. In particular, we examine a representative failure case in which direct addition and concatenation produce false-positive leakage in a strongly reflective region, and compare their spectral behavior with HCOCF. This analysis complements the dataset-level segmentation comparison in Table 7 by examining how different fusion strategies modify the relative distribution of DCT-domain energy.

For a DCT-transformed feature $F^{\mathcal{D}}\in {\mathbb{R}}^{C\times H\times W}$, we define the normalized radial frequency as

$$\rho (u,v)={\displaystyle \frac{1}{\sqrt{2}}}\sqrt{{\left({\displaystyle \frac{u}{H-1}}\right)}^2+{\left({\displaystyle \frac{v}{W-1}}\right)}^2},$$

(57)

where $u=0,\dots ,H-1$ and $v=0,\dots ,W-1$ denote the DCT frequency-bin indices along the height and width dimensions, respectively. The high-frequency region is defined as

$${\mathsf{\Omega }}_{\mathrm{HF}}=\left\{(u,v)\mid \rho (u,v)\ge 0.5\right\}.$$

(58)

We then measure the relative high-frequency energy by summing the squared DCT magnitude over all feature channels, where $q\in \{1,\dots ,C\}$ denotes the channel index:

$$R_{\mathrm{HF}}=100{\displaystyle \frac{{\displaystyle {\displaystyle \sum _{q=1}^C}\sum _{(u,v)\in {\mathsf{\Omega }}_{\mathrm{HF}}}{\left|F^{\mathcal{D}}(q,u,v)\right|}^2}}{{\displaystyle {\displaystyle \sum _{q=1}^C}\sum _{u,v}{\left|F^{\mathcal{D}}(q,u,v)\right|}^2}}}.$$

(59)

The change in relative high-frequency energy is measured as

$$\mathsf{\Delta }{R}_{\mathrm{HF}}=R_{\mathrm{HF}}^{\mathrm{after}}-R_{\mathrm{HF}}^{\mathrm{before}},$$

(60)

and is reported in percentage points.

As shown in Table 10, direct addition increases the relative high-frequency energy from 9.88% to 10.93%, corresponding to an increase of 1.05 percentage points. Concatenation exhibits a similar behavior, increasing the ratio from 10.49% to 11.60% ($+1.11$ percentage points). In contrast, HCOCF reduces the relative high-frequency energy from 12.33% to 8.13%, corresponding to a decrease of 4.20 percentage points.

Table 10. Relative high-frequency energy before and after branch interaction for the representative failure case shown in Figure 9.

Fusion Method	Before (%)	After (%)	Change (pp)
Addition	9.88	10.93	$+1.05$
Concatenation	10.49	11.60	$+1.11$
HCOCF	12.33	8.13	$-4.20$

Figure 9 provides a corresponding qualitative comparison. For this representative failure case, the post-fusion DCT maps of addition and concatenation retain stronger relative high-frequency responses than their respective pre-fusion representations. Their predicted masks also exhibit false-positive leakage in the upper reflective region. In contrast, HCOCF substantially attenuates the spectral response after fusion and produces a cleaner prediction that more closely follows the ground-truth mirror region. The qualitative behavior is consistent with the relatively high-frequency energy changes reported in Table 10.

Figure 9. DCT-domain feature-response analysis for different branch-interaction strategies. (a) Input image. (b) Ground-truth mask. (c) Channel-averaged DCT magnitude before fusion. (d) Channel-averaged DCT magnitude after fusion. (e) Predicted mask. The three rows correspond to Addition, Concatenation, and HCOCF, respectively. The non-English text visible in panel (a) is part of the original benchmark input image. All DCT magnitude maps are visualized using a common global normalization scale, so that identical colors correspond to identical normalized coefficient magnitudes. The DCT frequency indices u and v increase along the vertical and horizontal axes, respectively.

We emphasize that this observation does not imply that all high-frequency components are undesirable. High-frequency coefficients may also contain useful boundary and structural information, which is consistent with the placement analysis showing that applying HCOCF to very shallow stages can degrade fine-detail preservation. Rather, the representative example demonstrates that direct spatial-domain fusion can amplify excessive high-frequency responses in a reflection-dominated region, whereas HCOCF can suppress such responses through cross-conditioned spectral attenuation. Together with the dataset-level results in Table 7, this analysis provides empirical support for the frequency-domain motivation of HCOCF.

4.5.6. Effect of the Attenuation Coefficient Map Predictor

CrossHCO employs gMLP as the predictor for generating the cross-conditioned attenuation coefficient map. To validate this design choice, we replaced the gMLP-based predictor with a conventional MLP and a self-attention-based predictor and compared their performance. For a fair comparison, all other architectural components and training and evaluation settings were kept unchanged. Each predictor was implemented as a single block, and the hidden-width ratio of the MLP was set to 1. All three predictors used the same input and output feature dimensions, such that only their internal interaction mechanisms differed.

As shown in Table 11, the MLP-based predictor achieved an IoU of 86.33%, an $F_{\beta }$ of 0.939, and an MAE of 0.031, while the self-attention-based predictor achieved an IoU of 87.05%, an $F_{\beta }$ of 0.938, and an MAE of 0.031. In contrast, HCOCF with the gMLP-based predictor achieved an IoU of 88.47%, an $F_{\beta }$ of 0.944, and an MAE of 0.029, yielding the best overall performance among the three predictor designs. In particular, gMLP improved IoU by 2.14 percentage points over the MLP and by 1.42 percentage points over self-attention, while also achieving the highest $F_{\beta }$ and the lowest MAE.

Table 11. Comparison of CrossHCO attenuation coefficient predictors. Higher values (↑) are better for IoU and $F_{\beta }$; lower values (↓) are better for MAE. Bold denotes the best result for each metric.

Predictor	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
MLP	86.33	0.939	0.031
Self-attention	87.05	0.938	0.031
gMLP (HCOCF)	88.47	0.944	0.029

Although the three predictors share the same input and output feature dimensions, they differ in how they model interactions among positions. The conventional MLP does not explicitly perform the spatial mixing and multiplicative gating employed by gMLP, whereas self-attention models interactions through input-dependent pairwise attention. In contrast, the spatial gating unit of gMLP mixes information across positions through a learned spatial projection and multiplicative gating. Under the same experimental settings, the gMLP-based predictor achieved the best overall performance among the three predictor designs. These results support the use of gMLP as the predictor for generating the cross-conditioned attenuation coefficient map in HCOCF.

4.5.7. Effect of HCOCF Placement

As shown in Table 12, applying HCOCF only at the deepest stage ($S_4$) is insufficient to fully capture cross-branch correlations. The best result is obtained by applying HCOCF at stages $S_{3,4}$, where semantic structures are stable enough for global diffusion to be beneficial. Extending HCOCF to lower-level stages ($S_1$/$S_2$) slightly degrades performance, likely because low-level features contain strong high-frequency boundary details that can be over-smoothed by diffusion.

Table 12. Effect of HCOCF placement across stages on MSD. $S_n$ denotes the n-th stage. Applying HCOCF to stages 3 and 4 yields the best trade-off between global consistency and fine-detail preservation. Higher values (↑) are better for IoU and $F_{\beta }$; lower values (↓) are better for MAE. Bold denotes the best result for each metric.

Placement	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
$S_4$	85.84	0.931	0.032
$S_{3,4}$	88.47	0.944	0.029
$S_{2,3,4}$	86.89	0.940	0.033
$S_{1,2,3,4}$	87.12	0.938	0.033

4.5.8. Sensitivity to the CrossSE Reduction Ratio

The heat-conduction-based spectral attenuation in HCOCF does not require a manually selected diffusion-related hyperparameter, since the attenuation coefficients are adaptively predicted from the cross-branch features. We therefore examine the main architectural hyperparameter of HCOCF, namely the reduction ratio r of the MLP bottleneck in CrossSE.

As shown in Table 13, the performance remains relatively stable for $r\in \{8,16,32\}$. The IoU varies by only 0.52 percentage points across the tested settings. The setting $r=16$ achieves the highest IoU (88.47%) and the lowest MAE (0.029), while maintaining a competitive $F_{\beta }$ of 0.944. Although $r=32$ slightly improves $F_{\beta }$ to 0.947, it results in lower IoU and higher MAE. We therefore use $r=16$ in all main experiments as it provides the best overall performance across the three evaluation metrics.

Table 13. Sensitivity analysis of the CrossSE reduction ratio r on MSD. Higher values (↑) are better for IoU and $F_{\beta }$; lower values (↓) are better for MAE. Bold denotes the best result for each metric.

r	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
8	88.07	0.941	0.032
16	88.47	0.944	0.029
32	87.95	0.947	0.031

4.5.9. Multi-Seed Robustness Analysis

To evaluate whether the performance improvement of the proposed framework is robust to stochastic variations during training, we conducted five matched training runs for each model on MSD using random seeds 7, 42, 2026, 2077, and 1557. We compare the complete proposed model with SATNet, which serves as the direct symmetry-aware architectural baseline of our framework. Both models were trained using the same dataset split, optimization settings, and matched random seeds.

As shown in Table 14, the proposed model achieves an average IoU of $88.18\pm 0.30\%$, compared with $85.57\pm 1.07\%$ for SATNet. The proposed model also achieves $F_{\beta }=0.948\pm 0.005$ and an MAE of $0.0304\pm 0.0009$, whereas SATNet obtains $F_{\beta }=0.928\pm 0.006$ and an MAE of $0.0358\pm 0.0023$. In addition to achieving better mean performance, the proposed method exhibits a lower standard deviation in IoU, indicating more consistent segmentation performance across the tested random seeds.

Table 14. Multi-seed robustness analysis on MSD. Results are reported as mean ± standard deviation over five matched runs using random seeds 7, 42, 2026, 2077, and 1557. The p-values are obtained using a two-sided paired t-test across matched random seeds. Higher values (↑) are better for IoU and $F_{\beta }$; lower values (↓) are better for MAE. Bold denotes the best mean result for each metric.

Method	IoU ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
SATNet	$85.57\pm 1.07$	$0.928\pm 0.006$	$0.0358\pm 0.0023$
Ours	$\mathbf{88.18}\pm \mathbf{0.30}$	$\mathbf{0.948}\pm \mathbf{0.005}$	$\mathbf{0.0304}\pm \mathbf{0.0009}$
p-value	0.0097	0.0029	0.0043

We further performed a two-sided paired t-test using the results obtained from the matched random seeds. The differences between the proposed model and SATNet are statistically significant for IoU, $F_{\beta }$, and MAE at the significance level of $p<0.05$. These results provide additional evidence that the performance gain over the direct architectural baseline is consistently observed across the tested random seeds rather than being dependent on a particular random initialization.

4.5.10. Ablation on Loss Functions

As defined in Equation (44), HCOCF uses pixel-wise cross-entropy (CE) loss for four deeply supervised decoder outputs, which are combined using the output-level supervision weights $\mathsf{\lambda }=[1.25,1.25,1.0,1.5]$. To investigate whether additional overlap-based or boundary-aware supervision improves the segmentation performance, we constructed two variants by adding Dice Loss or the Boundary Loss proposed by Kervadec et al. [33] to the original CE objective. The Boundary Loss follows the signed-distance-based formulation proposed in [33].

For the CE + Dice Loss variant, the objective function is formulated as

$${\mathcal{L}}_{\mathrm{CE}+\mathrm{Dice}}={\displaystyle \sum _{i=1}^4}{\lambda }_i\left[{\mathcal{L}}_{ce}({\tilde{P}}_i,G)+{\lambda }_{\mathrm{Dice}}{\mathcal{L}}_{\mathrm{Dice}}({\tilde{P}}_i,G)\right],$$

(61)

where ${\lambda }_{\mathrm{Dice}}=1.0$. Similarly, the CE + Boundary Loss variant is formulated as

$${\mathcal{L}}_{\mathrm{CE}+\mathrm{BD}}={\displaystyle \sum _{i=1}^4}{\lambda }_i\left[{\mathcal{L}}_{ce}({\tilde{P}}_i,G)+{\lambda }_{\mathrm{BD}}{\mathcal{L}}_{\mathrm{BD}}({\tilde{P}}_i,G)\right],$$

(62)

where ${\lambda }_{\mathrm{BD}}=0.1$. In both variants, the additional loss was applied to all four deeply supervised decoder outputs. The original output-level supervision weights, network architecture, and all other training and evaluation settings were kept unchanged.

As shown in Table 15, CE combined with Dice Loss achieved an IoU of 87.41%, an $F_{\beta }$ of 0.944, and an MAE of 0.032, while CE combined with Boundary Loss achieved an IoU of 87.21%, an $F_{\beta }$ of 0.951, and an MAE of 0.033. In comparison, the original CE objective achieved an IoU of 88.47%, an $F_{\beta }$ of 0.944, and an MAE of 0.029, yielding the highest IoU and the lowest MAE among the three loss configurations.

Table 15. Ablation study on different loss functions. Higher values (↑) are better for IoU and $F_{\beta }$; lower values (↓) are better for MAE. Bold denotes the best result for each metric.

Loss Function	IoU (%) ↑	${\mathit{F}}_{\mathsf{\beta }}$ ↑	MAE ↓
CE + Dice Loss	87.41	0.944	0.032
CE + Boundary Loss	87.21	0.951	0.033
CE (original)	88.47	0.944	0.029

Compared with the original CE objective, adding Dice Loss reduced IoU by 1.06 percentage points and increased MAE by 0.003, while $F_{\beta }$ remained unchanged. Adding Boundary Loss increased $F_{\beta }$ by 0.007, but reduced IoU by 1.26 percentage points and increased MAE by 0.004. Thus, Boundary Loss resulted in a trade-off among the evaluation metrics rather than a consistent overall improvement, while Dice Loss did not provide an additional performance gain. Based on these results, we retained the original CE objective as the default training objective of HCOCF, as it provided the best overall segmentation performance among the evaluated loss configurations. Moreover, retaining a simple CE objective avoids introducing additional specialized loss engineering, allowing the architectural contribution of HCOCF to be evaluated more directly.

4.5.11. Complexity Analysis

Table 16 provides a component-wise complexity analysis of the SATNet baseline [10], HCOCF, the CSF decoder, and the complete model. All four configurations use the same $512\times 512$ input resolution, Swin-S backbone, codebase, and measurement protocol.

Table 16. Component-wise comparison of computational complexity and inference speed. All configurations use an input resolution of $512\times 512$ and a Swin-S backbone. MACs denote multiply–accumulate operations, and FPS denotes network-forward inference throughput measured under the same protocol for all configurations. Lower MACs and parameter counts (↓) and higher FPS (↑) are preferred.

Configuration	Fusion	Decoder	MACs (G) ↓	Params (M) ↓	FPS ↑
SATNet baseline	SAAM	CFDM	153.58	139.36	52.2
Baseline + HCOCF	HCOCF	CFDM	151.03	120.61	51.8
Baseline + CSF	SAAM	CSF	201.96	166.95	47.4
Ours (HCOCF + CSF)	HCOCF	CSF	199.41	148.20	45.0

The four configurations are designed to isolate the computational effects of the two architectural replacements. The SATNet baseline uses SAAM for high-level branch interaction and the original CFDM decoder. In the Baseline + HCOCF variant, SAAM is replaced by HCOCF while CFDM is retained. In the Baseline + CSF variant, SAAM is retained while CFDM is replaced by CSF. The complete model applies both replacements. This factorized configuration allows the computational contribution of HCOCF and CSF to be evaluated separately.

Replacing SAAM with HCOCF reduces the operation count from 153.58 G to 151.03 G MACs, corresponding to a reduction of 2.55 G (1.66%). The parameter count is also reduced from 139.36 M to 120.61 M, corresponding to a reduction of 18.75 M (13.45%). The measured inference speed changes only slightly from 52.2 to 51.8 FPS. The same complexity reduction is observed under the CSF decoder: replacing SAAM with HCOCF decreases MACs from 201.96 G to 199.41 G and parameters from 166.95 M to 148.20 M. Thus, HCOCF consistently reduces 2.55 G MACs and 18.75 M parameters relative to the SAAM interaction that it replaces, independently of the decoder configuration.

This behavior reflects the different computational structures of the two high-level interaction mechanisms. SATNet employs SAAM for attention-based interaction between the main and symmetry branches, whereas HCOCF performs cross-conditioned DCT-domain spectral attenuation together with attenuation-coefficient prediction and channel recalibration. In our implementation, the resulting HCOCF interaction requires fewer measured MACs and learnable parameters than the SAAM-based interaction.

In contrast, replacing the original CFDM decoder with CSF introduces the main additional computational cost. With SAAM retained, the decoder replacement increases the operation count from 153.58 G to 201.96 G MACs, corresponding to an increase of 48.38 G (31.50%), and increases the parameter count from 139.36 M to 166.95 M, an increase of 27.59 M (19.80%). The same increase is observed in the HCOCF-based configuration, where MACs increase from 151.03 G to 199.41 G and parameters increase from 120.61 M to 148.20 M.

The additional cost of CSF is consistent with its stronger decoder refinement. The original CFDM-style decoder mainly relies on addition-based cross-scale interaction, whereas CSF separately refines the main, mixed, and symmetry-aligned representations under a shared top-down condition before subsequent CCL processing and projection. These branch-specific conditional refinement operations account for most of the additional computational cost of the complete framework.

Overall, the complete model requires 199.41 G MACs and 148.20 M parameters, compared with 153.58 G MACs and 139.36 M parameters for the SATNet baseline. This corresponds to increases of 45.83 G MACs (29.84%) and 8.84 M parameters (6.34%). Therefore, most of the additional arithmetic cost originates from the stronger CSF decoder, while replacing SAAM with HCOCF partially offsets this increase by reducing both MACs and parameter count.

For the inference-speed measurement, latency was evaluated on an NVIDIA RTX 5090 GPU with a batch size of 1 and an input resolution of $512\times 512$. All configurations were evaluated in model.eval() mode under torch.no_grad() using FP32 inference. Fifty forward passes were performed as GPU warm-up and excluded from the measurement. The complete network forward pass was then timed with torch.cuda.synchronize() immediately before and after each inference to ensure that asynchronous CUDA execution was completed within the measured interval.

The timing interval covers all operations executed within the network. For configurations containing HCOCF, this includes the DCT/IDCT operations and spectral attenuation in addition to the backbone and decoder operations. Data loading, input resizing and normalization, tensor conversion, CPU-to-GPU transfer, output post-processing, metric computation, and prediction saving are excluded. The same timing procedure was applied to all configurations in Table 16. The mean per-image forward-pass latency was measured over the MSD test set, and FPS was calculated as the reciprocal of this mean latency. Accordingly, the reported FPS represents network-only inference throughput rather than end-to-end system throughput.

Although replacing SAAM with HCOCF slightly reduces the measured MAC count, the corresponding throughput changes only marginally from 52.2 to 51.8 FPS. This indicates that MAC count and wall-clock inference speed characterize different aspects of computational efficiency. In particular, transform- and memory-related operations such as DCT/IDCT contribute to the measured forward-pass latency but are not necessarily reflected proportionally in the MAC count. The complete model maintains a network-forward throughput of 45.0 FPS, providing a practical reference for the accuracy–efficiency trade-off of the proposed framework.

5. Conclusions

This paper addressed mirror segmentation under challenging reflection conditions, where mirrors exhibit weak intrinsic cues and strong, scene-dependent reflective patterns. We observed that many existing approaches rely on direct spatial-domain fusion, which can amplify reflection-induced high-frequency variations and cause leakage, shape distortion, and unstable boundaries. To overcome these limitations, we proposed a symmetry-aware framework that stabilizes cross-branch interaction through a heat-conduction-inspired cross-conditioned spectral attenuation mechanism. The core contribution of this work is HCOCF, which performs cross-conditioned spectral attenuation for symmetry-aware mirror segmentation. For multi-scale decoding, we adapted the baseline three-branch decoder by replacing simple addition with conditional feature aggregation, which supports boundary recovery and small-region reconstruction. Ablation studies confirm that HCOCF provides the main high-level branch-interaction gain, while the adapted decoder further supports boundary-sensitive reconstruction.

Extensive evaluations on MSD, PMD, and RGBD-Mirror demonstrate strong performance relative to representative supervised mirror-specific baselines. On RGBD-Mirror, the RGB-only HCOCF model achieves competitive performance without requiring depth input. Our additional controlled ablation further shows that incorporating a parallel depth branch improves IoU from 80.87% to 83.06%, indicating that depth can provide complementary information when appropriately integrated. Thus, rather than viewing depth as inherently unreliable in reflective scenes, our results show that HCOCF is effective as an RGB-only framework while remaining compatible with additional depth cues.

Future work will extend the proposed fusion strategy to a broader range of specular and reflective materials and investigate its application to video mirror segmentation. Specifically, the frame-wise spectral attenuation of HCOCF could be retained while temporally aligned features from neighboring frames are incorporated as additional context for the attenuation-coefficient predictor. In this way, the cross-conditioned attenuation coefficients could reflect both within-frame symmetry and temporally stable information across neighboring frames. In addition, a temporal consistency constraint between the current prediction and motion-aligned predictions from neighboring frames could be introduced to reduce frame-to-frame instability, while unreliable correspondences caused by occlusion or large motion could be excluded from this constraint. Integration with geometry-aware downstream tasks, including 3D reconstruction and neural rendering, will also be explored in future work.