3.1. Overall Framework
Figure 1 presents the architecture of the proposed dual-domain reverse distillation algorithm. The framework is developed based on reverse distillation and consists of a frozen teacher encoder E, WHAEM, OCBE, CSGEM, and a student decoder D. The teacher encoder extracts multi-scale features from input samples, and the OCBE module compresses these features into a low-dimensional embedding space that preserves only normal sample patterns. As a result, the student decoder can reconstruct only normal features and fails to restore features in defective regions. To overcome the limitation of relying solely on spatial-domain processing, we combine frequency-domain and spatial-domain information for joint feature enhancement. Specifically, WHAEM leverages wavelet transform to achieve frequency-domain enhancement, while CSGEM performs spatial-domain global feature modeling. The two modules cooperate to strengthen defect feature representation. Deployed on the branch of teacher encoder E, WHAEM amplifies feature differences between teacher and student networks to distinguish defects from normal textures. Embedded after the OCBE module, CSGEM optimizes global semantic features and further boosts the detection performance. At each of the three encoder stages, the WHAEM-enhanced teacher feature Ei is compared against the corresponding student-reconstructed feature Di via cosine similarity to compute a scale-wise discrepancy term Li = 1 − cos(Ei, Di) (i = 1, 2, 3), as illustrated in
Figure 1. These three terms are summed to form the total training loss and also serve as the basis for generating the multi-scale anomaly score map at test time.
The specific embedding positions of WHAEM and CSGEM are not arbitrary design choices, but are dictated by the underlying principle of reverse distillation: because the teacher-student pair is trained exclusively on defect-free samples, the student decoder can only faithfully reconstruct feature patterns that are representative of normal texture, whereas at test time it systematically fails to reconstruct patterns associated with unseen defects, and this failure manifests as a large reconstruction error that serves as the anomaly indicator. Both modules are designed to widen this training-inference discrepancy specifically at defective regions, but they act on different stages of the pipeline and therefore play complementary, non-interchangeable roles.
WHAEM is attached to the teacher encoder branch E, prior to the OCBE bottleneck, rather than to the student decoder D. Because the teacher branch defines the distillation target that the student is trained to reproduce, placing WHAEM at this stage embeds the frequency-domain enhancement directly into the target representation: during training, the student decoder learns to reconstruct the WHAEM-enhanced high-frequency patterns characteristic of normal fiber texture. At test time, when a defect is present, its high-frequency response is structurally inconsistent with the normal-texture patterns the student has learned, and this response is further amplified by WHAEM before being passed to the teacher-side target, while the student decoder, having never observed such patterns, cannot reproduce them—thereby widening the feature discrepancy specifically at defect locations. If WHAEM were instead placed on the student branch, the enhancement would act on the student’s own reconstructed, already normal-biased features rather than on the teacher’s target, offering no additional discriminative signal and potentially amplifying reconstruction noise rather than genuine defect cues. Moreover, embedding WHAEM before, rather than after, the OCBE compression ensures that fine-grained high-frequency detail is enhanced while the multi-scale spatial resolution of the teacher encoder is still preserved; applying this enhancement after OCBE’s bottleneck would operate on an already information-impoverished embedding in which subtle high-frequency defect cues have been attenuated.
CSGEM, in contrast, is embedded between OCBE and the student decoder D, i.e., on the student-side reconstruction path rather than on the teacher branch. This placement is likewise dictated by the reverse distillation mechanism: OCBE compresses the teacher’s multi-scale features into a low-dimensional embedding that retains only the statistical regularities of normal samples, and it is the student decoder’s task to expand this embedding back into full-resolution features. By inserting CSGEM immediately after OCBE, the student decoder’s reconstruction is conditioned on globally aggregated contextual information that reflects the long-range spatial regularity typical of normal fabric texture. For normal regions, this global context is consistent with the local content and therefore facilitates accurate reconstruction, keeping the residual error low. For defective regions, however, the defect locally violates the long-range structural regularity that CSGEM aggregates from the surrounding normal context; as a result, the self-attention-guided reconstruction is biased toward the incorrect, normal-consistent pattern, which further enlarges the discrepancy with the teacher’s target feature at the defect location. Critically, embedding CSGEM on the student side rather than the teacher side preserves the asymmetry between the two branches that reverse distillation fundamentally relies on: if CSGEM were instead applied to the teacher branch, the resulting globally regularized target would already encode long-range consistency that the student could trivially match even in the presence of local anomalies, thereby weakening rather than strengthening the reconstruction-error-based anomaly signal.
In summary, WHAEM operates on the teacher branch prior to bottleneck compression to inject frequency-domain discriminability into the distillation target itself, while CSGEM operates on the student branch after bottleneck compression to regularize the reconstruction process with global spatial context. Together, the two modules widen the teacher-student reconstruction discrepancy at defective regions from complementary frequency and spatial perspectives, while preserving the teacher-student asymmetry that is essential for the reverse distillation framework to remain sensitive to unseen defects.
3.2. Wavelet High-Frequency Deformable Enhancement Module
In defect detection, defect features correspond to high-frequency signals relative to background features. Therefore, we propose to enhance detection performance in the frequency domain by designing a frequency-domain enhancement module named the Wavelet High-frequency Deformable Enhancement Module (WHAEM). Inspired by recent advances in Discrete Wavelet Transform (DWT) and Inverse Discrete Wavelet Transform (IDWT) [
20,
21,
22,
23,
24,
25,
26], which have become active research topics in deep learning, WHAEM incorporates wavelet-based operations to strengthen high-frequency feature representation. The structure of the module is illustrated in
Figure 2.
The overall framework of the Wavelet High-frequency Deformable Enhancement Module (WHAEM) first employs the Haar wavelet transform to decompose the input feature map into four frequency subbands:
The low-frequency subband retains the global structure of the input feature map, while the three high-frequency subbands capture texture detail; the biorthogonal property of the wavelet transform ensures no information loss [
27]. The high-frequency subbands are processed by the Atrous Deformable Attention Module (ADAM), while the low-frequency subband is left unchanged. Finally, IDWT converts the processed subbands back to the spatial domain to obtain the output feature map:
Formally, the Haar wavelet is an orthogonal wavelet whose scaling function is a piecewise-constant function equal to one over the unit interval from zero to one and equal to zero elsewhere, and whose wavelet function is equal to one over the first half of that unit interval, equal to negative one over the second half, and equal to zero elsewhere. Correspondingly, the associated one-dimensional low-pass filter consists of two equal coefficients, each equal to one over the square root of two, while the associated one-dimensional high-pass filter consists of a first coefficient equal to one over the square root of two and a second coefficient equal to negative one over the square root of two; both filters are used for decomposition and reconstruction in this work. In the proposed WHAEM module, these two one-dimensional filters are combined through outer products to construct four separable two-dimensional kernels: the low-low kernel is obtained from the outer product of the low-pass filter with itself, the low-high kernel from the outer product of the low-pass filter with the high-pass filter, the high-low kernel from the outer product of the high-pass filter with the low-pass filter, and the high-high kernel from the outer product of the high-pass filter with itself. These four kernels correspond, respectively, to the LL (low-low), LH (low-high), HL (high-low), and HH (high-high) sub-bands, and are applied as strided two-dimensional convolutions to implement the discrete wavelet transform described above, with their transposed counterparts used for the inverse transform.
The Haar wavelet is specifically adopted, rather than other wavelet families such as Daubechies or Symlet, for three main reasons. First, the Haar wavelet has the simplest computational form among orthogonal wavelets, involving only addition and subtraction operations without floating-point multiplication overhead, which makes it well suited for real-time industrial inspection and potential deployment on resource-constrained embedded platforms. Second, the Haar wavelet possesses compact support and strict orthogonality, which guarantees precise spatial localization of frequency components without introducing redundant or overlapping information across sub-bands—a property that is particularly desirable for accurately localizing small and sharply bounded surface defects. Third, the multi-scale, multi-directional decomposition provided by the Haar wavelet naturally separates the low-frequency global texture structure from high-frequency directional details (horizontal, vertical, and diagonal), which aligns well with the goal of disentangling the random background texture of non-woven fabrics from the localized, high-frequency signatures of genuine defects.
WHAEM adaptively strengthens high-frequency defect features while suppressing background noise. However, traditional attention mechanisms model features along a single dimension with a fixed receptive field, often losing fine details of tiny defects [
28]. To address this, we propose the Atrous Deformable Attention Module (ADAM), which expands the receptive field via multi-scale atrous convolution, achieves precise focus on high-frequency defects through mean and maximum feature fusion, and adaptively learns key feature weights, thereby enhancing defect representation and suppressing noise. The structure of the module framework is presented in
Figure 3.
ADAM is applied in the comparison stage of the teacher–student model, leveraging local channel information to highlight the defect features extracted by the teacher model [
29]. As shown in
Figure 3, given an input multi-channel feature map, ADAM sequentially performs six steps: channel statistical information extraction, bottleneck compression, atrous multi-scale feature encoding, deformable gated interaction, attention weight generation, and weighted feature output, ultimately producing an attention-calibrated feature map. For the input convolutional feature map:
First, global average pooling and global maximum pooling are applied along the channel dimension to obtain two single-channel feature maps that respectively capture the overall feature distribution and salient local responses, while preserving the original spatial resolution and avoiding early information loss.
These two maps are then compressed by a 1 × 1 convolution to reduce redundancy and computational overhead, followed by a 3 × 3 atrous convolution (dilation rate 2) that expands the receptive field for multi-scale context modeling without downsampling or changing the spatial size. In the deformable gated interaction stage, independent convolutional layers—zero-initialized for training stability—learn modulation weights for the mean and maximum branches; after activation, the two branches undergo cross-gated interaction to achieve complementary feature enhancement and distortion correction, expressed as:
A backpropagation-scaling mechanism is further applied to the two branches to suppress gradient oscillation and improve convergence stability. In the attention weight generation step, the interacted features are concatenated in the channel dimension and fused via a 3 × 3 convolution, followed by a Sigmoid activation [
30] to generate a full-resolution spatial attention mask, given as:
This avoids the need for upsampling interpolation. Finally, the mask is multiplied pixel-wise with the input features to enhance key target regions and suppress background information, followed by a ReLU activation [
31] to output the calibrated feature map, expressed as:
By preserving full spatial resolution, expanding the receptive field without added parameters, and adaptively modeling feature deformation, ADAM effectively captures fine-grained defect details in a lightweight manner, while maintaining strong modeling capability for multi-scale and complex distortion scenarios.
Figure 4 illustrates the intermediate feature representations of the WHAEM module at each key processing stage on a representative WFB nonwoven fabric patch containing subtle defects (a low-contrast patch and a small dot defect, visible in the bottom-right corner of the input). As the starting point of WHAEM, the Input fabric patch is the original grayscale image directly fed into the teacher encoder stage E1. After DWT decomposition, the low-frequency subband LL (low-frequency, kept as-is) preserves the rough global texture structure of the fabric. Defect regions with low contrast are nearly indistinguishable at this stage, confirming that the low-frequency components carry minimal defect-discriminative information in this context and require no further enhancement. Meanwhile, the raw high-frequency subbands LH (raw, after DWT split), HL (raw, after DWT split), and HH (raw, after DWT split) before ADAM processing capture horizontal, vertical, and diagonal detail components, respectively. Although defect-related high-frequency responses are already present in these subbands, they are mixed with broadband background texture noise, making it difficult to disentangle the true defect clues from the background. Additionally, the bright edge artifacts visible in the LH and HL subbands originate from the boundary padding applied prior to the DWT transformation.
To address the issue of high-frequency noise mixing, the ADAM-processed high-frequency subbands LH after DeformableAttention, HL after DeformableAttention, and HH after DeformableAttention exhibit a critical feature evolution: the defect-related feature responses are selectively amplified by ADAM, while the spatially uniform background texture noise is significantly suppressed. This enhancement effect is intuitively reflected in the corresponding spatial attention masks, namely the LH attention mask, HL attention mask, and HH attention mask; these masks assign higher attention weights (bright regions) to subtle defect areas, while assigning lower weights (dark regions) to uniform background areas and boundary artifacts. Finally, the Output after IDWT (reconstructed) integrates the unmodified LL low-frequency structure with the three ADAM-enhanced high-frequency subbands via inverse discrete wavelet transform, generating a spatial-domain feature map with superior defect discriminability. This output serves as the WHAEM-enhanced teacher feature and is subsequently passed to the teacher-student distillation stage.
3.3. Convolutional Self-Attention Spatial Global Enhancement Module
This paper proposes the Convolutional Self-Attention Spatial Global Enhancement Module (CSGEM). Based on convolutional self-attention, this module realizes global dependency modeling in the spatial domain. It preserves local detailed features through residual connections and captures long-range spatial correlations via global self-attention, achieving fine-grained enhancement of spatial features and global information fusion. It significantly improves the model’s capability to perceive and locate spatial features of defective regions. The overall structure of the module is illustrated in
Figure 5. The input to the module is a batch of high-order convolutional feature maps output by the network, where B, C, H, and W denote the batch size, number of channels, height, and width of the feature maps, respectively, which is defined as:
where B, C, H, and W denote the batch size, channel dimension, and spatial height and width, respectively. CSGEM sequentially performs three processes: local convolutional feature refinement, multi-head global self-attention modeling, and adaptive residual feature fusion.
First, a 3 × 3 convolution with symmetric padding extracts local fine-grained features (e.g., textures, edges) while preserving spatial resolution, followed by batch normalization and ReLU activation to stabilize training and introduce nonlinearity.
The core of the module is the multi-head global self-attention branch, which overcomes the fixed receptive field of standard convolutions to model long-range spatial dependencies. The refined features are projected via 1 × 1 convolutions into Query, Key, and Value representations, then flattened into a spatial sequence and processed in multiple heads, with correlation weights computed via scaled dot-product attention:
To ease gradient propagation, CSGEM applies an adaptive residual connection: a 1 × 1 convolution aligns the residual branch when input–output dimensions or strides differ, otherwise identity mapping is used. The residual features are fused element-wise with the attention-enhanced features and calibrated by a ReLU activation:
By integrating local convolutional feature refinement with global self-attention modeling and adaptive residual fusion, CSGEM effectively combines fine-grained local details with long-range global context, enhancing feature discriminability while maintaining stable and efficient training.