Frequency-Aware Multi-Rate Resampling with Multi-Band Deep Supervision for Modular Speech Denoising

Abstract

Conventional waveform-based speech enhancement models prioritize temporal modeling, often neglecting the irreversible spectral information loss triggered by standard downsampling. Consequently, this study introduces a novel frequency-aware framework. The proposed approach incorporates a modular, multi-rate resampling module with principled anti-aliasing to precisely control each layer’s effective frequency band, complemented by a multi-band loss function for deep supervision. Integrating this module into a standard Wave-U-Net and an attention-enhanced variant confirmed its effectiveness. The findings show a significant improvement over the baseline, yielding an average Perceptual Evaluation of Speech Quality gain of 0.40, with further benefits when paired with an advanced temporal model at a permissible increase in computational complexity. Furthermore, tests on novel noise types validate the generalizability of the proposed principles, establishing structured frequency band allocation as a fundamental, modular design strategy for improving end-to-end models.

1. Introduction

High-quality speech signals are crucial for various modern intelligent systems, spanning high-fidelity audio restoration to real-time communication tools. However, ambient noise remains a major limitation in intelligent signal processing. Historically, the field relied on simplified statistical approaches like spectral subtraction [1] and Wiener filtering [2]. Although minimum mean-square error (MMSE) estimators [3] were more rigorous, their efficacy remained limited by assumptions about stationary noise, deteriorating performance in dynamic environments and introducing artifacts [1].

Early machine learning models like non-negative matrix factorization (NMF) [4] bridged the gap between conventional and modern methods by learning basis vectors directly from data, offering greater flexibility than their fixed statistical predecessors. Nevertheless, their linear structure constrained performance in highly complex, non-stationary noise environments, necessitating a paradigm shift.

The emergence of deep learning has transformed speech enhancement, creating two streams: Short-Time Fourier Transform (STFT)-based techniques, which operate on time-frequency magnitude but rely on noisy phase estimates [5], and end-to-end (E2E) models, which process raw waveforms directly with notable effectiveness [6]. These models learn complex and non-linear mappings from noisy to clean speech, enabling effective noise suppression without explicit assumptions. Architectures like Wave-U-Net [7] exemplify E2E’s effectiveness in multi-scale feature learning in the time domain, reducing reliance on STFT-phase assumptions.

Recent advancements in speech enhancement primarily stem from temporal modeling using recurrent neural networks (RNNs) [8] and self-attention mechanisms [9]. In addition, the field has advanced with other prominent streams over the last three years. These include diffusion and generative models that integrate denoising with high-fidelity restoration, sub-band and hybrid approaches that integrate full- and sub-band interactions, and waveform-based or low-latency systems for real-time efficiency, such as DeepFilterNet2, which emphasize computational efficiency without sacrificing perceptual quality [10,11,12,13,14,15,16,17].

Although these approaches are central to modern speech processing and have yielded substantial gains, a common, overlooked challenge remains: the irreversible loss of high-frequency spectral detail caused by standard downsampling techniques (strided convolutions) in several encoder–decoder architectures.

Instead of proposing another end-to-end backbone, this study introduces a novel, frequency-aware multi-rate resampling module. This module explicitly sets each layer’s passband and enforces principled anti-aliasing, mitigating irreversible spectral loss and aliasing-induced spectral distortion. To maximize effectiveness, we employ a complementary multi-band deep supervision strategy that emphasizes perceptually critical bands and suppresses residual artifacts. We demonstrate that our approach is backbone-agnostic and parameter-neutral, complements existing temporal modules, is robust against unseen noise, and performs favorably against representative same-type models like CleanUNet [9] and Conv-TasNet [18].

The primary objective is to propose and validate this modular approach rather than introducing a novel state-of-the-art model. The remainder of this paper is organized as follows. Section 2 reviews the limitations of existing models. Section 3 details the proposed framework. Section 4 outlines the validation setup. Section 5 reports the empirical results. Finally, Section 6 concludes with key findings.

2. Related Work

2.1. Wave-U-Net and the Focus on Temporal Modeling

E2E models operating directly on raw waveforms have garnered significant attention, with Wave-U-Net serving as a foundational architecture [6,7]. As Figure 1a illustrates, it adapts the U-Net’s symmetric encoder–decoder structure [19] to 1D audio. The encoder uses strided downsampling to extract hierarchical features, and the decoder mirrors this process, fusing multi-scale information via skip connections for precise reconstruction. In multi-rate digital signal processing, downsampling is ideally preceded by a low-pass anti-aliasing filter to prevent irreversible aliasing [20]. Section 2.3 analyzes the impact of omitting this filter and its interaction with temporal modules.

Accordingly, subsequent studies extend Wave-U-Net to capture longer-range temporal dependencies using methods like inserting an LSTM at the bottleneck to model sequential patterns [8] or applying attention to represent non-local relationships across the signal [9]. Although these mechanisms enhance temporal modeling and yield performance improvements, they remain constrained by artifacts introduced by early-stage decimation.

2.2. Comparison with Sub-Band and Hybrid Approaches

The proposed frequency-aware design differs from related sub-band and hybrid full/sub-band approaches in both objective and implementation. FullSubNet-style models operate in the STFT domain and explicitly utilize sub-bands and their cross-band interactions [21,22]; however, DTLN primarily processes STFT magnitudes in a two-stage pipeline without parallel sub-band decomposition [23]. In contrast, the proposed method works on raw waveforms and employs principled multi-rate resampling to develop a hierarchy where each layer has an explicitly controlled bandwidth [20]. Rather than parallel inter-band modeling [21,22], the emphasis is on layer-wise bandwidth allocation with explicit pre-decimation low-pass filtering [20], directly targeting aliasing and spectral loss due to conventional downsampling, a challenge orthogonal to primary sub-band research objectives [24].

2.3. Spectral Information Loss from Downsampling

Although temporal enhancements are effective, encoder–decoder architectures relying on strided decimation reduce the internal sampling rate of each branch, thereby limiting its maximum representable frequency (Nyquist–Shannon sampling theorem [24]; Figure 1b). When high-frequency content is not preserved via skip paths or parallel high-rate branches, the necessary low-pass filtering before decimation discards energy above the new Nyquist limit [20]; if anti-aliasing is insufficient, this energy folds back as aliasing [20]. In both situations, information outside the reduced band is lost, degrading cues crucial for clarity and intelligibility. Hence, we adopt the original Wave-U-Net as the baseline configuration to isolate and quantify the contribution of the proposed frequency-aware module [6,7].

2.4. Recent Advancements (2022–2025)

Recent research in speech enhancement has evolved along three closely related directions that frame our contribution. Sub-band and hybrid full/sub-band models integrate band-aware processing with full-band context; for example, TF-GridNet integrates global information with sub-band attention [11] and band-split RNN variants refine high-fidelity enhancement by operating on band-structured representations [12]. Our method stands out by assigning layer-wise bandwidths via principled resampling with anti-aliasing, addressing aliasing and irreversible spectral loss stemming from conventional downsampling.

A separate line of research explores diffusion and other generative models, exhibiting competitive quality and generalization in speech enhancement. Recent designs minimize sampling steps to improve practicality [10,16]; however, iterative reverse processes increase latency and implementation complexity. Conversely, our approach is discriminative and single-pass, focusing on front-end spectral integrity that can support both generative and non-generative backbones.

Research on waveform domain has also emphasized runtime efficiency and low latency. Foundational time-domain systems like Conv-TasNet established strong baselines for streaming separation [18], and subsequent designs (CleanUNet [9] and Hybrid Demucs [13]) improved the quality–efficiency trade-off via architectural and training innovations. Accordingly, our module is backbone-agnostic and parameter-neutral: it introduces a fixed, modest computational budget while delivering consistent gains in PESQ [25] and STOI [26], remaining compatible with resource-efficient systems.

3. Proposed Model Architecture

3.1. Design Principle and Overall Structure

We define the noisy mixture as $y\left[n\right]=s\left[n\right]+d\left[n\right]$, where $s\left[n\right]$ and $d\left[n\right]$ denote the clean target signal and additive noise, respectively. The network estimates $\widehat{s}\left[n\right]$ from $y\left[n\right]$, aiming to maximally match $s\left[n\right]$. For brevity, the time-sample index is omitted when the context is clear.

The proposed framework (Figure 2) constructs a frequency-aware hierarchy by explicitly controlling each layer’s passband. We achieve this by inserting a frequency-aware multi-rate resampling module (PR2) and applying a multi-band deep supervision strategy (PR3). The PR2 module applies principled anti-aliased decimation and anti-imaging interpolation, following the internal sampling-rate schedule detailed in

Table 1. Layer-wise internal sampling rates and effective frequency bands for the conventional and proposed Wave-U-Net. All values are in kHz, with the effective band defined as half of the internal sampling rate.

Encoder–Decoder Level	Conventional Fs	Conventional Effective Band	Proposed Fs	Proposed Effective Band
0	16.00	8.00	16.00	8.00
1	8.00	4.00	14.00	7.00
2	4.00	2.00	12.00	6.00
3	2.00	1.00	10.00	5.00
4	1.00	0.50	8.00	4.00
5	0.50	0.25	6.00	3.00
6	0.25	0.125	4.00	2.00
7	0.125	0.0625	2.00	1.00
8	0.0625	0.0313	1.50	0.75
9	0.0313	0.0156	1.00	0.50
10	0.0200	0.0100	0.50	0.25
11	0.0100	0.0050	0.20	0.10

. This minimizes aliasing-induced spectral distortion early, allowing subsequent modules to operate on cleaner representations. The PR3 stage then refines residual artifacts by emphasizing perceptually critical bands.

This architecture’s flow differs from conventional U-Nets. In the encoder path, the signal passes through an encoder block (Conv/BN/LeakyReLU) followed by PR2 resampling, which decimates the signal ([ConvBlock] $\to$ [Resample] order). In the decoder path, the flow is reversed: the signal is first upsampled by PR2, then processed by the decoder block, which also receives the skip connection. PR3 taps compute auxiliary losses at intermediate stages. This sequential process provides explicit, layer-wise passband control. Figure 2 (accompanying note) details the specific hyperparameters, resampling configuration, and resulting feature lengths for reproducibility.

3.2. Multi-Rate Resampling Module

To precisely control the frequency band at each layer, we substitute the strided convolutions and their upsampling counterparts with a resampling module based on multi-rate signal processing techniques [20]. We investigate two variants of this module to experimentally validate the importance of principled anti-aliasing.

The first variant, PR1, functions as a controlled baseline designed to explicitly illustrate aliasing effects. It implements resampling by a rational factor $I/D$ using a naive two-step process. First, the input signal $x\left[n\right]$ (where $n$ denotes the input sample index) is up-sampled by a factor of $I$ to generate an intermediate signal $\nu \left[m\right]$ (where $m$ is the intermediate index):

$$\nu \left[m\right]=\left\{\begin{array}{cc}\hfill x\left[m/I\right],& \mathrm{if} m \mathrm{is} \mathrm{a} \mathrm{multiple} \mathrm{of} I\hfill \\ \hfill 0 ,& \mathrm{otherwise} \hfill \end{array}\right.$$

(1)

Subsequently, the intermediate signal $\nu \left[m\right]$ is downsampled by a factor of $D$ to generate the final output $y\left[k\right]$ (where $k$ represents the output index) by retaining only every $D$-th sample according to the expression:

$$y\left[k\right]=\nu \left[k·D\right].$$

(2)

Although this process attains the target frequency bands in Table 1, omitting a pre-decimation anti-aliasing low-pass filter is likely to cause substantial spectral aliasing [20].

Conversely, the second variant, PR2, mitigates these artifacts using a rigorously designed anti-aliasing filter. This module employs a windowed-sinc low-pass prototype to theoretically minimize aliasing, defined as:

$$h\left[\mathcal{l}\right]=2f_c\mathrm{s}\mathrm{i}\mathrm{n}\mathrm{c}\left(2\pi f_c\left(\mathcal{l}-{\displaystyle \frac{N-1}{2}}\right)\right)w\left[\mathcal{l}\right], 0\le \mathcal{l}<N,$$

(3)

where $N$, $\mathcal{l}$, $w\left[\mathcal{l}\right]$, and $f_c$ represent the filter length, index for the filter taps, a window function, and normalized cutoff relative to this resampling stage’s input rate, respectively. To suppress aliasing from decimation and imaging during interpolation, the cutoff is set to:

$$f_c={\displaystyle \frac{1}{2 \mathrm{m}\mathrm{a}\mathrm{x}(I,D)}}$$

(4)

Interpolation, filtering, and decimation are integrated into a single operation to yield $y\left[k\right]$:

$$y\left[k\right]=\sum _{\mathcal{l}=0}^{N-1}h\left[\mathcal{l}\right]x_{\uparrow I}\left[kD-\mathcal{l}\right]$$

(5)

where $x_{\uparrow I}$ denotes $x\left[n\right]$ upsampled by a factor of $I$. Equation (5) is algebraically equivalent to upsampling, applying the filter $h\left[\mathcal{l}\right]$, and then decimation by $D$.

Table 1 summarizes the layer-wise internal sampling rate and corresponding effective frequency bands for the conventional Wave-U-Net and our proposed design. The values are based on a 16-kHz input sampling rate and scale linearly for others. In the conventional model, the “internal rate” (Table 1) reflects the equivalent rate from stride-two downsampling in the encoder, with the effective band at each level being half that rate under ideal anti-aliasing [24]. This shows the progressive reduction of high-frequency content with depth.

In the proposed design, each level regulates its internal rate via explicit resampling with an anti-aliasing filter [20]. The internal rates are scheduled as 16.00, 14.00, 12.00, 10.00, 8.00, 6.00, 4.00, 2.00, 1.50, 1.00, 0.50, and 0.20 kHz. The corresponding effective bands are 8.00, 7.00, 6.00, 5.00, 4.00, 3.00, 2.00, 1.00, 0.75, 0.50, 0.25, and 0.10 kHz. This schedule preserves more mid- and high-frequency information in upper layers while progressively narrowing bandwidth near the bottleneck, where coarse low-frequency structure suffices.

Regarding the architecture’s depth, the diagram sometimes refers to “11 encoder and 11 decoder blocks” by excluding the input level. Our schedule comprises 12 levels, indexed 0–11 (Table 1), which aligns with both the internal sampling-rate plan and channel progression (24–288). This depth effectively balances the receptive field and band-allocation fidelity for 16-kHz inputs. Deeper or shallower variants could be used to target different trade-offs between computation and quality.

3.3. Multi-Band Deep Supervision

Although the PR2 module establishes a frequency-aware architecture, an effective training strategy is essential to preserve multi-scale representation fidelity. Consequently, we introduce a multi-band loss that applies deep supervision [27] at intermediate decoder stages. Rather than utilizing the Speech Intelligibility Index (SII) as a training metric, we leverage its band-importance guidance to select the supervised bands [28]. The resulting model, (PR3), combines the PR2 architecture with this targeted training strategy. The total PR3 training loss is a weighted combination of the primary and auxiliary losses:

$${\mathcal{L}}_{total}=\left(1-\alpha \right){\mathcal{L}}_{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{r}\mathrm{y}}+\alpha {\mathcal{L}}_{\mathrm{a}\mathrm{u}\mathrm{x}},$$

(6)

where $\alpha \in \left[0, 1\right]$ governs the balance between the two loss terms and is set to 0.2 in our experiments; when auxiliary supervision is not applied, $\alpha$ is set to $0$. Here, we define the two loss components. The primary loss, ${\mathcal{L}}_{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{r}\mathrm{y}}$, applied to the final output, is the negative scale-invariant signal-to-distortion ratio (SI-SDR) [29] applied to the final output:

$${\mathcal{L}}_{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{m}\mathrm{a}\mathrm{r}\mathrm{y}}=-\mathrm{S}\mathrm{I}-\mathrm{S}\mathrm{D}\mathrm{R}(\widehat{\mathit{s}},\mathit{s}).$$

(7)

The auxiliary loss, ${\mathcal{L}}_{\mathrm{a}\mathrm{u}\mathrm{x}}$, is defined as a weighted sum of SI-SDR losses (denoted ${\mathcal{L}}_j$) calculated at $K-1$ intermediate decoder outputs, providing targeted supervision:

$${\mathcal{L}}_{\mathrm{a}\mathrm{u}\mathrm{x}}=\sum _{j=1}^{K-1}{\lambda }_j{\mathcal{L}}_j,\ge 0,$$

(8)

where ${\mathcal{L}}_j=-\mathrm{S}\mathrm{I}-\mathrm{S}\mathrm{D}\mathrm{R}({\widehat{\mathit{s}}}^{\left(j\right)},{\mathit{s}}^{\left(\mathit{j}\right)})$ represents the negative SI-SDR computed at the j-th intermediate reconstruction. $K$ denotes the total number of supervised decoder stages, including the final output. The choice of intermediate taps is task-specific and independent of the frequency schedule in Table 1. Guided by SII-based band importance [28], we prioritize bands centered at approximately 0.25, 0.50, 0.75, and 4.00 kHz by setting their corresponding ${\lambda }_j$ to 1.0 and 0 for all others.

The full-length estimated $\widehat{\mathit{s}}\left[n\right]$ and clean $\mathit{s}\left[n\right]$ signals are represented as vectors $\widehat{\mathit{s}}$ and $\mathit{s}$, respectively. Subsequently, SI-SDR is defined with a small constant $\epsilon$ for numerical stability as:

$$\mathrm{S}\mathrm{I}-\mathrm{S}\mathrm{D}\mathrm{R}(\widehat{\mathit{s}},\mathit{s})=10{\mathrm{l}\mathrm{o}\mathrm{g}}_{10}\left({\displaystyle \frac{{‖{\mathit{s}}_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}}‖}^2}{{‖{\mathit{e}}_{res}‖}^2+\epsilon }}\right),$$

(9)

with

$${\mathit{s}}_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}}={\displaystyle \frac{〈\widehat{\mathit{s}},\mathit{s}〉}{{‖\mathit{s}‖}^2}}\mathit{s},{\mathit{e}}_{\mathrm{r}\mathrm{e}\mathrm{s}}=\widehat{\mathit{s}}-{\mathit{s}}_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}},$$

(10)

where ${\mathit{s}}_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}}$ and ${\mathit{e}}_{\mathrm{r}\mathrm{e}\mathrm{s}}$ represent the projection of the estimated signal onto the clean target vector and residual error orthogonal to the target, respectively.

3.4. Analysis of the Proposed Framework

Table 2. Overview of the computational complexity analysis. RTF is measured on an NVIDIA A40 GPU using a 1.024-s input (16 384 samples at 16 kHz).

Model	Parameters (M)	GFLOPs	RTF
Baseline Wave-U-Net	10.13	4.92	0.0042
Proposed (PR1)	10.13	38.28	0.0048
Proposed (PR2)	10.13	38.18	0.0142
Proposed (PR3)	10.13	38.18	0.0142

summarizes the computational complexity of the proposed framework, highlighting the trade-off between the computational investment and perceptual quality. Computational requirements are quantified using Giga Floating-point Operations (GFLOPs) and Real-Time Factor (RTF).

The proposed resampling module incurs a fixed computational cost of ~33 GFLOPs measured at 16 kHz (Table 2), representing a significant increase over the lightweight Wave-U-Net baseline. However, the absolute cost remains practical for most applications. The final RTF of 0.0142 is well below the real-time processing threshold (RTF < 1), which indicates throughput faster than real time on that setup. Because RTF measures throughput, not algorithmic latency, a fast non-causal system can still be unsuitable for low-latency streaming.

These perceptual improvements are achieved without increasing the model’s parameter count, offering a fixed computational increase that delivers substantial quality gains. This observation aligns with establishing upper-bound potential under a non-causal setting. Adapting the module to a computationally optimized causal design for low-latency applications is a key avenue for future research.

4. Experimental Setup

4.1. Experimental Design, Datasets, and Metrics

To provide a scientifically rigorous evaluation, we conducted a controlled ablation study using two baseline models. The first, the original Wave-U-Net [6], functioned as a testbed for isolating spectral aliasing. The second, an attention-enhanced Wave-U-Net with a shuffle attention module [30] at the bottleneck, assessed the proposed approach’s modularity and complementary benefits.

To test for complementarity, we included an attention block at the bottleneck to strengthen temporal context modeling [30]. Although this temporal module does not constitute the proposed study’s novelty, it serves as a representative temporal enhancement. The resulting models evaluated herein include the baseline Wave-U-Net, Model A (Baseline + Attention modules), Model B (Baseline + PR2/PR3 modules), and Model C (Attention + PR2/PR3 modules).

Experiments utilized clean speech from the TIMIT database [31] and noise from the NOISEX-92 database [32]. All audio was resampled to 16 kHz. The training set comprised 4620 clean speech utterances from the TIMIT-TRAIN partition. The validation set was created by holding out 10% of these training utterances at the speaker level to prevent speaker overlap. The test set used speakers from the standard TIMIT-TEST partition, ensuring all test speakers were disjoint from those in the training and validation sets.

Training mixtures were constructed dynamically. For each clean utterance, we randomly selected one of the six noise types (babble, f16, factory, pink, Volvo, and white) from NOISEX-92. We also selected a signal-to-noise ratio (SNR) uniformly from the set {−10, −5, 0, 5, 10, 15} dB. Given a clean signal $s\left[n\right]$ and a randomly cropped noise segment $d\left[n\right]$, the noise was scaled to the target SNR via RMS energy matching to produce the mixture: $y\left[n\right]=\mathrm{s}\left[\mathrm{n}\right]+\mathsf{\gamma }d\left[\mathrm{n}\right],$ where $\mathsf{\gamma }=\sqrt{{\displaystyle \frac{{\sum }_n{s\left[n\right]}^2}{{\sum }_n{d\left[n\right]}^2}}·{10}^{-SNR/10}}$. Validation and test mixtures were achieved via the same procedure with their respective data splits. This process ensures a balanced range of difficulty, with a uniform SNR histogram and equal probability for all six noise categories.

Performance was rigorously evaluated using two standard objective metrics: PESQ [25] and STOI [26]. PESQ, following the ITU-T P.862 standard, correlates with perceived speech quality. For 16-kHz narrow-band signals, these scores typically span from approximately 1 to 4.5. STOI estimates intelligibility on a scale of 0–1, where values closer to 1 indicate better intelligibility. Here, we report “improvement” relative to the unprocessed noisy signal; otherwise, we report absolute scores. We specify in each figure and table caption whether the values are absolute or relative and the SNR ranges over which the aggregation is performed.

4.2. Implementation Details

Our implementation adopts the Wave-U-Net backbone architecture (Figure 2). Figure 2 (accompanying note) presents the specific architectural hyperparameters, including kernel sizes, channel counts, activation functions, and the precise resampling configuration (window function, filter width, and rolloff) for clarity and reproducibility. Optimization employs Adam with a fixed learning rate.

Table 3. Network architecture and training hyperparameters.

Hyperparameter	Setting
Optimizer	Adam
Learning Rate	5 × 10−4 (fixed)
Adam Betas	(0.9, 0.999)
Batch Size	32
Max Epochs	1200
Early Stopping Patience	300
Encoder Kernel Size	15
Decoder Kernel Size	5
Activation Function	LeakyReLU (slope = 0.1)
Final Activation Function	Tanh
Encoder Channels	24, 48,..., 288

summarizes other key training hyperparameters not included in the core architecture.

All experiments were run on three NVIDIA A40 GPUs in a multi-GPU configuration. However, the RTF measurement context (Table 2) was conducted on a single GPU. We report the operating system, software stack versions, hardware, and controls for randomness and determinism. Unless otherwise noted, all training and evaluation adopted FP32 precision. The employed operating system and language runtime were Ubuntu 22.04 LTS and Python 3.10.12, respectively. Our deep learning stack comprised PyTorch 2.7.1 with CUDA 12.6. We utilized three NVIDIA A40 GPUs (48-GB memory each) for all multi-GPU training and inference. For randomness and determinism, we set a global seed of 1337, set torch.backends.cudnn.benchmark to False, and enabled deterministic algorithms via torch.use_deterministic_algorithms. For data loading, we used eight workers with persistent workers and pin memory enabled. The precision was FP32, with Automatic Mixed Precision disabled. Our RTF protocol utilized a batch size of 1, an input length of 1.024 s, and was averaged over 100 trials on a single GPU.

5. Results and Discussion

We evaluated three training losses, including mean squared error (MSE), weighted SDR (wSDR) [29], and SI-SDR [29], to determine the criterion for all experiments. Using the baseline Wave-U-Net, we trained models with each loss function and averaged results across all test conditions. With an average PESQ of 2.12, SI-SDR yielded the highest perceptual quality, outperforming wSDR (2.09) and MSE (2.03). Therefore, SI-SDR was adopted for all subsequent experiments.

5.1. Performance Analysis of Proposed Methods

5.1.1. Effect of Principled Resampling with Anti-Aliasing

To contextualize performance gains, we first evaluated the absolute quality of the unprocessed noisy signals. On average, these signals achieved STOI and PESQ values of 0.749 and 1.467, respectively, with PESQ varying across noise types from 1.266 (white noise) to 1.938 (Volvo noise). Although this variability in intelligibility and quality establishes a challenging baseline, the standard Wave-U-Net achieved a substantial initial enhancement, averaging 0.882 and 2.25 in STOI and PESQ, respectively.

Figure 3 illustrates each model’s enhancement relative to the original noisy signal. The labels “Low SNRs” and “High SNRs” correspond to −10–0 dB and 5–15 dB, respectively. The baseline (Wave-U-Net) achieves a significant gain; however, PR1 and PR2 outperform it across all conditions, with the most significant advantage observed at “Low SNRs,” demonstrating superior robustness against heavy noise. PR1, employing naive resampling, already outperforms the baseline Wave-U-Net, highlighting the crucial role of effective frequency bands in denoising.

PR2, which employs sinc interpolation-based resampling, outperforms the other two models. Its consistent gains reflect the effectiveness of an anti-aliasing filter in mitigating the aliasing effects that limit PR1, thus confirming that principled anti-aliasing enhances time-domain feature fidelity and reduces spectral distortion.

Figure 4 provides a qualitative comparison of denoising performance for 0-dB babble noise, presenting log-magnitude spectrograms. The figure illustrates the noisy input (a), clean target (b), conventional Wave-U-Net output (c), and proposed PR2 output (d). A close inspection reveals the superiority of our approach in two key areas.

First, the harmonic structure appears as distinct horizontal lines in the clean target (b). The proposed PR2 output (d) preserves this structure with significant clarity, closely resembling the clean target. Conversely, the baseline output (c) exhibits considerable spectral distortion in the form of smearing artifacts between these harmonics, indicating a loss of spectral fidelity. The proposed PR2 output significantly suppresses this distortion, presenting a cleaner representation.

This visual evidence demonstrates that the proposed principled anti-aliasing approach effectively prevents spectral folding and distortion. This enhanced spectral integrity directly correlates with higher perceived quality (improved PESQ scores) and clearer speech representation (improved STOI scores).

Furthermore, a comparative analysis of the time-domain plots (Figure 4, left column) supports this finding. Both the Baseline Wave-U-Net (c) and PR2 (d) outputs effectively suppress gross noise; however, the Baseline (c) retains structural distortion, which forms residual artifacts. In contrast, the PR2 module demonstrates reduced distortion of the original speech components. This is evident as the PR2 output (d) more closely tracks the overall shape of the clean target’s amplitude envelope (b) compared to the Baseline Wave-U-Net (c), thus validating our design’s focus on signal integrity.

5.1.2. Effect of Multi-Band Deep Supervision

Having established the importance of principled resampling with PR2, we evaluate the additional impact of the PR3 multi-band deep supervision strategy. Guided by SII-based band importance [28], the auxiliary loss is computed over four perceptually salient bands centered at 4.00, 0.75, 0.50, and 0.25 kHz. Concretely, we supervise the decoder outputs whose effective passbands most closely cover these centers; under the current schedule in Table 1, this corresponds to levels 4, 8, 9, and 10, respectively. The band weights are set to ${\lambda }_j$ = 1.0 for these taps and 0 otherwise, and we set the mixing coefficient in Equation (6) to $\alpha =0.2$.

Table 4. Absolute STOI scores (maximum 1.0) for PR2 and PR3 across six noise types. Values denote the score averaged over Low SNR conditions (–10–0 dB), with the score for High SNR conditions (5–15 dB) presented in parentheses.

Methods	Noise Conditions
	Babble	F16	Factory	Pink	Volvo	White
PR2	0.720 (0.960)	0.761 (0.964)	0.734 (0.955)	0.752 (0.957)	0.977 (0.994)	0.789 (0.955)
PR3	0.719 (0.962)	0.767 (0.966)	0.730 (0.958)	0.752 (0.959)	0.977 (0.994)	0.796 (0.956)

and

Table 5. Absolute PESQ scores (maximum 4.5) for PR2 and PR3 across six noise types. Values denote the score averaged over Low SNR conditions (–10–0 dB), with the score for High SNR conditions (5–15 dB) presented in parentheses.

Methods	Noise Conditions
	Babble	F16	Factory	Pink	Volvo	White
PR2	1.301 (2.642)	1.444 (2.718)	1.327 (2.529)	1.382 (2.604)	3.257 (4.112)	1.405 (2.513)
PR3	1.335 (2.674)	1.477 (2.752)	1.351 (2.610)	1.418 (2.656)	3.231 (4.150)	1.443 (2.612)

compare the multi-band loss-based approach (PR3) with the single- loss approach (PR2), presenting absolute scores across low SNR conditions (−10–0 dB; main values) and high SNR conditions (5–15 dB; value in parentheses). A detailed analysis reveals that PR3 impacts STOI minimally, indicating that the multi-band loss did not improve intelligibility within our setup. Conversely, PR3 consistently improves PESQ across most conditions, suggesting that it primarily impacts perceived audio quality.

PR3 yields greater PESQ improvements, particularly under the high SNR conditions (values in parentheses) across several noise types, including factory, pink, and white. This pattern indicates that the multi-band loss functions as a fine-tuner, refining the model’s output, to suppress residual artifacts in perceptually critical frequency bands once the primary signal structure is restored.

An exception is the slight PESQ degradation for the stationary Volvo noise at the low SNR condition. Unlike broadband stationary noise like white noise, Volvo noise concentrates energy in persistent harmonic components within the low-to-mid frequency regions. At low SNR values, noise harmonics overlap with the frequency bands emphasized by the auxiliary loss, potentially causing the model to retain noise as speech and slightly degrade perceived quality. This effect is absent at high SNRs, where the speech dominates and minimizes such ambiguity.

5.2. Further Analysis and Robustness

To demonstrate that the frequency-aware framework represents a generalizable principle and not a solution confined to a specific architecture or dataset, we conducted three additional experiments evaluating its modularity, robustness, and comparative performance.

First, we validated modularity and potential synergy (

Table 6. Performance comparison of the combined effectiveness of the temporal enhancement module (Model A) and proposed frequency-aware module (Model B). All scores are averaged across all test conditions.

Model	Description	STOI	PESQ
Baseline	Original Wave-U-Net	0.882	2.25
Model A	Baseline + Attention Module (Temporal)	0.915	2.58
Model B	Baseline + Proposed Module (Frequency-Aware)	0.911	2.65
Model C	Baseline + Attention + Proposed (Combined)	0.926	2.77

Note: Metrics are mean absolute scores over all test conditions, as described in Section 4.1. The SNR set includes −10, −5, 0, 5, 10, and 15 dB. The adopted noise set is from NOISEX-92, which includes babble, f16, factory, pink, Volvo, and white noise, mixed with TIMIT test utterances.

). This test compares three models: Model A (Baseline + Attention), Model B (Baseline + our PR3 module), and Model C (Baseline + Attention + PR3). The findings indicate that Model A’s temporal enhancement and Model B’s frequency enhancement both surpass the original baseline, validating their effectiveness. Model B preserves speech intelligibility similar to the attention-enhanced baseline while achieving superior perceptual quality, underscoring the importance of spectral fidelity. Crucially, Model C, which integrates the proposed module with the attention-enhanced Wave-U-Net, outperforms Model A across both metrics. This reflects the complementary functions of the two modules: the frequency-aware module mitigates spectral distortions and the attention module subsequently captures temporal dependencies more accurately on this refined representation. This result highlights the core trade-off between computational investment and perceptual quality. For instance, under equal parameters, the proposed PR3 module (Model B) improves PESQ by +0.40 over the baseline (Table 6) at a fixed compute increase (Table 2). This quantifies a favorable trade-off, as the model remains real-time on our GPU (RTF = 0.0142) while achieving significant perceptual gains.

Second, after verifying architectural robustness, we investigated robustness against unseen acoustic environments. To evaluate generalization beyond NOISEX-92, we conducted a snapshot test on two non-stationary noise types from the Deep Noise Suppression Challenge dataset [33], without any retraining. The absolute scores decline under these more challenging noises (

Table 7. Robustness evaluation on unseen non-stationary noise conditions, demonstrating that the proposed method (Model B) generalizes effectively beyond the training dataset.

Noise Type	Model	STOI	PESQ
Typing Noise	Baseline	0.78	1.85
	Model B	0.82	2.01
Siren Noise	Baseline	0.75	1.79
	Model B	0.79	1.95

Note: Noise types are non-stationary “Typing” and “Siren” clips from a DNS-style corpus, utilized as a held-out snapshot test without any retraining. Reported numbers are mean absolute scores at the same SNR set used in Table 6.

); however, Model B consistently outperforms the original baseline. These results indicate that the frequency-aware optimization leverages a generalizable denoising principle, resisting overfitting to the training noises and maintaining robustness in realistic acoustic scenarios.

Third, for an exhaustive comparison with same-type methods, we expanded our evaluation to include two representative waveform-based models: CleanUNet [9] and Conv-TasNet [18]. We trained and tested these models under the same conditions as our own.

Table 8. Comparative analysis of ∆STOI and ∆PESQ (improvement scores) against same-type waveform models on NOISEX-92 test.

Model	∆STOI (Avg.)	∆PESQ (Avg.)
Baseline (Wave-U-Net)	0.116	0.78
PR1	0.125	0.90
PR2	0.128	0.93
PR3	0.129	0.97
CleanUNet	0.120	0.92
Conv-TasNet	0.113	0.73

summarizes the performance measured by the improvement in STOI (∆STOI) and PESQ (∆PESQ) over the noisy input.

Table 8 reveals two key findings. First, the internal ablation (Baseline vs. PR3) confirms our module’s innovation. By adding only our frequency-aware module and multi-band supervision, the ∆PESQ improves significantly from 0.78 to 0.97, while maintaining the same parameter count as the baseline Wave-U-Net.

Second, the external comparison indicates that our final model (PR3) performs favorably against both representative “same-type” models. However, this comparison must be understood in the context of the models’ design objectives, particularly causality. The proposed framework, built on the non-causal Wave-U-Net [6], is optimized for high-fidelity offline processing, enabling it to maximize the entire signal context. In contrast, both CleanUNet and Conv-TasNet are causal models, specifically designed for low-latency, real-time streaming operations. This constrains them to utilizing only past information. Therefore, our non-causal model’s superior performance (0.97 ∆PESQ) over CleanUNet (0.92) and Conv-TasNet (0.73) is expected, as it leverages the full signal context. This comparison validates that our modular, non-causal approach is highly effective for high-fidelity tasks, and it clearly demonstrates that this performance gain stems from our novel frequency-aware design.

Finally, linking innovation to design decisions, the observed gains in both Table 6 and Table 8 align with the design in Table 1. By explicitly scheduling internal sampling rates and enforcing pre-decimation low-pass and post-interpolation anti-imaging filters, the PR2 and PR3 models preserve mid and high-frequency cues that are otherwise lost or aliased under standard strided sampling. Subsequently, multi-band supervision in PR3 targets residual artifacts in perceptually weighted bands, elucidating the additional PESQ gains (Table 4 and Table 5) without increasing parameters (Table 2).

6. Conclusions

This study addressed the spectral information loss inherent in waveform-based speech enhancement caused by standard downsampling methods. Our main contribution is a validated, modular approach for spectral optimization, which integrates a principled resampling module with a multi-band deep supervision strategy.

Evaluations on the standard Wave-U-Net and its attention-enhanced version demonstrate the proposed framework’s key advantages. First, the frequency-aware approach achieves performance comparable to models that exclusively target temporal patterns by reducing signal distortion directly. Second, it functions as a versatile, complementary module that enhances performance when integrated with attention-based models. Tests on novel noise types verify robust generalization, achieved with a modest and practical computational overhead, highlighting a favorable performance–efficiency trade-off.

These findings establish frequency-aware design as a fundamental component for future E2E models alongside temporal modeling. In addition, its modularity allows integration with advanced architectures. A crucial next step is adapting the resampling module to a causal configuration for real-time applications like online communication and hearing aids.