Parallel Enhancement and Bandwidth Extension of Coded Speech

Abstract

An important use case of speech bandwidth extension (BWE) is generating high-frequency components from band-limited speech processed by a speech codec. Recent works on BWE have demonstrated remarkable capabilities in generating high-quality, high-band components using deep learning techniques. Among them, Streaming SEANet (StrmSEANet) has also been shown to be effective for BWE with reduced delay and computational complexity, making it suitable for real-time speech processing. However, the effect of the coding artifact in the lower band of the input signal has not been sufficiently considered in many deep learning-based BWE methods. In this work, we propose Parallel Enhancement and Bandwidth Extension of coded speech (PEBE), where two lightweight networks, referred to as Compact Streaming SEANet (CompSEANet), for coded speech enhancement (CSE) and BWE are configured in parallel. The CSE and BWE models are separately trained with the task-specific training settings, thereby effectively improving the reconstruction quality of the band-limited speech signals degraded by coding artifacts. Experimental results demonstrate that the proposed PEBE significantly outperforms the baseline AP-BWE, StrmSEANet, and standalone CompSEANet in reconstructing wideband (WB) and fullband speech from Opus-coded narrowband and WB signals. The proposed method achieves the highest scores in the subjective MUSHRA test while providing the fastest inference among all compared methods, with real-time factors (RTF) of 33.95× and 18.38× measured on a Samsung SM-F711 mobile device under single-thread execution.

1. Introduction

In voice communication systems, speech signals are degraded by bandwidth (BW) limitations introduced by speech codecs such as AMR-NB [1], AMR-WB [2], Opus [3], and EVS [4]. BW limitation is primarily due to standards for speech communication rather than the codecs themselves. Recently proposed neural network–based speech bandwidth extension (BWE) models [5,6,7,8] have shown remarkable capability in reconstructing high-frequency components of band-limited speech signals.

In ref. [5], an autoencoder convolutional neural network model [9] was adopted for BWE, trained with a time-frequency loss function to jointly optimize the model in both domains. To facilitate on-device deployment in communication systems, Streaming SEANet (StrmSEANet) [7] was proposed based on the original Sound EnhAncement Network (SEANet) [10], utilizing the causal convolutions and reducing the dimensions of the internal feature maps for real-time processing with low latency and complexity. In ref. [6], a BWE model operating on the complex spectrogram from the short-time Fourier transform (STFT) was proposed to better handle phase information. AP-BWE [8] adopted a ConvNeXt-based backbone [11] and introduced a dual-stream structure to separately process the amplitude and phase spectra of an input signal, effectively extending the frequency bandwidth and improving the speech quality. To further enhance the perceptual quality of the extended speech, a multi-resolution amplitude discriminator (MRAD) and a multi-resolution phase discriminator (MRPD) [8] were introduced, inspired by the multi-resolution spectrogram discriminator [12]. Although these approaches demonstrate strong reconstruction performance, their BWE performance on coded speech signals has not been extensively evaluated.

Several studies have aimed to address the restoration of high-frequency components while considering coding artifacts [13,14]. In ref. [13], a cascaded architecture of coded speech enhancement (CSE) and BWE modules was proposed, trained via supervision with a time-domain L2 loss and frequency-domain L2 losses on multi-resolution log spectral amplitude and log mel-spectrograms. A side-information-based approach to restoring original wideband (WB) speech from coded versions using a narrowband (NB) codec named Codec 2 [15] was proposed in [14]. Recently, numerous approaches [16,17,18,19,20,21] have been proposed as a result of the URGENT 2025 Challenge [22] to solve the universal speech enhancement problem, where input speech signals with diverse sampling rates are corrupted not only by bandwidth limitation and coding artifacts but also by additive noise, reverberation, clipping, packet loss, and wind noise.

Another approach to restoring high-band components, distinct from conventional BWE, was proposed in the name of Alias-and-Separate [23], where intentional aliasing is utilized to hide the high-frequency components of a wideband (WB) speech signal in the lower band region when the signal is downsampled to a narrowband (NB) one. After compression with an NB speech codec, the original signal can be reconstructed by separating the original lower band and aliased high band components. An improved version of the Alias-and-Separate was proposed in [24] to reduce algorithmic delay and enhance the quality of the reconstructed speech, as well as enable fullband (FB) speech coding tasks with a WB speech codec.

On the other hand, various approaches to coded speech enhancement (CSE) have been investigated [25,26,27,28,29] in recent years. Several studies have focused on exploiting codec-derived features such as LPC coefficients, pitch lags, and quantization gains, or side information transmitted from the encoder to improve the quality of the reconstructed signal [25,26,27]. In parallel, the authors of [28,29] have proposed approaches based on generative adversarial networks (GAN) [30] to effectively enhance the quality of coded speech.

In this work, we first propose a Compact Streaming SEANet (CompSEANet) to increase the computational efficiency of post-processing coded speech while minimizing performance degradation. We further propose a Parallel Enhancement and Bandwidth Extension of coded speech (PEBE) to effectively enhance coded speech signals and extend their spectral bandwidth. In the PEBE, two separate models for CSE and BWE individually produce the short-time Fourier transform (STFT) coefficients of the outputs from a band-limited input. The final output waveform is then reconstructed via an inverse STFT (iSTFT) of the combined outputs in the lower and upper bands.

Implemented with CompSEANet, the proposed PEBE demonstrates superior performance compared to the baseline StrmSEANet [7] and AP-BWE [8], achieving the highest subjective speech quality in the MUSHRA test and the fastest inference speed among all compared models in two tasks: reconstruction of wideband (WB) speech from coded narrowband (NB) speech (NB-to-WB) and reconstruction of fullband (FB) speech from coded wideband (WB) speech (WB-to-FB).

The remainder of this paper is organized as follows: Section 2 describes the baseline StrmSEANet architecture and training procedure. Section 3 presents the proposed CompSEANet and PEBE frameworks. Section 4 reports the experimental results. Finally, Section 5 concludes the paper and discusses future work.

2. Streaming SEANet

StrmSEANet [7] was proposed for real-time speech BWE, featuring a lighter network structure and lower algorithmic delay than the original version [10], while maintaining competitive reconstruction performance in terms of SI-SDR. As depicted in Figure 1, the coded input signal is upsampled with an upsampling ratio R (defined as the ratio between the target sampling rate $R{F}_s$ and the input sampling rate $F_s$) and then processed by an initial 1D convolutional layer with the kernel size of seven, four encoder blocks, four decoder blocks [10] with residual skip connections between encoder and decoder feature maps of identical time scale, and a final 1D convolutional layer. Although the original StrmSEANet [7] does not explicitly describe the upsampling procedure of the input signals, the input coded signal should be upsampled to the same sampling rate as the target speech waveform. Therefore, we explicitly include the upsampling process in our implementation of the StrmSEANet.

Figure 1. Architectures of StrmSEANet (left) and CompSEANet (right).

The encoder blocks consist of a strided convolution and three residual units [10], where dilated convolutions with dilation factors of 1, 3, and 9 are sequentially applied to enlarge the receptive field without increasing model size. Each encoder block reduces temporal resolution according to the stride factor while doubling the channel dimension. Conversely, each decoder block mirrors this process using a transposed convolution for upsampling followed by the same residual units, progressively restoring temporal resolution and reducing channels. All convolutions in the StrmSEANet [7] are causal, and no normalization across the time axis is employed in the generator. In refs. [7,10], the strides $(S_1,S_2,S_3,S_4)$ were set to (2, 2, 8, 8). The StrmSEANet is trained following the GAN-based training procedure [30] with multi-scale discriminators (MSD) [7], originally introduced in [31].

The loss function for adversarial training of the StrmSEANet is composed of a weighted sum of the hinge loss ${\mathcal{L}}_{adv}^{hg}$[10] and feature-matching loss ${\mathcal{L}}_{fm}$ [10,31] as follows:

$$\begin{array}{c}\hfill {\mathcal{L}}_G={\displaystyle \frac{1}{M}}\sum _{m=1}^M[{\mathcal{L}}_{adv}^{hg}(\widehat{x};D^m)+\lambda {\mathcal{L}}_{fm}(\widehat{x},x;D^m)],\end{array}$$

(1)

$$\begin{array}{c}\hfill {\mathcal{L}}_{adv}^{hg}(\widehat{x};D^m)={\displaystyle \frac{1}{T^m}}\sum _{\tau =1}^{T^m}max(0,1-D_{\tau }^m\left(\widehat{x}\right)),\end{array}$$

(2)

$$\begin{array}{c}\hfill {\mathcal{L}}_{fm}(\widehat{x},x;D^m)={\displaystyle \frac{1}{L^m}}\sum _l^{L^m}{\displaystyle \frac{1}{T_l^m}}\sum _{\tau =1}^{T_l^m}|D_{l,\tau }^m\left(\widehat{x}\right)-D_{l,\tau }^m\left(x\right)|,\end{array}$$

(3)

where x and $\widehat{x}$ are the real and generated waveforms, M is the number of discriminators, $D^m$ is the m-th discriminator, $D_{\tau }^m(·)$ is the $\tau$-th element in the output of $D^m$ among $T^m$ elements in total, $L^m$ is the number of layers in $D^m$, and $D_{l,\tau }^m(·)$ is the $\tau$-th element in the l-th internal feature map of $D^m$ among $T_l^m$ elements in total. In refs. [7,10], $\lambda$ was set to 100. Each discriminator $D^m$ is trained with a corresponding loss function to ${\mathcal{L}}_{adv}^{hg}$ defined as

$$\begin{array}{c}\hfill {\mathcal{L}}_{D^m}^{hg}(\widehat{x},x)={\displaystyle \frac{1}{T^m}}\sum _{\tau =1}^{T^m}\left[max(0,1+D_{\tau }^m\left(\widehat{x}\right))+max(0,1-D_{\tau }^m\left(x\right)\right)].\end{array}$$

(4)

3. Proposed Methods

3.1. Compact Streaming SEANet

Firstly, we design a lightweight post-processing model based on the StrmSEANet to improve computational efficiency while minimizing performance degradation. Inspired by iSTFTNet [32], we propose CompSEANet, a compact version of the StrmSEANet where the first two encoder blocks and the last two decoder blocks are replaced with STFT and iSTFT operations, respectively. A detailed illustration of the CompSEANet architecture is presented in Figure 1. We set the DFT sizes for the STFT and iSTFT to match their respective window sizes. The real and imaginary components in each frame are treated as separate channels and concatenated along the frequency axis to be processed by the first convolutional layer. The last convolutional layer outputs the real and imaginary components for the output waveform, concatenated along the frequency axis. Similarly to the StrmSEANet, residual skip connections are employed between feature maps to facilitate effective information flow across blocks. The STFT coefficients of the input are added to the corresponding lower-band part of the final output. Finally, the output waveform is reconstructed by the iSTFT with window and hop sizes R times larger than those of the STFT, where R corresponds to the upsampling ratio.

We used a loss function for adversarial training composed of the sum of the adversarial loss ${\mathcal{L}}_{adv}$, feature-matching loss ${\mathcal{L}}_{fm}$, and waveform reconstruction loss ${\mathcal{L}}_{rec}$ as follows:

$$\begin{array}{cc}\hfill {\mathcal{L}}_G=& {\displaystyle \frac{1}{M}}\sum _{m=1}^M[{\alpha }^m{\mathcal{L}}_{adv}(\widehat{x},x;D^m)+{\lambda }^m{\mathcal{L}}_{fm}(\widehat{x},x;D^m)]\hfill \\ & +{\mathcal{L}}_{rec}(\widehat{x},x),\hfill \end{array}$$

(5)

where x and $\widehat{x}$ are the real and generated waveforms. In addition to the hinge loss [10], we also consider the least-squares GAN (LSGAN) loss function [33] for the adversarial loss ${\mathcal{L}}_{adv}$, as used in [34], to find a more suitable loss function for coded speech post-processing,

$$\begin{array}{c}\hfill {\mathcal{L}}_{adv}^{ls}={\displaystyle \frac{1}{M}}\sum _{m=1}^M{\displaystyle \frac{1}{T^m}}\sum _{\tau =1}^{T^m}{(D_{\tau }^m\left(\widehat{x}\right)-1)}^2.\end{array}$$

(6)

The discriminators are trained with

$$\begin{array}{c}\hfill {\mathcal{L}}_{D^m}^{ls}(\widehat{x},x)={\displaystyle \frac{1}{T^m}}\sum _{\tau =1}^{T^m}[{\left(D_{\tau }^m\left(\widehat{x}\right)\right)}^2+{(D_{\tau }^m\left(x\right)-1)}^2].\end{array}$$

(7)

As utilized and shown to be effective for BWE in [6], we employ the multi-resolution STFT loss function [35], defined as

$$\begin{array}{c}\hfill {\mathcal{L}}_{rec}={\displaystyle \frac{1}{P}}\sum _{p=1}^P{\mathcal{L}}_{sc}(\widehat{x},x;{\theta }_p)+{\mathcal{L}}_{mag}(\widehat{x},x;{\theta }_p),\end{array}$$

(8)

$$\begin{array}{c}\hfill {\mathcal{L}}_{sc}(\widehat{x},x;{\theta }_p)={\displaystyle \frac{\left|\right||{\widehat{X}}_p|-|X_p{\left|\right||}_F}{\left|\right||X_p{\left|\right||}_F}},\end{array}$$

(9)

$$\begin{array}{c}\hfill {\mathcal{L}}_{mag}(\widehat{x},x;{\theta }_p)={\displaystyle \frac{1}{T_p{F}_p}}\left|\right|log\left(\right|{\widehat{X}}_p\left|\right)-log\left(\right|X_p{\left|\right)\left|\right|}_{1,1},\end{array}$$

(10)

where P is the number of STFT configurations ${\left\{{\theta }_p\right\}}_{p=1}^P$, ${\widehat{X}}_p$ and $X_p$ are the STFT coefficients of $\widehat{x}$ and x transformed with ${\theta }_p$, and $T_p$ and $F_p$ are the number of time steps and frequency bins, respectively. The operator $|·|$ denotes the magnitude of a complex-valued STFT coefficient, and ${\parallel ·\parallel }_F$ and ${\parallel ·\parallel }_{1,1}$ are the Frobenius norm and matrix norm, respectively. $log(·)$ denotes the natural logarithm applied to each element.

We present the results of our preliminary experiment for the StrmSEANet and CompSEANet in Table 1. Both models are trained with the loss function in Equation (5), as we confirmed that utilizing the waveform reconstruction loss improved speech quality. The strides for the StrmSEANet were set to (2, 4, 5, 8) for NB-to-WB and (4, 5, 6, 8) for WB-to-FB tasks. For CompSEANet, strides were set to (5, 8) for both tasks. We set ${\alpha }^m=1$ and ${\lambda }^m=100$ as defined in Equation (5) for this experiment.

Table 1. Preliminary experimental results for StrmSEANet and CompSEANet trained with MSD, ${\alpha }^m=1$, and ${\lambda }^m=100$. PESQ and ViSQOL-A scores are measured for NB-to-WB and WB-to-FB tasks, respectively. The best performance for each task is highlighted in bold.

Task	Model	RTF	Adversarial Loss	Score
NB-to-WB	StrmSEANet ($C=8$)	0.0384 (26.02×)	Hinge	2.71
	StrmSEANet ($C=8$)	0.0384 (26.02×)	LSGAN	2.88
	CompSEANet ($C=8$)	0.0114 (87.77×)	Hinge	2.51
	CompSEANet ($C=8$)	0.0114 (87.77×)	LSGAN	2.72
WB-to-FB	StrmSEANet ($C=12$)	0.1163 (8.60×)	Hinge	3.11
	StrmSEANet ($C=12$)	0.1163 (8.60×)	LSGAN	3.12
	CompSEANet ($C=12$)	0.0242 (41.41×)	Hinge	2.92
	CompSEANet ($C=12$)	0.0242 (41.41×)	LSGAN	3.16

All results indicate that using the LSGAN loss was beneficial for the reconstruction quality. Even though CompSEANet for the NB-to-WB task showed slightly degraded performance compared to StrmSEANet in terms of Perceptual Evaluation of Speech Quality (PESQ) [36], it dramatically reduced inference time by 70%. For the WB-to-FB task, CompSEANet achieved the best performance in terms of Virtual Speech Quality Objective Listener in audio mode (ViSQOL-A) [37,38], even with 80% reduced real-time factors (RTFs) compared to the StrmSEANet.

We also present experimental results in Table 2, where discriminators and weights (${\alpha }^m,{\lambda }^m$) were varied, along with C for CompSEANet in Figure 1. The use of MPD, MRAD, and MRPD [8] alongside MSD improved the PESQ score for both models compared to using MSD alone. Although the PESQ for CompSEANet with $C=8$ is not yet close to the StrmSEANet, the gap is reduced. When the RTF of the StrmSEANet approaches that of the CompSEANet with $C=16$, the PESQ for the CompSEANet exceeds that of the StrmSEANet.

Table 2. Second preliminary experimental results comparing PESQ scores for StrmSEANet and CompSEANet in the NB-to-WB task by varying discriminator combinations and loss weights. All models were trained with LSGAN loss. The best performance for each task is highlighted in bold.

Model	RTF	Discriminators	$({\mathit{\alpha }}^{\mathit{m}},{\mathit{\lambda }}^{\mathit{m}})$	PESQ
StrmSEANet ($C=8$)	0.0384 (26.02×)	MSD	(1, 100)	2.88
		MSD, MPD, MRAD, MRPD	(1, 100) for MSD and MPD (0.1, 0.1) for MRAD and MRPD	2.83
			(1, 1) for MSD and MPD (0.1, 0.1) for MRAD and MRPD	2.93
CompSEANet ($C=8$)	0.0114 (87.77×)	MSD	(1, 100)	2.72
		MSD, MPD, MRAD, MRPD	(1, 100) for MSD and MPD (0.1, 0.1) for MRAD and MRPD	2.73
			(1, 1) for MSD and MPD (0.1, 0.1) for MRAD and MRPD	2.87
CompSEANet ($C=16$)	0.0393 (25.46×)	MSD, MPD, MRAD, MRPD	(1, 1) for MSD and MPD (0.1, 0.1) for MRAD and MRPD	3.04

3.2. Parallel Enhancement and Bandwidth Extension of Coded Speech

Based on the preliminary experiments presented in sec:compseanet, we also propose PEBE to effectively enhance an input speech signal band-limited and coded during the coding procedure and extend the spectral bandwidth of the input signal by adopting separate parallelized models for CSE and BWE, each of which is optimized to the corresponding task. Compared to a cascaded structure of the CSE and BWE models, the parallel structure is beneficial for the total latency and design flexibility of each model. The total latency becomes the sum of the latencies of the CSE and BWE when they are cascaded, while the latency becomes the larger of the two models when parallelized. The modularity between the CSE and BWE models facilitates finding the optimal training settings or architectures for the models and improves the computational scalability, as the complexity of the CSE and BWE models can be adjusted separately according to the computational resources.

Figure 2 describes the processing procedure of the PEBE implemented with CompSEANet for CSE and BWE. The input speech waveform $x_{in}\in {\mathbb{R}}^{N_{in}}$ is transformed into STFT coefficients $X_{in}\in {\mathbb{C}}^{F_{in}\times T}$, where $N_{in}$ is the number of samples and $F_{in}$ and T are the numbers of frequency bins and frames, respectively. The CSE and BWE models process $X_{in}$ and output $X_{CSE}\in {\mathbb{C}}^{F_{CSE}\times T}$ and $X_{BWE}\in {\mathbb{C}}^{F_{BWE}\times T}$, respectively. $X_{CSE}$ is added to the corresponding part of $X_{in}$, and the output spectra $X_{out}\in {\mathbb{C}}^{F_{out}\times T}$, where $F_{out}=F_{CSE}+F_{BWE}$, are composed as follows:

$$\begin{array}{c}\hfill X_{out}(f,t)=\left\{\begin{array}{cc}{X}_{CSE}(f,t)+X_{in}(f,t)\hfill & 0\le f<F_{CSE}\hfill \\ X_{BWE}(f-F_{CSE},t)\hfill & F_{CSE}\le f<F_{out}\hfill \end{array}\right..\end{array}$$

(11)

Finally, the iSTFT is applied to $X_{out}$ to reconstruct the final output waveform $x_{out}$.

Figure 2. Architecture of PEBE adopting CompSEANet for CSE and BWE models.

Since PEBE utilizes CSE and BWE models in parallel, they can be trained separately with schemes suitable for each task. In this work, we trained the CSE and BWE models with individual loss functions and discriminator sets by applying iSTFT to each of the outputs of the CSE and BWE models, as given in Figure 3. To fill empty bands, the lower band for $X_{BWE}$ and the high band for $X_{CSE}$, corresponding components of the clean signal X, are utilized. The waveforms to train the CSE and BWE models, ${\tilde{x}}_{CSE}$ and ${\tilde{x}}_{BWE}$, are obtained by applying iSTFT to their corresponding STFT coefficients ${\tilde{X}}_{CSE}$ and ${\tilde{X}}_{BWE}$, defined as:

$$\begin{array}{c}\hfill {\tilde{X}}_{CSE}(f,t)=\left\{\begin{array}{cc}{X}_{CSE}(f,t)\hfill & 0\le f<F_{CSE}\hfill \\ X(f-F_{CSE},t)\hfill & F_{CSE}\le f<F_{out}\hfill \end{array}\right.,\end{array}$$

(12)

$$\begin{array}{c}\hfill {\tilde{X}}_{BWE}(f,t)=\left\{\begin{array}{cc}X(f,t)\hfill & 0\le f<F_{CSE}\hfill \\ X_{BWE}(f-F_{CSE},t)\hfill & F_{CSE}\le f<F_{out}\hfill \end{array}\right..\end{array}$$

(13)

With respective discriminator sets for CSE and BWE, ${\left\{D_{CSE}^m\right\}}_{m=1}^{M_{CSE}}$ and ${\left\{D_{BWE}^m\right\}}_{m=1}^{M_{BWE}}$, we define the respective loss functions as

$$\begin{array}{cc}\hfill {\mathcal{L}}_{CSE}& ={\displaystyle \frac{1}{M_{CSE}}}\sum _{m=1}^{M_{CSE}}[{\alpha }_{CSE}^m{\mathcal{L}}_{adv}(x,{\tilde{x}}_{CSE};D_{CSE}^m)+{\lambda }_{CSE}^m{\mathcal{L}}_{fm}(x,{\tilde{x}}_{CSE};D_{CSE}^m)]\hfill \\ & +{\eta }_{CSE}{\mathcal{L}}_{rec}(x,{\tilde{x}}_{CSE}),\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{BWE}& ={\displaystyle \frac{1}{M_{BWE}}}{\displaystyle \sum _{m=1}^{M_{BWE}}}[{\alpha }_{BWE}^m{\mathcal{L}}_{adv}(x,{\tilde{x}}_{BWE};D_{BWE}^m)+{\lambda }_{BWE}^m{\mathcal{L}}_{fm}(x,{\tilde{x}}_{BWE};D_{BWE}^m)]\hfill \\ & +{\eta }_{BWE}{\mathcal{L}}_{rec}(x,{\tilde{x}}_{BWE}).\hfill \end{array}$$

(15)

Figure 3. Block diagram describing the computation of loss functions for CSE and BWE models.

4. Experiments

4.1. Experimental Settings

We used the clean speech signals from the Valentini speech dataset [39] which is a subset of the VCTK corpus [40]. The dataset includes FB speech waveforms sampled at 48 kHz from 84 English speakers for training and from 2 speakers for evaluation. For the NB-to-WB task, the FB speech waveforms were resampled to 8 kHz and coded using the Opus NB codec operating at 8 kbps in constant bitrate (CBR) mode. The corresponding target WB signals with a sampling rate of 16 kHz were obtained by resampling the original FB speech waveforms. For the WB-to-FB task, the resampled WB signals were coded using the Opus WB at 10 kbps in CBR mode. The frame sizes for both codecs were set to 20 ms.

For the StrmSEANet, strides $(S_1,S_2,S_3,S_4)$ were set to (2, 4, 5, 8) for the NB-to-WB task and (4, 5, 6, 8) for the WB-to-FB task. Similarly, strides $(S_3,S_4)$ for the CompSEANet were set to (5, 8). STFT window and hop sizes for CompSEANet were 8 and 4 for NB-to-WB, and 16 and 8 for WB-to-FB. Accordingly, iSTFT window and hop sizes were 16 and 8 for the NB-to-WB task, and 48 and 24 for the WB-to-FB task. With these configurations, both StrmSEANet and CompSEANet introduce the algorithmic delay of 20 ms for both tasks.

We set the dimension of the first feature map (C) in the StrmSEANet to 8 and 12 for NB-to-WB and WB-to-FB tasks, respectively. For the CompSEANet, C was set to 16 and 24. For the CompSEANets in the PEBE framework, C was set to 8 and 12. $F_{CSE}$ and $F_{BWE}$ were 4 and 5 for the NB-to-WB task, and 6 and 19 for the WB-to-FB task.

We used the Adam optimizer [41] with ${\beta }_1=0.5,{\beta }_2=0.9$, and a learning rate of 0.0001. For the PEBE, BWE models were trained with the MSD [7], MPD [34], and MRAD and MRPD in [8]. Only the first discriminator without downsampling in the MSD was adopted for BWE. CSE models were trained with MSD and MPD. For the StrmSEANet and CompSEANet, we set ${\alpha }^m=1$ and ${\lambda }^m=1$ for the MSD and MPD, and ${\alpha }^m=0.1$ and ${\lambda }^m=0.1$ for the MRAD and MRPD. For the PEBE, we set ${\alpha }_{CSE}^m=1$, ${\lambda }_{CSE}^m=2$, ${\eta }_{CSE}=45$, and ${\eta }_{BWE}=1$, while ${\alpha }_{BWE}^m$ and ${\lambda }_{BWE}^m$ were set to 1 for the MSD and MPD and 0.1 for the MRAD and MRPD. We used the LSGAN loss [33] based on preliminary results.

In this work, we have adapted the AP-BWE source code from its open-source repository https://github.com/yxlu-0102/AP-BWE (accessed on 25 January 2026), modifying the internal convolutional layers in the ConvNeXt blocks [11] to be causal and replacing layer normalization (LN) [42] with cumulative layer normalization [43] for real-time operation. We set the ConvNeXt feature map dimension to 256. The window and hop sizes were 320 and 80 for NB-to-WB and 960 and 240 for WB-to-FB, respectively, with a DFT size of 1024 for both, which leads to 15 ms algorithmic delay for both tasks.

4.2. Experimental Results

We present spectrograms of clean and reconstructed WB speech signals in Figure 4. AP-BWE and PEBE reconstructed a clearer harmonic structure in the voiced region (green box) compared to the StrmSEANet and CompSEANet. PEBE also suppressed more artifacts in the high-band region (blue box). These tendencies are more prominent in Figure 5 for the WB-to-FB task. While the AP-BWE, CompSEANet, and PEBE produced fewer artifacts in the lower band than the StrmSEANet, only PEBE generated high-band components without severe over-smoothing.

Figure 4. Spectrograms for (a) a clean WB speech and the reconstructed WB speech signals by (b) AP-BWE, (c) StrmSEANet, (d) CompSEANet, and (e) PEBE.

Figure 5. Spectrograms for (a) a clean FB speech and the reconstructed FB speech signals by (b) AP-BWE, (c) StrmSEANet, (d) CompSEANet, and (e) PEBE.

Table 3 presents objective evaluation results. PESQ and ViSQOL-S were used for NB-to-WB, and ViSQOL-A and NISQA [44] for WB-to-FB. RTFs were measured on a Samsung SM-F711 mobile device (Samsung, Suwon, Republic of Korea) with single-thread operations. For the NB-to-WB task on the Valentini test set, the PEBE achieved the highest PESQ and ViSQOL-S scores, outperforming the AP-BWE, StrmSEANet, and CompSEANet. Even when the model size of the CompSEANet was adjusted to match the RTF of the PEBE, the PEBE consistently exhibited higher scores, yielding the lowest RTF, which demonstrates superior reconstruction quality with high computational efficiency. On the PTDB-TUG dataset, PEBE achieved the highest ViSQOL-S score, while its PESQ score, although slightly lower than that of AP-BWE, remained comparable to the other models, indicating stable generalization performance. In the WB-to-FB task on the Valentini test set, the PEBE achieved the highest NISQA score of 4.38. Although it obtained a slightly lower ViSQOL-A score than other models, the difference may be marginal. Notably, the PEBE also recorded the lowest RTF, offering the fastest inference speed with superior or comparable objective quality scores. On the PTDB-TUG test set, PEBE achieved the best performance in terms of NISQA, while exhibiting comparable ViSQOL-A results to other models, with only marginal performance differences observed across models.

Table 3. PESQ, ViSQOL, and NISQA scores for the baseline and proposed models on the Valentini test set and whole excerpts from the PTDB-TUG dataset, along with their real-time factors (RTFs). The best performance for each task is highlighted in bold.

Task	Model	RTF	Valentini		PTDB-TUG
			PESQ	ViSQOL-S	PESQ	ViSQOL-S
NB-to-WB	AP-BWE	0.1045 (9.57×)	2.94	4.26	2.42	3.67
	StrmSEANet ($C=8$)	0.0384 (26.02×)	2.93	4.24	2.24	3.73
	CompSEANet ($C=16$)	0.0393 (25.46×)	3.04	4.26	2.38	3.68
	PEBE ($C=8$)	0.0295 (33.95×)	3.07	4.27	2.38	3.78
Task	Model	RTF	Valentini		PTDB-TUG
			NISQA	ViSQOL-A	NISQA	ViSQOL-A
WB-to-FB	AP-BWE	0.1160 (8.62×)	4.07	3.22	3.95	3.80
	StrmSEANet ($C=12$)	0.1163 (8.60×)	4.18	3.22	4.05	3.76
	CompSEANet ($C=24$)	0.0775 (12.90×)	4.35	3.24	4.08	3.78
	PEBE ($C=12$)	0.0544 (18.38×)	4.38	3.20	4.16	3.77

We present MUSHRA [45] test results for both tasks in Figure 6. 11 experienced listeners evaluated the models. In both sessions, the PEBE exhibited significant improvement in speech quality. CompSEANet, StrmSEANet, and AP-BWE showed no significant difference in the NB-to-WB task, as shown in Figure 6a. However, for the WB-to-FB task, a significant difference was observed between AP-BWE and the other models as presented in Figure 6b, while the CompSEANet and StrmSEANet performed comparably. Despite superior objective scores of the CompSEANet in Table 3, it showed slightly lower average MUSHRA scores than StrmSEANet, possibly due to weak artifacts near 6 kHz.

Figure 6. MUSHRA test results for (a) NB-to-WB and (b) WB-to-FB tasks.

5. Conclusions

In this work, we proposed a Parallel Enhancement and Bandwidth Extension (PEBE) framework to address the post-processing of coded speech. We also introduced Compact Streaming SEANet (CompSEANet) to improve computational efficiency, utilized within PEBE for both coded speech enhancement and bandwidth extension. The proposed PEBE, implemented with the CompSEANet, outperformed the baseline StrmSEANet and AP-BWE models in two speech reconstruction tasks, NB-to-WB and WB-to-FB, while achieving faster inference speeds. While the current PEBE adopts independently trained CSE and BWE models, the two models may learn complementary features that are not explicitly shared in the current architecture. Future work will investigate feature-level interactions between the CSE and BWE models within the PEBE framework, while maintaining the advantages of the parallel architecture.