Efficient Speech Enhancement via Flow Matching with Gated Bidirectional Mamba2

Abstract

Speech enhancement (SE) aims to recover clean speech from noisy speech while preserving intelligibility, speaker identity, and runtime efficiency. Existing language-model (LM)-based methods may lose fine acoustic details due to discretization, whereas diffusion models often require many iterative denoising steps. This study proposes an efficient speech enhancement framework based on flow matching and a gated bidirectional Mamba2 backbone. The model predicts a continuous velocity field in the Mel-spectrogram domain and introduces a DiMamba block that captures past and future context through shared-weight bidirectional state-space modeling with adaptive gating. Experiments on the DNS Challenge test set and additional VoiceBank test data show that the proposed method achieves strong perceptual quality and speaker preservation while substantially reducing inference cost. The model reaches a real-time factor of 0.31, more than five times faster than diffusion baselines, and achieves a word error rate of 4.7% and a quality mean opinion score of 3.58. These results indicate that flow matching combined with gated bidirectional Mamba2 provides an effective quality–efficiency trade-off for offline speech enhancement.

1. Introduction

Speech enhancement (SE) has always been a fundamental task in signal processing. Whether in the field of speech generation or multi-modal processing, SE is widely required as a front-end processing step that recovers clean speech from noisy recordings, and it is applied in a range of practical scenarios such as podcast and broadcast post-production, archival recording restoration, and ASR pre-processing for non-real-time transcription, where high perceptual quality is more important than strict low-latency operation. Although conventional enhancement methods can effectively suppress additive noise under relatively simple acoustic conditions, they often struggle to maintain the naturalness, intelligibility, and speaker fidelity of enhanced speech when the input is heavily corrupted by reverberation, environmental noise, or bandwidth degradation.

Early SE systems mainly rely on regression or masking strategies that learn a direct mapping from noisy to clean speech or estimate a time–frequency mask to suppress noise components [1,2,3]. While computationally efficient, these discriminative approaches often struggle under complex acoustic interference, where they may over-suppress useful speech components or introduce perceptual artifacts. Generative modeling has therefore emerged as a promising alternative for high-fidelity speech enhancement, following two main paradigms. The first applies language models (LMs) to discrete speech tokens [4,5,6], while more recent extensions such as AnyEnhance [7] and LLaSE-G1 [8] further explore unified multi-task formulations and LLaMA-based generalization. Despite their strong sequence modeling capability, the reliance on discretization tends to lose fine acoustic details that are critical for speaker fidelity. The second paradigm builds on diffusion or score-based generative models [9,10,11], which achieve strong restoration quality but require many iterative denoising steps, leading to substantial inference cost.

Flow matching has recently attracted increasing attention as an efficient generative framework [12]. Instead of simulating a long reverse diffusion chain, flow matching learns a continuous velocity field that transports samples from a simple source distribution to the target data distribution, thereby enabling generation with substantially fewer ODE integration steps. Recent studies have demonstrated the potential of flow matching for SE [13,14], including dedicated SE flow matching models and the more recent StuPASE [15], which incorporates flow matching into a phonologically anchored two-stage system. However, existing flow matching SE systems still commonly rely on Transformer or U-Net-style backbones. Although effective, such architectures can be computationally demanding for long sequences, which weakens the efficiency advantage that makes flow matching attractive in the first place. Therefore, we focus on improving the backbone design within the flow matching framework.

Mamba [16] provides an alternative to Transformer-based sequence modeling through linear-complexity selective state-space computation, showing competitive results in speech enhancement and related audio tasks [17]. Among recent SSM variants, Mamba2 [18] reformulates the selective SSM under the Structured State-Space Duality (SSD) framework, achieving substantially improved hardware efficiency and stronger sequence modeling capability than the original Mamba. Therefore, we introduce Mamba2 to flow matching, attempting to strike a balance between the efficiency and quality of generative models in SE. However, Mamba2 is a causal model, whereas SE typically requires bidirectional contextual information. A natural remedy is to bidirectionalize the SSM, as adopted by Vision Mamba (Vim) [19], Mamba-SEUNet [20], and the BiMamba variants of Zhang et al. [21]. They typically instantiate two independent Mamba modules, processing the forward and backward sequences separately and concatenating their outputs, which essentially doubles the parameter count of the SSD layer. To avoid this overhead, we adopt a weight-sharing strategy; a single Mamba2 module is applied to both directions, reusing the same parameters. This strategy has been shown to be effective in other sequence architectures [22,23].

Beyond bidirectionalization, the fusion of forward and backward representations also requires careful design. Most prior bidirectional SSM blocks fuse the two directions through concatenation followed by a linear projection, or through additive combination, both of which treat the two directions as parallel linear contributions. This follows the success of multiplicative gating in language modeling, where Gated Linear Units have been shown to boost expressivity through cross-stream interaction [24,25]. Therefore, we adopt a multiplicative fusion where the forward and backward representations are combined via element-wise product, which acts as a feature-wise agreement gate that retains only information consistent in both temporal directions. Then, we further incorporate Adaptive Layer Normalization (AdaLN) for timestep conditioning, following the design choice established by DiT [26] for diffusion Transformers.

Based on the above research, we propose FMM, a flow-matching-based SE framework built on a gated bidirectional Mamba2 backbone. The DiMamba block is the core building unit of FMM; it combines bidirectional Mamba2 for richer contextual modeling, weight sharing across the two directions to reduce parameter overhead, multiplicative fusion to retain direction-consistent information, and Adaptive Layer Normalization for timestep conditioning. The model operates in the Mel-spectrogram domain, uses the noisy Mel-spectrogram as the conditioning signal, and is optimized with a conditional flow-matching objective augmented by classifier-free guidance. Rather than aiming for state-of-the-art performance on any single metric, FMM is designed to improve the trade-off among perceptual quality, speaker preservation, and inference efficiency in generative SE while also providing one of the first explorations of selective state-space models within the flow-matching paradigm. The main contributions of this work are summarized as follows:

(1)

We are among the first to integrate flow matching with selective state-space models for SE, achieving a real-time factor of $0.31$—more than five times faster than representative diffusion-based baselines while maintaining comparable perceptual quality.

(2)

We design DiMamba, a parameter-efficient bidirectional Mamba2 that shares weights across the forward and backward directions and fuses them through a multiplicative agreement gate, reducing the parameter overhead of bidirectionalization by approximately $21\%$ relative to the concatenation-based alternative.

(3)

We show through experiments that the proposed framework provides a favorable trade-off among perceptual quality, speaker similarity, intelligibility, and runtime efficiency when compared with representative discriminative, diffusion-based, LM-based, and flow-matching baselines.

2. Materials and Methods

2.1. Overall Architecture

Figure 1 illustrates the overall architecture of FMM. During training, DiMamba predicts the probability flow and is trained by updating the loss function. During inference, the flow predicted by the DiMamba guides an ODE solver to generate enhanced spectral features.

The DiMamba backbone learns a conditional velocity field that transports samples from a Gaussian source distribution to the clean distribution under the guidance of the noisy condition. Finally, the decoder reconstructs the enhanced speech.

2.2. Flow Matching for Speech Enhancement

In diffusion models, noisy speech is injected into the network as a conditioning. The diffusion process starts from a pure Gaussian distribution, and its reverse denoising procedure is a multi-step iterative process (typically with T ranging from 100 to 1000 steps). Flow matching [12] provides an efficient alternative by learning a continuous velocity field that directly transports a simple source distribution to the target data distribution along near-optimal paths.

As illustrated in Figure 2, diffusion models formulate generation as a discrete-time stochastic reverse process. Starting from Gaussian noise ${\mathbf{x}}_T\sim \mathcal{N}(\mathbf{0},\mathbf{I})$, the model iteratively denoises the sample over T discrete timesteps. In contrast, flow matching processing in a continuous-time, where a neural network directly parameterize a time-dependent velocity field ${\mathbf{v}}_{\theta }$ that transports samples from the source distribution ${\mathbf{x}}_0\sim \mathcal{N}(\mathbf{0},\mathbf{I})$ to the target data distribution through an ordinary differential equation. Integrating this ODE yields smooth, near-straight trajectories along the learned vector field, as illustrated in Figure 2b; this is also why high-quality generation can be achieved with fewer iteration steps.

In our formulation, the source distribution is a standard Gaussian ${\mathbf{M}}_0\sim \mathcal{N}(0,\mathbf{I})$, and the target distribution corresponds to the clean speech Mel-spectrogram ${\mathbf{M}}_x$. We do not use ${\mathbf{M}}_y$ as the initial state of the transport path. A direct noisy-to-clean path would make the trajectory closer to a deterministic restoration mapping from a corrupted observation to its clean counterpart. In contrast, the Gaussian-to-clean path preserves the generative nature of flow matching and allows the model to recover clean spectral details under the guidance of the noisy observation. The noisy Mel-spectrogram is incorporated into the conditional velocity field ${\mathbf{v}}_{\theta }({\varphi }_t,{\mathbf{M}}_y,t)$ and is fused with the current flow state through the input embedding layer at every time step. Thus, although ${\mathbf{M}}_y$ is not part of the linear interpolation path, it directly controls the predicted transport direction throughout the ODE integration.We define the interpolated state at time $t\in [0,1]$ as:

$${\varphi }_t=(1-t){\mathbf{M}}_0+t{\mathbf{M}}_x$$

(1)

where ${\mathbf{M}}_0$ is sampled Gaussian noise and ${\mathbf{M}}_x$ is the clean Mel-spectrogram. The choice of a Gaussian-to-clean interpolation path follows the standard conditional flow matching formulation, where a simple source distribution is transported to the target data distribution. In the speech enhancement setting, the target distribution is the clean speech Mel-spectrogram distribution, while the noisy Mel-spectrogram provides utterance-specific conditioning information. Therefore, our model learns the conditional distribution $p\left({\mathbf{M}}_x\right|{\mathbf{M}}_y)$ rather than an unconditional clean speech distribution. The ground-truth velocity field is then:

$${\mathbf{v}}^{*}={\mathbf{M}}_x-{\mathbf{M}}_0$$

(2)

The model learns a parameterized velocity field ${\mathbf{v}}_{\theta }({\varphi }_t,{\mathbf{M}}_y,t)$ to approximate ${\mathbf{v}}^{*}$. During inference, we initialize the ODE state as ${\mathbf{M}}_0\sim \mathcal{N}(0,\mathbf{I})$. Given the noisy Mel-spectrogram ${\mathbf{M}}_y$ as the fixed condition, we solve the following ODE:

$${\displaystyle \frac{d\mathbf{M}\left(t\right)}{dt}}={\mathbf{v}}_{\theta }(\mathbf{M}\left(t\right),{\mathbf{M}}_y,t),t\in [0,1]$$

(3)

In practice, we use a fixed-step Euler solver. Let K denote the number of ODE integration steps, $t_k=k/K$, and $\Delta t=1/K$. The discrete update rule is:

$${\mathbf{M}}^{(k+1)}={\mathbf{M}}^{\left(k\right)}+\Delta t·{\mathbf{v}}_{\theta }({\mathbf{M}}^{\left(k\right)},{\mathbf{M}}_y,t_k),k=0,1,\dots ,K-1$$

(4)

where ${\mathbf{M}}_0\sim \mathcal{N}(0,\mathbf{I})$. The final enhanced Mel-spectrogram is then obtained as ${\widehat{\mathbf{M}}}_x={\mathbf{M}}^{\left(K\right)}$.

Classifier-Free Guidance: to improve generation quality, we adopt Classifier-Free Guidance (CFG) [27]. During training, the noisy speech condition ${\mathbf{M}}_y$ is randomly dropped (replaced with zeros) with probability $p_{\mathrm{drop}}$:

$${\tilde{\mathbf{M}}}_y=\left\{\begin{array}{cc}{\mathbf{M}}_y,\hfill & \mathrm{with}\mathrm{probability}1-p_{\mathrm{drop}}\hfill \\ \mathbf{0},\hfill & \mathrm{with}\mathrm{probability}{p}_{\mathrm{drop}}\hfill \end{array}\right.$$

(5)

This enables the model to learn both conditional and unconditional velocity predictions. At inference, the guided velocity is computed as:

$$\widehat{\mathbf{v}}={\mathbf{v}}_{\theta }^{\mathrm{cond}}+s·({\mathbf{v}}_{\theta }^{\mathrm{cond}}-{\mathbf{v}}_{\theta }^{\mathrm{uncond}})$$

(6)

where s is the guidance scale and ${\mathbf{v}}_{\theta }^{\mathrm{uncond}}$ denotes the prediction with ${\mathbf{M}}_y$ set to zero.

2.3. Flow Matching with Mamba2

2.3.1. Encoder

Given the raw noisy waveform $\mathbf{y}\in {\mathbb{R}}^L$, we extract the Mel-spectrogram using the Short-Time Fourier Transform (STFT) followed by Mel filter banks:

$${\mathbf{M}}_y=f_{\mathrm{enc}}\left(\mathbf{y}\right)\in {\mathbb{R}}^{T\times F}$$

where T is the number of frames and F is the number of Mel filters.

2.3.2. Input Embedding

The interpolated state ${\varphi }_t\in {\mathbb{R}}^{T\times F}$ and the conditioning noisy Mel-spectrogram ${\mathbf{M}}_y\in {\mathbb{R}}^{T\times F}$ are concatenated along the feature dimension and projected into the hidden space via a linear layer:

$${\mathbf{h}}_0=\mathrm{Linear}\left([{\varphi }_t\parallel {\mathbf{M}}_y]\right)\in {\mathbb{R}}^{T\times D}$$

where $[·\parallel ·]$ denotes concatenation and D is the hidden dimension. During CFG training, ${\mathbf{M}}_y$ is replaced with zeros according to Equation (5). A convolutional position embedding (Conv PE) is then applied with a residual connection:

$$\mathbf{h}=\mathrm{ConvPE}\left({\mathbf{h}}_0\right)+{\mathbf{h}}_0$$

2.3.3. DiMamba Block

We employ N stacked DiMamba blocks as the backbone of the flow matching model. Each block consists of two sub-layers: a gated bidirectional Mamba2 (BiMamba2) layer for sequence mixing and a feedforward network (FFN) for channel mixing, both modulated by Adaptive Layer Normalization (AdaLN).

Following DiT [26], we use AdaLN to inject the timestep information into each sub-layer. Given the timestep embedding ${\mathbf{t}}_{\mathrm{emb}}$, the AdaLN module produces six modulation parameters via a linear projection:

$$({\gamma }_1,{\beta }_1,{\alpha }_1,{\gamma }_2,{\beta }_2,{\alpha }_2)=\mathrm{Linear}\left(\mathrm{SiLU}\left({\mathbf{t}}_{\mathrm{emb}}\right)\right)$$

(7)

where $({\gamma }_i,{\beta }_i)$ are the scale and shift parameters and ${\alpha }_i$ is the gating parameter for the i-th sub-layer. The original Mamba is a strictly causal model, processing sequences only in the forward direction. This is suboptimal for speech enhancement, where future context is equally important.

Compared to Mamba, the Structured State Space Duality (SSD) formulation supports a substantially larger state expansion at comparable cost, providing the additional representational capacity required to model the rich harmonic and temporal structure of clean speech. Then, we adopt Mamba2 as the core sequence mixer in DiMamba block. A common way to bidirectionalize an SSM-based block is to instantiate two independent modules and concatenate their outputs, as adopted by previous bidirectional Mamba designs for speech enhancement [20]. While effective, this approach doubles the parameter count of the SSD layer and therefore weakens the efficiency advantage that initially motivates the use of state-space models. Meanwhile, parameter tying has likewise been shown to be an effective regularizer in Transformer architectures, where it reduces overfitting and improves parameter efficiency without significantly degrading performance [22]. Therefore, we propose a weight-shared bidirectional extension based on Mamba. Beyond the direct reduction in parameter footprint, this design encodes an explicit inductive bias, where the selective state-space dynamics are constrained to be direction agnostic, and the same transition rules describe how past frames influence the present and how future frames do.

Specifically, given the modulated input, we process it through a shared Mamba2 module in both the original and time-reversed directions:

$${\mathbf{o}}_{\mathrm{fwd}}={\mathcal{M}}_{\theta }\left(\mathbf{x}\right),{\mathbf{o}}_{\mathrm{bwd}}=\mathrm{Flip}\left({\mathcal{M}}_{\theta }\left(\mathrm{Flip}\left(\mathbf{x}\right)\right)\right)$$

(8)

where $\mathrm{Flip}(·)$ denotes the operation that reverses the sequence along the time axis, and ${\mathcal{M}}_{\theta }(·)$ denotes the shared selective state-space operator, instantiated as a Mamba2 block in our implementation. The inner Flip reverses the input so that the shared Mamba2 module processes the sequence in the backward direction, while the outer Flip restores the resulting representation to the original forward temporal order. As a result, both ${\mathbf{o}}_{\mathrm{fwd}}$ and ${\mathbf{o}}_{\mathrm{bwd}}$ are aligned at every time step t, and the element-wise fusion is well-defined:

$$\mathbf{f}={\mathbf{o}}_{\mathrm{fwd}}\odot {\mathbf{o}}_{\mathrm{bwd}}\in {\mathbb{R}}^{T\times D}$$

(9)

To further control the information flow, the gate is defined as:

$$\mathbf{g}=\mathrm{SiLU}\left({\mathbf{W}}_g\mathbf{x}\right)\in {\mathbb{R}}^{T\times D}$$

where ${\mathbf{W}}_g\in {\mathbb{R}}^{D\times D}$ are learnable parameters. The output of the bidirectional Mamba2 layer is:

$$\mathbf{o}={\mathbf{W}}_o\left(\mathbf{f}\odot \mathbf{g}\right)$$

(10)

where ${\mathbf{W}}_o\in {\mathbb{R}}^{D\times D}$ is the output projection. In our implementation, we adopt $\mathrm{SiLU}$ (also known as Swish) as the gating activation rather than sigmoid or ReLU, following the design choices established in Mamba [16] and Mamba2 [18], where $\mathrm{SiLU}$ has been shown to be effective for selective state-space sequence modeling.

Compared to the concatenation-based bidirectional design that requires two independent Mamba2 modules ($2.30\times$ the parameters of a unidirectional Mamba2), our shared-weight scheme requires only $1.30\times$ the parameters while achieving multiplicative interaction between forward and backward representations. Each Mamba2 module internally follows the Structured State Space Duality (SSD) algorithm, which achieves linear complexity $\mathcal{O}\left(T\right)$ with respect to sequence length through a chunked computation strategy. Figure 3 illustrates a comparison between direct bidirectional concatenation and our designed shared-weight structure.

Each Mamba2 module internally follows the Structured State Space Duality (SSD) algorithm, which achieves linear complexity $\mathcal{O}\left(T\right)$ with respect to sequence length through a chunked computation strategy.

2.3.4. Training Objective

The model is optimized with the flow matching loss, which minimizes the mean squared error between the predicted and ground-truth velocity fields:

$${\mathcal{L}}_{\mathrm{FM}}={\mathbb{E}}_{t,{\mathbf{M}}_0,{\mathbf{M}}_x}\left[\parallel {\mathbf{v}}_{\theta }({\varphi }_t,{\mathbf{M}}_y,t)-{\mathbf{v}}^{*}{\parallel }_2^2\right]$$

(11)

Additionally, to further encourage the predicted velocity to produce spectrograms that are spectrally consistent with the clean target, an ${\ell }_1$ reconstruction loss on the generated Mel-spectrogram is incorporated:

$${\mathcal{L}}_{\mathrm{mel}}={\mathbb{E}}_{t,{\mathbf{M}}_0,{\mathbf{M}}_x,{\mathbf{M}}_y}\left[\parallel {\mathbf{M}}_x-{\widehat{\mathbf{M}}}_x^{\left(t\right)}{\parallel }_1\right]$$

(12)

where ${\widehat{\mathbf{M}}}_x^{\left(t\right)}$ is a single-step endpoint estimate computed directly from the current interpolated state ${\varphi }_t$ and the network’s velocity prediction:

$${\widehat{\mathbf{M}}}_x^{\left(t\right)}={\varphi }_t+(1-t)·{\mathbf{v}}_{\theta }({\varphi }_t,{\mathbf{M}}_y,t)$$

(13)

We do not unroll the inference ODE of Equation (4) at training time, and the multi-step Euler integration is used only at inference. The total training objective is:

$$\mathcal{L}={\mathcal{L}}_{\mathrm{FM}}+\lambda {\mathcal{L}}_{\mathrm{mel}}$$

(14)

where $\lambda$ is the weighting coefficient balancing the two terms. In our implementation, we set $\lambda =0.1$, which we found to provide a stable auxiliary signal without overwhelming the primary flow matching objective.

2.3.5. Decoder

The enhanced Mel-spectrogram ${\widehat{\mathbf{M}}}_x$ is converted back to a waveform. We adopt Vocos [28] as the vocoder to reconstruct waveforms from the predicted spectral features. Unlike conventional time-domain GAN vocoders, which rely on stacked transposed convolutions for upsampling, Vocos directly predicts the magnitude and phase of the short-time Fourier transform (STFT) and synthesizes the waveform via inverse STFT. This Fourier-domain formulation avoids the costly upsampling stages and yields an order-of-magnitude faster inference while maintaining audio fidelity competitive with state-of-the-art time-domain vocoders. The fully convolutional and time-aliased-free design also delivers improved spectral consistency and superior reconstruction of high-frequency components, making Vocos particularly well-suited for our speech enhancement framework.

$$\widehat{\mathbf{x}}=f_{\mathrm{dec}}\left({\widehat{\mathbf{M}}}_x\right)$$

3. Experimental Setup

3.1. Datasets

3.1.1. Dataset Configuration

We constructed a large-scale training set by combining clean speech, noise, and room impulse responses (RIRs) from publicly available corpora. Clean speech was drawn from three sources: LibriSpeech [29] (the train-other-500 subset, ∼500 h), VoiceBank (VCTK) [30] (∼44 h, 110 speakers), and a ∼100 h subset of GigaSpeech [31], giving a total of approximately 640 h of clean speech for training. Noise samples were drawn from the entirety of DEMAND [32], which spans 18 acoustic environments, and from the urban-noise portion of WHAM! [33]. RIRs were taken from all of OpenSLR-26 and OpenSLR-28 [34], covering both simulated and real-recorded room responses.

3.1.2. Training, Validation, and Test Splits

Splits were constructed in a speaker disjoint manner, so that no speaker appearing in any validation or test set was present in the training set. This is particularly important for the speaker-similarity (Spk-Sim) evaluation, which would otherwise be biased toward memorized speakers.

(1)

LibriSpeech. We used the train-other-500 subset for training and the official dev-clean and test-clean subsets for validation and testing, respectively. These official splits are speaker-disjoint by construction.

(2)

VCTK. Speakers were partitioned into train/validation/test splits with a ratio of $8:1:1$, ensuring that no speaker appears in more than one split. Utterances within each split were used in full.

(3)

GigaSpeech. A ∼100 h subset was drawn from the GigaSpeech training partition for training only; GigaSpeech was not used for evaluation.

The final validation set, used for hyperparameter selection (e.g., $\lambda$ in Equation (14), the CFG scale s, and the number of inference steps K), aggregates the LibriSpeech dev-clean subset and the VCTK validation split. None of the validation utterances were used to compute any score reported in Section 4.

3.1.3. Online Degradation Pipeline

Noisy–clean training pairs were generated during training so that each clean utterance is paired with a freshly sampled degradation at every epoch. The degradation pipeline applied the following operations sequentially:

(1)

Reverberation. With probability $0.25$, an RIR was randomly sampled (uniformly) from the OpenSLR-26/28 pool and convolved with the clean utterance; otherwise, the utterance was kept anechoic.

(2)

Additive noise. A noise segment was randomly cropped from the combined DEMAND + WHAM! noise pool and added to every training sample. The SNR was drawn from a four-bin categorical distribution that emphasizes moderately noisy conditions: bins $[-5,0)$, $[0,5)$, $[5,10)$, and $[10,20]$ dB were selected with probabilities $0.05$, $0.30$, $0.45$, and $0.20$, respectively, and the SNR was sampled uniformly within the chosen bin. This allocates more probability mass to the moderate-SNR regime, which is closer to typical real-world conditions while still exposing the model to both severe and mild cases.

(3)

Bandwidth limitation. With probability $0.25$, the mixture was further low-pass filtered with a cutoff frequency drawn uniformly from $\{2,4,8\}$ kHz, simulating bandwidth-limited recording conditions.

(4)

Loudness normalization. The noisy mixture and its clean reference were jointly scaled to a target dBFS level drawn from $[-35,-15]$ dBFS, with the same gain applied to both signals to preserve their relative amplitude and the target SNR.

Because the pipeline runs online, each training epoch generates a fresh set of degraded mixtures, providing strong implicit data augmentation across the full training schedule.

3.1.4. Test Sets

We evaluate the methods on three test sets, all disjoint from the training data: the DNS-Challenge 2021 test set [35], the LibriSpeech test-clean subset, and the VCTK test split. The LibriSpeech and VCTK test mixtures were generated using the same online degradation pipeline as in training, but with held-out noise and RIR samples and a fixed random seed, so that all compared methods are evaluated on exactly the same noisy inputs.

3.2. Evaluation Metrics

We use DNSMOS [36], a non-intrusive neural metric predicting ITU-T P.835 ratings in DNS-Challenge, and report its SIG, BAK, and OVL scores. We additionally report a quality MOS (QMOS) computed with the DNS-Challenge P.808 neural predictor, which approximates ITU-T P.808 overall quality and is, like DNSMOS, an objective non-intrusive estimate rather than a human listening score. PESQ and STOI are not reported, since both require sample-level waveform alignment between enhanced and reference signals, which does not hold for generative models such as flow matching and diffusion-based methods. The same choice has been adopted by recent generative SE works [10,13]. And, it is directly aligned with the DNS Challenge benchmark used in our evaluation, which ensures a fair comparison with prior work. Other non-intrusive metrics such as OSQA-SI and S3QA could provide complementary perspectives, and we leave their integration for future work.

Speaker similarity (Spk-Sim) is computed as the cosine similarity between WeSpeaker embeddings [37] of the enhanced and clean speech. Word error rate (WER) is obtained by transcribing the enhanced speech with the Whisper Turbo model (large-v3-turbo) [38] under default greedy decoding and computing WER with jiwer using the same ASR configuration for all methods. All RTF measurements were taken on a single V100 GPU under bf16 inference, and the length of all test audio was standardized to 8 s. All reported scores are means across the corresponding test subset. For QMOS, we additionally report a $95\%$ confidence interval computed from per-utterance scores assuming a t-distribution.

3.3. Training Setup

Input waveforms (originally 16 kHz) were upsampled to 24 kHz, then Mel-spectrograms were extracted from 24 kHz audio using a hop length of 256 samples and 100 Mel filter banks. The DiMamba backbone consists of $N=12$ blocks with hidden dimension $D=1024$, following the SSD configuration of Mamba2 [18]. We used the AdamW optimizer with default hyperparameters and a peak learning rate of $7.5\times {10}^{-5}$, with a $25,000$-step linear warm-up followed by a cosine decay schedule. Training was run for up to 200 epochs on 8 NVIDIA V100 GPUs (32 GB) using bf16 mixed precision, with a per-GPU batch size of 16 and a 4 s random crop per utterance. To improve efficiency, we adopted early stopping with patience 20 on the validation flow-matching loss, and the best-validation checkpoint was used for all reported results.

For Classifier-Free Guidance (Equation (5)), the noisy condition was randomly dropped with probability $p_{\mathrm{drop}}=0.1$ during training. At inference, we used a guidance scale of $s=1.0$ and $K=20$ Euler integration steps by default; further increasing K yielded only marginal improvement.

3.4. Baselines

We compare FMM against three families of representative speech enhancement methods: discriminative models (Conv-TasNet [39], Demucs [40]); diffusion-based generative models (CDiffuSE [9], SGMSE [11], StoRM [10]); and LM-based and flow-matching generative models (SELM [5], MaskSR [4], FlowSE [13]).

For all baselines with publicly available pre-trained models, we used the official released checkpoints and recomputed all metrics on exactly the same test mixtures as FMM. SELM and MaskSR provide neither an open-source implementation nor a public checkpoint. For these two methods, we report results on the DNS-Challenge test set, which is the common benchmark adopted in their original papers. We note that this is an approximate comparison, since training data and preprocessing may differ across publications.

4. Results and Discussion

4.1. Overall Comparison with Baselines

Table 1. Overall comparison on the DNS Challenge test set. “Diff” and “D” represent the diffusion and discriminative models. “Flow” represent the Flow matching models. ↑ = higher is better.

Model	Type	With Reverb				Without Reverb				Real Recording
		DNSMOS ↑			Spk Sim ↑	DNSMOS ↑			Spk Sim ↑	DNSMOS ↑
		SIG	BAK	OVL		SIG	BAK	OVL		SIG	BAK	OVL
Noisy	-	1.760	1.497	1.392	0.942	3.392	2.618	2.483	0.970	3.053	2.509	2.255
Conv-TasNet	D	2.415	2.710	2.010	0.815	3.092	3.341	3.001	0.802	3.102	2.975	2.410
Demucs	D	2.510	2.641	2.215	0.804	3.124	3.257	3.010	0.802	2.974	2.870	2.291
CDiffuSE	Diff	2.541	2.300	2.190	0.761	3.294	3.641	3.047	0.765	3.201	3.104	2.781
SGMSE	Diff	2.730	2.741	2.430	0.764	3.501	3.710	3.137	0.782	3.297	2.894	2.793
StoRM	Diff	2.947	3.141	2.516	0.790	3.514	3.941	3.205	0.798	3.410	3.379	2.940
FlowSE	Flow	3.391	3.957	3.109	0.861	3.556	4.032	3.291	0.876	3.481	4.004	3.119
FMM(Mamba2)	Flow	3.443	3.803	3.049	0.868	3.581	4.091	3.323	0.886	3.502	4.061	3.227

evaluates all models on the DNS-Challenge test set under three conditions: with reverberation, without reverberation, and real-world recordings. Since generative models lack strict waveform alignment with the reference signal, traditional metrics (PESQ, STOI, SI-SNR) are inapplicable. We therefore adopt DNSMOS as the perceptual quality metric and Spk Sim to assess speaker characteristic preservation.

In Table 1, Discriminative models directly estimate and subtract the noise component. While this preserves relatively high Spk-Sim under clean conditions, the enhanced speech suffers from audible distortions under reverberant scenarios, yielding limited DNSMOS improvements. Diffusion-based models progressively improve perceptual quality through multi-step iterative refinement. However, the iterative process introduces cumulative distortions that degrade Spk-Sim, lower than discriminative methods. Compared with FlowSE, FMM achieves only a small advantage on SIG and Spk-Sim while showing comparable or slightly lower scores on BAK and OVL under reverberant conditions. We do not regard this as a clear quality improvement over FlowSE in absolute terms. Rather, the goal of FMM is to better balance the trade-off between quality and inference efficiency. Along this trade-off, FMM offers a substantial efficiency advantage over the diffusion-based baselines in Figure 4, while attaining perceptual quality on par with the state-of-the-art flow matching baseline. Overall, the proposed method provides a competitive operating point under the joint consideration of quality and efficiency rather than a dominant improvement on any single metric.

4.2. Speaker Preservation and Intelligibility Analysis

To further examine whether the proposed method preserves linguistic content and overall perceptual quality, we report the WER and QMOS results in

Table 2. WER and MOS scores with 95% confidence intervals on the simulated testset. ↑ = higher is better, ↓ = lower is better.

Model	Type	WER ↓	QMOS ↑
Noisy	-	25.20	-
Conv-TasNet	D	18.6	3.00 ± 0.07
Demucs	D	17.4	3.07 ± 0.10
CDiffuSE	Diff	16.8	3.13 ± 0.08
SGMSE	Diff	15.3	3.27 ± 0.07
StoRM	Diff	15.1	3.37 ± 0.10
FMM	Flow	4.7	3.58 ± 0.10

. QMOS is computed with the DNS-Challenge P.808 neural predictor and approximates ITU-T P.808 overall quality, providing an objective non-intrusive estimate of perceptual quality. Compared with diffusion-based baselines, the proposed method yielded lower WER ($4.7\%$), indicating that the enhanced speech retained clearer linguistic information for downstream recognition. And, the highest QMOS ($3.58$) suggests that the perceptual gains observed in DNSMOS are also reflected in a P.808-based estimate.

Taken together with the speaker-similarity scores in Table 1, these results indicate that FMM improves perceptual quality and intelligibility while still preserving speaker characteristics rather than trading one off against the others. And, we note that QMOS is an objective neural network-predicted score rather than a true human listening test; we therefore interpret the QMOS comparison as an additional perceptual signal alongside DNSMOS rather than as a substitute for subjective evaluation. A controlled human listening study would provide a more definitive assessment and is left for future work.

4.3. Runtime Efficiency Analysis

Figure 4 compares the runtime efficiency of representative generative baselines. As expected, diffusion-based methods required substantially more computation because they relied on iterative denoising over many inference steps. In contrast, FMM achieves a markedly lower RTF of $0.31$, more than five times faster than the diffusion baselines (RTF $1.69$–$1.80$), reflecting the advantage of learning a continuous transport field that can be integrated with far fewer ODE steps.

To assess the actual end-to-end deployment cost, we additionally measured the RTF of the Vocos vocoder under the same hardware and obtained an additional cost of only $0.01$–$0.02$. The end-to-end RTF including waveform reconstruction is therefore approximately $0.32$–$0.33$, which remains substantially below all diffusion-based baselines and confirms that the vocoder stage does not materially affect the efficiency advantage of FMM.

Importantly, the gain in runtime did not come at the expense of a severe drop in perceptual quality or speaker preservation. When considered together with the results in

Table 3. DNSMOS metrics on the VCTK and Librispeech test set; NFE represents the number of running a DNN model to evaluate scores in the inference phase. ↑ = higher is better.

Model	Type	NFE	Reverb Set ↑			No Reverb Set ↑
			SIG	BAK	OVL	SIG	BAK	OVL
CDiffuSE	Diff	>50	2.451	2.310	2.228	3.225	3.214	2.687
SGMSE	Diff	>50	2.853	2.812	2.510	3.267	2.956	2.834
StoRM	Diff	>50	2.967	3.241	2.627	3.401	3.476	2.914
FMM	Flow	10	3.380	3.944	3.100	3.477	4.046	3.198
FMM	Flow	15	3.445	4.012	3.165	3.472	4.052	3.200
FMM	Flow	20	3.473	4.036	3.189	3.474	4.065	3.202

and Figure 4, our method achieves significant improvement in inference efficiency while maintaining high perceptual quality. It shows that the proposed method offers a more favorable quality–efficiency trade-off than diffusion-based baselines.

4.4. Ablation Study on the Backbone Design

We compare three backbone variants under the same flow matching framework in

Table 4. DNSMOS metrics in VCTK and LibriSpeech test set; Concat Mamba2 represents the direct bidirectional concatenation design. ↑ = higher is better.

Model	Type	Parameters	Reverb Set ↑			No Reverb Set ↑
			SIG	BAK	OVL	SIG	BAK	OVL
Noisy	-	-	1.870	1.678	1.428	3.021	2.543	2.168
Mamba	Flow	261 M	2.875	3.330	2.502	3.288	3.700	2.877
Concat Mamba2	Flow	364 M	3.269	3.913	2.950	3.451	4.021	3.166
FMM(our)	Flow	286 M	3.380	3.944	3.100	3.477	4.046	3.198

. Replacing the unidirectional Mamba backbone with our gated bidirectional Mamba2 (FMM) substantially improves DNSMOS-OVL on both reverberant ($+0.598$) and non-reverberant ($+0.321$) subsets, with only a $9.6\%$ increase in parameters (261 M → 286 M). Compared with the direct concatenation-based bidirectional design (Concat Mamba2), FMM achieves consistent improvements across all DNSMOS sub-scores while using approximately $21\%$ fewer parameters (364 M → 286 M). These results indicate that the shared-weight design exploits bidirectional context more effectively than parameter-heavier concatenation.

Despite this efficiency improvement over the concatenation variant, the resulting 286 M parameter count is still non-negligible. While it is in line with other recent generative speech enhancement models [4,13], deployment of FMM in practice still requires reasonably capable hardware, and porting the model to resource-constrained settings would benefit from further parameter reduction. We view exploration of lighter backbone configurations and model-compression strategies as a natural extension of the present design rather than a fundamental limitation and leave a systematic study of these directions to future work.

4.5. Speaker Reconstruction Experiment

Meanwhile, compared to LM-based models, discretizing speech and integrating it into a Transformer architecture enables high-quality speech enhancement. However, the information loss and speech artifacts resulting from this discretization are inevitable. Therefore, in

Table 5. DNSMOS and Spk-Sim metrics on the DNS Challenge test set. ’LM’ represent models based on LMs. ↑ = higher is better.

Model	Type	Spk-Sim	Reverb Set ↑			No Reverb Set ↑
			SIG	BAK	OVL	SIG	BAK	OVL
Noisy	-	-	1.760	1.497	1.392	3.392	2.618	2.483
SELM	LM	0.793	3.160	3.577	2.695	3.508	4.086	3.258
Masksr	LM	0.816	3.542	4.016	3.223	3.575	4.082	3.307
FMM (our)	Flow	0.868	3.443	3.803	3.049	3.581	4.091	3.323

, we compare the differences in the Spk-sim and DNSMOS metrics.

On Spk-Sim, FMM reaches $0.868$, clearly outperforming MaskSR ($0.816$, $+0.052$) and SELM ($0.793$, $+0.075$). Because FMM operates on continuous Mel-spectrograms and recovers speech via a continuous flow field, it avoids the codebook-induced loss of fine-grained speaker information inherent to LM-based methods. On DNSMOS, FMM and MaskSR perform comparably, FMM is best on the non-reverberant subset across all three sub-metrics, while MaskSR is slightly higher on BAK and OVL under reverberation. Overall, FMM preserves speaker identity more faithfully than LM-based alternatives while achieving comparable perceptual quality.

5. Conclusions

In this study, we proposed FMM, an efficient generative speech enhancement framework that combines flow matching with a gated bidirectional Mamba2 backbone. The main goal of this work was to improve the trade-off among perceptual quality, speaker preservation, intelligibility, and inference efficiency in offline speech enhancement. The proposed method formulates speech enhancement as a conditional flow matching problem in the Mel-spectrogram domain and introduces a DiMamba block to model bidirectional acoustic context with adaptive directional fusion. Compared with conventional Transformer, U-Net, or direct bidirectional concatenation-based designs, the proposed backbone provides a more parameter-efficient way to exploit both past and future information for speech restoration.

Experimental results on the DNS Challenge test set and an additional VoiceBank test set demonstrate that FMM achieves a favorable balance between enhancement quality and inference efficiency. Specifically, the model attains DNSMOS, Spk-Sim, WER, and QMOS scores comparable to the strongest generative baselines while reaching a real-time factor of $0.31$ on the DiMamba backbone, several times faster than diffusion-based methods. Ablation results further indicate that the gated bidirectional Mamba2 design is more effective than unidirectional Mamba and direct bidirectional concatenation under a comparable parameter budget.

While these results are encouraging, several limitations remain. The reported RTF reflects only the DiMamba backbone. Including the Vocos vocoder, the end-to-end RTF rises only marginally to approximately $0.32$–$0.33$, which is still well below diffusion-based baselines. The 286M-parameter model also requires reasonably capable hardware, and the offline, full-utterance bidirectional formulation is not directly suited to low-latency streaming. We plan to address these issues along three directions in future work: (i) extending the framework to chunk-wise or streaming inference, leveraging the stateful nature of Mamba2; (ii) reducing the parameter footprint via lighter backbone configurations and model compression to enable deployment on resource-constrained devices; and (iii) tightening the integration between the flow-matching backbone and the waveform reconstruction stage to mitigate the two-stage inference cost and potential spectral artifacts.