AMUSE++: A Mamba-Enhanced Speech Enhancement Framework with Bi-Directional and Advanced Front-End Modeling

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This paper, building upon the MUSE++ model, introduces an enhanced speech enhancement framework named AMUSE++. The core innovation lies in the deep optimization and extension of the Mamba module within MUSE++: firstly, by upgrading the original unidirectional Mamba structure to a bidirectional architecture to more comprehensively capture the temporal contextual dependencies of speech signals; secondly, by extending it from one-dimensional to two-dimensional, enabling joint modeling of both the time and frequency dimensions of the spectrogram. Furthermore, the paper designs a "Preliminary Denoising Module" (PDM) composed of cascaded 2D bidirectional Mamba modules to preprocess the input features before they are handled by the main network, effectively enhancing the model's noise reduction capabilities. The paper is well-structured and logically rigorous, demonstrating a systematic approach from theoretical motivation and model design to experimental validation. Through extensive experiments on the standard VoiceBank+DEMAND dataset, including comparisons with baseline models (MUSE, MUSE++) and various advanced lightweight models, as well as detailed ablation studies, the effectiveness of the proposed method is thoroughly validated. The results indicate that AMUSE++, while maintaining a low parameter count (0.31M), achieves significant and consistent improvements across multiple key objective evaluation metrics such as PESQ, CSIG, CBAK, and COVL, providing valuable research insights and practical solutions for the development of lightweight speech enhancement technology.

However, the paper presents areas for further improvement regarding model generalization, comprehensiveness of experimental validation, and practical application considerations. Firstly, all experiments were conducted on the VoiceBank+DEMAND dataset, which is generated by artificially mixing clean speech with noise. Although a standard benchmark in the field, it does not fully represent the complex and variable acoustic environments of the real world. The paper lacks validation on genuinely recorded, unprocessed audio from noisy environments, which casts uncertainty on whether the model's excellent performance on simulated data can be directly transferred to practical applications.

Secondly, the proposed AMUSE++ model is primarily designed and optimized for single-channel, offline processing scenarios. Its adaptability and effectiveness in scenarios crucial for many real-world applications—such as multi-channel microphone array enhancement and low-latency, streaming real-time processing (e.g., in smart conference systems, hearing aids)—have not been verified. This, to some extent, limits the direct applicability of the technical solution.

Furthermore, the performance evaluation of the paper relies entirely on objective metrics (e.g., PESQ, STOI). While widely adopted in academia, their correlation with human subjective auditory perception is not perfect. The study fails to incorporate subjective listening tests (such as MUSHRA or MOS scoring), where human listeners directly evaluate the quality of the enhanced speech. Consequently, there is a lack of direct evidence to support whether the improvements in objective metrics equate to a substantial enhancement in user auditory experience.

Additionally, although the paper emphasizes the model's lightweight nature, supported by its parameter count, it does not provide key performance indicators such as actual inference speed or Real-Time Factor (RTF) on specific hardware platforms (e.g., CPU or embedded devices). These metrics are crucial for assessing the feasibility of deploying the model on resource-constrained devices, and their absence makes the claims of "lightweight" and "efficient" less concrete and substantiated.

Finally, while the "Future Work" section mentions model compression and hardware-aware optimization, the current version does not delve into the model's robustness against different types of noise, especially highly non-stationary or transient noises not included in the training set. Moreover, the study does not explicitly state whether the source code and pre-trained models for AMUSE++ will be publicly released, which could hinder the reproducibility of the research and the community's ability to build upon it.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This article is both interesting and relevant. The authors' main contribution is the AMUSE++ system, which significantly expands the capabilities of MUSE++ by enabling bidirectional, two-dimensional Mamba modelling. The authors transformed the one-dimensional, unidirectional Mamba module into a bidirectional, two-dimensional architecture for the first time, making it capable of explicitly capturing spectro-temporal dependencies in both the time and frequency domains. Additionally, they developed a preliminary noise suppression module (PDM) that processes input speech features early on and increases the efficiency of the subsequent improvement stage. The proposed approach is supported by a multi-purpose loss function that provides direct control and stable training of the extended front-end module.

At the same time, I have some comments on the work:

1) The authors should check for an error in the term 'hardware-aware' (carryover from line 563 to line 564).

2) In subsection 5.3, the description of improvements is based on visual observations, such as clarity of harmonics and darkening of interharmonic regions. However, this description is not supported by additional local or frequency-dependent metrics that could objectively confirm the claimed qualitative differences between the methods.

3) The authors of the article only performed the evaluation on VoiceBank+DEMAND. No experiments were conducted on other corpora. This makes it difficult to assess the generalisability of AMUSE++ in different noise and speech conditions.

4) The authors should provide information on the computational complexity. The evaluation was performed only on VoiceBank+DEMAND. There are no experiments on other datasets. This makes it difficult to assess AMUSE++'s ability to generalise in different noise and speech conditions. How does AMUSE++ perform at an extremely low SNR? What is the computation time?

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The authors present an improved neural network for noisy speech intelligibility enhancement. Advantages of the text: 1) Enhanced neural-network architecture. 2) Training based on open databases. 3) Careful assessment of the enhancer. Disadvantages: 1) The algorithm compares only with other NN-based algorithms. No non-ML algorithms are accounted for comparison. 2) AMUSE++ algorithm gives only incremental enhancement of standard scores (circa 3%) in comparison with MUSE++, but the number of parameters increases by two times. No discussion found about real profits from the fatter model. 3) Figure 6 reveals that AMUSE++ tends to insert long sections of silence even in time slots that are evidently noisy in the original speech. It is usually very distracting for the listener. Please comment on it.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf