Dynamic Ultrasonic Jamming via Time–Frequency Mosaic for Anti-Eavesdropping Systems

Abstract

To combat microphone eavesdropping on devices like smartphones, ultrasonic-based methods offer promise due to human inaudibility and microphone nonlinearity. However, existing systems suffer from low jamming efficiency, poor energy utilization, and weak robustness. Based on these problems, this paper proposes a novel ultrasonic-based jamming algorithm called the Time–Frequency Mosaic (TFM) technique, which can be used for anti-eavesdropping. The proposed TFM technique can generate short-time, frequency-coded jamming signals according to the voice frequency characteristics of different speakers, thereby achieving targeted and efficient jamming. A jamming prototype using the Time–Frequency Mosaic technique was developed and tested in various scenarios. The test results show that when the signal-to-noise ratio (SNR) is lower than 0 dB, the text Word Error Rate (WER) of the proposed method is basically over 60%; when the SNR is 0 dB, the WER of the algorithm in this paper is on average more than 20% higher than that of current jamming algorithms. In addition, when the jamming system maintains the same distance from the recording device, the algorithm in this paper has higher energy utilization efficiency compared with existing algorithms. Experiments prove that in most cases, the proposed algorithm has a better jamming effect, higher energy utilization efficiency, and stronger robustness.

1. Introduction

With the rapid development of recording technology and the widespread proliferation of electronic devices equipped with microphones, people are able to conveniently conduct voice calls and audio recordings. However, they are also confronted with the risk of being eavesdropped. Hackers exploit disguised recording devices or voice assistants found in smartphones to eavesdrop on private conversations and meetings [1,2,3], resulting in the leakage of user privacy. Covert microphones play a significant role in these espionage activities [4], which has drawn attention to anti-eavesdropping security.

Traditional anti-eavesdropping techniques primarily include audible acoustic interference and electromagnetic interference [5]. The audible acoustic interference method employs white noise generators [6] or high-frequency noise sources [7] to mask acoustic information. Although highly effective, this approach significantly compromises the quality of normal communication. Electromagnetic interference countermeasures operate by introducing electromagnetic signals to disrupt microphone sensor systems [8,9]. However, this method has high power consumption requirements, and many current microphone recording devices are equipped with built-in anti-interference algorithms, which results in a poor interference effect of anti-eavesdropping interference signals [5]. In addition, it may violate electromagnetic spectrum management regulations, creating legal risks [10].

Since ultrasonic sounds are inaudible to the human ear, they can be covertly utilized for anti-eavesdropping in smart devices [5], leading to the proposal of ultrasonic microphone jammers [11]. The inherent nonlinear characteristics of microphone systems enable the leakage of ultrasonic energy into the audible frequency band [12,13], thereby pioneering new avenues for covert acoustic interference and establishing an emerging research direction in information security [14,15,16]. Compared with traditional audible-band jamming methods, ultrasonic interference demonstrates distinctive advantages: (1) The inaudibility of high-frequency carriers ensures user conversational comfort; (2) directional transmission characteristics enable precise spatial coverage; (3) near-field energy concentration enhances power utilization efficiency.

Current research on anti-eavesdropping technologies based on ultrasonic interference has primarily developed along three technical pathways. Firstly, the ultrasonic carrier modulation technique leverages nonlinear intermodulation between ultrasonic waves and low-frequency noise to generate audible-band interference in microphone circuits [17]. Building on this, Chen et al. [18] designed a portable ultrasonic anti-eavesdropping gadget, while Shen et al. [19] developed a directional ultrasonic coverage system. Secondly, the high-power single-tone jamming technique induces microphone circuit saturation through high-intensity single-frequency ultrasonic waves. In this context, Gui et al. [20] proposed an audio amplitude attenuation system. Thirdly, the speech feature adaptation technique dynamically generates ultrasonic adversarial signals aligned with the temporal–spectral characteristics of speech. For instance, Reference [21] introduced an interference method based on human vocal feature matching to enhance jamming efficiency. Additionally, Reference [22] presented an auditory information masking technique that reduces speech recognition accuracy by minimizing the perceptual gap between noise and target speech.

Based on these three technical pathways, some anti-eavesdropping systems that utilize ultrasonic waves for interference have been proposed. The literature [23] presents an anti-eavesdropping system capable of resisting deep learning denoising and feature denoising. It addresses the security vulnerability in traditional ultrasonic interference, where 75% of content can be recovered by using adaptive gain compensation and broad-band coverage, but it does not consider the design of system power consumption and cost. The literature [5] converts random noise into ultrasonic signals through DSB modulation and generates full-band audible noise via self-demodulation, achieving progress in concealment and operating distance; however, it does not consider specific optimization for human voices. The literature [3] solves the pain point of indiscriminate interference in traditional schemes through a selective interference mechanism and triple anti-recovery security design, but it has residual low-frequency noise and fails to fundamentally eliminate the impact of nonlinear effects. The literature [1] proposes an evaluation framework and standards for ultrasonic anti-eavesdropping systems, reveals the general vulnerability of existing UMJ schemes, and puts forward optimization paths for the design of ultrasonic interference systems. Nevertheless, its framework has insufficient coverage of dynamic scenarios and certain limitations.

Based on the above analysis, existing anti-eavesdropping interference systems based on ultrasonic waves still face three major limitations: (1) Spectral Mismatch—Most methods generate uniform-spectrum interference, failing to account for the nonuniform energy distribution characteristics inherent to human speech [24,25]; (2) Temporal Blindness—Existing systems lack real-time adaptation to speech dynamics, resulting in energy wastage during speech pauses [22]; (3) Hardware Constraints—The limited conversion efficiency of ultrasonic transducers restricts practical deployment scenarios.

Therefore, how to design an anti-eavesdropping system with good jamming effect, high jamming efficiency, and strong robustness based on ultrasonic interference remains a problem worthy of attention. To address these issues, this work proposes the Time–Frequency Mosaic (TFM) jamming paradigm, fundamentally redefining the design of ultrasonic anti-eavesdropping systems. The core innovations of this work include the following:

• Time–Frequency Joint Interference: A novel joint jamming method that integrates temporal and frequency domain characteristics is proposed for the first time. Unlike typical noise injection techniques, TFM not only adjusts its frequency but also modulates its temporal sequence.
• Adaptive Ultrasonic Encoding Strategy: Based on the state of the speech frame, the jamming strategy is dynamically switched to enhance the robustness of the jamming.
• Energy-Constrained Hardware Co-Design: The amplitude of the jamming signal is dynamically adjusted according to the energy of the speech frame, ensuring effective jamming while reducing power consumption.

The remainder of this paper is organized as follows: Section 2 analyzes the principles of the anti-eavesdropping system; Section 3 introduces the principles of the proposed TFM technology; Section 4 constructs the hardware platform of the anti-eavesdropping system and conducts tests; Section 5 concludes the research.

2. System Principle and Analysis

2.1. Out-of-Band Signal Conversion in Microphone

The signal path of the speech acquisition hardware consists of microphones, amplifiers, low-pass filters, and analog-to-digital converters(ADCs), as shown in Figure 1a. And the frequency range for human hearing and speech communication usually covers 20 Hz to 20 kHz. As a result, most microphone systems operate within this range. Even if a microphone is capable of detecting frequencies above 20 kHz, a low-pass filter efficiently removes these high-frequency signals. The typical sampling rate of analog-to-digital converters is 44.1 kHz as per the Nyquist sampling theorem, which states that the digital signal frequency should be below 22 kHz. For recording systems, in-band audible speech signals up to 22 kHz and inaudible ultrasonic signals above 25 kHz are typical out-of-band signals.

Although electronics, such as amplifiers, are typically designed with a linear input–output relationship, the actual devices exhibit nonlinear characteristics. Due to the nonlinear characteristics, the electronics may suffer from intermodulation distortion (IMD), which results in the presence of frequency components in the output that are not present in the input signal. The nonlinear nature of the microphone can be approximated as the transfer function equation of the input and output signals having a nonlinear quadratic square term. When the input signal is not of a single frequency, the quadratic term in the nonlinear equation generates harmonics and cross-products of the input signal, which cause the output signal to contain new frequencies. Therefore, as shown in Figure 1b,c, the inherent nonlinear properties of the device can be utilized to achieve the effect of converting out-of-band inaccessible signals into in-band audible signals within the voice recording system by designing suitable ultrasonic input signals.

For two signals a and b with multiple frequency components, the nonlinear components $s_{nl}\left(t\right)$ generated by the system can be expressed as

$$\begin{array}{c}\hfill s_{nl}\left(t\right)=\sum _i\sum _j\beta A_{a,i}{A}_{b,j}cos\left(2\pi \left(f_{a,i}\pm f_{b,j}\right)t\right)\end{array}$$

(1)

where $\beta$ is the nonlinearity coefficient, $A_{a,i}$ represents the amplitudes of the i-th component in signal a, $A_{b,j}$ represents the j-th in signal b, $f_{a,i}$ and $f_{b,j}$ represent the i-th frequency component of a and the j-th frequency component of b, respectively.

2.2. Eavesdropping Models Analysis

Consider the scenario where the eavesdropper’s objective is to record private conversations in a closed and confidential setting, such as small conference rooms or private offices. Eavesdropping devices come in various forms, including ordinary smartphones or other covert recording devices. However, all of these devices have microphone recording systems at their core. The device for eavesdropping can be controlled by the eavesdropper either manually, by switching it on and off, or through software. Additionally, we assume that the eavesdropping device is carried by one of the interlocutors. It is probable that the device is hidden on the opponent’s body or at least in close proximity, such as being secretly placed under the table.

For the anti-eavesdropping scenario, it is assumed that the two parties are adults conversing at a normal volume in a relatively quiet environment. It is important to note that the general hearing range of the adult ear is below 18 kHz; therefore, 25 kHz ultrasonic sound should be inaudible to the interlocutor. As shown in Figure 2, users wishing to counter eavesdrop can preemptively perform speech analysis on the jamming device and activate the jamming device when a conversation is required, and the jamming device emits jamming noise to the approximate location of the possible eavesdropping device.

3. Time–Frequency Mosaic Interference

Based on the time–frequency characteristics of speech signals, this work proposes a TFM interference strategy. By jointly attacking the critical time–frequency features of speech signals through adaptive mosaic-style coverage of key speech components, this method achieves precise interference in both temporal and spectral domains to effectively disrupt Automatic Speech Recognition (ASR) systems while maintaining imperceptibility to human auditory perception. The algorithm operates in three stages:

• Real-time Speech Hybrid Feature Analysis;
• Dynamic Interference Signal Synthesis;
• Energy-Constrained Synthesis and Robustness Enhancement.

The general structure of the TFM interference system can be observed in Figure 3 and comprehensively features two core elements: the analysis module and the hardware module. The analysis segment functions primarily to formulate interference signals, while the hardware segment produces the necessary output signals based on the interference library advanced by the analysis module. The ultrasound transducer array subsequently transmits these signals.

3.1. Real-Time Speech Hybrid Feature Analysis

TFM first conducts a mixed feature analysis on the voice signal.

Continuous speech constitutes a nonstationary signal, necessitating its decomposition into quasi-stationary segments to facilitate localized feature analysis. For a speech signal $x\left(n\right)$ with sampling rate $f_s$, the k-th frame signal $x_k\left(n\right)$ is defined as

$$x_k\left(n\right)=x(n+kM),0\le n<N$$

(2)

Here, $N=0.02f_s$ (20 ms frame length), and $M=0.5N$ (50% overlap).

A Hamming window is applied to the framed quasi-stationary segments during the windowing process to suppress spectral leakage. The Hamming window function $w\left(n\right)$ is given in (3), and the Hamming-window signal $x_k^{win}\left(n\right)$ is given in (4).

$$w\left(n\right)=0.54-0.46cos\left({\displaystyle \frac{2\pi n}{N-1}}\right)$$

(3)

$$x_k^{win}\left(n\right)=x_k\left(n\right)·w\left(n\right)$$

(4)

Subsequently, the Short-Time Fourier Transform (STFT) is computed for each windowed frame to obtain time–frequency representations. For the k-th frame $X_k\left(m\right)$, its STFT is defined as

$$X_k\left(m\right)=\sum _{n=0}^{N-1}{x}_k^{win}\left(n\right)e^{-j2\pi mn/N},0\le m<N$$

(5)

Then, the formant structure is extracted using a 12th-order linear predictive coding (LPC) model, and its power spectral density formula $P_{\mathrm{LPC}}\left(f\right)$ is shown in (6) [26,27].

$$P_{\mathrm{LPC}}\left(f\right)={\displaystyle \frac{{\sigma }^2}{{\left|1-{\sum }_{i=1}^{12}{a}_i{e}^{-j2\pi fi}\right|}^2}}$$

(6)

Here, ${\sigma }^2$ is the noise variance, and $a_i$ represents the LPC coefficients optimized through autocorrelation minimization.

The dual-spectral analysis approach is adopted to facilitate hybrid feature extraction, where STFT provides coarse-grained time–frequency localization, while LPC precisely identifies ASR-dependent formant regions.

3.2. Dynamic Interference Signal Synthesis

Based on feature analysis, TFM will generate dynamic jamming signals.

The concentration of speech information and energy within critical frequency components (e.g., fundamental frequency, formants) necessitates localized feature targeting based on spectral analysis outcomes. A differentiated interference strategy is then developed to maximally disrupt ASR-discriminative features.

First, perform mel scale energy focusing by converting the STFT magnitudes into the mel scale using 40 triangular filters $H_m\left(k\right)$. The mel scale emulates the human ear’s nonlinear auditory characteristics and covers the primary speech frequency range (300 Hz–4 kHz). The top 5% of mel bands with the highest energy are selected for prioritized interference. This energy-concentrated band screening avoids resource waste from full-band interference. $H_m\left(k\right)$ is shown in (7) [28].

$$H_m\left(k\right)=\left\{\begin{array}{cc}0\hfill & k<f(m-1)\hfill \\ \\ {\displaystyle \frac{k-f(m-1)}{f\left(m\right)-f(m-1)}}\hfill & f(m-1)\le k\le f\left(m\right)\hfill \\ \\ {\displaystyle \frac{f(m+1)-k}{f(m+1)-f\left(m\right)}}\hfill & f\left(m\right)<k\le f(m+1)\hfill \\ \\ 0\hfill & k>f(m+1)\hfill \end{array}\right.$$

(7)

And mel scale $f_{\mathrm{mel}}$ is given by (8) [29,30]

$$f_{\mathrm{mel}}=2595{log}_{10}\left(1+{\displaystyle \frac{f}{700}}\right)$$

(8)

where

$$f=700({10}^{m/2595}-1)$$

(9)

Perform formant neighborhood detection to precisely disrupt the ASR-dependent formant features while reducing the interference bandwidth. The formant bandwidth typically ranges from 50 to 100 Hz. A Gaussian window is applied to cover their energy diffusion range. The first three formant frequencies F1, F2, and F3 are extracted from $P_{LPC}\left(f\right)$, defining the interference frequency bands $B_i$ as

$${\mathcal{B}}_i=[F_i-\Delta f,F_i+\Delta f],\Delta f=50\mathrm{Hz},i=1,2,3$$

(10)

Based on the selected interference bands, compute the psychoacoustic masking thresholds. The masking threshold $T_b$ shown in (11) is calculated within 24 critical bands of the Bark scale, where $E_b$ denotes the energy of band b, and $S_b$ represents the predefined masking offset specified in the ISO/IEC 11172-3 standard [31].

$$T_b=10{log}_{10}\left(E_b\right)-S_b$$

(11)

After obtaining the interference frequency bands and masking values, the interference signal can be generated. This work designs two strategies for interference signal generation: low-frequency phase-inverted noise interference and mid-to-high-frequency swept-frequency interference. The fundamental frequency and harmonic structure are core features of speech periodicity, and phase-inverted interference can disrupt the temporal waveform correlation. In the mid-to-high-frequency range, formant frequencies vary over time (e.g., vowel transitions). Swept-frequency signals provide dynamic coverage, while noise introduces randomness to prevent ASR systems from recovering speech content by tracking formant trajectories.

A phase-inverted signal $s_{\mathrm{low}}\left(t\right)$ shown in (12) is injected into the formant regions below 500 Hz to disrupt the fundamental frequency harmonic structure, where $n_{low}\left(t\right)$ represents band-limited Gaussian noise with a bandwidth $B=100$ Hz.

$$s_{\mathrm{low}}\left(t\right)=A·sin(2\pi F_{0}t+\pi )+n_{\mathrm{low}}\left(t\right)$$

(12)

A linear chirp signal $s_{\mathrm{sweep}}\left(t\right)$ shown in (13) is generated in the mid-to-high-frequency bands to mask time-varying formants, with $T=20$ ms, and $F_{end}=F_i+100$ Hz.

$$s_{\mathrm{sweep}}\left(t\right)=A·sin\left(2\pi \left(F_{i}t+{\displaystyle \frac{\beta }{2}}{t}^2\right)\right),\beta ={\displaystyle \frac{F_{\mathrm{end}}-F_i}{T}}$$

(13)

3.3. Energy-Constrained Synthesis and Robustness Enhancement

This section will introduce the energy constraint mechanism and robustness enhancement mechanism in the TFM so as to improve the energy utilization efficiency and robustness of the overall system.

Due to significant fluctuations in speech energy (e.g., abrupt increases in plosive sounds), real-time adjustment of interference intensity is required to prevent insufficient interference in low-energy frames and overload distortion in high-energy frames. Dynamic amplitude control enforces energy constraints by adjusting the interference amplitude $A_b$ in each sub-band $B_i$ to satisfy

$$A_b=min\left(\sqrt{{\eta }_b},0.8·\sqrt{E_b}\right)$$

(14)

Hardware experiments indicate that interference exceeding 80% of the original signal’s amplitude tends to cause temporal waveform distortion (e.g., clipping). Therefore, dual constraints are implemented to balance stealthiness and interference intensity. Here, ${\eta }_b$ ($T_b$) represents the masking threshold, and the coefficient 0.8 is experimentally optimized to ensure signal stability.

The time-domain interference signal $s\left(n\right)$ shown in (15) is reconstructed through inverse STFT and the Overlap-Add (OLA) method. Each $s_k$ corresponds to the slow (low-frequency) and sweep (mid–high frequency) components of different frequency bands. The OLA method ensures inter-frame signal continuity, prevents “click” artifacts, enhances the naturalness of the interference signal, and reduces human auditory sensitivity.

$$s\left(n\right)=\sum _k{s}_k(n-kM)·w(n-kM)$$

(15)

After completing the interference signal synthesis, robustness enhancement is primarily achieved through voiced/unvoiced state-adaptive control and formant tracking smoothing, ensuring sustained effectiveness of interference signals against dynamic speech.

Firstly, voiced/unvoiced detection is performed to distinguish speech frames as voiced sounds (periodic vocal fold vibrations) or unvoiced sounds (random noise characteristics), enabling dynamic adjustment of interference strategies. Unvoiced sounds (e.g., /s/, /t/) exhibit low-energy and high-frequency concentration, necessitating a switch to broad-band noise interference to avoid failure of narrow-band interference designed for voiced sounds in unvoiced segments.

Voiced/unvoiced detection utilizes short-term energy E and zero-crossing rate $ZCR$ to classify speech frames as follows:

$$E={\displaystyle \frac{1}{N}}\sum _{n=0}^{N-1}{x}_k^2\left(n\right)$$

(16)

$$ZCR={\displaystyle \frac{1}{2N}}\sum _{n=1}^{N-1}|\mathrm{sgn}\left(x_k\left(n\right)\right)-\mathrm{sgn}\left(x_k(n-1)\right)|$$

(17)

The specific decision strategy is

$$Voicedframe\leftrightarrow E>{\theta }_E,ZCR<{\theta }_Z$$

(18)

where ${\theta }_E=0.3max\left(E\right),{\theta }_Z=0.1$.

Following detection, dynamic strategy adjustments are implemented.

For voiced frames: Apply low-frequency formant-enhanced interference targeting the fundamental frequency F0 and harmonic components by increasing the phase-inverted noise amplitude Alow, simultaneously, to suppress high-frequency interference to reduce energy allocation of swept-frequency signals in mid–high frequencies and avoid spectral overload.

For unvoiced frames: Inject mid-to-high-frequency noise in fricative/plosive regions (e.g., 4–8 kHz) by augmenting broad-band noise interference. Concurrently, attenuate low-frequency interference to avoid energy waste in base-band unvoiced segments lacking fundamental frequency components.

Moreover, formant frequencies vary smoothly between adjacent frames (e.g., during sustained vowel segments), requiring smoothing processing to avoid abrupt jumps in interference signals and enhance the obfuscation effect. Specifically, exponential smoothing is applied to the formant frequencies to suppress inter-frame abrupt changes in formant frequencies, thereby improving the continuity of interference signals.

$${\widehat{F}}_i^{\left(k\right)}=\alpha {\widehat{F}}_i^{(k-1)}+(1-\alpha )F_i^{\left(k\right)},\alpha =0.9$$

(19)

The formants $F_i\left(k\right)$ in (19) extracted from the k-th frame are smoothed with $\alpha$ = 0.9, achieving a relative balance between tracking speed and stability. This prevents discontinuities in the interference band $B_i$ shown in (20) caused by abrupt formant transitions, ensuring gradual variation of the frequency modulation (FM) slope $\beta$ in the swept-frequency signal $s_{sweep}\left(t\right)$.

$${\mathcal{B}}_i^{\left(k\right)}=[{\widehat{F}}_i^{\left(k\right)}-\Delta f,{\widehat{F}}_i^{\left(k\right)}+\Delta f]$$

(20)

Based on the smoothed formant frequencies ${\widehat{F}}_i$, recalculate the interference frequency bands and adjust the swept-frequency signal parameters $F_i$ and $F_{end}$ to ensure that the interference consistently covers the smoothed target frequency bands.

Robustness enhancement achieves dynamic interference optimization through two core technologies: These include voiced/unvoiced state-adaptive control and formant tracking smoothing. By dynamically allocating interference band energy based on voiced/unvoiced classification—for instance, focusing on high-frequency interference instead of low-frequency formant coverage when plosive sounds like /t/ occur—stealthiness is improved. Simultaneously, exponential smoothing filtering eliminates inter-frame jumps in formant tracking, ensuring time-frequency continuity of interference signals. Additionally, the voiced/unvoiced adaptive strategy reduces the total energy of interference signals by approximately 18%, optimizing energy efficiency while maintaining attack effectiveness. The synergistic interaction of these two technologies enables the synthesized interference signals to adapt to dynamic speech characteristics, resist the short-term spectral recovery capabilities of ASR systems, and ultimately enhance the robustness and stealthiness of adversarial attacks.

4. Evaluation and Discussion

In this section, a physical platform was built based on the proposed TFM, and the TFM was evaluated in terms of energy utilization efficiency, jamming effect, robustness, and performance in real environments to verify its performance.

4.1. Experimental Set Up

The hardware circuit implementation of the Time–Frequency Mosaic interference system is displayed in Figure 4. The system mainly includes the power module, the interference signal generation and processing module, and the ultrasonic transmission array. The interference signal generation and processing module consists of FPGA ZYNQ7020 (Produced by MicroPhase, Shanghai, China), digital-to-analog converter DAC8554 (Produced by Mindesigner, Changsha, China), low-pass filters, and two power amplifiers. For optimum interference effectiveness, the ultrasonic transmission array has been fitted with probes that are differentially distributed. Each probe transmits two distinctive Time–Frequency Mosaic interference signals.

Figure 5 is a simplified schematic of the experiment. Testers for the experiment were preregistered on the TFM system. During the experiment, the ultrasound array was directed toward the recording device. First, a recording was performed without interference. Then, the ultrasound jamming device was turned on, and the recording device was jammed with white noise and the TFM, respectively. We used a Redmi Note 11T Pro smartphone as the recording device.

Evaluating the interference effect of speech interference algorithms typically involves the use of a series of objective and subjective assessment metrics to ensure the performance and practical usability of the interference algorithms. The Mean Opinion Score (MOS) method is a subjective evaluation approach involving experts (listeners) who rate speech. Testing the interfered speech with a Speech-To-Text System (STT) and obtaining a transcribed text, followed by a comparison with the correct text, allows for the calculation of the Word Error Rate (WER) [32]. To reduce the subjectivity of the evaluation, this paper employed the STT testing method, using the WER to indicate the quality of interference. A higher WER suggests poorer recognition by the STT system, which in turn indicates a better interference effect.

4.2. Interference Energy Utilization Efficiency Analysis

The recording device was positioned at a consistent distance of 20 cm from the interfering device. The human voice was recorded both with and without interference from white noise and the TFM. The speaker and spoken text remained constant throughout the recordings. The resulting audio files were exported and analyzed.

To compare the effects of interference between TFM noise and white noise on the same sound, we plotted the time domain and time–frequency domain waveforms of the pure sound, the sound interfered with by white noise, and the sound interfered with by the TFM in Figure 6. The power of white noise is uniformly dispersed throughout the time–frequency domain, whereas TFM noise can be targeted to cover the specific composition of the human voice to achieve a better masking effect.

Meanwhile, it can be observed from Figure 6 that the energy of the TFM appears to be higher than that of white noise at the same distance. To confirm the energy transfer efficiency of the interference signal, we recorded the white noise interference signal and the TFM interference signal at the same distance. We calculated the SNR of the two recorded interference noises with the same segment of pure human voice signal distribution. We then obtained the SNR under different distances, as shown in Figure 7. The distance refers to the distance from the ultrasonic array to the sound-receiving device. Figure 7 shows that the noise generated by the TFM interference signal is higher than that of the white noise interference signal in the low-distance case by 2 dB. In the long-distance case, the two SNR curves rapidly increase and converge due to the limited power of the transmitting equipment and ultrasonic propagation in the air with loss, resulting in a rapid reduction of the noise generated by the interference signal. Therefore, the TFM jamming signal requires less power than white noise to achieve the same signal-to-noise ratio, demonstrating the superior energy efficiency of the TFM jamming system.

4.3. Interference Effect Analysis

To comprehensively evaluate the interference effect, we conducted tests using four different STT systems: two major commercial STT systems, Tencent STT and Xunfei STT; a free STT system, Diyun STT; and a commonly used open-source STT system, DeepSpeech [33].

Tencent STT boasts core strengths in high real-time performance and engineering capabilities. Its low-latency feature makes it the first choice for scenarios such as real-time subtitles in Tencent Meetings and voice input in WeChat. iFLYTEK Hear (Xunfei STT), developed by iFLYTEK, excels in professional field recognition and becomes a benchmark solution in the judicial and medical industries. Diyun AI Speech (Diyun STT), as a representative of embedded speech recognition, is a lightweight model that is compatible with Internet of Things (IoT) chips, meeting the stringent requirements of industrial inspection and smart home with ultra-low-power consumption. DeepSpeech, as an open-source project by Mozilla, supports cross-language recognition of over 80 languages, providing a a flexible open-source foundation for academic research and multilingual customization.

Note that the TFM produces higher power noise at the same distance. To fairly compare the interference effect of the noise produced by the interfering signal, we extracted the noise and adjusted the power ratio of the two types of noise to one. Then, we mixed the pure speech signals based on Mandarin with the noise signals in the digital domain and fed them into the STT system for comparison.

A comparison was performed between TFM interference and the jamming method in Backdoor [17], where the noise parameters replicated Backdoor’s design: band-limited white noise (0–12 kHz) was amplitude-modulated onto a 40 kHz ultrasonic carrier to exploit microphone nonlinearity. Under various SNRs, the recordings with TFM interference and white noise interference were input into the STT systems. The result is shown in Figure 8.

Figure 8 illustrates the WER values of each STT system within the SNR range of [−5, 5] dB. Among them, the red line represents the change in WER of the proposed STT with SNR, the white line represents the change in WER of white noise with SNR, and the black line represents the change in WER without any interference with SNR.

Figure 8 shows that our TFM interference design performed better than white noise interference, particularly when the SNR was below 0 dB. When white noise was present at an SNR of −5 dB, Tencent STT and Xunfei STT produced a much lower WERs compared to Diyun STT and Deep Speech. Additionally, the interference effects of other signal-to-noise ratios of white noise were weaker for Tencent STT and Xunfei STT. This suggests that Tencent STT and Xunfei STT have anti-interference designs for white noise, which greatly attenuate the white noise interference effect.

At an SNR of 0 dB, the average WER for TFM interference was 20% higher than for white noise. For the two main commercial STT systems, there was a greater difference in interference performance between TFM interference and white noise interference at low SNRs than in open-source STT systems. This variation can be associated with the use of noise reduction algorithms in commercial STT systems, showing the weaker suppression of TFM interference by current commercial noise reduction methods. This illustrates the resilience of TFM interference.

It should be noted that Diyun STT had a similar white noise interference effect to TFM at a low SNR. However, its WER dropped rapidly after the SNR was above 0 dB, indicating that the white noise mainly suppressed Diyun STT due to the lack of internal speech noise reduction and speech recovery extraction capabilities in this STT system.

In order to better illustrate the effect of TFM, we used the same equipment to transmit different interference signals, namely, noise in TFM, Backdoor [17], and noise in Patronus [34], and tested the actual interference effect in 20 cm, 40 cm, and 60 cm. The WER results of the three methods at different distances are shown in

Table 1. Average recognition results at different distances.

Distance (cm)	Recognition Results: Avg WER (%)
	Noise in Patronus [34]	Noise in Backdoor [17]	TFM
20	62.1	41.7	89.3
40	35.2	25.9	60.3
60	15.3	5.6	39.4

.

From Table 1, it can be found that TFM had the best interference effect, while white noise was the worst. At a distance of 20 cm, TFM was 43.8% better than Patronus; at a distance of 60 cm, TFM was 157.5% better than Patronus, which proves that the interference effect of TFM attenuates the least with distance. Meanwhile, TFM can generate higher-power interference in the interfered devices under the condition that the transmitting devices have the same power. This reflects that TFM interference has high energy utilization efficiency.

4.4. Robustness Evaluation of Speech Enhancement

In addition to testing the speech interference effect of the TFM system, the robustness of the TFM method needs to be evaluated. Generally speaking, the anti-eavesdropping interference signal may be denoised by the hardware filtering built-in in the recording equipment and the speech enhancement algorithm of the internal speech processing chip, which may weaken the effect of the anti-eavesdropping interference system.

Given that the 0–8 kHz filter has a higher power to filter out white noise, and considering the signal-to-noise ratio experiments conducted in the previous section with the same-distance recording device, the power of the white noise delivered to the smart device is relatively low compared to the TFM. As previously done, we adjusted the noise generated by the two interferences to the same power. Then, we mixed it with pure speech in the digital domain to obtain the same power of the speech signal masked by the different interfering signals.

The SOTA algorithm [35] was used to enhance the mixed speech signal, which was then input into the STT system. Figure 9 displays the recognition effect of the speech-enhanced interfering signals by each STT system with different signal-to-noise ratios.

The results indicate that the recognition rates of Xunfei STT and Deepspeech STT for speech-enhanced mixed speech remained unchanged, and the effect of speech enhancement was limited. The recognition rate of Tencent STT for white noise remained unchanged.

However, the WER for TFM increased. This may be due to a conflict between the speech enhancement algorithm used in Tencent STT and the TFM interference after the SOTA algorithm, resulting in an amplification of the interference effect. Xunfei STT, Deepspeech, and Tecent STT have built-in speech enhancement algorithms, which means that the system’s speech recognition accuracy remains relatively unchanged even after being enhanced by the SOTA algorithm. However, Diyun STT, which lacks a built-in speech enhancement algorithm, experienced a significant change in recognition accuracy.

Figure 9 demonstrates that Diyun STT produced a significant improvement in the speech recognition rate after speech enhancement, reducing the WER under white noise by an average of 25%. This suggests that the speech enhancement algorithm greatly weakens the effect of white noise interference on Diyun STT, indicating poor robustness of the white noise interference signal. Compared to white noise interference, the error rate of text recognition for TFM-interfered speech was only reduced by about 5% on average after speech enhancement by the SOTA algorithm. Despite this improvement, the overall interference efficiency of TFM interference on speech remained high, which makes the Diyun STT robust to speech enhancement. Additionally, the noise generated by the TFM interference signal showed robustness to the SOTA speech enhancement algorithm.

4.5. Testing in Real-World Scenarios

Previous experiments were conducted to jam a single speaker. To verify the universality of the jamming system, multiple speaker scenarios were considered. Five different speakers were selected and allowed to have a conversation while following the same text. The audio was recorded and jammed using an ultrasonic eavesdropping prevention system. After adjusting the SNR, the audio was fed into four STT systems. The results presented in Figure 10 indicate that the interference effect of white noise remained consistent in the case of multiple speakers. However, the interference effect of TFM decreased while still maintaining a good level of interference. This is due to the fact that the speech of different individuals has varying acoustic characteristics, such as fundamental frequency and resonance peak frequency. Additionally, the interference band of the TFM signals was limited to achieve exceptional time–frequency domain coverage, which led to a degradation of the interference effect.

In the previous multispeaker test, our experimenters were all male. To ensure objectivity, future tests will include experimenters of different genders. In order to better evaluate the performance of TFM interference during real scenarios, we selected Xunfei STT and Tecent STT. These commercial STT systems have better stability and universality, making them closer to real scenarios. Keeping the SNR at 0 dB, we tested the interference effect on three groups: single male, single female, and mixed. Each group consisted of five interlocutors who followed the same conversation script. Interference was introduced using white noise and TFM, and the average speech recognition results are shown in Figure 11. From Figure 11, we can find that the interference effect of white noise in the different genders basically did not change much, and the interference effect on single women was relatively weak. In terms of TFM interference, there was a more pronounced difference in the interference effect between genders, with the best result being 65.2% for women and the worst being 57.4% for mixed genders.

Regarding the experimental results above, it is suggested that the concentration of frequency in women’s voices and their faster speech speed may have contributed to the observed differences. Women’s voices have a relatively high resonance peak frequency and a narrower bandwidth, while men’s voices have lower fundamental and resonance peak frequencies, smaller spacing between overtones, and a larger bandwidth. Additionally, white noise may have a relatively poor effect on the interference of relatively high-frequency sounds with narrower bandwidths. TFM interference is designed by obtaining certain sound information in advance, making it specialized. Female voices have higher frequencies, larger spacing, and fewer main harmonics, simplifying the design of TFM interference signals and resulting in a stronger interference effect. Male voices, on the other hand, have a wider frequency bandwidth and more harmonics, making it difficult for TFM jamming to completely cover them, resulting in a weaker jamming effect. When both males and females are present in a group, the frequency at which they experience jamming increases, resulting in a more severe jamming effect.

5. Conclusions

This paper employed time–frequency analysis methods and proposed an efficient anti-eavesdropping system based on TFM interference, with its core contributions including the proposal of a novel time–frequency joint interference strategy that performs interference in both the time domain and frequency domain to enhance the system’s interference capability, dynamic ultrasonic encoding that switches interference strategies according to voice states to boost the system’s robustness, and hardware platform design and experiments where we have developed a low-power hardware co-design platform and designed experiments from aspects such as energy efficiency, the interference effect, robustness evaluation, and real-scenario testing, fully demonstrating its excellent interference effect, robustness, and energy efficiency.

Experimental findings showcase that TFM interference significantly disrupted STT systems at an SNR of 0 dB, yielding WERs exceeding 40%. This underscores the robustness against noise reduction algorithms in low-SNR scenarios. Moreover, TFM exhibited markedly superior interference efficiency compared to white noise in high-SNR conditions with open-source STT systems, highlighting its outstanding energy utilization efficiency. This study emphasizes TFM’s potential as a practical audio security solution for safeguarding privacy and information in an inter-connected world.