Desynchronization Resilient Audio Watermarking Based on Adaptive Energy Modulation

Abstract

With the rapid proliferation of social media platforms and user-generated content, audio data is frequently shared, remixed, and redistributed online. This raises urgent needs for copyright protection and traceability to safeguard the integrity and ownership of such content. Resilience to desynchronization attacks remains a significant challenge in audio watermarking. Most existing techniques face a trade-off between embedding capacity, robustness, and imperceptibility, making it difficult to meet all three requirements effectively in real-world applications. To address this issue, we propose an improved patchwork-based audio watermarking algorithm. Each audio frame is divided into two non-overlapping segments, from which mid-frequency energy features are extracted and modulated for watermark embedding. A linearly decreasing buffer compensation mechanism balances imperceptibility and robustness. Additionally, an optimization algorithm is incorporated to enhance watermark transparency while maintaining resistance to desynchronization attacks. During watermark extraction, each bit of the watermark is recovered by analyzing the intra-frame energy relationships. Furthermore, we provide a theoretical analysis demonstrating that the proposed method is robust against various types of attack. Extensive experimental results demonstrate that the proposed scheme ensures high audio quality, strong robustness against desynchronization attacks, and a higher embedding capacity than existing methods.

1. Introduction

In the era of social networks and online media platforms, audio content is widely disseminated through short videos, live streaming, and podcasts. Protecting the copyright of such content and enabling reliable traceability has become a critical requirement for ensuring data security and rightful ownership in user-generated media. In recent years, the rapid development of internet technology has significantly facilitated the dissemination of digital content, offering unprecedented access to information. However, this convenience has also heightened concerns over copyright protection, particularly for traditional media such as audio. The widespread adoption of large language models (LLMs) has further compounded these challenges, driving increased demand for content traceability and the advancement of specialized watermarking techniques [1]. Within this context, audio watermarking remains a crucial research focus, offering effective solutions for copyright verification and content tracking through the imperceptible embedding of information into audio signals. Nonetheless, the growing accessibility of audio editing tools has made operations such as playback speed adjustment and timbre modification increasingly common, which can severely impair the accuracy of watermark extraction. As a result, enhancing the robustness of audio watermarking against time-scale modification (TSM) and pitch-scale modification (PSM) has become a pressing challenge in current research.

Embedding and extracting audio watermarks present greater challenges compared to their image counterparts, primarily due to the heightened sensitivity of the human auditory system (HAS) relative to the human visual system (HVS). The HVS exhibits a degree of insensitivity to local variations in image details, allowing watermarks to be imperceptibly embedded in high-frequency regions. In contrast, the HAS is acutely sensitive to minute changes in amplitude, frequency, and phase, where even slight distortions can degrade perceptual audio quality. Furthermore, image watermarks are embedded in a two-dimensional space characterized by high spatial redundancy, a property widely exploited by various transform-domain techniques such as those based on DWT, SVD, and chaotic maps [2,3,4]. In contrast, audio signals are one-dimensional temporal sequences with lower redundancy and strong temporal-frequency correlations [5].

Over the past decade, numerous audio watermarking algorithms have been developed, which can be broadly categorized into time-domain and frequency-domain approaches [6,7]. Time-domain techniques, such as amplitude modulation, histogram-based methods, and echo hiding, embed watermarks by directly modifying the temporal characteristics of the audio signal [8,9,10]. In contrast, frequency-domain methods employ transformations like the discrete Fourier transform (DFT) [11,12], discrete cosine transform (DCT) [13,14], and discrete wavelet transform (DWT) [15,16,17] to embed watermark information by adjusting specific transform coefficients. Furthermore, auxiliary techniques including spread spectrum [18,19,20], patchwork schemes [21,22,23], quantized index modulation (QIM) [24,25,26], and singular value decomposition (SVD) [27,28,29] are frequently integrated to improve the robustness of watermarking systems. These methods aim to embed watermark data into the audio signal in an imperceptible way, while ensuring reliable extraction under various conditions.

Desynchronization attacks have long posed a significant challenge in audio watermarking by disrupting the alignment between the embedded watermark and the host signal, which greatly complicates accurate extraction. To address this issue, a variety of synchronization strategies have been explored, including exhaustive search, explicit synchronization, implicit synchronization, and invariant watermarking [30,31,32]. Exhaustive search techniques are straightforward in principle but often demand high computational cost, making them unsuitable for real-time scenarios. Explicit synchronization involves inserting additional markers or sequences into the audio signal to guide alignment during extraction. Although effective, this method typically reduces payload capacity and becomes vulnerable to signal processing operations that distort or remove the markers. In contrast, implicit synchronization determines alignment points by analyzing inherent audio features, such as amplitude fluctuations, energy variations, or changes in signal envelope. These features act as intrinsic landmarks for alignment, enabling robust synchronization without additional markers.

Several representative methods have emerged from these strategies. The technique proposed in ref. [33] employs a patchwork-based method, embedding watermark bits in the DCT domain and synchronization codes in the logarithmic DCT (LDCT) domain. Temporal distortions are compensated for by estimating scaling factors from the LDCT domain, leading to strong robustness against PSM, TSM, and jitter. However, the increased complexity of LDCT processing and the separation between synchronization and watermarking domains limit both efficiency and embedding capacity. To improve upon local-feature-based schemes, Jiang et al. [15] introduce a global-characteristic-based method that applies adaptive frame segmentation and embeds watermark data using statistical features from the wavelet domain. Indirect synchronization is achieved through a logical frame index, yielding solid robustness to 30% TSM and 1/10 jitter, though the method remains less effective against amplitude variations. Another noteworthy contribution is the audio-lossless robust watermarking (ALRW) framework in ref. [34], which embeds watermark information into the histogram structure of the low-frequency DWT subband. The distortion introduced during embedding is fully reversible in the absence of attacks, ensuring lossless recovery. This method offers excellent resistance to desynchronization attacks such as cropping, TSM, and jitter, yet its limited embedding capacity restricts broader applicability. A further improvement is presented in ref. [35], which combines explicit synchronization with invariant feature embedding. Watermark and synchronization codes are embedded into frequency-domain logarithmic mean (FLDM) features using a patchwork structure. This enables frame realignment when temporal distortions occur. Despite improved robustness to misalignment, the method offers limited performance under amplitude scaling and suffers from constrained capacity. Recently, some research has focused on constructing watermarking schemes in the time domain to resist desynchronization attacks. Among them, the scheme proposed by Zhao et al. [36] based on the Time Domain Fragment Energy Relationship is a representative work. This method achieves effective resistance against TSM attacks by modifying the energy ratio between adjacent audio fragments. However, directly modifying full-band signal samples in the time domain poses a significant challenge to maintaining the imperceptibility of the watermark.

Despite significant progress in audio watermarking, achieving an effective balance among robustness, imperceptibility, and embedding capacity remains a persistent challenge. Many existing methods perform poorly under desynchronization attacks such as jitter, TSM, and PSM, or lack stability when subjected to common signal processing operations. To address these limitations, this paper proposes a novel audio watermarking algorithm that combines robustness with efficiency. The method first segments the audio into frames and equally divides each frame into two parts. Each part is processed using DCT, and mid-frequency coefficients are selected for embedding. Watermark bits are embedded based on the energy comparison between the two segments. To further improve resistance to TSM and PSM while maintaining audio quality, an optimization strategy and a linearly decreasing buffer compensation mechanism are introduced to reduce distortion caused by energy adjustment. During extraction, the decoder reverses the embedding procedure by analyzing the inter-segment energy relationships to accurately recover the watermark. Theoretical analysis demonstrates the method’s reliability against a variety of attacks, and extensive experiments confirm its ability to maintain a favorable trade-off among audio fidelity, embedding rate, and robustness, showing strong practical applicability and adaptability.

The main contributions of this paper are as follows:

• We propose a linearly decreasing buffer compensation mechanism that enhances the robustness of audio watermarking against desynchronization attacks, such as TSM and PSM, while reducing the distortion caused by energy adjustment.
• We theoretically analyze the resilience of the proposed method against both common signal processing and desynchronization attacks, and experimentally validate its effectiveness through comprehensive evaluations.
• Experimental results show that the proposed method maintains stable performance under common signal processing operations, with the highest BER being only 4.01%, and exhibits enhanced robustness against desynchronization attacks such as jitter, TSM, and PSM, with the BER never exceeding 12.31% under all tested conditions.

2. Proposed Methods

Figure 1 shows the general framework of the proposed method, which mainly consists of two modules that are energy feature extraction and watermark embedding. In the following, we provide the details and give an analysis in terms of parameter optimization and robustness.

2.1. Energy Feature Extraction

Given the original audio signal $\mathbf{x}=\{x_1,x_2,\dots ,x_L\}$, where L is the total number of samples, we adopt a patchwork-based segmentation strategy to divide $\mathbf{x}$ into N non-overlapping frames of equal length M such that $L=NM$. For simplicity, we assume that $M\mid L$; otherwise, the remaining samples at the end are discarded. Each frame can be denoted as ${\mathbf{x}}_i=\{x_{(i-1)M+1},x_{(i-1)M+2},\dots ,x_{iM}\}$, where $i\in [1,N]$. Each frame ${\mathbf{x}}_i$ is further divided into two disjoint segments of equal length, i.e., ${\mathbf{x}}_i=[{\mathbf{x}}_{i,1},{\mathbf{x}}_{i,2}]$, where ${\mathbf{x}}_{i,j}=\{x_{(i-1)M+(j-1)S+1},x_{(i-1)M+(j-1)S+2},\dots ,x_{(i-1)M+jS}\}$, $j\in \{1,2\}$, and $S=M/2$. Here we also assume $S\mid M$. Thereafter, each segment ${\mathbf{x}}_{i,j}$ is transformed using the DCT to obtain its frequency representation ${\mathbf{X}}_{i,j}=\{X_{i,j}\left(1\right),X_{i,j}\left(2\right),\dots ,X_{i,j}\left(S\right)\}$.

Owing to the energy compaction property of the DCT, most signal energy is concentrated in the low-frequency components. Modifying these components may degrade the perceived audio quality, whereas high-frequency components are more susceptible to signal processing operations such as compression, filtering, and noise, which may lead to the loss of embedded watermark information. To achieve a balance between robustness and perceived audio quality, the watermark is embedded into the mid-frequency band. Mathematically, the mid-frequency range can be defined by $[f_{\mathrm{low}},f_{\mathrm{high}}]$, where $f_{\mathrm{low}}$ and $f_{\mathrm{high}}$ are the lower and upper bounds, respectively. These are computed as $f_{\mathrm{low}}={\eta }_{1}S$ and $f_{\mathrm{high}}={\eta }_{2}S$, where ${\eta }_1$ and ${\eta }_2$ are proportional parameters. The selected mid-frequency coefficients can be expressed as

$${\mathbf{X}}_{i,j}^{\mathrm{mid}}=\{X_{i,j}\left(f_{\mathrm{low}}\right),X_{i,j}(f_{\mathrm{low}}+1),\dots ,X_{i,j}\left(f_{\mathrm{high}}\right)\}.$$

(1)

The energy feature of the segment is then computed over the mid-frequency range as

$$E_{i,j}=\sum _{k=f_{\mathrm{low}}}^{f_{\mathrm{high}}}{X}_{i,j}^2\left(k\right).$$

(2)

TSM and PSM attacks can be approximately seen as linear scaling operations along the frequency axis, during which the signal’s energy features also change linearly. This characteristic gives patchwork-based watermarking methods a degree of robustness against scaling attacks. In addition, the temporal and spectral stability of energy features, as well as their resistance to noise, further strengthens the overall robustness of watermarking schemes.

2.2. Watermark Embedding

The proposed method builds upon a patchwork-based strategy and incorporates further improvements inspired by Ref. [37]. In this method, watermark information is embedded by adjusting the energy ratio between two adjacent segments within the mid-frequency band of each frame. Let $\mathbf{w}=\{w_1,w_2,\dots ,w_N\}\in {\{0,1\}}^N$ represent the binary watermark sequence. The general embedding rule is defined as follows:

$$\left\{\begin{array}{cc}{E}_{i,1}^{\prime }\ge \alpha E_{i,2}^{\prime },\hfill & \mathrm{if}{w}_i=0,\hfill \\ E_{i,2}^{\prime }\ge \alpha E_{i,1}^{\prime },\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(3)

In this formulation, $\alpha$ is the embedding strength factor, and $E_{i,1}^{\prime }$ and $E_{i,2}^{\prime }$ denote the energies of the two modified segments. By creating a buffer zone, $\alpha$ improves extraction reliability. However, a small $\alpha$ may reduce robustness, while a large value may cause audible distortion. The energy comparison is conducted based on the magnitude of the DCT coefficients in the frequency domain. The detailed embedding procedure is described as follows:

Let $E_{i,1}$ and $E_{i,2}$ denote the energies of two non-overlapping segments within a frame. In case $w_i=0$, the watermark bit is embedded by adaptively adjusting the energy relationship between the two segments. If the original energy values satisfy $E_{i,1}>\alpha E_{i,2}$, no modification is applied. Otherwise, the segment energies are adjusted to meet Equation (4) using two scaling factors $\beta$ and $\gamma$, i.e.,

$$\left\{\begin{array}{c}{E}_{i,1}^{\prime }={\beta }^2{E}_{i,1},\hfill \\ E_{i,2}^{\prime }={\gamma }^2{E}_{i,2}.\hfill \end{array}\right.$$

(4)

To minimize the auditory distortion caused by watermark embedding, $\beta$ and $\gamma$ are optimized and defined as follows:

$$\left\{\begin{array}{c}\beta ={\displaystyle \frac{\alpha E_{i,2}+E_{i,2}\sqrt{\alpha E_{i,1}{E}_{i,2}}}{\alpha E_{i,2}+E_{i,1}{E}_{i,2}}},\hfill \\ \gamma ={\displaystyle \frac{\sqrt{\alpha E_{i,1}{E}_{i,2}}+E_{i,1}{E}_{i,2}}{\alpha E_{i,2}+E_{i,1}{E}_{i,2}}}.\hfill \end{array}\right.$$

(5)

The details of the optimization strategy are described in Section 2.4.

In case $w_i=1$, the energy adjustment follows a similar procedure. If the condition $E_{i,2}>\alpha E_{i,1}$ is satisfied, the energy values remain unchanged. Otherwise, the energies are modified as

$$\left\{\begin{array}{c}{E}_{i,1}^{\prime }={\beta }^2{E}_{i,1},\hfill \\ E_{i,2}^{\prime }={\gamma }^2{E}_{i,2},\hfill \end{array}\right.$$

(6)

where $\beta$ and $\gamma$ are given by

$$\left\{\begin{array}{c}\beta ={\displaystyle \frac{E_{i,2}+E_{i,2}\sqrt{\alpha E_{i,1}{E}_{i,2}}}{E_{i,2}+\alpha E_{i,1}{E}_{i,2}}},\hfill \\ \gamma ={\displaystyle \frac{\sqrt{\alpha E_{i,1}{E}_{i,2}}+\alpha E_{i,1}{E}_{i,2}}{E_{i,2}+\alpha E_{i,1}{E}_{i,2}}}.\hfill \end{array}\right.$$

(7)

To enhance the robustness against TSM and PSM attacks, a buffer compensation mechanism is introduced to ensure a smooth transition between the watermarked and non-watermarked frequency regions. Since the scaling factors $\beta$ and $\gamma$ only affect the frequency range $[f_{\mathrm{low}},f_{\mathrm{high}}]$, setting a large embedding strength factor $\alpha$ may lead to significant differences between the modified coefficients within this band and the unmodified coefficients outside it. Some attacks may cause the feature extraction process to involve coefficients beyond the embedding band, thereby introducing severe distortion to the extracted features. To address this issue, we define a buffer zone with width $\xi$ on both sides of the embedding band and apply a gradual scaling adjustment to the coefficients within this zone, achieving a smooth transition between the embedded and non-embedded regions. The scaling process is defined as

$$X_{i,j}^{\prime }\left(k\right)=\left\{\begin{array}{cc}\left[1+\left(\sqrt{{\displaystyle \frac{E_{i,j}^{\prime }}{E_{i,j}}}}-1\right)·{\displaystyle \frac{k-(f_{\mathrm{low}}-\xi )}{\xi }}\right]·X_{i,j}\left(k\right),\hfill & k\in [f_{\mathrm{low}}-\xi ,f_{\mathrm{low}}],\hfill \\ \sqrt{{\displaystyle \frac{E_{i,j}^{\prime }}{E_{i,j}}}}·X_{i,j}\left(k\right),\hfill & k\in [f_{\mathrm{low}},f_{\mathrm{high}}],\hfill \\ \left[1+\left(\sqrt{{\displaystyle \frac{E_{i,j}^{\prime }}{E_{i,j}}}}-1\right)·\left(1-{\displaystyle \frac{k-f_{\mathrm{high}}}{\xi }}\right)\right]·X_{i,j}\left(k\right),\hfill & k\in [f_{\mathrm{high}},f_{\mathrm{high}}+\xi ].\hfill \end{array}\right.$$

(8)

Here, $X_{i,j}^{\prime }\left(k\right)$ denotes the watermarked coefficient at frequency index k after energy adjustment. It should be noted that the frequency coefficients within the $[f_{\mathrm{low}},f_{\mathrm{high}}]$ band strictly follow the predefined embedding rules and represent the region of coefficients used for energy extraction during the watermark detection process. In summary, the buffer compensation mechanism ensures a smooth energy transition at the boundaries of the embedding region, thereby improving the robustness of the proposed method under frequency distortions.

After the above adjustments, the modified DCT coefficients are concatenated with the unaltered coefficients to reconstruct the complete frame. Applying the inverse DCT to all frames yields the final watermarked audio signal, denoted by ${\mathbf{x}}^{\prime }$.

2.3. Watermark Extraction

Let ${\mathbf{x}}^{″}$ denote the received audio at the decoder, which may be an attacked version of the original watermarked signal ${\mathbf{x}}^{\prime }$. Clearly, if no attack occurs, we have ${\mathbf{x}}^{″}={\mathbf{x}}^{\prime }$. Without loss of generality, suppose we are extracting a watermark bit $w_i^{″}$ from the segment ${\mathbf{x}}_i^{″}$. According to the feature extraction procedure in Section 2.1, the energy features $E_{i,1}^{″}$ and $E_{i,2}^{″}$ are first calculated within the frequency range $[f_{\mathrm{low}},f_{\mathrm{high}}]$. The watermark bit $w_i^{″}$ is then extracted as

$$w_i^{″}=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}{E}_{i,1}^{″}\ge E_{i,2}^{″},\hfill \\ 1,\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(9)

Based on Equation (9), the full watermark sequence ${\mathbf{w}}^{″}$ can be extracted from ${\mathbf{x}}^{″}$. Ideally, we expect that the extracted watermark ${\mathbf{w}}^{″}$ equals the originally embedded watermark $\mathbf{w}$.

2.4. Parameter Optimization

In this section, we discuss our optimization strategy for selecting the parameters $\beta$ and $\gamma$, with the objective of minimizing the distortion introduced to the DCT coefficients while preserving the watermark embedding conditions. Without loss of generality, we assume the embedded watermark bit is $w_i=0$; the case for $w_i=1$ can be addressed similarly. To ensure that the embedding rule remains unchanged, we aim to satisfy the following condition:

$${\displaystyle \frac{E_{i,1}^{\prime }}{E_{i,2}^{\prime }}}=\alpha .$$

(10)

Using the energy modulation rules, this can be rewritten as

$${\beta }^2{E}_{i,1}=\alpha {\gamma }^2{E}_{i,2}.$$

(11)

Solving for $\beta$, we obtain the relation between $\beta$ and $\gamma$:

$$\beta =\gamma \sqrt{{\displaystyle \frac{\alpha E_{i,2}}{E_{i,1}}}}.$$

(12)

We define the distortion $D_i$ introduced by modifying the DCT coefficients as

$$D_i=\sum _{j\in [1,2]}\sum _{k\in [f_{\mathrm{low}},f_{\mathrm{high}}]}{\displaystyle \frac{{(X_{i,j}^{\prime }\left(k\right)-X_{i,j}\left(k\right))}^2}{\left|{\mathbf{X}}_{i,j}[f_{\mathrm{low}},f_{\mathrm{high}}]\right|}}.$$

(13)

The distortion metric in Equation (13) is computed over the core embedding band $[f_{\mathrm{low}},f_{\mathrm{high}}]$, where the embedding rule and detection features are defined. The adjacent buffer zones $[f_{\mathrm{low}}-\xi ,f_{\mathrm{low}})$ and $(f_{\mathrm{high}},f_{\mathrm{high}}+\xi ]$ apply a linear taper (Equation (8)) to ensure smooth coefficient transitions. The resulting per-coefficient distortion in the buffer is attenuated compared to the core band, and its total energy is much smaller due to the narrow width of the buffer. Therefore, we only consider the total distortion over the core embedding band. Based on Equation (11) and the coefficient scaling rule, the distortion can be reformulated as

$$\begin{array}{cc}\hfill D_i& =\sum _{j\in [1,2]}{\left(\sqrt{{\displaystyle \frac{E_{i,j}^{\prime }}{E_{i,j}}}}-1\right)}^2{E}_{i,j}\hfill \\ \hfill & ={(\beta -1)}^2{E}_{i,1}+{(\gamma -1)}^2{E}_{i,2}\hfill \\ \hfill & ={\left(\gamma \sqrt{{\displaystyle \frac{\alpha E_{i,2}}{E_{i,1}}}}-1\right)}^2{E}_{i,1}+{(\gamma -1)}^2{E}_{i,2}.\hfill \end{array}$$

(14)

Obviously, $D_i$ is a function of $\gamma$. To minimize $D_i$, we take the derivative with respect to $\gamma$ and set it to zero:

$${\displaystyle \frac{\mathrm{d}{D}_i}{\mathrm{d}\gamma }}=2\left(\gamma \sqrt{{\displaystyle \frac{\alpha E_{i,2}}{E_{i,1}}}}-1\right)\sqrt{{\displaystyle \frac{\alpha E_{i,2}}{E_{i,1}}}}{E}_{i,1}+2(\gamma -1)E_{i,2}=0.$$

(15)

Solving Equation (15) yields

$$\gamma ={\displaystyle \frac{\sqrt{\alpha E_{i,1}{E}_{i,2}}+E_{i,1}{E}_{i,2}}{\alpha E_{i,2}+E_{i,1}{E}_{i,2}}}.$$

(16)

Substituting Equation (16) into Equation (11) gives

$$\beta ={\displaystyle \frac{\alpha E_{i,2}+E_{i,2}\sqrt{\alpha E_{i,1}{E}_{i,2}}}{\alpha E_{i,2}+E_{i,1}{E}_{i,2}}}.$$

(17)

It is evident that the above results satisfy Equation (5).

2.5. Theoretical Analysis of Robustness

This section theoretically analyzes the robustness of the proposed method against common signal processing and desynchronization attacks. When the embedded watermark bit is $w_i=0$, the energy features after embedding satisfy the condition

$${\displaystyle \frac{E_{i,1}^{\prime }}{E_{i,2}^{\prime }}}\ge \alpha .$$

(18)

Without loss of generality, we consider the equality case

$${\displaystyle \frac{E_{i,1}^{\prime }}{E_{i,2}^{\prime }}}=\alpha ,$$

(19)

as the basis for robustness analysis. After the audio signal is subjected to various attacks, the intra-frame energy relationship can be modeled as

$${\displaystyle \frac{E_{i,1}^{″}}{E_{i,2}^{″}}}={\displaystyle \frac{E_{i,1}^{\prime }+{\Delta }_1}{E_{i,2}^{\prime }+{\Delta }_2}},$$

(20)

where ${\Delta }_1=E_{i,1}^{″}-E_{i,1}^{\prime }$ and ${\Delta }_2=E_{i,2}^{″}-E_{i,2}^{\prime }$. It is evident that the proposed method is inherently resistant to closed-loop attacks, in which ${\Delta }_1={\Delta }_2=0$. For low-pass filtering, MP3 compression, and AAC compression attacks, which primarily affect high-frequency components, the mid-frequency band used for feature embedding remains mostly intact. Thus, we can reasonably assume that ${\Delta }_1\approx {\Delta }_2\approx 0$ under these conditions. In the case of amplitude scaling attacks, all frequency coefficients within a frame are scaled by the same factor $\sigma$, resulting in

$${\displaystyle \frac{E_{i,1}^{″}}{E_{i,2}^{″}}}={\displaystyle \frac{{\sigma }^2{E}_{i,1}^{\prime }}{{\sigma }^2{E}_{i,2}^{\prime }}}={\displaystyle \frac{E_{i,1}^{\prime }}{E_{i,2}^{\prime }}},$$

(21)

which demonstrates that the proposed method is robust to amplitude scaling. For echo addition, resampling, quantization, and additive noise attacks, the intra-frame energy ratio can be rewritten as

$${\displaystyle \frac{E_{i,1}^{″}}{E_{i,2}^{″}}}={\displaystyle \frac{\alpha E_{i,2}^{\prime }+{\Delta }_1}{E_{i,2}^{\prime }+{\Delta }_2}}.$$

(22)

To ensure correct watermark extraction (i.e., $w_i^{″}=0$), we require

$${\displaystyle \frac{E_{i,1}^{″}}{E_{i,2}^{″}}}\ge 1,$$

(23)

which leads to the condition

$$E_{i,2}^{\prime }\ge {\displaystyle \frac{{\Delta }_2-{\Delta }_1}{\alpha -1}}.$$

(24)

Since ${\Delta }_1$ and ${\Delta }_2$ reflect the energy perturbation in adjacent sub-segments within the same frame, we can reasonably assume ${\Delta }_2-{\Delta }_1\approx 0$. Moreover, $E_{i,2}^{\prime }$ captures the dominant energy in the sub-segment, and is typically far greater than ${\Delta }_2-{\Delta }_1$. Additionally, the threshold $\alpha$ can be adjusted to introduce a margin of tolerance, making the inequality easily satisfied. Therefore, the proposed method is robust against echo addition, resampling, quantization, and noise attacks.

As stated in Ref. [28], time-scale and pitch-scale modification attacks can be modeled as stretching or compressing operations along the frequency axis. These distortions may cause frequency components outside the embedding region to be involved in feature extraction, while some originally embedded coefficients may be excluded. To mitigate this issue, the proposed buffer compensation mechanism smooths the transition between watermarked and non-watermarked regions, thereby reducing the influence of incorrectly involved coefficients during extraction. Assuming the embedded watermark bit is 0 (the analysis for bit 1 is analogous), we have

$$E_{i,j}^{″}=E_{i,j}^{\prime }+{\Delta }_{\mathrm{interf}}^{\left(j\right)}.$$

(25)

Here, ${\Delta }_{\mathrm{interf}}^{\left(j\right)}$ represents the interference energy caused by the inclusion of non-embedded coefficients and the loss of embedded ones. To correctly extract the watermark bit, the following condition must hold:

$${\displaystyle \frac{E_{i,1}^{″}}{E_{i,2}^{″}}}={\displaystyle \frac{E_{i,1}^{\prime }+{\Delta }_{\mathrm{interf}}^{\left(1\right)}}{E_{i,2}^{\prime }+{\Delta }_{\mathrm{interf}}^{\left(2\right)}}}\ge 1.$$

(26)

This can be simplified as

$$\alpha \ge {\displaystyle \frac{1+{\Delta }_{\mathrm{interf}}^{\left(2\right)}-{\Delta }_{\mathrm{interf}}^{\left(1\right)}}{E_{i,2}^{\prime }-1}}.$$

(27)

Next, we examine ${\Delta }_{\mathrm{interf}}^{\left(2\right)}$ and ${\Delta }_{\mathrm{interf}}^{\left(1\right)}$ in conjunction with Equation (8). A linear scaling factor smaller than 1 is applied to ${\Delta }_{\mathrm{interf}}^{\left(2\right)}$, while a factor greater than 1 is applied to ${\Delta }_{\mathrm{interf}}^{\left(1\right)}$. This results in a reduced numerator; when the TSM and PSM attack intensities are moderate, $E_{i,2}^{\prime }-1$ is typically much greater than $\alpha$, ensuring that the above inequality is satisfied.

In the case of moderate jittering attacks, where a small proportion of samples is randomly removed, we also assume ${\Delta }_1-{\Delta }_2\approx 0$. This suggests that the proposed method retains its robustness against jittering as well.

3. Experimental Results and Analysis

3.1. Setup

To evaluate the robustness and imperceptibility of the proposed method, all experiments are conducted using the same audio dataset as in Ref. [16]. The dataset consists of audio clips from 25 different genres. Each audio sample is uniformly processed to a duration of 20 s, with a sampling rate of 44.1 kHz and a quantization precision of 16 bits. All audio samples are embedded with watermarks using the proposed algorithm. Specifically, the lower and upper frequency bounds are set to $f_{\mathrm{low}}=3000$ Hz and $f_{\mathrm{high}}=4000$ Hz, corresponding to the proportional parameters ${\eta }_1=0.1361$ and ${\eta }_2=0.1814$, respectively. The buffer compensation width is configured as $\xi =500$ Hz, and the embedding strength parameter is set to $\alpha =8$. All experiments are conducted on a workstation equipped with a 12th Gen Intel(R) Core(TM) i7-12700H CPU at 2.30 GHz, 16 GB RAM, and an NVIDIA RTX 3060 GPU, using MATLAB R2020b as the software environment.

Metrics

To assess the perceptual quality of the watermarked audio, two commonly used objective metrics are employed: Signal-to-Noise Ratio (SNR) and Objective Difference Grade (ODG). According to the guidelines established by the International Federation of the Phonographic Industry (IFPI), an SNR value above 20 dB is generally required to ensure acceptable perceived audio quality. The SNR is computed as follows:

$$\mathrm{SNR}=10{\log }_{10}\left({\displaystyle \frac{{\sum }_{i=1}^{n}X{\left(i\right)}^2}{{\sum }_{i=1}^n{\left(X\left(i\right)-X^{\prime }\left(i\right)\right)}^2}}\right),$$

(28)

ODG is a key output of the Perceptual Evaluation of Audio Quality (PEAQ) standard, which simulates human auditory perception based on psychoacoustic models. ODG values typically range from $-4$ (very annoying) to 0 (imperceptible), with values between $-1$ and 0 generally indicating no perceptible degradation in audio quality.

To evaluate the robustness of the embedded watermark against common signal processing attacks, two additional metrics are adopted: Bit Error Rate (BER) and Normalized Cross-Correlation (NCC).

3.2. Imperceptibility Experiment

To further validate the transparency of the proposed embedding strategy, we compare the waveforms before and after watermark embedding for six different music genres at an embedding rate of 30 bps. As depicted in Figure 2, the waveforms remain visually indistinguishable before and after watermarking, demonstrating that our method introduces negligible perceptual distortion.

Figure 3a shows the mean SNR values under different embedding strength factors $\alpha$. As expected, both higher embedding rates and stronger embedding strengths negatively affect the SNR. Specifically, the SNR gradually decreases as $\alpha$ increases. Nevertheless, the proposed method maintains a high perceptual quality even under large $\alpha$ values.

Figure 3b illustrates the mean ODG values at 30 bps for different values of $\alpha$. As anticipated, ODG decreases with increasing embedding strength. However, even at a relatively large embedding strength (e.g., $\alpha =10$), the ODG remains greater than $-1$, indicating that the proposed method maintains high perceptual transparency. This result can be attributed to our embedding optimization strategy, which minimizes the distortion power while preserving the desired embedding proportion.

3.3. Robustness Against Common Attacks

To evaluate the robustness of the proposed watermarking method under realistic signal processing operations, we conduct a series of experiments involving various common audio attacks. These attacks are representative of distortions that may occur during typical transmission, compression, and playback scenarios. The goal is to assess whether the embedded watermark can be reliably extracted after such distortions.

• Non-attack: The watermark is extracted directly from the watermarked audio signal without any modification.
• Noise attack: Additive white Gaussian noise (AWGN) is added to the watermarked audio.
• Quantization attack: The watermarked audio is re-quantized before watermark extraction.
• Amplitude attack: The amplitude of the watermarked audio is scaled.
• Echo attack: An echo signal with a time delay is added to the watermarked audio.
• MP3 attack: The watermarked audio is compressed using MPEG-1 Layer III encoding.
• AAC attack: The watermarked audio is compressed using MPEG-4 Advanced Audio Coding (AAC).
• Resampling attack: The watermarked audio is first downsampled and then upsampled.
• Low-pass filter attack: A low-pass filter is applied to the watermarked signal.
• High-pass filter attack: A high-pass filter is applied to the watermarked signal.

We compare the proposed method with three advanced audio watermarking algorithms, namely those presented in Refs. [21,28,35]. To ensure a fair comparison, the embedding strength of each method is adjusted such that the resulting watermarked audio achieves a similar perceptual quality, with the SNR uniformly set to 25 dB. Typical parameters are selected for each type of attack. Specifically, for the noise attack, the SNR between the watermarked audio and the added noise is set to 20 dB and 30 dB, respectively. In the quantization attack, the bit depth is reduced from 16 bits to 8 bits. For amplitude scaling, scaling factors of 0.8 and 1.2 are applied. For the echo attack, the delay is set to 0.5 s, and the echo amplitude is 20% of the original signal. For MP3 compression, bitrates of 128 kbps and 96 kbps are used, with the same settings applied for AAC compression. In the resampling attack, the watermarked audio is downsampled from 44.1 kHz to 22.05 kHz and then upsampled back to 44.1 kHz. For low-pass filtering, the cutoff frequency is 8 kHz, while for high-pass filtering, it is 0.5 kHz.

As shown in

Table 1. The mean BERs (%) and mean NCC of different methods against common attacks at an embedding rate of 30 bps.

Common Attacks		Ref. [21]		Ref. [35]		Ref. [28]		Proposed
		BER	NCC	BER	NCC	BER	NCC	BER	NCC
Non-attack		0.00	1.000	0.00	1.000	0.00	1.000	0.00	1.000
Noise	30 dB	0.05	0.999	22.59	0.688	0.17	0.998	0.13	0.999
	20 dB	1.27	0.987	42.44	0.300	2.08	0.978	1.45	0.985
Quantization		0.11	0.999	14.37	0.856	0.19	0.998	0.13	0.999
Amplitude	0.8	0.00	1.000	0.00	1.000	0.00	1.000	0.00	1.000
	1.2	0.00	1.000	0.00	1.000	0.00	1.000	0.00	1.000
Echo		1.04	0.989	7.29	0.919	2.12	0.978	4.01	0.960
MP3	128 kbps	0.00	1.000	2.05	0.976	0.00	1.000	0.00	1.000
	96 kbps	0.00	1.000	8.41	0.902	0.00	1.000	0.00	1.000
AAC	128 kbps	0.00	1.000	0.02	1.000	0.00	1.000	0.00	1.000
	96 kbps	0.00	1.000	0.02	1.000	0.00	1.000	0.00	1.000
Resampling		0.00	1.000	30.78	0.593	0.00	1.000	0.00	1.000
Low-pass filtering		0.00	1.000	0.04	1.000	0.00	1.000	0.00	1.000
High-pass filtering		0.00	1.000	44.15	0.355	0.03	1.000	0.00	1.000

, the proposed method, along with the methods in Refs. [21,28], demonstrates strong robustness against common signal processing attacks, achieving low BER. In contrast, the method in Ref. [35] exhibits weaker robustness under noise, low-pass filtering, and resampling attacks. This limitation stems from its reliance on residual-based FDLM features, which are inherently sensitive to noise and sampling variations. Moreover, its feature extraction is performed on the top 75% of the frequency band coefficients, which includes a significant proportion of high-frequency components, rendering the method more susceptible to low-pass filtering.

One of the most significant contributions of this paper is its robustness against desynchronization attacks. Common types of desynchronization attacks considered include the following:

• Jitter: A certain proportion of sample points from each segment of the watermarked signal is randomly removed.
• TSM: The duration of the signal is modified while the pitch remains unchanged.
• PSM: The pitch of the signal is altered while maintaining its duration.

Representative parameters are selected for each attack. Specifically, for the jittering attack, 1 sample is randomly removed from every 100 samples. For TSM, the time scaling factor is selected from the range [0.8, 1.2]. Similarly, for PSM, the pitch scaling factor is varied within [0.8, 1.2] while keeping the duration unchanged. The experimental results under desynchronization attacks are presented in

Table 2. The mean BERs (%) and mean NCC of different methods under desynchronization attacks at an embedding rate of 30 bps.

Desynchronization Attacks		Ref. [21]		Ref. [35]		Ref. [28]		Proposed
		BER	NCC	BER	NCC	BER	NCC	BER	NCC
Jitter (1%)		3.39	0.963	37.93	0.446	9.58	0.906	10.55	0.902
TSM	80%	0.25	0.997	25.47	0.676	1.83	0.981	1.24	0.987
	90%	0.13	0.999	22.26	0.724	1.91	0.980	0.68	0.993
	110%	0.15	0.998	19.19	0.767	2.27	0.976	1.93	0.980
	120%	0.17	0.998	21.33	0.737	6.51	0.933	7.40	0.923
PSM	80%	31.94	0.639	33.18	0.544	11.36	0.882	7.03	0.928
	90%	49.13	0.441	27.83	0.639	3.49	0.963	2.17	0.978
	110%	52.10	0.382	20.47	0.750	8.02	0.916	3.71	0.962
	120%	30.83	0.663	24.11	0.697	16.48	0.829	12.31	0.872

. It can be observed that the proposed method achieves significant superiority in resisting desynchronization, with all BERs remaining below 13%, which aligns well with the theoretical analysis discussed in Section 2.5. In comparison, the method in [21] exhibits partial robustness against jitter and TSM attacks, but fails completely under PSM, while the method in [35] suffers from BERs exceeding 20% under TSM and PSM due to the high embedding rate of 30 bps, which exceeds its operational capacity. Both the proposed method and the method in [28] show strong resistance to TSM and PSM attacks; however, our method consistently outperforms [28], owing to the incorporation of a linearly decreasing buffer compensation mechanism. In general, desynchronization attacks disrupt watermark embedding positions, significantly increasing the difficulty of accurate extraction. The compared methods fail to effectively address this challenge due to the lack of robust synchronization features or mechanisms. In contrast, our method exhibits the best overall performance, demonstrating its superior robustness against various forms of desynchronization.

3.4. Runtime Analysis

In addition to robustness and imperceptibility evaluations, we also assess the computational overhead of the proposed embedding algorithm to verify its suitability for real-time applications. The runtime measurements were conducted on a 12th Gen Intel(R) Core(TM) i7-12700H CPU with 16 GB RAM, running MATLAB 2020b.

In our implementation, each audio clip has a length of 20 s sampled at 44.1 kHz, resulting in a total of 882,000 samples. The signal is divided into $N_f=600$ equal-length frames, giving $L_f=1470$ samples per frame, corresponding to approximately 33.33 ms of audio.

Table 3. Average runtime per frame for the proposed method. Std values are only reported for the total runtime.

Step	Avg Runtime (ms)	Std (ms)	Percentage
DCT + IDCT	0.185	–	88.5%
Buffer Scaling	0.007	–	3.3%
Other Operations	0.017	–	8.2%
Total	0.209	0.016	100%

presents the average runtime per frame for each main processing step. The results show that the proposed method achieves an average embedding runtime of 0.209 ms per frame (std: 0.016 ms), corresponding to a throughput of approximately 4784 frames/s, i.e., more than 160 s of audio per second of processing time. Given that typical real-time audio processing requires fewer than 100 frames/s, the proposed method easily satisfies real-time requirements. Notably, the DCT/IDCT operations dominate the total runtime, while buffer scaling accounts for only a small fraction. The mean SNR remains at 25.61 dB, indicating that the imperceptibility is preserved.

3.5. Ablation Study

To verify the effectiveness of the optimization strategy, we conduct comparative experiments under scenarios with and without parameter optimization. We evaluate the audio quality using the Signal-to-Noise Ratio (SNR) and Objective Difference Grade (ODG) as objective metrics. As shown in Figure 4, the proposed optimization yields significant improvements in both SNR and ODG across varying embedding strengths. This improvement can be attributed to the reduced energy distortion achieved by the analytical optimization of $\beta$ and $\gamma$.

To further evaluate the impact of buffer window length $\xi$ on watermarking performance, a detailed analysis is conducted under four different desynchronization attack scenarios. As illustrated in Figure 5a, the mean BERs are plotted for TSM and PSM attacks with scaling factors of 0.8 and 1.2, respectively, across a range of $\xi$ values from 0 to 0.75 kHz. The results show that BER consistently decreases as $\xi$ increases under all conditions. In particular, when $\xi =0$, the method still exhibits a certain degree of robustness, but the BER is noticeably higher—especially under PSM with a scaling factor of 1.2. This demonstrates that the buffer compensation mechanism can effectively alleviate the distortions caused by desynchronization attacks. As $\xi$ increases, more adjacent coefficients are involved in the embedding process, providing a buffer compensation effect that suppresses desynchronization and significantly reduces BER, thereby enhancing overall robustness.

We also investigate the influence of increasing $\xi$ on perceptual audio quality. As shown in Figure 5b, both the average SNR and ODG decrease as $\xi$ grows. This trend is expected, as a wider embedding region introduces greater energy modifications. Notably, when $\xi \ge 0.75$ kHz, the ODG value drops below −1.0, suggesting that the distortion introduced by watermarking may become perceptible to human listeners. Nevertheless, the SNR remains above 24 dB, and the ODG does not fall below typical acceptability thresholds.

In summary, increasing the buffer window length significantly improves robustness, particularly under severe desynchronization attacks such as PSM. However, it also introduces a trade-off in perceptual audio quality. To balance robustness and audio quality, a buffer length of $\xi =0.5$ kHz is selected in our experiments.

To further examine the limitations of the proposed method, we conduct supplementary tests under extremely strong desynchronization conditions (130% TSM/PSM). Figure 6 presents the mean BER versus $\xi$ for TSM 0.7, TSM 1.3, PSM 0.7, and PSM 1.3. The results show that for PSM 0.7, the BER remains above 20% when $\xi \le 0.5$ kHz, and for TSM 1.3, the BER exceeds 8% even at $\xi =0.75$ kHz. These findings confirm that while the buffer compensation mechanism effectively mitigates moderate desynchronization, extremely strong scaling factors still cause noticeable degradation. At the same time, the results validate our choice of $\xi =0.5$ kHz, which offers a good balance between robustness improvement and imperceptibility under both normal and extreme attack scenarios.

4. Conclusions and Discussion

This paper presents a novel patchwork-based audio watermarking method designed to address the persistent challenge of desynchronization attacks in robust audio watermarking. The proposed method enhances resistance to typical desynchronization distortions, including TSM, PSM, and jitter, while maintaining high audio quality and a reasonable embedding capacity. Specifically, the audio signal is first divided into frames, and each frame is further split into two non-overlapping segments. Watermark bits are embedded by modulating the energy ratio of mid-frequency DCT coefficients between the two segments. A key innovation of this method lies in the introduction of a linearly decreasing buffer compensation mechanism, which alleviates boundary inconsistencies between watermark and non-watermark coefficients. By gradually reducing the scaling factors near the boundaries, this mechanism not only mitigates distortion caused by coefficient modification but also suppresses interference from non-watermark coefficients during feature extraction, thereby significantly improving watermark detection accuracy under TSM and PSM attacks. Experimental results demonstrate that the proposed method achieves a favorable balance among imperceptibility, embedding rate, and robustness. Under a wide range of desynchronization attacks, including TSM, PSM, and jitter, the BER consistently remains below 13%, validating the effectiveness of the buffer compensation mechanism and optimization strategy. Compared with existing state-of-the-art techniques, the proposed method exhibits superior robustness and perceptual transparency, confirming its applicability to practical audio watermarking scenarios.