Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks

Abstract

As training Deep Neural Networks (DNNs) becomes more expensive, the interest in protecting the ownership of the models with watermarking techniques increases. Uchida et al. proposed a digital watermarking algorithm that embeds the secret message into the model coefficients. However, despite its appeal, in this paper, we show that its efficacy can be compromised by the optimization algorithm being used. In particular, we found through a theoretical analysis that, as opposed to Stochastic Gradient Descent (SGD), the update direction given by Adam optimization strongly depends on the sign of a combination of columns of the projection matrix used for watermarking. Consequently, as observed in the empirical results, this makes the coefficients move in unison giving rise to heavily spiked weight distributions that can be easily detected by adversaries. As a way to solve this problem, we propose a new method called Block-Orthonormal Projections (BOP) that allows one to combine watermarking with Adam optimization with a minor impact on the detectability of the watermark and an increased robustness.

1. Introduction

Deep learning has substantially impacted technology over the last years, becoming an important center of attention for researchers all over the world. Such a great impact stems from the versatility it offers as well as the excellent results that DNNs achieve on multiple tasks, like image classification or speech recognition, which often reach and even surpass human-level performance [1,2].

However, far from being a simple task, the design of new DNNs is generally expensive not only in terms of human effort and time needed to build effective model architectures but mostly because of the large volume of suitable data that must be gathered and the vast amount of computational resources and power used for training. Consequently, businesses owning costly models are interested in protecting them from any illicit use, and this growing need has recently led researchers to a common concern on how to embed watermarks on DNNs. As a result, several frameworks for protecting the intellectual property of neural networks were proposed in the literature, and can be classified as black-box or white-box approaches. The first kind of methods (i.e., black-box) do not need to access model parameters for detecting the presence of watermarks. Instead, key inputs are introduced as special triggers to identify the original network, either by the use of the so-called adversarial examples [3] or backdoor poisoning [4].

White-box approaches, in contrast, directly affect the model parameters in order to embed watermarks. One of the most significant white-box contributions was proposed by Uchida et al. in [5,6], the algorithm under study in this paper. This watermarking framework employs a regularization term that defines a cost function for embedding the secret message. This regularizer, which transforms weights into bits by means of a projection matrix in a similar way to spread–spectrum techniques used in watermarking [7], is added to the main loss function and can be applied either at the beginning of the training phase (i.e., from scratch) or during fine-tuning steps.

On the other hand, when training a DNN, there are several optimization algorithms to choose from. With their pros and cons, some of the most popular are the classic Stochastic Gradient Descent (SGD) [8] and Adaptive Moment Estimation (Adam) [9]. We found that, in order to perform watermarking following the approach in [5,6], it is necessary to pay close attention to the optimization algorithm being used. In this paper, we prove that Adam, although being an appealing algorithm for lots of applications—especially because of its efficiency and training speed—poses a big problem when implementing this watermarking method.

As any other watermarking algorithm, it is important to meet some minimal requirements regarding fidelity, robustness, payload, efficiency, and undetectability [5,6,10]. This latter property involves the need for concealing any clue that would let unauthorized parties know if a watermark was embedded, along with its further consequences. In other words, from a steganographic point of view, watermarks should be embedded into the DNN without leaving any detectable footprint.

We show that, unlike SGD, Adam significantly alters the distribution of the coefficients at the embedding layer in a very specific way and, thus, it compromises the undetectability of the watermark. In particular, when using Adam, the histogram of the weights at the embedding layer shows that these coefficients tend to group together at both sides of zero, originating two visible spikes that grow in magnitude with the number of iterations, so that the initial distribution ends up changing completely. As we will see later, the update direction given by Adam depends on the projection matrix used for watermarking; specifically, it is based on the sign function, responsible for the symmetric two-spiked shape that emerges. This behavior is somewhat surprising, as with a pseudorandom projection matrix, one might expect that the weights evolve with random speeds; in contrast, the weights tend to move in unison in a way that is reminiscent of ants after they have established a solid path from the nest to a food source. Then, the statistical dependence of the weights with the projection matrix is weak, and this is mainly due to the information collapse induced by the sign function; this loss is invested in making the weights more conspicuous and, therefore, the watermark more easily detectable. As we will confirm, the sign phenomenon does not occur in other algorithms like SGD. The appearance of the sign was already pointed out and studied—without considering watermarking applications—by the authors in [11], who explained the adverse effects of Adam on generalization [12] as a consequence of this aspect. However, instead of delving into the general performance of the optimization algorithm itself, we show that the sign function which is involved in Adam’s update direction is detrimental for watermarking purposes, so we highlight the need for being careful with the selection of the optimization algorithm in these cases.

In this paper, we carry out a theoretical and experimental analysis and compare the results using Adam and SGD optimization. The analysis can be extended to other optimization algorithms. As a way to measure the similarity between the original weight distribution and the resulting one after the watermark embedding, we will use the Kullback–Leibler divergence (KLD) [13]. We will show that, as expected, when we use Adam, the KLD between both distributions is considerably larger than when we use SGD, thus confirming that, as opposed to SGD, Adam modifies the original distribution to a great extent. Furthermore, in order to perform this kind of watermarking and, at the same time, enjoy the advantages that Adam optimization provides, we propose a new method called Block-Orthonormal Projections (BOP), which uses a secret transformation matrix in order to reduce the detectability of the watermark generated by Adam. As we will see, BOP allows us to considerably reduce the KLD to small values which are comparable to those obtained with SGD. Therefore, we show that BOP allows us to preserve the original shape of the weight distribution.

In summary, this work makes the following two-fold contribution:

• We provide mathematical and experimental evidence for SGD and Adam to show that: (1) in contrast to SGD, the changes in the distribution of weights caused by Adam can be easily detected when embedding watermarks following the approach in [5,6] and, hence, (2) the use of Adam considerably increases the detectability of the watermark. For the purpose of carrying out this analysis, we use FFDNet [14]—a DNN that performs image denoising tasks—as the host network.
• We introduce a novel method based on orthogonal projections to solve the detectability problem that arises when watermarking a DNN which is being optimized with Adam. A side effect of this novel method is an increased robustness against weight pruning.

The remainder of this paper is organized as follows: Section 1.1 introduces the notation and Section 2 explains the frameworks and algorithms used in this study—host network, optimization algorithms, and watermarking method—in more detail. Section 3 presents the mathematical core that allows us to model the observed effects on the histograms of weights once the embedding process has finished. Then, in Section 4, we introduce BOP as a solution for using Adam and watermarking simultaneously, Section 5 presents the information-theoretic measures that we will implement, and Section 6 shows the experimental results. Finally, we point out some concluding remarks in Section 7. Two appendices give additional details on mathematical derivations (Appendix A) and the validity of certain assumptions (Appendix B).

Notation

In this paper, we use the following notation. Matrices and vectors are denoted by upper-case and lower-case boldface characters, respectively, while random variables and their realizations are respectively represented by upper-case and lower-case characters.

For matrix and vector operations, we proceed as follows. As an example, let $\mathbf{A}$ be a matrix. Then, its transpose is denoted by ${\mathbf{A}}^T$. Moreover, we use $\mathrm{Tr}[\mathbf{A}]$ to represent the trace of $\mathbf{A}$ and ${(\mathbf{A})}_{i,j}$ to denote the $(i,j)$th element of $\mathbf{A}$. ${\mathbf{I}}_N$ refers to the $N\times N$ identity matrix. We use column vectors unless otherwise stated. In addition, we use $\mathbf{0}$ to denote a column vector of zeros and $\mathbf{1}$ for a column vector of ones. Let $\mathbf{w}$ be a column vector of length N; then, ${\mathbf{\nabla }}_{\mathbf{w}}$ is the gradient operator with respect to $\mathbf{w}$ that is:

$${\mathbf{\nabla }}_{\mathbf{w}}\doteq {\left(\frac{\partial }{\partial w_1},\cdots ,\frac{\partial }{\partial w_N}\right)}^T$$

We use the operator ∘ to denote the Hadamard (i.e., sample-wise) product and ⊗ for the Kronecker product. Finally, $\mathrm{E}\{·\}$ and $\mathrm{Var}\{·\}$ denote the mathematical expectation and the variance, respectively.

2. Preliminaries

2.1. Host Network: FFDNet

The rapid development of Deep Learning over the last few years has led to new advances in the field of image restoration [15]. Several Convolutional Neural Networks (CNNs) have been designed to replace classical methods and, often, they offer new competitive advantages. This is the case of FFDNet [14], which performs image denoising tasks and is used in our work as the exemplary host network that is watermarked by means of the algorithm proposed in [5,6].

Image denoising is the task of removing noise from a given image. Let $\mathbf{y}$ be the input noisy image, $\mathbf{x}$ the clean image, and $\mathbf{n}$ the noise, which is usually modeled as zero-mean Additive White Gaussian Noise (AWGN), then we have $\mathbf{y}=\mathbf{x}+\mathbf{n}$ and we wish to obtain an estimate $\widehat{\mathbf{x}}$ of the clean image. As opposed to other CNNs for image denoising, FFDNet works on downsampled sub-images, and it is able to adapt to several noise levels using only a single network. For that purpose, a noise level map $\mathbf{M}$ is included also as an input, so that the function FFDNet aims to learn can be expressed as: $\widehat{\mathbf{x}}=\mathcal{F}(\mathbf{y},\mathbf{M};\mathbf{w})$, where $\mathbf{w}$ represents the parameters of the network. Furthermore, FFDNet can handle spatially variant noise and offers competitive inference speed without sacrificing denoising performance [14].

Figure 1 shows the architecture of FFDNet. As we can see, it is composed of a downscaling operation on the input images, a nonlinear mapping consisting of Convolutional Layers, Batch Normalization steps [16], and ReLU activation functions; and, finally, an upscaling process to generate denoised images with the original size. Let $\left\{({\mathbf{y}}_i,{\mathbf{x}}_i)\right\}$ be L noisy-clean image pairs from the training dataset, the denoising cost function is:

$$f_0(\mathbf{w})=\frac{1}{2L}\sum _{i=1}^L||\mathcal{F}({\mathbf{y}}_i,{\mathbf{M}}_i;\mathbf{w})-{\mathbf{x}}_i{||}^2$$

(1)

FFDNet can be used for grayscale or color images.

Table 1. FFDNet configurations for grayscale and RGB image denoising.

	Grayscale	RGB
Conv layers	15	12
Feature maps per layer	64	96
Receptive field	$62\times 62$	$50\times 50$

shows the main differences between both configurations. As we can see, the total number of convolutional layers can be set to 15 or 12 for grayscale or RGB denoising, respectively. The number of feature maps and the size of the receptive field also differ, but this is not the case of the kernel size, which is kept to $3\times 3$ for either grayscale or RGB. In this paper, we implement the RGB version of FFDNet and proceed as follows: (1) train FFDNet from scratch using Adam optimization without embedding any watermark and (2) fine-tune the network to embed the desired watermark, using both Adam and SGD to compare the results.

2.2. Optimization Algorithms

The training of a DNN is an iterative process that makes it possible for the model to learn how to perform a given task. This is certainly the most challenging optimization problem when implementing deep learning models from scratch. In order to increase the efficiency of the training process, researchers have developed several optimization techniques in the last years, each of them with its own advantages and drawbacks [17].

An optimization algorithm determines the weight update rule that must be applied at every iteration. The goal is usually to minimize a cost function—also known as loss function—which generally compares predictions with expected values and computes an error metric that evaluates the performance of the network. The learning rate $\mu$—or step size—is the hyperparameter that controls how much the weights can change after each iteration of the optimization algorithm.

In the following sections, we briefly review the mechanics of two widely known optimization algorithms: SGD and Adam.

2.2.1. SGD Optimization

SGD [8] is a classic optimization algorithm based on gradient descent and one of the most used. However, unlike standard gradient descent techniques that use all the training samples to compute the gradient of the cost function, SGD uses a small number of samples from the dataset—a minibatch—and then takes the average over these samples to get an estimate of the gradient, ${\mathbf{\nabla }}_{\mathbf{w}}f(\mathbf{w})$. Then, the update rule is given by:

$${\mathbf{w}}^{(k)}={\mathbf{w}}^{(k-1)}-\mu {\mathbf{\nabla }}_{\mathbf{w}}f({\mathbf{w}}^{(k-1)}),k\ge 1$$

(2)

where ${\mathbf{w}}^{(k)}$ is the weight vector at iteration k and ${\mathbf{w}}^{(0)}$ is its initial value.

2.2.2. Adam Optimization

Adam [9] is a popular optimization algorithm that combines the ideas of AdaGrad [18] and RMSProp [19]. Some of the advantages of Adam include, among others, the fact of being easy to implement and computationally efficient, as well as being fast and suitable for complex settings with noisy and sparse gradients.

Just like SGD, Adam estimates the gradient from the samples of a minibatch, but, as opposed to SGD, it uses estimates of the first and second moments of the gradients to compute individual adaptive learning rates for each parameter in the network. In order to do that, exponential moving averages of the gradient and the squared gradient are calculated using two hyperparameters to control the decay rate, ${\beta }_1$ and ${\beta }_2$, respectively. Let ${\mathbf{m}}^{(0)}=\mathbf{0}$ and ${\mathbf{v}}^{(0)}=\mathbf{0}$ be the initial values for the first and second moment vectors, respectively, then the steps of this algorithm at the kth iteration are the following [9]:

$$\begin{array}{ccc}\hfill {\widehat{\mathbf{g}}}^{(k)}& =& {\mathbf{\nabla }}_{\mathbf{w}}f({\mathbf{w}}^{(k-1)})\hfill \end{array}$$

(3)

$$\begin{array}{ccc}\hfill {\mathbf{m}}^{(k)}& =& {\beta }_1{\mathbf{m}}^{(k-1)}+(1-{\beta }_1){\widehat{\mathbf{g}}}^{(k)}\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill {\mathbf{v}}^{(k)}& =& {\beta }_2{\mathbf{v}}^{(k-1)}+(1-{\beta }_2)({\widehat{\mathbf{g}}}^{(k)}\circ {\widehat{\mathbf{g}}}^{(k)})\hfill \end{array}$$

(5)

$$\begin{array}{ccc}\hfill {\widehat{\mathbf{m}}}^{(k)}& =& \frac{{\mathbf{m}}^{(k)}}{(1-{\beta }_1^k)}\hfill \end{array}$$

(6)

$$\begin{array}{ccc}\hfill {\widehat{\mathbf{v}}}^{(k)}& =& \frac{{\mathbf{v}}^{(k)}}{(1-{\beta }_2^k)}\hfill \end{array}$$

(7)

$$\begin{array}{ccc}\hfill w_j^{(k)}& =& w_j^{(k-1)}-\mu \frac{{\widehat{m}}_j^{(k)}}{\sqrt{{\widehat{v}}_j^{(k)}}+ϵ},j=1,\cdots ,N.\hfill \end{array}$$

(8)

where $w_j^{(k)}$, ${\widehat{m}}_j^{(k)}$, ${\widehat{v}}_j^{(k)}$ are the jth element of ${\mathbf{w}}^{(k)}$, ${\widehat{\mathbf{m}}}^{(k)}$, ${\widehat{\mathbf{v}}}^{(k)}$, respectively, $f(\mathbf{w})$ is the cost function and $ϵ$ is a very small number that avoids diving by zero. The default set-up for the hyperparameters is [9]: ${\beta }_1=0.9$, ${\beta }_2=0.999$ and $ϵ=1·{10}^{-8}$.

2.3. Digital Watermarking Algorithm

In this paper, we analyze the digital watermarking algorithm proposed in [5,6]. As we mentioned previously, we employ the fine-tune-to-embed approach; therefore, the embedding function is applied only during some additional epochs after convergence is achieved for the original task—in this case, image denoising—.

2.3.1. Embedding Elements

We wish to embed a T-bit sequence, $\mathbf{b}\in {\{0,1\}}^T$, into a certain layer l of the host DNN. For this specific layer, let $(S,S)$, I and F represent the size of the convolution filter, the depth of input to the convolutional layer and the number of filters in that layer, respectively. Then, the weights can be represented by a tensor ${\mathbf{W}}_l$ with dimensions $S\times S\times I\times F$ and then rearranged to form a vector ${\mathbf{w}}_l$ of length $N=S^{2}IF$.

One important point here is that, instead of directly using the weight vector ${\mathbf{w}}_l$, the authors in [5,6] suggest including an initial transformation of these coefficients. In order to reflect this, we must calculate the mean of ${\mathbf{W}}_l$ over the F kernels. As a result, we obtain a new flattened vector ${\widehat{\mathbf{w}}}_l$ with M elements, where $M=S^{2}I$. This transformation can also be formulated if we introduce a new matrix $\mathbf{\Theta }$ of size $N\times M$:

$$\mathbf{\Theta }\doteq {\mathbf{I}}_M\otimes \mathbf{h}$$

so that ${\widehat{\mathbf{w}}}_l={\mathbf{\Theta }}^T{\mathbf{w}}_l$. Here, $\mathbf{h}$ is a column vector of length F with all of its elements set to $1/F$, i.e., ${\mathbf{h}}^T\mathbf{h}=1/F$.

In order to move to the T-dimensional space, the authors in [5,6] introduce a secret projection matrix $\mathbf{\Phi }$. The size of this projection matrix—also referred to as regularizer parameter in [5,6]—is $M\times T$ so that each column corresponds to a particular projection vector ${\mathit{\varphi }}^{(i)}$, $i=1,\cdots ,T$.

For the purpose of the subsequent theoretical analysis, we pair up matrices $\mathbf{\Phi }$ and $\mathbf{\Theta }$ and, therefore, we ascribe the initial transformation to the projection matrix, preserving the notation with the original weight vector ${\mathbf{w}}_l$. To that end, we define a new $N\times T$ projection matrix:

$$\widehat{\mathbf{\Phi }}=\mathbf{\Theta }\mathbf{\Phi }$$

(9)

so that now the projection vectors can be expressed as ${\widehat{\mathit{\varphi }}}^{(i)}=\mathbf{\Theta }{\mathit{\varphi }}^{(i)}$.

2.3.2. Embedding Process

Now that we have introduced the basic elements, the embedding procedure can be described as follows [5,6]. Let $\mathbf{w}$ be a vector containing all the parameters of the network, then the watermarking regularizer, $f_{\mathsf{watermark}}({\mathbf{w}}_l)$, is added to the global cost function $f(\mathbf{w})$:

$$f(\mathbf{w})=f_0(\mathbf{w})+\lambda f_{\mathsf{watermark}}({\mathbf{w}}_l)$$

where $f_0(\mathbf{w})$ is the original cost function and $\lambda$ is the regularization parameter. The regularizer term is a composition of two functions: cross-entropy and the sigmoid,

$$f_{\mathsf{watermark}}({\mathbf{w}}_l)\doteq -\sum _{i=1}^T(b_{i}log(y_i)+(1-b_i)log(1-y_i))$$

(10)

with $y_i=1/\left(1+exp(-{({\widehat{\mathit{\varphi }}}^{(i)})}^T{\mathbf{w}}_l)\right)$. In order to minimize (10), $y_i$ will approximate to the value of $b_i$. Therefore, the sigmoid function will force each projection ${({\widehat{\mathit{\varphi }}}^{(i)})}^T{\mathbf{w}}_l$, $i=1,\cdots ,T$, to progressively move towards $+\infty$ and $-\infty$ depending on whether $b_i=1$ or $b_i=0$, respectively. For successfully embedding the secret message, it is generally enough to guarantee that each projection lies on the proper side of the horizontal axis. When this happens, we reach a Bit Error Rate (BER) of 0 and all the projected weights are aligned with their corresponding bits, that is, they are positive when the bit is 1 and, conversely, negative when the bit is 0.

2.3.3. Detectability Issues

In this paper, we employ the Random technique proposed by the authors, in which the values of the projection matrix $\mathbf{\Phi }$—before applying the transformation in (9)—are independent samples from a standard normal distribution $\mathcal{N}(0,1)$. Their results [5,6] show that this Random approach is the most appealing design method for $\mathbf{\Phi }$ because, as they indicate, it does not significantly alter the distribution of weights at the embedding layer. However, there are some detectability issues here that should be considered.

On the one hand, the authors in [20] show that the standard deviation of the distribution of weights grows with the length of the embedded message. This information can be used by adversaries for detecting the watermark and even overwriting it.

On the other hand, one of the main conclusions of our work is that the presence or absence of alterations to the shape of the weight distributions is a consequence of the optimization algorithm used during the watermark embedding. In particular, the authors in [5,6] employ SGD with momentum in their experiments and the distributions of weights remain unchanged, yet the use of Adam would significantly alter the shape of the distributions even though we apply the same Random technique. In this paper, we will show that the results in [5,6] regarding the undetectability of the watermark do not hold when we use Adam optimization.

As an example to visualize this peculiar behavior shown by Adam when we employ the watermarking algorithm proposed in [5,6], we plot in Figure 2 the resulting histograms when $T=256$ and $\lambda =1$; specifically, the histograms of the weights before and after the embedding (corresponding to k = 32,140) and the histogram of the weight variations, respectively. As we can see, the distribution of the original weights has significantly changed, turning into a two-spiked shape that could be easily detected by an adversary. The complete set of histograms will be later shown in Section 6.

2.3.4. Gaussian and Orthogonal Projection Vectors

In addition to the Random technique suggested by the authors of this watermarking algorithm—whose projection vectors will be referred to as Gaussian projection vectors in this paper—we will also implement orthonormal projectors. In order to build these kinds of projectors, we first generate the projection matrix following the Random technique; that is, samples are drawn from a standard normal distribution $\mathcal{N}(0,1)$. Then, from the Singular Value Decomposition of this projection matrix, we obtain an orthonormal basis for the column space so that we have ${\mathbf{\Phi }}^T\mathbf{\Phi }={\mathit{I}}_T$. Notice that once we apply the initial transformation (i.e., $\widehat{\mathbf{\Phi }}=\mathbf{\Theta }\mathbf{\Phi }$) the resulting projection vectors ${\widehat{\mathit{\varphi }}}^{(i)}$, $i=1,\cdots ,T$, will still preserve the orthogonality between them, although they will not be normalized:

$${(\widehat{\mathbf{\Phi }})}^T\widehat{\mathbf{\Phi }}={(\mathbf{\Theta }\mathbf{\Phi })}^T\mathbf{\Theta }\mathbf{\Phi }={\mathbf{\Phi }}^T{\mathbf{\Theta }}^T\mathbf{\Theta }\mathbf{\Phi }={\mathbf{\Phi }}^T({\mathit{I}}_M\otimes {\mathbf{h}}^T)({\mathit{I}}_M\otimes \mathbf{h})\mathbf{\Phi }=\frac{1}{F}{\mathbf{\Phi }}^T\mathbf{\Phi }=\frac{1}{F}{\mathit{I}}_T$$

Therefore, these kinds of projectors will be referred to as orthogonal. As we will see later from KLD results and histograms, implementing orthogonal projectors may help us to better preserve both the original shape of the weight distribution and the denoising performance.

3. Theoretical Analysis

From now on, we will omit the sub-index l for the sake of clarity although we are always addressing the coefficients of the embedding layer. The experiments clearly illustrate that the use of Adam optimization together with the watermarking algorithm proposed in [5,6] originates noticeable changes in the distribution of weights, as we see in Figure 2. In the following analysis, we delve into the reasons why this happens. To that end, we aim to get a theoretical expression of $\Delta \mathbf{w}={\mathbf{w}}^{(k)}-{\mathbf{w}}^{(0)}$. This will allow us to prove and understand the nature of the observed behavior of the weights when watermark embedding is carried out. We start off by defining vector $\widehat{\mathit{\phi }}$ and matrix $\widehat{\mathbf{\Psi }}$:

$$\begin{array}{ccc}\hfill \widehat{\mathit{\phi }}& \doteq & \sum _{i=1}^T{\widehat{\mathit{\varphi }}}^{(i)}\hfill \end{array}$$

(11)

$$\begin{array}{ccc}\hfill \widehat{\mathbf{\Psi }}& \doteq & \sum _{i=1}^T{\widehat{\mathit{\varphi }}}^{(i)}{({\widehat{\mathit{\varphi }}}^{(i)})}^T\hfill \end{array}$$

(12)

Notice that $\widehat{\mathbf{\Psi }}={\widehat{\mathbf{\Psi }}}^T$. In addition, for the case of orthogonal projectors, the following properties can be straightforwardly proven:

$$\begin{array}{ccc}\hfill \widehat{\mathbf{\Psi }}\widehat{\mathit{\phi }}& =& \frac{1}{F}\widehat{\mathit{\phi }}\hfill \end{array}$$

(13)

$$\begin{array}{ccc}\hfill \widehat{\mathbf{\Psi }}\widehat{\mathbf{\Psi }}& =& \frac{1}{F}\widehat{\mathbf{\Psi }}\hfill \end{array}$$

(14)

These properties will come in handy later on in several theoretical derivations.

Firstly, in order to simplify the analysis and understand more clearly how the watermarking cost function impacts on the movement of the weights when using both SGD and Adam optimization, we will just consider the presence of the regularization term, that is, we will not include the denoising cost function for now. The influence of the denoising part will be studied in Section 3.3. Therefore, given our embedding cost function in (10) and assuming for simplicity (and without loss of generality) that all embedded symbols are $+1$, we have:

$$\begin{array}{c}\hfill \tilde{f}(\mathbf{w})\doteq \lambda f_{\mathsf{watermark}}(\mathbf{w}|\mathbf{b}=\mathbf{1})=\lambda \sum _{i=1}^{T}log(1+exp(-{({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}))\end{array}$$

If we compute the gradient of this function, we obtain:

$${\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}(\mathbf{w})=-\lambda \sum _{i=1}^T{\widehat{\mathit{\varphi }}}^{(i)}\frac{1}{1+exp\left({({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}\right)}$$

(15)

In order to simplify the subsequent analysis, we introduce a series of assumptions which are based on empirical observations or hypotheses that will be duly verified.

By construction, it is possible to show that the mean of ${({\widehat{\mathit{\varphi }}}^{(i)})}^T{\mathbf{w}}^{(0)}$ is zero and its variance is $(N/F^2)\mathrm{Var}\{{\mathbf{w}}^{(0)}\}$ for Gaussian projectors and $(1/F)\mathrm{Var}\{{\mathbf{w}}^{(0)}\}$ for orthogonal projectors (see Appendix A.1). Since the variance of the weights at the initial iteration is generally very small—in our experiments, it is $0.0012$—it can be considered that the variance of ${({\widehat{\mathit{\varphi }}}^{(i)})}^T{\mathbf{w}}^{(0)}$ will also be small enough so that we can assume $|{({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}|\ll 1$ for all $i=1,\cdots ,T$. Although this assumption might not be strictly true for all k—especially once we have crossed the linear region of the sigmoid function—it is reasonably good and it allows us to use a first-order Taylor expansion for ${\left(1+exp\left({({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}\right)\right)}^{-1}$ around ${({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}=0$:

$${\left(1+exp\left({({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}\right)\right)}^{-1}\approx \frac{1}{2}-\frac{{({\widehat{\mathit{\varphi }}}^{(i)})}^T\mathbf{w}}{4}$$

(16)

Plugging (16) into (15) and using the definitions given above, we can write:

$${\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}(\mathbf{w})\approx -\frac{\lambda }{2}\widehat{\mathit{\phi }}+\frac{\lambda }{4}\widehat{\mathbf{\Psi }}\mathbf{w}$$

(17)

We introduce now one important hypothesis in this theoretical analysis to handle the previous equation: we assume that ${\mathbf{w}}^{(k)}$ grows approximately affinely with k:

$${\mathbf{w}}^{(k)}\approx {\mathbf{w}}^{(0)}+k·\mu ·\mathit{\eta }$$

(18)

where $\mathit{\eta }$ is a vector that contains the slopes for each weight, and it is to be determined in the following sections. We hypothesize this affine-like growth for the weights and, later, we will verify that this is consistent with the rest of the theory and the experiments (see Appendix B.1 for more details). Therefore, we can write the weight variations as:

$$\Delta \mathbf{w}={\mathbf{w}}^{(k)}-{\mathbf{w}}^{(0)}\approx k·\mu ·\mathit{\eta }$$

(19)

3.1. Analysis for SGD

We first analyze the behavior of SGD optimization when we implement digital watermarking as proposed in [5,6]. Recall the SGD update rule in (2). If we use the approximation for the gradient in (17) and the affine growth hypothesis for the weights introduced in (18), we have:

$${\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}(\mathbf{w})\approx -\frac{\lambda }{2}\widehat{\mathit{\phi }}+\frac{\lambda }{4}\widehat{\mathbf{\Psi }}({\mathbf{w}}^{(0)}+k\mu \mathit{\eta })$$

(20)

To simplify the analysis, we consider from now on that ${\mathbf{w}}^{(0)}\approx \mathbf{0}$. We confirm the validity of this assumption in Appendix B.2. Then, we can write:

$$\mathit{\eta }=-{\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}(\mathbf{w})\approx \frac{\lambda }{2}\widehat{\mathit{\phi }}-\frac{\lambda k\mu }{4}\widehat{\mathbf{\Psi }}\mathit{\eta }$$

(21)

If we consider orthogonal projectors, we can arrive at a more explicit expression for $\mathit{\eta }$. In particular, if we multiply (21) by $\widehat{\mathbf{\Psi }}$ and use the properties (13) and (14), we obtain:

$$\widehat{\mathbf{\Psi }}\mathit{\eta }=\frac{2\lambda }{4F-\lambda k\mu }\widehat{\mathit{\phi }}$$

(22)

Then, substituting (22) into (21), we can get a more concise expression for $\mathit{\eta }$:

$$\mathit{\eta }\approx \frac{\lambda }{2}\left(1-\frac{\lambda k\mu }{4F-\lambda k\mu }\right)\widehat{\mathit{\phi }}$$

(23)

Thus, if F is large compared to $\lambda k\mu$—this certainly holds for our experimental set-up, cf. Section 6.1—$\mathit{\eta }$ will be approximately proportional to $\widehat{\mathit{\phi }}$. Then, the coefficients will follow an affine-like growth as we hypothesized in (18) (see Appendix B.1 for the empirical confirmation of this hypothesis). Now, the weight variations can be expressed as:

$$\Delta \mathbf{w}\approx k·\mu ·\mathit{\eta }=\frac{\lambda k\mu }{2}\left(1-\frac{\lambda k\mu }{4F-\lambda k\mu }\right)\widehat{\mathit{\phi }}$$

(24)

As we can see, when we use SGD, $\Delta \mathbf{w}$ will approximately follow a zero-mean Gaussian distribution, as induced by [9]. Because of this, and unlike Adam (as we will see later), the weights will evolve with random speeds when we embed watermarks using SGD optimization. Therefore, the impact on the original shape of the weight distribution will be small. However, the variance of the weight distribution may change considerably as stated in [20]. Since we have $\mathrm{Var}\{\widehat{\mathit{\phi }}\}=T/(FN)$ for orthogonal projectors, the variance of $\Delta \mathbf{w}$ can be computed as:

$$\mathrm{Var}\{\Delta \mathbf{w}\}={\left[\lambda k\mu \frac{(2F-\lambda k\mu )}{(4F-\lambda k\mu )}\right]}^2\frac{T}{FN}$$

Thus, considering that ${\mathbf{w}}^{(0)}$ and $\Delta \mathbf{w}$ are uncorrelated—we check this statement in Section 6.2.2—we arrive at the following expression for the variance of the weights at the kth iteration:

$$\mathrm{Var}\{{\mathbf{w}}^{(k)}\}=\mathrm{Var}\{{\mathbf{w}}^{(0)}\}+{\left[\lambda k\mu \frac{(2F-\lambda k\mu )}{(4F-\lambda k\mu )}\right]}^2\frac{T}{FN}$$

(25)

As we can see from (25), when implementing the digital watermarking algorithm in [5,6] with SGD optimization and orthogonal projectors, the variance of the resulting weight distribution might change considerably. In order to preserve the original weight distribution when using SGD, it is important to take care with the values of T, F and N, especially. In addition, the standard deviation of the weights will (approximately) increase linearly with the number of iterations so it may be also important to limit the value of k. This is in line with the expected behavior: the weights will move away from their original value and they will be further if we perform more iterations.

Because the analysis for Gaussian projectors becomes considerably difficult, in this paper, we just address the study of SGD with orthogonal projectors. A more comprehensive analysis for Gaussian projectors that can be linked to the results obtained in [20] is left for future research. Regardless of this, the whole analysis for both kinds of projection vectors will be developed in the next section for Adam optimization.

3.2. Analysis for Adam

In the next sections, we will delve into the theory behind Adam optimization for DNN watermarking. In particular, we will obtain an expression for the mean and the variance of the gradient and then, as we did with SGD, we will analyze the update term to get an expression of the weight variations.

3.2.1. Mean of the Gradient

We are interested in computing the mean of the gradient that is used in Adam. Considering $\tilde{f}(\mathbf{w})$ as the global cost function, then, from (4), we can rewrite the mean at the kth iteration as:

$${\mathbf{m}}^{(k)}=(1-{\beta }_1)\sum _{i=1}^k{\beta }_1^{k-i}{\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}({\mathbf{w}}^{(i)})$$

(26)

We use the gradient in (17) and do some derivations to find an explicit expression for ${\widehat{\mathbf{m}}}^{(k)}={\mathbf{m}}^{(k)}/(1-{\beta }_1^k)$ under the hypothesis in (18). Finally, we arrive at the following expression for the bias-corrected mean gradient when k is sufficiently large (see Appendix A.2 for all the mathematical details):

$$\begin{array}{ccc}\hfill {\widehat{\mathbf{m}}}^{(k)}& \approx & -\frac{\lambda }{2}\widehat{\mathit{\phi }}+\frac{\lambda }{4}·\widehat{\mathbf{\Psi }}({\mathbf{w}}^{(0)}+k\mu \mathit{\eta })\hfill \\ & =& \frac{\lambda }{2}({\mathbf{m}}_0+\mu ·k·{\mathbf{m}}_1),k\gg \frac{{\beta }_1}{1-{\beta }_1}\hfill \end{array}$$

(27)

where ${\mathbf{m}}_0\doteq -\widehat{\mathit{\phi }}+\frac{1}{2}\widehat{\mathbf{\Psi }}{\mathbf{w}}^{(0)}$ and ${\mathbf{m}}_1\doteq \frac{1}{2}\widehat{\mathbf{\Psi }}\mathit{\eta }$. As we see from (27), the mean of the gradient also grows affinely with k.

3.2.2. Variance of the Gradient

Let $\mathbf{g}(\mathbf{w})\doteq {\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}(\mathbf{w})\circ {\mathbf{\nabla }}_{\mathbf{w}}\tilde{f}(\mathbf{w})$. The approximation in (17) for the jth element of this vector $\mathbf{g}(\mathbf{w})$, denoted by $g_j(\mathbf{w})$, is the following:

$$\begin{array}{ccc}\hfill g_j(\mathbf{w})& \approx & \frac{{\lambda }^2}{4}\sum _{m=1}^T\sum _{l=1}^T{\widehat{\mathit{\varphi }}}_j^{(m)}{\widehat{\mathit{\varphi }}}_j^{(l)}\left(1-{({\widehat{\mathit{\varphi }}}^{(m)})}^T\mathbf{w}+\frac{1}{4}{({\widehat{\mathit{\varphi }}}^{(m)})}^T\mathbf{w}{\mathbf{w}}^T{\widehat{\mathit{\varphi }}}^{(l)}\right)\hfill \\ & =& \frac{{\lambda }^2}{4}\left(a_j-{\mathbf{b}}_j^T\mathbf{w}+\frac{1}{4}{\mathbf{c}}_j^T\mathbf{w}{\mathbf{w}}^T{\mathbf{c}}_j\right)\hfill \end{array}$$

where:

$$\begin{array}{ccc}\hfill a_j& \doteq & {\widehat{\mathit{\phi }}}_j^2\hfill \end{array}$$

(28)

$$\begin{array}{ccc}\hfill {\mathbf{c}}_j& \doteq & \sum _{m=1}^T{\widehat{\mathit{\varphi }}}_j^{(m)}{\widehat{\mathit{\varphi }}}^{(m)}\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\hfill {\mathbf{b}}_j& \doteq & {\widehat{\mathit{\phi }}}_j·{\mathbf{c}}_j\hfill \end{array}$$

(30)

Following the hypothesis in (18), we can write:

$$\begin{array}{c}\hfill g_j({\mathbf{w}}^{(k)})\approx \frac{{\lambda }^2}{4}\left(a_j-{\mathbf{b}}_j^T{\mathbf{w}}^{(0)}+\frac{1}{4}{\mathbf{c}}_j^T{\mathbf{w}}^{(0)}{({\mathbf{w}}^{(0)})}^T{\mathbf{c}}_j-k\mu {\mathbf{b}}_j^T\mathit{\eta }+\frac{1}{2}k\mu {\mathbf{c}}_j^T{\mathbf{w}}^{(0)}{\mathit{\eta }}^T{\mathbf{c}}_j+\frac{1}{4}{k}^2{\mu }^2{\mathbf{c}}_j^T\mathit{\eta }{\mathit{\eta }}^T{\mathbf{c}}_j\right)\end{array}$$

In summary, for this affine-like growth, the square gradient vector can be written as:

$$g_j({\mathbf{w}}^{(k)})\approx \frac{{\lambda }^2}{4}(p_j+q_{j}k\mu +r_j{k}^2{\mu }^2)$$

(31)

for some vectors $\mathbf{p},\mathbf{q},\mathbf{r}$ whose jth component can be defined as:

$$\begin{array}{ccc}\hfill p_j& \doteq & a_j-{\mathbf{b}}_j^T{\mathbf{w}}^{(0)}+\frac{1}{4}{\mathbf{c}}_j^T{\mathbf{w}}^{(0)}{({\mathbf{w}}^{(0)})}^T{\mathbf{c}}_j\hfill \\ \hfill q_j& \doteq & -{\mathbf{b}}_j^T\mathit{\eta }+\frac{1}{2}{\mathbf{c}}_j^T{\mathbf{w}}^{(0)}{\mathit{\eta }}^T{\mathbf{c}}_j\hfill \end{array}$$

(32)

$$\begin{array}{ccc}\hfill r_j& \doteq & \frac{1}{4}{\mathbf{c}}_j^T\mathit{\eta }{\mathit{\eta }}^T{\mathbf{c}}_j\hfill \end{array}$$

(33)

Now, from (5), we can rewrite the variance of the gradient that is used in Adam as:

$$v_j^{(k)}=(1-{\beta }_2)\sum _{i=1}^k{\beta }_2^{k-i}{g}_j({\mathbf{w}}^{(i)})$$

(34)

The bias-corrected term ${\widehat{v}}_j^{(k)}$ is obtained after dividing $v_j^{(k)}$ by $(1-{\beta }_2^k)$. Applied to the special case of (31), this yields (see Appendix A.3):

$$\begin{array}{ccc}\hfill {\widehat{v}}_j^{(k)}& =& \frac{{\lambda }^2}{4}\left(p_j-\frac{{\beta }_2}{1-{\beta }_2}\mu q_j+\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}{\mu }^2{r}_j\right)\hfill \\ & +& \frac{{\lambda }^2}{4(1-{\beta }_2^k)}k\left(\mu q_j-\frac{2{\beta }_2}{1-{\beta }_2}{\mu }^2{r}_j\right)+\frac{{\lambda }^2}{4(1-{\beta }_2^k)}{k}^2{\mu }^2{r}_j\hfill \end{array}$$

3.2.3. Update Term

Because $\mu$ is usually very small—we use $\mu =1·{10}^{-6}$ in our experiments—we can assume that $k\mu$ will be small enough to obtain an approximation of the update used in Adam. Recall that, for the jth weight, this is $u_j^{(k)}\doteq {\widehat{m}}_j^{(k)}/\left(\sqrt{{\widehat{v}}_j^{(k)}}+ϵ\right)$, implying that $w_j^{(k)}=w_j^{(k-1)}-\mu u_j^{(k)}$. Let:

$$s_j\doteq \left(p_j-\frac{{\beta }_2}{1-{\beta }_2}\mu q_j+\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}{\mu }^2{r}_j\right)$$

(35)

Then, assuming that $\mu k\ll 1$, we can make a zero-order approximation of the update term, i.e.,:

$$u_j^{(k)}=\frac{{\widehat{m}}_j^{(k)}}{\sqrt{{\widehat{v}}_j^{(k)}}+ϵ}\approx \frac{m_{0,j}}{\sqrt{s_j}},\mu k\ll 1$$

(36)

This approximation is accurate enough for the set of experiments we perform. In particular, for the orthogonal case, we could deal with $k_{max}$ = 625,000 and still get a correlation coefficient of $0.9900$ between $({\lambda }^2/4)s_j$ and ${\widehat{v}}_j^{(k_{max})}$. In our experiments, we actually reach a BER of zero for values of k quite below $k_{max}$ (cf. Section 6.2).

From (36), we observe that the updated jth coefficient approximately follows the hypothesized growth, i.e., ${\mathbf{w}}^{(k)}={\mathbf{w}}^{(0)}+k·\mu ·\mathit{\eta }$, where ${\mathit{\eta }}_j=-m_{0,j}/\sqrt{s_j}$. Notice that, as expected, the update does not depend on $\lambda$, following Adam’s property that the update is invariant to rescaling the gradients [9]. Finding a more explicit expression runs into the problem that $\mathit{\eta }$ depends on $\mathbf{s}$, which in turn is a function of $\mathit{\eta }$ through (32) and (33). The following subsections are devoted to solving this problem by conjecturing a form for $\mathit{\eta }$ and refining it.

To simplify the analysis, we consider from now on that ${\mathbf{w}}^{(0)}\approx \mathbf{0}$ since most of the values of the weights at the initial iteration are very small (see Figure 2a). We will verify the accuracy of this approximation in Appendix B.2.

3.2.4. Rationale for the Sign Function

Recall the expression (23) that we obtained for $\mathit{\eta }$ when analyzing SGD, where we found $\mathit{\eta }$ to be approximately proportional to $\widehat{\mathit{\phi }}$. Now, for Adam, we take this as a starting point, so we conjecture first that $\mathit{\eta }=\gamma ·\widehat{\mathit{\phi }}$, for some real positive $\gamma$. Here, we consider orthogonal projection vectors and use the property introduced in (13) and the following:

$${\mathbf{c}}_j^T\widehat{\mathit{\phi }}=\frac{{\widehat{\mathit{\phi }}}_j}{F}$$

In this particular case, we have the following identities:

$$\begin{array}{ccc}\hfill m_{0,j}& \approx & -{\widehat{\mathit{\phi }}}_j\hfill \\ \hfill p_j& \approx & a_j={\widehat{\mathit{\phi }}}_j^2\hfill \\ \hfill q_j& \approx & -\frac{\gamma }{F}{\widehat{\mathit{\phi }}}_j^2\hfill \\ \hfill r_j& =& \frac{{\gamma }^2}{4F^2}{\widehat{\mathit{\phi }}}_j^2\hfill \end{array}$$

Substituting these values into (35), we find that

$$\sqrt{s_j}=\mathrm{sgn}({\widehat{\mathit{\phi }}}_j)·{\widehat{\mathit{\phi }}}_j·\sqrt{1+(\gamma /F)\mu {\beta }_2/(1-{\beta }_2)+({\gamma }^2/(4F^2)){\mu }^2({\beta }_2+{\beta }_2^2)/{(1-{\beta }_2)}^2}$$

When we divide $m_{0,j}$ by $\sqrt{s_j}$, we obtain:

$$u_j^{(k)}\approx \frac{-\mathrm{sgn}({\widehat{\mathit{\phi }}}_j)}{\sqrt{1+(\gamma /F)\mu {\beta }_2/(1-{\beta }_2)+({\gamma }^2/(4F^2)){\mu }^2({\beta }_2+{\beta }_2^2)/{(1-{\beta }_2)}^2}}$$

(37)

It is then clear that $\mathit{\eta }$ cannot be written in the form $\mathit{\eta }=\gamma ·\widehat{\mathit{\phi }}$, as was conjectured at the beginning of this section.

3.2.5. A Theoretical Expression for $\Delta \mathbf{w}$

Although the conjectured form for $\mathit{\eta }$ in Section 3.2.4 does not hold, the appearance of the sign function in (37) gives a key clue for an alternative approach, since the sign seems to reveal the reason behind the two-spiked histograms like the one shown in Figure 2c.

Therefore, let us write ${\mathit{\eta }}_j$ to explicitly contain the sign of ${\mathit{\phi }}_j$ and allow ${\gamma }_j$ to take different (non-negative) values with j to reflect the varying magnitude (recall that even in Section 3.2.4 the conjectured value could be written as ${\mathit{\eta }}_j=\gamma |{\mathit{\phi }}_j|·\mathrm{sgn}({\mathit{\phi }}_j)$). Let $\gamma$ be the column vector containing ${\gamma }_j$, $j=1,\cdots ,N$, then $\mathit{\eta }=\gamma \circ \mathrm{sgn}(\widehat{\mathit{\phi }})$. Since ${\mathbf{c}}_j^T(\gamma \circ \mathrm{sgn}(\widehat{\mathit{\phi }}))={\gamma }^T({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }}))$, we can write:

$$\begin{array}{ccc}\hfill m_{0,j}& \approx & -{\widehat{\mathit{\phi }}}_j\hfill \\ \hfill p_j& \approx & a_j={\widehat{\mathit{\phi }}}_j^2\hfill \\ \hfill q_j& \approx & -{\mathbf{b}}_j^T\mathit{\eta }=-{\widehat{\mathit{\phi }}}_j{\gamma }^T({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }}))\hfill \\ \hfill r_j& =& \frac{1}{4}{\left[{\gamma }^T({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }}))\right]}^2\hfill \end{array}$$

Now,

$$u_j^{(k)}\approx \frac{-{\widehat{\mathit{\phi }}}_j}{\sqrt{{\widehat{\mathit{\phi }}}_j^2+\mu {\widehat{\mathit{\phi }}}_j{\gamma }^T\left({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }})\right)\frac{{\beta }_2}{(1-{\beta }_2)}+\frac{{\mu }^2}{4}{\left[{\gamma }^T({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }}))\right]}^2\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}}}$$

In addition, thus, in order to meet the condition ${\mathit{\eta }}_j={\gamma }_j·\mathrm{sgn}({\widehat{\mathit{\phi }}}_j)$, the following nonlinear equation should be solved for all ${\gamma }_j$, $j=1,\cdots ,N$:

$${\gamma }_j=\frac{|{\widehat{\mathit{\phi }}}_j|}{\sqrt{{\widehat{\mathit{\phi }}}_j^2+\mu {\widehat{\mathit{\phi }}}_j{\gamma }^T\left({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }})\right)\frac{{\beta }_2}{(1-{\beta }_2)}+\frac{{\mu }^2}{4}{\left[{\gamma }^T({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }})\right]}^2\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}}}$$

(38)

This equation can be solved with a fixed-point iteration method [21]. To that end, we should initialize $\gamma$ and then iterate the following: (1) compute the right-hand side of (38), and (2) use it to update $\gamma$ on the left-hand side. This process will converge to the solution of (38). Even though this method can be implemented to give the specific values for each ${\gamma }_j$, we are more interested in obtaining a statistical characterization rather than a deterministic one. As we will see, the statistical approach offers a deeper explanation for the two-spiked distribution of $\Delta \mathbf{w}$ which we ultimately seek.

We thus aim at finding the pdf of $\Gamma$, now considered as a random variable for which ${\gamma }_j$, $j=1,\cdots ,N$ are nothing but realizations. Once again, Equation (38) can be solved iteratively (e.g., with Markov-chain Monte Carlo methods [22]) to yield the equilibrium distribution for $\Gamma$. Instead, we can resort to the results in Section 6 where we conclude that the pdf of $\Gamma$ is strongly concentrated around its mode. With this observation, it is possible to consider that ${\gamma }^T({\mathbf{c}}_j\circ \mathrm{sgn}(\widehat{\mathit{\phi }}))$ approximately corresponds to realizations of $\Gamma ·({\mathbf{c}}_j^T\mathrm{sgn}(\widehat{\mathit{\phi }}))$.

In order to simplify the analysis even further, we are interested in decomposing ${\mathbf{c}}_j^T\mathrm{sgn}(\widehat{\mathit{\phi }})$ using its statistical projection onto ${\widehat{\mathit{\phi }}}_j$, i.e., ${\mathbf{c}}_j^T\mathrm{sgn}(\widehat{\mathit{\phi }})=\alpha ·{\widehat{\mathit{\phi }}}_j+z_j$. Here, $\alpha$ is a real multiplier and $z_j$ is zero-mean noise uncorrelated with ${\widehat{\mathit{\phi }}}_j$. More generally, if we define matrix $\mathbf{C}\doteq [{\mathbf{c}}_0,\cdots ,{\mathbf{c}}_N]$, then we seek to write ${\mathbf{C}}^T\mathrm{sgn}(\widehat{\mathit{\phi }})=\alpha ·\widehat{\mathit{\phi }}+\mathbf{z}$. We do the analysis for the cases of Gaussian projectors and orthogonal projectors separately (refer to Appendix A.4 for the derivations). For the Gaussian case, we get:

$$\begin{array}{ccc}\hfill \alpha & =& \frac{1}{FT}\sqrt{\frac{2}{\pi T}}\left(F{T}^2+(N+F)T-F\right)\hfill \end{array}$$

(39)

$$\begin{array}{ccc}\hfill \mathrm{Var}(z_j)& =& -\frac{2}{\pi F^3{T}^2}\left(F{T}^4+4F{T}^3+(N+F)T^2-4FT+F\right)+\frac{T}{F^3}(N+F(T+1))\hfill \end{array}$$

(40)

On the other hand, for the orthogonal projectors, we get instead:

$$\begin{array}{ccc}\hfill \alpha & =& \sqrt{\frac{2N}{\pi TF}}\hfill \end{array}$$

(41)

$$\begin{array}{ccc}\hfill \mathrm{Var}(z_j)& =& \left(1-\frac{2}{\pi }\right)\frac{T}{FN}\hfill \end{array}$$

(42)

Recall that, by construction, $\widehat{\mathit{\phi }}$ can be seen as a random vector. In fact, we have $\widehat{\mathit{\phi }}\sim \mathcal{N}(\mathbf{0},\mathbf{I}·T/F^2)$ for Gaussian projection vectors, and $\widehat{\mathit{\phi }}$ approximately follows $\mathcal{N}(\mathbf{0},\mathbf{I}·T/(FN))$ for orthogonal projectors. Let $\Xi$, Z be random variables with the distribution of a single element of $\widehat{\mathit{\phi }}$ and $\mathbf{z}$, respectively, then $q_j$ and $r_j$ can be seen as realizations of (approximately): $Q=-\Gamma \Xi (\alpha \Xi +Z)$ and $R=\frac{{\Gamma }^2}{4}{(\alpha \Xi +Z)}^2$, so a stochastic version of (38) is:

$$\Gamma =\frac{|\Xi |}{\sqrt{{\Xi }^2+\Gamma \mu \Xi \left(\alpha \Xi +Z\right)\frac{{\beta }_2}{(1-{\beta }_2)}+\frac{{\Gamma }^2{\mu }^2}{4}{\left(\alpha \Xi +Z\right)}^2\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}}}$$

Squaring both sides, we find that, for a given realization ($\xi$, z) of ($\Xi$, Z), $\Gamma$ must take the positive value $\gamma$ that satisfies the following fourth degree equation:

$$\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}\frac{{\mu }^2}{4}{\left(\alpha \xi +z\right)}^2{\gamma }^4+\frac{{\beta }_2}{1-{\beta }_2}\mu \xi \left(\alpha \xi +z\right){\gamma }^3+{\xi }^2{\gamma }^2-{\xi }^2=0$$

(43)

From (43), it is easy to generate samples $\gamma$ of $\Gamma$ and, accordingly, samples of $\Delta \mathbf{w}$, by recalling that:

$$\Delta w\approx k·\mu ·\gamma ·\mathrm{sgn}(\xi )$$

(44)

We note that, for the particular case when ${\beta }_2$ is very close to 1, $\frac{{\beta }_2+{\beta }_2^2}{{(1-{\beta }_2)}^2}\approx 2{\left(\frac{{\beta }_2}{1-{\beta }_2}\right)}^2$. This simplification allows us to approximate (43) as

$${\gamma }^2{\left(\frac{\gamma \mu }{2}·\frac{{\beta }_2}{1-{\beta }_2}(\alpha \xi +z)+\xi \right)}^2-{\xi }^2\approx 0$$

(45)

which leads to the following fixed-point equation:

$$\gamma \approx \frac{\xi }{\frac{\gamma \mu }{2}·\frac{{\beta }_2}{1-{\beta }_2}(\alpha \xi +z)+\xi }$$

(46)

When the noise term z is very small compared to $\alpha \xi$ (which occurs with a fairly large probability, especially for the case of orthogonal projectors), then the solution to (46), denoted by ${\gamma }_s$, will be independent of the value of $\xi$. This will cause the probability of $\Gamma$ to be concentrated around ${\gamma }_s$, and in turn this will make the pdf $\Delta w$ have two spikes centered at $\pm k\mu {\gamma }_s$. We will see these spikes appearing time and again in the experiments carried out with Adam (Section 6).

3.3. The Denoising Term

Thus far, we have considered only that our cost function is $\tilde{f}(\mathbf{w})=\lambda f_{\mathsf{watermark}}(\mathbf{w})$; however, as we know, there is an additional term, the original denoising function, so our real cost function is: $f(\mathbf{w})=f_{\mathsf{0}}(\mathbf{w})+\tilde{f}(\mathbf{w})$.

The gradients corresponding to this function, $f_{\mathsf{0}}(\mathbf{w})$, will try to pull the weight vector towards the original optimal ${\mathbf{w}}^{(0)}$ in a relatively hard to model way. In order to analyze this behavior, we can approximate the gradient of the denoising function at the kth iteration with respect to the jth coefficient as a sum of a constant term, $d_j$, and a noisy one, ${\tilde{n}}_j^{(k)}$, which follows a zero-mean Gaussian distribution and is associated with the use of different training batches on each step. We will refer to this noise as batching noise. Thus, for each coefficient j, we can write:

$$\frac{\partial f_{\mathsf{0}}({\mathbf{w}}^{(k)})}{\partial w_j}\approx d_j+{\tilde{n}}_j^{(k)}$$

(47)

Like we did in the previous section, we can formulate a stochastic version of (47). To that end, we notice that the constant term of this gradient, $d_j$, can take different values with j, as well as the variance of the batching noise, $h_j$ that is, ${\tilde{n}}_j$ is drawn from $\mathcal{N}(\mathbf{0},h_j)$. Therefore, in order to reflect the variability of these terms along the j-elements, we introduce two random variables with the distribution of the mean gradient and the variance of the batching noise, D and H, respectively, for which $d_j$ and $h_j$ are realizations. The pdf of these distributions will be obtained empirically in Section 6.2. Then, we can see ${\tilde{n}}_j^{(k)}$ as a realization of $\tilde{N}\sim \mathcal{N}(0,H)$.

3.3.1. SGD

Similarly to Section 3.2.5, let $\Xi$ be a random variable with the distribution of $\widehat{\mathit{\phi }}$. Let ($\xi$, $\delta$, $\tilde{n}$) be a realization of ($\Xi$, D, $\tilde{N}$), respectively, then for SGD using orthogonal projectors we can compute samples of $\Delta \mathbf{w}$ adding both functions, i.e., denoising and watermarking:

$$\Delta w\approx k\mu (\delta +\tilde{n})+\frac{\lambda k\mu }{2}\left(1-\frac{\lambda k\mu }{4F-\lambda k\mu }\right)\xi$$

(48)

3.3.2. Adam

The variance of the batching noise computed by Adam will be approximately given by the random variable V, whose realizations can be expressed as $v_j=\frac{1-{\beta }_2}{1-{\beta }_2^k}{\sum }_{i=1}^k{\beta }_2^{(k-i)}{({\tilde{n}}_j^{(i)})}^2$. Notice that, for each realization of V, as the sum takes places over i, we must work with a fixed value $h_j$ for the variance of the batching noise. Then, with this variance, we generate k samples of $\tilde{N}$ to be used in the sum that produces $v_j$. With this characterization, we can easily analyze how the denoising cost function shapes the distribution of the weight variations. Notice that this analysis could be adapted for any host network. Let $\delta$ and $\nu$ be realizations of D and V, respectively, then, we can generate samples of $\Delta \mathbf{w}$ without including the gradients from the watermarking function as:

$$\Delta w\approx k·\mu ·\frac{\delta }{\sqrt{{\delta }^2+\nu }}$$

Moreover, in order to get a more accurate description of the problem, we can combine both functions: denoising and watermarking. The analysis becomes somewhat complicated, but, as we will check in Section 6, the distributions resulting from this analysis do capture better the shapes observed in the empirical ones. See Appendix A.5 for the results of this analysis.

4. Block-Orthonormal Projections (BOP)

Here, we discuss BOP, the solution we propose to solve the detectability problem posed by Adam optimization when implementing the watermarking algorithm proposed in [5,6]. In order to hide the noticeable weight variations that appear when we use Adam—as seen in Figure 2—we introduce a prior transformation using a secret $N\times N$ matrix $\mathbf{X}$ (the details for its construction are given below). The procedure we follow has three steps per each iteration of Adam.

Firstly, we project the weights and gradients from the embedding layer using $\mathbf{X}$:

$$\begin{array}{ccc}\hfill \mathbf{y}& =& \mathbf{X}\mathbf{w}\hfill \\ \hfill {\mathbf{\nabla }}_{\mathbf{y}}f(\mathbf{y})& =& \mathbf{X}{\mathbf{\nabla }}_{\mathbf{w}}f(\mathbf{w})\hfill \end{array}$$

Then, we run Adam optimization on the projected weights, $\mathbf{y}$, using the projected gradients, ${\mathbf{\nabla }}_{\mathbf{y}}f(\mathbf{y})$, as well, i.e., steps (3)–(8) are taken using $\mathbf{y}$ and ${\mathbf{\nabla }}_{\mathbf{y}}f({\mathbf{y}}^{(k-1)})$ instead of $\mathbf{w}$ and ${\mathbf{\nabla }}_{\mathbf{w}}f({\mathbf{w}}^{(k-1)})$, respectively. The key of BOP relies on the following: if we execute Adam on $\mathbf{y}$ instead of $\mathbf{w}$, we can break the natural bond created by Adam between $\mathrm{sgn}(\widehat{\mathit{\phi }})$ and $\mathbf{w}$—as we saw in the previous sections—responsible for the ant-like behavior of the weights and, consequently, the appearance of side spikes in their histograms. These undesired effects disappear when we de-project $\mathbf{y}$ using ${\mathbf{X}}^{-1}$ to get back to the weight vector $\mathbf{w}$:

$$\begin{array}{ccc}\hfill \mathbf{w}& =& {\mathbf{X}}^{-1}\mathbf{y}\hfill \end{array}$$

In order to reduce the computational complexity and the memory requirements of this method—recall that N is generally a very large number and we must project and de-project the weights on each iteration—we consider $\mathbf{X}$ to be a block diagonal matrix with B identical $\frac{N}{B}\times \frac{N}{B}$ blocks. In this way, we only have to build and work with a single block ${\mathbf{X}}_B$, for which we can choose the size by simply adjusting the value of B. The values of this block are drawn from a standard normal distribution. In addition, ${\mathbf{X}}_B$ is built as an orthonormal matrix so that ${\mathbf{X}}_B^{-1}={\mathbf{X}}_B^T$. Let ${\mathbf{y}}^{(i)}$ and ${\mathbf{w}}^{(i)}$ be the ith block of $\mathbf{y}$ and $\mathbf{w}$, respectively, both of them of length $\frac{N}{B}$; therefore, we just compute:

$$\begin{array}{ccc}\hfill {\mathbf{y}}^{(i)}& =& {\mathbf{X}}_B{\mathbf{w}}^{(i)}\hfill \\ \hfill {\mathbf{\nabla }}_{{\mathbf{y}}^{(i)}}f({\mathbf{y}}^{(i)})& =& {\mathbf{X}}_B{\mathbf{\nabla }}_{{\mathbf{w}}^{(i)}}f({\mathbf{w}}^{(i)})\hfill \end{array}$$

After executing Adam, we can get back to ${\mathbf{w}}^{(i)}$:

$$\begin{array}{ccc}\hfill {\mathbf{w}}^{(i)}& =& {\mathbf{X}}_B^T{\mathbf{y}}^{(i)}\hfill \end{array}$$

As we will see in Section 6.2.4, BOP does not significantly alter the original distribution of weights, as opposed to standard Adam. This makes it possible to enjoy the advantages of Adam optimization when we implement the watermarking algorithm in [5,6] with a minimal increase in the detectability of the watermark. In addition, this has an advantage in terms of robustness: if the adversary is not able to infer which layer is watermarked, then he/she will have to exert his/her attack (e.g., noise addition, weight pruning) on every layer thus producing a larger impact on the performance of the network as measured by the original cost function. We will discuss this fact in the experimental section.

5. Information-Theoretic Measures

As already discussed, one of the potential weaknesses of any neural network watermarking algorithm is the detectability of the watermark. An adversary that detects the presence of a watermark on a certain subset of the weights can initiate an attack to remove or alter the watermark. For this reason, it is important that the weights statistically suffer the least modification possible while of course being able to convey the desired hidden message. To measure this statistical closeness, we propose using the KLD [13] between the distributions of weights before and after the watermark embedding. Let P and Q be two discrete probability distributions defined on the same alphabet $\mathcal{X}$; then, the KLD from Q to P is (notice that it is not symmetric):

$$\mathrm{KLD}(P||Q)\doteq \sum _{x\in \mathcal{X}}P(x)log\left(\frac{P(x)}{Q(x)}\right)$$

The KLD is always non-negative. The more similar the distributions P and Q are, the smaller the divergence. In the extreme case of two identical distributions, the divergence is zero.

It is interesting to note that the KLD has been proposed for similar problems in forensics, including steganographic security [23], distinguishability between forensic operators [24], or more general source identification problems [25].

In our case, the two compared distributions are those of ${\mathbf{w}}^{(0)}$ and ${\mathbf{w}}^{(k)}$, for k just producing convergence with no decoding errors. Since the KLD is not symmetric, it remains to assign those distributions to P and Q so that the measure is as informative as possible. In particular, we are interested in properly accounting for the possible lateral spikes in the pdf of ${\mathbf{w}}^{(k)}$. As those spikes often appear where the pdf of ${\mathbf{w}}^{(0)}$ is small if not negligible, this suggests assigning the latter pdf to Q and the former to P. However, this choice creates a problem in practice, as for some $x\in \mathcal{X}$, the empirical probabilities are such that $P(x)\ne 0$ and $Q(x)=0$, potentially leading to an infinite divergence. To circumvent this issue related to insufficient sampling, we use an analytical approximation to Q with infinite support, after noticing that the empirical distribution of ${\mathbf{w}}^{(0)}$ with 1000 discrete bins (see Figure 2a) can be approximated by a zero-mean Generalized Gaussian Distribution (GGD) with shape parameter $\beta =0.64$ and scale parameter $\alpha =0.01$, (for notational coherence with the literature, $\alpha$ is used in this section to denote a different quantity than in the rest of the paper.) for which the latter controls the spread of the distribution. As a reference, the KLD between the empirical distribution of ${\mathbf{w}}^{(0)}$ and its GGD best-fit is $0.0177$, which is smaller than any of the KLDs that we find in

Table 2. PSNR (dB) results with noise level $\sigma =25$, number of iterations k needed to converge, KLD and SIKLD between the distributions of ${\mathbf{w}}^{(k)}$ and ${\mathbf{w}}^{(0)}$.

	SGD		Adam				BOP
	Gaussian	Orth.	Gaussian		Orth.		Gaussian		Orth.
$\lambda$	5	20	$0.05$	1	$0.5$	10	$0.05$	1	$0.5$	10
CBSD68	$30.76$	$31.09$	$31.18$	$31.17$	$31.21$	$31.16$	$31.20$	$31.16$	$31.19$	$31.15$
Kodak24	$31.66$	$32.03$	$32.13$	$32.10$	$32.15$	$32.10$	$32.15$	$32.08$	$32.14$	$32.09$
k	42,780	98,510	43,590	32,140	27,110	57,230	40,880	14,840	33,180	7150
KLD	$0.0477$	$0.0238$	$0.1149$	$0.2779$	$0.0281$	$0.8118$	$0.0463$	$0.0443$	$0.0227$	$0.0253$
SIKLD	$0.0468$	$0.0206$	$0.0879$	$0.2112$	$0.0280$	$0.4707$	$0.0449$	$0.0266$	$0.0197$	$0.0226$

. In order to compute the KLD in our experiments, we use this infinite-support symmetric distribution for Q and the empirical one of ${\mathbf{w}}^{(k)}$ for P after quantizing both to 1000 discrete bins.

The use of the KLD is adequate to measure the detectability in those cases where the adversary has access to information about the ’expected’ distribution of the weights. For instance, when only one layer is modified, the expected distribution can be inferred from the weights of other layers. However, this may be still too optimistic in terms of adversarial success, as while the expected shape may be preserved—and thus, inferred—across layers, the scale (directly affecting the variance) may be not so. For instance, if the original weights were expected to be zero-mean Gaussian and they still are after watermarking, the KLD (which depends on the ratio of the respective variances) may be quite large, but the adversary will not be able to determine if watermarking took place if he/she does not know what the variance should be and only measures divergence with respect to a Gaussian. To reflect this uncertainty, quite realistic in practical situations, we minimize the KLD with respect to the scale parameter $\alpha$. This puts the adversary in a scenario where only the shape is used for detectability. Thus, let $Q_{\alpha }$ correspond to a GGD with scale parameter $\alpha$, then we define the Scale Invariant KLD (SIKLD) as:

$$\mathrm{SIKLD}\doteq \underset{\alpha }{min}\mathrm{KLD}(P||Q_{\alpha })=\underset{\alpha }{min}\sum _{x\in \mathcal{X}}P(x)log\left(\frac{P(x)}{Q_{\alpha }(x)}\right)$$

6. Experiments and Results

In this section, we show the experimental results and we compare them to the theory that we have developed. We use MATLAB R2018b to implement the expressions obtained in Section 3 and represent the theoretical histograms. As we will see, both theory and experiments match reasonably well. In particular, for Adam optimization, we are able to reproduce the same position of the side spikes seen in the empirical histograms of $\Delta \mathbf{w}$, as well as some effects which are attributable to the influence of the denoising cost function. We will also verify the BOP method proposed in Section 4. In addition, the KLD will be computed to give a more precise measure of the similarity between the distributions of ${\mathbf{w}}^{(k)}$ (i.e., after the embedding) and ${\mathbf{w}}^{(0)}$, when using SGD, Adam and BOP.

6.1. Experimental Set-Up

We employ the fine-tune-to-embed approach described in [5,6]. This means that the training process is divided into two phases, as we explained earlier: (1) training the host network from scratch, and (2) fine-tuning steps for embedding the watermark.

6.1.1. Training the Host Network

In order to perform the initial training of the host network FFDNet, we use the open-source implementation for PyTorch provided in [26]. We employ the FFDNet architecture for color images, which has a depth of 12 convolutional layers and 96 feature maps per layer. The training details are the same as in [26] and also the used datasets: Waterloo Exploration Database [27] for training and Kodak24 [28] for validation. We implement the cost function introduced in (1) and train 80 epochs with the milestones described in [26] on a GPU NVIDIA Titan Xp. We use Adam as the optimization algorithm with its hyperparameters set to their default values. After training the network, we test it on the CBSD68 [29] and Kodak24 datasets.

6.1.2. Watermark Embedding

Once we have trained and tested our host network, we embed our T-bit watermark, $\mathbf{b}=\mathbf{1}$, $T=256$, into the convolutional layer $l=2$ of FFDNet. In the next section, we present the results for both SGD and Adam optimization algorithms. The size of the convolutional filter is $3\times 3$ and the depth of input is $I=96$, as well as the number of filters in the layer, $F=96$. Therefore, we have: $M=96·3·3=864$, and $N=MF$ = 82,944. In addition, the learning rate $\mu$ is set to ${10}^{-6}$ during these fine-tuning steps and, also, we do not perform weight orthogonalization as we did during the initial training.

In addition, we use the following values for the regularizer parameter. When we use SGD we set $\lambda =5$ and $\lambda =20$ for Gaussian and orthogonal projectors, respectively. In addition, for Adam optimization, we use different values of $\lambda$ for each configuration to better reflect the influence of the denoising function. In particular, we set $\lambda =0.05$ and $\lambda =1$ when we use Gaussian projectors, and $\lambda =0.5$ and $\lambda =10$ when we employ orthogonal projectors. We finish our embedding process when we reach a BER of zero, that is, when all the projected weights are positive—recall that all the embedded bits are set to $+1$—, i.e., ${({\widehat{\mathit{\varphi }}}^{(m)})}^T\mathbf{w}>0$, for all $m=1,\cdots ,T$. Notice that these values of $\lambda$ were selected with the goal of reaching a BER of zero in a relatively fast way and, as it can be seen, they are not straightforwardly comparable for Gaussian and orthogonal projectors. Finally, to check the validity of our proposed method BOP, we use the same values of $\lambda$ as with Adam optimization, and set the number of blocks B to 12.

6.2. Experimental Results

Here, we present the experimental results. After the main training of the host network and before the watermark embedding, we obtain a PSNR of $31.18$ dB and $32.13$ dB on the CBSD68 and Kodak24 datasets, respectively, for a noise level of 25. These results are very close to those reported in [26]. Compare these values to those in Table 2, where we show the PSNR (dB) results on the CBSD68 and Kodak24 datasets for the same noise level after the watermark embedding was performed. As we see, when we embed the watermark using SGD with Gaussian projectors, the denoising performance drops about $0.45$ dB, while, if we employ orthogonal projectors, the performance drops only $0.1$ dB. Thus, employing orthogonal projectors with SGD optimization might be beneficial to better preserve the denoising performance. For Adam optimization and BOP, the original performance does not significantly drop and it even increases when using orthogonal projectors. Consequently, in order to keep a good performance after the watermark embedding, Adam would be preferable to SGD were it not for the conspicuousness of the weights. Our proposed method BOP is a good solution to bring the detectability of Adam down to similar levels as SGD and still enjoy the rest of advantages.

Table 2 also presents the number of iterations required for obtaining a BER of zero and the KLD and Scale Invariant KLD (SIKLD) between the distributions of ${\mathbf{w}}^{(k)}$ and ${\mathbf{w}}^{(0)}$ for each configuration.

6.2.1. Empirical Denoising Gradients

In order to analyze the influence of the denoising function, we need to get the empirical distributions of the mean denoising gradient and the variance of the batching noise. To that end, we proceed as follows: firstly, we extract the denoising gradients from the embedding layer $l=2$ and we average them for each coefficient over the number of iterations k to get the distribution of the mean. Then, the batching noise can be easily computed if, for each coefficient, we subtract the mean value from its corresponding denoising gradient value at each iteration. By computing the variance of this noise for each individual weight, we can estimate the overall distribution of the variance of the batching noise. Figure 3 shows the empirical distribution of the mean denoising gradient, D, and the variance of the batching noise, H.

6.2.2. SGD

We embed our watermark using SGD and its corresponding set-up, as we detailed in Section 6.1.2. We show the resulting histograms of ${\mathbf{w}}^{(k)}$ and $\Delta \mathbf{w}$ in Figure 4. As we can see, SGD does not significantly alter the original distribution of weights shown in Figure 2a. This is also reflected in the SIKLD, which is very small, especially when we use orthogonal projectors (see Table 2).

Now, we check the theory we developed for SGD in Section 3.1. Recall that out theoretical analysis just covers the case of orthogonal projectors. Using (24), we can generate samples of $\Delta \mathbf{w}$ without including the effect of the denoising cost function. The resulting histogram is shown in Figure 5a. Notice that the unusual appearance of this histogram can be attributed to the effects of applying the initial transformation explained in Section 2.3.1. In particular, each value of $\mathit{\phi }$ repeats F times to form vector $\widehat{\mathit{\phi }}$; hence, the discrete values in the y-axis of the histogram shown in Figure 5a. However, we see that the range of values of this theoretical histogram fits quite well the empirical one (Figure 4c). In order to get a more accurate representation, we can generate samples of $\Delta \mathbf{w}$ according to (48), so that we add the effect of noise coming from the denoising cost function. As we see in Figure 5b, the resulting histogram is now very similar to the one in Figure 4c.

In addition, we confirm that (25) can be used to compute the variance of the distribution of ${\mathbf{w}}^{(k)}$ when we implement orthogonal projectors. Firstly, we check the hypothesis that we made regarding the uncorrelatedness between ${\mathbf{w}}^{(0)}$ and $\Delta \mathbf{w}$. For our particular case of $\lambda =20$, the correlation coefficient between ${\mathbf{w}}^{(0)}$ and $\Delta \mathbf{w}$ is $2.539·{10}^{-4}$, a very small value that confirms our assumption. Using (25), we have that the variance of the empirical distribution of ${\mathbf{w}}^{(k)}$—red histogram in Figure 4c—is $1.192·{10}^{-3}$ while the theoretical variance is $1.193·{10}^{-3}$. As we see, these values are almost identical.

6.2.3. Adam

In the following experiments, we employ Adam optimization for the watermark embedding and use the same settings as in Section 6.1.2. The resulting histograms of ${\mathbf{w}}^{(k)}$ and $\Delta \mathbf{w}$ are shown in Figure 6 and Figure 7, respectively. As it can be observed, the shape of the weight distribution changes to a great extent for $\lambda =1$ and $\lambda =10$. For smaller values of $\lambda$, since the influence of the watermarking cost function is weaker, we can avoid having a significant alteration to the original distribution shape. This is also reflected on their SIKLD values in Table 2: as $\lambda$ increases the SIKLD also increases considerably. However, notice that, whatever the value of $\lambda$ is, the histograms of weight variations always present the characteristic side spikes. These footprints left by Adam can increase the detectability of the watermark. In addition, when $\lambda$ is small, we can observe in these histograms the influence of the denoising cost function: it causes the appearance of a central peak with values that spread till the location of both side spikes.

Figure 8 represents the pdf of $\Gamma$ obtained from (43). Notice that, as we stated in Section 3.2.5, the pdf is concentrated around its mode. Figure 9 shows the histograms of $\Delta \mathbf{w}$ obtained from (44) when only the watermarking loss $\tilde{f}(\mathbf{w})$ is optimized and the denoising component is set to zero. Compare these histograms to those in Figure 7: the theory developed in Section 3.2 is able to explain the two-spiked distributions of $\Delta \mathbf{w}$. Notice that these theoretical expressions provide a good enough approximation since they allow us to predict the position of the side spikes. We show in

Table 3. Position of both side spikes in the histograms of $\Delta \mathbf{w}$ obtained from theoretical and empirical results.

	$\mathit{\lambda }$	Theoretical	Empirical
Gaussian	$0.05$	$-0.04243$	$-0.04278$
		$0.04239$	$0.04276$
	1	$-0.03129$	$-0.03119$
		$0.03126$	$0.03125$
Orthogonal	$0.5$	$-0.02710$	$-0.02710$
		$0.02710$	$0.02709$
	10	$-0.05720$	$-0.05717$
		$0.05720$	$0.05718$

the values of these positions obtained from both theoretical (Figure 9) and empirical (Figure 7) results. As we can see, these side spikes are placed in almost identical positions by both theory and experiments, hence, we can confirm that the sign phenomenon in Adam is responsible for this ant-like behavior shown by the weights at the embedding layer.

Finally, in order to reflect the influence of the denoising function and obtain more realistic histograms, we can solve the fourth-degree Equation (A11) and, then, generate samples of $\Delta \mathbf{w}$ according to (A12). The resulting histograms are shown in Figure 10 and are now quite similar to those in Figure 7. As it can be seen, we can emulate the central peak and the dispersion of the values of the side spikes in the histograms and, thus, we can confirm that these effects are attributable to the influence of the denoising cost function.

6.2.4. BOP

Here, we represent the empirical histograms when we implement our method BOP. Figure 11 and Figure 12 show the histograms of ${\mathbf{w}}^{(k)}$ and $\Delta \mathbf{w}$, respectively. As we see from these histograms and the KLD and SIKLD values in Table 2, this method allows us to remove the side spikes of the histograms and much better preserve the original shape of the weight distribution. As a result, the detectability of the watermark due to Adam optimization is strongly reduced.

A positive side effect of the undetectability of the watermark is that the robustness is increased because an adversary will not know which layer must be modified in order to alter the embedded watermark. This is illustrated in Figure 13 where we compare the robustness of standard Adam with that of BOP against weight pruning. The network is trained long enough for both Adam and BOP to guarantee a similar BER vs. pruning rate, as shown in Figure 13a. Then, the PSNR obtained after training and pruning is shown in Figure 13b for the Kodak24 dataset which illustrates the following facts: (1) BOP produces a network that is more robust to pruning in terms of PSNR, which can be valuable towards model compression; for instance, for a pruning rate of 0.35 (that has no impact on the BER of the hidden information), BOP degrades the original PSNR by about 1 dB, whereas Adam would produce a degradation of more than 3 dB. (2) This robustness might be detrimental in case it is an attacker who does the pruning in an attempt to degrade the watermark; for instance, for a pruning rate of 0.82, the BER for both Adam and BOP rises to 0.02 (see Figure 13a). For this pruning rate, the PSNR of BOP is around 25 dB while Adam gives slightly less than 24 dB. Then, in the case of BOP, the adversary would be able to produce a network that performs closer to the original in terms of PSNR—notice, however, that the degradation in both cases is quite severe, so this heavy pruning would render a denoiser with little practical use —. (3) In any case, the previous comparison would assume that the adversary knows the layer that contains the watermark; as we have properly justified, this is reasonable for Adam but not so for BOP. If the adversary does not know the layer that must be pruned, then, to achieve the same target BER, he/she must prune all the layers. In this case, for the same pruning rate of 0.82 that causes the BER to increase to 0.02, the PSNR drops to less than 18 dB.

Similar conclusions can be extracted from Figure 13c that shows the PSNR vs. pruning rate for the CBSD68 dataset. These experiments clearly show the higher robustness brought about by the undetectability of BOP, as it prevents attacks targeted to a specific layer.

7. Conclusions

Throughout this paper, we have shown the importance of being careful with the optimization algorithm when we embed watermarks following the approach in [5,6]. The choice of certain optimization algorithms whose update direction is given by the sign function can originate footprints in the distributions of weights that are easily detectable by adversaries, thus compromising the efficacy of the watermarking algorithm.

In particular, we studied the mechanisms behind SGD and Adam optimization and found that the sign phenomenon that occurs in Adam is detrimental for watermarking, since it causes the appearance of two salient side spikes on the histograms of weights. As opposed to Adam, the sign function does not appear when we use SGD. Therefore, SGD does not significantly alter the original shape of the distribution of weights although, as we showed in the theoretical analysis, it slightly increases its variance. The analysis in this paper can be extended to other optimization algorithms.

In addition, we introduced orthogonal projectors and observed that, compared to the Gaussian case, they generally preserve the original performance and weight distribution better. However, a deeper analysis on this subject is left for further research.

Finally, we presented a novel method that uses orthogonal block projections to address the use of Adam optimization together with the watermarking algorithm under study. As we checked in the empirical section, this method allows us to solve the detectability problem posed by Adam and still enjoy the rest of advantages of this optimization algorithm.