Visual Saliency Model-Based Image Watermarking with Laplacian Distribution

Abstract

To improve the invisibility and robustness of the multiplicative watermarking algorithm, an adaptive image watermarking algorithm is proposed based on the visual saliency model and Laplacian distribution in the wavelet domain. The algorithm designs an adaptive multiplicative watermark strength factor by utilizing the energy aggregation of the high-frequency wavelet sub-band, texture masking and visual saliency characteristics. Then, the image blocks with high-energy are selected as the watermark embedding space to implement the imperceptibility of the watermark. In terms of watermark detection, the Laplacian distribution model is used to model the wavelet coefficients, and a blind watermark detection approach is exploited based on the maximum likelihood scheme. Finally, this paper performs the simulation analysis and comparison of the performance of the proposed algorithm. Experimental results show that the proposed algorithm is robust against additive white Gaussian noise, JPEG compression, median filtering, scaling, rotation attack and other attacks.

1. Introduction

Video, images or audio can be easily accessed and distributed over the Internet due to the continuous development of network and related information technologies. However, illegal copying or tampering will cause economic losses and potential copyright protection disputes when the digital multimedia product is authorized to be protected. Therefore, how to protect the copyright of multimedia products is an important issue in the field of information hiding. As an effective copyright protection technology, digital watermarking has been widely investigated by scholars at home and abroad [1,2,3,4,5,6]. It is used to identify the ownership of the original works by embedding some data (such as a logo, meaningful information, etc.) into the original video, image or audio products.

Generally speaking, digital image watermarking mainly includes the quantization-based method and multiplicative-based method. In terms of quantization-based watermarking, this approach usually quantizes the transform coefficients of the image by designing the corresponding quantizer. In the procedure of watermark detection, the corresponding watermark data can be extracted based on the quantization interval. The quantized-based watermarking method has high capacity, and furthermore, the method is a blind detection scheme, so it has been widely studied in recent years [7,8,9,10,11,12]. A classical quantization watermarking method is the quantization index modulation watermarking method proposed by Chen et al. in [7]. This method uses dirty paper coding theory and edge information to embed the watermark data into the host signal. However, the algorithm is sensitive to amplitude scaling due to the use of fixed quantization step size. In order to solve this problem, Jiao Li and Cox proposed an adaptive quantization watermarking method [8]. They designed an adaptive quantization step based on Watson’s perceptual model. Experimental results show that their method has strong robustness against scaling attack.

Inspired by the idea of $\mu$-law compression in speech communication, Kalantari et al. [9] proposed a logarithmic quantization watermarking method, which effectively improved the robustness of the watermark, but this method is very sensitive to gain attack. To address this problem, a quantization watermarking method based on the Lp norm was developed in [10]. This method is robust to gain attacks due to the use of the division function. Wan et al. [11] proposed an adaptive spread transform dither modulation watermarking method based on the perceptual visual model. This paper exploited a perceptual just-noticeable distortion model in the discrete cosine transform (DCT) domain. The major merit of the algorithm is its insensitivity to the changes caused by watermark encoding and attacks. In addition, to improve the robustness of quantization-based watermarking against geometric attacks, Liu et al. [12] proposed a quantization watermarking method based on the L1 norm in the dual-tree complex wavelet transform domain.

Different from the quantization-based watermarking method, the multiplicative-based watermarking method generally can be regarded as a communication system based on the spread spectrum mechanism. Cox et al. proposed a transform-based spread-spectrum watermarking method [13] as early as 1997, and they have developed various improved multiplicative watermarking methods. In [14], an image watermarking method has been presented based on the wavelet visual model. To improve the imperceptibility of the watermark, the method [14] combined the frequency sensitivity, luminance sensitivity and texture complexity of the image. However, the method was still sensitive to geometric attacks, which consist of cropping and rotation. The work in [15,16] focused on developing the detection method of multiplicative watermarking based on the alpha stable distribution and inverse Gaussian distribution model.

In recent years, Akhaee and others have proposed two novel multiplicative methods [17,18] based on the scaling scheme. Based on the entropy masking model [17,19] chose the image sub-block with high information entropy as the watermark embedding area and established a multi-objective optimization strategy to find the optimal value of the watermark strength factor. It has good robustness under attacks, covering additive noise and JPEG compression. However, the entropy value of image sub-blocks is inconsistent before and after watermark embedding, which makes it difficult to extract watermark data correctly. Therefore, it is easy to reduce the robustness of the watermark against synchronization attacks. Furthermore, the complexity of the multi-objective optimization mathematical model is high. Subsequently, Akhaee et al. [18] proposed a novel multiplicative watermarking method based on the contourlet transform. This method also uses the sub-blocks with high information entropy as the watermark embedding space. The difference between [17] and [18] is that the method in [18] utilizes the contourlet transform to develop the watermarking method. Similar to the method in [17], the computational efficiency of the watermarking method is low in [18]. Yadav et al. [20] proposed a multiplicative watermarking method based on the dynamic watermark strength factor. The adaptability of the watermark strength factor was mainly due to the use of the variance information of the image. In addition, the method in [20] has high computational efficiency, but it does not consider the texture masking effect and saliency model of the image.

Many image watermarking methods have been proposed based on the visual saliency model. Khalilian et al. [21] proposed a video watermark decoding strategy based on principal component analysis. In this paper, they embedded the watermark into the LLsub-band of the image in an adaptive manner based on visual saliency. Tian et al. [22] proposed a video watermarking algorithm by using the visual saliency and secret sharing strategy. Agarwal et al. [23] proposed a visible watermarking method to embed a binary logo watermark in the just-noticeable distortion of the image region. They found the important portions of the image based on the visual saliency strategy. Besides this, Castiglione et al. [24] proposed a fragile reversible watermarking method for the purpose of protecting the authenticity and integrity of functional magnetic resonance imaging (fMRI) data. Moreover, they have integrated their scheme within a commercial off-the-shelf fMRI system in their work. The work in [25] used digital watermarking technology to improve the security of images and dealt with watermarking approaches in the Hadamard transform domain. Furthermore, the work in [25] also discussed various approaches using the computational intelligence strategy to arrive at the optimum value of scaling and embedding parameters.

In this paper, we develop an adaptive watermark strength factor based on the energy aggregation of wavelet coefficients and the human visual perception model. In detail, we have made full use of the texture masking effect and visual saliency characteristics of the human visual perception model, respectively. The structure of the rest paper is organized as follows. Section 2 presents the proposed watermark embedding method. Section 3 introduces the watermark decoding. Section 4 provides the experimental results and discussions about the performance of the proposed watermarking method. Finally, Section 5 concludes this study. The main motivation behind such a structure is to improve the imperceptibility and robustness of watermarking. Therefore, the region of wavelet coefficients with high image energy is used as the watermark embedding space. Moreover, in terms of watermark detection, we describe the non-Gaussian characteristics of the wavelet coefficients of the image by the Laplacian distribution.

2. Proposed Watermark Embedding Algorithm

In accordance with the human visual system (HVS), strongly-textured areas of the image have a high just-noticeable difference (JND) threshold [26], so these regions could be able to hide more distortion and allow for selecting a higher value for the watermark strength factor. Inspired by the visual attention model [26,27,28], we select high-energy image blocks as the watermark embedding space in this study. Our proposed watermarking method focuses on two stages: embedding and detection. Figure 1 shows the flowchart of the proposed watermark embedding method, which is performed through the following steps.

2.1. Watermark Embedding

Figure 1 shows the block diagram of the proposed embedding method. The embedding steps of the watermark are performed as follows.

Step 1: Segment the original image into non-overlapping image sub-blocks with equal size; calculate the energy of each image sub-block. Then, select the image sub-block with high energy as the watermark embedding area based on the descending order of the energy value. The energy of each image sub-block can be calculated as follows:

$$E=\sum _{m=1}^L\sum _{n=1}^L{∥I(m,n)∥}^2,$$

(1)

where $L\times L$ represents the size of each sub-block, I represents each image sub-block, $m,n$ denote the rows and columns of the image sub-block, respectively, $I(m,n)$ represents the pixels at position $(m,n)$ and E represents the energy of the sub-block.

Step 2: Apply the wavelet transform to decompose each image sub-block, then extract the low-frequency sub-band image coefficients, and transform them into a one-dimensional coefficient vector denoted ${[x_1,x_2,\dots ,x_k]}^T$.

Step 3: Embed the watermark information into the host image as follows.

$$y=x\ast (1+\mu (2b-1\left)w\right),$$

(2)

where $b\in \{0,1\}$ represents the binary watermark bit, $w={[w_1,w_2,\dots ,w_k]}^T$ represents the pseudo-random sequence, $y={[y_1,y_2,\dots ,y_k]}^T$ represents the watermarked vector of the low-frequency wavelet coefficients of sub-band image; $x={[x_1,x_2,\dots ,x_k]}^T$ represents the host vector of the low-frequency wavelet coefficients of the sub-band image; * represents the vector multiplication in Equation (2), $\mu$ represents the adaptive watermark strength factor. The calculation of $\mu$ is introduced in the following Section 2.2. The value of the variable k is set to $\frac{L^2}{2^s\times 2^s}$. s denotes the scale of wavelet decomposition.

Step 4: The inverse wavelet transform is used for each sub-block to obtain the watermarked image sub-block.

Step 5: Repeat Steps 2–4 to get all the watermarked sub-blocks and then combine with the non-watermarked sub-blocks to obtain the whole watermarked image.

2.2. Calculation of the Adaptive Embedding Strength Factor

According to the just noticeable difference model proposed in [21], the region with rich texture in the image can tolerate more information distortion, that is it can increase the capacity of watermark embedding. Based on this characteristic of the image, the average energy value of the wavelet coefficients of high frequency sub-bands in i-th image sub-block can be calculated as:

$$E_{HF,i}=\frac{1}{3}(E_{HL}+E_{LH}+E_{HH}),i=1,2,\dots ,N$$

(3)

where $HL,LH,HH$ represents the high-frequency horizontal detail sub-band image, the high-frequency vertical detail sub-band image and the high-frequency diagonal detail sub-band image, respectively. The energy of the above three sub-band images is computed as:

$$E_{HL}=\sum _{m=1}^{L/2}\sum _{n=1}^{L/2}{∥HL(m,n)∥}^2,E_{LH}=\sum _{m=1}^{L/2}\sum _{n=1}^{L/2}{∥LH(m,n)∥}^2,E_{HH}=\sum _{m=1}^{L/2}\sum _{n=1}^{L/2}{∥HH(m,n)∥}^2,$$

(4)

where $HL(m,n)$ represents the wavelet coefficients of the high frequency horizontal detail sub-band image at position $(m,n)$, $LH(m,n)$ is the wavelet coefficients of the high frequency vertical detail sub-band image at position $(m,n)$ and $HH(m,n)$ is the wavelet coefficients of the high frequency diagonal detail sub-band image at position $(m,n)$. The average energy of the high frequency sub-band images of the N image blocks can be expressed as:

$$E_{HF}=\frac{1}{N}\sum _{i=1}^N{E}_{HF,i},$$

(5)

According to (5), the high-frequency portion of strength factor ${\mu }_1$ based on image texture masking [18] can be computed as:

$${\mu }_1=a-c\times exp\left(-\xi ·E_{HF}\right),$$

(6)

where the value of a is 1.023 and the value of c is set to 0.02, while $\xi$ is $3.5\times {10}^{-5}$.

In the next step, we calculate the saliency distance of each image sub-block based on the method provided in [27,28]. Concretely, saliency distance is represented as D, and the maximum saliency distance of each image sub-block is expressed as $D_{max}$. The saliency model-based factor ${\mu }_2$ can be represented as:${\mu }_2=1+\delta ·D$, where $\delta =\frac{0.02}{D_{max}}$ represents the scaling factor. In conclusion, the final watermark strength factor can be expressed as:

$$\mu =\left(a-c\times exp(-\xi ·E_{HF})\right)\times \left(1+\frac{0.02}{D_{max}}·D\right)-{\mu }_0,$$

(7)

where ${\mu }_0$ is a constant, with no loss of generality, and the value of ${\mu }_0$ is set to 1.0. Note that the final watermark strength factor $\mu$ depends on the texture masking effect and saliency model of the human visual model. Furthermore, the watermark strength factor can be varied adaptively with the change of the image texture and saliency. Therefore, visual saliency can be used to control the strength of watermark embedding appropriately, which results in improving the imperceptibility and robustness of watermarking.

3. Watermark Detection

Roughly speaking, the watermark extraction can be regarded as a communication process. Therefore, we can decode the watermark signal at the receiver. In this section, the watermark extraction process is performed as follows.

Step 1: Based on the statistical likelihood ratio, the hypothesis test can be expressed as:

$$HypothesisTest=\left\{\begin{array}{cc}{H}_0:\hfill & b=0\hfill \\ H_1:\hfill & b=1\hfill \end{array}\right.$$

(8)

where $H_0$ is the null hypothesis and $H_1$ denotes the alternative hypothesis. Then, we use the Laplacian distribution to model the wavelet coefficients of the image. The statistical distribution function of the watermarked image is represented as follows:

$$f_{y|b}\left(y_i\right)=\frac{\theta exp\left(-\theta |\frac{y_i}{1+\mu (2b-1)w_i}|\right)}{2(1+\mu (2b-1)w_i)},$$

(9)

where $\theta$ denotes the parameters of the Laplacian distribution, $\mu$ is the watermark strength factor, $w_i$ represents the pseudo-random sequence $i=1,2,\dots N$, $b\in \{0,1\}$ represents the binary bit and $f_{y|b}\left(y_i\right)$ represents the statistical distribution function of watermarked image $y_i$.

Step 2: Apply the maximum likelihood rule to extract watermark information. Specifically, extract the watermark data according to $\widehat{b}=argmaxf\left(y\right|b)$. The maximum likelihood function can be expressed as:

$$L\left(y\right)=\frac{f\left(y\right|b=1)}{f\left(y\right|b=0)}=\frac{{\displaystyle \prod _{i=1}^N}{f}_{y|b=1}\left(y_i\right)}{{\displaystyle \prod _{i=1}^N}{f}_{y|b=0}\left(y_i\right)},$$

(10)

where $L\left(y\right)$ represents the maximum likelihood function. If $L\left(y\right)>1$, it corresponds to the $H_1$ hypothesis, whereas if $L\left(y\right)<1$, it corresponds to the $H_0$ hypothesis. $f\left(y\right|b=1)$ represents the statistical distribution function of the watermarked image with watermark data “1”, and $f\left(y\right|b=0)$ is the statistical distribution function of the watermarked image with watermark data “0”. Then, substitute Equation (9) into (10) and take the logarithm on both sides; we have:

$$\sum _{i=1}^N\left|y_i\right|w_i>\frac{1-{\mu }^2}{2\mu \theta }\sum _{i=1}^N{w}_{i}log\left(\frac{1+\mu }{1-\mu }\right),H_1:b=1,$$

(11)

$$\sum _{i=1}^N\left|y_i\right|w_i<\frac{1-{\mu }^2}{2\mu \theta }\sum _{i=1}^N{w}_{i}log\left(\frac{1+\mu }{1-\mu }\right),H_0:b=0,$$

(12)

where $y_i$ represents the watermarked image and $w_i$ represents the pseudo random sequence. $\tau =\frac{1-{\mu }^2}{2\mu \theta }{\displaystyle \sum _{i=1}^N}{w}_{i}log\left(\frac{1+\mu }{1-\mu }\right)$ represents the detection threshold. According to Equation (11), when ${\displaystyle \sum _{i=1}^N}\left|y_i\right|w_i$ is greater than the detection threshold $\tau$, the extracted watermark data are “1”; whereas the extracted watermark data are “0”.

According to Step 1 and Step 2, the watermark data can be extracted without the original image information. Therefore, the proposed image watermarking algorithm is a blind method, which further improves the practicability of digital watermarking.

Figure 2 shows a histogram of sub-band wavelet coefficients together with a plot of the fitted Laplacian distribution for the Barbara and Lena images, respectively. As shown in the figure, the fits are both generally quite good. Since the wavelet coefficients of the image have the characteristics of the peak shape and a heavy tailed distribution and the tail of Laplacian distribution is flatter than the Gaussian distribution, therefore, the Laplacian distribution can model the marginal distribution of wavelet coefficients well.

4. Experimental Results and Analysis

In order to objectively evaluate the performance of the proposed algorithm, the peak signal noise-ratio (PSNR) and structural similarity index (SSIM) [29] were used to assess the image quality of the image. The PSNR is calculated as follows:

$$PSNR=10\times lo{g}_{10}\frac{MA{X}_I^2}{MSE}=20\times lo{g}_{10}\frac{MA{X}_I}{\sqrt{MSE}},$$

(13)

where $MA{X}_I$ is set to 255 in this work. $MSE$ denotes the mean square error, and it is defined as:

$$MSE=\frac{1}{mn}\sum _{i=0}^{m-1}\sum _{n=0}^{n-1}{∥I(i,j)-\tilde{I}(i,j)∥}^2,$$

(14)

where $I(i,j)$ and $\tilde{I}(i,j)$ represent two images of size $m\times n$, respectively.

The construction of SSIM is mainly based on the human visual perception model, which utilizes the structural features, luminance characteristics and contrast change information. The detailed calculation process of SSIM can be referred to in [29].

In the experiment, four images (Barbara, Lena, Bridge and Crowd) with dimensions of $512\times 512$ were used for testing. The size of the image sub-block in all images was $32\times 32$. One hundred twenty eight image sub-blocks with high energy were selected as the watermark embedding space. The watermark strength factor was determined adaptively according to each image sub-block and was calculated by Equation (7). Concretely, the watermark strength factor was set to 0.025, and the filter of the wavelet was “bior4.4”. Finally, simulation was carried out through the MATLAB2016 environment. Figure 3 shows the original images and their watermarked versions obtained by the proposed watermarking method. In this figure, the left image represents the original image, and the right one represents the watermarked image in each row. It can be seen from the figure that the proposed watermarking algorithm had good invisibility.

In order to assess the image visual quality of the proposed algorithm,

Table 1. Performance results of the Barbara and Lena images against common attacks.

Image	Barbara	Lena
	PSNR (dB)/SSIM	PSNR (dB)/SSIM
Gau.noise (var = 20)	43.53/0.9541	42.57/0.9132
Gau. noise (var = 30)	41.08/0.9103	40.36/0.8769
JPEG (20%)	40.17/0.9026	39.60/0.9008
JPEG (60%)	44.15/0.9620	43.73/0.9582
Median filtering ($3\times 3$)	45.64/0.9690	45.92/0.9883
Gaussian filtering ($5\times 5$)	43.22/0.9638	42.89/0.9456
Scaling (0.75)	45.85/0.9754	46.09/0.9772
Scaling (1.20)	44.76/0.9420	45.90/0.9691
Rotation (${10}^{\circ }$)	40.94/0.9280	41.25/0.9158
Rotation (${30}^{\circ }$)	38.39/0.8218	38.77/0.8506
Contr.enhan.[0.2,0.8]	39.78/0.9116	39.22/0.9005
Cropping ($\frac{1}{8}$)	39.34/0.9107	39.08/0.8982

and

Table 2. Performance results of the Bridge and Crowd images against common attacks.

Image	Bridge	Crowd
	PSNR (dB)/SSIM	PSNR (dB)/SSIM
Gau. noise (var = 20)	41.75/0.9061	42.63/0.9309
Gau. noise (var = 30)	39.84/0.8722	40.23/0.9026
JPEG (20%)	39.17/0.8694	40.18/0.9157
JPEG (60%)	42.36/0.9423	43.45/0.9498
Median filtering ($3\times 3$)	46.03/0.9987	46.48/0.9993
Gaussian filtering ($5\times 5$)	43.77/0.9637	44.34/0.9765
Scaling (0.75)	45.34/0.9653	45.76/0.9718
Scaling (1.20)	45.39/0.9546	44.17/0.9452
Rotation (${10}^{\circ }$)	42.12/0.9458	42.79/0.9489
Rotation (${30}^{\circ }$)	39.56/0.9074	40.24/0.9117
Contr.enhan. [0.2,0.8]	40.38/0.9123	40.79/0.9223
Cropping ($\frac{1}{8}$)	39.33/0.9173	39.76/0.9145

give the experimental results of the PSNR (dB) and SSIM results on the Barbara, Lena, Bridge and Crowd images under various attacks, covering Gaussian noise, JPEG compression, median filtering, Gaussian filtering, amplitude scaling, rotation, image contrast enhancement and cropping attacks. To evaluate the robustness of the proposed watermarking algorithms, we have performed several experiments under these above attacks.

Table 3. BER (%) results of the extracted watermark under various attacks.

Attacks	Barbara	Lena	Bridge	Crowd
Gau. noise (var = 20)	3.39	2.03	2.77	2.91
Gau. noise (var = 30)	4.60	4.12	4.94	4.25
JPEG (20%)	3.78	2.24	2.90	3.10
JPEG (60%)	0.00	0.00	0.00	0.00
Median filtering ($3\times 3$)	8.44	9,75	6.37	7.58
Gaussian filtering ($5\times 5$)	4.16	5.20	5.33	4.84
Scaling (0.75)	1.73	0.78	1.15	1.56
Scaling (1.20)	2.34	1.62	1.48	1.90
Rotation (${10}^{\circ }$)	9.20	8.98	9.17	9.35
Rotation (${30}^{\circ }$)	18.69	17.43	19.06	19.83
Contr.enhan. [0.2,0.8]	12.05	13.57	12.73	14.62
Cropping ($\frac{1}{8}$)	14.78	15.94	16.64	15.88

shows the bit error ratio (BER) results of the extracted watermark. It can be seen from Table 1, Table 2 and Table 3 that the proposed watermarking algorithm is satisfactory.

Furthermore, to evaluate the robustness of the proposed watermarking method in comparison to other approaches, we compared our watermarking algorithm with two methods proposed in [17,20]. The results have been presented with average values obtained from a set of images, which includes the Barbara, Lena, Bridge and Crowd images. Figure 4, Figure 5, Figure 6 and Figure 7 present the average results obtained from the image set. In this regard, distortion attacks include JPEG compression, additive noise, scaling attack and rotation attack, respectively. For a fair comparison, we did the experiments by embedding the same watermark length of 1024 bits at PSNR of about 42 dB after watermarking as in [17,20] into these images. Besides this, the size of the image block in all algorithms was 16 × 16. In [17], the presented method was based on wavelet transformation and the scaling-based strategy. Furthermore, in [20], the maximum likelihood detection method in the wavelet domain had been proposed. The detailed configuration of the simulation parameters of our method is shown in

Table 4. Configuration of the simulation parameters.

Parameter	Configuration
Simulation platform	Window 7, MATLABR2016
Test images	Barbara, Lena, Bridge and Crowd
Image size	512 × 512
Wavelet filters	“bior 4.4”
Embedding bits	1024
Decomposition scale	Three
Watermark strength factor	0.025
Image assessment metrics	PSNR and SSIM
Robustness metric	BER

.

In Figure 4, the range of JPEG compression quality factor is [5, 50]. As seen in the figure, the proposed watermarking algorithm performed better than those in [17,20]. The performance of [20] was worse than that of [17] when the compression strength of JPEG compression was relatively high (when the compression factor was less than or equal to 10%), and other cases were similar.

In Figure 5, Figure 6 and Figure 7, the proposed watermarking algorithm is also superior to those in [17,20]. The main reason consists of two aspects. On the one hand, we designed a watermark strength factor based on the texture information and visual saliency. Thus, the watermark embedding strength can be controlled adaptively. Therefore, the tradeoff between the invisibility and robustness of watermarking can be achieved. On the other hand, we inserted the watermark information into the image sub-block region with high energy. As a result, the robustness of the watermark against conventional attacks can be improved. Moreover, the watermark data can be easily detected due to the use of the Laplacian distribution model. In contrast, [17,20] only relied on mean and variance in noise estimation, which have a great influence on watermark detection. As a result, the proposed watermarking algorithm has good performance.

5. Conclusions

To further improve the imperceptibility and robustness of image watermarking, a content-adaptive multiplicative watermarking strength factor has been exploited by using the texture masking model and visual saliency model, and the corresponding watermark embedding method is developed in this paper. In terms of watermark decoding, the wavelet coefficients of the image are modeled based on the Laplacian distribution, and a blind watermark detection approach is designed. Finally, the performance of the proposed watermarking algorithm is analyzed and discussed through simulation analysis and compared with the related watermarking algorithm. The results show that the proposed watermarking algorithm is imperceptible and robust to additive noise, JPEG compression, median filtering, scaling, rotation and other attacks. Future work will focus on developing novel watermarking methods by using other technologies, such as convolution neural network, generative adversarial networks, the binocular vision model, etc.