Perceptual Image Hashing Using Latent Low-Rank Representation and Uniform LBP

Robustness and discriminability are the two most important features of perceptual image hashing (PIH) schemes. In order to achieve a good balance between perceptual robustness and discriminability, a novel PIH algorithm is proposed by combining latent low-rank representation (LLRR) and rotation invariant uniform local binary patterns (RiuLBP). LLRR is first applied on resized original images to the principal feature matrix and to the salient feature matrix, since it can automatically extract salient features from corrupted images. Following this, Riulocal bin features are extracted from each non-overlapping block of the principal feature matrix and of the salient feature matrix, respectively. All features are concatenated and scrambled to generate final binary hash code. Experimental results show that the proposed hashing algorithm is robust against many types of distortions and attacks, such as noise addition, low-pass filtering, rotation, scaling, and JPEG compression. It outperforms other local binary patterns (LBP) based image hashing schemes in terms of perceptual robustness and discriminability.


Introduction
With the rapid development of multimedia information processing technology and the growing popularity of the Internet, the dissemination of digital contents such as digital images, audio and video via internet has become more and more popular. At the same time, however, the contents of digital data can easily be modified or forged without leaving any visible traces [1][2][3]. To verify the authenticity of digital images and to protect their intellectual property, perceptual image hashing (PIH) has emerged as an effective technology for image security and authentication and has attracted extensive attention [4,5]. A PIH function maps an input image to a fixed size binary string called image hash, based on an image's appearance to human eyes [6,7]. The hash values can be used to represent digital image contents, which should tolerate content preserving distortions but should reject malicious attacks that change image contents. Consequently, images with the same visual appearance should have similar hash values, while visually distinct images should have totally different hash values [8][9][10].
General PIH schemes consist of three steps: pre-processing, feature extraction and hash generation, which in the past decades have found extensive applications in many fields, such as image authentication, image retrieval, image recognition and digital watermarking [11][12][13][14][15][16][17]. One of the key steps in a PIH scheme is robust features extraction. A high performance PIH scheme is dependent on suitable features. A local binary pattern (LBP) is originally proposed by Ojala et al. [18] and is always an effective texture feature extraction method, due to its rotation and scale invariance [19][20][21].

Latent Low-Rank Representation
The low-rank representation (LRR) method aims at finding the lowest-rank representation among all the candidates. When the observed data matrix is used as the dictionary A, recovering the low-rank representation from the given observation X o can be written as the following convex optimization problem [32,33]: min where · * denotes the nuclear norm of a matrix, i.e., the sum of the singular values of the matrix. · 1 is the L1-norm characterizing the sparse noise E. λ > 0 is a regularization parameter for balancing the influences of the sparsity error term. Wang et al. [33] applied LRR to multi-view spectral clustering by separately imposing a low-rank constraint on each view and achieved the multi-view agreement via an iterative fashion. While A = X o , this assumption may be invalid, and the data sampling is insufficient. So, LRR may not represent the subspaces effectively, and the recovery robustness may be weakened. LLRR can be regarded as an enhanced version of LRR, which constructs the dictionary A using both observed data X o and unobserved hidden data X H ; it is more accurate and robust to noise than LRR for subspace representation [34,35]. To resolve the problem of insufficient sampling and to improve the robustness to noise corruption, LLRR is exploited to extract suitable image features during the PIH generation. An approximate recovery can be achieved by analyzing the properties of the hidden effects as follows: The hidden effects recovery problem for corrupted data in Equation (2) can be solved by the following convex optimization problem: where Z O|H and L H|O correspond to the principal component and the salient component, respectively.
For the sake of simplicity, we replace X o , Z O|H and L H|O with X, Z and L, respectively. Thus, the convex optimization problem in Equation (3) can be rewritten as: where X ∈ R d×n , Z ∈ R n×n and L ∈ R d×d . The parameters d and n are the feature vector size and the number of features, respectively. This problem can be solved via the Augmented Lagrange Multiplier (ALM) [36] method.

Local Binary Pattern
There are many LBP operators and LBP feature extraction methods are reported in image recognition and image security fields, among which the RiuLBP operator is one of most popular texture operators due to its rotation invariance and low dimension [37]. The basic LBP is a gray-scale invariant which transforms the neighborhood pixels into a set of binary codes by taking the center pixel as a threshold representing the center pixel, and it is defined as follows: where R denotes the scale of the radius of neighborhoods, P denotes the number of sampling points, g c is the gray value of the center pixel, g p is the circularly symmetric neighbor, and s(*) is the sign function that returns the sign of the specified number.
When there are at most two bitwise 0/1 transitions, the pattern is called a uniform one. The number of uniform patterns is P(P − 1) + 2, which is less than the number 2 P of the basic LBP feature. A function U(∆) is defined to return the number of spatial transitions (bitwise 0/1 transitions) in the pattern ∆, and it can be written as: where the function Mod(x, y) returns the remainder after a number x is divided by a divisor y.
In order to achieve good discriminability and robustness of the perceptual image hash, the RiuLBP feature descriptor is utilized to extract stable image features. Instead of the ordered binary coding of the basic LBP, the center pixel of RiuLBP is denoted by simply counting ones in the basic LBP coding for uniform patterns [38], denoted by LBP riu P,R , as shown below: From Equation (7), we can see that the RiuLBP feature only has P + 2 distinct patterns.

Proposed Perceptual Image Hashing Algorithm
In order to achieve a good balance between discriminability and robustness, a new PIH scheme is proposed in this paper by combining LLRR and LBP features (called LLRR-RiuLBP). In the proposed scheme, LLRR is first employed in order to obtain the principal and salient components of the original images, considering its robustness of salient feature extraction to corrupted data; following this, the RiuLBP feature extraction is applied to the principal and salient components and generates the perceptual hash. The proposed PIH scheme consists of three main stages: pre-processing, feature extraction, and hash generation. The block diagram of the proposed image hashing scheme is shown in Figure 1, and the whole hash generation process is depicted as follows.
where the function ( ) Mod x, y returns the remainder after a number x is divided by a divisor y.
In order to achieve good discriminability and robustness of the perceptual image hash, the RiuLBP feature descriptor is utilized to extract stable image features. Instead of the ordered binary coding of the basic LBP, the center pixel of RiuLBP is denoted by simply counting ones in the basic LBP coding for uniform patterns [38], denoted by riu P,R LBP , as shown below: From Equation (7), we can see that the RiuLBP feature only has P + 2 distinct patterns.

Proposed Perceptual Image Hashing Algorithm
In order to achieve a good balance between discriminability and robustness, a new PIH scheme is proposed in this paper by combining LLRR and LBP features (called LLRR-RiuLBP). In the proposed scheme, LLRR is first employed in order to obtain the principal and salient components of the original images, considering its robustness of salient feature extraction to corrupted data; following this, the RiuLBP feature extraction is applied to the principal and salient components and generates the perceptual hash. The proposed PIH scheme consists of three main stages: pre-processing, feature extraction, and hash generation. The block diagram of the proposed image hashing scheme is shown in Figure 1, and the whole hash generation process is depicted as follows. Step 1: For color images, only the luminance component is considered because it contains significant information on the input images. An original color image I is first converted to a grayscale image Ig.
Step 2: In order to produce a fixed-length hash code, the image normalization of the bilinear interpolation is applied to the grayscale image, and a resized image Ir is generated with a M M × size.
Step 3: Following this, a pixel-wise adaptive Wiener filter is applied to the resized input image Ir in order to reduce disturbances caused by the image operation, such as noise addition and lossy compression; a filtered input image If is then generated.
Step 4: LLRR is applied to the resized input image Ir in order to obtain the principle feature matrix Z, salient feature matrix L and error matrix E using Equation (4).
Step 5: The principle feature matrix Z and salient feature matrix L are divided respectively into non-overlapping sub-blocks with a b b × size. For each image sub-block, the normalized histogram of the CSLBP codes is computed as follows. Consequently, two histograms  Step 1: For color images, only the luminance component is considered because it contains significant information on the input images. An original color image I is first converted to a grayscale image I g .
Step 2: In order to produce a fixed-length hash code, the image normalization of the bilinear interpolation is applied to the grayscale image, and a resized image I r is generated with a M × M size.
Step 3: Following this, a pixel-wise adaptive Wiener filter is applied to the resized input image I r in order to reduce disturbances caused by the image operation, such as noise addition and lossy compression; a filtered input image I f is then generated.
Step 4: LLRR is applied to the resized input image I r in order to obtain the principle feature matrix Z, salient feature matrix L and error matrix E using Equation (4).
Step 5: The principle feature matrix Z and salient feature matrix L are divided respectively into non-overlapping sub-blocks with a b × b size. For each image sub-block, the normalized histogram of the CSLBP codes is computed as follows. Consequently, two histograms H r , r ∈ {Z, L}, are built according to the principle feature matrix Z and salient feature matrix L, respectively. where Step 6: To reduce feature redundancy, zero-mean normalization is applied to the histogram feature to produce a normalized histogram feature H r by using Equation (10): where µ and δ are the mean and standard deviations of the feature set H r , and where ε is a minimal value in order to avoid division by zero.
Step 7: The normalized histogram H r is a P + 2 bins histogram, and the features of all the blocks are concatenated in order to generate a final LLRR-RiuLBP feature.
Step 8: The principal component analysis (PCA) is applied on the feature vector H in order to obtain an effective perceptual feature; the process can be written as follows: where m denotes the feature dimension after the dimension reduction, and where H i i = 1, · · · , m are the principal components after the PCA feature reduction [39].
Step 9: A binary sequence of perceptual features V is generated by mapping H PCA onto the binary bits.
Step 10: A secret key k is used in order to produce pseudorandom sequences W with the use of a chaotic logistic map [40].
In order to ensure the security of the PHI scheme, the sequence W is used to scramble the sequence W via a pixel-wise exclusive-or (XOR) operation between the V and W sequences; the scramble feature vector is the final image hash H f .

Experiments and Analysis
To test the performances of the proposed PIH scheme, extensive experiments are conducted on many standard images with a 256 × 256 size, which can be obtained from the CVG-UGR (Computer Vision Group, University of Granada) image database [41]. The normalized Hamming distance was adopted in our experiments in order to measure the similarity between two hashes.
where H f and H f are two hash sequences, and where m is the hash length.
In the experiments, the parameters of the resized image size M, sub-block size b, LBP radius R, LBP pixel number P of the neighbor, and final perceptual hash length m are set to 256, 4, 1, 8 and 500, respectively.
All the experiments are tested on a laptop, the Intel Core i-3630QM 2.66 GHz CPU (Intel Corp., Santa Clara, CA, USA), with an 8 GB memory and running MATLAB 2016a (Mathworks Inc., Natick, MA, USA). The average time cost is computed on test images with a 256 × 256 size, and the average running times of the different PIH methods [13,24,26] are listed in Table 1. Our method and Qin et al.'s method [13] need more time cost than the other two methods [24,26] because of the use of the LRR operations.

Perceptual Robustness
In order to evaluate the perceptual robustness of the proposed PIH scheme (LLRR-RiuLBP), we conducted some robustness experiments under the common content-preserving attacks, such as JPEG compression, Gaussian filtering, median filtering, noise addition, scaling and rotation (as listed in Table 2), based on the CVG-UGR image database [41]. Four of the standard test images derived from them are shown in Figure 2 [26] schemes are illustrated in Figure 3 in terms of the normalized Hamming distance. Note that each average normalized Hamming distance in Figure 3 is calculated according to all the hash pairs of all the test images and the corresponding attacked images.
It can be seen that the average normalized Hamming distance of the proposed scheme (LLRR-RuiLBP) is less than Liu et al.'s and Qin et al.'s methods. That is to say, our PIH scheme is more robust to content-preserving attacks than existing schemes [13,24,26]. This is partly because the LLRR adopted in the proposed scheme can effectively extract principle features from corrupted data. in Table 2), based on the CVG-UGR image database [41]. Four of the standard test images derived from them are shown in Figure 2.  [26] schemes are illustrated in Figure 3 in terms of the normalized Hamming distance. Note that each average normalized Hamming distance in Figure 3 is calculated according to all the hash pairs of all the test images and the corresponding attacked images.  That is to say, our PIH scheme is more robust to content-preserving attacks than existing schemes [13,24,26]. This is partly because the LLRR adopted in the proposed scheme can effectively extract principle features from corrupted data.

Discriminability
To evaluate the anti-collision performance of image hashing, 696 hash codes are generated via the proposed PIH scheme, based on 696 test images from the CVG-UGR image database [41]; following this, 378,400 normalized Hamming distances are calculated between the hash pairs of different images. The histogram of the normalized Hamming distances is shown in Figure 4. One finds that the distribution of the normalized Hamming distance proximately obeys a normal distribution with a mean of μ = 0.4825 and with a standard variation of δ = 0.0451. Consequently, given the threshold τ μ < , the collision probability c P can be computed as follows: where ( ) erfc • is a Gauss error function. The collision probabilities of the proposed PIH scheme for different thresholds τ are shown in Table 3. From this table, it can be concluded that the collision probability decreases with a decreasing threshold τ . Additionally, the hashes generated by the proposed PIH scheme have a better discriminability than some of the existing image hashing schemes [13,24,26].

Discriminability
To evaluate the anti-collision performance of image hashing, 696 hash codes are generated via the proposed PIH scheme, based on 696 test images from the CVG-UGR image database [41]; following this, 378,400 normalized Hamming distances are calculated between the hash pairs of different images. The histogram of the normalized Hamming distances is shown in Figure 4. One finds that the distribution of the normalized Hamming distance proximately obeys a normal distribution with a mean of µ = 0.4825 and with a standard variation of δ = 0.0451. Consequently, given the threshold τ < µ, the collision probability P c can be computed as follows: where er f c(•) is a Gauss error function. The collision probabilities of the proposed PIH scheme for different thresholds τ are shown in Table 3. From this table, it can be concluded that the collision probability decreases with a decreasing threshold τ. Additionally, the hashes generated by the proposed PIH scheme have a better discriminability than some of the existing image hashing schemes [13,24,26].

Discriminability
To evaluate the anti-collision performance of image hashing, 696 hash codes are generated via the proposed PIH scheme, based on 696 test images from the CVG-UGR image database [41]; following this, 378,400 normalized Hamming distances are calculated between the hash pairs of different images. The histogram of the normalized Hamming distances is shown in Figure 4. One finds that the distribution of the normalized Hamming distance proximately obeys a normal distribution with a mean of μ = 0.4825 and with a standard variation of δ = 0.0451. Consequently, given the threshold τ μ < , the collision probability c P can be computed as follows: where ( ) erfc • is a Gauss error function. The collision probabilities of the proposed PIH scheme for different thresholds τ are shown in Table 3. From this table, it can be concluded that the collision probability decreases with a decreasing threshold τ . Additionally, the hashes generated by the proposed PIH scheme have a better discriminability than some of the existing image hashing schemes [13,24,26].

Security
In our scheme, the image hash is dependent on the secret key, and different secret keys will produce distinct hashes. Figure 5 tests the security of the proposed PIH scheme based on the average normalized Hamming distance sequences (with a size of 1001), between the hash pairs generated by one correct secret key and those generated by 1000 wrong secret keys. One can observe that only the 500th normalized Hamming distance (with the correct secret key) is located in the vicinity of 0, and it is very difficult for an unauthorized user to get the same hash without the correct secret key. The proposed PIH scheme is therefore key-dependently secure.

Security
In our scheme, the image hash is dependent on the secret key, and different secret keys will produce distinct hashes. Figure 5 tests the security of the proposed PIH scheme based on the average normalized Hamming distance sequences (with a size of 1001), between the hash pairs generated by one correct secret key and those generated by 1000 wrong secret keys. One can observe that only the 500th normalized Hamming distance (with the correct secret key) is located in the vicinity of 0, and it is very difficult for an unauthorized user to get the same hash without the correct secret key. The proposed PIH scheme is therefore key-dependently secure.

Conclusions
In this paper, we propose an effective PIH scheme based on LLRR and rotation invariant uniform LBP. LLRR is first employed to obtain a principal feature matrix and a salient feature matrix. Following this, rotation invariant uniform LBP is used to extract robust features for perceptual hash generation. The ability of LLRR to extract salient features, along with the effective texture feature extraction ability of LBP, are both helpful to robustness and discriminability. Experiments show that our proposed perceptual hashing scheme is robust to content-preserving attacks such as JEPG compression, low-pass filter, noise addition, slight rotation and scaling, and that it has better robustness and discriminability performance than existing hashing schemes. In addition, the hashing scheme has high key-dependent security.

Conclusions
In this paper, we propose an effective PIH scheme based on LLRR and rotation invariant uniform LBP. LLRR is first employed to obtain a principal feature matrix and a salient feature matrix. Following this, rotation invariant uniform LBP is used to extract robust features for perceptual hash generation. The ability of LLRR to extract salient features, along with the effective texture feature extraction ability of LBP, are both helpful to robustness and discriminability. Experiments show that our proposed perceptual hashing scheme is robust to content-preserving attacks such as JEPG compression, low-pass filter, noise addition, slight rotation and scaling, and that it has better robustness and discriminability performance than existing hashing schemes. In addition, the hashing scheme has high key-dependent security.