3D Convex Hull-Based Registration Method for Point Cloud Watermark Extraction

Most 3D point cloud watermarking techniques apply Principal Component Analysis (PCA) to protect the watermark against affine transformation attacks. Unfortunately, they fail in the case of cropping and random point removal attacks. In this work, an alternative approach is proposed that solves these issues efficiently. A point cloud registration technique is developed, based on a 3D convex hull. The scale and the initial rigid affine transformation between the watermarked and the original point cloud can be estimated in this way to obtain a coarse point cloud registration. An iterative closest point algorithm is performed after that to align the attacked watermarked point cloud to the original one completely. The watermark can then be extracted from the watermarked point cloud easily. The extensive experiments confirmed that the proposed approach resists the affine transformation, cropping, random point removal, and various combinations of these attacks. The most dangerous is an attack with noise that can be handled only to some extent. However, this issue is common to the other state-of-the-art approaches.


Introduction
Presently, digital data are exchanging through the networks intensively, and are, in this way, exposed to various attacks. A small piece of additional information, i.e., a watermark, can be embedded within the data to identify their origin. Different approaches for watermarking of various digital multimedia data types, including images, audio, and video, were proposed in the past [1][2][3]. The watermark can be embedded in the spatial (for example, changing the grey levels of some pixels on the image) or frequency domains, where different transformations are applied, such as, for example, the discrete cosine or discrete wavelet transformations. The obtained coefficients of the transformed data can then be watermarked in the latest case. With the wide accessibility of 3D scanning devices [4,5], a huge amount of discrete points (i.e.,point clouds), acquired from the surfaces of 3D objects, are currently obtained easily. Their applicability is in various areas, such as, for example, cultural heritage [6][7][8], civil engineering [9,10], architecture [11][12][13], mechanical engineering [14,15], 3D simulation and animation [16][17][18], and indoor navigation [19][20][21]. There is, therefore, also a need to watermark the point cloud data. Unfortunately, the actual point cloud watermarking methods do not address all possible watermark attacks adequately. For example, because they use Principal Component Analysis (PCA) [22], they cannot protect the watermarked point clouds against cropping and random removal attacks.
A novel approach for point cloud watermarking is presented in this paper. It uses a new registration procedure, based on a 3D convex hull, applied before extraction of the watermark. This registration method determines the proper scale and alignment between the watermarked  [31] between the source and the target point cloud [33][34][35]. However, it is still a challenging task to estimate a non-rigid transformation (the simplest among them is scaling). A comprehensive survey on these topics can be found in [36]. Moreover, the ICP algorithms depend on a good introductory alignment to achieve accurate convergence to the global minimum. Thus, various approaches were proposed for the initial coarse alignment [37][38][39][40][41]. Unfortunately, these approaches still have difficulties with non-rigid transformation estimation, such as scaling. Only a few recent works consider this type of transformation. Mellado et. al. proposed a relative scale estimation and 3D registration of multi-modal geometry using growing least square that can handle situations with 3D models with different sampling densities, scale, and various levels of noise [42]. Their method supports both a semi-automatic scale estimation (the user needs to specify a scale range and a pair of corresponding points), and an automatic approach by selecting seeds and finding correspondences between them. In the final stage, a random sample consensus (RANSAC) is applied to estimate the relative scale and initial alignment [43]. Recently, Fan et al. proposed a registration approach that applies a 3D convex hull [44]. They tried to find the similarity transformation by determining a translation vector, a rotation matrix and a scaling factor. The best transformation is determined by a randomly selected triangle from the source convex hull and a triangle on the target convex hull by applying RANSAC. The same authors extended their work, and projected the cloud points onto the reference plane and then performed matching [45]. None of the above methods are used in the watermarking of a 3D point cloud. Thus, we are not convinced that they can handle different attacks properly, especially the cropping attacks. A more robust approach is needed that can handle the point clouds damaged by various attacks. Thus, we propose an improved registration method used in the process of watermark extraction, which also applies a 3D convex hull to determine the initial scaling factor between the watermarked (possibly attacked) and the original point clouds.

The Proposed 3D Point Registration Method
The main contribution of this paper is a new registration method for a relative scale estimation using a 3D convex hull in the process of the watermark extraction from a point cloud [46,47]. The main motivation was to overcome the main disadvantage of methods using the RANSAC algorithm and its variants. Namely, RANSAC is an iterative algorithm, operating on all points from the point cloud. The points are unstructured and, therefore, difficult for the registration. The number of geometric entities were reduced and the structured entity (i.e., the 3D convex hull) is obtained, which is more suitable for the registration. The proposed approach is an extension of our method for watermarking of georeferenced airborne LiDAR data [48], which does not consider the affine transformation attacks, because these types of attacks are meaningless for LiDAR data. On the contrary, the presented extension works on point clouds in general, and can handle affine transformation, cropping, and random removal attacks. In the continuation, a brief overview of the method for watermarking georeferenced airborne LiDAR data is given in the next subsection [48]. After that, the new registration method for watermark extraction from the point cloud is explained in detail.

An Overview of LiDAR Data Watermarking
The watermarking methods consist of two weakly coupled tasks: Watermark embedding and watermark extraction.
• Watermark embedding. The watermark embedding defines randomly distributed marker areas, firstly on the XY-plane (see Figure 1). Then, the X and Y coordinates of the points within these areas are modified slightly as follows: Each marker area is divided further into smaller circular parts. The distances between the centroids of the points within the circular parts and the centres of the circular parts are calculated and used as an input vector to Discrete Cosine Transformation (DCT) [30]. The last DCT-coefficient is modified, and the Inverse Discrete Cosine Transformation (IDCT) is performed. In this way, the modified vector of distances is obtained, and it is used for small disturbances of the centroids. Finally, the coordinates of the LiDAR points are modified slightly to match the new centroid positions (details of the approach are given in [48]). • Watermark extraction. The watermark extraction from the LiDAR point cloud is performed identically to the watermarking, except for the last step [48]. The marker areas are determined first. Then, the distances are calculated, and the vector of the DCT-coefficients is built. Finally, the last DCT-coefficient is checked to determine the value of the embedded watermark bit. The process continues for all marker locations to construct the whole watermark [48].

Point Cloud Watermarking
Contrary to the georeferenced LiDAR points, point clouds, consisting of points scanned from the surfaces of 3D models, can be positioned, scaled, and oriented in many different ways. This is the reason watermarking of such point clouds differs from the LiDAR point clouds' watermarking. In the continuation of the paper, the original point cloud is denoted as I, while the possibly attacked point cloud as W A .

Watermark Embedding
The watermark embedding process should change the coordinates of the points by the same amount regardless of the point cloud size. The input point cloud is normalised first; it is scaled proportionally to map its height between the values −1.0 and 1.0 (see Figure 2). The so-called normalised point cloud I N is obtained from I in such a way. The bounding box of I is needed only in this step. The normalization of the watermarked, and possible attacked point cloud, is not needed in the process of the point cloud registration. The proposed registration approach is based on the scale ratios between identified triangles of the source and target convex-hulls, as explained in the continuation. The same watermark embedding process is then performed, as described briefly in Section 3.1.

Watermark Extraction
The first step of the watermark extraction from W A is its registration. However, before the registration, W A is cleaned of possible outliers. In our case, the statistical outlier removal algorithm was applied from the Point Cloud Library (PCL) [49,50].

Convex Hull Point Cloud Registration
The convex hull of a set I of points in Euclidean space is the smallest convex set that contains I [51]. There are various algorithms for constructing convex hulls. In our approach, the Quickhull [46] has been used for I and W A . Let H W (the source convex-hull) and H I (the target convex-hull) be convex hulls of W A and I N , correspondingly. The surface of a 3D convex hull consists of triangles and, therefore, T W = {t i } denotes a set of triangles of H W , and T I = {t j } a set of triangles of H I . The registration algorithm attempts to find the pairs of corresponding triangles from T W and T I , i.e., t i ≈ t j . If a matching pair of triangles is found, the appropriate scale factor (i.e., the needed affine transformation) can be calculated easily. Unfortunately, various attacks can change H W considerably. Consequently, the number of triangles in both sets can be different, i.e., |T I | = |T W |, and the best matching triangles are difficult to identify. Even worse, as explained later, a triangle from T W can have multiple matches with triangles from T I . However, if W A and I N are similar to some extent, there should be enough corresponding triangles among T I and T W that the scale and the rigid affine transformation can be estimated. These are then applied to W A for its alignment with I N . An Iterative Closest Point algorithm (ICP) was applied for this task [33][34][35].

Relative Scale Estimation
The areas A i and A j of each triangle t i and t j are calculated first. The triangles of each convex hull are then sorted separately in decreasing order of their areas, and, after that, compared by examining the edge ratios. However, small triangles are not convenient for the comparison, because even the slightest additive noise can change their edge ratios significantly. Thus, 40% of the smallest triangles from T W and 20% of the tiniest triangles from T I are discarded (these percentages were determined experimentally). Next, the lengths of each triangle edges (d 1 , d 2 , and d 3 ) are calculated, and arranged in increasing order, i.e., d 1 ≤ d 2 ≤ d 3 . The edges are used to calculate the edge ratios α and β according to Equation (1): The similarity ratios σ i,j and ρ i,j are then calculated as follows: where τ is a given threshold. Arrays of scale factors S i are formed for each t i and matching similar triangles candidates t k ⊆ T I by applying Equation (3): All arrays S i are then sorted in decreasing order. The first elements of each S i are the best matching triangles and the candidates for the final scale factor estimation (see Figure 3). An array one with the best scale factor), while i i and k i are indices of corresponding similar triangles t i ∈ T W and t k ∈ T I , respectively. Array B is then sorted in decreasing order according to the b i . If W A is obtained by scaling I N with some scale factor s and no other attacks occur, then the largest triangles from H W with the similarity ratios σ i,j and ρ i,j match with the largest triangles from H I with the same similarity ratios. In such cases, the scale factor s would be equal to b i , i.e., s = b i . Unfortunately, attacks cause that b i may have different values. Thus, array B is processed in order to determine the most frequent value of the scale factor. An array H = {s i , f i , Ψ i } is established, which is populated by applying Algorithm 1. The scale factor s i is calculated as an average of scale factors within the range [b i − ξ, b i + ξ] (ξ is the range threshold, defined experimentally), Ψ i is an array of matching triangle pairs, and f i is the number of scale factors b i within the considered range (see Figure 4). If only a scale attack has occurred, or if there are no attacks at all, at most one scale factor s i ∈ H with the highest value f i exists. On the other hand, the attacks can cause that the scale factors are dispersed. The wrong scale factors should be isolated, and the correct scale factor should be determined as precisely as possible. Thus, the relatively small range threshold tolerance (ξ = 0.0002) is used. As the scale factors (b i+1 , b i+2 , . . .), that are within range [b i ± ξ], are already handled, the next element for processing from B is the first scale factor that is outside the range [b i ± ξ] (the scale factor b i+3 in Figure 4).
j ← j − 1 12: end while 13: j ← i + 1 14: 15: s i ← (s i + B.b j )/2 16: f i ← f i + 1 17: end while 20: i ← j 21: end while Any scale factor s i with f i < φ is discarded in the continuation applying Algorithm 2. φ is an occurrence threshold, which was set experimentally to φ = 3 (see Figure 5). The isolated wrong scale factors are discarded in this way. Depending on the attacks, there can be more than one s i with f i ≥ φ. In such case, scale factors are joined, calculating weighted average s k (see Figure 5) with Equation (4). This is performed only if s j < s i + η (see Figure 5) (the range threshold η = 0.01 was defined experimentally).
Triangle pairs Ψ i and Ψ j are joined, too. Finally, H is sorted according to f i and the first s i with the highest f i is the best scale estimation. if H. f i > 0 then 3:

Rotation and Translation Estimation
The first s i with the highest f i in the array H also contains an array of matching triangles Ψ i . They are used for estimation of rotation and translation. Two auxiliary point clouds are built, from the first and the second matching pair of triangles B.i i and B.k i . The centres of the triangles are used as the points of these clouds. The source triangles are scaled by s i . Because the auxiliary cloud contains considerably fewer points, rotation and translation is performed fast. The rigid transformation matrix (i.e.,rotation and translation) is determined between newly created point clouds by applying the Single Value Decomposition-based alignment estimation [52]. Because the correspondences of the points between these point clouds have been already obtained from the scale estimation process, applying this algorithm is the most efficient solution. The scaling and the rigid transformation are then applied to the watermarked point cloud W A . In this way, a better initial alignment hint is assured for the ICP algorithm. The ICP algorithm is applied at the end for the fine alignment of W A to I N . As a good initial alignment of the source cloud is achieved, only a few iterations of the ICP are needed.

Extraction of the Watermark Bits
The same steps are performed as in the process of watermark embedding after aligning W A to I N (see Watermark Embedding in Section 3.2 and [48]), except for the last step. The markers are determined; the distances are calculated, and a vector of the DCT coefficients is built. The last DCT coefficient is checked to determine the value of the embedded watermark bit. This process is repeated for all marker locations to reconstruct the whole watermark.

Experimental Results and Discussions
Stanford 3D models (http://graphics.stanford.edu/data/3Dscanrep/) were used in our experiments ( Table 2). Equation (5) was applied to estimate the marker radius r out .
where P = d x d y is the projected area in plane XY (d x and d y are the sides of the bounding box) (see Figure 1) and n p is the number of points in I. Value n p = 150 was used to achieve that the number of points per marker was in range [150,250]. This guarantees that the watermark can be embedded successfully and sustain a certain level of the distortion caused by various attacks. The 64-bit watermark 63687732303137 16 was applied in all experiments. The match percentage m p between the inserted watermark w and the extracted watermark w * was calculated as: where n q is the number of equal bits in both watermarks, and M is the size of the watermark in bits. m p indicates the success of watermark extraction from the W I and W A . Various experiments were performed to evaluate the proposed approach, and to find how the parameters of the watermarking process affect the presented method.

Setting the Parameters for Watermark Embedding
In this subsection, a short explanation of parameter setting while inserting the watermark into the W I is given (details are in [48]). Table 3 shows the values of the parameters, where N * is the number of markers, T is the number of smaller circular areas within the circular marker (i.e.,the number of DCT coefficients), γ is the value of modulating amplitude of the last DCT coefficient that is used to embed a watermark bit, and d max is the maximum allowable distance used to control displacement of the points due to embedding of the watermark.  Figure 6 shows the results of experiments. The finest results were achieved if the number of smaller circular areas within the marker or the size of the input vector of DCT was between 128 and 160. The best match percentages were obtained when the bit coding amplitude (the amount that the last DCT coefficient was changed) was between 0.02 and 0.04. This parameter has an impact on the level of the point displacement, which is controlled by the parameter d max . It was impossible to embed the watermark successfully if this parameter value was too small (below 0.01).

Evaluating the Resistance of the Method Against Different Attack Types
The resistance of the proposed method against different types of attacks is considered in the continuation. Tables 4-7 show the average (m p avg ) and the minimum match percentage (m p min ), together with the number of successful watermark extractions (n e ) above threshold m p th . The m p th value is set to 67.19% as determined in [48]. The last column of Tables 4-7 represents the average m p avg and the minimum m p min match percentage of all test cases (marked with * ), while n e is the total number of successful watermark extractions (marked with †).

Affine Transformation Attacks
The main advantage of the PCA-based approaches is robustness against affine transformation attacks (see Table 1 in Section 2). Therefore, the proposed method was faced firstly with this type of attack. One-hundred random test cases were performed for each point cloud. The scaling factor was chosen randomly between 0.1 and 4.0, the translation in each coordinate direction between −2.0 and 2.0, and the rotation around each axis between 0.0 and 2π radians. The results of individual and combined affine transformation attacks are given in Table 4. The watermark was unable to extract only in 3 out of 500 cases. Although the majority of test cases succeeded, we have checked the failed cases. It turns out that the orientation of W A was inappropriate for the ICT algorithm in all three cases. A slight rotation of W A remedied the problem. Figure 7a shows an example of the combined affine transformation attacks, where W A is in the red colour. The point cloud was aligned completely with the original one and, the match percentage m p = 100%.

Cropping Attacks
PCA-based methods cannot handle cropping attacks because of calculating the point cloud centroid. Even the smallest cropping may, therefore, have a huge impact on the position of a centroid. On the contrary, the proposed method copes well with the cropping attacks. In our case, W A was cropped from the top by various amounts as shown in Figure 8. The watermark was extracted, even when W A was cropped to 80% of the points. A combination of affine transformations and cropping attacks was the next experiment. The amount of croppings were changing randomly between 0% and 50%. The majority of the cases were solved successfully, as shown in Table 5. Only 9 out of 500 test cases failed, i.e.,the watermark was not extracted. Figure 7b shows an example where 60% of W A points was cropped (the attacked 3D point cloud is plotted in red, the aligned model in blue, while the original points cloud is displayed in black). Another example of a cropping attack is presented in Figure 7e, where the bottom of W T was cropped with a slantwise cut at approximately 50%. The watermark was extracted successfully in both cases.

Random Removal Attacks
The cluster-based approach, developed by Agarwal and Prabhakaran [27], constructs a cluster-tree using the nearest neighbour heuristic. Thus, their method is vulnerable to random removal attacks that have an impact on the cluster construction. Figure 9 shows that the proposed method can extract the watermark, even up to 70% of randomly removed points. The combined cropping and random removal attacks were performed (Table 6) in the next experiments. All 500 test cases succeeded. The amount of cropping and the percentage of removed points were changed randomly between 0% and 50%. Table 6 also shows the results of the combined affine transformation, cropping, and random removal attacks. Our method extracted the watermarks in 488 out of the 500 test cases. Figure 7c shows an example where 50% of points were cropped, and then 50% of the remaining points were removed randomly. The method was successful in this case. Another example of a cropping attack is presented in Figure 7f, where even more complex cropping was performed (large portions of the model are removed from the top, while the bottom of the model is removed completely).

Noise Attacks
This kind of attack is considered to be the most difficult. As the watermark is embedded by changing the coordinates of the points, any further alteration can damage the embedded watermark regardless of the watermarking approach. Thus, none of the existing methods can resist this kind of attack adequately. Indeed, the watermark can be protected only to a certain extent. A local noise attack is less vulnerable than a global noise attack. The watermark can resist a higher amount of locally, rather than globally, added noise (see Figures 10 and 11). This can also be seen in Table 7 that summarises the results of the combined attacks. Figure 11d shows an example where the noise between 0.00 and 0.18 (according to the height of the 3D model, normalised between −1 and 1) was added randomly in the upper part of W A . The noise may have an impact on the convex hull. It can change the edges of triangles in such a way that the correspondence between triangles t i and t j cannot be found.

Conclusions
A new method for 3D point cloud watermarking is considered in this work. The scale estimation and the registration of a possibly attacked point cloud with the source point cloud are done through the use of a 3D convex hull. The method consists of the following three steps:

•
Constructing the convex hulls of watermarked and original point clouds; • Matching the triangles of both convex hulls to determine the affine transformation between them; • Performing point cloud registration by applying the obtained affine transformation and the Iterative Closest Point registration algorithm.
Extensive testing was done of the proposed method. It was confirmed that the method can handle cropping attacks, which cannot be tackled by the existing PCA-based approaches. The main benefit of these approaches is their resistance to affine transformation attacks. It was shown that the proposed method copes with this type of attack equally well. The method is also successful with other attacks, such as random removal attacks, affine transformation attacks, and combinations of all the mentioned attacks. The most devastating would be an attack with noise, because it cancels out the modifications in 3D point positions, made during the process of the watermark embedding. However, this issue is common with all known 3D point clouds watermarking methods.