Data Hiding in Symmetric Circular String Art

Abstract

Creative digital artwork is usually the outcome of a long period of intellectual creation and labor of an artist. Similarly, computer-created digital artwork is an outcome of a large amount of machine time and computational resources. However, such intellectual properties can be easily copied by illegal users. Copyright protection of digital art is increasingly more important than before. Recently, using a computational approach to generate string art tends to be popular and attractive. To protect the illegal usage of the digital form of string art, we propose a data hiding algorithm specifically designed for string art. A digital string art image consists of a sequence of string lines, each specified by two nails fixed at the two ends of that line. The encrypted secret data (the watermark) is embedded into the list of line segments by odd–even modulation, where a bit ‘1’ is embedded by forcing the next node to be an odd node, and a bit ‘0’ is embedded by forcing the next node to be an even node. To minimize the impact of data embedding on the quality of the original string art image, a local optimization algorithm is developed to select the nodes that produce minimal distortion. To quantify the embedding distortion, we introduce a smoothing filter model for the human vision system (HVS) specifically tailored to string art image. Experimental results show that using the proposed algorithm, the distortion between the original string art image and the watermarked string art image is unnoticeable. The modified string art image is statistically indistinguishable from the original string art, and hence is secure under steganalysis. To our best knowledge, this is the first work towards data hiding and copyright protection of digital string art.

1. Introduction

String art is a new form of abstract art that employs strings to render a natural image or simple patterns [1,2]. It was first proposed by artist Petros Vrellis in 2016 [3], who uses a private algorithm to generate monochrome and color string art. The first work on using a computer algorithm to generate string art was published in 2018 by M. Birsak et al. They designed and published an optimization algorithm to generate string art from a given digital image [4]. However, their manufacturing machine is an industry robotic arm that is not accessible to the general public. In 2019, S. Je et al. designed a low-cost fabrication machine and a spider-web-inspired algorithm to generate string art images. In addition to its artistic value, the generation process of the string art is very useful for studying the fabrication process. In 2020, I. Ostanin applied the string art to the design and manufacturing of composite materials and structures [5]. The manufactured string art can be used for interior and public space decoration. Furthermore, the manufacturing process using a robotic arm is very attractive for showing to the general public what a robotic arm can do in a public exhibition [6].

Both the design process and the manufacturing process of string art images are time-consuming. A completed string art is an accumulation of a large amount of creative work and labor, and hence is intellectual property. To protect it from illegal copying and usage, the creator or the owner of the digital art may need to embed his personal identification information into it before releasing to public. The same concern may be raised after the string art is extended to composite material design [5], since these composite materials are usually protected by laws of intellectual property right. Data hiding and watermarking have been shown to be a very powerful tool to protect the copyright of digital multimedia works [7].

Data hiding in multimedia signals has been developed for more than three decades [7,8,9,10]. Starting from the digital image, the type of host (to which the data are embedded) signal was extended to video, speech, audio, 3D mesh model and point clouds, etc. [11,12,13,14,15]. For the general data hiding problem, embedding rate and embedding distortion are two major performance metrics. According to application requirements, two types of systems can be adopted: irreversible data hiding and reversible data hiding [16]. For irreversible data hiding, secret data are embedded into host signal by irreversible operations such as addition (such as spread spectrum approach) [17,18,19,20], or quantization (such as quantization index modulation) [21,22,23,24]. Reversible watermarking explores the inherent redundancy in multimedia signals [25,26]: space/time correlation, entropy redundancy, etc., to find room for hiding extra data. This technique is further extended to encrypted images and secret sharing, hence is applicable to cloud storage and cloud computing [27,28,29,30].

However, all these available data hiding algorithms cannot be directly used for string art images. While existing digital multimedia signals use pixels/voxels as basic elements [31], the string art image uses overlapped line segments as basic elements [4]. The relationship between the line elements and the rendered image is more complicated than digital images. Thus, controlling the embedding distortion for string art images is a challenging problem. To our best knowledge, currently there are no data hiding algorithms developed specifically for this type of host signal.

The contributions of our work can be summarized and highlighted as follows:

• This work is the first work addressing data hiding in string art images for copyright protection and secure communication.
• In order to minimize embedding distortion, a data hiding framework by odd–even modulation on node histogram is proposed, which gives sufficient freedom in choosing a string segment pair to modify.
• Four data embedding algorithms are designed in the proposed framework, in an effort to minimize the embedding distortion at low computational complexity.

This paper is organized as follows. Section 2 introduces the data structure of string art image. The embedding and extraction algorithms are described in Section 3. Experiments are discussed in Section 4. Finally, we conclude this paper in Section 5.

Notation: Unless otherwise stated, scalars are denoted as lower-case letters, such as a variable n. Vectors and matrices are represented as low-case and upper-case boldface letters, respectively, such as a vector $\mathbf{b}$ and a matrix $\mathbf{A}$. Functions that operate on an entire image are represented as calligraphic form, such as operators $\mathcal{D}(\mathbf{A})$ and $\mathcal{R}(\mathbf{A})$. Set is represented in upper case San-serif font, such as a set $\mathsf{J}$.

2. The Digital String Art

A string art image is usually formed by passing a string through an array of nails fastened on a wooden plate. There are other variations of this basic form. For example, the nails can be replaced by teeth-shaped ends carved from an acrylic sheet [32]. Depending on whether the nails are regularly spaced or irregularly spaced, the string art can be classified into two types.

• Regularly Spaced Nails. For this type, the nails are regularly spaced on a wooden plate. The most popular arrangement of the nail positions is to put the nails on a circle. This will be the focus of our work. Other regular arrangements of nails are also possible, for example, by placing nails on a two-dimensional grid.
• Irregularly Spaced Nails. To better render the original image, the feature of the original image should be used to guide the placement of the nails. For example, in regions with more texture or structure, more nails should be placed.

2.1. Data Structure for Digital String Art Images

Before fabricating the digital string art image into a real string art, the digital string art image is stored in a text file. Unlike the ordinary digital image where the pixels are stored as a matrix, a string art image is stored as a sequence of line segments, connected heads to tails. For a string art image with N nodes and L line segments, we have L rows and two columns, as illustrated in

Table 1. Data structure for string art image.

$s_1$	$d_1$
$s_2$	$d_2$
⋯	⋯
$s_L$	$d_L$

. A toy example of the data structure for string art image is shown in Figure 1a. The corresponding string art image is rendered as shown in Figure 1b.

Each line segment starts from node $s_i$ and ends at $d_i$, where $s_i$ and $d_i$ are two node numbers. Let $\mathsf{N}=\left\{1,2,\dots ,N\right\}$ be the set of node indices, then $s_i\in \mathsf{N}$, $d_i\in \mathsf{N}$ and in general $s_i\ne d_i$. To make the line segments connected sequentially, the ending point of line segment i must be the starting point of the next line segment, i.e., $s_{i+1}=d_i$. The node $d_i$ is called the joint of two line segments $〚s_i,d_i〛$ and $〚s_{i+1},d_{i+1}〛$.

For convenience, we will use a matrix $\mathbf{A}$ to record this data structure. Each row of this matrix corresponds to one row of Table 1. So we have $a_{i,1}=s_i$ and $a_{i,2}=d_i$, for all $i\in \left\{1,2,\dots ,L\right\}$. Since this matrix has the same structure as a string art file, we will use the term string art $\mathbf{A}$ to denote either the file or the data structure as a matrix.

To describe our algorithm, we need to fix the following terms and notations.

• Node: A node of a string art image corresponds to one of the nails fastened at the edge of the circular plate. Each node is specified by its index number $n\in \left\{1,2,\dots ,N\right\}$.
• Segment: A line segment is determined by two nodes. For example, for a line segment starting at node m and ending at node n, it is denoted as $〚m,n〛$.
• Joint: A joint is a common node shared by two adjacent line segments. For example, for two line segments $〚m,n〛$ and $〚n,\ell 〛$, the node n is a joint between them. For example, the node 2 is a joint for line segments $〚0,2〛$ and $〚2,3〛$.
• Segment pair: Two line segments connected by a joint. For example, a segment pair formed by lines $〚m,n〛$ and $〚n,\ell 〛$ is denoted as $〚m,n,\ell 〛$.

3. Data Hiding by Odd–Even Modulation

In this section, we describe our data hiding framework by utilizing odd–even modulation in string art images. For a string art image having N nodes, this algorithm can hide up to $N-1$ bits. The basic idea is to let the number of visits of each node reflect the hidden message bits. The overall block diagram of the proposed system is shown in Figure 2. It consists of two stages: data embedding and data extraction. In the data embedding stage, encrypted secret data $b_n$ is embedded into string art image $\mathbf{A}$ to generate modified string art image $\widehat{\mathbf{A}}$. During data extraction, the secret data $\widehat{b}$ is extracted from the modified string art image $\widehat{\mathbf{A}}$. The critical problems here are (1) minimizing distortion between $\mathbf{A}$ and $\widehat{\mathbf{A}}$, and (2) keeping $\mathbf{A}$ and $\widehat{\mathbf{A}}$ statistically indistinguishable. To achieve these, our framework is based on a special histogram, node histogram, that we proposed for string art.

3.1. Node Histogram

A node histogram for string art image records the number of visits of each node. This is similar to the image histogram that records the number of occurance of values that pixels can take in a given image.

The node histogram can be obtained by sequentially visiting the entries of the matrix $\mathbf{A}$ and recording the number of occurance of the node indices. After checking the line segment matrix $\mathbf{A}$, we may notice that except for the first node $a_{1,1}$ and the last node $a_{L,2}$, all other nodes appear even number of times. This is because each of these nodes appear twice in two connected segments, i.e., $a_{i,2}=a_{i+1,1}$.

Let $f(n)$ be the number of occurrence of node n in the entries of $\mathbf{A}$, then the node histogram can be calculated by:

$$\begin{array}{c}\hfill h\left(n\right)=\left\{\begin{array}{c}\lceil \frac{f(n)}{2}\rceil ,n=1,a_{L,2};\\ \frac{f\left(n\right)}{2},\mathrm{otherwise}\end{array}\right.\end{array}$$

(1)

where the ceiling function $\lceil x\rceil$ returns the nearest integer towards $+\infty$. The node stored in $a_{L,2}$ is the ending node of the whole string art image.

For example, given a string art image Mona Lisa, the corresponding node histogram is shown in Figure 3. Two observations can be made from Figure 3b. First, the node histogram is correlated with the local blackness of the image near the node. For example, the nodes near the top-left corner of the image have lower histogram bins because the top-left corner is bright. Second, the node histogram exhibits local similarity, i.e., adjacent bins have similar heights. This local smoothness reflects the local smoothness of the string art image, which comes from the local smoothness of the original Mona Lisa image. This local smoothness is crucial to ensure that our algorithm is secure under steganalysis, as will be shown in Section 4.5.

3.2. Data Embedding

The basic idea of our data hiding algorithm is to let the height of the histogram bin reflect the hidden message. More specifically, given a sequence of secret message $b_n\in \left\{0,1\right\}$, $n=1,2,\dots ,N-1$, we let the parity of the histogram bin $h(n)$ be equal to $b_n$, i.e.,

$$\widehat{h}(n)=b_n(\mathrm{mod}2),n\in \left\{1,2,\dots ,N-1\right\}.$$

(2)

This is equivalent to replacing the least significant bit (LSB) of $h(n)$ by $b_n$:

$$\widehat{h}(n)=\mathcal{E}\left(h(n),b_n\right)=\sum _{i=1}^B{h}_{n,i}{2}^i+b_n,$$

(3)

where B is the bit depth of $h(n)$ and $h_{n,i}$ is the $i$-th bit of $h(n)$. If the parity of $h(n)$ is equal to $b_n$, then no changes of line segments connected to node n are needed. But, if the parity of $h(n)$ is different from $b_n$, then we need to move the common end of two line segments connected to node n to another node.

To increase security, a random node is selected as the first node. This initial node can be used as a secret key and should be shared between the embedder and the extractor. Without loss of generality, suppose that the node k is chosen as the first node. Then we sequentially embed $b_n$ into the node $k+n-1$, i.e.,

$$\widehat{h}(n+k-1)=b_n(\mathrm{mod}2),n\in \left\{1,2,\cdots ,N-1\right\}.$$

(4)

To enforce the parity as determined in Equation (4), we need to move line joint from one node to other nodes. If $h(n+k-1)\ne b_n$, then we need to choose one line joint from the $h(n+k-1)$ line joints at node $(n+k-1)$. Two technical issues need to be solved at this stage. (1) Which joint in the current node should be moved? and (2) where to move this joint to? As with data hiding in an image, modifying the string art causes distortion to the rendered image. So, when embedding data, we should try to minimize the introduced distortion.

Ideally, we may formulate the embedding as an optimization problem. Let $\mathcal{R}$ be a rendering function that renders the line segment matrix $\mathbf{A}$ into an image. This involves either physically manufacturing the string art image or simulating the string art image as a two-dimensional digital image.

To measure distortion introduced by data hiding, we also need to introduce a metric. Given two rendered digital images, this metric should mimic the behavior of a human vision system (HVS) in evaluating the fidelity of the modified image with respect to the original image. Many available HVS models describe the low-pass behavior of the HVS, for example, the Gaussian low-pass model [33]. Without loss of generality, we set the initial node $k=1$ when discussing non-security related aspects of our algorithm. So, the data hiding problem can be formulated as:

$$\begin{array}{cc}\hfill \underset{\widehat{\mathbf{A}}}{min}& \mathcal{D}(\mathcal{R}(\mathbf{A})-\mathcal{R}(\widehat{\mathbf{A}})),\hfill \\ \hfill \mathrm{subject} \mathrm{to}& \widehat{h}(n)=b_n(\mathrm{mod}2),\forall n\in \left\{1,2,\dots ,N-1\right\},\hfill \end{array}$$

(5)

where $\widehat{h}(n)$ is the node histogram of the modified string art $\widehat{\mathbf{A}}$.

On average, only half of the N nodes may fail to meet the requirement that $mod(\widehat{h}(n),2)=b_n$. So we need to find one joint from each of these $N/2$ nodes and move it to another such node. The number of solutions (i.e., the space of feasible solutions) can be estimated as follows. Let the set of nodes to be $\left\{n_1,n_2,\cdots ,n_{N/2}\right\}$ and we choose one joint from each of these nodes, then the number of choices is

$$\prod _{i=1}^{N/2}h(n_i).$$

(6)

Considering the worst case, where the node histogram is uniform, this number of combinations is:

$$\prod _{i=1}^{N/2}h(n_i)={\left(\frac{L}{N}\right)}^{\frac{N}{2}}.$$

(7)

For typical parameters $N=200$ and $L=3000$, this number is ${15}^{100}$, which is prohibitive to exhaust if a brute-force approach is used to search the space of feasible solutions.

To reduce the computational complexity, we resort to heuristic local greedy searching approaches. In this paper, we consider the following solutions: (1) random deletion, (2) random selection, and (3) minimal distance selection.

3.3. Random Deletion

If $h(n)\ne b_n(\mathrm{mod}2)$, then we randomly choose a joint from the node n and delete it from the list. Let $〚m,n,\ell 〛$ be a segment pair passing through the node n, then we remove the two segments $〚m,n〛$ and $〚n,\ell 〛$ from the list and add a new line segment $〚m,\ell 〛$. A drawback of this approach is that the newly added line segment $〚m,\ell 〛$ may be quite different from the original line segments $〚m,n〛$ and $〚n,\ell 〛$. For example, referring to Figure 1, the two segments $〚1,2〛$ and $〚2,3〛$ are near the edge of the image, but the new segment $〚1,3〛$ passes through the center of the string art image.

3.4. Random Selection

For this approach, we randomly select one joint from the current node and move it to the node next to it. The rationality behind it is that adjacent nodes are closely located with each other. So, when randomly moving a joint from one node to its adjacent neighbor, the impact to the visual quality is limited to a small region which can then be smoothed by HVS. Furthermore, since the node histogram is smooth, moving a node to its neighbor has the least impact on the histogram. As a result, it ensures security against steganalysis.

More specifically, let the current node to be n and that $\mathrm{mod}\left(h(n),2\right)\ne b_n$. For each node, we maintain a table that records all the line segments connected to that node. This table can be represented as a list of row indices of $\mathbf{A}$ with node n as the destination node:

$${\mathsf{J}}_n=\left\{\ell \in \left\{1,\cdots ,L\right\}:d_{\ell }=n\right\},$$

(8)

where L is the total number of line segments. To move a joint from node n to node $n+1$, we randomly select one joint from ${\mathsf{J}}_n$:

$$\ell \sim \mathcal{U}\left({\mathsf{J}}_n\right),$$

(9)

where $\mathcal{U}(\mathsf{B})$ denotes uniform distribution over the set B, and ∼ denotes sampling operation. Then we modify the line segment list $\mathbf{A}$ by letting $a_{\ell ,2}=a_{\ell +1,1}=n+1$ and keeping other entries unchanged.

3.5. Minimal Distance Selection

The string art images are mostly portraits of human faces. So the central region of the image is more visually important than the regions near nodes. When modifying the image, we should try not to change the line segments passing through the central part and should prefer line segments near the edges. The distance from the center of the circular plate to the line segment should be maximized when choosing line segments to move.

For convenience, we set the origin of the coordinate system to be the center of the circular plate. Given a segment $〚m,n〛$ and the radius r of the circular plate, we first determine the coordinates of the two nodes $(x_m,y_m)$ and $(x_n,y_n)$: $x_n=rcos(n\delta ),y_n=rsin(n\delta ),x_m=rcos(m\delta ),y_m=rsin(m\delta )$, where $\delta$ is the angle between two adjacent nodes and can be found by $\delta =\frac{2\pi }{N}$. The parameter N is the number of nodes. Thus, the distance between the plate center and this segment can be calculated by

$$L_{m,n}=\frac{\left|x_m{y}_n-x_n{y}_m\right|}{\sqrt{{\left(x_n-x_m\right)}^2+{\left(y_n-y_m\right)}^2}}.$$

(10)

For a segment pair $〚m,n,\ell 〛$, the sum of distances $L_{m,n,\ell }=L_{m,n}+L_{n,\ell }$ should be maximized in order to choose a pair far away from the center of the plate. Furthermore, for a given sum $L_{m,n,\ell }$, we should choose the pair with similar lengths. Combining the two requirements, we can formulate the segment pair selection problem as an optimization problem

$$\underset{〚m,n,\ell 〛\in {\mathsf{P}}_n}{max}\alpha \frac{L_{m,n}+L_{n,\ell }}{2r}+\beta \frac{L_{m,n}·L_{n,\ell }}{r^2},$$

(11)

where $\alpha$ and $\beta$ are two positive numbers and $\alpha +\beta =1$. The denominators $2r$ and $r^2$ are normalizing terms. The second term is a penalty term that favors segment pairs consisting of two line segments with similar lengths. Thus, a parameter $\beta$ controlling the strength of the penalties should be applied. The set ${\mathsf{P}}_n$ consists of all segment pairs at node n:

$${\mathsf{P}}_n\triangleq \left\{〚m,n,\ell 〛:\exists k,s.t.a_{k,1}=m,a_{k,2}=n,a_{k+1,1}=n,a_{k+1,2}=\ell \right\},$$

(12)

and $a_{i,j}$ is the $(i,j)$ entry of matrix $\mathbf{A}$.

For example, given a hypothetical string art image having four nodes ($N=4$) and the following string art matrix:

$$\mathbf{A}=\left[\begin{array}{cc}0& 2\\ 2& 3\\ 3& 1\\ 1& 2\\ 2& 0\\ 0& 3\end{array}\right],$$

(13)

we can find the set ${\mathsf{P}}_2$ as

$${\mathsf{P}}_2=\left\{〚0,2,3〛,〚1,2,0〛\right\}.$$

(14)

3.6. Simplified Minimum Distance Selection

The computational load of the optimization problem in Equation (11) consists of the following:

• Calculating the distances $L_{m,n}$ and $L_{n,\ell }$.
• Searching the space of feasible solutions.

To calculate the distance $L_{m,n}$, we need to calculate the coordinates first. But the distance calculation in Equation (10) involves division that is more costly than addition/subtraction. The radius should also be known. It can be observed from the string art image in Figure 3a that if two consecutive line segments are near the edge, then both of them are short. So, the sum of the lengths should be a reasonable measure for thread selection. The computation of $L_{m,n}$ can be replaced by the index distance as defined below.

For any two nodes m and n, their index distance is the absolute difference between the two indices, i.e., $\left|m-n\right|$. But since the nodes are placed along a circle, the index distance should be adjusted to account for this. Referring to Figure 4, the node ‘0’ is located on the x-axis (horizontal axis) and the indices increase counter-clockwise. Let the current node be node n, then the index distance between n and m can be calculated directly as $\left|m-n\right|$. But the index distance between n and ℓ should be smaller than $\left|\ell -n\right|$. Considering this, we define the index distance as:

$$D_{m,n}=\left\{\begin{array}{cc}\left|n-m\right|,\hfill & \mathrm{if}\left|n-m\right|\le \frac{N}{2};\hfill \\ N-\left|n-m\right|,\hfill & \mathrm{if}\left|n-m\right|>\frac{N}{2}.\hfill \end{array}\right.$$

(15)

Using the index distance, we can re-formulate the segment pair selection problem as

$$\underset{〚m,n,\ell 〛\in {\mathcal{P}}_n}{min}\widehat{\alpha }\frac{D_{m,n}+D_{n,\ell }}{N}-\widehat{\beta }\frac{D_{m,n}·D_{n,\ell }}{N^2/4},$$

(16)

where $\widehat{\alpha }>0$, $\widehat{\beta }>0$ and $\widehat{\alpha }+\widehat{\beta }=1$. The denominators N and $N^2/4$ are maximum values of $D_{m,n}+D_{n,\ell }$ and $D_{m,n}·D_{n,\ell }$, respectively. They are included in Equation (16) to normalize each term before weighted summing. Please note that now we have a minimization problem to solve, while Equation (11) is a maximization problem. The distance calculation in Equation (15) operates on integers and does not involve trigonometric function or division, nor does it need to know the radius of the plate. The divisions in Equation (16) can be included into the coefficients $\widehat{\alpha }$ and $\widehat{\beta }$, and can be pre-calculated. This accelerates the distance calculation significantly.

To show that the index distance $D_{m,n}$ is related to distance $L_{m,n}$, we plot the $D_{m,n}$∼$L_{m,n}$ curve for $m=0$ and $n\in \left\{1,2,...,N-1\right\}$, as shown in Figure 5. The radius is set as 180 mm [32]. Figure 5a shows that as n increases from 1 to 100, the distance D increases accordingly. When n increases from 101 to 200, the distance D decreases accordingly. The trend of D is opposite to the trend of L. Figure 5b shows that L decreases monotonically as D increases. Thus, by minimizing D, the distance L can be maximized.

3.7. Data Extraction

During data extraction, the modified string art file is given and we need to extract the embedded secret data. From the modified string art image, we first construct the the modified string art matrix $\widehat{\mathbf{A}}$, as described in Section 2.1. Then, the node histogram $\widehat{h}(n)$ can be constructed as:

$$\begin{array}{c}\hfill \widehat{h}\left(n\right)=\left\{\begin{array}{c}\lceil \frac{\widehat{f}(n)}{2}\rceil ,n=1,a_{L,2};\\ \frac{\widehat{f}\left(n\right)}{2},\mathrm{otherwise}\end{array}\right.\end{array}$$

(17)

where the ceiling function $\lceil x\rceil$ returns the nearest integer towards $+\infty$, and $\widehat{f}(n)$ records the number of occurrence of node n in the entries of $\widehat{\mathbf{A}}$. The embedded bits $\widehat{b}(n)$ can be extracted by calculating the parity of $\widehat{h}(n)$:

$$\widehat{b}(n)=\widehat{h}(n)\mathrm{mod}2,\forall n\in \left\{1,\dots ,N-1\right\}.$$

(18)

If the bits $b(n)$ were encrypted before embedding, a decryption operation needs to be applied to the extracted bits $\widehat{b}(n)$, in order to recover the plain text.

4. Experimental Evaluation

4.1. Performance Metric

As a typical data hiding system, rate and distortion are two main performance indices. Rate is defined as the number of bits embedded per dimension. For string art image, ‘dimension’ is related to the number of nodes since more nodes lead to better visual quality. For the proposed system, we embed $N-1$ bits into N nodes, so the rate R is a constant

$$R=\frac{N-1}{N}.$$

(19)

This rate increases with N and approaches 1 bit/node in the limit $N\to \infty$.

Distortion measures the difference between the perceptual quality of the original string art image $\mathcal{R}(\mathbf{A})$ and the modified string art image $\mathcal{R}(\widehat{\mathbf{A}})$. Let $\mathbf{I}=\mathcal{R}(\mathbf{A})$ and $\widehat{\mathbf{I}}=\mathcal{R}(\widehat{\mathbf{A}})$ be rendered string art images. We use HPSNR (Human Peak Signal to Noise Ratio) to measure the distortion of $\widehat{\mathbf{I}}$ with respect to $\mathbf{I}$ [34]. First, a Gaussian low-pass filter is used to smooth $\mathbf{I}$ and $\widehat{\mathbf{I}}$. This filter models the low-pass behavior of a human vision system (HVS), which is widely used in digital halftoning. The kernel of this filter is:

$$g(m,n)=\frac{1}{2\pi {\sigma }^2}exp\left(-\frac{m^2+n^2}{2{\sigma }^2}\right),$$

(20)

where $\sigma$ is the parameter of this HVS model. The smoothed images are denoted as ${\mathbf{I}}_{\sigma }$ and ${\widehat{\mathbf{I}}}_{\sigma }$, respectively. Then the HPSNR can be calculated as:

$$D=10{log}_{10}\frac{{\left(max({\mathbf{I}}_{\sigma })\right)}^2}{{\sum }_{m=1}^{Nr}{\sum }_{n=1}^{Nc}{\left({\widehat{I}}_{\sigma }(m,n)-I_{\sigma }(m,n)\right)}^2}(\mathrm{dB}),$$

(21)

where $Nr$ and $Nc$ are the number of rows and the number of columns of the rendered images, respectively.

As the central region of the rendered string art image is more visually important than region near the border, a weighted HPSNR can be designed to emphasize the central region. Before calculating the HPSNR, each filtered image is further multiplied by a weighting mask. Such a mask is shown in Figure 6.

For a symmetric string art image, we use a Gaussian mask with standard deviation ${\sigma }_M=min\left(Nr,Nc\right)/4$, such that the rendered image covers $4{\sigma }_M$ range of the Gaussian mask. The corresponding weighted HPSNR can be calculated as in Equation (21) using weighted and smoothed images.

To further characterize the structure distortion, we employ the SSIM (structure similarity measure) metric [35]. After applying the weighting mask as in Figure 6, we also get a weighted SSIM that emphasizes the central region of a string art before comparison.

4.2. Parameter Setup and Optimization

Several parameters of the proposed algorithms need to be set in our experiment. First, we summarize the chosen parameter values in

Table 2. Parameter setting.

	Parameter	Value	Setting Method
Original string art	N: Number of nodes	200	Typical value as used in [32]
	L: Number of line segments	3000	Typical value as used in [32]
	r: Radius of the plate	180 mm	Typical value as used in [32]
Data hiding algorithm	$\alpha$: weighting parameter for minimum distance selection	0.35	Parameter optimization
	$\widehat{\alpha }$: weighting parameter for simplified minimum distance selection	0.65	Parameter optimization
	$\sigma$: parameter of the Gaussian mask for HPSNR calculation	1.2	From [34].

, and then we present the rationality of these choices.

For the original string art, we use $N=200$ nodes and $L=3000$ line segments as suggested in [32]. Increasing N may bring us a string art with more details, but it will increase the computational time and fabrication time [6]. For $N=200$, using more line segments other than $L=3000$ brings us less improvement in the quality of the rendered image [32]. Using a plate of radius $r=180$ mm is suitable for manual knitting since the human arm feels comfortable when working in this range while the artist may need to spend several hours knitting the string art. The parameters for the proposed data hiding algorithm are also summarized Table 2 (rows 5 to 7). For calculating HPSNR, we set $\sigma =1.2$ as suggested in [34]. This value is suitable for observing a digital image displayed on a screen. The size of the smoothing kernel is set as $7\times 7$ to make sure that the kernel can cover $\pm 3\sigma$ range of the kernel [34].

To implement the Minimal Distance Selection (Section 3.5) and Simplified Minimal Distance Selection (Section 3.6), we must determine the parameters $\alpha$, $\beta$, $\widehat{\alpha }$ and $\widehat{\beta }$, where $\alpha +\beta =1$, and $\widehat{\alpha }+\widehat{\beta }=1$. So only two parameters $\alpha$ and $\widehat{\alpha }$ need to be set. Using a set of 4 typical string art images as testing images, we plot the average weighted HPSNR as $\alpha$ and $\widehat{\alpha }$ increase from 0 to 1. For each image, random secret bits are generated and embedded 10 times.

The testing string art images are shown in Figure 7. They are produced using the algorithm in [32] that are designed for a low-cost string art image printer.

The result is shown in Figure 8. For Simplified Minimal Distance Selection, we observe a jump of weighted HPSNR as $\widehat{\alpha }$ increases from 0.5 to 0.6. This can be explained as follows. If $\widehat{\alpha }$ is small, then the optimization process selects line segment pairs with similar lengths. So more selected segment pairs may pass through the central region of the string art image, thus producing more visually salient distortion. As $\widehat{\alpha }\to 1$, segment pairs having a small total length are preferred. For $\widehat{\alpha }=0.65$, the weighted HPSNR reaches its maximum. Therefore, we set $\widehat{\alpha }=0.65$ in the following experiments. Similar observations can be get from Figure 8a, but now the optimal $\alpha =0.35$.

4.3. Distortion Testing and Comparison

In this experiment, we test the distortion introduced by data hiding to rendered string art images. We implemented the four algorithms designed in Section 3: random deletion, random selection, minimum distance selection, and simplified minimum distance selection. Totally, $N-1=199$ bits are embedded. All these bits can be extracted correctly from the modified string art.

For the string art image Mona Lisa, the rendered string art images after data embedding are shown in Figure 9. We also included the original string art image for reference. Close inspection reveals that (1) Both the random deletion and random selection algorithm modify line segments crossing the central region of the image, and (2) The minimum distance selection and simplified minimum distance selection algorithms select line segments near the circumference. This observation is supported by the weighted HPSNR metrics. Random deletion and random selection have similar weighted HPSNR and both of them are around 30 dB. In contrast, the minimum distance selection and simplified minimum distance selection algorithms have a much higher weighted HPSNR. Both of them are around 47 dB, a 17 dB improvement. Repeated experiment using other randomly generated secret bits b gives a similar result.

To show that the distortion comparisons are consistent across different string art images, we use the four testing images as shown in Figure 7, and test the four algorithms on each image. The resulting weighted HPSNRs and weighted SSIMs are plotted in Figure 10. From Figure 10, we observe that the two minimum distance selection algorithms outperform random deletion and random selection consistently on four testing string art images. Even though a simplified distance calculation is adopted in the simplified minimum distance selection algorithm, its performance is comparable to the original minimum distance selection algorithm.

Furthermore, we observe a close correlation between the two distortion measures. The (simplified) minimum distance selection algorithms outperforms random deletion and random selection in terms of both weighed HPSNR and weighted SSIM.

A batch test is also conducted on a set of string art generated for human faces. To avoid possible copyright issues, we employ the styleGAN algorithm to generate a set of artificial faces [36]. This data set consists of 20 faces, as shown in Figure 11a. The host string arts for data embedding are then prepared using the StringArtGenerator package [37] as shown in Figure 11b, where $N=200$, $L=3000$ and the sizes of the rendered images are $1000\times 1000$.

The distortion testing result is shown in Figure 12. It is evident that the two minimum selection algorithms consistently outperform the random deletion and random selection algorithms, both in terms of weighted HPSNR and weighted SSIM.

To further investigate the coherence between the two distortion measures, we use a scatter plot to show how the weighted SSIM varies with weighted HPSNR, as shown in Figure 13. For random deletion and random selection algorithms, the correlation coefficient is relatively low, only 0.21 and 0.37. These low coefficients can be attributed to the random behavior of these two algorithms. Since a random line segment is selected or deleted, its impact on weighted SSIM and weighted HPSNR is not coherent. Close inspection of some selected or deleted line segments reveals that, for some selection/deletion, it affects tone similarity more than structure similarity. But for other selection/deletion, the reverse is true. However, for the two minimum distance selection algorithms, the correlation coefficients are 0.94 and 0.95, respectively. Using the minimum distance selection, we avoid changes of line segments across the central region of the string art. Thus, it improves not only the metrics but also their coherence. Furthermore, the correlation coefficient for all the four algorithms is 0.96.

Even though most string arts are designed for human portraits, it would be interesting to extend them to other types of images. We also test our algorithm on a set of biomedical images, as suggested by one of our reviewers. Three typical biomedical images are taken from an online data set [38]. The original image, the string art, and the string art after data embedding are shown in Figure 14, respectively. We use the simplified minimum distance selection algorithm, which is the best among the four proposed algorithms. Please note that our focus here is data embedding, not designing string art to resemble the original image. So, we will focus on distortion between the original string art and the modified string art. The weighted HPSNRs are 45.6 dB, 40.2 dB, and 40.8 dB, respectively. For weighted SSIM, we get 0.99, 0.98, and 0.98, respectively. This test shows that our algorithm can be applied to any type of image once it is rendered as a string art image.

4.4. Machine Time

In this experiment, we compare the two minimum distance selection algorithms for time efficiency. As analyzed in Section 3.5 and Section 3.6, using index distance may reduce the distance computation load. Since distance computation is called frequently when finding the optimal segment pairs to modify, we expect that reducing the computational complexity of distance calculation may reduce the machine time of the implementations. We focus on the machine time on computing the distances in Equations (10) and (15), which avoids the influence of machine time of other parts of the whole algorithm. Using the four testing images, we embed random bits 100 times and record the average machine time of both algorithms. The result is shown in

Table 3. Machine time comparison on distances calculation.

	Images	Mona Lisa	Lena	Fab	Ordy
Algorithms
Distance using Equation (10)		0.015	0.014	0.022	0.013
Simplified distance using Equation (15)		0.011	0.011	0.017	0.010
Number of calls		6772	6600	10664	5880

. On average, a 23.4% reduction of machine time can be achieved by using the simplified distance calculation.

4.5. Security

Security of the embedded data can be ensured by encrypting it before embedding. To resist steganalysis, we have to ensure that the statistical properties of the modified string art image are indistinguishable from the original image. In this section, we show that, for the testing images, their node histograms after data embedding are statistically indistinguishable from the original node histograms.

To see the similarity between the two node histograms $h(n)$ and $\widehat{h}(n)$, we find the emperical cumulative distribution functions (CDFs) first, which are:

$$\begin{array}{ccc}\hfill H(n)& =& \sum _{i=1}^{n}h(i),\hfill \end{array}$$

(22)

$$\begin{array}{ccc}\hfill \widehat{H}(n)& =& \sum _{i=1}^n\widehat{h}(i)\hfill \end{array}$$

(23)

Then, we plot $H(n)$ against $\widehat{H}(n)$, as shown in Figure 15 for string art image Mona Lisa. We observe a curve that is almost a straight line, indicating that the two distributions $h(n)$ and $\widehat{h}(n)$ are nearly indistinguishable.

To test if the two string art node lists come from the same underlying distribution, we employ the two-sample Kolmogorov–Smirnov test, where the test statistic is:

$$T=\underset{n}{max}\left(\left|H(n)-\widehat{H}(n)\right|\right).$$

(24)

The significance level is set as 0.05. The null hypothesis is that the two node lists come from the same underlying distribution. All the four images pass this hypothesis testing with a p-value 1, indicating that the two node histograms $h(n)$ and $\widehat{h}(n)$ are indistinguishable. Thus, we conclude that using the testing images in this paper, the node histogram after embedding is indistinguishable from the node histogram before embedding.

4.6. Qualitative Comparison with Other Copyright Protection Approaches

This paper focuses on using data hiding for copyright protection of string art. There are other approaches that one can employ for this purpose, such as a registration-based approach and visible watermarking [39,40,41,42,43]. In this section, we compare these three approaches qualitatively and demonstrate the superiority of the data hiding approach. The comparison result is summarized in

Table 4. Qualitative comparison with other copyright protection approaches.

	Distortion	Security	Flexibility
Visible watermarking [39,40]	High	Low	High
Registration-based approach [41,42,43]	Low	High	Low
Data hiding approach	Low	High	High

.

• Visible watermarking: Visible watermarking overlays a small-sized image (watermark) on the image to be protected, for notifying users of copyright issues [39,40]. This overlay brings high distortion to the original image. Furthermore, its security is low since the very existence of the watermark is known to everyone. This approach has higher flexibility than a registration-based approach since the owner is free to change his/her watermark.
• Registration-based approach: This approach is widely used in digital rights management (DRM) of multimedia signals shared over networks, such as Internet, DVD and CCTV (Closed Circuit Television) [41,42,43]. A salient feature of this approach is that the quality of the media signal is not affected. Furthermore, it has high security and is supported by display devices from various vendors. However, this approach is not flexible since the owner of the content needs to register his/her media to a centralized organization.
• Data hiding approach: Compared with visible watermarking, data hiding brings low distortion to the multimedia signal. Furthermore, it has higher security than visible watermarking since the watermark is hidden and encrypted. Compared with the registration-based approach, the data hiding approach has higher flexibility because the copyright message is embedded in the media signal itself. No centralized organization is needed to extract the copyright message.

5. Conclusions

To address the problem of data hiding in string art images, we proposed a framework based on odd–even modulation of the node histogram. Four algorithms were designed to minimize the embedding distortion at a low computational cost. The experimental results show that these algorithms can successfully embed secret data into a string art, without bringing perceptual distortion to the host string art. Using the minimum distance selection, an average of 17 dB reduction of weighted HPSNR is obtained on a set of testing images. The superiority of the minimum distance selection algorithm is also confirmed from the weighted SSIM metric. The two distortion metrics show close correlation for minimum distance selection algorithms. Furthermore, the simplified minimum distance selection approach leads to an average of 23.4% reduction of execution time. The modified node histograms after data embedding are indistinguishable from the corresponding original node histograms. Thus, the proposed algorithms are also secure under steganalysis.

One limitation of the proposed algorithm is that the payload of the data is relatively small. It can be improved by embedding into the connection sequence of line segments, instead of embedding into nodes. Another limitation is that the current algorithms are designed for black thread, which are not directly applicable to color threads.

The proposed embedding framework and algorithms can be extended to a string art image having irregularly-placed nodes. The node histogram can be applied to it directly. But since there is no natural ordering for the nodes, the embedding algorithm has to find a node near the current node as the target node. Our future work will focus on improving the embedding payload and designing the embedding algorithm for irregular string art.