Improved Splitting-Integrating Methods for Image Geometric Transformations: Error Analysis and Applications

Abstract

Geometric image transformations are fundamental to image processing, computer vision and graphics, with critical applications to pattern recognition and facial identification. The splitting-integrating method (SIM) is well suited to the inverse transformation $T^{-1}$ of digital images and patterns, but it encounters difficulties in nonlinear solutions for the forward transformation T. We propose improved techniques that entirely bypass nonlinear solutions for T, simplify numerical algorithms and reduce computational costs. Another significant advantage is the greater flexibility for general and complicated transformations T. In this paper, we apply the improved techniques to the harmonic, Poisson and blending models, which transform the original shapes of images and patterns into arbitrary target shapes. These models are, essentially, the Dirichlet boundary value problems of elliptic equations. In this paper, we choose the simple finite difference method (FDM) to seek their approximate transformations. We focus significantly on analyzing errors of image greyness. Under the improved techniques, we derive the greyness errors of images under T. We obtain the optimal convergence rates $O\left(H^2\right)+O(H/N^2)$ for the piecewise bilinear interpolations ($\mu =1$) and smooth images, where $H(\ll 1)$ denotes the mesh resolution of an optical scanner, and N is the division number of a pixel split into $N^2$ sub-pixels. Beyond smooth images, we address practical challenges posed by discontinuous images. We also derive the error bounds $O\left(H^{\beta }\right)+O(H^{\beta }/N^2)$, $\beta \in (0,1)$ as $\mu =1$. For piecewise continuous images with interior and exterior greyness jumps, we have $O\left(\sqrt{H}\right)+O(\sqrt{H}/N^2)$. Compared with the error analysis in our previous study, where the image greyness is often assumed to be smooth enough, this error analysis is significant for geometric image transformations. Hence, the improved algorithms supported by rigorous error analysis of image greyness may enhance their wide applications in pattern recognition, facial identification and artificial intelligence (AI).

1. Introduction

Graphics, pictures, computer images, computer vision and digital patterns are often distorted by linear or nonlinear transformations. The algorithms that correctly convert images and patterns under these transformations and restore their original images and patterns are essential for applications in image processing, pattern recognition and artificial intelligence (AI). We solicited numerical algorithms for image geometric transformations and proposed basic algorithms in a book [1] in 1989, which were developed from [2,3,4,5]. Since then, we have focused on refining existing numerical algorithms, with significant advancements reported in [1,6,7]. Geometric transformations heavily depend on shape boundaries, which are particularly important in applications like face fusion [8]. This paper extends the splitting algorithms of [9], with a focus on rigorous error analysis and broader applications. AI relies on three elements: data, models and algorithms. The new trend in AI is to significantly improve algorithmic efficiency, (see DeepSeek [10]); such a trend has guided our study for many years. A systematic summary of new numerical algorithms for image geometric transformations is provided in the new book [11].

Let us introduce the integral model of images and its basic algorithms (the model and algorithms as AI’s terminology). The digital image in 2D is denoted by a matrix with non-negative entries representing the pixel greyness. Each pixel greyness can also be regarded as the mean of a greyness function over a small pixel region, leading to a 2D integral. Hence, the image transformation can be reduced to numerical integration. Based on this idea, we develop the splitting-integrating method (SIM) in [1], which is, indeed, the composite centroid rule in numerical analysis (see Atkinson [12] and Davis and Rabinowitz [13]). The advantage of SIM is the simplicity of algorithms, in particular for the restoration of images because nonlinear solutions are not required. Although the smooth features of the piecewise integrand in the integral are different in different subregions, the same centroid rule is always chosen. In applications, we choose the simple piecewise constant interpolations as $\mu =0$ and the piecewise linear interpolations as $\mu =1$. Our goal is to achieve the optimal convergence rates $O\left({\displaystyle \frac{1}{N^{\mu +1}}}\right)$ of sequential image greyness errors, where N is the division number of a pixel split into $N^2$.

We may apply SIM for the forward transformation T, combine the SIM for $T^{-1}$, and develop a combination CIIM for a cycle transformation $T^{-1}T$. A drawback of SIM for T is that nonlinear solutions may be involved. When the nonlinear transformations are not complicated, the Newton iteration method may be used to carry out the nonlinear solutions. In this case, the combination CIIM is studied in ([11], Chapters 4 and 5) to provide the optimal sequential convergence rates $O\left({\displaystyle \frac{1}{N^{\mu +1}}}\right)$, $\mu =0,1,2,3$. The CIIM can also be applied to n-dimensional image transformation reported in ([11], Chapter 2). However, for complicated nonlinear transformations T, solving the associated nonlinear equations becomes challenging, limiting the practicality of SIM and CIIM. For image conversion under a geometric transformation T, the splitting-shooting method (SSM) does not need nonlinear solutions. Hence, we may combine SSM for T and SIM for $T^{-1}$ to develop a combination CSIM for $T^{-1}T$. A strict analysis of the CSIM was given in Li and Bai [7], to reveal that only the low convergence rates $O\left({\displaystyle \frac{1}{N}}\right)$ of image greyness can be obtained for both $\mu =0$ and $\mu =1$. Also, by probability analysis, a higher convergence rate as $O_p\left({\displaystyle \frac{1}{N^{1.5}}}\right)$ in probability can be reached in [7]. For 256 greyness-level images under a nonlinear transformation with a double enlargement, the division number N should be chosen as large as $N=32$ or 64 to provide satisfactory pictures (see [7]). For 256 real greyscale images, CSIM becomes impractical due to the prohibitively large number of pixels. In general, the original CSIM is suited well to a few (≤16) greyness level images. A sophisticated partition technique is also invented for CSIM, leading to the advanced combination $\mathrm{C}\overline{\mathrm{S}}\mathrm{IM}$ and $\mathrm{C}\overline{\mathrm{S}}\overline{\mathrm{I}}\mathrm{M}$ in ([11], Chapter 7), to reduce the division number N significantly.

Now, we recast CIIM and study new techniques embedded into the algorithm SIM for T to reduce and even bypass completely the nonlinear solutions. This is important to the wide application of CIIM. An interpolation technique is given in [6] to reduce the number of solutions to nonlinear equations from $O\left(M^2{N}^2\right)$ down to $O\left(M^2\right)$, where $M^2$ is the total number of image pixels. The piecewise bilinear interpolations are chosen in [6] based on the exact values of $\xi$ and $\eta$ at the semi-points $((I+{\displaystyle \frac{1}{2}})H,(J+{\displaystyle \frac{1}{2}})H)$. In this paper, we also solicit the interpolation to reduce the iteration number but based on the exact values of ${\xi }_{IJ}=\xi (IH,JH)$ and ${\eta }_{IJ}=\eta (IH,JH)$ at the pixel points $(IH,JH)$. The algorithms in [6] are denoted as ${\mathrm{SI}}^{*}\mathrm{M}$ for T and ${\mathrm{CI}}^{*}\mathrm{IM}$ for $T^{-1}T$.

Moreover, since the values of ${\xi }_{IJ}$ and ${\eta }_{IJ}$ are unnecessary to be exact, we may also use the interpolation to obtain the approximate values of ${\widehat{\xi }}_{IJ}$ and ${\widehat{\eta }}_{IJ}$ to provide the improved algorithms $\mathrm{S}{\mathrm{I}}^{\#}\mathrm{M}$ for T and ${\mathrm{CI}}^{\#}\mathrm{IM}$ for $T^{-1}T$ without any nonlinear solutions. The improved algorithms are proposed in [9], but no error analysis exists so far. This is due to the improved techniques where the combination ${\mathrm{CI}}^{\#}\mathrm{IM}$ can be applied to the images under complicated transformations, e.g., those governed by partial differential equations (PDEs). In this paper, we also discuss the transformations of the Poisson and blending models by using the simple finite difference method (FDM) but apply the ${\mathrm{CI}}^{\#}\mathrm{IM}$ to image transformations undergone with these PDE models.

From the analysis in Section 4, the optimal convergence rates of greyness errors as $O\left(H^2\right)+O\left({\displaystyle \frac{H}{N^2}}\right)$ can be maintained for continuous pictures by combination ${\mathrm{CI}}^{\#}\mathrm{IM}$ as well, where $H(\ll 1)$ is the small image resolution. Another aspect of the analysis in this paper is to deal with image discontinuity. Real image greyness often has discontinuity. In this paper, we solicit the Sobolev norms and give an error analysis of the boundary discontinuity of image greyness. Using Sobolev norms, we derive the absolute errors $O\left(H^{\alpha }\right)$, $0<\alpha <1$. This study is an important remedy for error analysis made for continuous images [7]. Geometric image transformations play a vital role in image processing, computer vision, computer graphics and AI. For pattern recognition and facial identification, finding and removing the geometric transformations involved are often a critical step. Hence, the improved algorithms with error analysis can be applied to enhance their performance in these areas.

The organization of this paper is as follows. In Section 2, the SIM is introduced for $T^{-1}$ and T, and in Section 3, the interpolation techniques are proposed to reduce and even to bypass the nonlinear solutions. In Section 4, error analysis is made for both continuity and discontinuity of image greyness, and error bounds are derived. In Section 5, the PDE models of complicated transformations are approximated by the FDM, and then applied to image transformations by the new combination ${\mathrm{CI}}^{\#}\mathrm{IM}$. In Section 6, some graphical and numerical experiments are provided to verify the theoretical analysis made, and to demonstrate the effectiveness of the proposed algorithms. Concluding remarks are given in Section 7, and a list of symbols used is provided in Appendix A.

2. The Splitting-Integrating Method and Its Combinations

2.1. Numerical Algorithms

Consider the cycle transformation $T^{-1}T$: $\widehat{W}\stackrel{T}{⟶}\widehat{Z}\stackrel{T^{-1}}{⟶}\widehat{W}$, where T is a nonlinear transformation defined by

$$\begin{array}{ccc}\hfill & & T:(\xi ,\eta )\to (x,y),x=x(\xi ,\eta ),y=y(\xi ,\eta ).\hfill \end{array}$$

(1)

In this paper, we assume that only the functions $x(\xi ,\eta )$ and $y(\xi ,\eta )$ are given explicitly, or implicitly governed by the PDE solutions. Denote by $\widehat{W}=\left\{{\widehat{W}}_{ij}\right\}$ and $\widehat{Z}=\left\{{\widehat{Z}}_{IJ}\right\}$ the original and distorted images, where ${\widehat{W}}_{ij}$ and ${\widehat{Z}}_{IJ}$ are the image pixels located at $(i,j)$ and $(I,J)$ in the two Cartesian coordinates $\xi o\eta$ and $XOY$, respectively:

$$\begin{array}{c}\hfill (i,j)=\{({\xi }_i,{\eta }_j),{\xi }_i=iH,{\eta }_j=jH\},(I,J)=\{(x_I,y_J),x_I=IH,y_J=JH\},\end{array}$$

(2)

and H is the mesh resolution of the optical scanner.

We solicit numerical algorithms shown in Figure 1 consisting of eight steps. Let $G_k(=1,2,\dots ,q)$ denote the k-th level in the q-level system, where $G_1$ and $G_q$ are the whiteness and darkness, respectively. In steps 1 and 5, the pixel greyness is converted by

$$\begin{array}{ccc}\hfill & & {\mathsf{\Phi }}_{ij}={\displaystyle \frac{k-1}{q-1}}\mathrm{if}{\widehat{W}}_{ij}=\prime *\prime \mathrm{with}{G}_k;B_{IJ}={\displaystyle \frac{k-1}{q-1}}\mathrm{if}{\widehat{Z}}_{IJ}=\prime *\prime \mathrm{with}{G}_k.\hfill \end{array}$$

(3)

In steps 8 and 4, the conversions between the pixels and greyness in the q-level system are given by

$$\begin{array}{ccc}\hfill & & {\widehat{W}}_{ij}=\left\{\begin{array}{ccc}\prime *\prime \mathrm{with}{G}_q,\hfill & \mathrm{if}\hfill & {\mathsf{\Phi }}_{ij}\ge 1-{\displaystyle \frac{1}{2(q-1)}},\hfill \\ \prime *\prime \mathrm{with}{G}_k,\hfill & \mathrm{if}\hfill & {\displaystyle \frac{k-\frac{3}{2}}{q-1}}\le {\mathsf{\Phi }}_{ij}<{\displaystyle \frac{k-\frac{1}{2}}{q-1}},k=2,3,\dots ,q-1,\hfill \\ \prime \prime ,\hfill & \mathrm{if}\hfill & {\mathsf{\Phi }}_{ij}<{\displaystyle \frac{1}{2(q-1)}},\hfill \end{array}\right.\hfill \end{array}$$

(4)

$$\begin{array}{ccc}\hfill & & {\widehat{Z}}_{IJ}=\left\{\begin{array}{ccc}\prime *\prime \mathrm{with}{G}_q,\hfill & \mathrm{if}\hfill & B_{IJ}\ge 1-{\displaystyle \frac{1}{2(q-1)}},\hfill \\ \prime *\prime \mathrm{with}{G}_k,\hfill & \mathrm{if}\hfill & {\displaystyle \frac{k-\frac{3}{2}}{q-1}}\le B_{IJ}<{\displaystyle \frac{k-\frac{1}{2}}{q-1}},k=2,3,\dots ,q-1,\hfill \\ \prime \prime ,\hfill & \mathrm{if}\hfill & B_{IJ}<{\displaystyle \frac{1}{2(q-1)}}.\hfill \end{array}\right.\hfill \end{array}$$

(5)

Case II consists of all eight steps in Figure 1; Case I consists of steps 1–3 and 6–8, where the distorted image $\widehat{Z}$ is converted from $\widehat{B}=\left\{B_{IJ}\right\}$ but with no feedback from $\widehat{Z}$ to $\widehat{B}$ in step 5.

Figure 1. Schematic steps of numerical methods for image transformations.

In steps 2 and 6, we choose the simple piecewise constant $(\mu =0)$ and the bilinear interpolations $(\mu =1)$, ${\varphi }_{\mu }(\xi ,\eta )$ and $b_{\mu }(x,y)$, defined by

$$\begin{array}{ccc}\hfill & & {\varphi }_0={\mathsf{\Phi }}_{ij}\mathrm{in}{\square }_{ij},\mathrm{or}{\varphi }_1\mathrm{in}{\stackrel{-}{\square }}_{ij};b_0=B_{IJ}\mathrm{in}{\square }_{IJ},\mathrm{or}{b}_1\mathrm{in}{\stackrel{-}{\square }}_{IJ},\hfill \end{array}$$

(6)

where ${\varphi }_1$ and $b_1$ are piecewise bilinear interpolations based on the ${\mathsf{\Phi }}_{ij}$ and $B_{IJ}$, respectively. In (6), the square small domains are defined by

$$\begin{array}{ccc}& & {\square }_{ij}=\left\{(\xi ,\eta )\right|(i-{\displaystyle \frac{1}{2}})H\le \xi <(i+{\displaystyle \frac{1}{2}})H,(j-{\displaystyle \frac{1}{2}})H\le \eta <(j+{\displaystyle \frac{1}{2}})H\},\hfill \\ & & {\stackrel{-}{\square }}_{ij}=\left\{(\xi ,\eta )\right|iH\le \xi <(i+1)H,jH\le \eta <(j+1)H\},\hfill \\ & & {\square }_{IJ}=\left\{(x,y)\right|(I-{\displaystyle \frac{1}{2}})H\le x<(I+{\displaystyle \frac{1}{2}})H,(J-{\displaystyle \frac{1}{2}})H\le y<(J+{\displaystyle \frac{1}{2}})H\},\hfill \\ & & {\stackrel{-}{\square }}_{IJ}=\left\{(x,y)\right|IH\le x<(I+1)H,JH\le y<(J+1)H\}.\hfill \end{array}$$

Hence, the original and the distorted image domains are given by $\Omega ={\bigcup }_{ij}{\stackrel{-}{\square }}_{ij}$ and $S={\bigcup }_{IJ}{\stackrel{-}{\square }}_{IJ}$.

2.2. A Splitting-Integrating Method for $T^{-1}$

The greyness ${\mathsf{\Phi }}_{ij}$ and $B_{IJ}$ can be represented by the mean of continuous (or piecewise continuous) greyness functions $\varphi (\xi ,\eta )$ and $b(x,y)$:

$$\begin{array}{ccc}\hfill & & {\mathsf{\Phi }}_{ij}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{ij}}\varphi (\xi ,\eta )d\xi d\eta ,B_{IJ}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}b(x,y)dxdy,\hfill \end{array}$$

(7)

where $\varphi (\xi ,\eta )=b\left(x\right(\xi ,\eta ),y(\xi ,\eta \left)\right)=b(x,y)$. The linkage of digital images and integrals in (7) enables us to develop numerical algorithms for image transformations under T and $T^{-1}T$ easily. The greyness functions $\varphi (\xi ,\eta )$ and $b(x,y)$ can be formed by the cubic spline interpolations based on ${\mathsf{\Phi }}_{ij}$ and $B_{IJ}$, respectively (see [11], Chapter 5). Note that even $H(\ll 1)$ is small, but not infinitesimal yet. In our previous analysis, we always assume that the piecewise cubic spline functions have

$$\begin{array}{ccc}\hfill & & \varphi (\xi ,\eta )\in {\mathbf{C}}^2(\Omega ),b(x,y)\in {\mathbf{C}}^2\left(S\right).\hfill \end{array}$$

(8)

In this paper, we assume

$$\begin{array}{ccc}\hfill & & \varphi (\xi ,\eta )\in {\mathbf{H}}^2(\Omega ),b(x,y)\in {\mathbf{H}}^2\left(S\right),\hfill \end{array}$$

(9)

where ${\mathbf{H}}^2\left(S\right)$ is the Sobolev space. Equation (7) is called the integral model of images, and numerical algorithms are developed based on (7) (see [1]).

First, consider the inverse transformation $T^{-1}:(x,y)\to (\xi ,\eta )$ based on the known distorted image pixels $\left\{{\widehat{Z}}_{IJ}\right\}$, where the approximate functions $b(x,y)\approx b_{\mu }(x,y)={\varphi }_{\mu }(\xi (x,y),\eta (x,y))$. The composite centroid rule will be used to evaluate the integration values in (7). Let ${\square }_{ij}$ be split into $N\times N$ uniform squares ${\square }_{ij,k\ell }$, i.e., ${\square }_{ij}={\bigcup }_{k,\ell }{\square }_{ij,k\ell }$, where

$$\begin{array}{ccc}& & {\square }_{ij,k\ell }=\left\{\begin{array}{cc}(\xi ,\eta ),\hfill & (i-{\displaystyle \frac{1}{2}})H+(k-1)h\le \xi <(i-{\displaystyle \frac{1}{2}})H+kh,\hfill \\ & (j-{\displaystyle \frac{1}{2}})H+(\ell -1)h\le \eta <(j-{\displaystyle \frac{1}{2}})H+\ell h\hfill \end{array}\right\},\hfill \end{array}$$

and h is the boundary length of ${\square }_{ij,k\ell }$, given by $h={\displaystyle \frac{H}{N}}$. For the small subpixels ${\square }_{ij,k\ell }$, the coordinates of the center of gravity are given by

$$\begin{array}{ccc}\hfill & & {\xi }_{\dot{G}}={\xi }_{{\dot{G}}_{ij,k\ell }}=(i-{\displaystyle \frac{1}{2}})H+(k-{\displaystyle \frac{1}{2}})h,{\eta }_{\dot{G}}={\eta }_{{\dot{G}}_{ij,k\ell }}=(j-{\displaystyle \frac{1}{2}})H+(\ell -{\displaystyle \frac{1}{2}})h.\hfill \end{array}$$

(10)

By the composite centroid rule in Davis and Rabinowitz [13], we have

$$\begin{array}{ccc}\hfill & & \int {\int }_{{\square }_{ij,k\ell }}\varphi (\xi ,\eta )d\xi d\eta =\int {\int }_{{\square }_{ij,k\ell }}b(x(\xi ,\eta ),y(\xi ,\eta ))d\xi d\eta \hfill \\ \hfill & & \approx h^{2}b(x({\xi }_{{\dot{G}}_{ij,k\ell }},{\eta }_{{\dot{G}}_{ij,k\ell }}),y({\xi }_{{\dot{G}}_{ij,k\ell }},{\eta }_{{\dot{G}}_{ij,k\ell }})).\hfill \end{array}$$

(11)

Hence, the normalized greyness of ${\widehat{W}}_{ij}$ is obtained by

$$\begin{array}{ccc}\hfill & & {\mathsf{\Phi }}_{ij}\approx {\displaystyle \frac{h^2}{H^2}}\sum _{k,\ell =1}^N{b}_{\mu }(x({\xi }_{{\dot{G}}_{ij,k\ell }},{\eta }_{{\dot{G}}_{ij,k\ell }}),y({\xi }_{{\dot{G}}_{ij,k\ell }},{\eta }_{{\dot{G}}_{ij,k\ell }})).\hfill \end{array}$$

(12)

Note that the computational algorithms (11) do not involve any nonlinear solutions, and the sequential errors are proven to be $O\left(N^{-2}\right)$ as $\mu =1$ (see [11], Chapter 4).

2.3. A Splitting-Integrating Method for T

We now apply the splitting-integrating method to images under the forward transformation T based on the given ${\mathsf{\Phi }}_{ij}$. The piecewise interpolation functions ${\varphi }_{\mu }(\xi ,\eta )$ are chosen, and $b_{\mu }(x,y)={\varphi }_{\mu }(\xi (x,y),\eta (x,y))$. Let the pixel ${\square }_{IJ}={\bigcup }_{k\ell }{\square }_{IJ,k\ell }$ in $XOY$ be split into $N\times N$ small sub-pixels ${\square }_{IJ,k\ell }$ with the boundary length $h=H/N$, where

$$\begin{array}{c}\hfill {\square }_{IJ,k\ell }=\left\{\begin{array}{cc}(x,y),\hfill & (I-{\displaystyle \frac{1}{2}})H+(k-1)h\le x<(I-{\displaystyle \frac{1}{2}})H+kh,\hfill \\ & (J-{\displaystyle \frac{1}{2}})H+(\ell -1)h\le y<(J-{\displaystyle \frac{1}{2}})H+\ell h\hfill \end{array}\right\}.\end{array}$$

(13)

Also denote the coordinates of center of gravity of ${\square }_{IJ,k\ell }$ by

$$\begin{array}{ccc}\hfill & & x_{\dot{g}}=x_{{\dot{g}}_{IJ,k\ell }}=(I-{\displaystyle \frac{1}{2}})H+(k-{\displaystyle \frac{1}{2}})h,y_{\dot{g}}=y_{{\dot{g}}_{IJ,k\ell }}=(J-{\displaystyle \frac{1}{2}})H+(\ell -{\displaystyle \frac{1}{2}})h.\hfill \end{array}$$

(14)

Similarly, we may evaluate pixel greyness by the composite centroid rule:

$$\begin{array}{ccc}\hfill & & B_{IJ}={\displaystyle \frac{1}{H^2}}\sum _{k,\ell =1}^N\int {\int }_{{\square }_{IJ,k\ell }}b(x,y)dxdy\approx {\left({\displaystyle \frac{h}{H}}\right)}^2\sum _{k,\ell =1}^N{b}_{\mu }(x_{\dot{g}},y_{\dot{g}})\hfill \\ & & =B_{IJ}^{\left(N\right)}={\left({\displaystyle \frac{h}{H}}\right)}^2\sum _{k,\ell =1}^N{\varphi }_{\mu }(\xi (x_{\dot{g}},y_{\dot{g}}),\eta (x_{\dot{g}},y_{\dot{g}})).\hfill \end{array}$$

(15)

In (15), we do also need the inverse values ${\xi }_{\dot{g}}=\xi (x_{\dot{g}},y_{\dot{g}})$ and ${\eta }_{\dot{g}}=\eta (x_{\dot{g}},y_{\dot{g}})$, which are unknown. Then, we need to solve the following nonlinear equations:

$$\begin{array}{ccc}\hfill & & x=x(\xi ,\eta )=f(\xi ,\eta ),y=y(\xi ,\eta )=g(\xi ,\eta ).\hfill \end{array}$$

(16)

The Newton iteration method is suggested as

$$\begin{array}{ccc}\hfill & & z^{(k+1)}=z^{\left(k\right)}-{\left[D\left(z^{\left(k\right)}\right)\right]}^{-1}{r}^{\left(k\right)},k=0,1,\dots ,\hfill \end{array}$$

(17)

where $z^{\left(0\right)}$ is an initial approximation, and vectors and the Jacobian matrix are given by

$$\begin{array}{ccc}\hfill & & z^{\left(k\right)}=\left\{\begin{array}{c}{\xi }^{\left(k\right)}\hfill \\ {\eta }^{\left(k\right)}\hfill \end{array}\right\},r^{\left(k\right)}=\left\{\begin{array}{c}f({\xi }^{\left(k\right)},{\eta }^{\left(k\right)})-x\hfill \\ g({\xi }^{\left(k\right)},{\eta }^{\left(k\right)})-y\hfill \end{array}\right\},D\left(z\right)=\left[\begin{array}{cc}{\displaystyle \frac{\partial x}{\partial \xi }}\hfill & {\displaystyle \frac{\partial x}{\partial \eta }}\hfill \\ {\displaystyle \frac{\partial y}{\partial \xi }}\hfill & {\displaystyle \frac{\partial y}{\partial \eta }}\hfill \end{array}\right].\hfill \end{array}$$

(18)

For the given quadratic models of T, five to seven iterations may reduce the initial errors by a factor ${\displaystyle \frac{1}{2}}\times {10}^{-6}$. Of course, other iteration methods may also be chosen (see Atkinson [12]). The combination CIIM is called when the SIM is used for images under both T and $T^{-1}$. Note that the nonlinear solutions are involved only for the forward transformation T.

3. Improved Algorithms of SIM for Images Under T

The nonlinear solutions for seeking $(\xi ,\eta )$ from $(x,y)$ in (16) by the Newton iterations (17) may encounter some difficulties, e.g., choices of good initial values and multiple solutions. Below, we develop improved techniques to reduce and even to bypass the nonlinear solutions completely. Such improved techniques may also be incorporated with the PDE solutions in the harmonic, Poisson and blending models provided in Section 5.

Technique I: Reduction of nonlinear solutions.

In Li [6], the reduction technique was first used for evaluating $\xi$ and $\eta$ only at the center of gravity of ${\square }_{IJ}$ in $XOY$, i.e., at the mid-points $(x_{I+{\displaystyle \frac{1}{2}}},y_{J+{\displaystyle \frac{1}{2}}})=((I+{\displaystyle \frac{1}{2}})H,(J+{\displaystyle \frac{1}{2}})H)$. It is better to carry out (16) only at the pixel points $(x_I,y_J)=(IH,JH)$. The number of solutions to (16) is only $O\left(M^2\right)$ as in Li [6], where $M^2$ is the total pixel number of the image. The approximations of $\xi =\xi (x,y)$ and $\eta =\eta (x,y)$ in the quadrilateral $\square \overline{A}\overline{B}\overline{C}\overline{D}$ in $\xi o\eta$ may be obtained by the piecewise bilinear approximations (see Figure 2), where the gravity center $(x_{\dot{g}},y_{\dot{g}})\in {\square }_{IJ}$, $IH\le x_{\dot{g}}<(J+1)H$ and $JH\le y_{\dot{g}}<(J+1)H$. Then, the values of $({\xi }_{\dot{g}},{\eta }_{\dot{g}})$ at $x=x_{\dot{g}}$ and $y=y_{\dot{g}}$ can be evaluated by the bilinear approximation:

$$\begin{array}{ccc}\hfill & & \xi (x,y)\approx \widehat{\xi }(x,y)={\xi }_{I+1,J+1}{\displaystyle \frac{(x-IH)(y-JH)}{H^2}}+{\xi }_{I,J+1}{\displaystyle \frac{(I+1)H-x\left)\right(y-JH)}{H^2}}\hfill \\ & & +{\xi }_{I+1,J}{\displaystyle \frac{(x-IH)\left(\right(J+1)H-y)}{H^2}}+{\xi }_{I,J}{\displaystyle \frac{\left(\right(I+1)H-x)\left(\right(J+1)H-y)}{H^2}}.\hfill \end{array}$$

(19)

Moreover, the following linear approximations may be used (see Figure 3). If $x+y\le (I+J+1)H$, we have

$$\begin{array}{ccc}\hfill & & \xi (x,y)\approx \widehat{\xi }(x,y)={\xi }_{I,J}+({\xi }_{I+1,J}-{\xi }_{I,J}){\displaystyle \frac{(x-IH)}{H}}+({\xi }_{I,J+1}-{\xi }_{I,J}){\displaystyle \frac{(y-JH)}{H}}.\hfill \end{array}$$

(20)

If $x+y>(I+J+1)H$,

$$\begin{array}{ccc}\hfill & & \xi (x,y)\approx \widehat{\xi }(x,y)={\xi }_{I+1,J+1}+({\xi }_{I,J+1}-{\xi }_{I+1,J+1}){\displaystyle \frac{(I+1)H-x}{H}}\hfill \\ \hfill & & +({\xi }_{I+1,J}-{\xi }_{I+1,J+1}){\displaystyle \frac{(J+1)H-y}{H}}.\hfill \end{array}$$

(21)

The formula for $\widehat{\eta }(x,y)$ is similar. The advantage of (19) in Li [6] is that the greyness errors in the computation are smaller. The algorithms in [6] are denoted by ${\mathrm{CI}}^{*}\mathrm{IM}$ for $T^{-1}T$. Note that when $N=1$, the ${\mathrm{CI}}^{*}\mathrm{IM}$ using Technique I is just the original CIIM given in Section 2.3.

Figure 2. The inverse transformations at the vertices of ${\square }_{IJ}$.

Figure 3. The inverse transformation at $(I,J)$ in $XOY$.

Technique II: Bypassing nonlinear solutions.

The values $({\xi }_{IJ},{\eta }_{IJ})$ in (19)–(21) used in Technique I do not need to be exact, either, if their approximate values $({\widehat{\xi }}_{IJ},{\widehat{\eta }}_{IJ})$ have small allowable errors. The nonlinear transformation T may also be approximated by a piecewise linear transformation $\widehat{T}$ on triangles ${\Delta }_{ij,\ell }$, where ${\square }_{ij}={\Delta }_{ij,1}\cup {\Delta }_{ij,2}$ in Figure 3. Hence, the values for $({\xi }_{IJ},{\eta }_{IJ})\stackrel{T^{-1}}{⟵}(IH,JH)$ can be evaluated approximately by those for $({\widehat{\xi }}_{IJ},{\widehat{\eta }}_{IJ})\stackrel{{\widehat{T}}^{-1}}{⟵}(IH,JH)$. Note that the inverse transformation of ${\widehat{T}}^{-1}$ is just a linear transformation locally. This avoids the nonlinear solutions in Algorithm SIM for T. The values ${\widehat{\xi }}_{IJ}$ and ${\widehat{\eta }}_{IJ}$ can be found using the following four steps.

Step 1. 

Compute all $x_{ij}$ and $y_{ij}$ by $x_{ij}=x({\xi }_i,{\eta }_j)=x(iH,jH)$ and $y_{ij}=y({\xi }_i,{\eta }_j)=y(iH,jH)$.

Step 2. 

Find all potential $(I,J)\in {\Omega }_{ij}\left({\Omega }_{ij}\stackrel{T}{⟵}{\square }_{ij}\right)$ in XOY, which can be determined by $I_{min}\le I\le I_{max}$ and $J_{min}\le J\le J_{max}$, where

$$\begin{array}{ccc}& & I_{max}=max\left\{\lfloor {\displaystyle \frac{x_{ij}}{H}}\rfloor ,\lfloor {\displaystyle \frac{x_{i+1,j}}{H}}\rfloor ,\lfloor {\displaystyle \frac{x_{i,j+1}}{H}}\rfloor ,\lfloor {\displaystyle \frac{x_{i+1,j+1}}{H}}\rfloor \right\},\hfill \\ & & J_{max}=max\left\{\lfloor {\displaystyle \frac{y_{ij}}{H}}\rfloor ,\lfloor {\displaystyle \frac{y_{i+1,j}}{H}}\rfloor ,\lfloor {\displaystyle \frac{y_{i,j+1}}{H}}\rfloor ,\lfloor {\displaystyle \frac{y_{i+1,j+1}}{H}}\rfloor \right\},\hfill \end{array}$$

where $\lfloor x\rfloor$ is the largest integer $\le x$. The definitions of $I_{min}$ and $J_{min}$ are the same.

Step 3. 

Split ${\square }_{ij}$ into two triangles ${▵}_{ij,1}$ and ${▵}_{ij,2}$ as ${▵}_{ij,\ell }\stackrel{\widehat{T}}{⟶}{\widehat{\Omega }}_{ij,\ell }$, where ${\widehat{\Omega }}_{ij,\ell }$ are also triangles, and find all possible $(I,J)$ such that

$$\begin{array}{ccc}\hfill & & (IH,JH)\in {\widehat{\Omega }}_{ij,\ell }.\hfill \end{array}$$

(22)

The area coordinates ${\sigma }_1$, ${\sigma }_2$ and ${\sigma }_3$ are obtained from the following algebraic system:

$$\begin{array}{ccc}\hfill & & \left\{\begin{array}{c}{x}_I={\sigma }_1{x}_a+{\sigma }_2{x}_b+{\sigma }_3{x}_c,\hfill \\ y_J={\sigma }_1{y}_a+{\sigma }_2{y}_b+{\sigma }_3{y}_c,\hfill \\ 1={\sigma }_1+{\sigma }_2+{\sigma }_3,\hfill \end{array}\right.\hfill \end{array}$$

(23)

where $a,b,c$ are the vertices of triangles ${\widehat{\Omega }}_{ij,\ell },\ell =1,2$ in Figure 3. The sufficient and necessary conditions for (22) are all nonnegative area coordinates $0\le {\sigma }_i\le 1,i=1,2,3$.

Step 4. 

Obtain the approximate values of $({\xi }_{IJ},{\eta }_{IJ})$ related to $(IH,JH)$ in (22) by

$$\begin{array}{ccc}\hfill & & {\widehat{\xi }}_{IJ}={\sigma }_1{\xi }_{\overline{a}}+{\sigma }_2{\xi }_{\overline{b}}+{\sigma }_3{\xi }_{\overline{c}},{\widehat{\eta }}_{IJ}={\sigma }_1{\eta }_{\overline{a}}+{\sigma }_2{\eta }_{\overline{b}}+{\sigma }_3{\eta }_{\overline{c}},\hfill \end{array}$$

(24)

where $\overline{a}$,$\overline{b}$ and $\overline{c}$ are the vertices of ${▵}_{ij,\ell }$ shown in Figure 3.

Note that Steps 1–4 above are easy to carry out. The computation complexity is also only $O\left(M^2\right)$, where $M^2$ is the total pixel number. More importantly, the computation for values, $({\widehat{\xi }}_{IJ},{\widehat{\eta }}_{IJ})\stackrel{{\widehat{T}}^{-1}}{⟵}(x_I,y_J),$ does not involve any nonlinear solutions. Based on the known $({\widehat{\xi }}_{IJ},{\widehat{\eta }}_{IJ})$, the bilinear approximations $({\widehat{\xi }}^{*},{\widehat{\eta }}^{*})$ are formulated by

$$\begin{array}{ccc}\hfill & & \xi (x,y)\approx \widehat{{\xi }^{*}}(x,y)={\widehat{\xi }}_{I+1,J+1}{\displaystyle \frac{(x-IH)(y-JH)}{H^2}}+{\widehat{\xi }}_{I,J+1}{\displaystyle \frac{(I+1)H-x\left)\right(y-JH)}{H^2}}\hfill \\ & & +{\widehat{\xi }}_{I+1,J}{\displaystyle \frac{(x-IH)(y-(J+1\left)H\right)}{H^2}}+{\widehat{\xi }}_{I,J}{\displaystyle \frac{\left(\right(I+1)H-x)\left(\right(J+1)-y)}{H^2}}.\hfill \end{array}$$

(25)

The improved SIM using Techniques I and II is given by

$$\begin{array}{ccc}\hfill & & B_{IJ}\approx {\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)}={\displaystyle \frac{h^2}{H^2}}\sum _{k,\ell =1}^N{\widehat{\varphi }}_{\mu }({\widehat{\xi }}^{*}(x_{\dot{g}},y_{\dot{g}}),{\widehat{\eta }}^{*}(x_{\dot{g}},y_{\dot{g}})),\hfill \end{array}$$

(26)

where $\dot{g}={\dot{g}}_{IJ,k\ell }$ given in (14). The computation complexity of (25) and (26) is $O\left(M^2{N}^2\right)$, where $N={\displaystyle \frac{H}{h}}$. Hence, the CPU time by Techniques I and II is insignificant if comparing $O\left(M^2\right)$ to $O\left(M^2{N}^2\right)$. Note that no nonlinear solutions are needed in (26) either. To distinguish the algorithms using Techniques I and II from SIM and CIIM, we denote (26) by $\mathrm{S}{\mathrm{I}}^{\#}\mathrm{M}$ and the combination of (12) and (26) by $\mathrm{C}{\mathrm{I}}^{\#}\mathrm{IM}$. Technique I was proposed in [6] with a preliminary error analysis, but Technique II was proposed in [9] without error analysis. The error analysis is important and challenging for improved $\mathrm{S}{\mathrm{I}}^{\#}\mathrm{M}$ for T via both Techniques I and II.

Below, we will derive error bounds in the next section by Sobolev norms, which can be applied to discontinuous images.

4. Error Analysis

Algorithm analysis (such as error analysis) is essential to efficiency for numerical partial differential equations (PDEs). In this section, the new error analysis of image greyness by Techniques I and II is twofold.

I. Compared to error analysis in the original CIIM in Li [6], the inverse coordinates for $\xi =\xi (x,y)$ and $\eta =\eta (x,y)$ are not exact. First, the piecewise linear (or bilinear) interpolations $\widehat{\xi }(x,y)$ and $\widehat{\eta }(x,y)$ are used based on the exact values ${\xi }_{I,J}$ and ${\eta }_{I,J}$ in Technique I. We give a strict analysis (ref. [6], Section 6). Next, in Technique II, the inverse values ${\xi }_{I,J}$ and ${\eta }_{I,J}$ are approximated by ${\widehat{\xi }}_{I,J}$ and ${\widehat{\eta }}_{I,J}$, also by using the piecewise linear interpolations. The linear (or bilinear) interpolations $\widehat{\xi }$ and $\widehat{\eta }$ on ${\square }_{IJ}$ yield the same order $O\left(H^2\right)$ of the errors, and the approximations ${\widehat{\xi }}_{I,J}$ and ${\widehat{\eta }}_{I,J}$ on ${\Delta }_{ij,\ell }$ also yield the same error $O\left(H^2\right)$ for continuous images. The improvements in Section 3 will maintain the optimal convergence rates $O\left(H^2\right)+O(H/N^2)$ for $\mu =1$ as in [11] (Chapter 4). Therefore, Algorithms ${\mathrm{SI}}^{\#}\mathrm{M}$ and ${\mathrm{CI}}^{\#}\mathrm{IM}$ circumvent the troublesome nonlinear solutions in the original CIIM in Section 2. For the rather complicated harmonic and blending models, the PDE solutions in Section 5 may be incorporated easily into the discrete algorithms proposed. Hence, the improved algorithms ${\mathrm{SI}}^{\#}\mathrm{M}$ and ${\mathrm{CI}}^{\#}\mathrm{IM}$ may be applied to a wide scale of applications in digital images and patterns.

II. To estimate the errors of discontinuous images, we have to face the image discontinuity; this is a challenging task because discontinuity of image greyness always exists, and because the severe discontinuity may become useless for the continuous analysis of image transformations. In Section 4.2, we do not need the assumption (9) but still solicit the Sobolev norms for error analysis all the way. For the improved techniques in Section 3, the greyness errors, $O\left(H^{\beta }\right)+O\left({\displaystyle \frac{H^{\beta }}{N^2}}\right)$, are given in Corollary 1, where $\beta \in (0,1)$ and $0<H\ll 1$. This is a significant development in error analysis for image transformations, noting that a high smooth greyness of images was often required in our previous papers [6,7]. New splitting techniques in this paper can also be applied to image processing, pattern recognition and AI under geometric transformation in [14,15,16,17].

4.1. Error Analysis for ${\mathrm{SI}}^{\#}\mathrm{M}$

Since the detailed analysis of the combinations for images under $T^{-1}T$ is given in Li [6], we only derive the distinct analysis for ${\mathrm{SI}}^{\#}\mathrm{M}$ for T as $\mu =1$. Let the division number $N=N_p=2^p,p=0,1,2,\dots$, and then, we define the global greyness errors:

$$\begin{array}{ccc}\hfill & & E_2^{\left(N\right)}\left(\widehat{B}\right)={\left\{\sum _{IJ}{\displaystyle \frac{{({\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)}-B_{IJ})}^2}{I_{Num}\left(\widehat{Z}\right)}}\right\}}^{{\displaystyle \frac{1}{2}}},E^{\left(N\right)}\left(\widehat{B}\right)=\sum _{IJ}{\displaystyle \frac{\left|{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)}-B_{IJ}\right|}{I_{Num}\left(\widehat{Z}\right)}},\hfill \end{array}$$

(27)

where ${\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)}$ are given in (26), and the sequential errors

$$\begin{array}{ccc}\hfill & & ▵E_2^{\left(N_p\right)}\left(\widehat{B}\right)={\left\{{\displaystyle \frac{1}{I_{Num}\left(\widehat{Z}\right)}}\sum _{IJ}{\left({\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N_p\right)}-{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N_{p-1}\right)}\right)}^2\right\}}^{{\displaystyle \frac{1}{2}}},\hfill \end{array}$$

(28)

$$\begin{array}{ccc}\hfill & & ▵E^{\left(N_p\right)}\left(\widehat{B}\right)={\displaystyle \frac{1}{I_{Num}\left(\widehat{Z}\right)}}\sum _{IJ}\left|{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N_p\right)}-{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N_{p-1}\right)}\right|,\hfill \end{array}$$

(29)

and the pixel number $I_{Num}\left(\widehat{Z}\right)=M^2\approx {\displaystyle \frac{\left|S\right|}{H^2}},M\gg 1$. The Sobolev norms over ${\mathbf{H}}^n\left(S\right)$ are defined by

$$\begin{array}{ccc}& & {\parallel v\parallel }_{n,S}={\left\{\sum _{\left|\alpha \right|\le n}\int {\int }_S{\left|D^{\alpha }v\right|}^{2}dxdy\right\}}^{1/2},{\left|v\right|}_{n,\Omega }={\left\{\sum _{\left|\alpha \right|=n}\int {\int }_S{\left|D^{\alpha }v\right|}^{2}dxdy\right\}}^{1/2}.\hfill \end{array}$$

We consider the forward transformations T, and construct a cubic spline function passing over $\varphi (iH,jH)={\varphi }_{ij}$ such that $\varphi (\xi ,\eta )\in {\mathbf{C}}^2(\Omega )$ and $b(x,y)=\varphi (\xi (x,y),\eta (x,y))\in {\mathbf{C}}^2\left(S\right)$. Define

$$\begin{array}{ccc}& & B_{IJ}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}b(x,y)dxdy,{\widehat{B}}_{IJ}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}\widehat{b}(x,y)dxdy,\hfill \end{array}$$

where $\widehat{b}(x,y)={\widehat{\varphi }}_{\mu }(\xi (x,y),\eta (x,y))$. The ${\widehat{\varphi }}_0$ and ${\widehat{\varphi }}_1$ denote the piecewise constant and bilinear functions, respectively. The transformation T is said to be regular if $x(\xi ,\eta )\in {\mathbf{C}}^2(\Omega )$, $y(\xi ,\eta )\in {\mathbf{C}}^2(\Omega )$ and the Jacobian determinant J satisfies $0<c_0\le J_0\le J\le J_M$, where $J_0$ and $J_M$ are bounded constants. In this subsection, we assume that the image greyness is continuous to satisfy $\varphi (\xi ,\eta )\in {\mathbf{H}}^2(\Omega )$ and $b(x,y)\in {\mathbf{H}}^2\left(S\right)$ as in (9), where ${\mathbf{H}}^2\left(S\right)$ is the Sobolev space. Such relaxed assumptions in the Sobolev space will be extended to the discontinuous greyness functions $\varphi (\xi ,\eta )$ and $b(x,y)$ in the sequential subsection. We have the following lemma.

Lemma 1. 

Let T be regular, $b(x,y)\in {\mathbf{H}}^2\left(S\right)$ hold, and the interpolation functions with $\mu =0,1$ be used. There exists a bounded constant C independent of μ and h such that

$$\begin{array}{ccc}\hfill & & \left|{\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{\left(N\right)}\right|\le C\left\{\left({\displaystyle \frac{h^2}{H}}\right)\parallel \widehat{b}{\parallel }_{2,{\square }_{IJ}}+{\displaystyle \frac{h^{\mu +\frac{1}{2}}}{H^{\frac{3}{2}}}}{\parallel \widehat{b}\parallel }_{\mu ,\partial {\stackrel{\sim }{\square }}_{IJ}}\right\},\hfill \end{array}$$

(30)

where ${\displaystyle \partial {\stackrel{\sim }{\square }}_{IJ}=\bigcup _{\stackrel{k,\ell }{\mathrm{Case}\mathrm{B}}}{\square }_{IJ,k\ell }}$, and Case B denotes the ${\square }_{IJ,k\ell }$ whose inverse transformed shapes by $T^{-1}$ fall across the boundary $\partial {\square }_{ij}$ of ${\square }_{ij}$ in $\xi o\eta$. For steps 2 and 3 in Figure 1, we have

$$\begin{array}{ccc}\hfill & & {\left\{{\displaystyle \frac{1}{M^2}}\sum _{I,J}|{\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{\left(N\right)}{|}^2\right\}}^{{\displaystyle \frac{1}{2}}}\le {\displaystyle \frac{C}{{\left|S\right|}^{\frac{1}{2}}}}\left\{h^2\parallel \widehat{b}{\parallel }_{2,S}+{\displaystyle \frac{h^{\mu +\frac{1}{2}}}{H^{\frac{1}{2}}}}{\parallel \widehat{b}\parallel }_{\mu ,\partial \tilde{S}}\right\},\hfill \end{array}$$

(31)

where $S={\bigcup }_{IJ}{\square }_{IJ}$ and ${\displaystyle \partial \tilde{S}=\bigcup _{IJ}\partial {\stackrel{\sim }{\square }}_{IJ}}$.

Proof. 

The bounds in (30) are obtained similarly from [11] (Chapter 4), by replacing $\varphi$ and ${\mathsf{\Phi }}_{ij}$ with b and $B_{IJ}$, respectively. Next, from (30), we have

$$\begin{array}{ccc}& & {\left\{{\displaystyle \frac{1}{M^2}}\sum _{I,J}|{\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{\left(N\right)}{|}^2\right\}}^{{\displaystyle \frac{1}{2}}}\le {\displaystyle \frac{C}{M}}{\left\{\sum _{I,J}\left({\left({\displaystyle \frac{h^2}{H}}\right)}^2\parallel \widehat{b}{\parallel }_{2,{\square }_{IJ}}^2+{\left({\displaystyle \frac{h^{\mu +\frac{1}{2}}}{H^{\frac{3}{2}}}}\right)}^2{\parallel \widehat{b}\parallel }_{\mu ,\partial {\stackrel{\sim }{\square }}_{IJ}}^2\right)\right\}}^{{\displaystyle \frac{1}{2}}}\hfill \\ \hfill & & ={\displaystyle \frac{C}{M}}\left\{{\displaystyle \frac{h^2}{H}}\parallel \widehat{b}{\parallel }_{2,S}+{\displaystyle \frac{h^{\mu +\frac{1}{2}}}{H^{\frac{3}{2}}}}{\parallel \widehat{b}\parallel }_{\mu ,\partial S_0}\right\}={\displaystyle \frac{C}{{\left|S\right|}^{\frac{1}{2}}}}\left\{h^2\parallel \widehat{b}{\parallel }_{2,S}+{\displaystyle \frac{C{h}^{\mu +\frac{1}{2}}}{H^{\frac{1}{2}}}}{\parallel \widehat{b}\parallel }_{\mu ,\partial \tilde{S}}\right\}\hfill \\ \hfill & & =O\left({\displaystyle \frac{H^2}{N^2}}\right)+O\left({\displaystyle \frac{H^{\mu }}{N^{\mu +1}}}\right),\hfill \end{array}$$

where $\left|S\right|\approx {\left(MH\right)}^2=O\left(1\right)$ and $|\partial \tilde{S}|\approx M^{2}Hh={\displaystyle \frac{{\left(MH\right)}^2}{N}}=O\left({\displaystyle \frac{1}{N}}\right)$. This completes the proof of Lemma 1. □

For simplicity, we consider only the case of $\mu =1$ and achieve the optimal convergence rates $O\left(H^2\right)+O(H/N^2)$. For $\mu =0$, the analysis may be carried out leading to $O\left(N^{-1}\right)$ and $O_p\left(N^{-1.5}\right)$ in probability in Li and Bai [7]. The difference in analysis between ${\mathrm{SI}}^{\#}\mathrm{M}$ of Section 3 and SIM of Section 2 lies in the evaluation on the approximate errors resulting from $\widehat{\xi }(x_{\dot{g}},y_{\dot{g}})$, $\widehat{\eta }(x_{\dot{g}},y_{\dot{g}})$, ${\widehat{\xi }}^{*}(x_{\dot{g}},y_{\dot{g}})$ and ${\widehat{\eta }}^{*}(x_{\dot{g}},y_{\dot{g}})$ in (19) and (25), instead of the exact values, $\xi (x_{\dot{g}},y_{\dot{g}})$ and $\eta (x_{\dot{g}},y_{\dot{g}})$. Now, we give the grayness errors from Technique I from Strang and Fix [18].

Lemma 2. 

Let $\widehat{\xi }(x,y)$ and $\widehat{\eta }(x,y)$ in (19) be the piecewise bilinear interpolations of $\xi (x,y)$ and $\eta (x,y)$ on ${\square }_{IJ}$, based on the exact values of ${\xi }_{IJ}=\xi (IH,JH)$ and ${\eta }_{IJ}=\eta (IH,JH)$ satisfying the two equations $IH=x({\xi }_{IJ},{\eta }_{IJ})$ and $JH=y({\xi }_{IJ},{\eta }_{IJ})$ exactly. Suppose that $\xi (x,y)\in {\mathbf{H}}^2\left(S\right)$ and $\eta (x,y)\in {\mathbf{H}}^2\left(S\right)$. There exists a bounded constant C independent of H such that

$$\begin{array}{ccc}& & \parallel \xi -\widehat{\xi }{\parallel }_{0,S}+\parallel \eta -\widehat{\eta }{\parallel }_{0,S}\le C{H}^2{\left(\right|\xi |}_{2,S}+{\left|\eta \right|}_{2,S}).\hfill \end{array}$$

In Technique II, we obtain the approximations $({\widehat{\xi }}_{IJ}^{*},{\widehat{\eta }}_{IJ}^{*})$ in (25) from the piecewise linear interpolations $\xi =\widehat{\xi }(x,y)$ and $\eta =\widehat{\eta }(x,y)$ (see Figure 3). Below we prove a lemma.

Lemma 3. 

Let ${\widehat{\xi }}^{*}(x,y)$ and ${\widehat{\eta }}^{*}(x,y)$ be the piecewise linear interpolation based on the approximations: ${\xi }_{IJ}^{*}=\widehat{\xi }(IH,JH)$ and ${\eta }_{IJ}^{*}=\widehat{\eta }(IH,JH)$, where $\widehat{\xi }$ and $\widehat{\eta }$ are the piecewise linear interpolations on ${\bigcup }_{ij,\ell }{\Delta }_{ij,\ell }$ in Lemma 2. Then, there exist the error bounds:

$$\begin{array}{ccc}\hfill & & \parallel {\widehat{\xi }}^{*}(x,y)-\widehat{\xi }{(x,y)\parallel }_{0,S}\le C{\parallel \xi (x,y)-\widehat{\xi }(x,y)\parallel }_{0,S},\hfill \end{array}$$

(32)

$$\begin{array}{ccc}\hfill & & \parallel {\widehat{\eta }}^{*}(x,y)-\widehat{\eta }{(x,y)\parallel }_{0,S}\le C{\parallel \eta (x,y)-\widehat{\eta }(x,y)\parallel }_{0,S}.\hfill \end{array}$$

(33)

Proof. 

Take the linear approximation (20) as an example. We have

$$\begin{array}{ccc}& & {\widehat{\xi }}^{*}(x,y)-\widehat{\xi }(x,y)=-{ϵ}_{I,J}{\displaystyle \frac{(x+y-(I+J+1\left)H\right)}{H}}+{ϵ}_{I+1,J}{\displaystyle \frac{(x-IH)}{H}}+{ϵ}_{I,J+1}{\displaystyle \frac{(y-JH)}{H}},\hfill \end{array}$$

where $ϵ(x,y)={\widehat{\xi }}^{*}(x,y)-\widehat{\xi }(x,y)$, ${ϵ}_{I,J}=ϵ(IH,JH)$, and $((x,y)\in {\square }_{IJ})\wedge ((x+y)\le (I+J+1)H)$. By using the affine transformation $r={\displaystyle \frac{x-IH}{H}}$ and $t={\displaystyle \frac{y-JH}{H}}$, the triangle ${\Delta }_{IJ,1}$ is transformed to a reference triangle $\Delta (r\ge 0,t\ge 0,r+t\le 1)$. Then, we have

$$\begin{array}{ccc}\hfill & & \int {\int }_{{\Delta }_{IJ,1}}{({\widehat{\xi }}^{*}(x,y)-\widehat{\xi }(x,y))}^{2}dxdy\hfill \\ \hfill & & =H^2\int {\int }_{\Delta }{((1-r-t){ϵ}_{IJ}+r{ϵ}_{I+1,J}+t{ϵ}_{I,J+1})}^{2}drdt\hfill \\ \hfill & & \le 3H^2\left\{{ϵ}_{IJ}^2\int {\int }_{\Delta }{(1-r-t)}^{2}drdt+{ϵ}_{I+1,J}^2\int {\int }_{\Delta }{r}^{2}drdt+{ϵ}_{I,J+1}^2\int {\int }_{\Delta }{t}^{2}drdt\right\},\hfill \end{array}$$

(34)

where the integrals are evaluated by calculus

$$\begin{array}{ccc}& & \int {\int }_{\Delta }{(1-r-t)}^{2}drdt={\int }_0^1\left\{{\int }_0^{1-t}{(1-r-t)}^{2}dr\right\}dt={\displaystyle \frac{1}{12}},\int {\int }_{\Delta }{r}^{2}drdt=\int {\int }_{\Delta }{t}^{2}drdt={\displaystyle \frac{1}{12}}.\hfill \end{array}$$

Then, it follows that

$$\begin{array}{ccc}\hfill & & \int {\int }_{{\Delta }_{IJ,1}}{({\widehat{\xi }}^{*}(x,y)-\widehat{\xi }(x,y))}^{2}dxdy\le {\displaystyle \frac{H^2}{4}}({ϵ}_{IJ}^2+{ϵ}_{I+1,J}^2+{ϵ}_{I,J+1}^2).\hfill \end{array}$$

(35)

Since all norms in finite dimensions are equivalent to each other, by noting $ϵ(x,y)=\widehat{\xi }(x,y)-\xi (x,y)=ϵ(r,t)$, we obtain

$$\begin{array}{ccc}\hfill & & ({ϵ}_{IJ}^2+{ϵ}_{I+1,J}^2+{ϵ}_{I,J+1}^2)\le C\int {\int }_{\Delta }{\left(ϵ(r,t)\right)}^{2}drdt\le {\displaystyle \frac{C}{H^2}}\int {\int }_{{\Delta }_{IJ,1}}{(\widehat{\xi }-\xi )}^{2}dxdy,\hfill \end{array}$$

(36)

where C is a bounded constant independent of H. Therefore, we obtain from (35) and (36)

$$\begin{array}{ccc}\hfill & & \parallel {\widehat{\xi }}^{*}-\widehat{\xi }{\parallel }_{0,S}^2=\sum _{IJ,\ell }\int {\int }_{{\Delta }_{IJ,\ell }}{({\widehat{\xi }}^{*}(x,y)-\widehat{\xi }(x,y))}^{2}dxdy\hfill \\ \hfill & & \le C\sum _{IJ,\ell }\int {\int }_{{\Delta }_{IJ,\ell }}{(\widehat{\xi }(x,y)-\xi (x,y))}^{2}dxdy=C{\parallel \widehat{\xi }-\xi \parallel }_{0,S}^2.\hfill \end{array}$$

(37)

This is the desired result (32), and Equation (33) holds similarly. □

Lemma 3 implies that the error order resulting from Technique II is no worse than that from Technique I. Now let us consider ${\mathrm{SI}}^{\#}\mathrm{M}$ using Techniques I and II, and the absolute errors are defined by

$$\begin{array}{ccc}\hfill & & {\left(E_2\left(\widehat{B}\right)\right)}^2=\sum _{IJ}{\displaystyle \frac{{({\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)}-B_{IJ})}^2}{M^2}},\hfill \end{array}$$

(38)

where $B_{IJ}$ and ${\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)}$ are defined in (15) and (26), respectively. Denote

$$\begin{array}{ccc}& & {\widehat{B}}_{IJ}^{\#}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}{\widehat{\varphi }}_1({\widehat{\xi }}^{*},{\widehat{\eta }}^{*})dxdy,{\widehat{B}}_{IJ}^{*}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}{\widehat{\varphi }}_1(\widehat{\xi },\widehat{\eta })dxdy,\hfill \\ & & {\widehat{B}}_{IJ}={\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}{\widehat{\varphi }}_1(\xi ,\eta )dxdy.\hfill \end{array}$$

We have

$$\begin{array}{ccc}\hfill & & {\left(E_2\left(\widehat{B}\right)\right)}^2\le {\displaystyle \frac{4}{M^2}}\{\sum _{IJ}{(B_{IJ}-{\widehat{B}}_{IJ})}^2+\sum _{IJ}{({\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{*})}^2\hfill \\ \hfill & & +\sum _{IJ}{({\widehat{B}}_{IJ}^{*}-{\widehat{B}}_{IJ}^{\#})}^2+\sum _{IJ}{({\widehat{B}}_{IJ}^{\#}-{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)})}^2\}.\hfill \end{array}$$

(39)

To provide the error bounds for (39), we first give two lemmas.

Lemma 4. 

Suppose that $\xi \in {\mathbf{H}}^2(\Omega )$ and $\eta \in {\mathbf{H}}^2(\Omega )$. There exist the error bounds

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{(B_{IJ}-{\widehat{B}}_{IJ})}^2\le {\displaystyle \frac{1}{\left|S\right|}}{\parallel \varphi -{\widehat{\varphi }}_1\parallel }_{0,S}^2,\hfill \end{array}$$

(40)

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{({\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{*})}^2\le {\displaystyle \frac{C}{\left|S\right|}}|{\widehat{\varphi }}_1{|}_{1,S}^2(\parallel \xi -\widehat{\xi }{\parallel }_{0,S}^2+\parallel \eta -\widehat{\eta }{\parallel }_{0,S}^2),\hfill \end{array}$$

(41)

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{({\widehat{B}}_{IJ}^{*}-{\widehat{B}}_{IJ}^{\#})}^2\le {\displaystyle \frac{C}{\left|S\right|}}|{\widehat{\varphi }}_1{|}_{1,S}^2(\parallel \xi -\widehat{\xi }{\parallel }_{0,S}^2+\parallel \eta -\widehat{\eta }{\parallel }_{0,S}^2).\hfill \end{array}$$

(42)

Proof. 

We have from $\left|S\right|=M^2{H}^2$ and the Schwarz inequality

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{(B_{IJ}-{\widehat{B}}_{IJ})}^2={\displaystyle \frac{1}{M^2}}\sum _{IJ}{\left({\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}(\varphi -{\widehat{\varphi }}_1)dxdy\right)}^2\hfill \\ \hfill & & \le {\displaystyle \frac{1}{M^2{H}^4}}\sum _{IJ}\left\{\int {\int }_{{\square }_{IJ}}{(\varphi -{\widehat{\varphi }}_1)}^{2}dxdy\right\}\times \left\{\int {\int }_{{\square }_{IJ}}dxdy\right\}\hfill \\ \hfill & & ={\displaystyle \frac{1}{M^2{H}^2}}\sum _{IJ}\int {\int }_{{\square }_{IJ}}{(\varphi -{\widehat{\varphi }}_1)}^{2}dxdy={\displaystyle \frac{1}{\left|S\right|}}{\parallel \varphi -{\widehat{\varphi }}_1\parallel }_{0,S}^2.\hfill \end{array}$$

(43)

This is the first bound (40).

Next, we have from (43) and the Schwarz inequality

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{({\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{*})}^2={\displaystyle \frac{1}{M^2}}\sum _{IJ}{\left({\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}({\widehat{\varphi }}_1(\xi ,\eta )-{\widehat{\varphi }}_1(\widehat{\xi },\widehat{\eta }))dxdy\right)}^2\hfill \\ \hfill & & \le {\displaystyle \frac{1}{\left|S\right|}}\parallel {\widehat{\varphi }}_1(\xi ,\eta )-{\widehat{\varphi }}_1(\widehat{\xi },\widehat{\eta }){\parallel }_{0,S}^2\le {\displaystyle \frac{C}{\left|S\right|}}\parallel {\widehat{\varphi }}_1{\parallel }_{1,S}^2(\parallel \xi -\widehat{\xi }{\parallel }_{0,S}^2+\parallel \eta -\widehat{\eta }{\parallel }_{0,S}^2).\hfill \end{array}$$

(44)

Third, from Lemma 3 we have similarly

$$\begin{array}{ccc}& & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{({\widehat{B}}_{IJ}^{*}-{\widehat{B}}_{IJ}^{\#})}^2={\displaystyle \frac{1}{M^2}}\sum _{IJ}{\left({\displaystyle \frac{1}{H^2}}\int {\int }_{{\square }_{IJ}}({\widehat{\varphi }}_1(\widehat{\xi },\widehat{\eta })-{\widehat{\varphi }}_1({\widehat{\xi }}^{*},{\widehat{\eta }}^{*}))dxdy\right)}^2\hfill \\ \hfill & & \le {\displaystyle \frac{1}{\left|S\right|}}\parallel {\widehat{\varphi }}_1(\widehat{\xi },\widehat{\eta })-{\widehat{\varphi }}_1({\widehat{\xi }}^{*},{\widehat{\eta }}^{*}){\parallel }_{0,S}^2\le {\displaystyle \frac{C}{\left|S\right|}}{\parallel {\widehat{\varphi }}_1(\xi ,\eta )-{\widehat{\varphi }}_1(\widehat{\xi },\widehat{\eta })\parallel }_{0,S}^2\hfill \\ \hfill & & \le {\displaystyle \frac{C}{\left|S\right|}}\parallel {\widehat{\varphi }}_1{\parallel }_{1,S}^2(\parallel \xi -\widehat{\xi }{\parallel }_{0,S}^2+\parallel \eta -\widehat{\eta }{\parallel }_{0,S}^2).\hfill \end{array}$$

This completes the proof of Lemma 4. □

The Jacobian determinant $J=\left|D\left(z\right)\right|=\left|\begin{array}{cc}{\displaystyle \frac{\partial x}{\partial \xi }}\hfill & {\displaystyle \frac{\partial x}{\partial \eta }}\hfill \\ {\displaystyle \frac{\partial y}{\partial \xi }}\hfill & {\displaystyle \frac{\partial y}{\partial \eta }}\hfill \end{array}\right|$ is given from (18). The nonsingular Jacobian determinant satisfies $J_M\ge J\ge J_0\ge c_0>0$. Denote by ${\sigma }_0(\ge c_0>0)$ the minimal singular value of Jacobian matrix $D\left(z\right)$. Then, we have $D^T\left(z\right)D\left(z\right)\ge {\sigma }_0^2{z}^{T}z$, and give the following lemma.

Lemma 5. 

Let the transformation T be regular. Then, under the inverse transformation $T^{-1}:(x,y)\to (\xi ,\eta )$, there exist the bounds

$$\begin{array}{ccc}\hfill & & {\left|\varphi \right|}_{k,S}\le \sqrt{J_M}{\sigma }_0^{-k}{\left|\varphi \right|}_{k,\Omega },k=0,1,2,\hfill \end{array}$$

(45)

where $J_M$ is the maximal value of the Jacobian determinant, and ${\sigma }_0$ is the minimal singular value of matrix $D\left(z\right)$ in (18).

Proof. 

We have

$$\begin{array}{ccc}\hfill & & {\parallel \varphi \parallel }_{0,S}^2=\int {\int }_S{\varphi }^{2}dxdy=\int {\int }_{\Omega }{\varphi }^{2}Jd\xi d\eta \le J_M\int {\int }_{\Omega }{\varphi }^{2}d\xi d\eta =J_M{\parallel \varphi \parallel }_{0,\Omega }^2.\hfill \end{array}$$

(46)

This is (45) for $k=0$.

Next, since

$$\begin{array}{ccc}& & \left[\begin{array}{c}{\varphi }_{\xi }\hfill \\ {\varphi }_{\eta }\hfill \end{array}\right]=D^T\left(z\right)\left[\begin{array}{c}{\varphi }_x\hfill \\ {\varphi }_y\hfill \end{array}\right],\hfill \end{array}$$

we have

$$\begin{array}{ccc}\hfill & & {\varphi }_{\xi }^2+{\varphi }_{\eta }^2=[{\varphi }_x,{\varphi }_y]D\left(z\right)D^T\left(z\right)\left[\begin{array}{c}{\varphi }_x\hfill \\ {\varphi }_y\hfill \end{array}\right]\ge {\sigma }_0^2({\varphi }_x^2+{\varphi }_y^2).\hfill \end{array}$$

(47)

From (47), we have ${\varphi }_{\xi \xi }^2+{\varphi }_{\xi \eta }^2\ge {\sigma }_0^2({\varphi }_{\xi x}^2+{\varphi }_{\xi y}^2)$ and ${\varphi }_{\eta \xi }^2+{\varphi }_{\eta \eta }^2\ge {\sigma }_0^2({\varphi }_{\eta x}^2+{\varphi }_{\eta y}^2)$. Hence, we obtain

$$\begin{array}{ccc}\hfill & & {\varphi }_{\xi \xi }^2+2{\varphi }_{\xi \eta }^2+{\varphi }_{\eta \eta }^2\ge {\sigma }_0^2({\varphi }_{\xi x}^2+{\varphi }_{\xi y}^2+{\varphi }_{\eta x}^2+{\varphi }_{\eta y}^2)={\sigma }_0^2\{({\varphi }_{x\xi }^2+{\varphi }_{x\eta }^2)+({\varphi }_{y\xi }^2+{\varphi }_{y\eta }^2)\}\hfill \\ \hfill & & \ge {\sigma }_0^4\{({\varphi }_{xx}^2+{\varphi }_{xy}^2)+({\varphi }_{yx}^2+{\varphi }_{yy}^2)\}={\sigma }_0^4({\varphi }_{xx}^2+2{\varphi }_{xy}^2+{\varphi }_{yy}^2).\hfill \end{array}$$

(48)

Consequently, we have from (47) and (48)

$$\begin{array}{c}{\left|\varphi \right|}_{1,S}^2=\int {\int }_S({\varphi }_x^2+{\varphi }_y^2)dxdy\le {\displaystyle \frac{1}{{\sigma }_0^2}}\int {\int }_{\Omega }({\varphi }_{\xi }^2+{\varphi }_{\eta }^2)Jd\xi d\eta \le {\displaystyle \frac{J_M}{{\sigma }_0^2}}\int {\int }_{\Omega }({\varphi }_{\xi }^2+{\varphi }_{\eta }^2)d\xi d\eta ={\displaystyle \frac{J_M}{{\sigma }_0^2}}{\left|\varphi \right|}_{1,\Omega }^2,\hfill \end{array}$$

and

$$\begin{array}{ccc}\hfill & & {\left|\varphi \right|}_{2,S}^2=\int {\int }_S({\varphi }_{xx}^2+2{\varphi }_{xy}^2+{\varphi }_{yy}^2)dxdy\le {\displaystyle \frac{1}{{\sigma }_0^4}}\int {\int }_{\Omega }{({\varphi }_{\xi \xi }^2+2{\varphi }_{\xi \eta }^2+{\varphi }_{\eta \eta }^2)}^{2}Jd\xi d\eta \le {\displaystyle \frac{J_M}{{\sigma }_0^4}}{\left|\varphi \right|}_{2,\Omega }^2.\hfill \end{array}$$

These are (45) for $k=1,2$, and completes the proof of Lemma 5. □

Now we prove a main theorem, which is not only important for error analysis of the improved algorithms in this paper, but also effective for error analysis of discontinuous greyness of images in Section 4.2.

Theorem 1. 

Let $x(\xi ,\eta )\in {\mathbf{H}}^2(\Omega )$ and $y(\xi ,\eta )\in {\mathbf{H}}^2(\Omega )$, and the conditions in Lemma 5 hold. Then, there exists the error bound for $\mu =1$,

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C{\left({\displaystyle \frac{J_M}{\left|S\right|}}\right)}^{{\displaystyle \frac{1}{2}}}\left\{{\displaystyle \frac{H^2}{{\sigma }_0}}|{\varphi }_1{|}_{1,\Omega }{\left(\right|\xi |}_{2,S}+{\left|\eta \right|}_{2,S})+H^2{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_0^2}}{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_{0}H}}{\parallel \varphi \parallel }_{1,\Omega }\right\}.\hfill \end{array}$$

(49)

Proof. 

From Lemmas 1 and 5 for $\mu =1$,

$$\begin{array}{ccc}\hfill & & {\left\{{\displaystyle \frac{1}{M^2}}\sum _{IJ}({\widehat{B}}_{IJ}^{\#}-{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)})\right\}}^{{\displaystyle \frac{1}{2}}}\le {\displaystyle \frac{C}{{\left|S\right|}^{\frac{1}{2}}}}\left\{h^2\parallel {\widehat{b}}_1^{*}{\parallel }_{2,\tilde{S}}+{\displaystyle \frac{h^{\frac{3}{2}}}{H^{\frac{1}{2}}}}{\parallel {\widehat{b}}_1^{*}\parallel }_{1,\partial \tilde{S}}\right\}\hfill \\ \hfill & & \le {\displaystyle \frac{C{J}_M^{\frac{1}{2}}}{{\left|S\right|}^{\frac{1}{2}}}}\left\{{\displaystyle \frac{h^2}{{\sigma }_0^2}}\parallel {\widehat{\varphi }}_1^{*}{\parallel }_{2,\tilde{\Omega }}+{\displaystyle \frac{h^{\frac{3}{2}}}{{\sigma }_0{\left(H\right)}^{\frac{1}{2}}}}{\parallel {\widehat{\varphi }}_1^{*}\parallel }_{1,\partial \tilde{\Omega }}\right\},\hfill \end{array}$$

(50)

where $\tilde{\Omega }$ and $\partial \tilde{\Omega }$ are the original domains of $\tilde{S}$ and $\partial \tilde{S}$. Since $\varphi$ can be regarded as piecewise bi-cubic spline functions, the following bounds hold due to finite-dimensional functions.

$$\begin{array}{ccc}\hfill & & |\partial \Omega |\le {\displaystyle \frac{1}{N}}|\Omega |=C{\displaystyle \frac{h}{H}}|\Omega |,|{\widehat{\varphi }}_1{|}_{1,\partial \Omega }\le C{\left({\displaystyle \frac{h}{H}}\right)}^{{\displaystyle \frac{1}{2}}}{\parallel \varphi \parallel }_{1,\Omega },\parallel {\widehat{\varphi }}_1^{*}{\parallel }_{2,\tilde{\Omega }}\le C{\parallel \varphi \parallel }_{2,\Omega }.\hfill \end{array}$$

(51)

It follows that

$$\begin{array}{ccc}\hfill & & {\left\{{\displaystyle \frac{1}{M^2}}\sum _{IJ}({\widehat{B}}_{IJ}^{\#}-{\left({\widehat{B}}_{IJ}^{\#}\right)}^{\left(N\right)})\right\}}^{{\displaystyle \frac{1}{2}}}\le {\displaystyle \frac{C{J}_M^{\frac{1}{2}}}{{\left|S\right|}^{\frac{1}{2}}}}\left\{{\displaystyle \frac{h^2}{{\sigma }_0^2}}{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_{0}H}}{\parallel \varphi \parallel }_{1,\Omega }\right\}.\hfill \end{array}$$

(52)

Also we have from Lemmas 4 and 5

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}{(B_{IJ}-{\widehat{B}}_{IJ})}^2\le {\displaystyle \frac{1}{\left|S\right|}}\parallel \varphi -{\widehat{\varphi }}_1{\parallel }_{0,S}^2\le C{\displaystyle \frac{J_M}{\left|S\right|}}\parallel \varphi -{\widehat{\varphi }}_1{\parallel }_{0,\Omega }^2\le C{\displaystyle \frac{J_M{H}^4}{\left|S\right|}}{\parallel \varphi \parallel }_{2,\Omega }^2.\hfill \end{array}$$

(53)

Moreover, we have $|{\widehat{\varphi }}_1{|}_{1,S}\le C{\displaystyle \frac{\sqrt{J_M}}{{\sigma }_0}}{\left|\varphi \right|}_{1,\Omega }$ from Lemma 5. Then, we obtain from Lemmas 4 and 2,

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{1}{M^2}}\sum _{IJ}\{{({\widehat{B}}_{IJ}-{\widehat{B}}_{IJ}^{*})}^2+{({\widehat{B}}_{IJ}^{*}-{\widehat{B}}_{IJ}^{\#})}^2\}\hfill \\ \hfill & & \le {\displaystyle \frac{C}{\left|S\right|}}|{\widehat{\varphi }}_1{|}_{1,S}^2(\parallel \xi -\widehat{\xi }{\parallel }_{0,S}^2+\parallel \eta -\widehat{\eta }{\parallel }_{0,S}^2)\le {\displaystyle \frac{C{J}_M{H}^4}{\left|S\right|{\sigma }_0^2}}{\left|\varphi \right|}_{1,\Omega }^2{\left(\right|\xi |}_{2,S}^2+{\left|\eta \right|}_{2,S}^2).\hfill \end{array}$$

(54)

Based on (39) we have from (52)–(54),

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C{\displaystyle \frac{J_M^{\frac{1}{2}}{H}^2}{{\left|S\right|}^{\frac{1}{2}}{\sigma }_0}}|{\varphi }_1{|}_{1,\Omega }{\left(\right|\xi |}_{2,S}+{\left|\eta \right|}_{2,S})+C{\displaystyle \frac{J_M^{\frac{1}{2}}{H}^2}{{\left|S\right|}^{\frac{1}{2}}}}{\parallel \varphi \parallel }_{2,\Omega }\hfill \\ \hfill & & +C{\displaystyle \frac{J_M^{\frac{1}{2}}}{{\left|S\right|}^{\frac{1}{2}}}}\left\{{\displaystyle \frac{h^2}{{\sigma }_0^2}}{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_{0}H}}{\parallel \varphi \parallel }_{1,\Omega }\right\},\hfill \end{array}$$

(55)

to give the desired results (49). This completes the proof of Theorem 1. □

Remark 1. 

For the regular transformation T, we have $\xi ,\eta \in {\mathbf{H}}^2\left(S\right)$, ${\sigma }_0\ge c_0>0$ and $J_M<C$. Then, Equation (49) leads to

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C\left\{H^2+h^2+{\displaystyle \frac{h^2}{H}}\right\}=O\left(H^2\right)+O\left({\displaystyle \frac{H^2}{N^2}}\right)+O\left({\displaystyle \frac{H^{\mu }}{N^2}}\right),\mu =1.\hfill \end{array}$$

(56)

The error bounds (56) are the optimal estimates of the SIM via transformation $T^{-1}$ in [11] (Chapters 3 and 4). The optimal convergence rates (56) remain for the improved $S{I}^{\#}M$ via T without nonlinear solutions. Hence, the $S{I}^{\#}M$ is beneficial for wide applications as in Section 5. Moreover, the error bounds (49) in Sobolev norms can be extended to discontinuous images below.

4.2. Error Analysis for Image Discontinuity

In our past study for image geometric transformations, the greyness functions $\varphi (\xi ,\eta )$ were assumed to be continuous $\varphi \in {\mathbf{H}}^2(\Omega )$ or even $\varphi \in {\mathbf{C}}^2(\Omega )$. However, since $H\ll 1$, we may study the discontinuity of digital images and patterns. In this subsection, we introduce the discontinuity degree. First, let us consider the binary images. If the entire image is either black or white, the discontinuity degree is zero. On the other hand, if the pixel is black when $i+j$ is even, and white when $i+j$ is odd, the discontinuity degree is one. Let $\Omega$ be split into a $2\times 2$ partition of ${\square }_{ij}$ and denoted by ${\square }_{ij}^{2\times 2}$. Denote by ${\stackrel{\sim }{\square }}_{ij}^{2\times 2}$ both the blacks and the whites existing at nine-pixel points (see Figure 4). Let the number $N_{dis}=Num\left({\stackrel{\sim }{\square }}_{ij}^{2\times 2}\right)$. Hence, the discontinuity degree is defined by

$$\begin{array}{ccc}\hfill & & {\rho }_{dis}={\displaystyle \frac{N_{dis}}{Numof{\square }_{ij}^{2\times 2}}}=4{\displaystyle \frac{N_{dis}}{M^2}}={\displaystyle \frac{4N_{dis}}{|\Omega |/H^2}}={\displaystyle \frac{4H^2}{|\Omega |}}{N}_{dis}.\hfill \end{array}$$

(57)

There are four cases of an image in 2D of the image discontinuity with the discontinuity degree from (57).

Figure 4. The square ${\square }^{*}$ and ${\square }_{i,j}^{2\times 2}$ in $3\times 3$ nodes.

(1)

There are only a few isolated pixels,

$$\begin{array}{ccc}& & N_{dis}\le C,{\rho }_{dis}=O\left(H^2\right).\hfill \end{array}$$

(2)

Greyness discontinuity exists only along the interior and exterior image boundary,

$$\begin{array}{ccc}& & N_{dis}=O\left(H^{-1}\right),{\rho }_{dis}=O\left(H\right).\hfill \end{array}$$

(3)

Greyness discontinuity is minor,

$$\begin{array}{ccc}& & N_{dis}=O\left(H^{-\gamma }\right),{\rho }_{dis}=O\left(H^{2-\gamma }\right),\gamma \in (0,2).\hfill \end{array}$$

(4)

All ${\square }_{ij}^{2\times 2}$ are ${\stackrel{\sim }{\square }}_{ij}^{2\times 2}$,

$$\begin{array}{ccc}& & N_{dis}=O\left(H^{-2}\right),{\rho }_{dis}=1.\hfill \end{array}$$

The analysis below in this paper is well suited to Cases (1)–(3). For Case (4), the large, useless errors $O\left(1\right)$ are obtained. Let us prove a new lemma.

Lemma 6. 

For nonuniform binary images in ${\stackrel{\sim }{\square }}_{ij}^{2\times 2}$, the error bounds of piecewise cubic-spline interpolant functions $\varphi (\xi ,\eta )$ passing all ${\varphi }_{ij}$ are given by

$$\begin{array}{ccc}\hfill & & {\left|\varphi \right|}_{\ell ,{\stackrel{\sim }{\square }}_{ij}^{2\times 2}}\le C{H}^{1-\ell },\hfill \end{array}$$

(58)

$$\begin{array}{ccc}\hfill & & {\left|\varphi \right|}_{\ell ,\Omega }\le C\left[1+\sqrt{N_{dis}}{H}^{1-\ell }\right],\ell =0,1,2.\hfill \end{array}$$

(59)

Proof. 

By the affine transformation: $T^{*}:(\xi ,\eta )\to ({\xi }^{*},{\eta }^{*})$,

$$\begin{array}{ccc}\hfill & & {\xi }^{*}={\displaystyle \frac{\xi -{\xi }_{i-1}}{2H}},{\eta }^{*}={\displaystyle \frac{\eta -{\eta }_{j-1}}{2H}},\hfill \end{array}$$

(60)

the region ${\square }_{ij}^{2\times 2}$ is transformed to a unit square ${\square }^{*}$ in Figure 4, where ${\square }^{*}=\left\{({\xi }^{*},{\eta }^{*})\right|0\le {\xi }^{*}\le 1,0\le {\eta }^{*}\le 1\}$. Then, we have ${\left|\varphi \right|}_{\ell ,{\stackrel{\sim }{\square }}_{ij}^{2\times 2}}={\left(2H\right)}^{1-\ell }{\left|{\varphi }^{*}\right|}_{\ell ,{\square }^{*}}$. Note that the same binary images at the nine-pixel points on ${\square }^{*}$ as in ${\stackrel{\sim }{\square }}_{ij}^{2\times 2}$. Let ${\widehat{\varphi }}_2^{*}$ be the bi-quadratic Lagrange polynomial passing through all pixel points: ${\varphi }_{ij}^{*}={\varphi }_{ij}$. Based on the equivalence of finite dimensional norms, we can prove $|{\varphi }^{*}{|}_{\ell ,{\square }^{*}}\le C{\left|{\widehat{\varphi }}_2^{*}\right|}_{\ell ,{\square }^{*}}$. Hence,

$$\begin{array}{ccc}\hfill & & {\left|\varphi \right|}_{\ell ,{\stackrel{\sim }{\square }}_{ij}^{2\times 2}}\le C{H}^{1-\ell }|{\widehat{\varphi }}_2^{*}{|}_{\ell ,{\square }^{*}}\le C{\left|{\widehat{\varphi }}_2\right|}_{\ell ,{\stackrel{\sim }{\square }}_{ij}^{2\times 2}},\hfill \end{array}$$

(61)

where ${\widehat{\varphi }}_2$ is also a bi-quadratic Lagrange interpolant polynomial in $\xi o\eta$.

For the binary images, the bi-quadratic interpolant gives ${\varphi }_2^{*}\stackrel{T}{⟶}{\widehat{\varphi }}_2^{*}$ and ${\widehat{\varphi }}_2(iH,jH)={\varphi }_{ij}$. It is easy to see

$$\begin{array}{ccc}\hfill & & |{\widehat{\varphi }}_2{|}_{\ell ,{\stackrel{\sim }{\square }}_{ij}^{2\times 2}}\le C{H}^{1-\ell }.\hfill \end{array}$$

(62)

The first desired result (58) follows from (61) and (62). Moreover

$$\begin{array}{ccc}& & {\left|\varphi \right|}_{\ell ,\Omega }^2=\sum _{ij}{\left|\varphi \right|}_{\ell ,{\square }_{ij}^{2\times 2}}^2=\sum _{ij}{\left|\varphi \right|}_{\ell ,{\stackrel{\sim }{\square }}_{ij}^{2\times 2}}^2+\sum _{ij\mathrm{left}}{\left|\varphi \right|}_{\ell ,{\square }_{ij}^{2\times 2}}^2=C{N}_{dis}{H}^{2(1-\ell )}+O\left(1\right).\hfill \end{array}$$

This complete the proof of Lemma 6. □

For the q-greyness levels, we may define the greyness jumps of images as

$$\begin{array}{ccc}\hfill & & {\displaystyle \frac{▵\varphi }{q-1}}\ge p,i.e.,p=32\mathrm{for}q=256.\hfill \end{array}$$

(63)

The analysis of binary images can be extended to that of other multiple level images. From Theorem 1 and Lemma 6, we have the following theorem.

Theorem 2. 

Let transformation T be regular and all conditions in Lemma 6 hold. When $N_{dis}\ge 1$, there exist the bounds:

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C\sqrt{N_{dis}}\left\{H+{\displaystyle \frac{H}{N^2}}\right\}.\hfill \end{array}$$

(64)

Proof. 

From Theorem 1 and Lemma 6, we have

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C{\displaystyle \frac{{\left(N_{dis}{J}_M\right)}^{\frac{1}{2}}H}{{\left|S\right|}^{\frac{1}{2}}}}\left\{{\displaystyle \frac{H}{{\sigma }_0}}{\left(\right|\xi |}_{2,S}+{\left|\eta \right|}_{2,S})+1+{\displaystyle \frac{1}{N^2}}({\displaystyle \frac{1}{{\sigma }_0^2}}+{\displaystyle \frac{1}{{\sigma }_0}})\right\}.\hfill \end{array}$$

(65)

The regular transformation T implies $\xi ,\eta \in {\mathbf{H}}^2\left(S\right)$, ${\sigma }_0\ge c_0>0$, $J_M<C$. We have

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C\sqrt{N_{dis}}\left\{H^2+H+{\displaystyle \frac{H}{N^2}}\right\}\le C\sqrt{N_{dis}}\left\{H+{\displaystyle \frac{H}{N^2}}\right\}.\hfill \end{array}$$

(66)

This is the desired result (64) and completed the proof of Theorem 2. □

From Theorem 2, we have the following corollary.

Corollary 1. 

Let all conditions in Theorem 1 hold. For the finite greyness jumps $N_{dis}\le C$, the absolute errors are

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)=O\left(H\right)+O(H/N^2).\hfill \end{array}$$

(67)

When the images have with greyness jumps of interior and exterior boundaries $N_{dis}=O\left(H^{-1}\right)$,

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)=O\left(\sqrt{H}\right)+O(\sqrt{H}/N^2).\hfill \end{array}$$

(68)

When $N_{dis}=O\left(H^{-\gamma }\right)$ with $\gamma \in (0,2)$, denote $\beta =1-{\displaystyle \frac{\gamma }{2}}\in (0,1)$. We have

$$\begin{array}{ccc}\hfill & & E_2=E_2\left(\widehat{B}\right)=O(H^{1-{\displaystyle \frac{\gamma }{2}}})+O(H^{1-{\displaystyle \frac{\gamma }{2}}}{N}^{-2})=O\left(H^{\beta }\right)+O(H^{\beta }{N}^{-2}).\hfill \end{array}$$

(69)

Remark 2. 

When the majority greyness of images is continuous, we have $N_{dis}=O\left(H^{-\beta }\right)$ with $\beta \approx 1$. The errors $E_2$ are small due to small $H(\ll 1)$ from (65). This is an important development from [6,7], where images are often assumed to be continuous. Note that the sequential errors $\Delta E_2=O\left(N^{-2}\right)$ as $\mu =1$ still hold. However for Case (4): the full discontinuous images, we have $E_2=O\left(1\right)$ from (65) with $\beta =0$ and $\gamma =2$, which is meaningless. Another error analysis for greyness discontinuity is reported for other kinds of splitting algorithms in [11] (Chapter 8).

4.3. Summary of Six Combinations for the Cycle Transformation $T^{-1}T$

Here let us give a summary of various combinations for the cycle transformation $T^{-1}T$. There are six combinations with $\mu =0$ and $\mu =1$ in 2D images under $T^{-1}T$: (1) $\mathrm{CSIM}$, (2) $\mathrm{C}\overline{\mathrm{S}}\mathrm{IM}$ and $\mathrm{C}\overline{\mathrm{S}}\overline{\mathrm{I}}\mathrm{M}$, (3) $\mathrm{C}{\overline{\mathrm{S}}}^{\#}{\overline{\mathrm{I}}}^{\#}\mathrm{M}$, (4) $\mathrm{CIIM}$, (5) ${\mathrm{CI}}^{*}\mathrm{IM}$, and (6) ${\mathrm{CI}}^{\#}\mathrm{IM}$ in this paper. We list their characteristics, accuracy and convergence in Table 1 with $\mu =1$ only. The details of algorithms and rigor analysis are provided in a recent book [11]. Furthermore, a list of symbols used is listed in Appendix A.

Table 1. Comparisons of six combinations for $T^{-1}T$ with $\mu =1$ in their characteristics, accuracy and convergence.

Combinations	References	Solve Nonlinear Eqs.	Increasing N	Sequential Errors	Absolute Errors (Continuous Greyness)	Absolute Errors (Discontinuous Greyness)
$\mathrm{CSIM}$	[1,7]	No	Yes	$O(1/N)$ and $O_p(1/N^{1.5})$	$O\left(H^2\right)$	/
$\mathrm{C}\overline{\mathrm{S}}\mathrm{IM}$	[11] (Chapter 7)	No	Yes	$O(1/N^2)$	$O\left(H^2\right)$	/
$\mathrm{C}{\overline{\mathrm{S}}}^{\#}{\overline{\mathrm{I}}}^{\#}\mathrm{M}$	[11] (Chapter 8)	No	Not but $\ge N_0$	/	$O\left(H\right)$	$O\left(H\right)$
$\mathrm{CIIM}$	[6]	Yes	Yes		$O\left(H^2\right)$	/
${\mathrm{CI}}^{*}\mathrm{IM}$	[6]	Yes	Yes	$O(1/N^2)$	$O\left(H^2\right)$	/
${\mathrm{CI}}^{\#}\mathrm{IM}$	[9] and this paper	No	Yes		$O\left(H^2\right)$	$O\left(H^{\alpha }\right),0<\alpha <1$

In summary, combination $\mathrm{CSIM}$ in [1,7] is the simplest, but suffers from low sequential errors $O(1/N)$ and $O_p(1/N^{1.5})$. Combination $\mathrm{CIIM}$ in [11] (Chapter 4) gains the better convergence rates, $O(1/N^{\mu +1})$. When $\mu =1$ using the piecewise bilinear interpolations ${\widehat{\varphi }}_1(\xi ,\eta )$ and ${\widehat{b}}_1(x,y)$, the sequential convergence rates $O(1/N^2)$ are optimal. For $\mu =0$ using the piecewise constant interpolations ${\widehat{\varphi }}_0(\xi ,\eta )$ and ${\widehat{b}}_0(x,y)$, the performance of $\mathrm{CIIM}$ is still satisfactory. However, $\mathrm{CIIM}$ suffers from nonlinear solutions during the forward transformation T. Consequently, for the inverse transformation $T^{-1}$, $\mathrm{CIIM}$ is recommended if the nonlinear solutions are easy using iteration methods. The original $\mathrm{CSIM}$ is suited well to low numbers of greyness-level images; but $\mathrm{CIIM}$ is suited to both low and high numbers of greyness level (≥256) images. The improved ${\mathrm{CI}}^{\#}\mathrm{IM}$ in [9] may completely bypass the nonlinear solutions, and its rigor analysis is explored in this paper. The improved ${\mathrm{CI}}^{\#}\mathrm{IM}$ are particularly beneficial for harmonic models with the finite element method, finite difference method and finite volume method because the approximate transformation $\widehat{T}$ obtained is already piecewise linear (see [8,9]).

To promote accurate images with no need for nonlinear solutions, $\mathrm{C}\overline{\mathrm{S}}\mathrm{IM}$ for $\mu =1$ and $\mathrm{C}\overline{\mathrm{S}}\overline{\mathrm{I}}\mathrm{M}$ for $\mu =0$ in [11] (Chapter 7) have been developed to have the optimal convergence rates $O(1/N^2)$. The sophisticated combinations $\mathrm{C}{\overline{\mathrm{S}}}^{\#}{\overline{\mathrm{I}}}^{\#}\mathrm{M}$ in [11] (Chapter 8) may be easily carried out without sequential errors when $N\ge N_0$. Moreover, an analysis in [11] (Chapter 8) reveals the better absolute error $O\left(H\right)$ for all kinds of image discontinuity.

5. Applications to Harmonic, Poisson and Blending Models

5.1. Descriptions of the Models

Let us consider the arbitrary geometric shape transformations T by the given boundaries in both original and distorted images (see Figure 5). Assume $x(\xi ,\eta ),y(\xi ,\eta )\in {\mathbf{C}}^2(\Omega )$, we may define the functions $x(\xi ,\eta )$ and $y(\xi ,\eta )$ by satisfying the Poisson equations

$$\begin{array}{ccc}\hfill & & ▵x={\displaystyle \frac{{\partial }^{2}x}{\partial {\xi }^2}}+{\displaystyle \frac{{\partial }^{2}x}{\partial {\eta }^2}}=f_1(\xi ,\eta )\mathrm{in}\Omega ,\hfill \end{array}$$

(70)

$$\begin{array}{ccc}\hfill & & ▵y={\displaystyle \frac{{\partial }^{2}y}{\partial {\xi }^2}}+{\displaystyle \frac{{\partial }^{2}y}{\partial {\eta }^2}}=f_2(\xi ,\eta )\mathrm{in}\Omega ,\hfill \end{array}$$

(71)

and the Dirichlet boundary conditions:

$$\begin{array}{ccc}\hfill & & {x|}_{\partial \Omega }=g_1{(\xi ,\eta ),y|}_{\partial \Omega }=g_2(\xi ,\eta ).\hfill \end{array}$$

(72)

The transformation of (70)–(72) is called the Poisson model. When $f_1(\xi ,\eta )=f_2(\xi ,\eta )=0$, Equations (70) and (71) are reduced to the harmonic equations:

$$\begin{array}{ccc}\hfill & & ▵x=▵y=0,(\xi ,\eta )\in \Omega .\hfill \end{array}$$

(73)

Figure 5. The transformation of arbitrary domains.

The transformation of (72) and (73) is called the harmonic model. The harmonic model was proposed in Li, et al. [1] and then in [9,11] for combinations CSIM and CIIM.

In this paper, we also propose blending models by assuming that the shapes can be blended as a thin elastic plate. This leads to the following two blending models.

• Simply supported blending models:

  $$\begin{array}{ccc}\hfill & & {▵}^{2}x=f_1(\xi ,\eta ),{▵}^{2}y=f_2(\xi ,\eta ),\hfill \end{array}$$

  (74)

  $$\begin{array}{ccc}\hfill & & {x|}_{\partial \Omega }=g_1{(\xi ,\eta ),y|}_{\partial \Omega }=g_2(\xi ,\eta ),\hfill \end{array}$$

  (75)

  $$\begin{array}{ccc}& & {▵x|}_{\partial \Omega }{=▵y|}_{\partial \Omega }=0.\hfill \end{array}$$

  (76)
• The clamped blending models: Equations (74), (75) and $x_{\nu }=y_{\nu }=0$ on $\partial \Omega$, where $x_{\nu }={\displaystyle \frac{\partial x}{\partial \nu }}$, and $\nu$ is the outward normal of $\partial \Omega$. In the blending models (74)–(76), we assume $x,y\in {\mathbf{C}}^4(\Omega )$.

Since the white pixels can be added to images without any changes, the original image domains may be assumed to be rectangular or even square. The rectangular and square images have many applications, e.g., pictures, TV, etc. Below we will use the simple finite difference method (FDM) for harmonic, Poisson and blending models. The practical approximations $\tilde{x}$ and $\tilde{y}$ can be obtained and easily embedded into ${\mathrm{SI}}^{\#}\mathrm{M}$ and ${\mathrm{CI}}^{\#}\mathrm{IM}$. The error analysis for Poisson models in Section 5.3 shows that the optimal sequential errors will be maintained as well. Furthermore, the error analysis of the CSIM is made for harmonic, models by the FEM in [11] (Chapter 10).

5.2. The Finite Difference Method

Let $\Omega$ be a unit square and the pixel points be chosen as the difference grid nodes. For the same partition $\Omega ={\bigcup }_{ij}{\square }_{ij}$, the standard five-node schemes of FDM for (70)–(72) are given by

$$\begin{array}{ccc}\hfill & & 4x_{i,j}-(x_{i+1,j}+x_{i-1,j}+x_{i,j+1}+x_{i,j-1})=-H^2{f}_{i,j},1\le i,j\le M-1,\hfill \end{array}$$

(77)

$$\begin{array}{ccc}\hfill & & x_{i,j}=g_1(iH,jH),i=0,M\wedge j=0,M,\hfill \end{array}$$

(78)

and $H=1/M$. The equations for $y_{ij}$ are similar to (77) and (78). The successive over-relaxation iteration (SOR) may be chosen to solve (77) and (78) by

$$\begin{array}{ccc}\hfill & & x_{i,j}^{(k+1)}=x_{i,j}^{\left(k\right)}-{\displaystyle \frac{w}{4}}\left\{4x_{i,j}^{\left(k\right)}-\left(x_{i-1,j}^{(k+1)}+x_{i,j-1}^{(k+1)}+x_{i+1,j}^{\left(k\right)}+x_{i,j+1}^{\left(k\right)}\right)+H^2{f}_{i,j}\right\},\hfill \end{array}$$

(79)

where w is the relaxation parameter. The iteration (79) is convergent when $0<w<2$ and optimal when (see Hageman and Young [19])

$$\begin{array}{ccc}\hfill & & w=w_{opt}={\displaystyle \frac{2}{1+\sqrt{1-{\rho }^2}}}={\displaystyle \frac{2}{1+sin\left(\pi H\right)}},\hfill \end{array}$$

(80)

where $\rho$ is the spectral radius of the Jacobian iteration matrix. For the case of (77) and (78), $\rho =cos\left(\pi H\right)$. Since the iteration number is $O\left(M\right)$, the total CPU time for the FDM solutions is $O\left(M^3\right)$. For Dirichlet problems of PDE, the domain boundary $\partial \Omega$ must be given. The harmonic modes by using the FDM in (77)–(80) are used for face imaging resembling (face morphing) in a recent paper [8], where face boundary formulation is studied in detail.

Next, for the simply supported blending model (74)–(76), we may split into two Poisson models. Let $v=▵x$ and $w=▵y$. We have two Poisson models.

(I)

Poisson Model A.

$$\begin{array}{ccc}\hfill & & ▵v=f_1(\xi ,\eta ),▵w=f_2(\xi ,\eta ),v{{|}_{\partial \Omega }=w|}_{\partial \Omega }=0.\hfill \end{array}$$

(81)

(II)

Poisson Model B.

$$\begin{array}{ccc}\hfill & & {▵x=v,▵y=w,x|}_{\partial \Omega }=g_1{(\xi ,\eta ),y|}_{\partial \Omega }=g_2(\xi ,\eta ).\hfill \end{array}$$

(82)

We may also apply the same SOR to two models (81) and (82). As to the clamped blending models, we may obtain the FDM (see Figure 6a)

$$\begin{array}{ccc}\hfill & & 20x_{i,j}-8(x_{i+1,j}+x_{i-1,j}+x_{i,j+1}+x_{i,j-1})+2(x_{i+1,j+1}+x_{i+1,j-1}+x_{i-1,j+1}+x_{i-1,j-1})\hfill \\ \hfill & & +(x_{i+2,j}+x_{i-2,j}+x_{i,j+2}+x_{i,j-2})=H^4{f}_1(iH,jH),\hfill \end{array}$$

(83)

where the boundary conditions are given by

$$\begin{array}{ccc}\hfill & & x_{i,j}=f_1(iH,jH),i=0,M\wedge j=0,M,\hfill \\ \hfill & & x_{-1,j}=x_{1,j},x_{M+1,j}=x_{M-1,j},j=0,M,\hfill \\ \hfill & & x_{i,-1}=x_{i,1},x_{i,M+1}=x_{i,M-1},i=0,M.\hfill \end{array}$$

(84)

The difference equations for $y_{i,j}$ are similar. The block SOR can be used for solving (83) and (84) (see Hageman and Young [19]).

Figure 6. Two partitions in the FDM.

When $\Omega$ is an arbitrary bounded domain, the finite element method (FEM) may be chosen for the Poisson and the blending models. However, the programming of the FEM is more complicated than that of the FDM, in particular for the biharmonic equations in (74). Furthermore, the finite volume method (FVM) with Delaunay triangulation is used in [11] (Chapter 10). The FDM with SOR is easier to carry out for image transformations. It is worth pointing out that the FDM (77) is also a kind of FEM using linear elements of Figure 6b, or bilinear elements in Figure 6a.

Remark 3. 

The standard five-node difference equations (77) are valid for uniform meshes as digital images. Rigor error analysis is given in Theorem 3 below for the improved $S{I}^{\#}M$ by Techniques I and II to give the error $E_2=O\left(H^2\right)+O\left(h^2\right)=O\left(H^2\right)+O\left({\displaystyle \frac{H^2}{N^2}}\right)$. The Laplace operator can be used as a contour filter to test the edges of digital images; see Winnicki et al. [20]. The contour filter may be used for pre-processing to face boundary formulation for harmonic/Poisson models in [8]. Various $3\times 3$ difference schemes of nine-nodes: $(i+k,j+\ell )$ with $k,\ell =0,\pm 1$, are discussed in [20]. Suppose the solutions $u\in {\mathbf{C}}^6(\Omega )$. The best $3\times 3$ difference scheme is found in [20] ((16)–(18)) (called the fourth-order filter), to offer higher accuracy of the Laplace operator. The bounds of (87) can be improved to $\parallel x-{\widehat{x}}_h{\parallel }_{0,\Omega }+{\parallel y-{\widehat{y}}_h\parallel }_{0,\Omega }=O\left(H^4\right)$. Such a better difference scheme may also be used for harmonic/Poisson models. They are beneficial for the numerical algorithms by using piecewise-spline interpolations with $\mu =2,3$ in [11] (Chapter 10) due to the high errors requested.

5.3. Error Analysis for the FDM Solutions in ${\mathrm{SI}}^{\#}\mathrm{M}$

The FDM was used for harmonic transformations in [8,9], but no error analysis exists so far. In this subsection, we provide a new error analysis. In Section 2, $x_{IJ}=x(IH,JH)$ and $y_{IJ}=y(IH,JH)$ are exact. Since the values $x(\xi ,\eta )$ and $y(\xi ,\eta )$ can also be approximately obtained by FDM, $\widehat{\xi }$ and $\widehat{\eta }$ may be obtained approximately from ${\tilde{x}}_{IJ}={\widehat{x}}_h(IH,JH)$ and ${\tilde{y}}_{IJ}={\widehat{y}}_h(IH,JH)$, which is similar to Section 3. To embed the FDM solutions $x_h$ and $y_h$ into SIM, we first give a lemma.

Lemma 7. 

Let $x_h$ and $y_h$ be the piecewise bilinear interpolant on ${\bigcup }_{ij,\ell }{\Delta }_{ij,\ell }$, and ξ and η be their inverse bilinear functions on ${\bigcup }_{ij,\ell }{\widehat{\Omega }}_{ij,\ell }$, where ${\widehat{\Omega }}_{ij,\ell }\stackrel{{\widehat{T}}^{-1}}{⟵}{\Delta }_{ij,\ell }$ by $\widehat{T}$. Then, there exist the bounds,

$$\begin{array}{ccc}\hfill & & \parallel \xi -{\widehat{\xi }}_h{\parallel }_{0,S}+{\parallel \eta -{\widehat{\eta }}_h\parallel }_{0,S}\le {\displaystyle \frac{\sqrt{J_M}}{{\sigma }_0}}\left(\parallel x-{\widehat{x}}_h{\parallel }_{0,\Omega }+{\parallel y-{\widehat{y}}_h\parallel }_{0,\Omega }\right),\hfill \end{array}$$

(85)

where ${\sigma }_0$ is the minimal value of matrix $D\left(z\right)$ in (18), and the Jacobian determinant J satisfies $0<J_0\le J\le J_M$.

Proof. 

From the Lagrange mean theorem in calculus, we have

$$\begin{array}{ccc}& & \left[\begin{array}{c}x-{\widehat{x}}_h\hfill \\ y-{\widehat{y}}_h\hfill \end{array}\right]=D\left(\tilde{z}\right)\left[\begin{array}{c}\xi -{\widehat{\xi }}_h\hfill \\ \eta -{\widehat{\eta }}_h\hfill \end{array}\right],\hfill \end{array}$$

where $\tilde{z}$ is a mean value. Then, we have ${(\xi -{\widehat{\xi }}_h)}^2+{(\eta -{\widehat{\eta }}_h)}^2\le {\displaystyle \frac{1}{{\sigma }_0^2}}({(x-{\widehat{x}}_h)}^2+{(y-{\widehat{y}}_h)}^2)$ to lead to

$$\begin{array}{ccc}& & \parallel \xi -{\widehat{\xi }}_h{\parallel }_{0,S}^2+{\parallel \eta -{\widehat{\eta }}_h\parallel }_{0,S}^2=\int {\int }_S({(\xi -{\widehat{\xi }}_h)}^2+{(\eta -{\widehat{\eta }}_h)}^2)dxdy\hfill \\ \hfill & & \le {\displaystyle \frac{1}{{\sigma }_0^2}}\int {\int }_{\Omega }({(x-{\widehat{x}}_h)}^2+{(y-{\widehat{y}}_h)}^2)Jd\xi d\eta \le {\displaystyle \frac{J_M}{{\sigma }_0^2}}\left(\parallel x-{\widehat{x}}_h{\parallel }_{0,\Omega }^2+{\parallel y-{\widehat{y}}_h\parallel }_{0,\Omega }^2\right).\hfill \end{array}$$

This completes the proof of Lemma 7. □

Theorem 3. 

Let all conditions in Theorem 1 hold. Suppose that ${\widehat{x}}_h$ and ${\widehat{y}}_h$ are the approximate solutions of the Poisson models by the FDM (or the linear FEM), there exist the error bounds of image errors for $\mu =1$.

$$\begin{array}{ccc}\hfill & & E_2\left(\widehat{B}\right)\le C{\left({\displaystyle \frac{J_M}{\left|S\right|}}\right)}^{{\displaystyle \frac{1}{2}}}\left\{{\displaystyle \frac{J_M^{\frac{1}{2}}{H}^2}{{\sigma }_0^2}}{\left|\varphi \right|}_{1,\Omega }{\left(\right|x|}_{2,\Omega }+{\left|y\right|}_{2,\Omega })+H^2{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_0^2}}{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_{0}H}}{\parallel \varphi \parallel }_{1,\Omega }\right\}.\hfill \end{array}$$

(86)

Proof. 

We evaluate the greyness errors of the FDM. The FDM can be regarded as the linear FEM in the regular triangulation (see Figure 6b). The FDM solutions, $\widehat{x}=x_h$ and $\widehat{y}=y_h$, have the error bounds for Poisson’s equation:

$$\begin{array}{ccc}\hfill & & \parallel x-{\widehat{x}}_h{\parallel }_{0,\Omega }+\parallel y-{\widehat{y}}_h{\parallel }_{0,\Omega }\le C{H}^2{\left(\right|x|}_{2,\Omega }+{\left|y\right|}_{2,\Omega }).\hfill \end{array}$$

(87)

From (55), (87), Lemma 7 and Theorem 1, we obtain

$$\begin{array}{ccc}& & E_2\left(\widehat{B}\right)\le {\displaystyle \frac{C}{{\left|S\right|}^{\frac{1}{2}}}}{\displaystyle \frac{J_M}{{\sigma }_0^2}}{\left|\varphi \right|}_{1,\Omega }(\parallel x-{\widehat{x}}_h{\parallel }_{0,\Omega }+\parallel y-{\widehat{y}}_h{\parallel }_{0,\Omega })\hfill \\ \hfill & & +C{\left({\displaystyle \frac{J_M}{\left|S\right|}}\right)}^{{\displaystyle \frac{1}{2}}}{H}^2{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{C{J}_M^{\frac{1}{2}}}{{\left|S\right|}^{\frac{1}{2}}}}\left({\displaystyle \frac{h^2}{{\sigma }_0^2}}{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_{0}H}}{\parallel \varphi \parallel }_{1,\Omega }\right)\hfill \\ \hfill & & \le C{\displaystyle \frac{J_M^{\frac{1}{2}}}{{\left|S\right|}^{\frac{1}{2}}}}\left\{{\displaystyle \frac{J_M^{\frac{1}{2}}{H}^2}{{\sigma }_0^2}}{\left|\varphi \right|}_{1,\Omega }{\left(\right|x|}_{2,\Omega }+{\left|y\right|}_{2,\Omega })+H^2{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_0^2}}{\parallel \varphi \parallel }_{2,\Omega }+{\displaystyle \frac{h^2}{{\sigma }_{0}H}}{\parallel \varphi \parallel }_{1,\Omega }\right\}.\hfill \end{array}$$

This is the desired bound (86) and completes the proof of Theorem 3. □

Since Theorems 1 and 2 are similar to each other, all conclusions of error analysis for discontinuous images in Section 4.2 are valid.

6. Numerical and Graphical Experiments

6.1. Binary Images

Consider binary images with the conversion,

$$\begin{array}{ccc}& & {\widehat{W}}_{ij}=\left\{\begin{array}{ccc}\prime *\prime \hfill & \mathrm{if}\hfill & {\mathsf{\Phi }}_{ij}\ge {\displaystyle \frac{1}{2}},\hfill \\ \prime +\prime \hfill & \mathrm{if}\hfill & {\displaystyle \frac{1}{10}}\le {\mathsf{\Phi }}_{ij}<{\displaystyle \frac{1}{2}},\hfill \\ \prime .\prime \hfill & \mathrm{if}\hfill & 0<{\mathsf{\Phi }}_{ij}<{\displaystyle \frac{1}{10}},\hfill \\ \prime \prime \hfill & \mathrm{if}\hfill & {\mathsf{\Phi }}_{ij}=0,\hfill \end{array}\right.\hfill \end{array}$$

where $\prime +\prime$ and $\prime .\prime$ are used to display the greyness errors less than ${\displaystyle \frac{1}{2}}$. We carry out combination ${\mathrm{CI}}^{\#}\mathrm{IM}$ for $T^{-1}T$ without nonlinear solutions. Choose the simple bi-quadratic model in [1] (p. 127). The greyness errors are listed in Table A1 and Table A2, and the graphical images are given in Figure A1 and Figure A2. In Table A1 and Table A2, E, $E_2$, $\Delta E$ and $\Delta E_2$ are given in (27)–(29). Also $\Delta I_1$, $\Delta I_2$ and $\Delta I_3$ denote the numbers of pixel errors for $\prime *\prime$, $\prime +\prime$ and $\prime .\prime$, respectively.

6.2. Harmonic and Poisson’s Models

Face recognition is an active research area due to wide applications, and we cite some recent reports [21,22,23,24,25,26]. Let us choose Lena’s images in [7,9], which consists of $256\times 256$ pixels with 256 greyness. Lena’s images were the standard model for testing in engineering. However, Lena’s images are no longer allowed in IEEE publications after 1 April 2024. Readers may refer to our original papers if necessary. We add a darkest square boundary for Lena’s images as in Figure A1 and choose the transformed boundary as that of the quadratic model, as in Figure A2. See Figures 12–18 in [7]. First, we use the ${\mathrm{CI}}^{\#}\mathrm{IM}$ for $T^{-1}T$ with the harmonic transformation T, with the same boundary of bi-quadratic transformation in [1] (p. 127). Next, consider the Poisson model (70)–(72), where the functions are given by

$$\begin{array}{ccc}& & f_1=f_2=1\mathrm{in}{\Omega }_0\subset \Omega ,f_1=f_2=0\mathrm{in}\Omega \setminus {\Omega }_0,\hfill \end{array}$$

and ${\Omega }_0$ is a smaller rectangle as ${\Omega }_0=\left\{(\xi ,\eta )\right|{\displaystyle \frac{1}{3}}\le \xi \le {\displaystyle \frac{2}{3}},{\displaystyle \frac{1}{3}}\le \eta \le {\displaystyle \frac{2}{3}}\}$. The exterior boundary is also chosen as that of the quadratic model. In this paper, we employ the FDM. The numerical results of greyness and pixel errors are listed in Table A3, Table A4, Table A5 and Table A6 for $\mu =0$ and $\mu =1$. In Table A3, Table A4, Table A5 and Table A6, $P.E.$ denotes the greyness-level errors in the 256 greyness-level system. Also, $Tot.Pix.$ denotes the total number of non-empty pixels of the distorted images under T. In Table A5 and Table A6, the absolute errors of E and $E_2$ under the transformation T are computed from (27), and the sequential errors of $\Delta E$ and $\Delta E_2$ are computed from (28) and (29), respectively. For the cycle transformation $T^{-1}T$, the sequential errors of $\Delta E$ and $\Delta E_2$ are defined similarly. The curves of $\Delta E$ under $T^{-1}T$ are drawn in Figure A3 from Table A5 and Table A6. We can see from Figure A3,

$$\begin{array}{ccc}\hfill & & \Delta E=O\left(N^{-2}\right)\mathrm{as}\mu =1,\hfill \end{array}$$

(88)

$$\begin{array}{ccc}\hfill & & \Delta E=O\left(N^{-1.5}\right)\mathrm{as}\mu =0.\hfill \end{array}$$

(89)

Equation (88) is consistent with (68) and Theorem 3 with $\mu =1$ by noting $\Delta E\le C\Delta E_2$. Equation (89) is consistent with $\Delta E=O_p\left(N^{-1.5}\right)$ in probability in [7]. Note that the greyness discontinuity does happen, and the error analysis in Section 4.2 is valid and important in practical applications.

When the face boundary is changeable, the face images are also changeable [8]. The face transformations may be applied to the moving pictures as Sora. The harmonic models developed by using FDM in this paper may also be applied for face recognition and face morphing (see [8]). In addition, we also cited other reports [21,22,23,24,25,26,27,28] for this subject.

To close this section, let us give a remark.

Remark 4. 

In this remark, we also apply the numerical algorithms to deep learning in AI. The numerical algorithms in Table 1 are the core of Chapters 2–8 of a new book [11]. The focus of this paper is error analysis for geometric transformations using Techniques I and II. The link between numerical algorithms and deep learning-based transformation may also be found in [11]. In [29,30,31], camera photos and the marine image videos are used; they may suffer from perspective transformations. The projective and perspective transformations are discussed in Chapter 1 of [11]. Moreover, in [32,33,34], the 3D videos and 3D geometric morphology are discussed, while the efficient numerical algorithms in 3D are explored in Chapter 9 of [11]. Feature-based approaches are discussed in [35,36,37], where line segments are basic features as handwritten characters. Their images are often found by cameras and X-rays. When geometric distortions and illumination effects are involved, the numerical algorithms in this paper and [11] can be used to remove them. Then, the correct pattern recognition can be made. Moreover, in [8,9] and Chapter 10 of [11], we provided more applicable examples in AI, such as handwriting recognition, hiding secret images, face image fusion, and morphing attack detection.

7. Concluding Remarks

Compared to our previous study of digital images under geometric transformations (see [1,6,7,9]), we address several novelties in this paper.

1. For the images under a nonlinear transformation T, the original SIM requires the nonlinear solutions, although the number of nonlinear equations may be reduced to $O\left(M^2\right)$ in Li [6]. In Li, et al. [9], we proposed interpolation techniques that entirely bypass nonlinear solutions for the geometric images. The new numerical algorithms are called the improved ${\mathrm{SI}}^{\#}\mathrm{M}$ for T and Combination ${\mathrm{CI}}^{\#}\mathrm{IM}$ under $T^{-1}T$. This paper provides error analysis for the improved ${\mathrm{SI}}^{\#}\mathrm{M}$ by using Techniques I and II.

2. The error bounds are given in Theorem 1 based on the Sobolev norms. The optimal convergence rates $O\left(H^2\right)+O(H/N^2)$ are obtained for the piecewise bilinear interpolations ($\mu =1$) and smooth images, where $H(\ll 1)$ is the mesh resolution of an optical scanner, and N is the division number of a pixel split into $N^2$ sub-pixels.

3. For the images in which the portion of discontinuity is minor, the error bounds are given in Theorem 2. For general cases of discontinuous images, the error bounds are given in Corollary 1 as $O\left(H^{\beta }\right)+O(H^{\beta }/N^2)$, $\beta \in (0,1)$ when $\mu =1$.

4. The combination ${\mathrm{CI}}^{\#}\mathrm{IM}$ can be applied to the harmonic, Poisson and blending models in Section 5 and their approximate solutions ${\widehat{x}}_h$ and ${\widehat{y}}_h$ may be obtained by the FDM using the SOR. The programming of the FDM is much easier than that of the FEM and the finite volume method (FVM), and the CPU time greatly reduced.

5. Numerical and graphical experiments are carried out in Section 6. The real images of $256\times 256$ (and $504\times 631$) pixels of 256 greyness-levels are carried out to show the importance of the improved algorithms. The numerical experiments in Section 6 also support the error analysis made.

6. New numerical algorithms for image geometric transformations in this paper are beneficial to computer vision, image processing and pattern recognition. Applications of numerical algorithms particularly to deep learning in AI are also given in Remark 4. Modern AI systems (e.g., ChatGPT, DeepSeek, Manus, Sora) rely on three core elements: data, models and algorithms. As emphasized in [8,10,11], efficient algorithms are critical for optimizing computational resources. Six combinations of numerical algorithms for the images under $T^{-1}T$ are summarized in Table 1.