Progressive Iterative Approximation with Preconditioners

: The progressive iterative approximation (PIA) plays an important role in curve and surface ﬁtting. By using the diagonally compensated reduction of the collocation matrix, we propose the preconditioned progressive iterative approximation (PPIA) to improve the convergence rate of PIA. For most of the normalized totally positive bases, we show that the presented PPIA can accelerate the convergence rate signiﬁcantly in comparison with the weighted progressive iteration approximation (WPIA) and the progressive iterative approximation with different weights (DWPIA). Furthermore, we propose an inexact variant of the PPIA (IPPIA) to reduce the computational complexity of the PPIA. We introduce the inexact solver of the preconditioning system by employing some state-of-the-art iterative methods. Numerical results show that both the PPIA and the IPPIA converge faster than the WPIA and DWPIA, while the elapsed CPU times of the PPIA and IPPIA are less than those of the WPIA and DWPIA.


Introduction
The progressive iterative approximation (PIA) is an important iterative method for fitting a given set of data points. Due to its clear geometric meaning, stable convergence and simple iterative scheme, the PIA has intrigued researchers for decades. In particular, said methodology has achieved great success in geometric design [1,2], data fitting [3,4], reverse engineering [5] and NURBS solid generation [6][7][8]. For more detailed research on this topic, we refer the reader to read a recent survey [9]. However, the PIA converges very slowly, especially when the number of interpolating points increases to some extent. This is because the collocation matrices resulting from some normalized totally positive (NTP) bases are usually ill-conditioned; see [10,11]. Therefore, much more attention was paid to accelerating the convergence rate of PIA; see the weighted progressive iteration approximation (WPIA) (also named modified Richardson method) in [12,13]; the QR-PIA in [11]; the Jacobi-PIA in [14]; the Chebyshev semi-iterative method in [15]; the progressive iterative approximation with different weights (DWPIA) in [16]; the GS-PIA in [17]; the PIA with mutually different weights in [18]; the Schulz iterative method in [19]; and a lot literature therein.
There are other remedy strategies, such as locally interpolating, i.e, interpolating partial points selected from the total given points, which results in the local progressive-iterative approximation format [20], the extended progressive-iterative approximation format [21] and the progressive iterative approximation format for least square fitting [22]. Let {b i (t)} n i=0 be an NTP basis, such as the Bernstein basis [23] or the Said-Ball basis [24]. Given a set of data points {v i } n i=0 in R 2 or R 3 , we assign a parameter value t i to the i-th point v i . For PIA, the initial interpolating curve is given by = v i , i = 0, 1, · · · , n, and the (k + 1)-th interpolating curve can be generated by Equation (1) can be rewritten as the following form which generates a sequence of control polygons u (k+1) i , for k = 0, 1, 2, · · · .
Let ∆ (k) = δ Then the matrix form of Equations (1) and (2) is where U (0) = V , ∆ (k) = V − BU (k) , I is the identity matrix and B is the collocation matrix resulting from the basis {b 0 (t), b 1 (t), · · · , b n (t)} at t i , i = 0, 1, · · · , n. We note in [23] that the collocation matrix B resulting from any NTP basis is a stochastic totally positive matrix. If lim k→∞ γ (k) (t i ) = v i , then the sequence of curves γ (k) (t) is said to have the PIA property and Equation (2) is referred to as the PIA format. It is shown in [25] that the PIA property holds for any NTP basis.
In fact, the PIA is mathematically equivalent to the Richardson iteration for solving the collocation matrix equation see [15,26]. Therefore, the state-of-the-art iterative method, the generalized minimal residual (GMRES) method for solving general linear systems of equations, was suggested in [26] for solving (4). It is known that the GMRES method works efficiently when the coefficient matrix B is positive definite. It is true that the GMRES has better convergence behavior than PIA in the same required precision when fitting 19 points on the helix. However, we found that the collocation matrix B resulting from some NTP bases is not always positive definite, even if B is a totally positive matrix. This led us to further study the PIA. Obviously, the convergence of PIA depends on the spectral radius of the iteration matrix I − B. The smaller the spectral radius is, the faster the PIA converges. This motivated us to use the preconditioning techniques to reduce the spectral radius of the iteration matrix I − B. The PIA consists of the interpolatory PIA and the approximating PIA [9]. In the first case, the number of the control points is the same as that of the given data points. In the second case, the number of the selected control points is less than that of the given data points. In this paper, we focus on preconditioning the interpolatory PIA. The proposed preconditioning technique can be extended straightforwardly to the approximating PIA. The paper is organized as follows. In Section 2, we first exploit the diagonally compensated reduction of the collocation matrix B to construct a class of preconditioners for B; then we establish the PPIA (preconditioned PIA) format and analyze its convergence. In order to improve the computational efficiency of the PPIA, we further propose an IPPIA (inexact PPIA) format and analyze its convergence in Section 3. Finally, some numerical examples are given to illustrate the effectiveness of our proposed methods in Section 4.

The Preconditioned Progressive Iterative Approximation (PPIA)
As mentioned earlier, the PIA does always converge although slowly. In this section, we therefore consider the preconditioning technique to accelerate the convergence rate of PIA. Preconditioning means replacing the initial system (4) with the system where P is an approximation of B with the following properties: P −1 B is well conditioned or has few extreme eigenvalues, and Pz = d is easy to solve or P −1 is easy to obtain. A careful and problem-dependent choice of P can often make the condition number of P −1 B much smaller than that of B and thus accelerate convergence dramatically.

Preconditioners
In this subsection, we focus on preconditioner P for the collocation matrix B resulting from an NTP basis. Very often, the preconditioner P should be chosen to preserve certain structures of the original matrix B. There are many applications that generate structured matrices, and by exploiting the structure, one may be able to design faster and/or more accurate algorithms; see for example [27][28][29]; furthermore, structure may also help in producing solutions which have more precise physical meanings. Structure comes in many forms, including Hamiltonian, Toeplitz, Vandermonde matrices and so on. Note that B is a stochastic matrix, so we hope that the preconditioner P is also a stochastic matrix. Moreover, in order to get P −1 easily, we restrict P to be a banded matrix with small bandwidth. Based on those two requirements, we construct P as follows.
We first split B into B = B q + R, where and B q = B − R is a banded matrix with bandwidth 2q + 1(0 ≤ q ≤ n). Then we define a diagonal matrix such that De = Re with e = (1, 1, · · · , 1) T . Finally, let be the diagonally compensated reduction matrix of B. The matrix R is called the reduced matrix and D is called the compensation matrix for R; see for instance [30,31]. It is easy to check that P is a stochastic banded matrix and its construction time is small enough and can be neglected, because it only involves a matrix-vector multiplication. If P in (8) is invertible, it can serve as a preconditioner for the system (4). Unfortunately, we cannot give a theoretical proof about the nonsingularity of P, but numerous numerical experiments demonstrate P is invertible for different order n with various q. Therefore, we have Accordingly, the PIA for (9) becomes the following format where U (0) = V , or equivalently where For convenience, we refer to (10) as the PPIA format.

Convergence Analysis of PPIA
Before analyzing the convergence, we first investigate some properties of the collocation matrix B resulting from the Bernstein basis.

Proposition 1.
For the Bernstein basis, if t i = i n , i = 0, 1, · · · , n, then the entries of the collocation matrix B are and possess the following properties: (a) The diagonal entry b j (t j ) is the maximum of all entries which locate at the j-th row and j-th column; i.e., Proof. It was proven in [23] that the Bernstein basis has the unimodality property in the interval [0, 1]; i.e., b j (t) has a unique extremum at t = j/n in the interval [0, 1]. Moreover, for any fixed t * ∈ [0, 1], there exists a corresponding index j such that In other words, b j (t) reaches its maximum at the value t * which is the closest to j/n; thus Property (a) holds. Properties (b) and (c) can be proved in a similar way, by using the unimodality property of the Bernstein basis.
On one hand, according to the definition of D in (7), we have From (8) and (11), we further have On the other hand, from (6), we have For each fixed j, it is easy to verify that both b j+q+1 (t j ) and b j−q−1 (t j ) are monotonically decreasing and finally decay to zero, as q increases. Hence, any of sum of each row's entries outside of the q-th diagonal decays to 0. This means that R ∞ approaches zero if q → n. Combined with (12), we conclude that the matrix P is a good approximation of B. Thus, the spectrum of P −1 B approaches 1 for large q.

Remark 1.
We remark here that the spectral radius ρ(I − P −1 B) may decrease as q increases; i.e., the bigger the q is, the faster the PPIA format converges. However, according to (10), one needs to compute P −1 ∆ (k) or equivalently solve an additional matrix equation PX (k) = ∆ (k) at the k(k = 0, 1, · · · )-th iteration. This will result in a large computational complexity, especially for large q. Therefore, we need to seek a tradeoff between the convergence rate and computational complexity. Experimentally, we found that q ≈ n 2 is a suitable choice.

Remark 2.
We emphasize that most of the NTP bases have the unimodality property, so Proposition 1 also holds for other NTP bases with unimodality. This means that our PPIA will work well for such a class of bases, such as the Said-Ball basis. This was experimentally verified in our numerical tests.

The Inexact Preconditioned Progressive Iterative Approximation (IPPIA)
As mentioned in Remark 1, we need to compute P −1 ∆ (k) or equivalently solve PX (k) = ∆ (k) at the k-th iteration. This is very costly and impractical in actual implementations for large n. To reduce the computational complexity, we propose an IPPIA format in which we inexactly solve PX (k) = ∆ (k) by employing a state-of-the-art iterative method, that is, the conjugate gradient (CG) method for solving the related system of normal equations.

The Inexact Solver
Denote by {Ũ (k) } ∞ k=0 the iteration sequence generated from IPPIA format defined below. Since the sequence Now, let X (l,k) be the l-th approximation of the solution of which can be obtained by employing the CG method to solve the system of normal equations The CG method used here can be thought of as an inner iterative format and terminates if where τ k is the prescribed stopping tolerance and · is the Frobenius norm.
Very often, to ensure the computational efficiency, the iteration (16) can also be terminated if the stopping criterion, is satisfied, where ε (k) defined by is the interpolation error of the k-th interpolating curve γ (k) (t).
Since the term P −1 ∆ (k) in (10) is replaced by an inexact solution X (l,k) , the iterative sequence {Ũ (k) } ∞ k=0 does not necessarily converge to the exact solution U * . Clearly, the selection of τ k will affect the convergence of iterative sequence {Ũ (k) } ∞ k=0 . If τ k equals to zero, the solution obtained by the CG method is the exact one for the system (14). In this case, the IPPIA format is reduced into the PPIA format. Therefore, one can expect that the iterative sequence has a good convergence rate if τ k is chosen to be small enough. The selection of τ k is discussed in the following subsection.

Convergence Analysis of IPPIA
Theorem 1. Let P be the preconditioner defined as in (8) for B and U * be the exact solution to (4). Suppose that the CG method starts from an initial guess X (0,k) , terminates if (15) is satisfied and produces an inner iterative sequence X (l,k) for each outer iterative index k, where X (l,k) is an approximation of P −1 ∆ (k) . Then, for the iterative sequence Ũ (k) ∞ k=0 generated by IPPIA, we have (10), we have Let U * be the exact solution of (4). Then According to (16), (20) and (21), we have Due to the nonsingularity of P, (22) can be transformed intõ Since BU * = V , from the hypothesis (15), we have Combining (23) with (24), we have Since I − P −1 B F < 1, we have from (25) that π max < 1 for small enough τ max . The proof is thus complete.
Theorem 1 tells us that the error of the IPPIA iteration consists of two parts: I − P −1 B F and τ k P −1 F P −T F B F . The first part is the error upper bound for PPIA iteration. The second part is the error upper bound resulting from the CG method, depending on the choice of the stopping tolerance τ k . This τ k is hard to be optimally determined between the convergence rate and computational costs in practice.
The following theorem presents one possible way of choosing the tolerance τ k .

Theorem 2.
Let the assumptions in Theorem 1 be satisfied. Suppose that {ζ k } is a nondecreasing and nonnegative sequence satisfying ζ (k) ≥ 1, and lim k→∞ sup ζ (k) = +∞, and that χ is a real constant in the interval (0, 1) satisfying where c is a nonnegative constant. Then we have i.e., the convergence rate of IPPIA is asymptotically the same as that of PPIA. (25) and (26), we have

Remark 3.
Theorem 2 tells us that the convergence rate of IPPIA is asymptotically the same as that of PPIA if the tolerance sequence {τ k } approaches 0 as k increases. To reduce the computational complexity, it is not necessary for τ k to approach zero in actual implementations, which will be shown numerically in the next section.

Numerical Experiments
In this section, some numerical examples are given to illustrate the effectiveness of our methods. All the numerical experiments were done on a PC with Intel(R) Core(TM) i5-5200U CPU @2.20 GHz by Matlab R2012b.
In our numerical experiments, we used the Bézier curves and the Said-Ball curves to test the following three examples [12,17,26].
For comparison, we also tested the WPIA and the DWPIA. The parameter values t i and the optimal weight ω used in our numerical experiments were the same as those used in [12]; i.e., t i = i/n, for i = 0, 1, · · · , n and ω = 2/(1 + λ n ) with λ n being the smallest eigenvalue of B. Moreover, we set the parameter a = 1.9 when we tested the DWPIA. Example 1. Consider the lemniscate of Gerono given by (x(t), y(t)) = (cos t, sin t cos t) , t ∈ − π 2 , 3π 2 .
A sequence of n + 1 points {v i } n i=0 is taken from the lemniscate of Gerono as follows:
A sequence of n + 1 points {v i } n i=0 is taken from the helix as below v i = 5 cos 6π n i , 5 sin 6π n i , 6π n i , i = 0, 1, · · · , n.
A sequence of n + 1 points {v i } n i=0 is taken from the four-leaf clover in the following way v i = 4 sin i 2π n sin i 8π n , 4 cos i 2π n sin i 8π n , i = 0, 1, · · · , n.

Tests for PPIA
Numerically, we found that the half-bandwidth of the preconditioner P in (8) q ≈ n/2 is a good choice. Therefore, when we used Bézier curves and Said-Ball curves to test Example 1 with n = 10, we took q = 5 and 6, respectively. Similarly, when we used Bézier curves and Said-Ball curves to test Example 2 with n = 18, we took q = 9 and 12, respectively; when we used Bézier curves and Said-Ball curves to test Example 3 with n = 21, we took q = 11 and 14, respectively.
The spectral distributions of I − B and I − P −1 B resulting from Bézier curves and Said-Ball curves in Examples 1-3 are illustrated in Figures 1-3, respectively. From those figures, we can see that most of the eigenvalues of the preconditioned iteration matrices I − P −1 B are far away from 1 and clustered around 0 in each of six cases.  To further illustrate the effectiveness of the PPIA, we list in Table 1 the spectral radii of the iteration matrices of the WPIA, DWPIA and PPIA formats resulting from the considered Bézier and Said-Ball curves. Clearly, the spectral radii of the iteration matrices of the PPIA are much less than those of the corresponding WPIA, DWPIA. Additionally, we see that the spectral radii of PPIA decline sharply when the half-bandwidth of the preconditioner P is near n/2. Those results entirely coincide with the aforementioned conclusions in Section 2. Therefore, we can expect that the PPIA method has a good convergence behavior.
We denote the number of iterations by k, the k-th interpolation error by ε (k) and the elapsed CPU time after k iterations by t (k) (in seconds), respectively. We emphasize here that the elapsed CPU time for the same routine in Matlab was different every time. Therefore, we took the average of all runtimes of 100 independent runs as the elapsed CPU time. The detailed numerical results of the PPIA, DWPIA and WPIA resulting from Bézier curves and Said-Ball curves are listed in Tables 2-4, respectively. Under the requirement of the same precision, the number of iterations by PPIA is much less than those of WPIA and DWPIA, and the runtime by PPIA is much less than the runtimes of WPIA and DWPIA. This means that our PPIA can accelerate the convergence rate of PIA significantly.
In Table 5, we show the detailed runtimes of WPIA, DWPIA and PPIA for when we tested Example 3 by using Bézier curves and Said-Ball curves. We denote the runtime of computing the optimal weight for WPIA by T 1 , the runtime of computing different weights for DWPIA by T 2 and the runtime of constructing the preconditioner P for PPIA by T 3 , respectively. We can see that the cost of constructing P is small enough and can be neglected.       We remark here that all curves in Figures 4-6 are those obtained at 10th iteration step by employing different iterations with different bases, respectively. In Figures 4-6, the curves generated by PPIA and WPIA formats are denoted by the solid and dashed lines respectively. We can see from Figures 4-6 that the approximations obtained by PPIA are better than those obtained by WPIA, no matter whether Bézier curves or Said-Ball curves are used. Clearly, the PPIA converges faster than the WPIA.

Tests for IPPIA
From the above numerical experiments, we know that the PPIA works much better than the WPIA and DWPIA if the number of the interpolating points is moderate. However, when we fit 53 points sampled from the lemniscate of Gerono, the helix and the four-leaf clover in Examples 1-3 by using Bézier curves and Said-Ball curves, we found that the WPIA and DWPIA converge very slowly. At the same time, we found that the PPIA does not work, because the condition number of P is very large and the solution to P∆ (k) =∆ (k) cannot be directly obtained. In this case, we can employ the IPPIA to get the corresponding fitting curves. The numerical results (including the number of iterations, the approximation errors and the runtime) obtained by the WPIA, DWPIA and IPPIA are listed in Tables 6-8 respectively, where the parameters in IPPIA are taken as q = 26, θ = 0.98 and τ k = 10 −2 .
From Tables 6-8, we can see that the IPPIA converges much faster than the WPIA and DWPIA. For example, when we fit the points sampled from the lemniscate of Gerono by using Bézier curves, we observed that the errors caused by performing 5 IPPIA iterations are about the same as those caused by performing 137 WPIA iterations or 381 DWPIA iterations. Again, when we fit 53 points sampled from the lemniscate of Gerono by using Said-Ball curves, we found that the errors caused by performing 4 IPPIA iterations are about the same as those caused by performing 291 WPIA iterations or 358 DWPIA iterations. Furthermore, under the requirement of the same accuracy, the runtime of IPPIA iterations was less than that of WPIA iterations and it was comparable to that of DWPIA iterations.
When fitting 53 points sampled from the lemniscates of Gerono, helix and four-leaf clover, we sketched the Bézier curves and the Said-Ball curves generated by performing five IPPIA and WPIA iterations in Figures 7-9, respectively. The solid and dashed lines in Figures 7-9 represent the curves generated by IPPIA and WPIA respectively. Clearly, the IPPIA outperforms the WPIA in the sense of the convergence rate.

Conclusions
In this paper, we have exploited the properties of collocation matrix B, resulting from an NTP basis, to design an efficient preconditioner P for PIA. The main tool to derive our preconditioner is the diagonally compensated reduction. By applying that P to PIA, we have developed a novel version of PIA, called the PPIA. By both convergence theory and numerical experiments, we have shown that the PPIA outperforms the WPIA and DWPIA, in the sense of convergence rate and elapsed CPU time. In particular, to reduce the computational complexity of PPIA, we have also proposed an inexact variant of PPIA (i.e., IPPIA) and shown its effectiveness, especially when the number of interpolating points is large.
As mentioned in Section 1, the PIA has extensive successful applications. Therefore, we can expect that our methods have a wide range of potential applications.