Derivative-Free Kurchatov-Type Accelerating Iterative Method for Solving Nonlinear Systems: Dynamics and Applications

: Two novel Kurchatov-type ﬁrst-order divided difference operators were designed, which were used for constructing the variable parameter of three derivative-free iterative methods. The convergence orders of the new derivative-free methods are 3, ( 5 + √ 17 ) /2 ≈ 4.56 and 5. The new derivative-free iterative methods with memory were applied to solve nonlinear ordinary differential equations (ODEs) and partial differential equations (PDEs) in numerical experiments. The dynamical behavior of our new methods with memory was studied by using dynamical plane. The dynamical planes showed that our methods had good stability.


Introduction
The main advantage of derivative-free methods is that a derivative-free method can be used for finding the solution of a non-differentiable nonlinear function F(t). Compared to Newton's method [1], derivative-free iterative methods do not require derivative in iterations. Numerous derivative-free iterative methods for finding the solution of nonlinear systems F(t) = 0 have been proposed, where F : D ⊆ R n → R n . Traub [2] proposed the well-known derivative-free iterative method s (k) = t (k) + VF(t (k) ), t (k+1) = t (k) − [s (k) , t (k) ; F] −1 F(t (k) ), (1) where V ∈ R − {0}. [s (k) , t (k) ; F] is the first-order divided difference operator, which is defined by [3] [s (k) , t (k) ; F] ij = The error equation of the method in (1) is ε (k+1) = D 2 (I + VF (η))(ε (k) ) 2 + O((ε (k) ) 3 where η ∈ R n is the solution of F(t) = 0, ε (k) = t (k) − η and D k = 1 k! F (η) −1 F (k) (η) ∈ L k (R n , R n ), (k = 2, 3, . . . ). By adding one new iterative step to the method in (1), Chicharro et al. [4] designed the following iterative method with order three: (k) , t (k) ; F] −1 F(t (k) ), t (k+1) = z (k) − [s (k) , z (k) ; F] −1 F(z (k) ), (4) satisfying the error equation ε (k+1) = D 2 2 (I + VF (η)) 2 (ε (k) ) 3 + O((ε (k) ) 4 ). (5) Let us assume that V = −F (η) −1 in (3) and (5); then, the order of the methods in (1) and (4) can be improved. Unfortunately, the solution η ∈ R n is unknown. Generally, we handle this problem by replacing the constant parameter V with a variable parameter V (k) . The method with variable parameter V (k) is called method with memory. The variable parameter V (k) is designed by using iterative sequences from the current and previous steps. Substituting the variable parameter V (k) = −[s (k−1) , t (k−1) ; F] −1 for constant parameter V in the method in (1), Ahmad et al. [5] and Petković et al. [6] designed some efficient multistep derivative-free iterative methods with memory. In order to design a new iterative method with memory, Kurchatov [7] obtained the following one-step method with memory: where ; F] is called Kurchatov's divided difference operator. Argyros et al. [8,9] studied the convergence properties of Kurchatov's method (6) in the Banach space and proved that Kurchatov's method is second-order convergent. Wang et al. [10] and Cordero et al. [11] presented some Kurchatov-type methods with memory by using Kurchatov's divided difference operator. [4] obtained the following method with memory: where The method in (7) is called FM3 in Chicharro's paper [4]. Substituting V (k) = −(A (k) ) −1 for the parameter V in (4), they obtained the following scheme with memory: The method in (8) is called FM5 in Chicharro's paper [4]. They concluded that the convergence orders of the FM3 and FM5 methods were 3 and 5, respectively. However, their conclusions about convergence order were incorrect and the results of numerical experiment were inconsistent with their theoretical results. The main reason of this mistake is that they wrongly used the following equation in the paper [4]: where For the iterative FM3 and FM5 methods, the error ε (k) is less than the error (ε (k−1) ) 2 . So, Equation (9) should be written as Theorem 1. Let the nonlinear function F : D ⊂ R n → R n be sufficiently differentiable and η ∈ R n be a zero of F. Let us assume that the initial value t (0) is close to η. Then, the FM3 and FM5 methods have convergence orders 1 + √ 3 ≈ 2.732 and 4, respectively.
Proof. Let r and p be the convergence orders of the FM3 and FM5 methods, respectively. The error relation of FM3 can be written as where A k,r is the asymptotic error constant. So, By replacing I + VF (η) of (3) with (11), we obtain the error expression of the FM3 method.
The asymptotic error constants A k,r A r k−1,r in (13) and D 2 D 3 A 2 k−1,r in (14) do not affect the convergence order of the iterative method. By equating the appropriate exponents of ε (k−1) in (13) and (14), we have By solving Equation (15), we obtain the positive root r = 1+ √ 3 ≈ 2.732. This implies that the order of the FM3 method is 2.732.
Similar to Equations (12) and (13), the error relation of the FM5 method can be given by and By replacing I + VF (η) of (5) with (11), we obtain the error relation From (17) and (18), we obtain By solving Equation (19), we obtain p = 4 and p = −1. The convergence order of iterative method should be positive, so p = −1 is discarded. This implies that the convergence order of the FM5 method is four.
Theorem 1 obtains the true order of the FM3 and FM5 methods.
Inspired by the FM3 and FM5 methods, we propose three iterative methods with a novel variable parameter in the next section.
The structure of this paper is as follows: The design of two novel divided difference operators as the variable parameters of three derivative-free iterative methods with memory for solving nonlinear systems is presented in Section 2. Using the new Kurchatovtype divided difference, the new derivative-free iterative methods reached the orders 3, (5 + √ 17)/2 ≈ 4.56 and 5, respectively. In Section 3, the application of the proposed methods to solve the ODEs, PDEs , the standard nonlinear systems and the non-differentiable nonlinear systems is presented. In Section 4, the dynamical behavior of the presented method is studied for analyzing the stability of the presented methods. In Section 5, we give a short summary.

Two New Kurchatov-Type Accelerating Derivative-Free Iterative Methods with Memory
In this section, two new first-order divided difference operators are used for constructing the variable parameter V (k) . Similar to Kurchatov's first-order divided difference, we construct the following first-order divided differences and We call (20) and (21) Kurchatov-type first-order divided differences. (10) and (11), we obtain where ε (10) and (11), we obtain where ε By substituting (22) for V (k) of the FM3 method, we obtain the following scheme with memory: The convergence order of the proposed scheme (24) is given by the following theorem.
Theorem 2. Let F : D ⊂ R n → R n be a sufficiently differentiable function in an open neighborhood D and η ∈ R n be a zero of function F. Let us assume that the initial value t (0) is close enough to η. Then, the convergence order of the iterative scheme in (24) is three.

Remark 2.
We note that the method in (24) and the FM3 method have the same computational costs with different convergence orders. Our method (24) has a higher convergence order than the FM3 method. So, the computational efficiency of the FM3 method is less than that of the method in (24). By replacing V (k) of the FM5 method with (22), we obtain the following scheme with memory: By replacing V (k) of the FM5 method with (23), we obtain The convergence orders of the proposed schemes in (30) and (31) are given by the following result. Theorem 3. Let F : D ⊂ R n → R n be a sufficiently differentiable function in an open neighborhood D and η ∈ R n be a zero of function F. Let us assume that the initial value t (0) is close enough to η. Then, the iterative methods in (30) and (31) have convergence orders (5+ √ 17)/2 ≈ 4.56 and 5, respectively.
From (37) and (38), we have and By letting lim k→∞ (r k+1 /r k ) = lim k→∞ (r k /r k−1 ) = Or, we obtain where Or is a constant. By solving Equation (41), we obtain Or = 5 + √ 17 /2. This implies that the convergence order of the method in (30) is Or = 5 + √ 17 /2 ≈ 4.56. From (5) and (23), we obtain and This implies that the order of the method with memory in (31) is five.
Remark 3. Theorem 3 shows that the convergence order of the method in (4) without memory is improved from three to five by using one accelerating parameter (23). The convergence orders of the methods in (30) and (31) are higher than that of the FM5 method. The computational efficiency index (CEI) [3] is defined by CEI = ρ 1 c , where ρ is the convergence order and c is the computational cost of the iterative method. The methods in (30) and (31) and the FM5 method have different convergence orders with the same computational costs. We know that the computational efficiency of our methods shown in (30) and (31) is superior to that of the FM5 method. (24) and (30) use the same accelerating parameter (22). In Theorems 2 and 3, we use the different error relations of the parameter (22). The main reason is that (ε (22) is less than ε (k) for the two-step method method in (24) and (ε (22) is more than ε (k) for the three-step method in (30).

Remark 5.
In order to simplify calculation, the computational scheme of the method in (30) can be written as 3 .

(44)
The computational scheme of the method in (31) can be written as 3 .

(45)
The linear systems in (37) and (38) were solved by LU decomposition in numerical experiments.
Tables 1-6 show the following information: ACOC [12] is the approximated computational order of convergence, NIT (number of iterations) is the number of iterations, EVL is evaluation of error ||t (k) − t (k−1) || at the last step, T is the CPU time and FU is function values at the last step. The iterative processes of the iterative methods are given by Figures 1-5.
The solution of problem 1 is η = {1, 1, · · · , 1} T and the initial value is t (0) = {1.25, 1.25, · · · , 1.25} T .   Figure 1 shows that the method in (31) had higher accuracy than the other methods. The accuracy of the method in (24) was similar to that of the FM3 method for Problem 1.

Problem 5.
x The solution is η = {1.0207, 0.27988} T and the initial value is x (0) = {0.9638, 0.2376} T .  Problem 6. Heat conduction problem [15]: This problem was transformed into nonlinear systems. The step size for t was h = 1/N and the step size for x was k = T/N.
The numerical results are given in Table 6. The exact solution z(x, t) = exp(−t) sin(πx) of this problem is shown in Figure 6. The absolute value of error of the solution and approximate solution obtained using the method in (31) are shown in Figures 7 and 8.

Dynamical Analysis
Recently, the dynamical analysis method has been applied to study the stability of the iterative method. Some basic concepts on complex dynamics can be found in references [16][17][18][19][20][21][22][23][24]. For brevity, we omit these concepts in this section. If an iterative method has good stability, it must has good properties for solving simple nonlinear equations. So, we compared our methods with other methods for solving complex equations z n − 1 = 0, (n = 2, 3, 4, 5). The field D = [−5.0, 5.0] × [−5.0, 5.0] ∈ C was divided into a grid of 500 × 500. If the initial point z 0 did not converge to the zero of the function after 25 iterations, it was painted with black. The tolerance |z − z * | < 10 −3 was used in our programs. Tables 7-10 show the computing time (Time) for drawing the dynamical planes and the percentage of points (POPs) which guaranteed the convergence to the roots of the complex equations z n − 1 = 0, (n = 2, 3, 4, 5). Figures 9-12 show that our method, shown in (24), is the most stable method and the stability of the method in (4) without memory is the worst among the tested methods.         Tables 7-10 show that, compared with other methods, the method in (24) had the highest percentage of points which guaranteed the convergence to the roots of the complex equations and the method in (31) costed the least computing time.

Conclusions
In this paper, three Kurchatov-type accelerating iterative schemes with memory were obtained by using two novel Kurchatov-type divided difference operator. The new methods avoided the evaluation of the derivative. The orders of convergence of our methods, shown in (24), (30) and (31), are 3, (5 + √ 17)/2 ≈ 4.56 and 5, respectively. We also corrected the order of convergence of Chicharro's methods, FM3 and FM5. In experimental application, our methods were applied to nonlinear ODEs and PDEs. The numerical results show that, compared with other methods, our methods, shown in (30) and (31), had higher computational accuracy. Dynamical planes were used to analyze the stability of the presented methods. It is worth noting that our method with memory, shown in (24), had better stability than the other methods in this paper. The stability of the method without memory in (4) was the worst. We can conclude that iterative methods with memory can effectively improve the stability of iterative methods without memory.