Thresholds of the Inner Steps in Multi-Step Newton Method

Abstract

We investigate the efficiency of multi-step Newton method (the classical Newton method in which the first derivative is re-evaluated periodically after m steps) for solving nonlinear equations, $F\left(x\right)=0,F:D\subseteq R^n\to R^n$. We highlight the following property of multi-step Newton method with respect to some other Newton-type method: for a given n, there exist thresholds of m, that is an interval $(m_i,m_s)$, such that for m inside of this interval, the efficiency index of multi-step Newton method is better than that of other Newton-type method. We also search for optimal values of m.

1. Introduction

There is currently an increasing interest in developing iterative methods for solving systems of nonlinear equations, $F\left(x\right)=0,F:D\subseteq R^n\to R^n$, with high order of convergence by using only the first derivative of F and its inverse. We will use the term “Newton-type method” for such an iterative procedure. A typical example of Newton-type method is the multi-step Newton method (other terms, “modified Newton method”, “multi-step frozen Jacobian version of the Newton method” [1], etc.) in which the first derivative of F is re-evaluated periodically after m steps. If x denotes the current iteration, the next iteration $x_{next}$ is obtained by the following scheme:

$$\begin{array}{c}x=currentiteration,y_1=x;\hfill \\ Compute{F}^{\prime }\left(x\right);\hfill \\ Fork=1tom\hfill \\ ComputeF\left(y_k\right);\hfill \\ Solvethelinearsystem{F}^{\prime }\left(x\right)s=-F\left(y_k\right);\hfill \\ y_{k+1}=y_k+s;\hfill \\ x_{next}:=y_{m+1}.\hfill \end{array}$$

The method has high convergence order, equal to $m+1$, and the computational cost is due by the inner steps for the computation of $y_k$. This cost can be estimated by the total number d of functional evaluations of F and $F^{\prime }$ (in our case $d=n^2+mn$). The efficiency of the method can be then measured by the Ostrowski index of efficiency, given by $I={\varrho }^{\frac{1}{d}}$, where $\varrho$ is the convergence order. A more accurate measurement of efficiency is the Traub index of efficiency, given by the same formula, but for which d is now the total number of arithmetical operations (usually products/divisions) necessary to obtain the next iteration (in our case $d=(n^3+3m{n}^2-n)/3$, that is, $(n^3-n)/3$ operations for $LU$ factorization and $m{n}^2$ operations for solving the triangular systems [2]).

A plethora of high order Newton-type methods also exist, in which the inner steps do not involve any loop calculation, such as, for example, the four order Jarratt method [3], the fifth order “M5” method [2] and sixth order WKG method [4] (these methods will be used in this note as test examples). It is worth mentioning that M5 method has excellent computational properties (convergence order and complexity) and, as a consequence, a high efficiency index.

The particular case $m=2$ was considered by Potra and Ptak [5]. Using non-discrete induction, they proved the order three of convergence and gave sharp a priori and a posteriori error for this particular case. Often it is called “Potra-Ptak” method [6,7]. In the case of a single equation, Potra-Ptak method was considered by Traub [8] (1982). Ortega and Rheinboldt [9] proved order three of convergence for Potra-Ptak method in n-dimensional spaces (Theorem 10.2.4, [9]). Note that Potra-Ptak method is a particular case of a multipoint iterative process with order three of convergence considered by Ezquerro and Hernandez [10].

Numerical experiments show that the Ostrowski/Traub efficiency index of multi-step Newton method has the following property with respect to the other Newton-type method: Under some conditions, for given n, there exists an interval $(m_i,m_s)$, such that for m inside of this interval, the efficiency index of multi-step Newton method is better than that of considered Newton-type method. We say that $m_i,m_s$ are the thresholds of m, inferior and superior, respectively. This property was tested experimentally by the author for several high order Newton-type methods, like classical Newton, Jarratt, M5, WKG.

In this paper, we give a simple condition ensuring that this property holds for some class of Newton-type methods. In particular, this condition is verified by classical Newton method and M5 method.

2. The Threshold

Lemma 1.

Let $g:[1,\infty )\times [1,\infty )\to \mathcal{R}$ be the real function in two variables defined by

$$g(m,n)=m-ln{(m+1)}^{m+1}+\frac{n^2-1}{3n},$$

where $ln$ is the natural logarithm. Then for any $n\in {\mathcal{N}}_{+},n\ge 2$ the equation $g(m,n)=0$ has a unique solution $m_e\in (1,\infty )$.

Proof. 

The proof involves simple elementary calculus, such that some details are omitted.

We have

$$g(1,n)=1-ln\left(4\right)+\frac{n^2-1}{3n}>0.$$

Suppose that

$$m>e^{\frac{n^2-1}{3n}}-1,$$

then $g(m,n)<0$. Therefore, there exists $m_e$ such that $g(m_e,n)=0$. The solution $m_e$ of the equation $g(m,n)=0$ is unique in $(1,\infty )$. Indeed, the derivative of g with respect to m is $g^{\prime }(m,n)=-ln(m+1)$ and the function g is strictly decreasing on $[1,\infty )$. ☐

Remark 1.

Suppose that, for some n and some $p,0<p<1$, the following is true

$$ln{(n^p+1)}^{n^p+1}>n^p+\frac{n^2-1}{3n}.$$

(1)

Then $n^p>m_e$ (the proof is elementary). Note that, for given p, (1) is not satisfied for any n, but it is satisfied on some interval of n, usually very large when $p\approx 1$. For example, if $p=2/3=0.666\cdots$ then this interval is $[1,1.756\times {10}^3]$; if $p=0.8$ then the interval is $[1,1.257\times {10}^8]$.

We consider next an iterative method of Newton-type which has $\varrho$ as convergence order and which uses only the first derivative and its inverse in inner steps. Jarratt, M5 and WKG methods are examples of such Newton-type method. The main processing consists of the solution of one or several linear systems having the first derivative as matrix. The total number of products/divisions of such a method is given by $d\left(n\right)=a{n}^3+P\left(n\right)$, where $a>0$ and P is a polynomial of degree 2.

The theorem below gives a comparative analysis of Traub efficiency indexes of multi-step Newton method and this Newton-type method. Recall that the Traub efficiency index of multi-step Newton method, is:

$$I_{m_s}(m,n)={(m+1)}^{\frac{3}{n^3+3m{n}^2-n}}.$$

Theorem 1.

Consider a Newton-type method with the convergence order $\varrho \ge 2$ and the number of products/divisions $d\left(n\right)$. Suppose that, for a given n, the following condition is satisfied

$$d\left(n\right)>n^2(m_e+1)ln\varrho ,$$

(2)

where $m_e$ is defined in Lemma 1. Then there exist the thresholds $m_i,m_s\in [1,\infty )$ such that

$$\left\{\begin{array}{c}{I}_{m_s}(m,n)>{\varrho }^{1/d\left(n\right)}ifm\in (m_i,m_s),\hfill \\ I_{m_s}(m,n)<{\varrho }^{1/d\left(n\right)}ifm\notin (m_i,m_s).\hfill \end{array}\right.$$

Proof. 

The derivative of $I_{m_s}$ with respect to m is

$$I_{m_s}^{\prime }(m,n)=9(n^2+3mn-1){(m+1)}^{\frac{3}{n^3+3m{n}^2-n}-1}g(m,n).$$

Thus, for any n, $sgn{I}_{m_s}^{\prime }(m,n)=sgng(m,n)$ and from Lemma 1 it results that $I_{m_s}^{\prime }(m,n)>0$ on $[1,m_e)$ and $I_{m_s}^{\prime }(m,n)<0$ on $(m_e,\infty )$. Therefore $I_{m_s}$ is strictly increasing on $(1,m_e)$ and strictly decreasing on $(m_e,\infty )$. For $m=m_e$ we have

$$I_{m_s}(m_e,n)=e^{\frac{1}{n^2(m_e+1)}}$$

and from (2) we obtain

$$I_{m_s}(m_e,n)>{\varrho }^{1/d\left(n\right)}.$$

(3)

As

$$\underset{m\to \infty }{lim}{I}_{m_s}(m,n)={\left[\underset{m\to \infty }{lim}{(m+1)}^{m+1}\right]}^{\frac{1}{n^2}}=1,$$

and ${\varrho }^{1/d\left(n\right)}>1$, for m sufficiently large, we have

$$I_{m_s}(m,n)<{\varrho }^{1/d\left(n\right)}.$$

(4)

From (3) and (4) it results that there exists $m_s\in (m_e,\infty )$ as superior threshold.

Next we have to consider two cases:

(1)

$I_{m_s}(1,n)\ge {\varrho }^{1/d\left(n\right)}$; in this case $I_{m_s}(m.n)>{\varrho }^{1/d\left(n\right)}$ for $m\in (1,m_e)$ and $m_i=1$ is the inferior threshold of m.

(2)

$I_{m_s}(1,n)<{\varrho }^{1/d\left(n\right)}$; then this inequality and (3) imply that there exists $m_i\in (1,m_e)$ such that $I_{m_s}(m,n)<d^{1/d\left(n\right)}$ for $m\in (m_i,m_e)$, that is $m_i$ is inferior threshold of m.

 ☐

The graphs of indexes for the three cases are given in Figure 1.

The case $I_{m_s}(1,n)={\varrho }^{1/d\left(n\right)}$ is illustrated by classical Newton method (Figure 1a), the case $I_{m_s}(1,n)>{\varrho }^{1/d\left(n\right)}$, by Jarratt method (Figure 1b) and the case $I_{m_s}(1,n)<{\varrho }^{1/d\left(n\right)}$, by M5 method (Figure 1c).

Remark 2.

It is desirable that condition (2) to be satisfied for a large interval of n, or even for all n. If (1) is satisfied for some n and p, then the condition (2) can be replaced by the following stronger condition

$$d\left(n\right)>n^2(n^p+1)ln\varrho ,$$

(5)

The condition (5) is easier to check, but it gives a more restrictive interval for n.

Particular Cases

We check that the condition (2) is satisfied for classical Newton method and for M5 method and therefore Theorem 1 can be applied.

It is easy to see that in the case of Newton method, the condition (2) is satisfied for all n. The stronger condition (5) gives for n the interval $[7,1.756\times {10}^3]$. For example, if $n=15$ then the threshold for m is $m_s=22.101\cdots$

In the case of M5 method, the condition (2) is satisfied for all $n\in [32,\infty )$. The stronger condition (5) gives for n the interval $[100,1.756\times {10}^3]$. For example, if $n=40$ then the thresholds for m are $m_i=5.229\cdots$, $m_s=14.431\cdots$

3. On the Optimal Number of Inner Steps in Multi-Step Newton Method

In [2] M5 method is compared with three high order method, Newton, Jarratt and WKG, and with respect to the both Ostrowski and Traub indexes. It is shown that the M5 method has better efficiency than all these methods. The previous analysis shows that the multi-step Newton method has still better Traub efficiency index, provided that $n\ge 32$ and m belong to the interval $(m_i,m_s)$. The following two problems arise: 1. Is there an m, independent of n, such that the multi-step Newton index is larger that M5 index? 2. Is there an optimal number $m_{opt}$ of multi-step Newton method for which the difference between the index of the multi-step Newton and the index of M5 method is maxim?

Both problems have affirmative answers.

• Several numerical tests show that the sequence $\left\{m_i\right\}$ is a decreasing sequence as n increases (the proof is elementary). Therefore we can take for m the larger term of the sequence $\left\{m_i\right\}$, which is $m=7$. Therefore $I_{m_s}(7,n)>I_{M5}\left(n\right)$ for $n\ge 32$ (Figure 2).
• Let I be the efficiency index of some iterative methods of Newton-type. The optimal value of the number of inner steps in multi-step Newton method with respect to some I, is the value of $m\in (m_i,m_s)$ for which $I_{m_s}(m,n)-I\left(n\right)$ is maximum. The properties of functions $I_{m_s}$ and I (for all considered cases) show that such optimal value exists and is unique.

For example, if $I=I_n$ (the Newton method) the value of m for which $I_{m_s}(m,n)-I_n\left(n\right)$ is maximum, is $m_{opt}=1.901\cdots$ (Ostrowski index) and $m_{opt}=2.311\cdots$ (Traub index). Thus $m_{opt}=2$. We can conclude that the Potra-Ptak method is the most efficient method with respect to the classical Newton method.

In the case of M5 method the optimal value is $m_{opt}=10$ (Traub index).

Remark 3.

The efficiency indexes of Newton-type methods tend to value 1 when $n\to \infty$. Thus, these indexes differ slightly from each other, even for n not very large. For example, if $n=40$, the thresholds of multi-step Newton method with respect to M5 method are $m_i=6,m_s=14$ and the maximum of $I_{m_s}$ for $m\in (m_i,m_s)$ is obtained by $m=6$ and has the value $1.000385$. The maximum efficiency index of Potra-Ptak method, for the same value of n, is $1.000312$. Therefore, the efficiency index increases very little from Potra-Ptak method to the most efficient Newton-type method, approx $0.006705$ in percent.