A Family of Iterative Methods for Solving Systems of Nonlinear Equations Having Unknown Multiplicity

Abstract

The singularity of Jacobian happens when we are looking for a root, with multiplicity greater than one, of a system of nonlinear equations. The purpose of this article is two-fold. Firstly, we will present a modification of an existing method that computes roots with known multiplicities. Secondly, will propose the generalization of a family of methods for solving nonlinear equations with unknown multiplicities, to the system of nonlinear equations. The inclusion of a nonzero multi-variable auxiliary function is the key idea. Different choices of the auxiliary function give different families of the iterative method to find roots with unknown multiplicities. Few illustrative numerical experiments and a critical discussion end the paper.

1. Introduction

We are interested in computing a real root $\mathsf{\alpha }$ of a function $\mathbf{F}:\mathrm{D}\subseteq {\mathrm{ℝ}}^{\mathrm{n}}⟶{\mathbb{R}}^{\mathrm{n}}$, that is a vector $\mathsf{\alpha }$ belonging to D such that $\mathbf{F}\left(\mathsf{\alpha }\right)=\mathbf{0}$. The most classical iterative method for solving a system of nonlinear equations, especially in the case of simple zeros, is the Newton method that offers quadratic convergence [1,2], under certain local regularity conditions. Many researchers have proposed further iterative methods for solving a system of nonlinear equations, that are efficient and with a high order of convergence [3,4,5,6,7,8,9]. However, as far as we are concerned with roots having multiplicity $\mathrm{m}(\ge 2)$, the classical Newton method deteriorates and its convergence rate is linear with convergence factor deteriorating when the multiplicity is higher: modified variants of the Newton method can offer a good alternative, by recovering quadratic convergence, under the hypothesis that the multiplicity is known in advance.

The most classical modified Newton, for scalar nonlinear equations, can be written as

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\mathrm{x}}_0=\text{initial}\text{guess}\hfill \\ {\mathrm{x}}_{\mathrm{k}+1}={\mathrm{x}}_{\mathrm{k}}-\mathrm{m}\frac{\mathrm{f}\left({\mathrm{x}}_{\mathrm{k}}\right)}{{\mathrm{f}}^{\prime }\left({\mathrm{x}}_{\mathrm{k}}\right)},\mathrm{k}=0,1,\cdots \hfill \end{array}\right.\end{array}$$

(1)

The straightforward generalization of Equation (1) is presented in [10]

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\mathbf{x}}_0=\text{initial}\text{guess},\hfill \\ {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^{-1}\text{diag}\left(\mathbf{m}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right),\mathrm{k}=0,1,\cdots \hfill \end{array}\right.\end{array}$$

(2)

where $\mathbf{m}={[{\mathrm{m}}_1,{\mathrm{m}}_2,\cdots ,{\mathrm{m}}_{\mathrm{n}}]}^{\mathrm{T}}$ is a vector of multiplicities for system of nonlinear equations $\mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}$ and diag$(·)$ represents a diagonal matrix that keeps the input vector at its main diagonal. The proof of quadratic convergence of Equation (2) is established in [10]. Wu [11] proposed a variant of Newton method with the help of an auxiliary function. To complete the reference, we give new details of developments proposed by Wu. Suppose we have a system of nonlinear equations $\mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}$ and we define a new system of nonlinear equation that have the same root

$$\begin{array}{c}\hfill \mathbf{U}\left(\mathbf{x}\right)={\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\odot \mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}\end{array}$$

(3)

where ⊙ is the component-wise multiplication of two vectors and $\mathbf{v}={[{\mathrm{v}}_1,{\mathrm{v}}_2,\cdots ,{\mathrm{v}}_{\mathrm{n}}]}^{\mathrm{T}}$. The Fréchet derivative of $\mathbf{U}\left(\mathbf{x}\right)$ is

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{U}}^{\prime }\left(\mathbf{x}\right)=\text{diag}\left({\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\right){\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right)\mathbf{v}\odot {\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\hfill \\ & {\mathbf{U}}^{\prime }\left(\mathbf{x}\right)=\text{diag}\left({\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\right)\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{v}\odot \mathbf{F}\left(\mathbf{x}\right)\right)\right)\hfill \end{array}\end{array}$$

(4)

The application of Newton method for Equation (3) is

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\mathbf{U}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^{-1}\mathbf{U}\left({\mathbf{x}}_{\mathrm{k}}\right)\hfill \\ & {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left(\text{diag}\left({\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\right)\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{v}\odot \mathbf{F}\left(\mathbf{x}\right)\right)\right)\right)}^{-1}{\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\odot \mathbf{F}\left(\mathbf{x}\right)\hfill \\ & {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{v}\odot \mathbf{F}\left(\mathbf{x}\right)\right)\right)}^{-1}\text{diag}{\left({\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\right)}^{-1}{\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\odot \mathbf{F}\left(\mathbf{x}\right)\hfill \\ & {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{v}\odot \mathbf{F}\left(\mathbf{x}\right)\right)\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}\end{array}$$

(5)

The rate of convergence of Equation (5) is quadratic, just because the proposed iteration coincides with the Newton method. Notice that vector $\mathbf{v}$ is a parameter that provides a degree of freedom in Equation (5). Jose et al. [10] proposed a modification in Equation (1) by defining a modified function

$$\begin{array}{c}\hfill \mathbf{U}\left(\mathbf{x}\right)={\mathrm{e}}^{\mathbf{v}\odot \mathbf{x}}\odot \mathbf{F}{\left(\mathbf{x}\right)}^{1/\mathbf{m}}=\mathbf{0}\end{array}$$

(6)

where $1/\mathbf{m}={[1/{\mathrm{m}}_1,1/{\mathrm{m}}_2,\cdots ,1/{\mathrm{m}}_{\mathrm{n}}]}^{\mathrm{T}}$ and power of $\mathbf{F}\left(\mathbf{x}\right)$ is component-wise. The application of Newton method to Equation (6) leads to the scheme

$$\begin{array}{cc}\hfill & {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{v}\odot \mathbf{F}\left(\mathbf{x}\right)\right)\right)}^{-1}\text{diag}\left(\mathbf{m}\right)\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}$$

(7)

2. Some Generalizations

The original idea of using a auxiliary function was proposed in [11]. The auxiliary function employed in Equation (3) is the exponential function. The question is, why we choose the exponential function? The answer is: it is a non-zeros function with non-zero derivative. So generalization is straightforward, we can choose any function that is non-zero everywhere in the vicinity of the root and in this way we will ensure the roots of $\mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}$ are not affected by the auxiliary function multiplicity. Let $\mathbf{G}\left(\mathbf{x}\right)$ be a non-zero auxiliary function in the neighborhood of the root of the system of nonlinear equations with unknown multiplicity and define a new system of nonlinear equations associated with $\mathbf{F}\left(\mathbf{x}\right)$ as below

$$\begin{array}{c}\hfill \mathbf{U}\left(\mathbf{x}\right)=\mathbf{G}\left(\mathbf{x}\right)\odot \mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}\end{array}$$

(8)

Notice that the roots of $\mathbf{U}\left(\mathbf{x}\right)=\mathbf{0}$ and $\mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}$ are the same because $\mathbf{G}\left(\mathbf{x}\right)\ne \mathbf{0}$ for all $\mathbf{x}$ in the neighborhood of root. The first order Fréchet derivative of Equation (8) can be computed as

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathrm{U}}_{\mathrm{i}}\left(\mathbf{x}\right)={\mathrm{F}}_{\mathrm{i}}\left(\mathbf{x}\right){\mathrm{G}}_{\mathrm{i}}\left(\mathbf{x}\right)\hfill \\ & \nabla {\mathrm{U}}_{\mathrm{i}}{\left(\mathbf{x}\right)}^{\mathrm{T}}={\mathrm{F}}_{\mathrm{i}}\left(\mathbf{x}\right)\nabla {\mathrm{G}}_{\mathrm{i}}{\left(\mathbf{x}\right)}^{\mathrm{T}}+{\mathrm{G}}_{\mathrm{i}}\left(\mathbf{x}\right)\nabla {\mathrm{F}}_{\mathrm{i}}{\left(\mathbf{x}\right)}^{\mathrm{T}},\mathrm{i}=1,2,\cdots ,\mathrm{n}\hfill \\ & \left[\begin{array}{c}\nabla {\mathrm{U}}_1{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ \nabla {\mathrm{U}}_2{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ \nabla {\mathrm{U}}_3{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ ⋮\\ \nabla {\mathrm{U}}_{\mathrm{n}}{\left(\mathbf{x}\right)}^{\mathrm{T}}\end{array}\right]=\left[\begin{array}{cccc}{\mathrm{F}}_1\left(\mathbf{x}\right)& 0& \cdots & 0\\ 0& {\mathrm{F}}_2\left(\mathbf{x}\right)& \cdots & 0\\ 0& 0& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathrm{F}}_{\mathrm{n}}\left(\mathbf{x}\right)\end{array}\right]\left[\begin{array}{c}\nabla {\mathrm{G}}_1{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ \nabla {\mathrm{G}}_2{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ \nabla {\mathrm{G}}_3{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ ⋮\\ \nabla {\mathrm{G}}_{\mathrm{n}}{\left(\mathbf{x}\right)}^{\mathrm{T}}\end{array}\right]+\left[\begin{array}{cccc}{\mathrm{G}}_1\left(\mathbf{x}\right)& 0& \cdots & 0\\ 0& {\mathrm{G}}_2\left(\mathbf{x}\right)& \cdots & 0\\ 0& 0& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& 0& \cdots & {\mathrm{G}}_{\mathrm{n}}\left(\mathbf{x}\right)\end{array}\right]\left[\begin{array}{c}\nabla {\mathrm{F}}_1{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ \nabla {\mathrm{F}}_2{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ \nabla {\mathrm{F}}_3{\left(\mathbf{x}\right)}^{\mathrm{T}}\\ ⋮\\ \nabla {\mathrm{F}}_{\mathrm{n}}{\left(\mathbf{x}\right)}^{\mathrm{T}}\end{array}\right]\hfill \end{array}\end{array}$$

(9)

From Equation (9), the Fréchet derivative of $\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)$ is

$$\begin{array}{c}\hfill {\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)}^{\prime }=\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right){\mathbf{G}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{G}\left(\mathbf{x}\right)\right){\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\end{array}$$

(10)

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{U}}^{\prime }\left(\mathbf{x}\right)=\text{diag}\left(\mathbf{G}\left(\mathbf{x}\right)\right){\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right){\mathbf{G}}^{\prime }\left(\mathbf{x}\right)\hfill \\ & {\mathbf{U}}^{\prime }\left(\mathbf{x}\right)=\text{diag}\left(\mathbf{G}\left(\mathbf{x}\right)\right)\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right)\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{G}}^{\prime }\left(\mathbf{x}\right)\right)\hfill \end{array}\end{array}$$

(11)

If we apply the Newton method to the system in Equation (8), then we obtain

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right)\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{G}}^{\prime }\left(\mathbf{x}\right)\right)}^{-1}\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}\mathbf{G}\left(\mathbf{x}\right)\odot \mathbf{F}\left(\mathbf{x}\right)\hfill \\ & {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right)\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{G}}^{\prime }\left(\mathbf{x}\right)\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}\end{array}$$

(12)

The convergence order of Equation (12) is two, under the usual regularity assumptions. The iterative method Equation (7) can be written as

$$\begin{array}{cc}\hfill & {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{F}\left(\mathbf{x}\right)\right)\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{G}}^{\prime }\left(\mathbf{x}\right)\right)}^{-1}\text{diag}\left(\mathbf{m}\right)\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}$$

(13)

Again the convergence order of Equation (13) is two. In numerical simulations, we show that numerical results can improve by choosing appropriate auxiliary functions: in other words, the use of an auxiliary function can improve the constant hidden in the quadratic convergence.

3. Proposed Method

For the purpose of motivation, we present some developments for single nonlinear equations and subsequently we establish results for the multi-dimensional case. Recently Noor et al. [12] constructed a family of methods for solving nonlinear equations with unknown multiplicities and this represents a crucial improvement wit respect to procedures requiring this information. What they have established is the following. Let $\mathrm{g}\left(\mathrm{x}\right)$ be a non-zero function and let us define a new function

$$\begin{array}{c}\hfill \mathrm{q}\left(\mathrm{x}\right)=\frac{\mathrm{f}\left(\mathrm{x}\right)\mathrm{g}\left(\mathrm{x}\right)}{{\mathrm{f}}^{\prime }\left(\mathrm{x}\right)}\end{array}$$

(14)

The application of a classical Newton method to the equation $\mathrm{q}\left(\mathrm{x}\right)=0$ leads to the iteration

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathrm{x}}_{\mathrm{k}+1}={\mathrm{x}}_{\mathrm{k}}-\frac{\mathrm{q}\left({\mathrm{x}}_{\mathrm{k}}\right)}{{\mathrm{q}}^{\prime }\left({\mathrm{x}}_{\mathrm{k}}\right)}\hfill \\ & {\mathrm{x}}_{\mathrm{k}+1}={\mathrm{x}}_{\mathrm{k}}-\frac{{\mathrm{f}}^{\prime }\left({\mathrm{x}}_{\mathrm{k}}\right)\mathrm{f}\left({\mathrm{x}}_{\mathrm{k}}\right)\mathrm{g}\left({\mathrm{x}}_{\mathrm{k}}\right)}{{\mathrm{f}}^{\prime }\left({\mathrm{x}}_{\mathrm{k}}\right){\left(\mathrm{f}\left({\mathrm{x}}_{\mathrm{k}}\right)\mathrm{g}\left({\mathrm{x}}_{\mathrm{k}}\right)\right)}^{\prime }-\underset{¯}{{\mathrm{f}}^{″}\left({\mathrm{x}}_{\mathrm{k}}\right)\mathrm{f}\left({\mathrm{x}}_{\mathrm{k}}\right)\mathrm{g}\left({\mathrm{x}}_{\mathrm{k}}\right)}}\hfill \end{array}\end{array}$$

(15)

The order of convergence of Equation (15) is two, under suitable regularity assumptions both on $\mathrm{f}(·)$ and $\mathrm{g}(·)$. We are interested in developing a possible multidimensional version of Equation (15). Let $\mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}$ be a system of nonlinear equations and having a root of unknown multiplicities. With the help of a non-zero auxiliary function $\mathbf{G}\left(\mathbf{x}\right)$, we define a new function $\mathbf{Q}\left(\mathbf{x}\right)$.

$$\begin{array}{c}\hfill \mathbf{Q}\left(\mathbf{x}\right)={\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\left(\mathbf{G}\left(\mathbf{x}\right)\odot \mathbf{F}\left(\mathbf{x}\right)\right)=\mathbf{0}\end{array}$$

(16)

The first order Fréchet derivative of Equation (16) can be written as

$$\begin{array}{c}\hfill {\mathbf{Q}}^{\prime }\left(\mathbf{x}\right)={\left({\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\right)}^2\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right){\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)}^{\prime }-{\mathbf{F}}^{\prime }\left(\mathbf{x}\right){\mathbf{F}}^{″}\left(\mathbf{x}\right){\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)\right)\end{array}$$

(17)

Further simplification of ${\mathbf{Q}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\mathbf{Q}\left(\mathbf{x}\right)$ gives

$$\begin{array}{c}\hfill {\mathbf{Q}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\mathbf{Q}\left(\mathbf{x}\right)={\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right){\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)}^{\prime }-\underset{¯}{{\mathbf{F}}^{\prime }(\mathbf{x}){\mathbf{F}}^{″}(\mathbf{x}){\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)}\right)}^{-1}{\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)\end{array}$$

(18)

When comparing the underlined expressions in Equations (15) and (18), we clearly see that it is not possible to simplify the expression ${\mathbf{F}}^{\prime }\left(\mathbf{x}\right){\mathbf{F}}^{″}\left(\mathbf{x}\right){\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)$, simply because ${\mathbf{F}}^{\prime }\left(\mathbf{x}\right)$ and ${\mathbf{F}}^{″}\left(\mathbf{x}\right)$ do not commute in general. Clearly in the scalar case the elimination of ${\mathrm{f}}^{\prime }\left(\mathrm{x}\right)$ is possible due to commutativity.

Our idea amounts in artificially eliminating ${\mathbf{F}}^{\prime }\left(\mathbf{x}\right)$ and ${\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}$ from the expression $\mathbf{F}{\left(\mathbf{x}\right)}^{\prime }{\mathbf{F}}^{″}\left(\mathbf{x}\right){\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)$: in this way we obtain a new iterative method for solving system of nonlinear equations with unknown multiplicities

$$\begin{array}{c}\hfill {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right){\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\odot \mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{\prime }-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\odot \mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\odot \mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)\end{array}$$

(19)

We clearly state that our proposed scheme i.e., the iterative method in Equation (19) is not the application of Newton method to Equation (16). However, our procedure is simpler and preserves the same quadratic convergence as the Newton method, under the very same regularity assumptions. In the next section, we will establish the proof of quadratic convergence for Equation (19).

4. Convergence

If we substitute $\mathbf{G}\left(\mathbf{x}\right)=1$ in Equation (19), then we obtain

$$\begin{array}{c}\hfill {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^2-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\end{array}$$

(20)

First we will establish the proof of quadratic convergence of the iterative procedure in Equation (20) and then for the main iterative method reported in Equation (19). Let ${\alpha }={[{\mathsf{\alpha }}_1,{\mathsf{\alpha }}_2,\cdots ,{\mathsf{\alpha }}_{\mathrm{n}}]}^{\mathrm{T}}$ be the root of $\mathbf{F}\left(\mathbf{x}\right)=\mathbf{0}$ with corresponding vector of multiplicities $\mathbf{m}={[{\mathrm{m}}_1,{\mathrm{m}}_2,\cdots ,{\mathrm{m}}_{\mathrm{n}}]}^{\mathrm{T}}$. As a consequence, there exists $\mathbf{H}\left(\mathbf{x}\right)$ such that $\mathbf{H}\left(\mathsf{\alpha }\right)={[{\mathrm{h}}_1\left(\mathsf{\alpha }\right),{\mathrm{h}}_2\left(\mathsf{\alpha }\right),\cdots ,{\mathrm{h}}_{\mathrm{n}}\left(\mathsf{\alpha }\right)]}^{\mathrm{T}}\ne \mathbf{0}$ for which the system of nonlinear equations can be written as

$$\begin{array}{c}\hfill \mathbf{F}\left(\mathbf{x}\right)={\left(\mathbf{x}-\mathsf{\alpha }\right)}^{\mathrm{m}}\odot \mathbf{H}\left(\mathbf{x}\right)\end{array}$$

(21)

where ${\left(\mathbf{x}-\mathsf{\alpha }\right)}^{\mathrm{m}}={[{({\mathrm{x}}_1-{\mathsf{\alpha }}_1)}^{{\mathrm{m}}_1},{({\mathrm{x}}_2-{\mathsf{\alpha }}_2)}^{{\mathrm{m}}_2},\cdots ,{({\mathrm{x}}_{\mathrm{n}}-{\mathsf{\alpha }}_{\mathrm{n}})}^{{\mathrm{m}}_{\mathrm{n}}}]}^{\mathrm{T}}$. The first order and second order Fréchet derivatives of Equation (21) can be computed as

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{F}}^{\prime }\left(\mathbf{x}\right)=\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathrm{m}}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\hfill \\ & {\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}=\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathrm{m}}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}+\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\mathbf{w}\hfill \\ & {\mathbf{F}}^{″}\left(\mathbf{x}\right)\mathbf{w}={\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}\right)}^{\prime }=\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\odot {\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}\right)+\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}}\right){\mathbf{H}}^{″}\left(\mathbf{x}\right)\mathbf{w}\hfill \\ & +\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\odot \mathbf{w}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left(\mathbf{m}\odot (\mathbf{m}-\mathbf{1})\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{2}}\odot \mathbf{w}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\hfill \end{array}\end{array}$$

(22)

where $\mathbf{w}$ is a vector that we use to compute second order Fréchet derivative. By replacing $\mathbf{w}$ in Equation (22) by $\mathbf{F}\left(\mathbf{x}\right)$, we get

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{F}}^{″}\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)=\text{diag}\left({\mathbf{m}}^2\odot {(\mathbf{x}-\mathsf{\alpha })}^{2\mathbf{m}-\mathbf{2}}\odot \mathbf{H}{\left(\mathbf{x}\right)}^2\right)-\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{2\mathbf{m}-\mathbf{2}}\odot \mathbf{H}{\left(\mathbf{x}\right)}^2\right)\hfill \\ & +\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\right)\text{diag}\left({\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathrm{m}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\right)\hfill \\ & +\text{diag}\left(\mathbf{m}{(\mathbf{x}-\mathsf{\alpha })}^{2\mathbf{m}-\mathbf{1}}\odot \mathbf{H}\left(\mathbf{x}\right)\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\hfill \\ & {\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^2=\text{diag}\left({\mathbf{m}}^2\odot {(\mathbf{x}-\mathsf{\alpha })}^{2\mathbf{m}-\mathbf{2}}\odot \mathbf{H}{\left(\mathbf{x}\right)}^2\right)+\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\hfill \\ & +\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathrm{m}}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\hfill \\ & +\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}-\mathbf{1}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\text{diag}\left({(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}}\right){\mathbf{H}}^{\prime }\left(\mathbf{x}\right)\hfill \end{array}\end{array}$$

(23)

By using Equations (22) and (23), and after proper simplifications, we have

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)=\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{2\mathbf{m}-\mathbf{1}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\left(\mathbf{I}+\mathrm{O}(\mathbf{x}-\mathsf{\alpha })\right)\hfill \\ & {\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^2-{\mathbf{F}}^{″}\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)=\text{diag}\left(\mathbf{m}\odot {(\mathbf{x}-\mathsf{\alpha })}^{2\mathbf{m}-\mathbf{2}}\odot \mathbf{H}\left(\mathbf{x}\right)\right)\left(\mathbf{I}+\mathrm{O}(\mathbf{x}-\mathsf{\alpha })\right)\hfill \\ & {\left({\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^2-{\mathbf{F}}^{″}\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)=(\mathbf{x}-\mathsf{\alpha })+\mathrm{O}\left({(\mathbf{x}-\mathsf{\alpha })}^2\right)\hfill \end{array}\end{array}$$

(24)

Theorem 1. 

Let $\mathbf{F}:\mathrm{D}\subseteq {\mathbb{R}}^{\mathrm{n}}⟶{\mathbb{R}}^{\mathrm{n}}$ and $\mathsf{\alpha }={[{\mathsf{\alpha }}_1,{\mathsf{\alpha }}_2,{\mathsf{\alpha }}_3,\cdots ,{\mathsf{\alpha }}_{\mathrm{n}}]}^{\mathrm{T}}\in \mathrm{D}$ is a root of $\mathbf{F}\left(\mathbf{x}\right)={(\mathbf{x}-\mathsf{\alpha })}^{\mathbf{m}}\odot \mathbf{H}\left(\mathbf{x}\right)=\mathbf{0}$ with corresponding multiplicities vector $\mathbf{m}={[{\mathrm{m}}_1,{\mathrm{m}}_2,\cdots ,{\mathrm{m}}_{\mathrm{n}}]}^{\mathrm{T}}$ and $\mathbf{H}\left(\mathsf{\alpha }\right)={[{\mathrm{h}}_1\left(\mathsf{\alpha }\right),{\mathrm{h}}_2\left(\mathsf{\alpha }\right),\cdots ,{\mathrm{h}}_{\mathrm{n}}\left(\mathsf{\alpha }\right)]}^{\mathrm{T}}\ne \mathbf{0}$ with ${\mathrm{h}}_{\mathrm{i}}\left(\mathbf{x}\right)\in {\mathit{C}}^2\left(\mathrm{D}\right)$. Then there exists a subset $\mathrm{S}\subseteq \mathrm{D}$ such that, if we choose ${\mathbf{x}}_0\in \mathrm{S}$ , the iterative method in Equation (20) has quadratic convergence in S.

Proof. 

We can write Equation (20) as

$$\begin{array}{c}\hfill {\mathbf{x}}_{\mathrm{k}+1}=R\left({\mathbf{x}}_{\mathrm{k}}\right)={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^2-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\end{array}$$

(25)

By dropping the index k, we obtain

$$\begin{array}{cc}\hfill & \mathbf{R}\left(\mathbf{x}\right)=\mathbf{x}-{\left({\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^2-{\mathbf{F}}^{″}\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}$$

(26)

By using Equations (24) and (26), the simplified expression for $\mathbf{R}\left(\mathbf{x}\right)$ is

$$\begin{array}{c}\hfill \begin{array}{cc}& \mathbf{R}\left(\mathbf{x}\right)=\mathbf{x}-(\mathbf{x}-\mathsf{\alpha })+\mathrm{O}\left({(\mathbf{x}-\mathsf{\alpha })}^2\right)\hfill \\ & {\mathbf{R}}^{\prime }\left(\mathbf{x}\right)=\mathbf{O}+\mathrm{O}\left(\text{diag}\left(\mathbf{x}-\mathsf{\alpha }\right)\right)\hfill \end{array}\end{array}$$

(27)

By substituting $\mathbf{x}=\mathsf{\alpha }$ in Equation (27), we deduce the crucial relationships

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill & \mathbf{R}\left(\mathsf{\alpha }\right)=\mathsf{\alpha }\hfill \\ & {\mathbf{R}}^{\prime }\left(\mathsf{\alpha }\right)=\mathbf{O}\hfill \end{array}\end{array}$$

(28)

and from Equation (28) we conclude that the iterative method in Equation (20) has at least quadratic convergence.  ☐

Finally, the quadratic convergence of the method reported in Equation (19) can be proven as follows. Let $\mathbf{e}=\mathbf{x}-\mathsf{\alpha }$ and after applying few simplifications, we can write Equation (19) in the form

$$\begin{array}{c}\hfill \begin{array}{cc}& \mathbf{M}\left(\mathbf{x}\right)=\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{F}}^{\prime }{\left(\mathbf{x}\right)}^{-1}{\mathbf{F}}^{″}\left(\mathbf{x}\right)\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)\hfill \\ & \mathbf{L}\left(\mathbf{x}\right)={\mathbf{F}}^{\prime }\left(\mathbf{x}\right)+\text{diag}\left({\mathrm{f}}^{\prime }\left(\mathbf{x}\right)\right)\text{diag}{\left(\mathbf{G}\left(\mathbf{x}\right)\right)}^{-1}{\mathbf{G}}^{\prime }\left(\mathbf{x}\right)\hfill \\ & {\mathbf{e}}_{\mathrm{k}+1}={\mathbf{e}}_{\mathrm{k}}-{\left(\mathbf{L}\left(\mathbf{x}\right)-\mathbf{M}\left(\mathbf{x}\right)\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\hfill \\ & {\mathbf{e}}_{\mathrm{k}+1}={\mathbf{e}}_{\mathrm{k}}-{\left(\mathbf{I}-\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{M}\left(\mathbf{x}\right)\right)}^{-1}\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\hfill \\ & {\mathbf{e}}_{\mathrm{k}+1}\approx {\mathbf{e}}_{\mathrm{k}}-\left(\mathbf{I}+\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{M}\left(\mathbf{x}\right)\right)\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\hfill \\ & {\mathbf{e}}_{\mathrm{k}+1}\approx \left({\mathbf{e}}_{\mathrm{k}}-\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\right)-\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{M}\left(\mathbf{x}\right)\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}\end{array}$$

(29)

From Equation (12), we can see the term ${\mathbf{e}}_{\mathrm{k}}-\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)=\mathrm{O}\left({\mathbf{e}}^2\right)$ and hence $\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)=\mathrm{O}\left(\mathbf{e}\right)$. Moreover, It can be seen easily that $\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{M}\left(\mathbf{x}\right)=\mathrm{O}\left(\mathbf{e}\right)$. It means $\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{M}\left(\mathbf{x}\right)\mathbf{L}{\left(\mathbf{x}\right)}^{-1}\mathbf{F}\left(\mathbf{x}\right)=\mathrm{O}\left({\mathbf{e}}^2\right)$. Hence we conclude that

$$\begin{array}{c}\hfill {\mathbf{e}}_{\mathrm{k}+1}=\mathrm{O}\left({\mathbf{e}}_{\mathrm{k}}^2\right)\end{array}$$

(30)

Notice that in Equation (24), we shows that the expression ${\left({\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^2-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)$ is independent from factor $\mathbf{m}$ and in Equation (29) the inclusion of auxiliary function does not disturb the quadratic convergence. The auxiliary function $\mathbf{G}\left(\mathbf{x}\right)$ works as a parameter that helps in rapid convergence by changing the path of convergence. And it is the vector quotient ${\left({\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^2-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)$ that actually makes the method independent from the information contained in the multiplicity.

5. Numerical Testing

It is important to test the computational convergence order (COC) of the iterative methods discussed so far. In all our simulations, we adopt the following definition of COC

$$\begin{array}{c}\hfill \text{COC}=\frac{\log \left(\left|\right|\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}+1}\right){\left|\right|}_{\infty }/\left|\right|\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right){\left|\right|}_{\infty }\right)}{\log \left(\left|\right|\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right){\left|\right|}_{\infty }/\left|\right|\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}-1}\right){\left|\right|}_{\infty }\right)}\text{or}\frac{\log \left(\left|\right|{\mathbf{x}}_{\mathrm{k}+1}-{\mathsf{\alpha }\left|\right|}_{\infty }/\left|\right|{\mathbf{x}}_{\mathrm{k}}-\mathsf{\alpha }{\left|\right|}_{\infty }\right)}{\log \left(\left|\right|{\mathbf{x}}_{\mathrm{k}}-{\mathsf{\alpha }\left|\right|}_{\infty }/\left|\right|{\mathbf{x}}_{\mathrm{k}-1}-\mathsf{\alpha }{\left|\right|}_{\infty }\right)}\end{array}$$

(31)

Next, we explain how to compute the term ${\mathbf{F}}^{″}\left(\mathbf{x}\right)(\mathbf{G}\left(\mathbf{x}\right)\odot \mathbf{F}\left(\mathbf{x}\right))$. Suppose, we have a system of three nonlinear equations

$$\begin{array}{c}\hfill \text{Problem}1=\left\{\begin{array}{cc}& {\mathrm{F}}_1\left(\mathbf{x}\right)={({\mathrm{x}}_1-1)}^4\exp \left({\mathrm{x}}_2\right)=0\hfill \\ & {\mathrm{F}}_2\left(\mathbf{x}\right)={({\mathrm{x}}_2-2)}^5({\mathrm{x}}_1{\mathrm{x}}_2-1)=0\hfill \\ & {\mathrm{F}}_3\left(\mathbf{x}\right)={({\mathrm{x}}_3+4)}^6=0\hfill \end{array}\right.\end{array}$$

(32)

The Jacobian ${\mathbf{F}}^{\prime }\left(\mathbf{x}\right)$ of Equation (32) is

$$\begin{array}{c}\hfill {\mathbf{F}}^{\prime }\left(\mathbf{x}\right)=\left[\begin{array}{ccc}4\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^3& \exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^4& 0\\ {\mathrm{x}}_2{({\mathrm{x}}_2-2)}^5& {\mathrm{x}}_1{({\mathrm{x}}_2-2)}^5+5({\mathrm{x}}_1{\mathrm{x}}_2-1){({\mathrm{x}}_2-2)}^4& 0\\ 0& 0& 6{({\mathrm{x}}_3+4)}^5\end{array}\right]\end{array}$$

(33)

Now we take a constant vector $\mathbf{w}={[{\mathrm{w}}_1,{\mathrm{w}}_2,{\mathrm{w}}_3]}^{\mathrm{T}}$. By multiplying ${\mathbf{F}}^{\prime }\left(\mathbf{x}\right)$ and $\mathbf{w}$, we get

$$\begin{array}{c}\hfill {\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}=\left[\begin{array}{c}4{\mathrm{w}}_1\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^3+{\mathrm{w}}_2\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^4\\ {\mathrm{w}}_2({\mathrm{x}}_1{({\mathrm{x}}_2-2)}^5+5({\mathrm{x}}_1{\mathrm{x}}_2-1){({\mathrm{x}}_2-2)}^4)+{\mathrm{w}}_1{\mathrm{x}}_2{({\mathrm{x}}_2-2)}^5\\ 6{\mathrm{w}}_3{({\mathrm{x}}_3+4)}^5\end{array}\right]\end{array}$$

(34)

Now again, we take the Jacobian of ${\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}$

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{F}}^{″}\left(\mathbf{x}\right)\mathbf{w}={\left({\mathbf{F}}^{\prime }\left(\mathbf{x}\right)\mathbf{w}\right)}^{\prime }=\hfill \\ & \left[\begin{array}{ccc}12{\mathrm{w}}_1\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^2+4{\mathrm{w}}_2\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^3& 4{\mathrm{w}}_1\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^3+{\mathrm{w}}_2\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^4& 0\\ {\mathrm{w}}_2(5{\mathrm{x}}_2{({\mathrm{x}}_2-2)}^4+{({\mathrm{x}}_2-2)}^5)& {\mathrm{w}}_1{({\mathrm{x}}_2-2)}^5+{\mathrm{w}}_2(10{\mathrm{x}}_1{({\mathrm{x}}_2-2)}^4+20({\mathrm{x}}_1{\mathrm{x}}_2-1){({\mathrm{x}}_2-2)}^3)+5{\mathrm{w}}_1{\mathrm{x}}_2{({\mathrm{x}}_2-2)}^4& 0\\ 0& 0& 30{\mathrm{w}}_3{({\mathrm{x}}_3+4)}^4\end{array}\right]\hfill \end{array}\end{array}$$

(35)

By replacing $\mathbf{w}$ with $\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)={[{\mathrm{f}}_1,{\mathrm{f}}_2,{\mathrm{f}}_3]}^{\mathrm{T}}$ in Equation (35), we obtain

$$\begin{array}{c}\hfill \begin{array}{cc}& {\mathbf{F}}^{″}\left(\mathbf{x}\right)\left(\mathbf{F}\left(\mathbf{x}\right)\odot \mathbf{G}\left(\mathbf{x}\right)\right)=\hfill \\ & \left[\begin{array}{ccc}12{\mathrm{f}}_1\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^2+4{\mathrm{f}}_2\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^3& 4{\mathrm{f}}_1\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^3+{\mathrm{f}}_2\exp \left({\mathrm{x}}_2\right){({\mathrm{x}}_1-1)}^4& 0\\ {\mathrm{f}}_2(5{\mathrm{x}}_2{({\mathrm{x}}_2-2)}^4+{({\mathrm{x}}_2-2)}^5)& {\mathrm{f}}_1{({\mathrm{x}}_2-2)}^5+{\mathrm{f}}_2(10{\mathrm{x}}_1{({\mathrm{x}}_2-2)}^4+20({\mathrm{x}}_1{\mathrm{x}}_2-1){({\mathrm{x}}_2-2)}^3)+5{\mathrm{f}}_1{\mathrm{x}}_2{({\mathrm{x}}_2-2)}^4& 0\\ 0& 0& 30{\mathrm{f}}_3{({\mathrm{x}}_3+4)}^4\end{array}\right]\hfill \end{array}\end{array}$$

(36)

For a large system of nonlinear equations, it is not practical that we compute symbolically second order Fréchet derivatives. There is a way to approximate the second order Fréchet derivative numerically by using the history of iterations [13]. However, then, it is hard to keep the quadratic convergence of the iterative method. On the other hand, the methods that do not use the second order Fréchet derivative, require the knowledge of multiplicities of roots. Again practically, it is hard to have the knowledge of root multiplicities for a general system of nonlinear equations. For single nonlinear equations, many authors propose some recipes to approximate the multiplicity of roots iteratively. Finally, we have two kinds of iterative method with quadratic convergence and with the inclusion of auxiliary function $\mathbf{G}\left(\mathbf{x}\right)$. We list them as

$$\begin{array}{cc}& {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\left(\text{diag}\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right){\mathbf{G}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)+\text{diag}\left(\mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right){\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\right)-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\odot \mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)\right)}^{-1}\hfill \\ & {\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\odot \mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)\hfill \end{array}$$

(37)

$$\begin{array}{cc}& {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)+\text{diag}\left(\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)\text{diag}{\left(\mathbf{G}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}{\mathbf{G}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}\text{diag}\left(\mathbf{m}\right)\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}$$

(38)

When we take $\mathbf{G}\left(\mathbf{x}\right)=\mathbf{1}$, the iterative methods Equations (37) and (38) reduce to the following forms

$$\begin{array}{cc}& {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\left({\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^2-{\mathbf{F}}^{″}\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\right)}^{-1}{\mathbf{F}}^{\prime }\left({\mathbf{x}}_{\mathrm{k}}\right)\mathbf{F}\left({\mathbf{x}}_{\mathrm{k}}\right)\hfill \end{array}$$

(39)

$$\begin{array}{cc}& {\mathbf{x}}_{\mathrm{k}+1}={\mathbf{x}}_{\mathrm{k}}-{\mathbf{F}}^{\prime }{\left({\mathbf{x}}_{\mathrm{k}}\right)}^{-1}\text{diag}\left(\mathbf{m}\right)\mathbf{F}\left(\mathbf{x}\right)\hfill \end{array}$$

(40)

respectively.

$$\begin{array}{c}\hfill \text{Problem}2=\left\{\begin{array}{c}{\mathrm{F}}_1\left(\mathbf{x}\right)={\mathrm{x}}_1{\mathrm{x}}_2=0\hfill \\ {\mathrm{F}}_2\left(\mathbf{x}\right)={\mathrm{x}}_2{\mathrm{x}}_3=0\hfill \\ {\mathrm{F}}_3\left(\mathbf{x}\right)={\mathrm{x}}_3{\mathrm{x}}_4=0\hfill \\ {\mathrm{F}}_4\left(\mathbf{x}\right)={\mathrm{x}}_4{\mathrm{x}}_1=0\hfill \end{array}\right.\end{array}$$

(41)

$$\begin{array}{c}\hfill \text{Problem}3=\left\{\begin{array}{c}{\mathrm{F}}_1\left(\mathbf{x}\right)=\sqrt{{\mathrm{x}}_1-1}{\mathrm{x}}_2{\mathrm{x}}_3=0\hfill \\ {\mathrm{F}}_2\left(\mathbf{x}\right)=\sqrt{{\mathrm{x}}_2-1}{\mathrm{x}}_1{\mathrm{x}}_3=0\hfill \\ {\mathrm{F}}_3\left(\mathbf{x}\right)=\sqrt{{\mathrm{x}}_3-1}{\mathrm{x}}_1{\mathrm{x}}_2=0\hfill \end{array}\right.\end{array}$$

(42)

In most of the cases, the iterative methods Equations (39) and (40) are badly conditioned. In Table 1, Table 2 and Table 3, we have shown that the exponential function is not the only and best choice to use as auxiliary function for rapid convergence. In the tables, one can see that a particular choice of auxiliary function for method Equation (37) gives the order of convergence greater than two. The multiplicities of roots in problem 3 are less than one and the iterative method in Equation (37) provides better accuracy in the solution of this problem. In the majority of cases, the performance of iterative method in Equation (37) with unknown multiplicity is better than that of the procedure in Equation (38).

Table 1. Problem 1: initial guess = $[2,1,-2]$, $\mathrm{m}=[4,5,6]$.

	$\mathrm{G}\left(\mathrm{x}\right)$	Iter	$\left|\right|\mathrm{x}-{\mathsf{\alpha }\left|\right|}_{\infty }$	Num. Stability	COC
Iterative method Equaiton (37)	$\mathbf{1}$	6	$\mathrm{O}\left({10}^{-43}\right)$	Badly-conditioned	$2.0$
	$6+\cos \left(\mathbf{x}\right)/10$	6	$\mathrm{O}\left({10}^{-51}\right)$	Well-conditioned	$2.05$
	$1+{\mathbf{x}}^3/1000$	6	$\mathrm{O}\left({10}^{-42}\right)$	Well-conditioned	$2.0$
	$\exp (-\mathbf{x}/100)$	6	$\mathrm{O}\left({10}^{-46}\right)$	Well-conditioned	$2.0$
Iterative method Equaiton (38)	$\mathbf{1}$	6	$\mathrm{O}\left({10}^{-30}\right)$	Badly-conditioned	$2.0$
	$6+\cos \left(\mathbf{x}\right)/10$	6	$\mathrm{O}\left({10}^{-30}\right)$	Well-conditioned	$2.0$
	$1+{\mathbf{x}}^3/1000$	6	$\mathrm{O}\left({10}^{-30}\right)$	Well-conditioned	$2.0$
	$\exp (\mathbf{x}/100)$	6	$\mathrm{O}\left({10}^{-30}\right)$	Well-conditioned	$2.0$

Table 1. Problem 1: initial guess = $[2,1,-2]$, $\mathrm{m}=[4,5,6]$.

	$\mathrm{G}\left(\mathrm{x}\right)$	Iter	$\left|\right|\mathrm{x}-{\mathsf{\alpha }\left|\right|}_{\infty }$	Num. Stability	COC
Iterative method Equaiton (37)	$\mathbf{1}$	6	$\mathrm{O}\left({10}^{-43}\right)$	Badly-conditioned	$2.0$
	$6+\cos \left(\mathbf{x}\right)/10$	6	$\mathrm{O}\left({10}^{-51}\right)$	Well-conditioned	$2.05$
	$1+{\mathbf{x}}^3/1000$	6	$\mathrm{O}\left({10}^{-42}\right)$	Well-conditioned	$2.0$
	$\exp (-\mathbf{x}/100)$	6	$\mathrm{O}\left({10}^{-46}\right)$	Well-conditioned	$2.0$
Iterative method Equaiton (38)	$\mathbf{1}$	6	$\mathrm{O}\left({10}^{-30}\right)$	Badly-conditioned	$2.0$
	$6+\cos \left(\mathbf{x}\right)/10$	6	$\mathrm{O}\left({10}^{-30}\right)$	Well-conditioned	$2.0$
	$1+{\mathbf{x}}^3/1000$	6	$\mathrm{O}\left({10}^{-30}\right)$	Well-conditioned	$2.0$
	$\exp (\mathbf{x}/100)$	6	$\mathrm{O}\left({10}^{-30}\right)$	Well-conditioned	$2.0$

Table 2. Problem 2: initial guess = $[1,2,4,3]$, $\mathrm{m}=[2,2,2,2]$.

	$\mathrm{G}\left(\mathrm{x}\right)$	Iter	${\left|\right|\mathrm{f}\left(\mathrm{x}\right)\left|\right|}_{\infty }$	Num. Stability	COC
Iterative method Equation (37)	$\mathbf{1}$	1	-	Badly-conditioned	-
	$6+\cos \left(\mathbf{x}\right)/10$	7	$\mathrm{O}\left({10}^{-1551}\right)$	Well-conditioned	$2.98$
	$1+{\mathbf{x}}^3/1000$	7	$\mathrm{O}\left({10}^{-8482}\right)$	Well-conditioned	$3.98$
	$\exp (\mathbf{x}/100)$	7	$\mathrm{O}\left({10}^{-376}\right)$	Well-conditioned	$2.00$
Iterative method Equation (38)	$\mathbf{1}$	1	-	Badly-conditioned	-
	$6+\cos \left(\mathbf{x}\right)/10$	20	$\mathrm{O}\left({10}^{-23}\right)$	Well-conditioned	$1.0$
	$1+{\mathbf{x}}^3/1000$	20	Not converging	Well-conditioned	-
	$\exp (\mathbf{x}/100)$	7	$\mathrm{O}\left({10}^{-443}\right)$	Well-conditioned	$2.0$

Table 2. Problem 2: initial guess = $[1,2,4,3]$, $\mathrm{m}=[2,2,2,2]$.

	$\mathrm{G}\left(\mathrm{x}\right)$	Iter	${\left|\right|\mathrm{f}\left(\mathrm{x}\right)\left|\right|}_{\infty }$	Num. Stability	COC
Iterative method Equation (37)	$\mathbf{1}$	1	-	Badly-conditioned	-
	$6+\cos \left(\mathbf{x}\right)/10$	7	$\mathrm{O}\left({10}^{-1551}\right)$	Well-conditioned	$2.98$
	$1+{\mathbf{x}}^3/1000$	7	$\mathrm{O}\left({10}^{-8482}\right)$	Well-conditioned	$3.98$
	$\exp (\mathbf{x}/100)$	7	$\mathrm{O}\left({10}^{-376}\right)$	Well-conditioned	$2.00$
Iterative method Equation (38)	$\mathbf{1}$	1	-	Badly-conditioned	-
	$6+\cos \left(\mathbf{x}\right)/10$	20	$\mathrm{O}\left({10}^{-23}\right)$	Well-conditioned	$1.0$
	$1+{\mathbf{x}}^3/1000$	20	Not converging	Well-conditioned	-
	$\exp (\mathbf{x}/100)$	7	$\mathrm{O}\left({10}^{-443}\right)$	Well-conditioned	$2.0$

Table 3. Problem 3: initial guess = $[2,4,3]$, $\mathrm{m}=[1/2,1/2,1/2]$.

	$\mathrm{G}\left(\mathrm{x}\right)$	Iter	${\left|\right|\mathrm{f}\left(\mathrm{x}\right)\left|\right|}_{\infty }$	Num. Stability	COC
Iterative method Equation (37)	$\mathbf{1}$	12	$\mathrm{O}\left({10}^{-2011}\right)$	Well-conditioned	$2.00$
	$6+\cos \left(\mathbf{x}\right)/10$	12	$\mathrm{O}\left({10}^{-1914}\right)$	Well-conditioned	$2.00$
	$1+{\mathbf{x}}^3/1000$	12	$\mathrm{O}\left({10}^{-1248}\right)$	Well-conditioned	$2.00$
	$\exp (-\mathbf{x}/10)$	12	$\mathrm{O}\left({10}^{-2767}\right)$	Well-conditioned	$2.00$
Iterative method Equation (38)	$\mathbf{1}$	1	-	Badly-conditioned	-
	$6+\cos \left(\mathbf{x}\right)/10$	12	$\mathrm{O}\left({10}^{-56}\right)$	Well-conditioned	$2.00$
	$1+{\mathbf{x}}^3/1000$	20	Not converging	Well-conditioned	-
	$\exp (-\mathbf{x}/10)$	7	$\mathrm{O}\left({10}^{-35}\right)$	Well-conditioned	$2.00$

Table 3. Problem 3: initial guess = $[2,4,3]$, $\mathrm{m}=[1/2,1/2,1/2]$.

	$\mathrm{G}\left(\mathrm{x}\right)$	Iter	${\left|\right|\mathrm{f}\left(\mathrm{x}\right)\left|\right|}_{\infty }$	Num. Stability	COC
Iterative method Equation (37)	$\mathbf{1}$	12	$\mathrm{O}\left({10}^{-2011}\right)$	Well-conditioned	$2.00$
	$6+\cos \left(\mathbf{x}\right)/10$	12	$\mathrm{O}\left({10}^{-1914}\right)$	Well-conditioned	$2.00$
	$1+{\mathbf{x}}^3/1000$	12	$\mathrm{O}\left({10}^{-1248}\right)$	Well-conditioned	$2.00$
	$\exp (-\mathbf{x}/10)$	12	$\mathrm{O}\left({10}^{-2767}\right)$	Well-conditioned	$2.00$
Iterative method Equation (38)	$\mathbf{1}$	1	-	Badly-conditioned	-
	$6+\cos \left(\mathbf{x}\right)/10$	12	$\mathrm{O}\left({10}^{-56}\right)$	Well-conditioned	$2.00$
	$1+{\mathbf{x}}^3/1000$	20	Not converging	Well-conditioned	-
	$\exp (-\mathbf{x}/10)$	7	$\mathrm{O}\left({10}^{-35}\right)$	Well-conditioned	$2.00$

6. Conclusions

We have shown that the exponential function is not the only choice to use as an auxiliary function (compare our conclusions with [10]). The scalar version of iterative method Equation (37) was developed in [12]. Moreover, we have shown that the vector method can not be constructed plainly by the same procedure. Even the iterative method Equation (37) is not a direct consequence of the Newton method, but our analysis shows that we have quadratic convergence as well. The computed COC confirms our claims regarding the order of convergence of different iterative methods. The validity and accuracy of constructed iterative methods are clearly depicted in our computed results for different problems.