Gauss–Newton–Secant Method for Solving Nonlinear Least Squares Problems under Generalized Lipschitz Conditions

Abstract

We develop a local convergence of an iterative method for solving nonlinear least squares problems with operator decomposition under the classical and generalized Lipschitz conditions. We consider the case of both zero and nonzero residuals and determine their convergence orders. We use two types of Lipschitz conditions (center and restricted region conditions) to study the convergence of the method. Moreover, we obtain a larger radius of convergence and tighter error estimates than in previous works. Hence, we extend the applicability of this method under the same computational effort.

1. Introduction

Nonlinear least squares problems often arise while solving overdetermined systems of nonlinear equations, estimating parameters of physical processes by measurement results, constructing nonlinear regression models for solving engineering problems, etc. The most used method for solving nonlinear least squares problems is the Gauss–Newton method [1]. In the case when the derivative can not be calculated, difference methods are used [2,3].

Some nonlinear functions have a differentiable and a nondifferentiable part. In this case, a good idea is to use a sum of the derivative of the differentiable part of the operator and the divided difference of the nondifferentiable part instead of the Jacobian [4,5,6]. Numerical study shows that these methods converge faster than Gauss–Newton type’s method or difference methods.

In this paper, we study the local convergence of the Gauss–Newton–Secant method under the classical and generalized Lipschitz conditions for first-order Fréchet derivative and divided differences.

Let us consider the nonlinear least squares problem:

$$\underset{x\in R^p}{min}{\displaystyle \frac{1}{2}}{(F\left(x\right)+G\left(x\right))}^T(F\left(x\right)+G\left(x\right)),$$

(1)

where residual function $F+G:R^p\to R^m$ ($m\ge p$) is nonlinear in x, F is a continuously differentiable function, and G is a continuous function, the differentiability of which, in general, is not required.

We propose the following modification of the Gauss–Newton method combined with the Secant-type method [4,6] for finding the solution to problem (1):

$$x_{n+1}=x_n-{\left(A_n^T{A}_n\right)}^{-1}{A}_n^T(F\left(x_n\right)+G\left(x_n\right)),n=0,1,\dots ,$$

(2)

where $A_n=F^{\prime }\left(x_n\right)+G(x_n,x_{n-1})$, $F^{\prime }\left(x_n\right)$ is a Fréchet derivative of $F\left(x\right)$; $G(x_n,x_{n-1})$ is a divided difference of the first order of function $G\left(x\right)$ [7] at points $x_n$, $x_{n-1}$; and $x_0$, $x_{-1}$ are given.

Setting $A_n=F^{\prime }\left(x_n\right)$, for solving problem (1), from (2) we obtain an iterative Gauss–Newton-type method:

$$x_{n+1}=x_n-{\left(F^{\prime }{\left(x_n\right)}^T{F}^{\prime }\left(x_n\right)\right)}^{-1}{F}^{\prime }{\left(x_n\right)}^T(F\left(x_n\right)+G\left(x_n\right)),n=0,1,\dots .$$

(3)

For $m=p$, problem (1) turns into a system of nonlinear equations:

$$F\left(x\right)+G\left(x\right)=0.$$

(4)

In this case, method (2) is transformed into the combined Newton–Secant method [8,9,10]:

$$x_{n+1}=x_n-{(F^{\prime }\left(x_n\right)+G(x_n,x_{n-1}))}^{-1}(F\left(x_n\right)+G\left(x_n\right)),n=0,1,\dots ,$$

(5)

and method (3) into the Newtons-type method for solving nonlinear equations [11]:

$$x_{n+1}=x_n-{\left(F^{\prime }\left(x_n\right)\right)}^{-1}(F\left(x_n\right)+G\left(x_n\right)),n=0,1,\dots .$$

(6)

The convergence domain is small (in general), and error estimates are pessimistic. These problems restrict the applicability of these methods. The novelty of our work is in the claim that these problems can be addressed without adding hypotheses. In particular, our idea is to use a center and restricted radius Lipschitz conditions. Such an approach to the study of the convergence of methods allows for extending the convergence ball of the method and improving error estimates.

The remainder of the paper is organized as follows: Section 2 deals with the local convergence analysis. The numerical experiments appear in Section 3. Section 4 contains the concluding remarks and ideas about future works.

2. Local Convergence Analysis

Let us consider, at first, some auxiliary lemmas needed to obtain the main results. Let D be an open subset of $R^p$.

Lemma 1

([4]). Let ${\displaystyle e\left(t\right)={\int }_0^{t}E\left(u\right)du}$, where E is an integrable and positive nondecreasing function on $[0,T]$. Then, $e\left(t\right)$ is monotonically increasing with respect to t on $[0,T]$.

Lemma 2

([1,12]). Let ${\displaystyle h\left(t\right)={\displaystyle \frac{1}{t}}{\int }_0^{t}H\left(u\right)du,}$ where H is an integrable and positive nondecreasing function on $[0,T]$. Then, $h\left(t\right)$ is nondecreasing with respect to t on $(0,T]$.

Additionally, $h\left(t\right)$ at $t=0$ is defined as $h\left(0\right)=\underset{t\to 0}{lim}\left({\displaystyle {\displaystyle \frac{1}{t}}{\int }_0^{t}H\left(u\right)du}\right)$.

Lemma 3

([13]). Let ${\displaystyle s\left(t\right)={\displaystyle \frac{1}{t^2}}{\int }_0^{t}S\left(u\right)udu,}$ where S is an integrable and positive nondecreasing function on $[0,T]$. Then, $s\left(t\right)$ is nondecreasing with respect to t on $(0,T]$.

Definition 1.

The Fréchet derivative $F^{\prime }$ satisfies the center Lipschitz condition on D with $L_0$ average if

$$\parallel F^{\prime }\left(x\right)-F^{\prime }\left(x^{\ast }\right)\parallel \le {\int }_0^{\rho \left(x\right)}{L}_0\left(u\right)du,\mathit{for}\mathit{each}x\in D\subset R^p,$$

(7)

where $\rho \left(x\right)=\parallel x-x^{\ast }\parallel$, $x^{\ast }\in D$ is a solution of problem (1), and $L_0$ is an integrable, positive, and nondecreasing function on $[0,T]$.

The functions $M_0,L,M,L_1$ and $M_1$ introduced next are as the function $L_0$: integrable, positive, and nondecreasing functions defined on $[0,2R]$.

Definition 2.

The first order divided difference $G(x,y)$ satisfies the center Lipschitz condition on $D\times D$ with $M_0$ average if

$$\parallel G(x,y)-G(x^{\ast },x^{\ast })\parallel \le {\int }_0^{\rho \left(x\right)+\rho \left(y\right)}{M}_0\left(u\right)du,\mathit{for}\mathit{each}x,y\in D.$$

(8)

Let $B>0$ and $\alpha >0$. We define function $\phi$ on $[0,+\infty )$ by

$$\phi \left(t\right)=B\left[2\alpha +{\int }_0^t{L}_0\left(u\right)du+{\int }_0^{2t}{M}_0\left(u\right)du\right]\left[{\int }_0^t{L}_0\left(u\right)du+{\int }_0^{2t}{M}_0\left(u\right)du\right].$$

(9)

Suppose that equation

$$\phi \left(t\right)=1$$

(10)

has at least one positive solution. Denote by $\gamma$ the minimal such solution. Then, we can define ${\mathsf{\Omega }}_0=D\cap \mathsf{\Omega }(x^{\ast },\gamma )$, where $\mathsf{\Omega }(x^{\ast },\gamma )=\{x:\parallel x-x^{\ast }\parallel <\gamma \}$.

Definition 3.

The Fréchet derivative $F^{\prime }$ satisfies the restricted radius Lipschitz condition on ${\mathsf{\Omega }}_0$ with L average if

$$\parallel F^{\prime }\left(x\right)-F^{\prime }\left(x^{\tau }\right)\parallel \le {\int }_{\tau \rho \left(x\right)}^{\rho \left(x\right)}L\left(u\right)du,x^{\tau }=x^{\ast }+\tau (x-x^{\ast }),0\le \tau \le 1,\mathit{for}\mathit{each}x\in {\mathsf{\Omega }}_0.$$

(11)

Definition 4.

The first order divided difference $G(x,y)$ satisfies the restricted radius Lipschitz condition on ${\mathsf{\Omega }}_0$ with M average if

$$\parallel G(x,y)-G(u,v)\parallel \le {\int }_0^{\parallel x-u\parallel +\parallel y-v\parallel }M\left(u\right)du,\mathit{for}\mathit{each}x,y,u,v\in {\mathsf{\Omega }}_0.$$

(12)

Definition 5.

The Fréchet derivative $F^{\prime }$ satisfies the radius Lipschitz condition on D with $L_1$ average if

$$\parallel F^{\prime }\left(x\right)-F^{\prime }\left(x^{\tau }\right)\parallel \le {\int }_{\tau \rho \left(x\right)}^{\rho \left(x\right)}{L}_1\left(u\right)du,\mathit{for}\mathit{each}x\in D.$$

(13)

Definition 6.

The first order divided difference $G(x,y)$ satisfies the radius Lipschitz condition on D with $M_1$ average if

$$\parallel G(x,y)-G(u,v)\parallel \le {\int }_0^{\parallel x-u\parallel +\parallel y-v\parallel }{M}_1\left(u\right)du,\mathit{for}\mathit{each}x,y,u,v\in D.$$

(14)

Remark 1.

It follows from the preceding definitions that $L=L(L_0,M_0)$, $M=M(L_0,M_0)$, and for each $t\in [0,\gamma ]$

$$\begin{array}{ccc}\hfill L_0\left(t\right)& \le & L_1\left(t\right),\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill L\left(t\right)& \le & L_1\left(t\right),\hfill \end{array}$$

(16)

$$\begin{array}{ccc}\hfill M\left(t\right)& \le & M_1\left(t\right),\hfill \end{array}$$

(17)

since ${\mathsf{\Omega }}_0\subseteq D$. By $L(L_0,M_0)$, we mean that L (or M) depends on $L_0$ and $M_0$ by the definition of ${\mathsf{\Omega }}_0$. In case any of (15)–(17) are strict inequalities, the following benefits are obtained over the work in [4] using $L_1,M_1$ instead of the new functions:

(a1) 

An at least as large convergence region leading to at least as many initial choices;

(a2) 

At least as tight upper bounds on the distances $\parallel x_n-x^{\ast }\parallel$, so at least as few iterations are needed to obtain a desired error tolerance.

These benefits are obtained under the same computational effort as in [4], since the new functions $L_0,M_0,L,$ and M are special cases of the functions $L_1$ and $M_1$. This technique of using the center Lipschitz condition in combination with the restricted convergence region has been used by us on Newton’s, Secant, Newton-like methods [14,15], and can be used on other methods, too, with the same benefits.

The proof of the next result follows as the corresponding one in [4], but there are crucial differences, where we use $(L_0,L)$ instead of $L_1$ and $(M_0,M)$ instead of $M_1$ used in [4].

We use the Euclidean norm. Note that the following equality is satisfied for the Euclidean norm $\parallel A-B\parallel =\parallel A^T-B^T\parallel ,$ where $A,B\in R^{m\times p}$.

Theorem 1.

Let $F+G:R^p\to R^m$ be continuous on an open convex subset $D\subset R^p$, F be a continuously differentiable function, and G be a continuous function. Suppose that problem (1) has a solution $x^{\ast }\in D$; the inverse operation

$${\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}={\left[{(F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast }))}^T(F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast }))\right]}^{-1}$$

(18)

exists, such that $\parallel {\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}\parallel \le B$; (7), (8), (11), and (12) hold, and γ given in (10) exists.

Furthermore,

$$\begin{array}{c}\parallel F\left(x^{\ast }\right)+G\left(x^{\ast }\right)\parallel \le \eta ,\parallel F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast })\parallel \le \alpha ;\end{array}$$

(19)

$$\begin{array}{c}{\displaystyle \frac{B}{R}}\left({\int }_0^R{L}_0\left(u\right)du+{\int }_0^{2R}{M}_0\left(u\right)du\right)\eta <1\end{array}$$

(20)

and $\mathsf{\Omega }=\mathsf{\Omega }(x^{\ast },r_{\ast })\subseteq D,$ where $r_{\ast }$ is the unique positive zero of the function q given by

$$\begin{array}{ccc}\hfill q\left(r\right)& =& B\left[\left(\alpha +{\int }_0^r{L}_0\left(u\right)du+{\int }_0^{2r}{M}_0\left(u\right)du\right)\left({\int }_0^{r}L\left(u\right)udu+{\int }_0^{r}M\left(u\right)du\right)\right.\hfill \\ \hfill & & +\left(2\alpha +{\int }_0^r{L}_0\left(u\right)du+{\int }_0^{2r}{M}_0\left(u\right)du\right)\left({\int }_0^r{L}_0\left(u\right)du+{\int }_0^{2r}{M}_0\left(u\right)du\right)\hfill \\ & & \left.+\left({\displaystyle \frac{1}{r}}{\int }_0^r{L}_0\left(u\right)du+{\displaystyle \frac{1}{r}}{\int }_0^{2r}{M}_0\left(u\right)du\right)\eta \right]-1.\hfill \end{array}$$

(21)

Then, for $x_0,x_{-1}\in \mathsf{\Omega }$, the iterative sequence $\left\{x_n\right\}$, $n=0,1,\dots ,$ generated by (2), is well defined, remains in Ω, and converges to $x^{\ast }$. Moreover, the following error estimates hold for each $n=0,1,2,\dots$:

$$\begin{array}{ccc}\hfill \parallel x_{n+1}-x^{\ast }\parallel & \le & C_1\parallel x_{n-1}-x^{\ast }\parallel +C_2\parallel x_n-x^{\ast }\parallel +C_3\parallel x_{n-1}-x^{\ast }\parallel \parallel x_n-x^{\ast }\parallel \hfill \\ & & +C_4{\parallel x_n-x^{\ast }\parallel }^2,\hfill \end{array}$$

(22)

where

$$\begin{array}{ccc}\hfill g\left(r\right)& =& {\displaystyle \frac{B}{1-\phi \left(r\right)}};C_1=g\left(r_{\ast }\right){\displaystyle \frac{1}{2r_{\ast }}}{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\eta ;\hfill \end{array}$$

(23)

$$\begin{array}{ccc}\hfill C_2& =& g\left(r_{\ast }\right)\left({\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\displaystyle \frac{1}{2r_{\ast }}}{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right)\eta ;\hfill \end{array}$$

(24)

$$\begin{array}{ccc}\hfill C_3& =& g\left(r_{\ast }\right)\left(\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right){\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}M\left(u\right)du;\hfill \end{array}$$

(25)

$$\begin{array}{ccc}\hfill C_4& =& g\left(r_{\ast }\right)\left(\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right){\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}L\left(u\right)udu.\hfill \end{array}$$

(26)

Proof. 

We obtain

$$\begin{array}{ccc}\hfill {\displaystyle \underset{r\to 0^{+}}{lim}{\displaystyle \frac{1}{r}}{\int }_0^r{L}_0\left(u\right)du}& \le & \underset{r\to 0^{+}}{lim}{\displaystyle \frac{L_0\left(r\right)r}{r}}\le L_0\left(0\right),\hfill \end{array}$$

(27)

$$\begin{array}{ccc}\hfill {\displaystyle \underset{r\to 0^{+}}{lim}{\displaystyle \frac{1}{r}}{\int }_0^{2r}{M}_0\left(u\right)du}& \le & \underset{r\to 0^{+}}{lim}{\displaystyle \frac{M_0\left(2r\right)2r}{r}}\le 2M_0\left(0\right),\hfill \end{array}$$

(28)

since $L_0$ and $M_0$ are positive and nondecreasing functions on $[0,R]$, and $[0,2R]$, respectively. Taking into account Lemma 1 for a sufficiently small $\eta$, $q\left(0\right)=B(L_0\left(0\right)+2M_0\left(0\right))\eta -1<0$. With a sufficiently large R, the inequality $q\left(R\right)>0$ holds. By the intermediate value theorem, the function q has a positive zero on $(0,R)$ denoted by $r_{\ast }$. Moreover, this zero is the only one on $(0,R)$. Indeed, according to Lemma 2, the function ${\displaystyle \left({\displaystyle \frac{1}{r}}{\int }_0^r{L}_0\left(u\right)du+{\displaystyle \frac{1}{r}}{\int }_0^{2r}{M}_0\left(u\right)du\right)\eta }$ is non-decreasing with respect to r on $(0,R]$. By Lemma 1, functions ${\displaystyle {\int }_0^{r}L\left(u\right)du}$, ${\displaystyle {\int }_0^{r}M\left(u\right)du}$, and ${\displaystyle {\int }_0^{2r}M\left(u\right)du}$ are monotonically increasing on $[0,R]$. Furthermore, by Lemma 3, the function ${\displaystyle {\int }_0^{r}L\left(u\right)udu=r^2\left({\displaystyle \frac{1}{r^2}}{\int }_0^{r}L\left(u\right)udu\right)}$ is monotonically increasing with respect to r on $(0,R]$. Therefore, $q\left(r\right)$ is monotonically increasing on $(0,R]$. Thus, the graph of function $q\left(r\right)$ crosses the positive r-axis only once on $(0,R)$. Finally, from the monotonicity of q and since $q\left(\gamma \right)>0$, we obtain $r_{\ast }<\gamma$, so $\mathsf{\Omega }(x^{\ast },r_{\ast })\subset {\mathsf{\Omega }}_0$.

We denote $A_n=F^{\prime }\left(x_n\right)+G(x_n,x_{n-1})$. Let $n=0$. By the assumption $x_0$, $x_{-1}\in \mathsf{\Omega }$, we obtain the following estimation:

$$\begin{array}{c}∥I-{\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}{A}_0^T{A}_0∥=∥{\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}(A_{\ast }^T{A}_{\ast }-A_0^T{A}_0)∥\\ =∥{\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}\left[A_{\ast }^T(A_{\ast }-A_0)+(A_{\ast }^T-A_0^T)(A_0-A_{\ast })+(A_{\ast }^T-A_0^T)A_{\ast }\right]∥\\ \le ∥{\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}∥\left[\parallel A_{\ast }^T\parallel \parallel A_{\ast }-A_0\parallel +\parallel A_{\ast }^T-A_0^T\parallel \parallel A_0-A_{\ast }\parallel +\parallel A_{\ast }^T-A_0^T\parallel \parallel A_{\ast }\parallel \right]\\ \le B\left[\alpha \parallel A_{\ast }-A_0\parallel +\parallel A_{\ast }^T-A_0^T\parallel \parallel A_0-A_{\ast }\parallel +\alpha \parallel A_{\ast }^T-A_0^T\parallel \right].\end{array}$$

(29)

Using conditions (11) and (12), we obtain

$$\begin{array}{ccc}\hfill \parallel A_0-A_{\ast }\parallel & =& \parallel (F^{\prime }\left(x_0\right)+G(x_0,x_{-1}))-(F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast }))\parallel \hfill \\ \hfill & =& \parallel F^{\prime }\left(x_0\right)-F^{\prime }\left(x_{\ast }\right)+G(x_0,x_{-1})-G(x^{\ast },x^{\ast })\parallel \hfill \\ \hfill & \le & \parallel F^{\prime }\left(x_0\right)-F^{\prime }\left(x^{\ast }\right)\parallel +\parallel G(x_0,x_{-1})-G(x^{\ast },x^{\ast })\parallel \hfill \\ \hfill & \le & {\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du,\hfill \end{array}$$

(30)

where ${\rho }_k=\rho \left(x_k\right)$. Then, from inequality (29) and the equation $q\left(r\right)=0$, we obtain by (10)

$$\begin{array}{c}\parallel I-{\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}{A}_0^T{A}_0\parallel \le B\left[2\alpha +{\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du\right]\\ \times \left[{\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du\right]\le B\left[2\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du\right.\\ \left.+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\left[{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]<1.\end{array}$$

(31)

Next, from (29)–(31) and the Banach lemma [16], it follows that ${\left(A_0^T{A}_0\right)}^{-1}$ exists, and

$$\begin{array}{ccc}\hfill ∥{\left(A_0^T{A}_0\right)}^{-1}∥& \le & g_0=B\left\{1-B\left[2\alpha +{\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du\right]\right.\hfill \\ \hfill & & {\left.\times \left[{\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du\right]\right\}}^{-1}\hfill \\ \hfill & \le & g\left(r_{\ast }\right)=B\left\{1-B\left[2\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\right.\hfill \\ & & {\left.\times \left[{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\right\}}^{-1}.\hfill \end{array}$$

(32)

Hence, $x_1$ is correctly defined. Next, we will show that $x_1\in \mathsf{\Omega }(x^{\ast },r_{\ast })$.

Using the fact

$$A_{\ast }^T(F\left(x^{\ast }\right)+G\left(x^{\ast }\right))={(F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast }))}^T(F\left(x^{\ast }\right)+G\left(x^{\ast }\right))=0,$$

(33)

$x_0,x_{-1}\in \mathsf{\Omega }(x^{\ast },r_{\ast })$ and the choice of $r_{\ast }$, we obtain the estimate

$$\begin{array}{ccc}\hfill \parallel x_1-x^{\ast }\parallel & =& ∥x_0-x^{\ast }-{\left(A_0^T{A}_0\right)}^{-1}\left[A_0^T(F\left(x_0\right)+G\left(x_0\right))-A_{\ast }^T(F\left(x^{\ast }\right)+G\left(x^{\ast }\right))\right]∥\hfill \\ \hfill & \le & \parallel -{\left(A_0^T{A}_0\right)}^{-1}\parallel \parallel -A_0^T[A_0-{\int }_0^1{F}^{\prime }(x_{\ast }+t(x_0-x^{\ast }))dt\hfill \\ & & -G(x_0,x^{\ast })](x_0-x^{\ast })+(A_0^T-A_{\ast }^T)(F\left(x^{\ast }\right)+G\left(x^{\ast }\right))\parallel .\hfill \end{array}$$

(34)

So, considering the inequalities

$$\begin{array}{c}\parallel A_0-{\int }_0^1{F}^{\prime }(x^{\ast }+t(x_0-x^{\ast }))dt-G(x_0,x^{\ast })\parallel \\ =\parallel F^{\prime }\left(x_0\right)-{\int }_0^1{F}^{\prime }(x^{\ast }+t(x_0-x^{\ast }))dt+G(x_0,x_{-1})-G(x_0,x^{\ast })\parallel \\ =\parallel {\int }_0^1\left[F^{\prime }\left(x_0\right)-F^{\prime }(x^{\ast }+t(x_0-x^{\ast }))\right]dt+G(x_0,x_{-1})-G(x_0,x^{\ast })\parallel \\ =\parallel {\int }_0^1\left[F^{\prime }\left(x_0\right)-F^{\prime }\left(x_0^t\right)\right]dt+G(x_0,x_{-1})-G(x_0,x^{\ast })\parallel \\ \le {\int }_0^1{\int }_{t{\rho }_0}^{{\rho }_0}L\left(u\right)dudt+{\int }_0^{{\rho }_{-1}}M\left(u\right)du={\int }_0^{{\rho }_0}L\left(u\right)udu+{\int }_0^{{\rho }_{-1}}M\left(u\right)du\\ \le {\displaystyle \frac{1}{r_{\ast }^2}}{\int }_0^{r_{\ast }}L\left(u\right)udu{\rho }_0^2+{\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}M\left(u\right)du{\rho }_{-1},\end{array}$$

(35)

$$\begin{array}{c}\parallel A_0\parallel \le \parallel A_{\ast }\parallel +\parallel A_0-A_{\ast }\parallel \le \alpha +{\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du,\end{array}$$

(36)

we obtain

$$\begin{array}{ccc}\hfill \parallel x_1-x^{\ast }\parallel & \le & g_0\left\{\left[\alpha +{\int }_0^{{\rho }_0}{L}_0\left(u\right)du+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du\right]\right.\hfill \\ \hfill & & \times \left[{\int }_0^{{\rho }_0}L\left(u\right)udu+{\int }_0^{{\rho }_{-1}}M\left(u\right)du\right]\parallel x_0-x^{\ast }\parallel +\eta [{\int }_0^{{\rho }_0}{L}_0\left(u\right)du\hfill \\ \hfill & & \left.+{\int }_0^{{\rho }_0+{\rho }_{-1}}{M}_0\left(u\right)du]\right\}\le g_0\left\{\left[\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\right.\hfill \\ \hfill & & \times \left[{\displaystyle \frac{1}{r_{\ast }^2}}{\int }_0^{r_{\ast }}L\left(u\right)udu{\rho }_0^2+{\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}M\left(u\right)du{\rho }_{-1}\right]\parallel x_0-x^{\ast }\parallel \hfill \\ & & \left.+\eta \left[{\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}{L}_0\left(u\right)du{\rho }_0+{\displaystyle \frac{1}{2r_{\ast }}}{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du({\rho }_0+{\rho }_{-1})\right]\right\}\hfill \\ \hfill & <& g\left(r_{\ast }\right)\left\{\left[\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\left[{\int }_0^{r_{\ast }}L\left(u\right)udu+{\int }_0^{r_{\ast }}M\left(u\right)du\right]\right.\hfill \\ & & \left.+{\displaystyle \frac{1}{r_{\ast }}}\left[{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\eta \right\}{r}_{\ast }=p\left(r_{\ast }\right)r_{\ast }=r_{\ast },\hfill \end{array}$$

(37)

where

$$\begin{array}{ccc}\hfill p\left(r\right)& =& g\left(r\right)\left\{\left[\alpha +{\int }_0^r{L}_0\left(u\right)du+{\int }_0^{2r}{M}_0\left(u\right)du\right]\left[{\int }_0^{r}L\left(u\right)udu+{\int }_0^{r}M\left(u\right)du\right]\right.\hfill \\ & & \left.+{\displaystyle \frac{1}{r}}\left[{\int }_0^r{L}_0\left(u\right)du+{\int }_0^{2r}{M}_0\left(u\right)du\right]\eta \right\}.\hfill \end{array}$$

(38)

Therefore, $x_1\in \mathsf{\Omega }(x^{\ast },r_{\ast })$, and estimate (22) holds for $n=0$.

Let us assume that $x_n\in \mathsf{\Omega }(x^{\ast },r_{\ast })$ for $n=0,1,...,k$ and estimate (22) holds for $n=0,1,...,k-1$, where $k\ge$1 is an integer. We shall show $x_{n+1}\in \mathsf{\Omega }$ and that the estimate (22) holds for $n=k$.

We can write

$$\begin{array}{c}\parallel I-{\left(A_{\ast }^T{A}_{\ast }^T\right)}^{-1}{A}_{{}^k}^T{A}_k\parallel =\parallel {\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}(A_{\ast }^T{A}_{\ast }-A_{{}^k}^T{A}_k)\parallel \\ =\parallel {\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}(A_{\ast }^T(A_{\ast }-A_k)+(A_{\ast }^T-A_{{}^k}^T)(A_k-A_{\ast })+(A_{\ast }^T-A_{{}^k}^T)A_{\ast })\parallel \\ \le B\left(\alpha \parallel A_{\ast }-A_k\parallel +\parallel A_{\ast }^T-A_{{}^k}^T\parallel \parallel A_k-A_{\ast }\parallel +\alpha \parallel A_{\ast }^T-A_{{}^k}^T\parallel \right)\\ \le B\left[2\alpha +{\int }_0^{{\rho }_k}{L}_0\left(u\right)du+{\int }_0^{{\rho }_k+{\rho }_{k-1}}{M}_0\left(u\right)du\right][{\int }_0^{{\rho }_k}{L}_0\left(u\right)du\\ +{\int }_0^{{\rho }_k+{\rho }_{k-1}}{M}_0\left(u\right)du]\le B\left[2\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\\ \times \left[{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]<1.\end{array}$$

(39)

Consequently, ${\left(A_{{}^k}^T{A}_k\right)}^{-1}$ exists, and

$$\begin{array}{ccc}\hfill \parallel {\left(A_{k+1}^T{A}_{k+1}\right)}^{-1}\parallel & \le & g_k=B\left\{1-B\left[2\alpha +{\int }_0^{{\rho }_k}{L}_0\left(u\right)du+{\int }_0^{{\rho }_k+{\rho }_{k-1}}{M}_0\left(u\right)du\right]\right.\hfill \\ & & {\left.\times \left[{\int }_0^{{\rho }_k}{L}_0\left(u\right)du+{\int }_0^{{\rho }_k+{\rho }_{k-1}}{M}_0\left(u\right)du\right]\right\}}^{-1}\le g\left(r_{\ast }\right).\hfill \end{array}$$

(40)

Therefore, $x_{k+1}$ is correctly defined, and the following estimate holds:

$$\begin{array}{c}\parallel x_{k+1}-x_{\ast }\parallel =\parallel x_k-x^{\ast }-{\left(A_k^T{A}_k\right)}^{-1}[A_k^T(F\left(x_k\right)+G\left(x_k\right))-A_{\ast }^T(F\left(x^{\ast }\right)\\ +G\left(x^{\ast }\right)\left)\right]\parallel \le \parallel -{\left(A_k^T{A}_k\right)}^{-1}\parallel \parallel -A_k^T[A_k-{\int }_0^1{F}^{\prime }(x^{\ast }+t(x_k-x^{\ast }))dt\\ -G(x_k,x_{\ast })](x_k-x^{\ast })+(A_k^T-A_{\ast }^T)(F\left(x^{\ast }\right)+G\left(x^{\ast }\right))\parallel \\ \le \parallel -{\left(A_k^T{A}_k\right)}^{-1}\parallel \parallel -A_k^T[A_k-{\int }_0^1{F}^{\prime }(x^{\ast }+t(x_k-x^{\ast }))dt\\ -G(x_k,x_{\ast })](x_k-x^{\ast })+(A_k^T-A_{\ast }^T)(F\left(x^{\ast }\right)+G\left(x^{\ast }\right))\parallel \\ \le g_k\left\{\left[\alpha +{\int }_0^{{\rho }_k}{L}_0\left(u\right)du+{\int }_0^{{\rho }_k+{\rho }_{k-1}}{M}_0\left(u\right)du\right][{\int }_0^{{\rho }_k}L\left(u\right)udu\right.\\ \left.+{\int }_0^{{\rho }_{k-1}}M\left(u\right)du]\parallel x_k-x^{\ast }\parallel +\eta \left[{\int }_0^{{\rho }_k}{L}_0\left(u\right)du+{\int }_0^{{\rho }_k+{\rho }_{k-1}}{M}_0\left(u\right)du\right]\right\}\\ \le g\left(r_{\ast }\right)\left\{\left[\alpha +{\int }_0^{r_{\ast }}{L}_0\left(u\right)du+{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du\right]\right.\\ \times \left[{\displaystyle \frac{1}{r_{\ast }^2}}{\int }_0^{r_{\ast }}L\left(u\right)udu{\rho }_k^2+{\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}M\left(u\right)du{\rho }_{k-1}\right]\parallel x_k-x^{\ast }\parallel \\ \left.+\eta \left[{\displaystyle \frac{1}{r_{\ast }}}{\int }_0^{r_{\ast }}{L}_0\left(u\right)du{\rho }_k+{\displaystyle \frac{1}{2r_{\ast }}}{\int }_0^{2r_{\ast }}{M}_0\left(u\right)du({\rho }_k+{\rho }_{k-1})\right]\right\}<p\left(r_{\ast }\right)r_{\ast }=r_{\ast }.\end{array}$$

(41)

This proves that $x_{k+1}\in \mathsf{\Omega }(x^{\ast },r_{\ast })$ and estimate (22) for $n=k$.

Thus, by the induction method, (2) is correctly defined, $x_n\in \mathsf{\Omega }(x^{\ast },r_{\ast })$, and estimate (22) holds for each $n=0,1,2,\dots$.

It remains to be proven that $x_n\to x^{\ast }$ for $n\to \infty$.

Let us define functions a and b on $[0,r_{\ast }]$ as

$$\begin{array}{ccc}\hfill a\left(r\right)& =& g\left(r\right)\left\{\left[\alpha +{\int }_0^r{L}_0\left(u\right)du+{\int }_0^{2r}{M}_0\left(u\right)du\right]\left[{\int }_0^{r}L\left(u\right)udu+{\int }_0^{r}M\left(u\right)du\right]\right.\hfill \\ \hfill & & \left.+\left[{\displaystyle \frac{1}{r}}{\int }_0^r{L}_0\left(u\right)du+{\displaystyle \frac{1}{2r}}{\int }_0^{2r}{M}_0\left(u\right)du\right]\eta \right\};\hfill \end{array}$$

(42)

$$\begin{array}{ccc}b\left(r\right)& =& g\left(r\right){\displaystyle \frac{1}{2r}}{\int }_0^{2r}M\left(u\right)du\eta .\end{array}$$

(43)

According to the choice of $r_{\ast }$, we obtain

$$a\left(r_{\ast }\right)\ge 0,b\left(r_{\ast }\right)\ge 0,a\left(r_{\ast }\right)+b\left(r_{\ast }\right)=1.$$

(44)

Using estimate (22), the definition of functions a, b and constants $C_i$ ($i=1,2,3,4$), we have

$$\begin{array}{ccc}\hfill \parallel x_{n+1}-x^{\ast }\parallel & \le & C_1\parallel x_{n-1}-x^{\ast }\parallel +(C_2+C_3{r}_{\ast }+C_4{r}_{\ast })\parallel x_n-x^{\ast }\parallel \hfill \\ \hfill & =& a\left(r_{\ast }\right)\parallel x_n-x^{\ast }\parallel +b\left(r_{\ast }\right)\parallel x_{n-1}-x^{\ast }\parallel .\hfill \end{array}$$

(45)

According to the proof in [17], under the conditions (42)–(45), the sequence $\left\{x_n\right\}$ converges to $x^{\ast }$ for $n\to \infty$. □

Corollary 1

([4]). The convergence order of method (2) for the problem (1) with zero residual is equal to ${\displaystyle \frac{1+\sqrt{5}}{2}}$.

If $\eta =0$, we have the nonlinear least squares problem with zero residual. Then, the constants $C_1=0$ and $C_2=0$, and estimate (22) takes the form

$$\parallel x_{n+1}-x^{\ast }\parallel \le C_3\parallel x_{n-1}-x^{\ast }\parallel \parallel x_n-x^{\ast }\parallel +C_4{\parallel x_n-x^{\ast }\parallel }^2.$$

(46)

This inequality can be written as

$$\parallel x_{n+1}-x^{\ast }\parallel \le (C_3+C_4)\parallel x_{n-1}-x^{\ast }\parallel \parallel x_n-x^{\ast }\parallel .$$

(47)

Then, we can write an equation for determining the convergence order as follows:

$$t^2-t-1=0.$$

(48)

Therefore, the positive root, $t^{\ast }={\displaystyle \frac{1+\sqrt{5}}{2}}$ of the latter equation is the order of convergence of method (2).

In case $G\left(x\right)\equiv 0$ in (1), we obtain the following consequences.

Corollary 2

([4]). The convergence order of method (2) for problem (1) with zero residual is quadratic.

Indeed, if $G\left(x\right)\equiv 0$, then $C_3=0$, and estimate (22) takes the form

$$\parallel x_{n+1}-x^{\ast }\parallel \le C_4{\parallel x_n-x^{\ast }\parallel }^2,$$

(49)

which indicates the quadratic convergence rate of method (2).

Remark 2.

If $L_0=L=L_1$ and $M_0=M=M_1$, our results specialize to the corresponding ones in [4]. Otherwise, they constitute an improvement as already noted in Remark 1. As an example, let $q_1,g_1,C_1^1,C_2^1,C_3^1,C_4^1,r_{\ast }^1$ denote the functions and parameters where $L_0,L,M_0,M$ are replaced by $L_1,L_1,M_1,M_1$, respectively. Then, we have in view of (15)–(17) that

$$\begin{array}{ccc}\hfill q\left(r\right)& \le & q_1\left(r\right),\hfill \end{array}$$

(50)

$$\begin{array}{ccc}\hfill g\left(r\right)& \le & g_1\left(r\right),\hfill \end{array}$$

(51)

$$\begin{array}{ccc}\hfill C_1& \le & C_1^1,\hfill \end{array}$$

(52)

$$\begin{array}{ccc}\hfill C_2& \le & C_2^1,\hfill \end{array}$$

(53)

$$\begin{array}{ccc}\hfill C_3& \le & C_3^1,\hfill \end{array}$$

(54)

and

$$\begin{array}{ccc}\hfill C_4& \le & C_4^1.\hfill \end{array}$$

(55)

Hence, we have

$$\begin{array}{ccc}\hfill r_{\ast }^1& \le & r_{\ast },\hfill \end{array}$$

(56)

the new error bounds (22) being tighter than the corresponding (6) in [4], and the rest of the advantages (already mentioned in Remark 1) holding true.

Next, we study the convergence of method (2) if $L_0,L,M_0,M$ are constants, as a consequence of Theorem 1.

Corollary 3.

Let $F+G:R^p\to R^m$ be continuous on an open convex subset $D\subset R^p$, F be a continuously differentiable, and G be a continuous function on D. Suppose that problem (1) has a solution $x^{\ast }\in D$, and the inverse operation

$${\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}={\left[{(F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast }))}^T(F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast }))\right]}^{-1}$$

(57)

exists, such that $\parallel {\left(A_{\ast }^T{A}_{\ast }\right)}^{-1}\parallel \le B$.

Suppose that the Fréchet derivative $F^{\prime }$ satisfies the classic Lipschitz conditions

$$\begin{array}{ccc}\hfill \parallel F^{\prime }\left(x\right)-F^{\prime }\left(x^{\ast }\right)\parallel & \le & L_0\parallel x-x^{\ast }\parallel ,\mathit{for}\mathit{each}x\in D,\hfill \end{array}$$

(58)

$$\begin{array}{ccc}\hfill \parallel F^{\prime }\left(x\right)-F^{\prime }\left(y\right)\parallel & \le & L\parallel x-y\parallel ,\mathit{for}\mathit{each}x,y\in {\mathsf{\Omega }}_0\hfill \end{array}$$

(59)

and the function G has a first order divided difference $G(x,y)$ that satisfies

$$\begin{array}{ccc}\hfill \parallel G(x,y)-G(x^{\ast },x^{\ast })\parallel & \le & M_0(\parallel x-x^{\ast }\parallel +\parallel y-x^{\ast }\parallel ),\mathit{for}\mathit{each}x,y\in D,\hfill \end{array}$$

(60)

$$\begin{array}{ccc}\hfill \parallel G(x,y)-G(u,v)\parallel & \le & M(\parallel x-u\parallel +\parallel y-v\parallel ),\mathit{for}\mathit{each}x,y,u,v\in {\mathsf{\Omega }}_0,\hfill \end{array}$$

(61)

where ${\mathsf{\Omega }}_0=D\cap \mathsf{\Omega }\left({\displaystyle x^{\ast },\frac{\sqrt{B^2{\alpha }^2+B}-B\alpha }{B(L_0+2M_0)}}\right)$.

Furthermore,

$$\parallel F\left(x^{\ast }\right)+G\left(x^{\ast }\right)\parallel \le \eta ,\parallel F^{\prime }\left(x^{\ast }\right)+G(x^{\ast },x^{\ast })\parallel \le \alpha ,B(L_0+2M_0)\eta <1$$

(62)

and $\mathsf{\Omega }=\mathsf{\Omega }(x^{\ast },r_{\ast })\subseteq D,$ where

$$r_{\ast }={\displaystyle \frac{4(1-B{T}_0\eta )}{B\alpha (4T_0+T)+\sqrt{B^2{\alpha }^2{(4T_0+T)}^2+8B{T}_0(2T_0+T)(1-B{T}_0\eta )}}},$$

(63)

$T_0=L_0+2M_0,T=L+2M.$ Then, for each $x_0,x_{-1}\in \mathsf{\Omega }$, the iterative sequence $\left\{x_n\right\}$, $n=0,1,...,$ generated by (2) is well defined, remains in Ω, and converges to $x^{\ast }$, such that the following error estimate holds for each $n=0,1,2,\dots$:

$$\begin{array}{ccc}\hfill \parallel x_{n+1}-x^{\ast }\parallel & \le & C_1\parallel x_{n-1}-x^{\ast }\parallel +C_2\parallel x_n-x^{\ast }\parallel \hfill \\ \hfill & & +C_3\parallel x_{n-1}-x^{\ast }\parallel \parallel x_n-x^{\ast }\parallel +C_4{\parallel x_n-x^{\ast }\parallel }^2,\hfill \end{array}$$

(64)

where

$$\begin{array}{ccc}\hfill g\left(r\right)& =& B{[1-B(2\alpha +(L_0+2M_0)r)(L_0+2M_0)r]}^{-1};\hfill \end{array}$$

(65)

$$\begin{array}{ccc}\hfill C_1& =& g\left(r_{\ast }\right)M_0\eta ;C_2=g\left(r_{\ast }\right)(L_0+M_0)\eta ;\hfill \end{array}$$

(66)

$$\begin{array}{ccc}\hfill C_3& =& g\left(r_{\ast }\right)(\alpha +(L_0+2M_0)r_{\ast })M;\hfill \end{array}$$

(67)

$$\begin{array}{ccc}\hfill C_4& =& g\left(r_{\ast }\right)(\alpha +(L_0+2M_0)r_{\ast }){\displaystyle \frac{L}{2}}.\hfill \end{array}$$

(68)

The proof of Corollary 3 is analogous to the proof of Theorem 1.

3. Numerical Examples

In this section, we give examples to show the applicability of method (2) and to confirm Remark 2. We use the norm $\parallel x\parallel =\sqrt{{\displaystyle \sum _{i=1}^p}{x}_i^2}$ for $x\in {\mathrm{R}}^{\mathrm{p}}.$

Example 1.

Let function $F+G:R^2\to R^3$ be defined by

$$\begin{array}{c}F\left(x\right)+G\left(x\right)=\left(\begin{array}{c}3u^{2}v+v^2-1+|u^2-1|\\ u^4+u{v}^3-1+\left|v\right|\\ v-0.3+|u-1|\end{array}\right),\end{array}$$

(69)

$$\begin{array}{c}F\left(x\right)=\left(\begin{array}{c}3u^{2}v+v^2-1\\ u^4+u{v}^3-1\\ v-0.3\end{array}\right),G\left(x\right)=\left(\begin{array}{c}|u^2-1|\\ \left|v\right|\\ |u-1|\end{array}\right),\end{array}$$

(70)

where $x=(u,v)$. The solution of this problem $x^{\ast }\approx (0.917889,0.288314)$ and $\eta \approx 0.079411$.

Let us give the number of iterations needed to obtain an approximate solution of this problem. We test method (2) for the different initial points $x_0=\delta {(1.1,0.5)}^T$, where $\delta \in R$, and use the stopping criterion $\parallel x_{n+1}-x_n\parallel \le \epsilon$. The additional point $x_{-1}=x_0+{10}^{-4}$. The numerical results are shown in

Table 1. Results for Example 1, $\epsilon ={10}^{-8}$.

	$\mathit{\delta }=0.1$	$\mathit{\delta }=1$	$\mathit{\delta }=5$	$\mathit{\delta }=10$	$\mathit{\delta }=100$
Number of iterations	12	8	15	17	25

.

In

Table 2. Iterative sequence, norm of growth, and residual for Example 1, $x_0={(0.8,0.2)}^T$, $\epsilon ={10}^{-6}$.

n	${\mathit{x}}_{\mathit{n}+1}$	$\parallel {\mathit{x}}_{\mathit{n}+1}-{\mathit{x}}_{\mathit{n}}\parallel$	$\parallel \mathit{F}\left({\mathit{x}}_{\mathit{n}+1}\right)+\mathit{G}\left({\mathit{x}}_{\mathit{n}+1}\right)\parallel$
0	(0.937901, 0.312602)	0.178033	0.143759
1	(0.918455, 0.290216)	2.965298 × 10${}^{-2}$	7.973496 × 10${}^{-2}$
2	(0.917850, 0.288333)	1.977741 × 10${}^{-3}$	7.941104 × 10${}^{-2}$
3	(0.917888, 0.288313)	4.346993 × 10${}^{-5}$	7.941092 × 10${}^{-2}$
4	(0.917889, 0.288314)	7.873833 × 10${}^{-7}$	7.941092 × 10${}^{-2}$

, we give values of $x_{n+1}$, $\parallel x_{n+1}-x_n\parallel$ and the norm of residual at each iteration.

Example 2.

Let function $F+G:D\subseteq \mathrm{R}\to {\mathrm{R}}^3$ be defined by [5]:

$$\begin{array}{c}F\left(x\right)+G\left(x\right)=\left(\begin{array}{c}x+\mu \\ \lambda x^3+x-\mu \\ \lambda |x^2-1|-\lambda \end{array}\right),\end{array}$$

(71)

$$\begin{array}{c}F\left(x\right)=\left(\begin{array}{c}x+\mu \\ \lambda x^3+x-\mu \\ 0\end{array}\right),G\left(x\right)=\left(\begin{array}{c}0\\ 0\\ \lambda |x^2-1|-\lambda \end{array}\right),\end{array}$$

(72)

where $\lambda ,\mu \in \mathrm{R}$ are two parameters. Here $x^{\ast }=0$ and $\eta =\sqrt{2}\left|\mu \right|$. Thus, if $\mu =0$, then we have a problem with zero residual.

Let us consider Example 2 and show that $r_{\ast }^1\le r_{\ast }$ and the new error estimates (64) are tighter than the corresponding ones in [4]. We consider the case of the classical Lipschitz conditions (Corollary 3). Error estimates from [4] are as follows:

$$\begin{array}{ccc}\hfill \parallel x_{n+1}-x^{\ast }\parallel & \le & C_1^1\parallel x_{n-1}-x^{\ast }\parallel +C_2^1\parallel x_n-x^{\ast }\parallel \hfill \\ \hfill & & +C_3^1\parallel x_{n-1}-x^{\ast }\parallel \parallel x_n-x^{\ast }\parallel +C_4^1{\parallel x_n-x^{\ast }\parallel }^2,\hfill \end{array}$$

(73)

where

$$\begin{array}{ccc}\hfill g^1\left(r\right)& =& B{[1-B(2\alpha +(L_1+2M_1)r)(L_1+2M_1)r]}^{-1};\hfill \end{array}$$

(74)

$$\begin{array}{ccc}\hfill C_1^1& =& g^1\left(r_{\ast }^1\right)M_1\eta ;C_2^1=g^1\left(r_{\ast }^1\right)(L_1+M_1)\eta ;\hfill \end{array}$$

(75)

$$\begin{array}{ccc}\hfill C_3^1& =& g^1\left(r_{\ast }^1\right)(\alpha +(L_1+2M_1)r_{\ast }^1)M_1;\hfill \end{array}$$

(76)

$$\begin{array}{ccc}\hfill C_4^1& =& g^1\left(r_{\ast }^1\right)(\alpha +(L_1+2M_1)r_{\ast }^1){\displaystyle \frac{L_1}{2}}.\hfill \end{array}$$

(77)

They can be obtained from (64) by replacing $r_{\ast },L_0,L,M_0,M$ in $g\left(r\right)$, $C_1$, $C_2$, $C_3$, $C_4$ by $r_{\ast }^1,L_1,L_1,M_1,M_1$, respectively. Similarly,

$$r_{\ast }^1={\displaystyle \frac{4(1-B{T}_1\eta )}{5B\alpha T_1+\sqrt{25B^2{\alpha }^2{T}_1^2+24B{T}_1^2(1-B{T}_1\eta )}}},T_1=L_1+2M_1.$$

(78)

Let us choose $D=(-0.5;0.5)$. Thus, we have $B=0.5$, $\eta =\sqrt{2}\left|\mu \right|$, $\alpha =\sqrt{2}$, $L_0=\underset{x\in D}{max}3\left|\lambda \right|\left|x\right|$, $L=\underset{x,y\in {\mathsf{\Omega }}_0}{max}3\left|\lambda \right||x+y|$, $L_1=\underset{x,y\in D}{max}3\left|\lambda \right||x+y|$, $M_0=M=M_1=\left|\lambda \right|$. Radii are written in

Table 3. Radii of convergence domains.

$\mathit{\lambda }$	$\mathit{\mu }$	${\mathit{L}}_0$	L	${\mathit{L}}_1$	M	${\mathit{r}}_{\ast }$	${\mathit{r}}_{\ast }^1$
0.4	0	0.6	1.004205	1.2	0.4	0.319259	0.235702
0.1	0.2	0.15	0.3	0.3	0.1	1.192633	0.885163

.

Table 4. Results for $\lambda =0.4$, $\mu =0$.

n	$|{\mathit{x}}_{\mathit{n}+1}-{\mathit{x}}^{\ast }|$	The Right Side of (64)	The Right Side of (73)
0	4.364164 × 10${}^{-3}$	0.125318	0.169740
1	1.425535 × 10${}^{-5}$	1.245455 × 10${}^{-3}$	1.529729 × 10${}^{-3}$
2	2.179258 × 10${}^{-11}$	8.675961 × 10${}^{-8}$	1.060957 × 10${}^{-7}$
3	3.542853 × 10${}^{-22}$	4.314684 × 10${}^{-16}$	5.272102 × 10${}^{-16}$

and

Table 5. Results for $\lambda =0.1$, $\mu =0.2$.

n	$|{\mathit{x}}_{\mathit{n}+1}-{\mathit{x}}^{\ast }|$	The Right Side of (64)	The Right Side of (73)
0	2.063103 × 10${}^{-3}$	5.909333 × 10${}^{-2}$	8.484100 × 10${}^{-2}$
1	5.453349 × 10${}^{-7}$	9.113893 × 10${}^{-3}$	1.080560 × 10${}^{-2}$
2	2.054057 × 10${}^{-14}$	9.051468 × 10${}^{-5}$	1.057648 × 10${}^{-4}$
3	1.447579 × 10${}^{-18}$	2.390964 × 10${}^{-8}$	2.792694 × 10${}^{-8}$

report the left and right side of error estimates (64) and (73). We obtained these results for $\epsilon ={10}^{-8}$ and starting approximations $x_{-1}=0.2001$, $x_0=0.2$. We see that the new error bounds (64) are tighter than the corresponding (73) from [4].

4. Conclusions

We developed an improved local convergence analysis of the Gauss–Newton–Secant method for solving nonlinear least squares problems with nondifferentiable operator. We use a center and restricted radius Lipschitz conditions to study the method. As a consequence, we obtain a larger radius of convergence and tighter error estimates under the same computational effort as in earlier papers. This idea can be used to extend the usage of other methods with inverses, such as Newton-type, Secant-type, single-step, or multi-step, to mention a few. This should be our future work. Finally, it is worth mentioning that except for the methods used in this paper, some of the most representative computational intelligence algorithms can be used to solve the problems, such as monarch butterfly optimization (MBO) [18], the earthworm optimization algorithm (EWA) [19], elephant herding optimization (EHO) [20], the moth search (MS) algorithm [21], the slime mould algorithm (SMA), and Harris hawks optimization (HHO) [22].