Extending the Applicability of Newton’s Algorithm with Projections for Solving Generalized Equations

Abstract

A new technique is developed to extend the convergence ball of Newton’s algorithm with projections for solving generalized equations with constraints on the multidimensional Euclidean space. This goal is achieved by locating a more precise region than in earlier studies containing the solution on which the Lipschitz constants are smaller than the ones used in previous studies. These advances are obtained without additional conditions. This technique can be used to extend the usage of other iterative algorithms. Numerical experiments are used to demonstrate the superiority of the new results.

1. Introduction

Let $F:D⟶{\mathbb{R}}^i$ be a continuously differentiable operator; $D\subseteq {\mathbb{R}}^i$ an open set, $K\subset D,K$ a closed convex set, and $H:D\rightrightarrows {\mathbb{R}}^i$ be a set-valued operator equipped with a nonempty closed graph. We shall study the generalized equation

$$x\in K,0\in F\left(x\right)+H\left(x\right).$$

(1)

We are interested in finding a solution $x^{\ast }$ of the generalized equation, since many problems in nonlinear programming and other disciplines can be reduced to this equation using mathematical modeling.

Wy utilize the Newton Inexact Projection Algorithm (NIPA) formally defined in [1] by:

Step a. 

Choose $x_0\in K$ and let $\left\{{\tau }_m\right\}\subset [0,\infty )$ be given, and set $n=0.$

Step b. 

If $0\in F\left(x_n\right)+H\left(x_n\right),$ then terminate; otherwise, compute $v_n\in {\mathbb{R}}^i$ so that

$$0\in F\left(x_n\right)+F^{\prime }\left(x_n\right)(v_n-x_n)+H\left(v_n\right).$$

Step c. 

If $v_n\in K,$ let $x_{n+1}=v_n;$ otherwise choose any $x_{n+1}\in K$ so that

$$x_{n+1}\in P_K(v_n,x_n;{\tau }_n).$$

Step d. 

Let $n\leftarrow n+1,$ and repeat Step a.

Here $P_K(.,x,\tau ):{\mathbb{R}}^i\rightrightarrows K$ is the operator given as

$$P:=P_K(y_1,x,\tau ):=\{y_2\in K:\langle y_1-y_2,y_3-y_2\rangle \le \tau \parallel y_1-x{\parallel }^2\mathrm{for\; all}{y}_3\in K\}.$$

Operator P is called a feasible inexact projection.

The study of generalized equations was inaugurated by S. M. Robinson back in the 1970s [1,2]. A plethora of results was produced ever since utilizing iterative algorithms since solutions in closed form are rarely attainable [1,2,3,4,5,6,7,8]. It is known that if $H\equiv \left\{0\right\},$ then (1) reduces to a constrained generalized equation [2,3,4,7,8,9]. In these studies the superlinear and/or quadratic local convergence has been established under the condition of the metric regularity or strong metric regularity of the partial linearization of the function that defines a generalized equation. There is a surge in studies of this type since they provide an abstract model for several applications such as equilibrium, linear and nonlinear complementary problems, and variational inequality problems [1,2,5,9]. A common feature in these studies is the realization that region of accessibility or more commonly called convergence ball is not large in general. Hence, the choice of initial points for the iterative algorithm that guarantee its convergence is limited. Moreover, the upper error estimations on the distances are pessimistic in most cases, so more than necessary iterations are carried out to achieve a certain error tolerance. Extending the uniqueness of the solution ball is also an important issue. Motivated by the elegant work by Oliveira et al. [6], which generalized the earlier ones [2,3,4,7], as well as the aforementioned problems, we develop a technique leading to smaller Lipschitz constants than before [6,7,8], and consequently to a finer ball convergence. It is important to notice that these extensions are obtained without additional conditions. In particular, our technique is given for the NIPA. But it can be used with the same advantages on other iterative algorithms.

In Section 2 the ball convergence of NIPA is given, whereas the examples appears in Section 3.

2. Ball Convergence

The concept of metric regularity that follows plays a role in the ball convergence of NIPA.

Definition 1

([1]). We say that a set valued operator $Q:D\rightrightarrows {\mathbb{R}}^i$ is metrically regular at $y_2\in D$ for $y_1\in {\mathbb{R}}^i,$ if for $y_1\in Q\left(y_2\right),$ the graph of Q is closed (locally) at $(y_1,y_2).$ Moreover, there exist constants $\lambda ,\alpha ,\beta >0$ such that

$$U(y_2,\alpha )\subset D\mathrm{and}d(w,Q^{-1}\left(z\right))\le \lambda d(z,Q\left(w\right))\mathrm{for\; all}(w,z)\in U(y_1,\alpha )\times U(y_2,\beta ).$$

To avoid repetitions more details and properties of the standard concepts developed can be found in [1,2,5,6] and the references therein. Let $x^{\ast }\in K$ be such that $0\in F\left(x^{\ast }\right)+H\left(x^{\ast }\right).$ The following Lipschitz-type conditions are useful.

Definition 2.

Suppose: there exists ${\ell }_0>0$ such that

$$\parallel F^{\prime }\left(x\right)-F^{\prime }\left(x^{\ast }\right)\parallel \le {\ell }_0\parallel x-x^{\ast }\parallel \mathrm{for\; each}x\in D.$$

(2)

Then, we say operator $F^{\prime }$ is center-Lipschitz on $D.$

Consider $D_0:=D\cap U(x^{\ast },\frac{1}{{\ell }_0}).$

Definition 3.

Suppose: there exists $\ell >0$ such that

$$\parallel F^{\prime }\left(x\right)-F^{\prime }\left(y\right)\parallel \le \ell \parallel x-y\parallel \mathrm{for\; each}x,y\in D_0.$$

(3)

Then, we say operator $F^{\prime }$ is restricted-Lipschitz on $D_0.$

Definition 4.

Suppose: there exists ${\ell }_1>0$ such that

$$\parallel F^{\prime }\left(x\right)-F^{\prime }\left(y\right)\parallel \le {\ell }_1\parallel x-y\parallel \mathrm{for\; each}x,y\in D.$$

(4)

Then, we say operator $F^{\prime }$ is Lipschitz on $D.$

Consider the partial linearization of $F+H$ at some $x\in D$ [6], $T_{F+H}(x,.):D\rightrightarrows {\mathbb{R}}^i$ to be given as

$$T_{F+H}(x,z):=F\left(x\right)+F^{\prime }\left(x\right)(z-x)+H\left(z\right).$$

Let $\rho :=sup\{s\ge 0:U(x^{\ast },s)\subseteq D\}.$

Remark 1.

Condition (4) was used in the proof of the main result ([6], Theorem 2). But it turns out that (2) and (3) can be used instead. We have that

$$D_0\subseteq D,$$

(5)

resulting

$${\ell }_0\le {\ell }_1$$

(6)

and

$$\ell \le {\ell }_1,$$

(7)

where $\frac{{\ell }_0}{{\ell }_1}$ can be small (arbitrarily) [9,10,11]. The iterates $\left\{x_n\right\}$ belong in $D_0$ which is a more accurate region than D used in [6] (see also the numerical section). That is why we obtain smaller Lipschitz constants. Then, we also suppose

$${\ell }_0\le \ell ;$$

(8)

otherwise our results hold with ${\ell }_0$ replacing $\ell .$

If one uses (2) where it is needed and (3) everywhere else instead of ${\ell }_1,$ in the proof of Theorem 2 in [6], we can prove our main ball convergence result for NIPA:

Theorem 1.

Suppose: there exist $x^{\ast }\in K,{\ell }_0>0,\ell >0$ such that $0\in F\left(x^{\ast }\right)+H\left(x^{\ast }\right).$

Conditions (2) and (3) hold;

$$D\ni z⟶T_{F+H}(x^{\ast },z)$$

is metrically regular at $x^{\ast }$ for $0,$ with parameters $\lambda >0,\alpha >0$ and $\beta >0;$

$${\rho }^{\ast }:=min\{\rho ,\alpha ,\sqrt{\frac{2\beta }{2{\ell }_0+\ell }},\frac{2(1-\sqrt{2\overline{\tau }})}{\lambda (\ell +2{\ell }_0+(\ell -2{\ell }_0)\sqrt{2\overline{\tau }}}\},$$

where

$$\overline{\tau }:=\underset{n}{sup}{\tau }_n<\frac{1}{2}.$$

Then, sequence $\left\{x_n\right\}$ starting at $x_0\in K\cap U(x^{\ast },{\rho }^{\ast })-\left\{x^{\ast }\right\}$ and generated by NIPA that solves (1), related to $\left\{{\tau }_n\right\}$ belongs in $U(x^{\ast },{\rho }^{\ast })\cap K,$ and converges to $x^{\ast }$ so that for each $n=0,1,2,\dots$

$$e_{n+1}:=\parallel x_{n+1}-x^{\ast }\parallel \le r_n{e}_n,$$

(9)

where

$$r_n=(1+\sqrt{2{\overline{\tau }}_n})\frac{\lambda \ell e_n}{2(1-\lambda {\ell }_0{e}_n)}+\sqrt{2\overline{\tau }}.$$

(10)

Moreover, if $0={lim}_{n⟶\infty }{\tau }_n,$ then, the convergence is superlinear, and if ${\tau }_n=0$ then,

$$e_{n+1}\le \frac{(2{\ell }_0+\ell )\lambda }{2}{e}_n^2.$$

(11)

Furthermore, $x^{\ast }$ is the only solution of (1) in $U(x^{\ast },{\rho }^{\ast })$ provided that operator $T_{F+H}$ is metrically strongly regular at $x^{\ast }$ for $0.$

Remark 2.

(a) 

If we further specialize K or ${lim\; sup}_{n⟶\infty }{\tau }_n,$ then we obtain an improved version of the results in [4].

(b) 

If ${\ell }_0=\ell ={\ell }_1,$ our results reduce to the ones in [6]. Otherwise, (i.e., if ${\ell }_0<{\ell }_1$ or $\ell <{\ell }_1$), then our results give a larger ball of convergence (so more initial points $x_0$ become available); the error bounds are tighter (i.e., less iterates to be computed to achieve a desired error tolerance), and the uniqueness of the solution ball is enlarged. More precisely, we have

$${\overline{\rho }}^{\ast }\le {\rho }^{\ast },$$

(12)

and

$$e_n\le {\overline{e}}_n,$$

(13)

where ${\overline{\rho }}^{\ast },{\overline{e}}_n$ stand for corresponding radius and error bounds found in [6] (using only ${\ell }_1$). If ${\ell }_0,\ell$ are replaced by ${\ell }_1$ (see also (5)–(8)) it is noticeable that these extensions are not obtained using additional conditions because ${\ell }_0$ and ℓ are specializations of parameter ${\ell }_1.$ Notice that the results in [6] specialize to the results in [2,3,4,7,8]. Hence, by extending the results in [6], we have also extended the results in these references too. Examples where (6)–(8), (12) and (13) are strict follow in Section 3. Other examples can be found in [9,10,11].

3. Numerical Examples

We present a simple example to show estimates (5)–(8), (12) and (13) can be strict, so the advantages we obtained apply. For simplicity, we set $H=\left\{0\right\},\lambda =1,{\tau }_n=0,$$i=3$ and $D=\overline{U}(0,1).$

Example 1.

Let us consider a system of differential equations governing the motion of an object and given by

$$G_1^{\prime }\left(v\right)=e^{v_1},G_2^{\prime }\left(v_2\right)=(e-1)v_2+1,G_3^{\prime }\left(v_3\right)=1$$

with initial conditions $G_1\left(0\right)=G_2\left(0\right)=G_3\left(0\right)=0.$ Let $G={(G_1,G_2,G_3)}^T={(e^{v_1},\frac{e-1}{2}{v}_2^2+v_2,v_3)}^T.$ Let $D=\overline{U}(0,1),x_{\ast }={(0,0,0)}^T.$ Define function G on D for $v={(v_1,v_2,v_3)}^T$ by

$$G\left(v\right)={(e^{v_1}-1,\frac{e-1}{2}{v}_2^2+v_2,v_3)}^T.$$

Then, the derivative is given as

$$G^{\prime }\left(v\right)=\left[\begin{array}{ccc}{e}^{v_1}& 0& 0\\ 0& (e-1)v_2+1& 0\\ 0& 0& 1\end{array}\right].$$

Notice that conditions (2)–(4) hold if, we set ${\ell }_0=e-1<\ell =e^{\frac{1}{e-1}}<{\ell }_1=e.$ Then, the results in [6] are extended, since

$${\overline{\rho }}^{\ast }=\frac{2}{3{\ell }_1}=0.2453<{\rho }^{\ast }=\frac{2}{2{\ell }_0+\ell }=0.3827.$$

Moreover, for the error bounds, we have

$$\frac{\ell }{2(1-{\ell }_0{e}_n)}<\frac{{\ell }_1}{2(1-{\ell }_1{e}_n)},$$

so

$$e_n<{\overline{e}}_n.$$

Finally, in view of ${\overline{\rho }}^{\ast }<{\rho }^{\ast }$ the uniqueness of the solution $x^{\ast }$ is extended.

Example 2.

Let $C[0,1],$ the space of continuous functions defined on $[0,1]$ be equipped with the max norm. Let $D=\overline{U}(0,1).$ Define function F on D by

$$F\left(\phi \right)\left(x\right)=\phi \left(x\right)-5{\int }_0^{1}x\theta \phi {\left(\theta \right)}^{3}d\theta .$$

(14)

We have that

$$F^{\prime }\left(\phi \left(\xi \right)\right)\left(x\right)=\xi \left(x\right)-15{\int }_0^{1}x\theta \phi {\left(\theta \right)}^2\xi \left(\theta \right)d\theta ,\mathrm{for\; each}\xi \in D.$$

We have $\lambda =1,$ so ${\ell }_0=7.5<\ell ={\ell }_1=15.$ Then, the results in [6] are extended, since

$${\overline{\rho }}^{\ast }=\frac{2}{3{\ell }_1}=0.0444<{\rho }^{\ast }=\frac{2}{2{\ell }_0+\ell }=0.0667.$$

The same advantages as in Example hold for error bounds and uniquenes results.

4. Conclusions

We have extended the applicability of NIPA without additional conditions. The novelty of our study lies in the observation that our new technique generates a subset $D_0$ of D also containing the iterates $x_n.$ But on $D_0$ the Lipschitz constants ${\ell }_0,\ell$ are tighter (see (5)–(7)) than ${\ell }_1$ used in [4,5,6,7,8]. Hence, the aforementioned advantages are obtained. It is also worth noticing that no additional conditions are used and that ${\ell }_0,\ell$ are specializations of ${\ell }_1.$ So, no additional computation is needed.

Our technique can be used to do the same on other algorithms [1,3,4,9].