Next Article in Journal
Augmentation of Densest Subgraph Finding Unsupervised Feature Selection Using Shared Nearest Neighbor Clustering
Next Article in Special Issue
Solving of the Inverse Boundary Value Problem for the Heat Conduction Equation in Two Intervals of Time
Previous Article in Journal
On the Numerical Treatment of the Temporal Discontinuity Arising from a Time-Varying Point Mass Attachment on a Waveguide
Previous Article in Special Issue
Overlapping Grid-Based Optimized Single-Step Hybrid Block Method for Solving First-Order Initial Value Problems
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Some Inexact Version of the Iterative Exponential Method for Solving Nonsmooth Equations

by
Marek J. Śmietański
Faculty of Mathematics and Computer Science, University of Lodz, Banacha 22, 90-238 Łódź, Poland
Algorithms 2023, 16(1), 27; https://doi.org/10.3390/a16010027
Submission received: 23 November 2022 / Revised: 21 December 2022 / Accepted: 22 December 2022 / Published: 3 January 2023

Abstract

:
This paper presents an inexact version of an exponential iterative method designed for solving nonlinear equations F ( x ) = 0 , where the function F is only locally Lipschitz continuous. The proposed algorithm is completely new as an essential extension of the iterative exponential method for solving nonsmooth equations. The method with backtracking is globally and superlinearly convergent under some mild assumptions imposed on F. The presented results of the numerical computations confirm both the theoretical properties of the new method and its practical effectiveness.

1. Introduction

Let F : R n R n be locally Lipschitz continuous. Consider the nonlinear equation in the following simplest form:
F ( x ) = 0 ,
where x = ( x 1 , , x n ) T , F ( x ) = ( f 1 ( x ) , , f n ( x ) ) T and f i : R n R , i = 1 , , n . Furthermore, we assume that there exists x R n , which is a solution to Equation (1) (i.e., F ( x ) = 0 ). It is worth recalling that Pang and Qi [1] presented significant motivations and a broad scope of applications for nonsmooth equations.
Some class of iterative exponential methods for solving one-dimensional nonlinear equations was proposed by Chen and Li [2]. One of the most interesting formulas, and simultaneously the simplest one, is the method which has the following form:
x k + 1 = x k exp f ( x k ) x k f ( x k ) , k = 0 , 1 , 2 ,
This method turns to the classic Newton method if we use the first-order Taylor series expansion of the expression exp f ( x k ) x k f ( x k ) . The above method in Equation (2) has (at least) quadratic convergence if the function f : R R is twice differentiable and f ( x ) 0 in a neighborhood of the solution. A substantial extension of the exponential method in Equation (2) to the nonsmooth case in R n was presented in [3]. This locally and superlinearly convergent method can be written in the following two-stage form:
V k h ( k ) = F ( x ( k ) ) x ( k + 1 ) = diag exp h i ( k ) x i ( k ) x ( k )
where the matrix V k is arbitrarily taken from the B-differential of F at x ( k ) .
On the other hand, one traditional approach for solving nonsmooth equations is the inexact generalized Newton method:
F ( x ( k ) ) + V k h ( k ) η k F ( x ( k ) ) ,
where the matrix V k could be arbitrarily taken from some subdifferential of F at x ( k ) (not only from the generalized Jacobian or B-differential but also from, for example, the ∗-differential) and 0 η k < 1 for all k = 0 , 1 , 2 , Martínez and Qi in [4] stated the local and superlinear convergence of the generalized Jacobian-based method with Equation (4) for solving equations with semismooth function F. Other versions of such Newton-type methods for solving nonsmooth equations were also considered among the others in [5,6,7]. The main concept of the inexact Newton-type method is to substitute the standard Newton equation V k h ( k ) = F ( x ( k ) ) with an inequality as in Equation (4), where the parameter η k [ 0 , 1 ) is the forcing term. Then, the Newton step h k may be determined inexactly, such as by the use of any iterative method with feasible inaccuracy, especially when the current approximation is far from the solution to Equation (1).
The main objective of this paper is to introduce an inexact version of the exponential method with a subdifferential for multidimensional nonsmooth cases. Therefore, we present some new iterative methods for solving nondifferentiable problems, which can be sufficiently effective in solving not only nonsmooth equations but also some important problems in nonsmooth optimization. The exponential method was introduced in [2] for finding single roots of univariate nonlinear equations and considered in [8] for solving unconstrained optimization problems. In both cases, the proposed formulas allow solving only smooth and one-dimensional problems. We discuss an algorithm which is intended to solve both nonsmooth and multi-dimensional problems. Our method allows solving Equation (1), in which the function is only Lipschitz continuous and need not be differentiable. A primary extension, shown in Equation (3), was introduced in [3], but this study focuses on the inexact approach. In this way, vector h ( k ) can be approximated in every iteration with some inaccuracy, which reduces the cost of determining it. However, all these methods are only locally convergent. A new approach also includes a backtracking procedure, and as a result, we obtained a global convergence of the inexact exponential method.
This paper is organized as follows. We recall some important needed notions and properties in Section 2. In Section 3, we not only introduce an algorithm but also prove its global and superlinear convergence. Section 4 features the results of numerical tests. In Section 5 at the end, we give some conclusions.

2. Preliminaries

Throughout the paper, · denotes the Euclidean norm on R n , where we regard the vector from R n as a column vector. However, all theoretical results do not depend on the choice of the norm.
We assume that the function F is locally Lipschitz continuous in the traditional sense. In other words, for every U R n , there exists a constant L > 0 such that
F ( x ) F ( y ) L x y   for   all   x , y U .
If F : R n R n is a differentiable function, we denote the Jacobian matrix of F at x as J F ( x ) and the set where F is differentiable as D F . The local Lipschitz continuity of the function F implies that it is differentiable almost everywhere, according to Rademacher’s theorem [9]. Then, we have
B F ( x ) = lim x i x J F x i , x i D F
which is called the Bouligand subdifferential (B-differential for short) of F at x [10]. Furthermore, the generalized Jacobian of function F at x (in the sense of Clarke [9]) is the convex hull of the Bouligand subdifferential such that
F ( x ) = conv B F ( x ) .
We say that the function F is BD-regular at x if F is locally Lipschitz continuous at x and if all matrices V from B F ( x ) are nonsingular. For BD-regular functions, Qi proved some important properties (see Lemma 2).
In turn, the set
b F ( x ) = B f 1 ( x ) × × B f n ( x ) ,
is called the b-differential of function F at x (defined by Sun and Han in [11]). It is easy to verify that in a one-dimensional case (i.e., when n = 1 ), the generalized Jacobian reduces to the Clarke generalized gradient and B F ( x ) = b F ( x ) i.e., the B-differential and b-differential coincide). Furthermore, a ∗-differential F ( x ) is a non-empty bounded set for any x such that
F ( x ) f 1 ( x ) × × f n ( x ) ,
where f i ( x ) denotes the generalized gradient of the component function f i at x. Obviously, for a locally Lipschitz function, all previously mentioned differentials, namely F ( x ) , B F ( x ) and b F ( x ) , are ∗-differentials (see Gao [12]). Obviously, if all components f i of function F are C 1 at x, then F ( x ) = B F ( x ) = { J F ( x ) } .
The notion of semismoothness was primarily introduced by Mifflin [13] for functionals. Later, Qi and Sun [14] extended the original definition to nonlinear operators. A function F is semismooth at x if F is locally Lipschitz continuous at x and for any h R n , the limit
lim V F ( x + t h ) , h h , t 0 V h .
exists. If function F is semismooth at x, then F is directionally differentiable at x, and  F ( x ; h ) is equal to the above limit:
Lemma 1
(Lemma 2.2 by Qi [15]). Suppose that the function F : R n R n is directionally differentiable in a neighborhood of x. The following statements are equivalent:
(1) 
F is semismooth at x;
(2) 
F ( · , · ) is semicontinuous at x;
(3) 
For any V F ( x + h ) , the following is true:
V h F ( x ; h ) = o ( h )   as   h 0 .
Moreover, we say that function F is p-order semismooth at x if for any matrix V from the generalized Jacobian F ( x + h ) , it holds that
V h F ( x , h ) = O ( h 1 + p ) as   h 0 ,
where 0 < p 1 . If  p = 1 , then F is just called strongly semismooth [16].
Remark 1.
(i) 
Clearly, semismoothness of any order implies semismoothness.
(ii) 
The piecewise C 2 function is strongly semismooth.
Lemma 2
(Lemma 2.6 by Qi [15]). If function F is BD-regular at x, then there is a neighborhood N of x and a constant C > 0 such that for any y N and matrix V B F ( y ) , V is nonsingular, and
V 1 C .
If the function F is also semismooth at y N , then for any h R n , the following is true:
h C F y ; h .
Qi and Sun showed in [14] that if a function F is semismooth at x, then
F ( x + h ) F ( x ) F ( x ; h ) = o h   for   any   h 0 .
Furthermore, if a function F is p-order semismooth at x, then
F ( x + h ) F ( x ) F ( x ; h ) = O ( h 1 + p )   for   any   h 0 .
Throughout this paper, N ( x , r ) denotes the closed ball with a center x and radius r in R n (if the center is known, and a radius is negligible, just N is used).
Furthermore, for a given diagonal matrix whose diagonal entries are d 1 , , d n such that
D = diag ( d 1 , , d n ) = d 1 0 0 0 d 2 0 0 0 d n ,
then exp ( D ) denotes a so-called exponential matrix, which has the following form:
exp ( D ) = e d 1 0 0 0 e d 2 0 0 0 e d n .

3. The Algorithm and Its Properties

In this section, we introduce a new algorithm, and we discuss its convergence. For solving Equation (1), we can consider a method in the descriptive two-stage form
determine   h ( k ) using   V k and   F ( x ( k ) ) compute   x ( k + 1 ) using   x ( k ) and   h ( k )
where matrix V k is taken from some subdifferential of F at x ( k ) . We suggest an approach which is a combination of two methods: the iterative exponential one and the inexact generalized Newton one with a decreasing function norm and backtracking. The proposed inexact approach is similar to the algorithm presented by Birgin et al. [17] for smooth equations, whereas the exponential formula was introduced in [3]. Because of the nondifferentiability of F, as iteration matrices, we use matrices from the B-differential instead of the Jacobians.
Remark 2.
(i) 
The matrix V k may be chosen absolutely arbitrarily from known elements of the Bouligand subdifferential (in Step 1 of Algorithm 1). Appropriate matrices are usually suitable Jacobians of F.
(ii) 
The backtracking approach used in Step 2 is a particular case of the minimum reduction algorithm.
(iii) 
The formula in Equation (10) for obtaining x ( k + 1 ) could be written in a substitute form, which explicitly shows how to generate a new approximation x ( k + 1 ) using components of the previous approximation x ( k ) :
x i ( k + 1 ) = x i ( k ) exp α k h i ( k ) x i ( k ) , i = 1 , , n .
(iv) 
Various strategies for the choice of the forcing sequence can be found in [18] but for the inexact methods for solving smooth equations. Therefore, most of these strategies would need to be modified with respect to the nondifferentiability of F. For example, we could set η k [ 0 , 1 ) as a small constant or use the following rule:
η k = V k h ( k ) + F ( x ( k ) ) F ( x ( k ) ) if   F ( x ( k ) ) 0 0 otherwise ,
where V k B F ( x ( k ) ) . Then, Equation (8) holds with equality.
Before the convergence of Algorithm 1 is proven, we present a necessary important assumption:
Algorithm 1: Inexact exponential method (IEM).
Assume that θ 0 , 1 , τ 1 , τ 2 0 , 1 and τ 1 < τ 2 . Let x ( 0 ) R n be an arbitrary starting approximation. The steps for obtaining x ( k + 1 ) from x ( k ) are as follows:
Step 1: Determine η k 0 , 1 and find an h ( k ) value that satisfies
V k h ( k ) + F ( x ( k ) ) η k F ( x ( k ) ) ,
where V k B F ( x ( k ) ) . Set α = 1 .
Step 2: If
F diag exp α h i ( k ) x i ( k ) x ( k ) 1 α θ 1 η k F ( x ( k ) )
then set α k = α and
x ( k + 1 ) = diag exp α k h i ( k ) x i ( k ) x ( k ) .
Otherwise, choose α τ 1 α , τ 2 α , set α = α , and repeat Step 2.
Assumption 1.
Function F satisfies Assumption 1 at x if for any y from a neighborhood N of x and any matrix V y B F ( y ) , it holds that
F ( y ) F ( x ) = V y y x + o y x .
Furthermore, function F satisfies Assumption 1 at x with a degree ρ if the following holds:
F ( y ) F ( x ) = V y y x + O ( y x ρ ) .
Remark 3.
(i) 
It is not difficult to indicate functions that satisfy Assumption 1. We can indicate three classes of such functions. Assumption 1 is implied not only by semismoothness but also by H-differentiability (introduced by Song et al. [19]) and second-order C-differentiability (introduced by Qi [10]).
(ii) 
If function F is BD-regular at x and satisfies Assumption 1 at x, then there is a neighborhood N of x and a constant C > 0 such that for any y N and matrix V B F ( y ) , the following is true:
y x C V y y x .
(iii) 
Lemma 2, proven in [7], states that if function F is BD-regular at x , satisfies Assumption 1 at x and l = max 2 β , 1 2 β + V y , where β = V y 1 and V y B F ( y ) , then
1 l y x F y l y x for   all   y N ,
where N is a neighborhood of x .
Now, we establish some theoretical foundations for Algorithm 1:
Lemma 3.
Assume that Algorithm 1 is carried out with η k 0 , 1 determined in such way that the inequality in Equation (9) holds. If the series k = 1 1 η k is divergent, then
lim k F ( x ( k ) ) = 0 .
Proof of Lemma 3.
Under Equation (9), the following is true:
F ( x ( k ) ) 1 α θ 1 η k F ( x ( k 1 ) ) F ( x ( 0 ) ) 0 j < k [ 1 α θ ( 1 η k ) ] F ( x ( 0 ) ) exp α θ 0 j < k ( 1 η k )
Since 1 η k 0 and α θ > 0 , convergence F ( x ( k ) ) to 0 follows from the divergence of k 0 ( 1 η k ) . □
The divergence of series k = 1 1 η k implies a satisfactory decrease in F over all steps such that F ( x ( k ) ) 0 , namely if the inequality in Equation (9) holds.
Lemma 4.
Let θ 0 , 1 and x be given, and assume that there is h ¯ , which satisfies V h ¯ + F x < F x , where V B F x . If function F satisfies Assumption 1 at x, then there is η min 0 , 1 such that for any η η min , 1 , there exists h, which satisfies
V h + F x η F x   and   F diag exp α h i x i x 1 α θ 1 η F x ,
where α 0 , 1 .
Proof of Lemma 4.
Obviously, F ( x ) 0 and h ¯ 0 .
Set
η ¯ = V h ¯ + F x F x , η min = max η ¯ , 1 1 η ¯ δ h ¯ , ϵ = 1 α θ 1 η ¯ F x h ¯ ,
where δ > 0 is sufficiently small for the following inequality to be satisfied:
F diag exp α h i x i x F x V h ϵ h ,
whenever h α δ . Under Assumption 1, such a value of δ exists. Since η ¯ < 1 , then η min < 1 .
Let
h = 1 η 1 η ¯ h ¯
for any η η min , 1 . Then, we have
V h + F x = 1 η V h ¯ + 1 η ¯ F x 1 η ¯ 1 η 1 η ¯ V h ¯ + F x + η η ¯ 1 η ¯ F x = 1 η 1 η ¯ η ¯ F x + η η ¯ 1 η ¯ F x = η F x .
In addition, since
h = 1 η 1 η ¯ h ¯ 1 η min 1 η ¯ h ¯ δ ,
then
F diag exp α h i x i x F diag exp α h i x i x F x V h + V h + F x ϵ 1 η 1 η ¯ h ¯ + η F x = 1 α θ 1 η F x + η F x = 1 α θ 1 η F x .
Lemma 4 above implies that the inexact exponential step is well-defined for the current approximation x ( k ) , and suitable η k and h ( k ) values exist for any θ 0 , 1 .
Now, we will prove that the method described by Algorithm 1 is convergent. The first theorem regarding convergence is analogous to the one presented in [20] for the inexact generalized Newton method for solving nonsmooth equations, albeit with some significant changes in the proof:
Theorem 1.
Let { x ( k ) } be a sequence generated by Algorithm 1 with η k 0 , and for each k, there is a constant M > 0 such that the following holds:
V k h ( k ) + F ( x ( k ) ) η k F ( x ( k ) )
and
h ( k ) M
for any matrix V k B F ( x ( k ) ) .
If function F satisfies Assumption 1 at x , then every limiting point of the sequence { x ( k ) } is a solution to Equation (1), and
lim k F ( x ( k ) ) = 0 .
Proof of Theorem 1.
Suppose that K is a sequence of indices such that lim k K x ( k ) = x . We have two cases due to the behavior of the sequence { α k } :
(i)
{ α k } is not convergent to 0.
Then, there is a sequence of indices K 0 K and α ¯ > 0 such that α k α ¯ > 0 for all k K 0 . Clearly, for all k K 0 , the following is true:
F ( x ( k + 1 ) ) F ( x ( k ) ) .
In turn, according to Equation (9), we have
F ( x ( k + 1 ) ) F ( x ( k ) ) α ¯ θ 1 η k F ( x ( k ) )   for   all   k K 0 .
By adding all these inequalities side by side, we obtain the following inequality:
α ¯ θ k K 0 1 η k F x ( k ) F x ( 0 ) .
Therefore, lim k K 0 F ( x ( k ) ) = 0 . Hence, F ( x ) = 0 .
(ii)
{ α k } is convergent to 0.
For k at large enough values, under the choice α (according to Step 2 of Algorithm 1), there is α k > α k , α k α k / τ 2 , α k / τ 1 such that
lim k K α k = 0
and
F diag exp α k h i ( k ) x i ( k ) x ( k ) > 1 α k θ 1 η k F ( x ( k ) ) .
Therefore, for some V k B F ( x ( k ) ) such that Equation (8) is satisfied, it holds that
1 α k θ 1 η k F ( x ( k ) ) < F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) + V k α k h ( k ) + F ( x ( k ) ) + V k α k h ( k ) F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) V k α k h ( k ) + α k F ( x ( k ) ) + V k h ( k ) + 1 α k F ( x ( k ) ) F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) V k α k h ( k ) + α k η k F ( x ( k ) ) + 1 α k F ( x ( k ) ) ,
which implies that
1 α k θ 1 η k F ( x ( k ) ) α k η k F ( x ( k ) ) 1 α k F ( x ( k ) ) < F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) V k α k h ( k ) ,
and finally
α k 1 θ 1 η k F ( x ( k ) ) < F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) V k α k h ( k ) .
Because α k > 0 , then
1 θ 1 η k F ( x ( k ) ) < F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) V k α k h ( k ) α k h ( k ) h ( k ) .
Since h ( k ) is bounded, α k and η k tend toward zero, Assumption 1 and the local Lipschitz continuity of function F imply that the product on the right side of the above inequality tends toward zero for large enough values of k. Hence, lim k K F ( x ( k ) ) = 0 and F ( x ) = 0 .
Let ε > 0 be arbitrary. Let j K be such that F ( x ( j ) ) < ε . Note that 0 1 α θ 1 η k < 1 . Therefore, according to Equation (9), the following holds:
F ( x ( k ) ) F ( x ( k 1 ) )   for   all   k 1 .
Therefore, if k > j , we have that F ( x ( k ) ) F ( x ( j ) ) ε , and thus
lim k F ( x ( k ) ) = 0 .
The main theoretical result of this section is presented in the theorem below. Theorem 2 establishes the sufficient conditions for superlinear convergence of the inexact exponential method for solving nonsmooth equations:
Theorem 2.
Let { x ( k ) } be a sequence generated by Algorithm 1 with η k 0 . If x is a limiting point of { x ( k ) } , function F is BD-regular at x and satisfies Assumption 1 at x , then the sequence { x ( k ) } is superlinearly convergent to x .
Proof of Theorem 2.
If matrix V k is nonsingular, then by Equation (8), we have that
h ( k ) = V k 1 V k h ( k ) = V k 1 V k h ( k ) + F ( x ( k ) ) V k 1 F ( x ( k ) ) V k 1 η k F ( x ( k ) ) + V k 1 F ( x ( k ) ) = V k 1 η k + 1 F ( x ( k ) ) .
From the BD-regularity of function F at x , there is a δ 1 > 0 such that for any x ( k ) N x , δ 1 and any V k B F ( x ( k ) ) , we have V k 1 C . Therefore, we have
h ( k ) 2 C F ( x ( k ) )   for   all   k .
Thus, from Theorem 1, it follows that
lim k F ( x ( k ) ) = 0
and every limiting point of { x ( k ) } is a solution to Equation (1). Furthermore, the construction of new approximation x ( k + 1 ) in Algorithm 1 and Equation (12) implies that
lim k x ( k + 1 ) x ( k ) = 0 .
Since function F is BD-regular at x , the inverse function theorem guarantees that there is δ 2 > 0 such that F ( x ) > 0 , provided 0 < x x < δ 2 . Let ε 0 , δ 2 be arbitrary. Then, the set N x , ε , δ 2 : = x R n : ε x x δ 2 cannot contain any solution. Therefore, not only does it not contain any limiting point, but it can also contain a finite number of approximations. Hence, there is k 0 such that, for all k k 0 , we have
x ( k ) N x , ε , δ 2 .
Let k 1 k 0 be such that
x ( k + 1 ) x ( k ) < δ 2 ε   for   all   k k 1 .
Since x is a limiting point, there is k k 1 such that x ( k ) x < ε , and hence
x ( k + 1 ) x x ( k + 1 ) x ( k ) + x ( k ) x < δ 2 .
However, we have that x ( k + 1 ) x < ε because k k 0 . Therefore, we also have that x ( k + j ) x ε for all integers j 1 . A number ε is arbitrary, so the sequence { x ( k ) } converges to a solution x to Equation (1).
According to Equation (8) and Assumption 1, for large enough k values, we have
F diag exp α k h i ( k ) x i ( k ) x ( k ) F ( x ( k ) ) + V k h ( k ) + o ( h ( k ) ) η k F ( x ( k ) ) + o ( h ( k ) ) .
Since both η k 0 and F ( x ( k ) ) 0 , we obtain
lim k F diag exp α k h i ( k ) x i ( k ) x ( k ) F x ( k ) = 0 .
Thus, for k values that are large enough, we have
F diag exp α k h i ( k ) x i ( k ) x ( k ) 1 θ F ( x ( k ) ) .
Therefore, the inequality in Equation (9) holds with α = 1 , and for k values that are large enough, we obtain
x ( k + 1 ) = diag exp α k h i ( k ) x i ( k ) x ( k ) .
Hence, we have
lim k F ( x ( k + 1 ) ) F ( x ( k ) ) = 0 .
Now, under the local Lipschitz continuity and BD-regularity of function F at x and Assumption 1, there exists l > 0 such that
1 l x x F x l x x
for all x in a neighborhood of x (see Remark 3 (iii)). Then, according to Equation (14), we have
lim k x ( k + 1 ) x x ( k ) x = 0 .
Remark 4.
(i) 
It is not hard to check that Theorem 2 is still true if our new Algorithm 1 is used with one of other subdifferentials mentioned in Section 1.
(ii) 
The method is quadratically convergent if the function F satisfies Assumption 1 at x with a degree of two.
Some versions of the inexact Newton methods have superlinear convergence under an assumption called a local error bound condition. It requires that
c x x F ( x )   for   all   x N
holds for a positive constant c in a neighborhood N of the solution to Equation (1). A local error bound holds, for example, for a piecewise linear function with non-unique solutions.
In solving smooth equations, a local error bound condition is noticeably weaker than the nonsingularity of the Jacobian F ( x ) (see [21]). In our nonsmooth case, this condition can be used as a weaker assumption instead of the BD-regularity. Obviously, for our inexact exponential method, both the BD-regularity and local error bound perform the same role as the nonsingularity of the Jacobian in a smooth case.

4. Numerical Experiments

Now, we present some numerical results which help us to confirm the theoretical properties of the algorithm. We solved a few different problems. Algorithm 1 was implemented in C++ with double precision arithmetic and performed using Apache NetBeans. In all examples, the crucial parameters were set to be alike as follows: θ = 0.999 , η k = 0.5 for all k and τ 1 = τ 2 = 0.5 . The stopping criterion was the condition F ( x ( k ) ) 2 1.0 × 10 7 . A flowchart of Algorithm 1 is presented in Figure 1. In our opinion, the results of the numerical computations were satisfactory, as we will now demonstrate.
In all examples, x ( 0 ) is the starting approximation, and N I E M is the number of iterations performed by Algorithm 1. For comparison of the efficiency of the new method, we also show the results obtained in tests by the inexact generalized Newton (IGM) method presented in [20] ( N I G M ), which also has superlinear convergence.
Example 1.
As the first (double) test problem, we chose the usual Equation (1) with two different nonsmooth functions:
F 1 ( x ) = 0.2 x x 1 + e x 0.5 1.05 , F 2 ( x ) = x 1 + ( x 2 1 ) 2 1 ( x 1 1 ) 2 + x 2 1
The equation with function F 1 has one solution x = 0.5 , while the equation with function F 2 has two solutions: ( 0 , 0 ) and ( 1 , 1 ) . Both equations have one nondifferentiability point each: x = 1 for F 1 and x = ( 0 , 0 ) for F 2 .
Both functions F 1 and F 2 are locally Lipschitz at their solution points. F 1 is C 1 at x because
F 1 ( x ) = 0.2 ( x 2 ) + e x 0.5   for   x < 1 , 0.2 ( x 2 ) + e x 0.5   for   x > 1 ,
Therefore, F 1 is also semismooth at x . In turn, Qi and Sun proved in [14] that the function is semismooth at some point if and only if all of its component functions are semismooth at this point. And the component functions of F 2 are semismooth as the sum of the semismooth functions (the local Lipschitz continuities of the absolute value function and quadratic function are easy to check). Moreover, the Bouligand subdifferential of F 1 at ( 0 , 0 ) has the form
B F ( ( 0 , 0 ) = 1 2 2 1 , 1 2 2 1 , 1 2 2 1 , 1 2 2 1 .
The equations can be solved by Algorithm 1. The only point of nondifferentiability of the function F 1 is not a solution, and function F 2 satisfies Assumption 1 (it follows from Remark 3(i)) and is BD-regular at x = ( 0 , 0 ) . The exemplary results are shown in Table 1 and Table 2.
Example 2.
The second test problem was Equation (1) with the function F : R n R n (used among the others in [4]), defined by
f i ( x ) = c 1 g i ( x )   for   g i ( x ) 0 , c 2 g i ( x )   for   g i ( x ) 0 ,
with
g i ( x ) = i j = 1 i cos ( x j 1 ) + j 1 cos ( x j 1 ) sin ( x j 1 ) .
Obviously, if c 1 = c 2 , then function F is differentiable. Therefore, the value c 1 c 2 may be treated as the degree of nondifferentiability of function F [22]. Equation (1) has an infinite number of solutions ( 1 + 2 k 1 π , , 1 + 2 k n π ) T , where k 1 , , k n Z are arbitrary. First, the starting approximation was 0 , , 0 . We used Algorithm 1 to solve one series of smooth problems (with c 1 = c 2 = 1 ) and three series of nonsmooth problems with various parameters c 1 = c 2 = 1 , 10 , 100 .
Function F is semismooth as a composite of semismooth functions (as mentioned in Example 1), and all functions f i are semismooth as piecewise smooth functions (see [13]).
In the smooth case, Algorithm 1 (IEM) stopped after six iterations, while Algorithm IGM generated seven successive approximations. The most interesting results we selected are presented in Table 3.
In these tests, we noticed that the choice of starting point did not affect the number of iterations very much when Algorithm 1 generated a convergent sequence. Therefore, we omitted the results for other starting points. The number of iterations was actually very similar for a small number of variables. However, Algorithm 1 showed its advantage for larger problems (i.e., when a system of nonlinear equations has at least a dozen unknowns).
Example 3.
The last test problem was the Walrasian production–price problem [23] with the demand function ξ defined in the following form:
ξ j π = i = 1 l a i j ϕ i π π j b i , j = 1 , , m .
Here, π R m is the price vector, a i j measures the i-th consumer’s intensity of demand for commodity j, and b i is the i-th consumer’s elasticity of substitution for i = 1 , , l . Furthermore, the following
ϕ i π = k = 1 m π k ω i k k = 1 m a i k π k 1 b i
is a function of the price chosen in such way to satisfy the budget constraint for each unit. Let B R m × n denote the technology, w R m denote the total endowment of the economy prior to production and y R n denote the production plan.
Then, the Walrasian model can be written as the nonlinear complementarity problem (NCP) in the form
min ( g ( x ) , f ( x ) ) = 0
with the following functions:
g π , y = π y   and   f π , y = ω ξ π + B y B T π ,
where the operator “min” denotes the component-wise minimum of the vectors.
Function F ( x ) = min ( g ( x ) , f ( x ) ) is locally Lipschitzian, semismooth and BD-regular, so we can use Algorithm 1 to solve nonsmooth Equation (1).
An approximate solution of such a problem (NCP) with m = 6 , n = 8 and l = 5 is
x ˜ = 0.220 , 0.251 , 0.161 , 0.055 , 0.106 , 0.207 , 0.463 , 0 , 3.940 , 0.006 , 0 , 0 , 0.438 , 0 .
We solved this problem using the inexact exponential method with various starting approximations from the neighborhood of x ˜ in the form
x ( 0 ) = x ˜ + a 1 e ˙ + a 2 e ¨ ,
where e ˙ = 1 , 1 , 1 , 1 , 1 , 1 , 0 , , 0 R 14 and e ¨ = 0 , 0 , 0 , 0 , 0 , 0 , 1 , , 1 R 14 . The exemplary results are shown in Table 4.
The Broyden-like method (from [24]) generated somewhat comparable results, which was not surprising since all these methods have the same order of convergence. The number of iterations was equal to 27, 9, −, 66 and 263 for the same initial points as in Table 4. In this test problem, the convergence significantly depended on the location of the starting point (parameters a 1 and a 2 ). Here, Algorithm 1 showed its advantage for initial points further from the solution.

5. Conclusions

We have proposed a new inexact exponential method for solving systems of nonsmooth equations (Equation (1)). We proved that the presented method has global and superlinear convergence under some mild assumptions when function F is locally Lipschitz continuous. In practice, Algorithm 1 (IEM) was significantly faster than the usual inexact generalized Newton (IGM) method. The used globalization technique turned out to be an effective approach to improve convergence behavior. The results of the numerical tests confirmed that the new inexact exponential method can be used to effectively solve miscellaneous nonsmooth problems such as nonsmooth equations, nonlinear complementarity problems or equlibrium problems.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Data sharing is not applicable to this article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Pang, J.; Qi, L. Nonsmooth equations: Motivation and algorithms. SIAM J. Optim. 1993, 3, 443–465. [Google Scholar] [CrossRef]
  2. Chen, J.; Li, W. On new exponential quadratically convergent iterative formulae. Appl. Math. Comput. 2006, 180, 242–246. [Google Scholar] [CrossRef]
  3. Śmietański, M.J. On a new exponential iterative method for solving nonsmooth equations. Numer. Linear Algebra Appl. 2019, 25, e2255. [Google Scholar] [CrossRef]
  4. Martínez, J.M.; Qi, L. Inexact Newton method for solving nonsmooth equations. J. Comput. Appl. Math. 1995, 60, 127–145. [Google Scholar] [CrossRef] [Green Version]
  5. Pu, D.; Tian, W. Globally convergent inexact generalized Newton’s methods for nonsmooth equations. J. Comput. Appl. Math. 2002, 138, 37–49. [Google Scholar] [CrossRef] [Green Version]
  6. Bonettini, S.; Tinti, F. A nonmonotone semismooth inexact Newton method. Optim. Meth. Soft. 2007, 22, 637–657. [Google Scholar] [CrossRef] [Green Version]
  7. Śmietański, M.J. Inexact quasi-Newton global convergent method for solving constrained nonsmooth equations. Int. J. Comput. Math. 2007, 84, 1157–1170. [Google Scholar] [CrossRef]
  8. Kahya, E. A class of exponential quadratically convergent iterative formulae for unconstrained optimization. Appl. Math. Comp. 2007, 186, 1010–1017. [Google Scholar] [CrossRef]
  9. Clarke, F.H. Optimization and Nonsmooth Analysis; SIAM: Philadelphia, PA, USA, 1990. [Google Scholar]
  10. Qi, L. C-Differential Operators, C-Differentiability and Generalized Newton Methods; Applied Mathematics Report AMR 96/5; University of New South Wales: Sydney, Australia, 1996. [Google Scholar]
  11. Sun, D.; Han, J. Newton and quasi-Newton methods for a class of nonsmooth equations and related problems. SIAM J. Optim. 1997, 7, 463–480. [Google Scholar] [CrossRef] [Green Version]
  12. Gao, Y. Newton methods for solving nonsmooth equations via a new subdifferential. Math. Meth. Oper. Res. 2001, 54, 239–257. [Google Scholar] [CrossRef]
  13. Mifflin, R. Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 1977, 15, 959–972. [Google Scholar] [CrossRef] [Green Version]
  14. Qi, L.; Sun, J. A nonsmooth version of Newton’s method. Math. Program. 1993, 58, 353–367. [Google Scholar] [CrossRef]
  15. Qi, L. Convergence analysis of some algorithms for solving nonsmooth equations. Math. Oper. Res. 1993, 18, 227–244. [Google Scholar] [CrossRef]
  16. Potra, F.A.; Qi, L.; Sun, D. Secant methods for semismooth equations. Numer. Math. 1998, 80, 305–324. [Google Scholar] [CrossRef]
  17. Birgin, E.; Krejić, N.; Martínez, J.M. Globally convergent inexact quasi-Newton methods for solving nonlinear systems. Numer. Algorithms 2003, 32, 249–260. [Google Scholar] [CrossRef]
  18. An, H.; Mo, Z.; Liu, X. A choice of forcing terms in inexact Newton method. J. Comput. Appl. Math. 2007, 200, 47–60. [Google Scholar] [CrossRef] [Green Version]
  19. Song, Y.; Gowda, M.S.; Ravindran, G. On the characterizations of P- and P0-properties in nonsmooth functions. Math. Oper. Res. 2000, 25, 400–408. [Google Scholar] [CrossRef]
  20. Śmietański, M.J. Some superlinearly convergent inexact quasi-Newton method for solving nonsmooth equations. Optim. Meth. Soft. 2012, 27, 405–417. [Google Scholar] [CrossRef]
  21. Yamashita, N.; Fukushima, M. On the rate of convergence of the Levenberg-Marquardt method. Computing 2001, 15, 237–249. [Google Scholar]
  22. Spedicato, E. Computational Experience with Quasi-Newton Algorithms for Minimization Problems of Moderately Large Size; Report CISE-N-175; Centro Informazioni Studi Esperienze: Milan, Italy, 1 October 1975. [Google Scholar]
  23. Scarf, H. The Computation of Economic Equilibria; Yale University Press: New Haven, CT, USA, 1973. [Google Scholar]
  24. Chen, X.; Qi, L. A parametrized Newton method and a quasi-Newton method for nonsmooth equations. Comput. Optim. Appl. 1994, 3, 157–179. [Google Scholar] [CrossRef]
Figure 1. Flowchart for Algorithm 1.
Figure 1. Flowchart for Algorithm 1.
Algorithms 16 00027 g001
Table 1. The numerical results obtained for the equation with function F 1 from Example 1.
Table 1. The numerical results obtained for the equation with function F 1 from Example 1.
x ( 0 ) N I E M N I G M
1 3441
0.11619
0.356
0.756
179
51316
101820
506974
100132147
Table 2. The numerical results obtained for the equation with function F 2 from Example 1 ( x indicates to which solution the sequence converged.).
Table 2. The numerical results obtained for the equation with function F 2 from Example 1 ( x indicates to which solution the sequence converged.).
x ( 0 ) N I E M N I G M x
( 100 , 100 )3335(0,0)
( 10 , 10 )2827(0,0)
( 10 , 5 )3438(0,0)
( 5 , 5 )2126(0,0)
( 1 , 1 )1517(0,0)
( 0.5 , 0.5 )1315(0,0)
(0.5,0.5)1719(1,1)
(5,5)810(1,1)
(5,10)912(1,1)
(10,10)1014(1,1)
(100,100)1619(1,1)
( 1 ,0.5)2731(0,0)
(2, 0.5 )3439(1,1)
Table 3. The numerical results obtained for the equation from Example 2 (n is the number of component functions.).
Table 3. The numerical results obtained for the equation from Example 2 (n is the number of component functions.).
c 1 = c 2 = 1 c 1 = c 2 = 10 c 1 = c 2 = 100
n N I E M N I G M N I E M N I G M N I E M N I G M
2565656
3565767
4677888
5687889
881099910
109109101011
12101110111111
15192123272530
20171919231821
Table 4. The numerical results obtained for the Walrasian production–price problem from Example 3 with various parameters a 1 and a 2 .
Table 4. The numerical results obtained for the Walrasian production–price problem from Example 3 with various parameters a 1 and a 2 .
a 1 0.020.20.00.52.0
a 2 0.020.00.20.52.0
N I E M 20117963224
N I G M 22129773253
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Śmietański, M.J. Some Inexact Version of the Iterative Exponential Method for Solving Nonsmooth Equations. Algorithms 2023, 16, 27. https://doi.org/10.3390/a16010027

AMA Style

Śmietański MJ. Some Inexact Version of the Iterative Exponential Method for Solving Nonsmooth Equations. Algorithms. 2023; 16(1):27. https://doi.org/10.3390/a16010027

Chicago/Turabian Style

Śmietański, Marek J. 2023. "Some Inexact Version of the Iterative Exponential Method for Solving Nonsmooth Equations" Algorithms 16, no. 1: 27. https://doi.org/10.3390/a16010027

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop