Next Article in Journal
Representations of a Comparison Measure between Two Fuzzy Sets

Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

# Two Bregman Projection Methods for Solving Variational Inequality Problems in Hilbert Spaces with Applications to Signal Processing

by
Lateef Olakunle Jolaoso
1,*,
Maggie Aphane
1 and
Safeer Hussain Khan
2
1
Department of Mathematics and Applied Mathematics, Sefako Makgatho Health Sciences University, P.O. Box 94, Medunsa 0204, South Africa
2
Department of Mathematics, Statistics and Physics, Qatar University, Doha 2713, Qatar
*
Author to whom correspondence should be addressed.
Symmetry 2020, 12(12), 2007; https://doi.org/10.3390/sym12122007
Submission received: 14 November 2020 / Revised: 24 November 2020 / Accepted: 25 November 2020 / Published: 5 December 2020

## Abstract

:
Studying Bregman distance iterative methods for solving optimization problems has become an important and very interesting topic because of the numerous applications of the Bregman distance techniques. These applications are based on the type of convex functions associated with the Bregman distance. In this paper, two different extragraident methods were proposed for studying pseudomonotone variational inequality problems using Bregman distance in real Hilbert spaces. The first algorithm uses a fixed stepsize which depends on a prior estimate of the Lipschitz constant of the cost operator. The second algorithm uses a self-adaptive stepsize which does not require prior estimate of the Lipschitz constant of the cost operator. Some convergence results were proved for approximating the solutions of pseudomonotone variational inequality problem under standard assumptions. Moreso, some numerical experiments were also given to illustrate the performance of the proposed algorithms using different convex functions such as the Shannon entropy and the Burg entropy. In addition, an application of the result to a signal processing problem is also presented.

## 1. Introduction

Let H be a real Hilbert space with norm $∥ · ∥$ and inner product $〈 · , · 〉 .$ We study the Variational Inequality Problem (shortly, VIP) of the form:
$Find v ¯ ∈ C such that 〈 A v ¯ , u − v ¯ 〉 ≥ 0 , ∀ u ∈ C ,$
where $A : H → H$ is a given operator and C is a nonempty closed convex subset of $H .$ We shall denote the set of solutions of the VIP (1) by $Ω V I .$ The VIP has drawn much attention from researchers because of its importance as a core model for studying many mathematical problems which include convex programming, equilibrium problem, inclusion problem, split feasibility problem, complimentarity problem, minimization problem, etc, see e.g., [1,2]. Recently, VIP has also been used for studying optimization problems arising in machine learning, signal processing and linear inverse problem; see, for instance [3,4,5].
Due to its importance, many researchers have developed several iterative methods for solving the VIP (1) in Hilbert and Banach spaces, see for example [6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22] and references therein. One of the simplest methods known for solving the VIP is the Extragradient Method (EM) introduced by Korpelevich [23] for a case where A is monotone and Lipschitz operator in a finite-dimensional space as follows: Given $a 0 ∈ C ⊆ R n ,$ compute
$b n = P C ( a n − γ A a n ) , a n + 1 = P C ( a n − γ A b n ) , n ≥ 1 ,$
where $γ ∈ ( 0 , 1 / L ) ,$ $A : H → H$ is monotone and L-Lipschitz continuous which has been extended to infinite-dimensional spaces by many authors, e.g., [8,9,13,19,24,25]. However, the EM requires computing the projection onto C twice per each iteration which is not cost effective in practice. As a result of this drawback, there has been an increasing effort in finding appropriate method for solving the VIP. Censor  et al. [13] introduced a Subgradient Extragradient Method (SEM) by replacing the secong projection in SEM with projection onto a half-space as follows:
Algorithm SEM: Given $a 0 ∈ C ,$ $γ ∈ ( 0 , 1 L )$, compute
$a 0 ∈ C , b n = P C ( a n − γ A a n ) , a n + 1 = P T n ( a n − γ A b n ) , where T n : = { x ∈ H : 〈 a n − γ A a n − b n , x − b n ≥ 0 } n ≥ 0 .$
Note that the set $T n$ is a half-space whose projection can be explicitly calculated (see for instance [26]). Under some mild assumptions, Censor et al. [13] obtained a weak convergence result for solving VIP using (3), and by modifying the SEM with Halpern iterative scheme (see [27]), they proved the strong convergences of the SEM under certain mild conditions (see for instance [14,15]).
It is worthy to mention that both the EM and SEM require two evaluations of A per each iteration. Popov [28] later proposed a nice modification of the EM that required the evaluation of A at only one point in each iteration. Malitsky and Semenov [29] modified the Popov’s extragradient method by replacing the second projection onto C with a projection onto a half-space. Further extension of Popov’s extragradient method was introduced by Nomirovskii et al. [30] by using Bregman distance function [31] in a finite-dimensional setting. Recently, Gibali [32] extended the methods of Malitsky and Semenov [29] and Nemirovskii et al. [30] to infinite-dimensional Hilbert space using a Bregman distance function. A weak convergence result is proved for the sequence generated by these methods provided the stepsize satisfies the condition $λ ∈ 0 , 2 − 1 L ,$ where A is monotone and L-Lipschitz continuous.
An obvious disadvantage in the above-mentioned results is that the stepsize of the algorithms requires a prior estimate of the Lipschitz constant L. It is known that the Lipschitz constant are too difficult to estimate in general. Even when it is possible, the estimate is often too small which affects the convergence of the algorithm. An effort in solving this problem for the case of monotone VIP was recently introduced by Hieu and Cholamjiak [33]. They introduced an extragradient method with Bregman distance which does not require a prior estimate of the Lipschitz constant. However, their algorithm can only be applicable to monotone VIP but not pseudomonotone. When the cost operator is not monotone, the result of [33] can not be applied to such VIP, for instance, see Example 3.12 and 4.2 of [19].
Motivated by these results, in the present paper, we first introduce two Bregman extragradient methods of Popov’s type for solving pseudomonotone variational inequalities in real Hilbert spaces. In the first algorithm, the stepsize requires a prior estimate of the Lipschitz constant. This extends the results of [30,32] from monotone VIP to pseudomonotone VIP in real Hilbert spaces. In the second algorithm, the stepsize is selected self-adaptively and does not require a prior estimate of the Lipschitz constant. This improves Algorithm 1 of the paper and also the results of  [30,32]. It also improves the method of [33,34] from the extragradient type method with Bregman distance to the subgradient extragradient method with Bregman distance. More so, it extends the result of [33,34] from monotone VIP to pseudomonotone VIP. We further provide some numerical examples using different types of convex functions and a numerical application of our result to the Least Absolute Shrinkage and Selection Operator (LASSO) problem in compressing sensing. We denote the strong and weak convergence of a sequence ${ a n } ⊆ H$ to a point $x ∈ H$ by $a n → x$ and $a n ⇀ x$ respectively.
The paper is organized as follows: We first recall some basic definitions and results in Section 2. We present our algorithm and discuss its convergence in Section 3. We give some numerical experiments in Section 4. We conclude with a final remark in Section 5.

## 2. Preliminaries

In this section, we recall some basic definitions and concepts that are needed for establishing our results.
Definition 1.
[20] Let$A : C → H$be an operator. Then A is called
(i) monotone if $〈 A u − A z , u − z 〉 ≥ 0$ for all $u , z ∈ C ;$
(ii) α-strongly pseudomonotone if
$〈 A u , z − u 〉 ≥ 0 ⇒ 〈 A z , z − u 〉 ≥ α ∥ u − z ∥ 2 ∀ u , z ∈ C , α > 0 ;$
(iii) pseudomonotone if for all $u , z ∈ C ,$
$〈 A u , z − u 〉 ≥ 0 ⇒ 〈 A z , z − u 〉 ≥ 0 ;$
(iv) Lipschitz continuous if there exists a constant $L > 0$ such that
$∥ A u − A z ∥ ≤ L ∥ u − z ∥ ∀ u , z ∈ C ;$
(v) weakly sequentially continuous if for each sequence ${ a n }$, we have:
$a n ⇀ p ⇒ A a n ⇀ A p .$
Definition 2.
A function $f : H → R ∪ { + ∞ }$ is said to be proper if its effective domain, i.e., $d o m f = { v ∈ H : f ( v ) < + ∞ }$ is nonempty. The Fenchel conjugate function of f is the convex function $f * : H → R$ defined by $f * ( b ) = sup { 〈 b , x 〉 − f ( x ) : x ∈ H } .$ f is said to be subdifferentiable at a point $v ∈ E$ if the set $∂ f ( v ) = { e ∈ H : f ( y ) − f ( v ) ≥ 〈 e , y − v 〉 , ∀ y ∈ H }$ is nonempty. An element $e ∈ ∂ f ( x )$ is called subgradient of $f .$ If f is continuously differentiable, then $∂ f ( x ) = { ∇ f ( x ) }$, where $∇ f ( x )$ denotes the gradient of f at $x .$; f is call Legendre if
(i) int(domf) $≠ ∅$ and $∂ f$ is single-valued on its domain
(ii) int(dom$f *$) $≠ ∅$ and $∂ f *$ is single-valued on its domain.
The function f is said to be strongly convex with strong convexity constant $σ > 0 ,$ if
$f ( u ) ≥ f ( v ) − 〈 ∇ f ( v ) , u − v 〉 + σ 2 ∥ u − v ∥ 2 , ∀ u ∈ dom f and v ∈ int ( dom f ) .$
A function $f : H → R$ is Gâteaux differentiable at $u ∈ i n t ( d o m f )$ if the limit
$f o ( u , v ) = lim λ → 0 + f ( u + λ v ) − f ( u ) λ ∀ v ∈ H ,$
exists for any $v ∈ H .$ The gradient of f at u is a linear function $∇ f ( u )$ defined by $〈 ∇ f ( u ) , y 〉 = f o ( u , v )$$∀ v ∈ H .$ When the limit in (5) holds for all $u ∈ i n t ( d o m f ) ,$ we say that f is Gâteaux differentiable. More so, when (5) is attained uniformly for any $v ∈ H$ with $∥ v ∥ = 1 ,$ we say that f is Fréchet differentiable. It is well-known that f is Fréchet differentiable if and only if the gradient $∇ f$ is norm-to-norm continuous at u (see [35]).
Given a Gâteaux differentiable function $f : H → R ∪ { + ∞ }$. The Bregman distance corresponding to the function f is defined by (see [31])
$Δ f ( v , u ) = f ( v ) − f ( u ) − 〈 ∇ f ( u ) , v − u 〉 ∀ v ∈ dom f and u ∈ int ( dom f ) .$
Remark 1.
The Bregman distance was introduced by Bregman [31] as a nice and effective method for designing and analyzing feasibility and optimization algorithms. It is easy to see that for any $v ∈ d o m f , u , w ∈ i n t ( d o m f )$, we have
$Δ f ( v , w ) = Δ f ( v , u ) + Δ f ( u , w ) + 〈 ∇ f ( u ) − ∇ f ( w ) , v − u 〉 .$
Moreover, when f is strongly convex with strong convexity modulus $σ > 0$, then the Bregman distance satisfies the following identity:
$Δ f ( v , u ) ≥ σ 2 ∥ v − u ∥ 2 .$
It is important we emphasize that various types of function f gives different Bregman distance. We give the following important examples of some practical important types of function f and their corresponding Bregman distance (see, e.g., [33,36]):
Example 1.
(i) If $f ( v ) = 1 2 ∥ v ∥ 2 ,$ then
$Δ f ( v , u ) = 1 2 ∥ v − u ∥ 2 ,$
which is the squared Euclidean distance (SE).
(ii) If $f ( v ) = ∑ i = 1 n a i log ( v i )$ called Shannon entropy, then its corresponding Bregman distance is given as
$Δ f ( v , u ) = ∑ i = 1 n ( v i log ( v i u i ) + u i − v i ) .$
This distance is called Kullback–Leibler distance (KL) and as become a very important tool in several areas of applied mathematics such as machine learning.
(iii) If $f ( v ) = − ∑ i = 1 n log ( v i )$ called Burg entropy, then its corresponding Bregman distance is given as
$Δ f ( v , u ) = ∑ i = 1 n ( log ( v i u i ) + v i u i − 1 ) .$
This is called Itakura–Saito distance (IS), which is very important in information theory.
(iv) If $f ( v ) = 1 2 v T D v$ where $v T$ is stands for the transpose of $v ∈ R n ,$ and $D = d i a g ( 1 , 2 , … , n ) ∈ R n$ is a positive semidefinite matrix, then
$Δ f ( v , u ) = 1 2 ( v − u ) D T ( v − u ) ,$
which is called the Mahalanobis distance (MD) used for cluster analysis.
Definition 3.
[37] The Bregman projection with respect to f of $v ∈$ int(domf) onto a nonempty closed convex set $C ⊂$ int(domf) is the unique vector $Π C ( v ) ∈ C$ satisfying
$Π C ( v ) : = inf { Δ f ( v , u ) : u ∈ C } .$
Similar with the metric projection, the Bregman projection is characterized by the following identities (see [37]):
$z = Π C ( v ) ⇔ 〈 ∇ f ( v ) − ∇ f ( z ) , u − z 〉 ≤ 0 , ∀ u ∈ C ,$
and
$Δ f ( u , Π C ( v ) ) + Δ f ( Π C ( v ) , v ) ≤ Δ f ( u , v ) ∀ u ∈ C , v ∈ int ( dom f ) .$
Lemma 1.
[38] Let E be a Banach space and let $f : E → ( − ∞ , ∞ ]$ be a proper strictly convex function so that it is G$a ^$teaux differentiable and ${ a n }$ is a sequence in E such that $a n ⇀ u$ for some $u ∈ E .$ Then
$lim sup n → ∞ Δ f ( u , a n ) < lim sup n → ∞ Δ f ( v , a n ) ,$
for all v in the interior of domf with $u ≠ v$.
The following result is well-known; see, e.g., [30,32].
Lemma 2.
Let ${ a n }$ and ${ b n }$ be two non-negative real sequences such that
$a n + 1 ≤ a n − b n .$
Then ${ a n }$ is bounded and $∑ n = 0 ∞ b n < ∞ .$
Definition 4.
[39,40] The Minty Variational Inequalities (MVI) are defined as finding a point $x ¯ ∈ C$ such that
$〈 A y , y − x ¯ 〉 ≥ 0 , ∀ y ∈ C .$
We denote by $M ( C , A )$, the set of solution of (11). Some existence results for the MVIP have been presented in [39]. The assumption that $M ( C , A ) ≠ ∅$ has already been used for solving $V I ( C , A )$ in finite dimensional spaces (see e.g., [41]).
Lemma 3.
[40] Consider the VI (1). If the mapping $h : [ 0 , 1 ] → E *$ defined as $h ( t ) = A ( t x + ( 1 − t ) y )$ is continuous for all $x , y ∈ C$ (i.e., h is hemicontinuous), then $M ( C , A ) ⊂ V I ( C , A )$. Moreover, if A is pseudomonotone, then $V I ( C , A )$ is closed, convex and $V I ( C , A ) = M ( C , A ) .$

## 3. Main Results

In this section, we present our algorithms and their convergence analysis. In the sequel, we assume that the following assumptions hold.
Assumption 1.
(A1) The feasible set $C ⊆ H$ is a nonempty, closed and convex set.
(A2) The mapping $A : C → H$ is pseudomonotone, L-Lipschitz continuous and weakly sequentially continuous on H.
(A3) The solution set $S o l : = Ω V I$ is nonempty.
(A4) The function $f : H → R ∪ { + ∞ }$ uniformly Fréchet differentiable, Legendre, strongly coercive and strongly convex with a strong convexity modulus $σ > 0 .$

#### 3.1. Bregman Projection Method with Fixed Stepsize

We present our first algorithm as follows:
Algorithm BPMFS
Initialization: Pick $a 0 , b 0 ∈ H$ arbitrarily and let $λ > 0 .$ Set $n = 0 .$
Step 1: Compute
$a 1 = Π C ( ▽ f * ( ∇ f ( a 0 ) − λ A b 0 ) ) , b 1 = Π C ( ∇ f * ( ∇ f ( a 1 ) − λ A b 0 ) ) .$
Step 2: Given the current iterates $a n , b n$ and $b n − 1 ,$ calculate $a n + 1$ and $b n + 1$ as follows:
$a n + 1 = Π T n ( ∇ f * ( ∇ f ( a n ) − λ A b n ) ) ,$
$b n + 1 = Π C ( ∇ f * ( ∇ f ( a n + 1 ) − λ A b n ) ) ,$
where
$T n : = { z ∈ H : 〈 ∇ f ( a n ) − λ A b n − 1 − ∇ f ( b n ) , z − b n 〉 ≤ 0 } .$
If $a n + 1 = a n$ and $b n = b n − 1$ or $A b n = 0 :$ STOP. Otherwise, set $n ← n + 1$ and repeat Step 2.
Remark 2.
The projection for $a n + 1$ in Step 2 can be calculated explicitly using the projection formula in [26]. Algorithm BPMFS is constructed based on the concepts of [28,29] using Bregman projections. As in [29], the main task of Algorithm BPMFS is to compute one projection for $b n + 1$ onto C and one evaluation of A at the current approximation $b n .$ More so, it is easy to see from (12) that $C ⊂ T n .$ Indeed from (12), we have that
$b n = Π C ( ∇ f * ( ∇ f ( a n ) − λ A b n − 1 ) ) ,$
hence, using (9), we obtain
$〈 ∇ f ( a n ) − λ A b n − 1 − ∇ f ( b n ) , ω − b n 〉 ≤ 0 ∀ ω ∈ C .$
It follows from (14) that $C ⊂ T n .$
Remark 3.
Moreover, if for some $n ≥ 0 ,$$a n + 1 = a n$ and $b n = b n − 1$ in Algorithm BPMFS. Then $b n$ is a solution of the VI (1).
Proof.
From $a n + 1 = a n ,$ this implies that
$〈 ∇ f ( a n + 1 ) − ∇ f ( a n ) + λ A b n , z − a n + 1 〉 ≥ 0 ∀ z ∈ T n .$
Hence
$λ 〈 A b n , z − a n 〉 ≥ 0 ∀ z ∈ T n .$
Equivalently
$〈 A b n , z − b n 〉 ≥ 〈 A b n , a n − b n 〉 ∀ z ∈ T n .$
Since $C ⊂ T n ,$ we get
$〈 A b n , z − b n 〉 ≥ 〈 A b n , a n − b n 〉 ∀ z ∈ C .$
On the other hand, it follows from the definition of $T n$ that
$〈 ∇ f ( a n ) − λ A b n − 1 − ∇ f ( b n ) , z − b n 〉 ≤ 0 ∀ z ∈ T n .$
Substituting $b n − 1 = b n$ and $z = a n$ into the above inequality, we get
$〈 ∇ f ( a n ) − λ A b n − ∇ f ( b n ) , a n − b n 〉 ≤ 0 .$
Then we have
$λ 〈 A b n , a n − b n 〉 ≥ 0 .$
Combining (15) and (16), we get
$〈 A b n , z − b n 〉 ≥ 0 ∀ z ∈ C .$
Therefore $b n$ is a solution of the VI (1). □
Lemma 4.
Let ${ a n }$ and ${ b n }$ be the sequences generated from Algorithm BPMFS. Then the following inequality holds:
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − 1 − λ L 2 σ Δ f ( a n + 1 , b n ) − 1 − λ L σ ( 1 + 2 ) Δ f ( b n , a n ) + λ L σ Δ f ( a n , b n − 1 ) ,$
for any $z ∈ S o l .$
Proof.
Since $z ∈ S o l ⊂ C ⊂ T n$ and $a n + 1 = Π T n ( ∇ f * ( ∇ f ( a n ) − λ A b n ) ) ,$ we have from (9)
$〈 ∇ f ( a n + 1 ) − ( ∇ f ( a n ) − λ A b n ) , z − a n + 1 〉 ≥ 0 .$
This implies that
$〈 ∇ f ( a n + 1 ) − ∇ f ( a n ) + λ A b n , z − a n + 1 〉 ≥ 0 ,$
thus we get
$λ 〈 A b n , z − a n + 1 〉 ≥ 〈 ∇ f ( a n + 1 ) − ∇ f ( a n ) , a n + 1 − z 〉 .$
By the three point identity (7), we have
$〈 ∇ f ( a n + 1 ) − ∇ f ( a n ) , a n + 1 − z 〉 = Δ f ( z , a n ) − Δ f ( z , a n + 1 ) − Δ f ( a n + 1 , a n ) .$
This implies that
$λ 〈 A b n , z − a n + 1 〉 ≥ Δ f ( z , a n ) − Δ f ( z , a n + 1 ) − Δ f ( a n + 1 , a n ) .$
Thus
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , a n ) + λ 〈 A b n , z − a n + 1 〉 .$
Since A is pseudomonotone and $b n ∈ C ,$ it follows that $〈 A b n , z − b n 〉 ≤ 0 .$ Then this implies that
$〈 A b n , z − a n + 1 〉 = 〈 A b n , b n − a n + 1 〉 + 〈 A b n , z − b n 〉 ≤ 〈 A b n , b n − a n + 1 〉 .$
Combining (19) and (20), we get
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , a n ) + λ 〈 A b n , b n − a n + 1 〉 + λ 〈 A b n , z − b n 〉 ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , a n ) + λ 〈 A b n , b n − a n + 1 〉 = Δ f ( z , a n ) − Δ f ( a n + 1 , a n ) + λ 〈 A b n − 1 , b n − a n + 1 〉 + λ 〈 A b n − A b n − 1 , b n − a n + 1 〉 .$
However, we have
$λ 〈 A b n − 1 , b n − a n + 1 〉 = 〈 ∇ f ( a n ) − λ A b n − 1 − ∇ f ( b n ) , a n + 1 − b n 〉 + 〈 ∇ f ( b n ) − ∇ f ( a n ) , a n + 1 − b n 〉 .$
Since $a n + 1 ∈ T n$ and by the definition of $T n ,$ we get
$λ 〈 A b n − 1 , b n − a n + 1 〉 ≤ 〈 ∇ f ( b n ) − ∇ f ( a n ) , a n + 1 − b n 〉 .$
Using the three points identity (7) in (22), we obtain
$λ 〈 A b n − 1 , b n − a n + 1 〉 ≤ Δ f ( a n + 1 , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) .$
Combining (21) and (23), we get
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) + λ 〈 A b n − A b n − 1 , b n − a n + 1 〉 .$
Note that
$λ 〈 A b n − 1 − A b n , a n + 1 − b n 〉 ≤ λ ∥ A b n − 1 − A b n ∥ ∥ a n + 1 − b n ∥ ≤ λ L ∥ b n − 1 − b n ∥ ∥ a n + 1 − b n ∥ ≤ λ L 1 2 2 ∥ b n − 1 − b n ∥ 2 + 1 2 ∥ a n + 1 − b n ∥ 2 ≤ λ L 2 2 ( ( 2 + 2 ) ∥ a n − b n ∥ 2 + 2 ∥ a n − b n − 1 ∥ 2 ) + λ L 2 ∥ a n + 1 − b n ∥ 2 ≤ λ L σ ( 1 + 2 ) Δ f ( b n , a n ) + λ L σ Δ f ( a n , b n − 1 ) + λ L 2 σ Δ f ( a n + 1 , b n ) ,$
where in (25), we have used the following basic inequalities with (8):
$a b ≤ ϵ 2 2 a 2 + 1 2 ϵ 2 b 2 and ( a + b ) 2 ≤ 2 a 2 + ( 2 + 2 ) b 2 .$
It follows from (24) and (25) that
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − ( 1 − λ L 2 σ ) Δ f ( a n + 1 , b n ) − ( 1 − λ L σ ( 1 + 2 ) ) Δ f ( b n , a n ) + λ L σ Δ f ( a n , b n − 1 ) .$
Hence, we obtained the desired result. □
Theorem 1.
Suppose that Assumption A1 holds, and let $λ ∈ ( 0 , ( 2 − 1 ) σ L ) .$ Then, the sequences ${ a n }$ and ${ b n }$ generated by Algorithm BPMFS converges weakly to a solution of the VI (1).
Proof.
Let $z ∈ S o l .$ First, we show that the sequence ${ a n }$ is bounded. From Lemma 4, we obtain
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) + λ L σ Δ f ( a n , b n − 1 ) .$
Now let $Γ n = Δ f ( z , a n ) + λ L σ Δ f ( a n , b n − 1 )$ and $Λ n = ( 1 − λ L ( 1 + 2 ) σ ) ( Δ f ( b n , a n ) + Δ f ( a n + 1 , b n ) ) .$ Note that (17) can be re-written as
$a n + 1 ≤ Γ n − Λ n .$
Hence, it follows from Lemma 2 that ${ a n }$ is bounded and $lim n → ∞ Δ f ( b n , a n ) = 0 .$ Consequently, ${ a n }$ is bounded and $∥ a n − b n ∥ → 0 ,$$∥ a n + 1 − b n ∥ → 0$ as $n → ∞ .$ This implies that $∥ a n + 1 − a n ∥ → 0$ as $n → ∞ .$ Furthermore
$∥ b n + 1 − b n ∥ ≤ ∥ b n + 1 − a n + 1 ∥ + ∥ a n + 1 − a n ∥ + ∥ a n − b n ∥ → 0 , as n → ∞ .$
Consequent
$lim n → ∞ ∥ a n + 1 − b n + 1 ∥ ≤ lim n → ∞ ( ∥ a n + 1 − b n ∥ + ∥ b n + 1 − b n ∥ ) = 0 .$
Since f is uniformly Fréchet differentiable, the $∇ f$ is norm-to-norm uniformly continuous on bounded subsets of E. Thus, we have
$lim n → ∞ ∥ ∇ f ( a n + 1 ) − ∇ f ( b n + 1 ) ∥ = 0 .$
Since ${ a n }$ is bounded, then we can choose a subsequence ${ a n k }$ of ${ a n }$ such that $a n k ⇀ p .$ Since $∥ a n − b n ∥ → 0 ,$ then $b n k ⇀ p .$ Since ${ b n } ⊂ C ,$ thus $p ∈ C .$ We now show that $p ∈ S o l .$ Since $b n + 1 = Π C ( ∇ f * ( ∇ f ( a n + 1 ) − λ A b n ) ) ,$ we have
$〈 ∇ f ( a n + 1 ) − λ A b n − ∇ f ( b n + 1 ) , v − b n + 1 〉 ≤ 0 , ∀ v ∈ C .$
Equivalently
$〈 ∇ f ( a n + 1 ) − ∇ f ( b n + 1 ) , v − b n + 1 〉 ≤ 〈 λ A b n , v − b n + 1 〉 ∀ v ∈ C .$
This implies that
$〈 ∇ f ( a n + 1 ) − ∇ f ( b n + 1 ) λ , v − b n + 1 〉 + 〈 A b n , b n + 1 − b n 〉 ≤ 〈 A b n , v − b n 〉 ∀ v ∈ C .$
Hence, we have
$〈 ∇ f ( a n k + 1 ) − ∇ f ( b n k + 1 ) λ , v − b n k + 1 〉 + 〈 A b n k , b n k + 1 − b n k 〉 ≤ 〈 A b n k , v − b n k 〉 ∀ v ∈ C .$
Taking limits of the above inequality as $k → ∞ ,$ we get
$lim inf k → ∞ 〈 A b n k , v − b n k 〉 ≥ 0 ∀ v ∈ C .$
We choose a sequence ${ ϵ k }$ of positive numbers such that ${ ϵ k }$ is decreasing and convergence to $0 .$ For each $k ≥ 1 ,$ we denote by $N k$ the smallest positive integer such that
$〈 A b n k , v − b n k 〉 + ϵ k ≥ 0 ∀ k ≥ N k .$
Since ${ ϵ k }$ is decreasing, it is easy to see that the sequence ${ N k }$ is non-decreasing. Thus, for each $k ≥ 1 ,$ since ${ b n } ⊂ C ,$ we have $A b N k ≠ 0$ and by setting
$ϑ N k = A b N k ∥ A b N k ∥ 2 ,$
we have $〈 A b N k , ϑ N k 〉 = 1$ for each $k ≥ 1 .$ Now, it follows from (29) that for each $k ≥ 1 ,$
$〈 A b N k , x + ϵ k ϑ N k − b N k 〉 ≥ 0 .$
Since A is pseudomonotone, we get
$〈 A ( v + ϵ k ϑ N k ) , v + ϵ k ϑ N k − b N k 〉 ≥ 0 .$
This implies that
$〈 A x , v − b N k 〉 ≥ 〈 A v − A ( x + ϵ k ϑ N k ) , v + ϵ k ϑ N k − b N k 〉 − ϵ k 〈 A x , ϑ N k 〉 .$
Now we show that $lim k → ∞ ϵ k ϑ N k = 0 .$ Indeed, since $a n k ⇀ p$ and $lim k → ∞ ∥ a n k − b n k ∥ = 0 ,$ we obtain that $b N k ⇀ p$ as $k → ∞ .$ Since ${ b n } ⊂ C$ and A is weakly sequentially continuous on H, it follows that ${ A b N k }$ converges weakly to $A p .$ We can suppose that $A p ≠ 0$ (otherwise, p is a solution). Since the norm is sequentially weakly lower semicontinuous, we have
$0 > ∥ A p ∥ ≤ lim inf k → ∞ ∥ A b n k ∥ .$
Since ${ b N k } ⊂ { b n k }$ and $ϵ k → 0$ as $k → ∞ ,$ we get
$0 ≤ lim sup k → ∞ ∥ ϵ k ϑ N k ∥ = lim sup k → ∞ ϵ k ∥ A b n k ∥ ≤ lim sup k → ∞ ϵ k lim inf k → ∞ ∥ A b n k ∥ = 0 ,$
which implies that $lim k → ∞ ϵ k ϑ N k = 0 .$ Thus, we get
$lim inf k → ∞ 〈 A x , v − b N k 〉 ≥ 0 .$
Hence for all $v ∈ C ,$ we get
$〈 A x , v − p 〉 = lim k → ∞ 〈 A x , x − b N k 〉 = lim inf k → ∞ 〈 A x , v − b N k 〉 ≥ 0 .$
Therefore by Lemma 3, $p ∈ S o l = Ω V I .$
Finally, we show that p is unique. Assume the contrary, i.e., there exists a subsequence ${ a n j }$ of ${ a n }$ such that $a n j ⇀ q$ with $q ≠ p .$ Following similar argument has above, we get $q ∈ Ω V I .$ It follows from the Bregman opial-like property of H (more precisely, Lemma 1) that
$lim n → ∞ Δ f ( p , a n ) = lim k → ∞ Δ f ( p , a n k ) > lim k → ∞ Δ f ( q , a n k ) = lim n → ∞ Δ f ( q , a n ) = lim j → ∞ Δ f ( q , a n j ) > lim j → ∞ Δ f ( p , a n j ) = lim n → ∞ Δ f ( p , a n ) ,$
which is a contradiction. Thus, we have $p = q$ and the desired result follows. This completes the proof. □
Remark 4.
We note that Algorithm BPMFS converges strongly to a solution of the VI (1) if the operator $A : E → E *$ is γ-strongly pseudomonotone and L-Lipschitz continuous.
Proof.
Indeed, let $p ∈ S o l .$ Following similar argument in the proof of Lemma 4 till (19), we have
$Δ f ( p , a n + 1 ) ≤ Δ f ( p , a n ) − Δ f ( a n + 1 , a n ) + λ 〈 A b n , p − a n + 1 〉 .$
Since p is a solution of the VI (1), we have $〈 A p , v − p 〉 ≥ 0$ for all $v ∈ C .$ Using the strong pseudomonotonicity of A, we obtain $〈 A x , v − p 〉 ≥ γ ∥ x − p ∥ 2$ for all $x ∈ C .$ Taking $v = b n ∈ C ,$ we get
$〈 A b n , p − b n 〉 ≤ − γ ∥ b n − p ∥ 2 .$
Hence, we have
$〈 A b n , p − a n + 1 〉 = 〈 A b n , p − b n 〉 + 〈 A b n , b n − a n + 1 〉 ≤ − γ ∥ b n − p ∥ 2 + 〈 A b n , b n − a n + 1 〉 .$
Therefore (29) and (30), we get
$Δ f ( p , a n + 1 ) ≤ Δ f ( p , a n ) − Δ f ( a n + 1 , a n ) + λ 〈 A b n , b n − a n + 1 〉 − γ λ ∥ b n − p ∥ 2 .$
Using (7), we have
$Δ f ( p , a n + 1 ) ≤ Δ f ( p , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) − 〈 ∇ f ( b n ) − ∇ f ( a n ) , a n + 1 − b n 〉 + λ 〈 A b n , b n − a n + 1 〉 − γ λ ∥ b n − p ∥ 2 = Δ f ( p , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) + 〈 ∇ f ( a n ) − λ A b n − ∇ f ( b n ) , a n + 1 − b n 〉 − γ λ ∥ b n − p ∥ 2 .$
Note that
$〈 ∇ f ( a n ) − λ A b n − ∇ f ( b n ) , a n + 1 − b n 〉 = 〈 ∇ f ( a n ) − λ A b n − 1 − ∇ f ( b n ) , a n + 1 − b n 〉 + λ 〈 A b n − 1 − A b n , a n + 1 − b n 〉 .$
Since $a n + 1 ∈ T n ,$ then we have
$〈 ∇ f ( a n ) − λ A b n − 1 − ∇ f ( b n ) , a n + 1 − b n 〉 ≤ 0 .$
Hence, we have
$〈 ∇ f ( a n ) − λ A b n − ∇ f ( b n ) , a n + 1 − b n 〉 ≤ λ 〈 A b n − 1 − A b n , a n + 1 − b n 〉 ≤ λ L ∥ b n − 1 − b n ∥ ∥ a n + 1 − b n ∥ ≤ λ L { 1 2 2 ∥ b n − 1 − b n ∥ 2 + 1 2 ∥ a n + 1 − b n ∥ 2 } ≤ λ L 2 2 ( ( 2 + 2 ) ∥ a n − b n ∥ 2 + 2 ∥ a n − b n − 1 ∥ 2 ) + λ L 2 ∥ a n + 1 − b n ∥ 2 ≤ λ L σ ( 1 + 2 ) Δ f ( b n , a n ) + λ L σ Δ f ( a n , b n − 1 ) + λ L 2 σ Δ f ( a n + 1 , b n ) .$
Combining (31) and (32), we get
$Δ f ( p , a n + 1 ) ≤ Δ f ( p , a n ) − 1 − λ L 2 σ Δ f ( a n + 1 , b n ) − 1 − λ L σ ( 1 + 2 ) Δ f ( b n , a n ) + λ L σ Δ f ( a n , b n − 1 ) − γ λ ∥ b n − p ∥ 2 .$
Following similar argument as in the Proof of Theorem 1, we have that the sequence ${ a n }$ is bounded and $∥ b n − a n ∥ → 0 ,$$∥ a n + 1 − b n ∥ → 0$ as $n → ∞ .$$∥ a n + 1 − a n ∥ → 0$ as $n → ∞ .$ Since f is continuous on bounded sets and $∇ f$ is weakly–weakly continuous, we get $∥ f ( a n + 1 ) − f ( a n ) ∥ → 0$ and $∥ ∇ f ( a n + 1 ) − ∇ f ( a n ) ∥ → 0$ as $n → ∞ .$ Therefore
$Δ f ( p , a n ) − Δ f ( p , a n + 1 ) = f ( p ) − f ( a n ) − 〈 ∇ f ( a n ) , p − a n 〉 − f ( p ) + f ( a n + 1 ) + 〈 ∇ f ( a n + 1 ) , p − a n + 1 〉 = f ( a n + 1 ) − f ( a n ) + 〈 ∇ f ( a n + 1 ) , p − a n + 1 〉 − 〈 ∇ f ( a n ) , p − a n 〉 = f ( a n + 1 ) − f ( a n ) + 〈 ∇ f ( a n + 1 ) − ∇ f ( a n ) , p − a n 〉 + 〈 ∇ f ( a n + 1 ) , a n − a n + 1 〉 .$
Thus, we have that
$lim n → ∞ ( Δ f ( p , a n ) − Δ f ( p , a n + 1 ) ) = 0 .$
Note that
$Δ f ( a n , b n − 1 ) = f ( a n ) − f ( b n − 1 ) − 〈 ∇ f ( b n − 1 ) , a n − b n − 1 〉 → 0 .$
It follows from (34) that
$γ λ ∥ b n − p ∥ 2 ≤ Δ f ( p , a n ) − Δ f ( p , a n + 1 ) + λ L σ Δ f ( a n , b n − 1 ) → 0 .$
Therefore, we get
$lim n → ∞ ∥ b n − p ∥ = 0 .$
Since $∥ a n − b n ∥ → 0 ,$ then $∥ a n − p ∥ → 0$ as $n → ∞ .$ This completes the proof. □

#### 3.2. Bregman Projection Method with Self-Adaptive Stepsize

Next, we propose a Bregman subgradient extragradient method whose convergence does not require a prior estimate of the Lipschitz constant of the cost operator A. The importance of the second algorithm is to cater for the situations when estimating the Lipschitz constant of A is very difficult. So, the algorithm uses a self-adaptive process for selecting its stepsize and does not require a prior estimate of the Lipschitz constant of A.
Algorithm BPMSAS
Initialization: Pick $a 0 , b 0 ∈ H , λ 0 > 0$ arbitrarily and $α ∈ ( 0 , 1 ) .$
Step 1: Compute
$a 1 = Π C ( ∇ f * ( ∇ f ( a 0 ) − λ n A b 0 ) ) , b 1 = Π C ( ∇ f * ( ∇ f ( a 1 ) − λ n A b 0 ) ) .$
Step 2: Given the current iterates $a n , b n$ and $b n − 1 ,$ and $λ n$, calculate $a n + 1$ and $b n + 1$ as follows:
$a n + 1 = Π T n ( ∇ f * ( ∇ f ( a n ) − λ n A b n ) ) , b n + 1 = Π C ( ∇ f * ( ∇ f ( a n + 1 ) − λ n + 1 A b n ) ) ,$
where
$λ n + 1 = { min { λ n , α ∥ b n − b n − 1 ∥ ∥ A b n − A b n − 1 ∥ } , A b n = A b n − 1 , λ n , otherwise ,$
and
$T n : = { z ∈ H : 〈 ∇ f ( a n ) − λ n A b n − 1 − ∇ f ( b n ) , z − b n 〉 ≤ 0 } .$
If $a n + 1 = a n$ and $b n = b n − 1$ or $A b n = 0 :$ STOP. Otherwise, set $n ← n + 1$ and repeat Step 2.
Remark 5.
Note that in Algorithm BPMSAS, the stepsize $λ n$ is chosen by a self-adaptive process. This means that the stepsize is improved along with the iterations. Moreso, the sequence ${ λ n }$ is monotonically non-increasing and bounded below by $min { λ 0 , α L } .$ Then, we can say that the limit $lim n → ∞ λ n$ exist, which we denote by λ, i.e., $lim n → ∞ λ n = λ > 0 .$
We now give the convergence of Algorithm BPMSAS.
Theorem 2.
Suppose that Assumption A1 holds, and let $α ∈ 0 , ( 2 − 1 ) σ .$ Then, the sequences ${ a n }$ and ${ b n }$ generated by Algorithm BPMSAS converges weakly to a solution of the VI (1).
Proof.
Let $z ∈ V I ( C , A ) .$ Following similar argument as in the proof of (24), we obtain
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) + λ n 〈 A b n − A b n − 1 , b n − a n + 1 〉 .$
Now, using Cauchy-Schwartz inequality and the defintion of $λ n + 1$ in (36), we get
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) + λ n λ n + 1 λ n + 1 ∥ A b n − A b n − 1 ∥ ∥ b n − a n + 1 ∥ ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n )$
$+ λ n 2 λ n + 1 α 1 2 ∥ b n − 1 − b n ∥ 2 + 2 ∥ b n − a n + 1 ∥ 2 ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n )$
$+ λ n α 2 2 λ n + 1 2 ∥ b n − 1 − a n ∥ 2 + ( 2 + 2 ∥ a n − b n ∥ 2 ) + λ n α 2 λ n + 1 2 ∥ b n − a n + 1 ∥ 2 = Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) + λ n α 2 λ n + 1 ∥ b n − 1 − a n ∥ 2 + ( 2 + 1 ) λ n α 2 λ n + 1 ∥ a n − b n ∥ 2 + λ n α 2 2 λ n + 1 ∥ b n − a n + 1 ∥ 2 ,$
where in (37) and (38), we have used the identities: $2 a b ≤ 1 2 a 2 + 2 b 2$ and $( a + b ) 2 ≤ 2 a 2 + ( 2 + 2 ) b 2$ respectively. Since f is strongly convex and $σ > 0$, it follows from (8) that
$Δ f ( z , a n + 1 ) ≤ Δ f ( z , a n ) − Δ f ( a n + 1 , b n ) − Δ f ( b n , a n ) + λ n α σ λ n + 1 Δ f ( a n , b n − 1 ) + ( 2 + 1 ) λ n α σ λ n + 1 Δ f ( b n , a n ) + λ n α 2 σ λ n + 1 Δ f ( a n + 1 , b n ) = Δ f ( z , a n ) − 1 − λ n α 2 σ λ n + 1 Δ f ( a n + 1 , b n ) − 1 − ( 2 + 1 ) λ n α σ λ n + 1 Δ f ( b n , a n ) + λ n α σ λ n + 1 Δ f ( a n , b n − 1 ) .$
Now put
$Γ n = Δ f ( z , a n ) + α λ n σ λ n + 1 Δ f ( a n , b n − 1 ) ,$
and
$Λ n = 1 − ( 2 + 1 ) α λ n σ λ n + 1 ( Δ f ( b n , a n ) + Δ f ( a n + 1 , b n ) ) .$
Then (39) can be rewritten as
$a n + 1 ≤ Γ n − Λ n − Ω n ,$
where $Ω n = α σ λ n λ n + 1 + λ n + 1 λ n + 2 Δ f ( a n + 1 , b n ) .$ It follows from Lemma 2 that ${ a n }$ is bounded. Note that since $lim n → ∞ λ n = λ > 0$ and $α ∈ ( 0 , ( 2 − 1 ) σ ) ,$ there exists a number $n 0 ≥ 0$ such that
$lim n → ∞ 1 − λ n α 2 σ λ n + 1 = 1 − α 2 σ > 0 ,$
$lim n → ∞ 1 − λ n α ( 2 + 1 ) σ λ n + 1 = 1 − ( 2 + 1 ) α σ > 0 .$
This implies that there exists $ϵ > 0$ such that
$( 1 − λ n α ( 2 + 1 ) σ λ n + 1 ) ≥ ϵ > 0 , ∀ n ≥ n 0 .$
Hence from Lemma 2, we obtain
$∑ n = 0 ∞ ϵ ( Δ f ( b n , a n ) + Δ f ( a n + 1 , b n ) ) < ∞ ,$
which implies that
$lim n → ∞ Δ f ( b n , a n ) = lim n → ∞ Δ f ( a n + 1 , b n ) = 0 .$
Consequently
$lim n → ∞ ∥ a n − b n ∥ = lim n → ∞ ∥ a n + 1 − b n ∥ = 0 .$
Hence
$lim n → ∞ ∥ a n + 1 − a n ∥ = lim n → ∞ ∥ b n + 1 − b n ∥ = lim n → ∞ ∥ a n + 1 − b n + 1 ∥ = 0 .$
Since f is uniformly Fréchet differentiable, then $∇ f$ is norm-to-norm continuous and therefore
$lim n → ∞ ∥ ∇ f ( a n + 1 ) − ∇ f ( b n + 1 ) ∥ = 0 .$
Since ${ a n }$ is bounded, then there exists a subsequence ${ a n k }$ of ${ a n }$ such that $a n k ⇀ x ¯ .$ Following similar argument from (27) of Theorem 1, we get that $x ¯ ∈ V I ( C , A ) .$ This completes the proof. □

## 4. Numerical Examples

In this section, we test the performance of Algorithm BPMFS and Algorithm BPMSAS for solving some VI available in the literature. All numerical computation are carried out using MATLAB program on a PC (with Intel(R) Core (TM) i7-5600U CPU @ 3.40GHz, RAM 8.00GB) in MATLAB 9.7 2019(a).
Example 2.
First, we consider the variational inequality problem for a finite dimensional space $R m .$ The cost operator $A : R m → R m$ is defined by $A x = G x + q$, such that
$G = S S T + Q + R ,$
where S is a $m × m$ matrix, Q is a $m × m$ skew-symmetric matrix, R is a $m × m$ diagonal matrix, whose diagonal entries are positive (so, G is positive definite) and q is a vector in $R m .$ The feasible set C is the closed convex polyhedron which is defined by $C = { x ∈ R m : B x ≤ b }$, where B is a $l × m$ ($l = 10$) matrix and b is a non-negative vector in $R m .$ It is easy to see that A is monotone (thus, pseudomonotone) and L-Lipschitz continuous with $L = ∥ G ∥ .$ For experimental purpose, all the entries of $S , Q , R , B$ and b are generated randomly such that their properties are satisfied and the starting points $a 0 , b 0$ are generated randomly in $R m$ ($m = 50 , 100 , 150 , 200$). The Bregman projection onto the feasible set C is easily computed using thefminconsolver in MATLAB Optimization toolbox. We set $q = 0 ∈ R m ,$ in this case, we know that the unique solution of the VI is ${ 0 } .$ We choose $f ( x ) = 1 2 ∥ x ∥ 2$ which is strongly convex with modulus 1. We test Algorithm BPMFS (named, Alg. 1), Algorithm BPMSAS (named, Alg. 2) and theSEM(Algorithm (3)) using the following parameters: for Algorithm BPMFS andSEM, we choose $λ = 0.3 L$ while for Algorithm BPMSAS, we choose $λ 0 = 0.3 .$ The computation is stopped when $D n = ∥ a n − 0 ∥ < 10 − 7$ is satisfied. We plot the graphs of $D n$ against the number of iteration in each case for each iteration. The numerical results are shown in Table 1 and Figure 1. The numerical results shows that Algorithm BPMSAS has computational advantage than Algorithm BPMFS and the SEM in terms of number of iteration and CPU time taken for computation. This is due to the fact that the stepsize is improve in each iteration compare to the Algorithm BPMFS and SEM which used fixed stepsize. More so, Algorithm BPMFS performs better than theSEM.
Example 3.
Here, we compare the performance of Algorithm BPMFS and Algorithm BPMSAS for different convex functions. Let C be defined by
$C = x = ( a 1 , … , a m ) T : ∥ x ∥ ≤ 1 and a i > 0 , i = 1 , … , m .$
Note that the corresponding values of $∇ f ( x )$ and $∇ f * ( x ) = ∇ f − 1 ( x )$ for the convex function given in Example 1 are respectively:
(i)
$∇ f ( x ) = x$ and $∇ f * ( x ) = x$ for SE,
(ii)
$∇ f ( x ) = 1 + log ( a 1 ) , … , 1 + log ( a m ) T$ and $∇ f * ( x ) = exp ( a 1 − 1 ) , … , exp ( a m − 1 ) T$ for KL,
(iii)
$∇ f ( x ) = − 1 x , … , 1 a m T$ and $∇ f * ( x ) = − 1 x , … , 1 a m T$ for IS,
(iv)
$∇ f ( x ) = Q x$ and $∇ f * ( x ) = Q − 1 ( x )$ for MD.
We test the algorithms using similar parameters as in Example 1 with $∥ a n − 0 ∥ > 10 − 4$ as stopping criterion. The numerical results are shown in Table 2. The computation result shows that the IS has the best performance while the MD has the poorest performance for both Algorithm BPMFS and Algorithm BPMSAS.
Example 4.
Next, we apply our algorithms to solve the Least Absolute Selection and Shrinkage Operator (LASSO) problem [42]. Let z be a vector in $R M$, B be a matrix of order $M × N$ ($M < < N )$ and $λ > 0 .$ The LASSO problem is given by
$min x ∈ R N 1 2 ∥ B x − z ∥ 2 2 + λ ∥ x ∥ 1 ,$
where $∥ x ∥ 2$ is the Euclidean norm of x and $∥ x ∥ 1 = ∑ i = 1 N | a i |$ is the $l 1$-norm of x. This problem has been considered as a great tool in several branches of science and engineering. Clearly, (40) is a convex unconstrained minimization problem which appears in compress sensing and image reconstruction, where the original signal (or image) is sparse in some orthogonal basis by the process
$z = B x + ξ ,$
where x is the original signal (or image), B is the blurring operator, ξ is a noise and z is the degraded or blurred data which needs to be recovered. Several iterative methods for solving (40) have been introduced with the earliest being the projection method by Figureido et al. [43]. Equivalently, the LASSO problem (40) can be expressed as a VI by setting $A = B T ( B x − z )$ (see, [44]). We apply our algorithms to solve the signal problem with $N = 4096$ which is generated by the uniform distribution on the interval $[ − 2 , 2 ]$ with 160 non-zero elements. The matrix B is generated by the normal distribution with mean zero and variance one while the observation z with $M = 512$ is generated by Gaussian noise with SNR = 40. The initial points $a 0 , y 0$ are picked randomly and in Algorithm 1, we choose $λ = 0.35 ,$ in Algorithm 2, $λ 0 = 0.02$ and in SEM, $λ = 0.35 .$ We compare the performance of each algorithm with respect to the numer of iterations, CPU time and MSE values, see Figure 2 and Figure 3.

## 5. Conclusions

In this paper, we introduce two Bregman subgradient extragradient method for solving variational inequalities with pseudomonotone and Lipschitz continuous operator in a real Hilbert space. We established some weak convergence results for the sequence generated by the algorithms. We also gives some numerical experiments to illustrate the performance of the proposed algorithms. The numerical results show that the proposed methods perform better than the usual subgradient extragradient method and their efficiency depends on the type of convex function used in the algorithms.

## Author Contributions

Conceptualization, L.O.J.; methodology, L.O.J.; validation, L.O.J., M.A., S.H.K.; formal analysis, L.O.J., S.H.K.; writing—original draft preparation, L.O.J.; writing—review and editing, M.A., S.H.K.; visualization, L.O.J.; supervision, L.O.J., M.A., S.H.K.; project administration, M.A.; funding acquisition, M.A.; All authors have read and agreed to the published version of the manuscript.

## Funding

This research was funded by Sefako Makgatho Health Sciences University Postdoctoral research fund and and the APC was funded by Department of Mathematics and Applied Mathematics, Sefako Makgatho Health Sciences University, Pretoria, South Africa.

## Acknowledgments

The authors acknowledge with thanks, the Department of Mathematics and Applied Mathematics at the Sefako Makgatho Health Sciences University for making their facilities available for the research.

## Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

## References

1. Glowinski, R.; Lions, J.L.; Trémolixexres, R. Numerical Analysis of Variational Inequalities; North-Holland: Amsterdam, The Netherlands, 1981. [Google Scholar]
2. Kinderlehrer, D.; Stampacchia, G. An Introduction to Variational Inequalities and Their Applications; Academic Press: New York, NY, USA, 1980. [Google Scholar]
3. Combettes, P.L.; Pesquet, J.-C. Deep neural structures solving variational inequalities. Set-Valued Var. Anal. 2020, 28, 491–518. [Google Scholar] [CrossRef] [Green Version]
4. Juditsky, A.; Nemirovski, A. Solving variational inequalities with monotone operators in domains given by linear minimization oracles. arXiv 2016, arXiv:1312.1073v2. [Google Scholar] [CrossRef]
5. Luo, M.J.; Zhang, Y. Robust solution sto box-contrained stochastic linear variational inequality problem. J. Ineq. Appl. 2017, 2017, 253. [Google Scholar] [CrossRef] [Green Version]
6. Abass, H.A.; Aremu, K.O.; Jolaoso, L.O.; Mewomo, O.T. An inertial forward-backward splitting algorithm for approximating solutions of certain optimization problems. J. Nonlinear Funct. Anal. 2020, 2020, 6. [Google Scholar]
7. Alakoya, T.O.; Jolaoso, L.O.; Mewomo, O.T. Modified Inertial Subgradient Extragradient Method with Self-Adaptive Stepsize for Solving Monotone Variational Inequality and Fixed Point Problems. Available online: https://www.tandfonline.com/doi/abs/10.1080/02331934.2020.1723586. (accessed on 1 November 2020).
8. Cruz, J.Y.B.; Iusem, A.N. A strongly convergent direct method for monotone variational inequalities in Hilbert spaces. Numer. Funct. Anal. Optim. 2009, 30, 23–36. [Google Scholar] [CrossRef]
9. Cai, G.; Gibali, A.; Iyiola, O.S.; Shehu, Y. A new double-projection method for solving variational inequalities in Banach space. J. Optim Theory Appl. 2018, 178, 219–239. [Google Scholar] [CrossRef]
10. Cegielski, A. Iterative Methods for Fixed Point Problems in Hilbert Spaces; Lecture Notes in Mathematics; Springer: Heidelberg, Germany, 2012; Volume 2057. [Google Scholar]
11. Ceng, L.C.; Teboulle, M.; Yao, J.C. Weak convergence of an iterative method for pseudomonotone variational inequalities and fixed-point problems. J. Optim. Theory Appl. 2010, 146, 19–31. [Google Scholar] [CrossRef]
12. Censor, Y.; Gibali, A.; Reich, S. Algorithms for the split variational inequality problem. Numer. Algorithms 2012, 59, 301–323. [Google Scholar] [CrossRef]
13. Censor, Y.; Gibali, A.; Reich, S. Extensions of Korpelevich’s extragradient method for variational inequality problems in Euclidean space. Optimization 2012, 61, 119–1132. [Google Scholar] [CrossRef]
14. Censor, Y.; Gibali, A.; Reich, S. The subgradient extragradient method for solving variational inequalities in Hilbert spaces. J. Optim. Theory Appl. 2011, 148, 318–335. [Google Scholar] [CrossRef] [Green Version]
15. Censor, Y.; Gibali, A.; Reich, S. Strong convergence of subgradient extragradient methods for the variational inequality problem in Hilbert space. Optim. Methods Softw. 2011, 26, 827–845. [Google Scholar] [CrossRef]
16. Chidume, C.E.; Nnakwe, M.O. Convergence theorems of subgradient extragradient algorithm for solving variational inequalities and a convex feasibility problem. Fixed Theory Appl. 2018, 2018, 16. [Google Scholar] [CrossRef]
17. Iusem, A.N. An iterative algorithm for the variational inequality problem. Comput. Appl. Math. 1994, 13, 103–114. [Google Scholar]
18. Jolaoso, L.O.; Alakoya, T.O.; Taiwo, A.; Mewomo, O.T. An inertial extragradient method via viscosity approximation approach for solving equilibrium problem in Hilbert spaces. Optimization 2020. [Google Scholar] [CrossRef]
19. Jolaoso, L.O.; Aphane, M. Weak and strong convergence Bregman extragradient schemes for solving pseudo-monotone and non-Lipschitz variational inequalities. J. Inequalities Appl. 2020, 2020, 195. [Google Scholar] [CrossRef]
20. Jolaoso, L.O.; Taiwo, A.; Alakoya, T.O.; Mewomo, O.T. A unified algorithm for solving variational inequality and fixed point problems with application to split equality problems. Comput. Appl. Math. 2020, 39, 38. [Google Scholar] [CrossRef]
21. Kanzow, C.; Shehu, Y. Strong convergence of a double projection-type method for monotone variational inequalities in Hilbert spaces. J. Fixed Point Theory Appl. 2018, 20. [Google Scholar] [CrossRef]
22. Khanh, P.D.; Vuong, P.T. Modified projection method for strongly pseudomonotone variational inequalities. J. Glob. Optim. 2014, 58, 341–350. [Google Scholar] [CrossRef]
23. Korpelevich, G.M. The extragradient method for finding saddle points and other problems. Ekon. Mat. Metody. 1976, 12, 747–756. (In Russian) [Google Scholar]
24. Apostol, R.Y.; Semenov, A.A.G.V.V. Iterative algorithms for monotone bilevel variational inequalities. J. Comput. Appl. Math. 2012, 107, 3–14. [Google Scholar]
25. Kassay, G.; Reich, S.; Sabach, S. Iterative methods for solving system of variational inequalities in reflexive Banach spaces. SIAM J. Optim. 2011, 21, 1319–1344. [Google Scholar] [CrossRef]
26. Bauschke, H.H.; Combettes, P.L. Convex Analysis and Monotone Operator Theory in Hilbert Spaces; (CMS Books in Mathematics); Springer: New York, NY, USA, 2011. [Google Scholar]
27. Halpern, B. Fixed points of nonexpanding maps. Proc. Am. Math. Soc. 1967, 73, 957–961. [Google Scholar] [CrossRef] [Green Version]
28. Popov, L.C. A modification of the Arrow-Hurwicz method for finding saddle point. Math. Notes 1980, 28, 845–848. [Google Scholar] [CrossRef]
29. Malitsky, Y.V.; Semenov, V.V. An extragradient algorithm for monotone variational inequalities. Cybern. Syst. Anal. 2014, 50, 271–277. [Google Scholar] [CrossRef]
30. Nomirovskii, D.A.; Rublyov, B.V.; Semenov, V.V. Convergence of two-step method with Bregman divergence for solving variational inequalities. Cybern. Syst. Anal. 2019, 55, 359–368. [Google Scholar] [CrossRef]
31. Bregman, L.M. The relaxation method for finding common points of convex sets and its application to the solution of problems in convex programming. USSR Comput. Math. Math. Phys. 1967, 7, 200–217. [Google Scholar] [CrossRef]
32. Gibali, A. A new Bregman projection method for solving variational inequalities in Hilbert spaces. Pure Appl. Funct. Anal. 2018, 3, 403–415. [Google Scholar]
33. Hieu, D.V.; Cholamjiak, P. Modified Extragradient Method with Bregman Distance for Variational Inequalities. Appl. Anal. 2020. Available online: https://www.tandfonline.com/doi/abs/10.1080/00036811.2020.1757078 (accessed on 1 November 2020).
34. Denisov, S.V.; Semenov, V.V.; Stetsynk, P.I. Bregman extragradient method with monotone rule of step adjustment. Cybern. Syst. Anal. 2019, 55, 377–383. [Google Scholar] [CrossRef]
35. Bauschke, H.H.; Borwein, J.M.; Combettes, P.L. Essential smoothness, essential strict convexity, and Legendre functions in Banach spaces. Commun. Contemp. Math. 2001, 3, 615–647. [Google Scholar] [CrossRef] [Green Version]
36. Beck, A. First-Order Methods in Optimization, Society for Industrial and Applied Mathematics. Philadelphia 2017. Available online: https://www.worldcat.org/title/first-order-methods-in-optimization/oclc/1002692951 (accessed on 1 November 2020).
37. Reich, S.; Sabach, S. A strong convergence theorem for proximal type- algorithm in reflexive Banach spaces. J. Nonlinear Convex Anal. 2009, 10, 471–485. [Google Scholar]
38. Huang, Y.Y.; Jeng, J.C.; Kuo, T.Y.; Hong, C.C. Fixed point and weak convergence theorems for point-dependent λ-hybrid mappings in Banach spaces. Fixed Point Theory Appl. 2011, 2011, 105. [Google Scholar] [CrossRef] [Green Version]
39. Lin, L.J.; Yang, M.F.; Ansari, Q.H.; Kassay, G. Existence results for Stampacchia and Minty type implicit variational inequalities with multivalued maps. Nonlinear Anal. Theory Methods Appl. 2005, 61, 1–19. [Google Scholar] [CrossRef]
40. Mashreghi, J.; Nasri, M. Forcing strong convergence of Korpelevich’s method in Banach spaces with its applications in game theory. Nonlinear Anal. 2010, 72, 2086–2099. [Google Scholar] [CrossRef]
41. Solodov, M.V.; Svaiter, B.F. A new projection method for variational inequality problems. SIAM J. Control Optim. 1999, 37, 765–776. [Google Scholar] [CrossRef]
42. Xiao, Y.; Zhu, H. A conjugate gradient method to solve convex constrained monotone equations with applications in compressive sensing. J. Math. Anal. Appl. 2013, 405, 310–319. [Google Scholar] [CrossRef]
43. Figueiredo, M.A.T.; Nowak, R.D.; Wright, S.J. Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 2007, 1, 586–597. [Google Scholar] [CrossRef] [Green Version]
44. Shehu, Y.; Iyiola, O.S.; Ogbuisi, F.U. Iterative method with inertial terms for nonexpansive mappings: Applications to compressed sensing. Numer. Algorithms 2020, 83, 1321–1347. [Google Scholar] [CrossRef]
Figure 1. Example 2, Top Left: $m = 50$; Top Right: $m = 100$, Bottom Left: $m = 150$; Bottom Right: $M = 200$.
Figure 1. Example 2, Top Left: $m = 50$; Top Right: $m = 100$, Bottom Left: $m = 150$; Bottom Right: $M = 200$.
Figure 2. Sparse signal recovery. From the top to the bottom is the original signal, the measurement, and the reconstructed signal using Algorithms 1 and 2 and SEM.
Figure 2. Sparse signal recovery. From the top to the bottom is the original signal, the measurement, and the reconstructed signal using Algorithms 1 and 2 and SEM.
Figure 3. Graphs of objective function value against number of iteration (Top) and objective function against CPU time (sec) (Bottom) for Algorithms 1 and 2 and SEM.
Figure 3. Graphs of objective function value against number of iteration (Top) and objective function against CPU time (sec) (Bottom) for Algorithms 1 and 2 and SEM.
Table 1. Computation result for Example 2.
Table 1. Computation result for Example 2.
Alg. 1Alg. 2SEM
$m = 50$Iter.292441
CPU time (sec)0.00500.00220.0083
$m = 100$Iter.322444
CPU time (sec)0.01650.01010.0203
$m = 150$Iter352444
CPU time (sec)0.01610.00750.0211
$m = 200$Iter.372445
CPU time (sec)0.03210.01460.0385
Table 2. Example 3 Numerical result showing number of iteration and CPU time (sec) for various types of function f using Algorithm BPMFS and Algorithm BPMSAS.
Table 2. Example 3 Numerical result showing number of iteration and CPU time (sec) for various types of function f using Algorithm BPMFS and Algorithm BPMSAS.
$m = 50$$m = 100$$M = 150$$m = 200$
Iter.CPUIter.CPUIter.CPUIter.CPU
SEAlg. 1230.0049260.0099180.0153180.0163
Alg. 2150.0021160.0039160.0109160.0138
KLAlg. 1150.0044170.0081180.0150180.00160
Alg 2.150.0040150.0058150.00109150.0143
ISAlg. 1140.0048140.0104140.0133120.0183
Alg. 270.002850.002540.004130.0057
MDAlg. 1240.0104270.0419280.0960300.1910
Alg. 2120.0094120.0254120.0423150.0698
 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Share and Cite

MDPI and ACS Style

Jolaoso, L.O.; Aphane, M.; Khan, S.H. Two Bregman Projection Methods for Solving Variational Inequality Problems in Hilbert Spaces with Applications to Signal Processing. Symmetry 2020, 12, 2007. https://doi.org/10.3390/sym12122007

AMA Style

Jolaoso LO, Aphane M, Khan SH. Two Bregman Projection Methods for Solving Variational Inequality Problems in Hilbert Spaces with Applications to Signal Processing. Symmetry. 2020; 12(12):2007. https://doi.org/10.3390/sym12122007

Chicago/Turabian Style

Jolaoso, Lateef Olakunle, Maggie Aphane, and Safeer Hussain Khan. 2020. "Two Bregman Projection Methods for Solving Variational Inequality Problems in Hilbert Spaces with Applications to Signal Processing" Symmetry 12, no. 12: 2007. https://doi.org/10.3390/sym12122007

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.