Adaptive Douglas–Rachford Algorithms for Biconvex Optimization Problem in the Finite Dimensional Real Hilbert Spaces

Abstract

In this paper, we delve into the realm of biconvex optimization problems, introducing an adaptive Douglas–Rachford algorithm and presenting related convergence theorems in the setting of finite-dimensional real Hilbert spaces. It is worth noting that our approach to proving the convergence theorem differs significantly from those in the literature.

1. Introduction

In science and engineering, convex optimization has been extensively studied and applied. For convex optimization problems, any local minimum is also a global minimum, simplifying the search for optimal solutions. In parameter estimation, particularly in the context of system identification, convexity ensures the convergence of estimates to the true parameters. However, some identification problems cannot always be formulated as convex optimization problems. For instance, the identification of block-oriented nonlinear systems often leads to a biconvex optimization problem rather than a convex one. Unlike convex optimization, biconvex optimization may have numerous local minima. Nevertheless, it exhibits convex substructures, as a biconvex optimization problem can be divided into multiple convex optimization subproblems. These substructures can be effectively utilized to solve the entire biconvex optimization problem.

From the literature [1], we observe that many optimization problems are multi-convex programming, and many published studies on practical multi-convex programming focus on special practical models, like [2,3,4,5,6,7,8,9,10,11,12,13]. In particular, Wen, Yin and Zhang [12] pointed out that multi-convex programming is an NP-hard problem.

Let $H_1$ and $H_2$ be two real Hilbert spaces, and let $f:H_1\times H_2\to \mathbb{R}\cup \{+\infty \}$ be an extended function. Then f is called a biconvex function if $x\to f(x,y)$ is convex for each $y\in H_2$, and $y\to f(x,y)$ is convex for each $x\in H_1$. Thus, every convex function is also biconvex, but a biconvex function may still be nonconvex.

The following is a type of biconvex optimization problem, also referred to as a block optimization problem.

$${\displaystyle (\mathbf{BOP}){\mathrm{argmin}}_{x\in H_1,y\in H_2}J(x,y)=f(x)+g(y)+h(x,y),}$$

where $H_1$ and $H_2$ are real Hilbert spaces, $h:H_1\times H_2\to \mathbb{R}$ is a block biconvex function, and  $f:H_1\to \mathbb{R}$ and $g:H_2\to \mathbb{R}$ are convex functions. Here, f and g are called the regularization functions of (BOP). In general, f and g could be supposed as Fréchet differentiable. It is important to note that the function $(x,y)\to f(x)+g(y)+h(x,y)$ can be nonconvex, even if f and g are convex functions.

The standard approach to solving the biconvex optimization problem is via the so-called Gauss–Seidel iteration scheme, popularized in the modern era under the name alternating minimization. Indeed, this method could be called the block coordinate descent algorithm (BCD).

$$(\mathrm{BCD})\left\{\begin{array}{c}{x}_1\in H_1\mathrm{and}{y}_1\in H_2\mathrm{are}\mathrm{given}\mathrm{arbitrarily},\hfill \\ x_{k+1}\in {\mathrm{argmin}}_{x\in H_1}f(x)+g(y_k)+h(x,y_k),\hfill \\ y_{k+1}\in {\mathrm{argmin}}_{y\in H_2}f(x_{k+1})+g(y)+h(x_{k+1},y),k\in \mathbb{N}.\hfill \end{array}\right.$$

In 1992, the proximal BCD algorithm was proposed by Auslender [14] to relax the requirements of the (BCD) convergence theorem.

$$(\mathrm{proximal}\mathrm{BCD})\left\{\begin{array}{c}\mathrm{choose}\tau >0\mathrm{and}\sigma >0,\hfill \\ x_1\in H_1\mathrm{and}{y}_1\in H_2\mathrm{are}\mathrm{given}\mathrm{arbitrarily},\hfill \\ x_{k+1}\in {\mathrm{argmin}}_{x\in H_1}f(x)+g(y_k)+h(x,y_k)+{\displaystyle \frac{1}{2\tau }}\left|\right|x-x_k{\left|\right|}^2,\hfill \\ y_{k+1}\in {\mathrm{argmin}}_{y\in H_2}f(x_{k+1})+g(y)+h(x_{k+1},y)+{\displaystyle \frac{1}{2\sigma }}\left|\right|y-y_k{\left|\right|}^2,k\in \mathbb{N}.\hfill \end{array}\right.$$

In 2013, Xu and Yin [13] gave the proximal linearized BCD.

$$\left\{\begin{array}{c}\mathrm{choose}{\tau }_k>0\mathrm{and}{\sigma }_k>0,\hfill \\ x_1\in H_1\mathrm{and}{y}_1\in H_2\mathrm{are}\mathrm{given}\mathrm{arbitrarily},\hfill \\ x_{k+1}\in {\mathrm{argmin}}_{x\in H_1}f(x)+{\displaystyle \frac{1}{2{\tau }_k}}\left|\right|x-x_k{\left|\right|}^2+⟨x,{\nabla }_{x}h(x_k,y_k)⟩,\hfill \\ y_{k+1}\in {\mathrm{argmin}}_{y\in H_2}g(y)+{\displaystyle \frac{1}{2{\sigma }_k}}\left|\right|y-y_k{\left|\right|}^2+⟨y,{\nabla }_{y}h(x_{k+1},y_k)⟩,k\in \mathbb{N}.\hfill \end{array}\right.$$

In 2014, Botle et al. [3] gave the proximal alternating linearized minimization.

$$(\mathrm{PALM})\left\{\begin{array}{c}{r}_1>1,r_2>1,\hfill \\ x_1\in H_1\mathrm{and}{y}_1\in H_2\mathrm{are}\mathrm{given}\mathrm{arbitrarily},\hfill \\ c_k=r_1{L}_1(y_k),\hfill \\ x_{k+1}\in pro{x}_{c_k}^f(x_k-{\displaystyle \frac{1}{c_k}}{\nabla }_{x}h(x_k,y_k)),\hfill \\ d_k=r_2{L}_2(x_{k+1}),\hfill \\ x_{k+1}\in pro{x}_{d_k}^g(y_k-{\displaystyle \frac{1}{d_k}}{\nabla }_{y}h(x_{k+1},y_k)),k\in \mathbb{N}.\hfill \end{array}\right.$$

In 2019, Nikolova and Tan [10] gave the alternating structure-adapted proximal gradient descent algorithm.

$$(\mathrm{ASAP})\left\{\begin{array}{c}\mathrm{choose}\tau \in (0,{\displaystyle \frac{2}{Lip({\nabla }_f)}}),\sigma \in (0,{\displaystyle \frac{2}{Lip({\nabla }_g)}}),\hfill \\ x_1\in H_1\mathrm{and}{y}_1\in H_2\mathrm{are}\mathrm{given}\mathrm{arbitrarily},\hfill \\ x_{k+1}\in {\mathrm{argmin}}_{x\in H_1}H(x,y_k)+{\displaystyle \frac{1}{2\tau }}\left|\right|x-x_k{\left|\right|}^2+⟨x,\nabla f(x_k)⟩,\hfill \\ y_{k+1}\in {\mathrm{argmin}}_{y\in H_2}H(x_{k+1},y)+{\displaystyle \frac{1}{2\sigma }}\left|\right|y-y_k{\left|\right|}^2+⟨y,\nabla g(y_k)⟩,k\in \mathbb{N}.\hfill \end{array}\right.$$

In fact, the algorithm of ASAP is equivalent to the following.

$$\left\{\begin{array}{c}\mathrm{choose}\tau \in (0,{\displaystyle \frac{2}{Lip({\nabla }_f)}}),\sigma \in (0,{\displaystyle \frac{2}{Lip({\nabla }_g)}}),\hfill \\ x_1\in H_1\mathrm{and}{y}_1\in H_2\mathrm{are}\mathrm{given}\mathrm{arbitrarily},\hfill \\ x_{k+1}={\mathrm{argmin}}_{x\in H_1}H(x,y_k)+{\displaystyle \frac{1}{2\tau }}\left|\right|x-x_k{\left|\right|}^2+⟨x-x_k,\nabla f(x_k)⟩+f(x_k),\hfill \\ y_{k+1}\in {\mathrm{argmin}}_{y\in H_2}H(x_{k+1},y)+{\displaystyle \frac{1}{2\sigma }}\left|\right|y-y_k{\left|\right|}^2+⟨y-y_k,\nabla g(y_k)⟩+g(y_k),k\in \mathbb{N}.\hfill \end{array}\right.$$

Therefore, the algorithm of ASAP can be written in the following form.

$$\left\{\begin{array}{c}\mathrm{choose}\tau \in (0,{\displaystyle \frac{2}{Lip({\nabla }_f)}}),\sigma \in (0,{\displaystyle \frac{2}{Lip({\nabla }_g)}}),\hfill \\ x_{k+1}=pro{x}_{\tau }^{h(·,y_k)}(x_k-\tau \nabla f(x_k)),\hfill \\ y_{k+1}=pro{x}_{\sigma }^{h(x_{k+1},·)}(y_k-\sigma \nabla g(y_k)),k\in \mathbb{N}.\hfill \end{array}\right.$$

On the other hand, the following is a well-known generalized convex optimization problem, and related Douglas–Rachford algorithm.

$$argmin\{\phi (x)+g(x):x\in H\},$$

where H is a real Hilbert space, and $\phi :H\to (-\infty ,\infty ]$ and $g:H\to (-\infty ,\infty ]$ are proper, lower semicontinuous, and convex functions (see Algorithm 1).

Algorithm 1: ([15], Corollary 27.4)

Let ${\left\{x_n\right\}}_{n\in \mathbb{N}}$ be generated by the following. $$\left\{\begin{array}{c}{x}_1\in H_1\mathrm{is}\mathrm{chosen}\mathrm{arbitrarily},\hfill \\ y_n:=pro{x}_{\beta ,\phi }(x_n),\hfill \\ z_n:=pro{x}_{\beta ,g}(2y_n-x_n),\hfill \\ x_{n+1}:=x_n+k_n(z_n-y_n),n\in \mathbb{N},\hfill \end{array}\right.$$ where $\phi ,g:H\to (-\infty ,\infty ]$ are proper, lower semicontinuous, and convex functions, $\beta$ is a positive real number, ${\left\{k_n\right\}}_{n\in \mathbb{N}}$ is a sequence of positive real numbers, $pro{x}_{\beta ,\phi }$ and $pro{x}_{\beta ,g}$ are proximal operators of g and h with $\beta$, respectively.

Indeed, it is well-known that the Douglas–Rachford algorithm is widely used for convex optimization problems and those involving convexity assumptions. Additionally, despite the lack of theoretical justification, the literature shows that the algorithm has been successfully applied to various practical non-convex problems [16,17]. For further details on the Douglas–Rachford and Peaceman–Rachford algorithms, please refer to [18,19,20,21,22] and related references.

Inspired by the above work, we propose the adaptive Douglas–Rachford algorithm to study the biconvex optimization problem in the finite dimensional real Hilbert spaces.

Remark 1.

In Algorithm 2, if ${\left\{{\mathcal{l}}_n\right\}}_{n\in \mathbb{N}}$ is given by Step 3-1, then this algorithm could be called the Douglas–Rachford algorithm. But, when ${\left\{{\mathcal{l}}_n\right\}}_{n\in \mathbb{N}}$ could be given by Step 3-2, we call this algorithm the adaptive Douglas–Rachford algorithm.

In this paper, we study the biconvex optimization problem and give an adaptive Douglas–Rachford algorithm and related convergence theorems in the setting of finite dimensional real Hilbert spaces. It is worth noting that our approach to proving the convergence theorem differs significantly from those in the literature.

Algorithm 2: Adaptive Douglas–Rachford Algorithm

Let $\tau >0$, and ${\left\{{\mathcal{l}}_n\right\}}_{n\in \mathbb{N}}$ be a sequence in $(0,2)$, and $[a,b]\subseteq (0,2)$ with $a+b<2$, and $x_1$ and $y_1$ be given, and let ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ be generated as follows. Step 1. $u_n=pro{x}_{\tau }^f(x_n)$ and $v_n=pro{x}_{\tau }^g(y_n)$. Step 2. $s_n=pro{x}_{\tau }^{h(·,v_n)}(2u_n-x_n)$ and $t_n=pro{x}_{\tau }^{h(s_n,·)}(2v_n-y_n)$. Step 3. ${\mathcal{l}}_n$ is selected through the following steps.     Step 3-1. If $f(u_n)-f(s_n)+g(v_n)-g(t_n)\ge 0$, then we choose ${\mathcal{l}}_n\in (a,b)$ arbitrarily.     Step 3-2. If $f(u_n)-f(s_n)+g(v_n)-g(t_n)<0$, then we set $p_n$ as $${\displaystyle p_n=\frac{a^2}{2\tau }·\left(\frac{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2]}{f(s_n)-f(u_n)+g(t_n)-g(v_n)}\right).}$$     (i) If $p_n\ge b$, then we choose ${\mathcal{l}}_n\in (a,b)$ arbitrarily.     (ii) If $a\le p_n<b$, then we set ${\mathcal{l}}_n=p_n$.     (iii) If $p_n<a$ and $\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2\ge 2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]$, then we set ${\mathcal{l}}_n=0.5$.     (iv) If $p_n<a$ and $\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2<2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]$, then we set ${\mathcal{l}}_n$ as $${\mathcal{l}}_n=min\left\{b,{\displaystyle \frac{2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]}{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2}}-{\displaystyle \frac{a}{2}}\right\}.$$ Step 4. $x_{n+1}=x_n+{\mathcal{l}}_n(s_n-u_n)$ and $y_{n+1}=y_n+{\mathcal{l}}_n(t_n-v_n)$. Set $n=n+1$ and go to Step 1.

2. Preliminaries

Let H be a real Hilbert space with inner product $⟨·,·⟩$ and norm $\left|\right|·\left|\right|$. We denote the strong and weak convergence of ${\left\{x_n\right\}}_{n\in \mathbb{N}}$ to $x\in H$ by $x_n\to x$ and $x_n\rightharpoonup x$, respectively. For each $x,y,u,v\in H$, and $\lambda \in \mathbb{R}$, we have

$$\left|\right|x+{y\left|\right|}^2={\left|\right|x\left|\right|}^2+2⟨x,y⟩+{\left|\right|y\left|\right|}^2,$$

(1)

$$\left|\right|\lambda x+{(1-\lambda )y\left|\right|}^2={\lambda \left|\right|x\left|\right|}^2+{(1-\lambda )\left|\right|y\left|\right|}^2-\lambda (1-\lambda )\left|\right|x-{y\left|\right|}^2,$$

(2)

$$2⟨x-y,u-v⟩=\left|\right|x-{v\left|\right|}^2+\left|\right|y-{u\left|\right|}^2-\left|\right|x-{u\left|\right|}^2-\left|\right|y-{v\left|\right|}^2.$$

(3)

Definition 1.

Let H be a real Hilbert space, $B:H\to H$ be a mapping, and $\rho >0$. Thus,

 (i)

B is monotone if $⟨x-y,Bx-By⟩\ge 0$ for all $x,y\in H$.

 (ii)

B is ρ-strongly monotone if $⟨x-y,Bx-By⟩\ge \rho \left|\right|x-{y\left|\right|}^2$ for all $x,y\in H$.

Definition 2.

Let H be a real Hilbert space, and $B:H⊸H$ be a set-valued mapping with domain $\mathcal{D}(B):=\{x\in H:B(x)\ne Ø\}$. Thus,

 (i)

B is monotone if $⟨u-v,x-y⟩\ge 0$ for any $u\in B(x)$ and $v\in B(y)$.

 (ii)

B is maximal monotone if its graph $\{(x,y):x\in \mathcal{D}(B),y\in B(x)\}$ is not properly contained in the graph of any other monotone mapping.

 (iii)

B is ρ-strongly monotone ($\rho >0$) if $⟨x-y,u-v⟩\ge \rho \left|\right|x-{y\left|\right|}^2$ for all $x,y\in H$, and all $u\in B(x)$, and $v\in B(y)$.

Definition 3.

Let H be a real Hilbert space, and $f:H\to \mathbb{R}$ be a function. Thus,

 (i)

f is proper if $\{x\in H:f(x)<\infty \}\ne Ø$.

 (ii)

f is lower semicontinuous if $\{x\in H:f(x)\le r\}$ is closed for each $r\in \mathbb{R}$.

 (iii)

f is convex if $f(tx+(1-t)y)\le tf(x)+(1-t)f(y)$ for every $x,y\in H$ and $t\in [0,1]$.

 (iv)

f is ρ-strongly convex ($\rho >0$) if

$$f(tx+(1-t)y)+{\displaystyle \frac{\rho }{2}}·t(1-t)\left|\right|x-{y\left|\right|}^2\le tf(x)+(1-t)f(y)$$

for all $x,y\in H$ and $t\in (0,1)$.

 (v)

f is Gâteaux differentiable at $x\in H$ if there is $\nabla f(x)\in H$ such that

$${\displaystyle \underset{t\to 0}{lim}\frac{f(x+ty)-f(x)}{t}=⟨y,\nabla f(x)⟩}$$

for each $y\in H$.

 (vi)

f is Fréchet differentiable at x if there is $\nabla f(x)$ such that

$$\underset{y\to 0}{lim}{\displaystyle \frac{f(x+y)-f(x)-⟨\nabla f(x),y⟩}{\left|\right|y\left|\right|}}=0.$$

Remark 2.

Let H be a real Hilbert space, and $f:H\to \mathbb{R}$ be a function. Then f is a convex function if and only if ${\displaystyle g(x)=f(x)+\frac{\rho }{2}·{\left|\right|x\left|\right|}^2}$ is a ρ-strongly convex function ([15], Proposition 10.6). Hence, it is easy to establish the relation between convex functions and strongly convex functions.

Definition 4.

Let $f:H\to (-\infty ,\infty ]$ be a proper lower semicontinuous and convex function. Then the subdifferential $\partial f$ of f is defined by

$$\partial f(x):=\{x^{*}\in H:f(x)+⟨y-x,x^{*}⟩\le f(y)\mathrm{for}\mathrm{each}y\in H\}$$

for each $x\in H$.

Lemma 1

([15,23]). Let $f:H\to (-\infty ,\infty ]$ be a proper lower semicontinuous and convex function. Then the following is satisfied.

 (i)

$\partial f$ is a set-valued maximal monotone mapping.

 (ii)

f is Gâteaux differentiable at $x\in \mathrm{int}(\mathcal{D})$ if and only if $\partial f(x)$ consists of a single element. That is, $\partial f(x)=\{\nabla f(x)\}$.

 (iii)

Suppose that f is Fréchet differentiable. Then f is convex if and only if $\nabla f$ is a monotone mapping.

Lemma 2

([15], Example 22.3(iv)). Let $\rho >0$, H be a real Hilbert space, and $f:H\to \mathbb{R}$ be a proper lower-semicontinuous and convex function. If f is ρ-strongly convex, then $\partial f$ is ρ-strongly monotone.

Lemma 3

([15], Proposition 16.26). Let H be a real Hilbert space, and $f:H\to (\infty ,\infty ]$ be a proper lower semicontinuous and convex function. If ${\left\{u_n\right\}}_{n\in \mathbb{N}}$ and ${\left\{x_n\right\}}_{n\in \mathbb{N}}$ are sequences in H with $u_n\in \partial f(x_n)$ for all $n\in \mathbb{N}$, and $x_n\rightharpoonup x$ and $u_n\to u$, then $u\in \partial f(x)$.

Lemma 4

([24]). Let H be a real Hilbert space, $B:H⊸H$ be a set-valued maximal monotone mapping, $\beta >0$, and  $J_{\beta }^B$ be defined by $J_{\beta }^B(x):={(I+\beta B)}^{-1}(x)$ for each $x\in H$. Then $J_{\beta }^B$ is a single-valued mapping.

Definition 5.

Let $\tau >0$, H be a real Hilbert space, and $g:H\to \mathbb{R}$ be a proper lower-semicontinuous and convex function. Then the proximal operator of g with τ is defined by

$$pro{x}_{\tau }^g(x):={\mathrm{argmin}}_{v\in H}\{g(v)+{\displaystyle \frac{1}{2\tau }}\left|\right|v-{x\left|\right|}^2\}$$

for each $x\in H$.

Lemma 5.

Let $f:H\to \mathbb{R}$ be a proper, lower semicontinuous, and convex function. Assume $\tau >0$ and $\overline{x}=pro{x}_{\tau }^f(x)$, we have

$$f(u)\ge f(\overline{x})-{\displaystyle \frac{1}{\tau }}·⟨u-\overline{x},\overline{x}-x⟩$$

for each $u\in H$.

3. Main Results

We are interested in solving nonconvex minimization problems of the form

$$(BOP){\mathrm{argmin}}_{x\in H_1,y\in H_2}J(x,y)=f(x)+g(y)+h(x,y),$$

where $H_1$ and $H_2$ are two real Hilbert spaces, $J:H_1\times H_2\to \mathbb{R}$, $f:H_1\to \mathbb{R}$, $g:H_2:\to \mathbb{R}$, and $h:H_1\times H_2\to \mathbb{R}$.

Here, if $(\overline{x},\overline{y})\in H_1\times H_2$ is a solution of the problem (BOP), then

$$f(\overline{x})+g(\overline{y})+h(\overline{x},\overline{y})\le f(x)+g(y)+h(x,y)\mathrm{for}\mathrm{all}(x,y)\in H_1\times H_2,$$

(4)

and this implies that

$$f(\overline{x})+g(\overline{y})+h(\overline{x},\overline{y})\le f(x)+g(\overline{y})+h(x,\overline{y}),$$

and

$$f(\overline{x})+g(\overline{y})+h(\overline{x},\overline{y})\le f(\overline{x})+g(y)+h(\overline{x},y)$$

for all $(x,y)\in H_1\times H_2$. That is,

$$\forall (x,y)\in H_1\times H_2:f(\overline{x})+h(\overline{x},\overline{y})\le f(x)+h(x,\overline{y})\&g(\overline{y})+h(\overline{x},\overline{y})\le g(y)+h(\overline{x},y).$$

(5)

This implies that

$$\forall (x,y)\in H_1\times H_2:f(\overline{x})+g(\overline{y})+2·h(\overline{x},\overline{y})\le f(x)+g(y)+h(x,\overline{y})+h(\overline{x},y).$$

(6)

So, it is natural to give a condition:

$${(BOP)}_{C}h(x,v)+h(u,y)=h(x,y)+h(u,v)$$

for all $x,u\in H_1$ and $y,v\in H_2$. Indeed, if this condition ${(BOP)}_C$ holds, then (4), (5), and (6) are equivalent.

Example 1.

Let $h:\mathbb{R}\times \mathbb{R}\to \mathbb{R}$ be defined as

$$h(x,y)=\left\{\begin{array}{ccc}log\left|x\right|+log\left|y\right|\hfill & \mathrm{if}\hfill & \left|x\right|>1\mathrm{and}\left|y\right|>1,\hfill \\ log\left|x\right|\hfill & \mathrm{if}\hfill & \left|x\right|>1\mathrm{and}\left|y\right|\le 1,\hfill \\ log\left|y\right|\hfill & \mathrm{if}\hfill & \left|x\right|\le 1\mathrm{and}\left|y\right|>1,\hfill \\ 0\hfill & \mathrm{if}\hfill & \left|x\right|\le 1\mathrm{and}\left|y\right|\le 1,\hfill \end{array}\right.$$

for all $(x,y)\in \mathbb{R}\times \mathbb{R}$. Then h satisfies the condition ${(BOP)}_C$.

Assumption 1.

Assume that:

 (i)

f and g are proper lower semicontinuous and convex functions;

 (ii)

h is continuous, and $x\to h(x,y)$ and $y\to h(x,y)$ are proper and convex functions;

 (iii)

J is lower bounded;

 (iv)

$h(x,v)+h(u,y)=h(x,y)+h(u,v)$ for all $x,u\in H_1$ and $y,v\in H_2$.

For a fixed y, the partial subdifferential of $J(·,y)$ at x is denoted by ${\partial }_{x}J(x,y)$. For a fixed x, the partial subdifferential of $J(x,·)$ at y is denoted by ${\partial }_{y}J(x,y)$. Hence, for $J(x,y)=f(x)+g(y)+h(x,y)$, we have

$$\partial J(x,y)={\partial }_{x}J(x,y)\times {\partial }_{y}J(x,y)=(\partial f(x)+{\partial }_{x}h(x,y))\times (\partial g(y)+{\partial }_{y}h(x,y)).$$

Fermat’s rule, extended to nonconvex and nonsmooth function, is given next.

Proposition 1

([25], Theorem 10.1). Let $f:{\mathbb{R}}^m\to \mathbb{R}\cup \{+\infty \}$ be a proper function. If f has a local minimum at $\overline{x}$, then $0\in \partial f(\overline{x})$.

Definition 6.

We say that $(\overline{x},\overline{y})$ is a critical point of J if $(0,0)\in \partial J(\overline{x},\overline{y})$. For simplicity, the set of the critical points of J is denoted by $\mathrm{crit}(J)$.

Remark 3.

We know

$$\begin{array}{cc}& (0,0)\in \partial J(\overline{x},\overline{y})\hfill \\ \hfill \iff & (0,0)\in {\partial }_{x}J(\overline{x},\overline{y})\times {\partial }_{y}J(\overline{x},\overline{y})\hfill \\ \hfill \iff & 0\in \partial f(\overline{x})+{\partial }_{x}h(\overline{x},\overline{y})\&0\in \partial g(\overline{y})+{\partial }_{y}h(\overline{x},\overline{y})\hfill \\ \hfill \iff & \forall (x,y)\in H_1\times H_2:f(\overline{x})+h(\overline{x},\overline{y})\le f(x)+h(x,\overline{y})\&g(\overline{y})+h(\overline{x},\overline{y})\le g(y)+h(\overline{x},y).\hfill \end{array}$$

Hence, if the condition ${(BOP)}_C$ is satisfied, then $(0,0)\in \partial J(\overline{x},\overline{y})$ if and only if $(\overline{x},\overline{y})$ is a solution of the problem $(BOP)$.

Here, we consider the first part of Algorithm 2.

Remark 4.

In Algorithm 3, for each $x\in H_1$ and $y\in H_2$, we set $u=pro{x}_{\tau }^f(x)$ and $v=pro{x}_{\tau }^g(y)$. Thus, $\left|\right|u_n-u{\left|\right|}^2\le ⟨x_n-x,u_n-u⟩$ and $\left|\right|v_n-v{\left|\right|}^2\le ⟨y_n-y,v_n-v⟩$. Further, if ${\left\{x_n\right\}}_{n\in \mathbb{N}}$ and ${\left\{y_n\right\}}_{n\in \mathbb{N}}$ are bounded, then ${\left\{u_n\right\}}_{n\in \mathbb{N}}$ and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ are bounded.

Algorithm 3: Adaptive Douglas–Rachford Algorithm (Part 1)

Let $\tau >0$, and ${\left\{{\mathcal{l}}_n\right\}}_{n\in \mathbb{N}}$ be a sequence in $(0,2)$, and $[a,b]\subseteq (0,2)$ with $a+b<2$, and $x_1$ and $y_1$ be given, and let ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ be generated as $$\left\{\begin{array}{l}{u}_n=pro{x}_{\tau }^f(x_n),\\ v_n=pro{x}_{\tau }^g(y_n),\\ s_n=pro{x}_{\tau }^{h(·,v_n)}(2u_n-x_n),\\ t_n=pro{x}_{\tau }^{h(s_n,·)}(2v_n-y_n),\\ x_{n+1}=x_n+{\mathcal{l}}_n(s_n-u_n),\\ y_{n+1}=y_n+{\mathcal{l}}_n(t_n-v_n),n\in \mathbb{N}.\end{array}\right.$$

Proof. 

Since $u_n=pro{x}_{\tau }^f(x_n)={(I+\tau \partial f)}^{-1}(x_n)$ and $u=pro{x}_{\tau }^f(x)$, we have

$${\displaystyle \frac{1}{\tau }}(x_n-u_n)\in \partial f(u_n)\mathrm{and}{\displaystyle \frac{1}{\tau }}(x-u)\in \partial f(u)$$

(7)

Since $\partial f$ is maximal monotone, we know

$$⟨(x_n-u_n)-(x-u),u_n-u⟩\ge 0,$$

(8)

and this implies that

$$\left|\right|u_n-{u\left|\right|}^2\le ⟨x_n-x,u_n-u⟩\le \left|\right|u_n-u\left|\right|·\left|\right|x_n-x\left|\right|.$$

(9)

Similarly, we have

$${\displaystyle \frac{1}{\tau }}(y_n-v_n)\in \partial g(v_n)\mathrm{and}{\displaystyle \frac{1}{\tau }}(y-v)\in \partial g(v),$$

(10)

and

$$\left|\right|v_n-{v\left|\right|}^2\le ⟨y_n-y,v_n-v⟩\le \left|\right|v_n-v\left|\right|·\left|\right|y_n-y\left|\right|.$$

(11)

Further, it is easy to see ${\left\{u_n\right\}}_{n\in \mathbb{N}}$ and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ are bounded when ${\left\{x_n\right\}}_{n\in \mathbb{N}}$ and ${\left\{y_n\right\}}_{n\in \mathbb{N}}$ are bounded.    □

Lemma 6.

Let ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ be generated from Algorithm 3. Then, for each $x\in H_1$ and $y\in H_2$, and $n\in \mathbb{N}$, we have

$$\begin{array}{cc}\hfill \left|\right|x_{n+1}-x{\left|\right|}^2\le & \left|\right|x_n-{x\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|s_n-u_n{\left|\right|}^2\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)-f(x)+h(s_n,v_n)-h(x,v_n)],\hfill \end{array}$$

(12)

and

$$\begin{array}{cc}\hfill \left|\right|y_{n+1}-y{\left|\right|}^2\le & \left|\right|y_n-{y\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|t_n-v_n{\left|\right|}^2\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [g(v_n)-g(y)+h(s_n,t_n)-h(s_n,y)].\hfill \end{array}$$

(13)

Proof. 

Take any $x\in H_1$ and $y\in H_2$, and let $x,y$ be fixed. First, it follows from $u_n=pro{x}_{\tau }^f(x_n)$ and Lemma 5 that

$$f(x)\ge f(u_n)-{\displaystyle \frac{1}{\tau }}·⟨x-u_n,u_n-x_n⟩,$$

(14)

and this implies that

$$\left|\right|u_n-{x\left|\right|}^2+2\tau [f(u_n)-f(x)]\le \left|\right|x_n-{x\left|\right|}^2-\left|\right|x_n-u_n{\left|\right|}^2.$$

(15)

Similar to (15), we have

$$\left|\right|v_n-{y\left|\right|}^2+2\tau [g(v_n)-g(y)]\le \left|\right|y_n-{y\left|\right|}^2-\left|\right|y_n-v_n{\left|\right|}^2.$$

(16)

Next, it follows from $s_n=pro{x}_{\tau }^{h(·,v_n)}(2u_n-x_n)$ and Lemma 5 that

$$h(x,v_n)\ge h(s_n,v_n)-{\displaystyle \frac{1}{\tau }}·⟨x-s_n,s_n-(2u_n-x_n)⟩,$$

(17)

and this implies that

$$2\tau [h(s_n,v_n)-h(x,v_n)]\le 2⟨x-s_n,s_n-(2u_n-x_n)⟩.$$

(18)

Here, we know

$$\begin{array}{cc}\hfill & 2⟨x-s_n,s_n-(2u_n-x_n)⟩\hfill \\ \hfill =& \left|\right|2u_n-x_n-{x\left|\right|}^2-\left|\right|s_n-{x\left|\right|}^2-\left|\right|2u_n-x_n-s_n{\left|\right|}^2\hfill \\ \hfill =& \left|\right|(u_n-x_n)+(u_n-x){\left|\right|}^2-\left|\right|s_n-{x\left|\right|}^2-\left|\right|(u_n-x_n)+(u_n-s_n){\left|\right|}^2\hfill \\ \hfill =& \left|\right|u_n-x_n{\left|\right|}^2+\left|\right|u_n-{x\left|\right|}^2+2⟨u_n-x_n,u_n-x⟩-\left|\right|s_n-x{\left|\right|}^2\hfill \\ \hfill & -\left|\right|u_n-x_n{\left|\right|}^2-\left|\right|u_n-s_n{\left|\right|}^2-2⟨u_n-x_n,u_n-s_n⟩\hfill \\ \hfill =& \left|\right|u_n-{x\left|\right|}^2-\left|\right|s_n-{x\left|\right|}^2-\left|\right|u_n-s_n{\left|\right|}^2+2⟨u_n-x_n,s_n-x⟩\hfill \\ \hfill =& \left|\right|u_n-{x\left|\right|}^2-\left|\right|s_n-{x\left|\right|}^2-\left|\right|u_n-s_n{\left|\right|}^2+\left|\right|u_n-{x\left|\right|}^2+\left|\right|s_n-x_n{\left|\right|}^2\hfill \\ \hfill & -\left|\right|u_n-s_n{\left|\right|}^2-\left|\right|x_n-x{\left|\right|}^2\hfill \\ \hfill =& 2\left|\right|u_n-{x\left|\right|}^2-\left|\right|s_n-{x\left|\right|}^2-2\left|\right|u_n-s_n{\left|\right|}^2+\left|\right|s_n-x_n{\left|\right|}^2-\left|\right|x_n-x{\left|\right|}^2.\hfill \end{array}$$

(19)

By (18) and (19), we have

$$\begin{array}{cc}\hfill & 2\tau [h(s_n,v_n)-h(x,v_n)]\hfill \\ \hfill \le & 2\left|\right|u_n-{x\left|\right|}^2-\left|\right|s_n-{x\left|\right|}^2-2\left|\right|u_n-s_n{\left|\right|}^2+\left|\right|s_n-x_n{\left|\right|}^2-\left|\right|x_n-x{\left|\right|}^2.\hfill \end{array}$$

(20)

Similar to (18), we have

$$2\tau [h(s_n,t_n)-h(s_n,y)]\le 2⟨y-t_n,t_n-(2v_n-y_n)⟩.$$

(21)

Hence, we know

$$\begin{array}{cc}\hfill & 2⟨y-t_n,t_n-(2v_n-y_n)⟩\hfill \\ \hfill =& \left|\right|2v_n-y_n-{y\left|\right|}^2-\left|\right|t_n-{y\left|\right|}^2-\left|\right|2v_n-y_n-t_n{\left|\right|}^2\hfill \\ \hfill =& \left|\right|(v_n-y_n)+(v_n-y){\left|\right|}^2-\left|\right|t_n-{y\left|\right|}^2-\left|\right|(v_n-y_n)+(v_n-t_n){\left|\right|}^2\hfill \\ \hfill =& \left|\right|v_n-{y\left|\right|}^2-\left|\right|t_n-{y\left|\right|}^2-\left|\right|v_n-t_n{\left|\right|}^2+2⟨v_n-y_n,t_n-y⟩\hfill \\ \hfill =& 2\left|\right|v_n-{y\left|\right|}^2-\left|\right|t_n-{y\left|\right|}^2-2\left|\right|v_n-t_n{\left|\right|}^2+\left|\right|y_n-t_n{\left|\right|}^2-\left|\right|y_n-y{\left|\right|}^2.\hfill \end{array}$$

(22)

By (21) and (22), we have

$$\begin{array}{cc}& 2\tau [h(s_n,t_n)-h(s_n,y)]\hfill \\ \le & 2\left|\right|v_n-{y\left|\right|}^2-\left|\right|t_n-{y\left|\right|}^2-2\left|\right|v_n-t_n{\left|\right|}^2+\left|\right|y_n-t_n{\left|\right|}^2-\left|\right|y_n-y{\left|\right|}^2.\end{array}$$

(23)

Next, we have

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-x{\left|\right|}^2\hfill \\ \hfill =& \left|\right|x_n+{\mathcal{l}}_n(s_n-u_n)-x{\left|\right|}^2\hfill \\ \hfill =& \left|\right|(1-{\displaystyle \frac{{\mathcal{l}}_n}{2}})(x_n-x)+{\displaystyle \frac{{\mathcal{l}}_n}{2}}(x_n+2s_n-2u_n-x){\left|\right|}^2\hfill \\ \hfill =& (1-{\displaystyle \frac{{\mathcal{l}}_n}{2}})\left|\right|x_n-{x\left|\right|}^2+{\displaystyle \frac{{\mathcal{l}}_n}{2}}\left|\right|x_n+2s_n-2u_n-{x\left|\right|}^2-{\displaystyle \frac{{\mathcal{l}}_n}{2}}(1-{\displaystyle \frac{{\mathcal{l}}_n}{2}})\left|\right|2s_n-2u_n{\left|\right|}^2\hfill \\ \hfill =& (1-{\displaystyle \frac{{\mathcal{l}}_n}{2}})\left|\right|x_n-{x\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|s_n-u_n{\left|\right|}^2\hfill \\ \hfill & +{\displaystyle \frac{{\mathcal{l}}_n}{2}}\left[\left|\right|x_n-{x\left|\right|}^2+4\left|\right|s_n-u_n{\left|\right|}^2+4⟨x_n-x,s_n-u_n⟩\right]\hfill \\ \hfill =& \left|\right|x_n-{x\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|s_n-u_n{\left|\right|}^2+2{\mathcal{l}}_n\left|\right|s_n-u_n{\left|\right|}^2\hfill \\ \hfill & +{\mathcal{l}}_n\left|\right|x_n-u_n{\left|\right|}^2+{\mathcal{l}}_n\left|\right|s_n-{x\left|\right|}^2-{\mathcal{l}}_n\left|\right|x_n-s_n{\left|\right|}^2-{\mathcal{l}}_n\left|\right|u_n-x{\left|\right|}^2.\hfill \end{array}$$

(24)

By (15), (20), and (24), we have

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-x{\left|\right|}^2\hfill \\ \hfill =& \left|\right|x_n-{x\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|s_n-u_n{\left|\right|}^2+2{\mathcal{l}}_n\left|\right|s_n-u_n{\left|\right|}^2\hfill \\ \hfill & +{\mathcal{l}}_n\left|\right|x_n-u_n{\left|\right|}^2+{\mathcal{l}}_n\left|\right|s_n-{x\left|\right|}^2-{\mathcal{l}}_n\left|\right|x_n-s_n{\left|\right|}^2-{\mathcal{l}}_n\left|\right|u_n-x{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-{x\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|s_n-u_n{\left|\right|}^2+2{\mathcal{l}}_n\left|\right|s_n-u_n{\left|\right|}^2\hfill \\ \hfill & +{\mathcal{l}}_n(||x_n-{x\left|\right|}^2-\left|\right|u_n-{x\left|\right|}^2-2\tau [f(u_n)-f(x)])\hfill \\ \hfill & +{\mathcal{l}}_n(2||u_n-{x\left|\right|}^2-2\left|\right|u_n-s_n{\left|\right|}^2+\left|\right|s_n-x_n{\left|\right|}^2-\left|\right|x_n-x{\left|\right|}^2\hfill \\ \hfill & -2\tau [h(s_n,v_n)-h(x,v_n)])-{\mathcal{l}}_n\left|\right|x_n-s_n{\left|\right|}^2-{\mathcal{l}}_n\left|\right|u_n-x{\left|\right|}^2\hfill \\ \hfill =& \left|\right|x_n-{x\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|s_n-u_n{\left|\right|}^2\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)-f(x)+h(s_n,v_n)-h(x,v_n)].\hfill \end{array}$$

(25)

Similar to (24), we have

$$\begin{array}{cc}\hfill \left|\right|y_{n+1}-y{\left|\right|}^2=& \left|\right|y_n-{y\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|t_n-v_n{\left|\right|}^2+2{\mathcal{l}}_n\left|\right|t_n-v_n{\left|\right|}^2\hfill \\ \hfill & +{\mathcal{l}}_n\left|\right|y_n-v_n{\left|\right|}^2+{\mathcal{l}}_n\left|\right|t_n-{y\left|\right|}^2-{\mathcal{l}}_n\left|\right|y_n-t_n{\left|\right|}^2-{\mathcal{l}}_n\left|\right|v_n-y{\left|\right|}^2.\hfill \end{array}$$

(26)

By (16), (23), and (26),

$$\begin{array}{cc}\hfill \left|\right|y_{n+1}-y{\left|\right|}^2\le & \left|\right|y_n-{y\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)\left|\right|t_n-v_n{\left|\right|}^2\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [g(v_n)-g(y)+h(s_n,t_n)-h(s_n,y)].\hfill \end{array}$$

(27)

So, we obtain the conclusion of Lemma 6.    □

The following is the second part of Algorithm 2.

Remark 5.

In Algorithm 4, we know the sequence ${\left\{{\mathcal{l}}_n\right\}}_{n\in \mathbb{N}}$ is chosen from the interval $(0,2)$ with

$${\displaystyle \frac{a}{2}}=min\left\{{\displaystyle \frac{a}{2}},{\displaystyle \frac{1}{2}}\right\}\le {\mathcal{l}}_n\le b.$$

Algorithm 4: Adaptive Douglas–Rachford Algorithm (Part 2)

In Algorithm 3, let ${\mathcal{l}}_n$ be selected through the following steps. Case 1. If $f(u_n)-f(s_n)+g(v_n)-g(t_n)\ge 0$, then we choose ${\mathcal{l}}_n\in (a,b)$ arbitrarily. Case 2. If $f(u_n)-f(s_n)+g(v_n)-g(t_n)<0$, then we set $p_n$ as $${\displaystyle p_n=\frac{a^2}{2\tau }·\left(\frac{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2]}{f(s_n)-f(u_n)+g(t_n)-g(v_n)}\right).}$$   (i) If $p_n\ge b$, then we choose ${\mathcal{l}}_n\in (a,b)$ arbitrarily.   (ii) If $a\le p_n<b$, then we set ${\mathcal{l}}_n=p_n$.   (iii) If $p_n<a$ and $\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2\ge 2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)],$ then we set ${\mathcal{l}}_n=0.5$.   (iv) If $p_n<a$ and $\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2<2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)],$ then we set ${\mathcal{l}}_n$ as $${\mathcal{l}}_n=min\left\{b,{\displaystyle \frac{2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]}{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2}}-{\displaystyle \frac{a}{2}}\right\}.$$

Theorem 1.

Let ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ be generated from Algorithms 3 and 4, and we assume that the solution set of the problem (BOP) is nonempty, and we assume that $H_1$ and $H_2$ are finite dimensional. Then there exist $\overline{x},\overline{u}\in H_1$, $\overline{y},\overline{v}\in H_2$, a subsequence ${\left\{(x_{n_k},y_{n_k},u_{n_k},v_{n_k})\right\}}_{k\in \mathbb{N}}$ of ${\left\{(x_n,y_n,u_n,v_n)\right\}}_{n\in \mathbb{N}}$ such that $x_{n_k}\to \overline{x}$, $y_{n_k}\to \overline{y}$, $u_{n_k}\to \overline{u}$, $v_{n_k}\to \overline{v}$, and $(\overline{u},\overline{v})$ is a solution of the problem $(BOP)$.

Proof. 

Let $(\widehat{u},\widehat{v})\in H_1\times H_2$ be any solution of problem (BOP). By (26), (27), and the assumption, we have

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)-f(\widehat{u})+h(s_n,v_n)-h(\widehat{u},v_n)]\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [g(v_n)-g(\widehat{v})+h(s_n,t_n)-h(s_n,\widehat{v})]\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)+g(v_n)+h(s_n,t_n)-f(\widehat{u})-g(\widehat{v})-h(\widehat{u},\widehat{v})]\hfill \\ \hfill =& \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(s_n)+g(t_n)+h(s_n,t_n)-f(\widehat{u})-g(\widehat{v})-h(\widehat{u},\widehat{v})]\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)-f(s_n)+g(v_n)-g(t_n)]\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)-f(s_n)+g(v_n)-g(t_n)].\hfill \end{array}$$

(28)

Next, we consider the following cases.

Case 1: If $f(u_n)-f(s_n)+g(v_n)-g(t_n)\ge 0$, then we choose ${\mathcal{l}}_n\in (a,b)\subseteq (0,2)$. Hence,

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & -2{\mathcal{l}}_n\tau [f(u_n)-f(s_n)+g(v_n)-g(t_n)]\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-a(2-b)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2).\hfill \end{array}$$

(29)

Case 2: If $f(u_n)-f(s_n)+g(v_n)-g(t_n)<0$, then we set $P_n$ as

$${\displaystyle p_n=\frac{a^2}{2\tau }·\left(\frac{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2]}{f(s_n)-f(u_n)+g(t_n)-g(v_n)}\right).}$$

Case 2 (i): If $p_n\ge b$, then we choose ${\mathcal{l}}_n\in (a,b)\subseteq (0,2)$. Hence,

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & +2{\mathcal{l}}_n\tau [f(s_n)-f(u_n)+g(t_n)-g(v_n)]\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-a(2-b)·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & +a^2·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-a(2-a-b)·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2).\hfill \end{array}$$

(30)

Case 2 (ii): If $a\le p_n<b$, then we set ${\mathcal{l}}_n=p_n$. Hence,

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-a(2-b)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & +2{\mathcal{l}}_n\tau [f(s_n)-f(u_n)+g(t_n)-g(v_n)]\hfill \\ \hfill =& \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-a(2-a-b)·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2).\hfill \end{array}$$

(31)

Case 2 (iii): If $p_n<a$ and $\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2\ge 2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]$, then we set ${\mathcal{l}}_n=0.5$ and have the following.

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & +2{\mathcal{l}}_n\tau [f(s_n)-f(u_n)+g(t_n)-g(v_n)]\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & +{\mathcal{l}}_n(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n(1-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\displaystyle \frac{1}{4}}·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2).\hfill \end{array}$$

(32)

Case 2 (iv): If $p_n<a$ and $\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2<2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]$, then we set ${\mathcal{l}}_n$ as

$${\mathcal{l}}_n=min\left\{b,{\displaystyle \frac{2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]}{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2}}-{\displaystyle \frac{a}{2}}\right\}.$$

Thus, we have the following.

$${\displaystyle \frac{a}{2}}\le {\mathcal{l}}_n\le b\mathrm{and}2-{\mathcal{l}}_n\ge 2+{\displaystyle \frac{a}{2}}-{\displaystyle \frac{2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]}{\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2}}.$$

So,

$$\begin{array}{cc}\hfill & (2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)-2\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]\hfill \\ \hfill \ge & (2+{\displaystyle \frac{a}{2}})(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)-4\tau ·[f(s_n)-f(u_n)+g(t_n)-g(v_n)]\hfill \\ \hfill \ge & (2+{\displaystyle \frac{a}{2}})(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)-2(\left|\right|s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill =& {\displaystyle \frac{a}{2}}·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2),\hfill \end{array}$$

(33)

and this implies that

$$\begin{array}{cc}\hfill & \left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{\mathcal{l}}_n[(2-{\mathcal{l}}_n)(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)\hfill \\ \hfill & -2\tau [f(s_n)-f(u_n)+g(t_n)-g(v_n)]]\hfill \\ \hfill \le & \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-{({\displaystyle \frac{a}{2}})}^2·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2).\hfill \end{array}$$

(34)

Set $r_n$ as

$$r_n=min\left\{a(2-b),a(2-a-b),{\displaystyle \frac{1}{4}},{({\displaystyle \frac{a}{2}})}^2\right\}=min\left\{a(2-a-b),{({\displaystyle \frac{a}{2}})}^2\right\}.$$

(35)

Hence, we obtain the following from (30), (31), (32), (34), and (35).

$$\left|\right|x_{n+1}-\widehat{u}{\left|\right|}^2+\left|\right|y_{n+1}-\widehat{v}{\left|\right|}^2\le \left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2-r_n·(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2).$$

Therefore, $\left\{\right||x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2{\}}_{n\in \mathbb{N}}$ is nondecreasing, and ${\displaystyle \underset{n\to \infty }{lim}\left|\right|x_n-\widehat{u}{\left|\right|}^2+\left|\right|y_n-\widehat{v}{\left|\right|}^2}$ exists, and this implies that ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ are bounded, and

$$\sum _{n=1}^{\infty }(||s_n-u_n{\left|\right|}^2+\left|\right|t_n-v_n{\left|\right|}^2)<\infty .$$

(36)

So,

$$\sum _{n=1}^{\infty }\left|\right|x_{n+1}-x_n{\left|\right|}^2=\sum _{n=1}^{\infty }{\mathcal{l}}_n^2·\left|\right|s_n-u_n{\left|\right|}^2\le 4\sum _{n=1}^{\infty }\left|\right|s_n-u_n{\left|\right|}^2<\infty$$

(37)

and

$$\sum _{n=1}^{\infty }\left|\right|y_{n+1}-y_n{\left|\right|}^2=\sum _{n=1}^{\infty }{\mathcal{l}}_n^2·\left|\right|t_n-v_n{\left|\right|}^2\le 4\sum _{n=1}^{\infty }\left|\right|t_n-v_n{\left|\right|}^2<\infty .$$

(38)

Since ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, and ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ are bounded, there exist $\overline{x},\overline{u}\in H_1$, $\overline{y},\overline{v}\in H_2$, a subsequence ${\left\{x_{n_k}\right\}}_{k\in \mathbb{N}}$ of ${\left\{x_n\right\}}_{n\in \mathbb{N}}$, a subsequence ${\left\{y_{n_k}\right\}}_{k\in \mathbb{N}}$ of ${\left\{y_n\right\}}_{n\in \mathbb{N}}$, a subsequence ${\left\{u_{n_k}\right\}}_{k\in \mathbb{N}}$ of ${\left\{u_n\right\}}_{n\in \mathbb{N}}$, and a subsequence ${\left\{v_{n_k}\right\}}_{k\in \mathbb{N}}$ of ${\left\{v_n\right\}}_{n\in \mathbb{N}}$ such that $x_{n_k}\to \overline{x}$, $y_{n_k}\to \overline{y}$, $u_{n_k}\to \overline{u}$, and $v_{n_k}\to \overline{v}$.

Next, by (7) and (10), we know

$$x_{n_k}-u_{n_k}\in \tau \partial f(u_{n_k})\mathrm{and}{y}_{n_k}-v_{n_k}\in \tau \partial g(v_{n_k}),$$

(39)

and this implies that

$$\overline{x}-\overline{u}\in \tau \partial f(\overline{u})\mathrm{and}\overline{y}-\overline{v}\in \tau \partial g(\overline{v}).$$

(40)

Since $s_n=pro{x}_{\tau }^{h(·,v_n)}(2u_n-x_n)$ and $t_n=pro{x}_{\tau }^{h(s_n,·)}(2v_n-y_n)$, we know

$$u_{n_k}-s_{n_k}+u_{n_k}-x_{n_k}\in \tau {\partial }_{x}h(s_{n_k},v_{n_k}),$$

(41)

and

$$v_{n_k}-t_{n_k}+v_{n_k}-y_{n_k}\in \tau {\partial }_{y}h(s_{n_k},t_{n_k}).$$

(42)

So, we have

$$\overline{u}-\overline{x}\in \tau \partial h_x(\overline{u},\overline{v})\mathrm{and}\overline{v}-\overline{y}\in \tau \partial h_y(\overline{u},\overline{v}).$$

(43)

This implies that

$$0\in \partial f(\overline{u})+\partial h_x(\overline{u},\overline{v})\mathrm{and}0\in \partial g(\overline{v})+\partial h_y(\overline{u},\overline{v}).$$

(44)

So, $(\overline{u},\overline{v})$ is a critical point of J and this implies that $(\overline{u},\overline{v})$ is a solution of the problem (BOP). □

Theorem 2.

In Theorem 1, if $\rho >0$ and we further assume that f and g are ρ-strongly convex, then there exist $\overline{u}\in H_1$, $\overline{v}\in H_2$ such that $u_n\to \overline{u}$ and $v_n\to \overline{v}$, where $(\overline{u},\overline{v})$ is a solution of the problem $(BOP)$.

Proof. 

In Theorem 1, there exist $\overline{x},\overline{u}\in H_1$, $\overline{y},\overline{v}\in H_2$, a subsequence ${\left\{(x_{n_k},y_{n_k},u_{n_k},v_{n_k})\right\}}_{k\in \mathbb{N}}$ of ${\left\{(x_n,y_n,u_n,v_n)\right\}}_{n\in \mathbb{N}}$ such that $x_{n_k}\to \overline{x}$, $y_{n_k}\to \overline{y}$, $u_{n_k}\to \overline{u}$, $v_{n_k}\to \overline{v}$, and $(\overline{u},\overline{v})$ is a solution of the problem $(BOP)$. Further, we have

$$\overline{x}-\overline{u}\in \tau \partial f(\overline{u})\mathrm{and}\overline{y}-\overline{v}\in \tau \partial g(\overline{v}),$$

(45)

and

$$\overline{u}-\overline{x}\in \tau \partial h_x(\overline{u},\overline{v})\mathrm{and}\overline{v}-\overline{y}\in \tau \partial h_y(\overline{u},\overline{v}).$$

(46)

Hence, if ${\left\{({\widehat{u}}_{n_k},{\widehat{v}}_{n_k})\right\}}_{k\in \mathbb{N}}$ is a subsequence of ${\left\{(u_n,v_n)\right\}}_{n\in \mathbb{N}}$ such that ${\widehat{u}}_{n_k}\to \widehat{u}$ and ${\widehat{v}}_{n_k}\to \widehat{v}$. Clearly, we know ${\left\{({\widehat{x}}_{n_k},{\widehat{y}}_{n_k})\right\}}_{k\in \mathbb{N}}$ is a bounded subsequence of ${\left\{(x_n,y_n)\right\}}_{n\in \mathbb{N}}$. So, without loss of generality, we may assume that ${\widehat{x}}_{n_k}\to \widehat{x}$ and ${\widehat{y}}_{n_k}\to \widehat{y}$. Next, it follows from the proof of Theorem 1 that $(\widehat{u},\widehat{v})$ is a solution of the problem $(BOP)$, and

$$\widehat{x}-\widehat{u}\in \tau \partial f(\widehat{u})\mathrm{and}\widehat{y}-\widehat{v}\in \tau \partial g(\widehat{v}),$$

(47)

and

$$\widehat{u}-\widehat{x}\in \tau \partial h_x(\widehat{u},\widehat{v})\mathrm{and}\widehat{v}-\widehat{y}\in \tau \partial h_y(\widehat{u},\widehat{v}).$$

(48)

By (45) and (47), and f is when $\rho$-strongly convex, we have

$$\tau ·\rho ·\left|\right|\overline{u}-\widehat{u}{\left|\right|}^2\le ⟨(\widehat{x}-\widehat{u})-(\overline{x}-\overline{u}),\widehat{u}-\overline{u}⟩,$$

(49)

and this implies that

$$\tau ·\rho ·\left|\right|\overline{u}-\widehat{u}{\left|\right|}^2\le ⟨\widehat{x}-\overline{x},\widehat{u}-\overline{u}⟩-\left|\right|\overline{u}-\widehat{u}{\left|\right|}^2$$

(50)

Since $x\to h(x,y)$ is convex, we have

$$0\le ⟨(\widehat{u}-\widehat{x})-(\overline{u}-\overline{x}),\widehat{u}-\overline{u}⟩=⟨-\widehat{x}+\overline{x},\widehat{u}-\overline{u}⟩+\left|\right|\widehat{u}-\overline{u}{\left|\right|}^2.$$

(51)

By (50) and (51), we know $\overline{u}=\widehat{u}$. Similarly, we have $\overline{v}=\widehat{v}$. Therefore, we know $u_n\to \overline{u}$ and $v_n\to \overline{v}$, and the proof is completed. □