An Adaptive Projection Gradient Method for Solving Nonlinear Fractional Programming

Abstract

In this study, we focus on solving the nonlinear fractional optimization problem in which the numerator is smooth convex and the denominator is smooth concave. To achieve this goal, we develop an algorithm called the adaptive projection gradient method. The main advantage of this method is that it allows the computations for the gradients of the considered functions and the metric projection to take place separately. Moreover, an interesting property that distinguishes the proposed method from some of the existing methods is the nonincreasing property of its step-size sequence. In this study, we also prove that the sequence of iterates that is generated by the method converges to a solution for the considered problem and we derive the rate of convergence. To illustrate the performance and efficiency of our algorithm, some numerical experiments are performed.

1. Introduction

Many practical situations that arise in economics [1,2,3,4], machine learning [5,6,7,8] and wireless communication [9,10,11] can be formulated as the problem of minimizing or maximizing objective functions that appear as ratios of functions. This problem is referred to as the fractional programming problem. Due to its broad range of applications, the fractional programming problem has received a lot of attention and has been studied for many decades.

In this paper, we consider the following nonlinear fractional programming problem:

$${\displaystyle {\theta }^{*}:=\underset{x\in C}{min}\frac{f\left(x\right)}{g\left(x\right)},}$$

(1)

where C is a nonempty closed and convex subset of a Euclidean space ${\mathbb{R}}^n$ with the inner product $\langle ·,·\rangle$ and its induced norm $\parallel ·\parallel$. We also assume that the following conditions hold:

($A_f$)

The function $f:{\mathbb{R}}^n\to \mathbb{R}$ is convex (Fréchet) differentiable, the gradient of which is Lipschitz continuous with a Lipschitz constant $L_f>0$ and $f\left(x\right)\ge 0$ for all $x\in C$.

($A_g$)

The function $g:{\mathbb{R}}^n\to \mathbb{R}$ is concave (Fréchet) differentiable, the gradient of which is Lipschitz continuous with a Lipschitz constant $L_g>0$ and there is $M>0$ such that $0<g\left(x\right)\le M$ for all $x\in C$.

We denote the solution set of (1) as

$$\mathcal{S}:=\left\{x^{*}\in C:\frac{f\left(x^{*}\right)}{g\left(x^{*}\right)}={\theta }^{*}\right\},$$

providing that the solution set $\mathcal{S}$ is nonempty.

It is worth mentioning that the nonlinear fractional optimization problem (1) covers a wide range of interesting practical applications, such as the optimization of risk-adjusted performance measures where the goal is to maximize objective functions that appear in the form of the Sharpe ratio. This problem can be reformulated as problem (1), which was studied by Chen et al. in [12]. As well as the financial field, problem (1) can also be found in the inventory routing problem in [13] and the environmental–economic power dispatch problem in [14]. To be specific, in [13], Archetti et al. formed a model that aimed to minimize objective functions in the form of logistic ratios, which related to the ratio of the total routing costs and the total distributed quantity. In [14], Chen et al. considered a minimization model for the ratio of the total fuel costs and the total emissions of power dispatch systems. Another example of an application of problem (1) is the recovery of sparse signals by minimizing the ratio of the ${\ell }_1$-norm and the Euclidean norm, which is a special case of problem (1) and was shown by Rahimi et al. in [15]. More comprehensive application overviews can also be found in [16,17,18].

There are some existing methods that can be used to solve the nonlinear fractional optimization problem (1). In the literature, one of the classical and popular methods for solving problem (1) is the so-called Dinkelbach’s algorithm [19], in which, for an arbitrary initial point $x_1\in C$, the iterate is defined by:

$${\displaystyle x_{n+1}\in \underset{x\in C}{argmin}[f\left(x\right)-{\theta }_{n}g\left(x\right)],}$$

(2)

for all $n\in \mathbb{N}$, where ${\theta }_n=\frac{f\left(x_n\right)}{g\left(x_n\right)}$. This method has been further developed by several authors (for instance, [20,21,22] and the references therein). The key feature of this method is that it exploits the benefits of the relationship between the nonlinear fractional programming problem and the nonlinear parametric programming problem [19,23]. The latter problem is presented in the form:

$${\displaystyle \underset{x\in C}{min}f\left(x\right)-{\theta }^{*}g\left(x\right),}$$

(3)

where ${\theta }^{*}\in \mathbb{R}$. It was stated in [19] that finding an optimal solution $x^{*}\in \mathcal{S}$ in problem (1) is essentially equivalent to finding an optimal solution to problem (3) when the optimal objective value of problem (3) is zero with ${\theta }^{*}=\frac{f\left(x^{*}\right)}{g\left(x^{*}\right)}$. However, the optimal solution to problem (3) is not necessarily known because solving each iteration in problem (2) requires high computational costs and cannot be performed easily in finite sub-iterations due to the additive structure of the objective function $f-{\theta }_{n}g$.

To overcome this limitation, it is natural to consider iterative algorithms that can perform the functions f and g separately. The first successful splitting method was proposed by Boţ and Csetnek [24], which had the form:

$${\displaystyle x_{n+1}\in \underset{x\in C}{argmin}\left[f\left(x\right)+\frac{1}{2{\eta }_n}{∥x-(x_n+{\eta }_n{\theta }_n\nabla g\left(x_n\right))∥}^2\right],}$$

(4)

where $x_1\in C$ is the initial point, ${\theta }_n=\frac{f\left(x_n\right)}{g\left(x_n\right)}$ is the step size and ${\eta }_n=\frac{1}{2L_g{\theta }_n}$. Since this representative starting point for dealing with the additive structure of $f-{\theta }_{n}g$ was proposed, several generalizations, variant settings and acceleration techniques have subsequently been investigated (for example, some recent contributions can be found in [7,8,25]). Notice that, when the constrained set C is not the whole space ${\mathbb{R}}^n$, method (4) requires the evaluation of the proximity operator of the sum ${\eta }_n(f+{\delta }_C)$ at point $x_n+{\eta }_n{\theta }_n\nabla g\left(x_n\right)$. However, it is well known that computing the proximity operators of additive structure functions can generally be fairly difficult since it can involve solving sub-problems with splitting-type algorithms (there are some further discussions in [26]).

To fulfill this practical limitation, we turn to the particular structure in which the function f is smooth and propose a type of projection gradient method to solve the nonlinear fractional optimization problem (1). The presented method allows us not only to compute the gradients of f, the gradient of g and the metric projection onto C separately but also to construct a nonincreasing step-size sequence ${\left\{{\eta }_n\right\}}_{n\ge 1}$. We prove that the sequence that is generated by the proposed method converges to a point in $\mathcal{S}$ and then subsequently derive the $\mathcal{O}(1/N)$ rate of convergence for the function value ${min}_{n=1,\dots ,N}{\theta }_n$ to the optimal value ${\theta }^{*}$. Finally, we perform numerical experiments to illustrate the convergence behaviors of the proposed method via various types of the considered nonlinear fractional programming problem.

The rest of this paper is organized as follows. In Section 2, we recall some preliminary tools and key propositions for proving convergence results. In Section 3, we present out adaptive projection gradient method and investigate its convergence properties. In Section 4, we present our numerical experiments. In the last section, we conclude the presented work.

2. Preliminaries

In this section, we recall some useful definitions and properties that are used in the subsequent sections to prove our convergence results.

Let C be a nonempty closed and convex set in ${\mathbb{R}}^n$. For every $x\in {\mathbb{R}}^n$, there exists a unique point $x^{*}\in C$, such that

$$\begin{array}{c}\hfill \parallel x-x^{*}\parallel \le \parallel x-y\parallel ,\end{array}$$

for every $y\in C$ ([27], Theorem 1.2.3). We call point $x^{*}$ a projection of x onto C and denote it as $P_C\left(x\right)$. The normal cone to C at $x\in {\mathbb{R}}^n$ is defined by

$$\begin{array}{c}\hfill N_C\left(x\right)=\{y\in {\mathbb{R}}^n:\langle y,z-x\rangle \le 0,\forall z\in C\}.\end{array}$$

Proposition 1.

([27], Lemma 1.2.9) For every $x\in {\mathbb{R}}^n$, the following statements are equivalent:

(i)  $y=P_C\left(x\right)$;

(ii) $y\in C$ and $x-y\in N_C\left(y\right)$.

Note that the following characterization of the metric projection holds true:

$$\begin{array}{c}\hfill \langle x-P_C\left(x\right),z-P_C\left(x\right)\rangle \le 0,\forall z\in C.\end{array}$$

The indicator function is defined as ${\delta }_C\left(x\right)=0$ when $x\in C$ and it is defined as ${\delta }_C\left(x\right)=\infty$ when $x\notin C$. It is worth mentioning that $\partial {\delta }_C=N_C$. The following two propositions play key roles in proving our convergence results.

Proposition 2.

([24], the descent lemma) Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be (Fréchet) differentiable with a gradient that is Lipschitz continuous with a Lipschitz constant $L\ge 0$. Then, for any $x,y\in {\mathbb{R}}^n$

$$f\left(y\right)\le f\left(x\right)+〈\nabla f\left(x\right),y-x〉+\frac{L}{2}{\parallel x-y\parallel }^2.$$

Proposition 3.

([28], Lemma 11) Let ${\left\{a_n\right\}}_{n\ge 1},{\left\{b_n\right\}}_{n\ge 1}$ and ${\left\{c_n\right\}}_{n\ge 1}$ be sequences of nonnegative real numbers, such that

$$\begin{array}{c}\hfill a_{n+1}+b_n\le a_n+c_n,\forall n\ge 1.\end{array}$$

If ${\sum }_{n\ge 1}{c}_n<\infty$, then ${lim}_{n\to \infty }{a}_n$ exists and ${\sum }_{n\ge 1}{b}_n<\infty$.

In order to show that sequence ${\left\{x_n\right\}}_{n\ge 1}$ converges to a point in $\mathcal{S}$, we need a variation statement of the Opial lemma, which is stated in the following proposition.

Proposition 4.

([29], Lemma 2) Let ${\left\{x_n\right\}}_{n\ge 1}$ be a sequence in ${\mathbb{R}}^n$ and ${\left\{{\beta }_n\right\}}_{n\ge 1}$ be a sequence of nonnegative real numbers. Let X be a nonempty subset of ${\mathbb{R}}^n$. Assume that the following conditions hold:

(i) ${\left\{x_n\right\}}_{n\ge 1}$ is bounded;

(ii) the cluster points of ${\left\{x_n\right\}}_{n\ge 1}$ belong to X;

(iii) it holds that

$$\begin{array}{c}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\parallel x_{n+1}{-x\parallel }^2+{\beta }_{n+1}\le {\parallel x_n-x\parallel }^2+{\beta }_n,\forall x\in X,\forall n\ge 1.\hfill \end{array}$$

Then, ${\left\{x_n\right\}}_{n\ge 1}$ converges to an element in X.

3. Algorithm and Convergence Results

In this section, we present our iterative method for solving the nonlinear fractional programming problem. We assume that the Lipschitz constants $L_f$ and $L_g$ are not equal to 0 simultaneously.

Algorithm 1: Adaptive projection gradient method.

Initialization: Choose an initial point $x_1\in C$, two real numbers $\underline{\eta }>0$ and   $0<a<1$. Set ${\theta }_1=\frac{f\left(x_1\right)}{g\left(x_1\right)}$ and $\hspace{1em}{\eta }_1=min\left\{\frac{g\left(x_1\right)}{M},\frac{a}{L_f+{\theta }_1{L}_g}\right\}$.  Step 1: For a current iterate $x_n\in C$ and a step size ${\eta }_n$ ($n=1,2,\dots$).  Set $${\theta }_n=\frac{f\left(x_n\right)}{g\left(x_n\right)},$$   and define $$x_{n+1}=P_C\left(x_n-{\eta }_n\nabla f\left(x_n\right)+{\eta }_n{\theta }_n\nabla g\left(x_n\right)\right).$$   Step 2: Update a step size ${\eta }_{n+1}$ as follows. If ${\eta }_n\le \underline{\eta }$, then set ${\eta }_{n+1}={\eta }_n$.   Otherwise, set $${\eta }_{n+1}=min\left\{\frac{{\eta }_{n}g\left(x_{n+1}\right)}{M},\frac{a}{L_f+{\theta }_{n+1}{L}_g}\right\}.$$   Let $n:=n+1$ and go to Step 1.

Remark 1.

(i) Since the iterate $x_n\in C$ for all $n\ge 1$, we obtain ${\theta }_n\ge {\theta }^{*}$ for all $n\ge 1$;

(ii) It is worth noting that when $n_0\in \mathbb{N}$ exists where ${\theta }_{n_0}=0$, then the iterate $x_{n_0}\in C$ is a solution of the fractional programming problem (1) since ${\theta }^{*}\ge 0$. Thus, in this work, we assume that Algorithm 1 does not terminate in any finite number of iterations $n\ge 1$.

Lemma 1.

The step-size sequence ${\left\{{\eta }_n\right\}}_{n\ge 1}$ is a positive and nonincreasing sequence. Furthermore, we also have that the limit ${lim}_{n\to \infty }{\eta }_n>0$.

Proof. 

We precede the proof of positivity of ${\left\{{\eta }_n\right\}}_{n\ge 1}$ by the induction statement. Since the value ${\theta }_1$ is nonnegative, it is clear that ${\eta }_1>0$. Then, we let k be a natural number, such that ${\eta }_k>0$. When ${\eta }_{k+1}=\frac{{\eta }_{k}g\left(x_{k+1}\right)}{M}$, then we obtain ${\eta }_k>0$ from the induction hypothesis, together with the positivities of $g\left(x_{k+1}\right)$ and M, that ${\eta }_{k+1}>0$. On the other hand, when ${\eta }_{k+1}=\frac{a}{L_f+{\theta }_{k+1}{L}_g}$, we obtain from the nonnegativity of ${\theta }_{k+1}$ that ${\eta }_{k+1}$ is also positive. Hence, we obtain that sequence ${\left\{{\eta }_n\right\}}_{n\ge 1}$ i a positive sequence.

Since $0<g\left(x_{n+1}\right)\le M$ for all $n\ge 0$, we obtain:

$${\eta }_{n+1}\le \frac{{\eta }_{n}g\left(x_{n+1}\right)}{M}\le {\eta }_n\hspace{1em}\hspace{1em}\forall n\ge 1.$$

This means that the sequence ${\left\{{\eta }_n\right\}}_{n\ge 1}$ is a nonincreasing sequence.

Next, we prove that the limit ${lim}_{n\to \infty }{\eta }_n>0$. When there is $n_0\ge 1$, such that ${\eta }_{n_0}\le \underline{\eta }$, then ${\eta }_n={\eta }_{n_0}$ for all $n\ge n_0$. This means that sequence ${\left\{{\eta }_n\right\}}_{n\ge n_0}$ is a constant sequence in which its value is equal to ${\eta }_{n_0}$ and hence, the limit ${lim}_{n\to \infty }{\eta }_n={\eta }_{n_0}>0$. When ${\eta }_n\ge \underline{\eta }$ for all $n\ge 1$, then $\underline{\eta }$ is the lower bound of the nonincreasing sequence ${\left\{{\eta }_n\right\}}_{n\ge 1}$, which implies that

$$\underset{n\to \infty }{lim}{\eta }_n=\underset{n\ge 1}{inf}{\eta }_n\ge \underline{\eta }>0.$$

This completes the proof. □

Remark 2.

Due to the fact that the limit of sequence ${\left\{{\eta }_n\right\}}_{n\ge 1}$ is more than 0, we have that ${\sum }_{n\ge 1}{\eta }_n=\infty$.

For the sake of simplicity, we denote

$${\Gamma }_n\left(x\right):={\parallel x_n-x\parallel }^2+2{\eta }_n{\theta }_{n}M\hspace{1em}\hspace{1em}\forall x\in C, \forall n\ge 1.$$

Our analysis of the proposed adaptive projection gradient method is based on the three-term inequality that is presented in the following lemma.

Lemma 2.

For every $n\ge 1$ and $x\in C$, we have

$${\Gamma }_{n+1}\left(x\right)\le {\Gamma }_n\left(x\right)-2{\eta }_{n}g\left(x\right)\left({\theta }_n-\frac{f\left(x\right)}{g\left(x\right)}\right)-(1-a){\parallel x_{n+1}-x_n\parallel }^2.$$

Proof. 

Let $n\ge 1$ and $x\in C$. Firstly, we note from the characterization of the metric projection $P_C$ that

$$〈x_n-{\eta }_n\nabla f\left(x_n\right)+{\eta }_n{\theta }_n\nabla g\left(x_n\right)-x_{n+1},x-x_{n+1}〉\le 0,$$

which is

$$〈x_n-x_{n+1},x-x_{n+1}〉\le {\eta }_n〈\nabla f\left(x_n\right),x-x_{n+1}〉+{\eta }_n{\theta }_n〈-\nabla g\left(x_n\right),x-x_{n+1}〉.$$

(5)

We note that

$$2〈x_n-x_{n+1},x-x_{n+1}〉=\parallel x_{n+1}{-x\parallel }^2-\parallel x_n{-x\parallel }^2+{\parallel x_{n+1}-x_n\parallel }^2,$$

so it follows that (5) becomes

$$\begin{array}{ccc}\hfill \parallel x_{n+1}{-x\parallel }^2& \le & \parallel x_n{-x\parallel }^2-{\parallel x_{n+1}-x_n\parallel }^2+2{\eta }_n〈\nabla f\left(x_n\right),x-x_{n+1}〉\hfill \\ & & +2{\eta }_n{\theta }_n〈-\nabla g\left(x_n\right),x-x_{n+1}〉.\hfill \end{array}$$

(6)

By using the convexity of f and applying the descent lemma (Proposition 2) to f, we obtain

$$\begin{array}{ccc}\hfill 2{\eta }_n〈\nabla f\left(x_n\right),x-x_{n+1}〉& =& 2{\eta }_n〈\nabla f\left(x_n\right),x-x_n〉+2{\eta }_n〈\nabla f\left(x_n\right),x_n-x_{n+1}〉\hfill \\ & \le & 2{\eta }_n(f\left(x\right)-f\left(x_n\right))\hfill \\ & & +2{\eta }_n\left(f\left(x_n\right)-f\left(x_{n+1}\right)+\frac{L_f}{2}{\parallel x_{n+1}-x_n\parallel }^2\right)\hfill \\ & =& 2{\eta }_n(f\left(x\right)-f\left(x_{n+1}\right))+{\eta }_n{L}_f{\parallel x_{n+1}-x_n\parallel }^2.\hfill \end{array}$$

(7)

Since the function g is concave, we know that $-g$ is convex. Furthermore, we note that the function $-g$ is differentiable with a Lipschitz gradient with the constant $L_g$. By applying these facts and the descent lemma to $-g$, we obtain

$$\begin{array}{ccc}\hfill 2{\eta }_n{\theta }_n〈-\nabla g\left(x_n\right),x-x_{n+1}〉& =& 2{\eta }_n{\theta }_n〈-\nabla g\left(x_n\right),x-x_n〉+2{\eta }_n{\theta }_n〈-\nabla g\left(x_n\right),x_n-x_{n+1}〉\hfill \\ & \le & 2{\eta }_n{\theta }_n(-g\left(x\right)+g\left(x_n\right))\hfill \\ & & +2{\eta }_n{\theta }_n\left(-g\left(x_n\right)+g\left(x_{n+1}\right)+\frac{L_g}{2}{\parallel x_{n+1}-x_n\parallel }^2\right)\hfill \\ & =& 2{\eta }_n{\theta }_n(-g\left(x\right)+g\left(x_{n+1}\right))+{\eta }_n{\theta }_n{L}_g{\parallel x_{n+1}-x_n\parallel }^2.\hfill \end{array}$$

(8)

By combining (6), (7) and (8), we obtain

$$\begin{array}{ccc}\hfill \parallel x_{n+1}{-x\parallel }^2& \le & \parallel x_n{-x\parallel }^2-(1-{\eta }_n(L_f+{\theta }_n{L}_g)){\parallel x_{n+1}-x_n\parallel }^2\hfill \\ & & +2{\eta }_n(f\left(x\right)-f\left(x_{n+1}\right))+2{\eta }_n{\theta }_n(-g\left(x\right)+g\left(x_{n+1}\right))\hfill \\ & \le & \parallel x_n{-x\parallel }^2-(1-{\eta }_n(L_f+{\theta }_n{L}_g)){\parallel x_{n+1}-x_n\parallel }^2\hfill \\ & & +2g\left(x\right){\eta }_n\left(\frac{f\left(x\right)}{g\left(x\right)}-{\theta }_n\right)+2{\eta }_n({\theta }_{n}g\left(x_{n+1}\right)-f\left(x_{n+1}\right)),\hfill \end{array}$$

It follows from the fact that $g\left(x_n\right)\le M$ for all $n\ge 1$ that

$$\begin{array}{ccc}\hfill \parallel x_{n+1}{-x\parallel }^2+2{\eta }_{n}f\left(x_{n+1}\right)& \le & \parallel x_n{-x\parallel }^2+2{\eta }_n{\theta }_{n}M-2g\left(x\right){\eta }_n\left({\theta }_n-\frac{f\left(x\right)}{g\left(x\right)}\right)\hfill \\ & & -(1-{\eta }_n(L_f+{\theta }_n{L}_g)){\parallel x_{n+1}-x_n\parallel }^2.\hfill \end{array}$$

(9)

We note that $0<1-a\le 1-{\eta }_n(L_f+{\theta }_n{L}_g)$ for all $n\ge 1$. Moreover, we also note from the fact that ${\eta }_{n+1}\le \frac{{\eta }_{n}g\left(x_{n+1}\right)}{M}$ that

$$\begin{array}{c}\hfill 2{\eta }_{n}f\left(x_{n+1}\right)\ge 2{\eta }_{n+1}\frac{f\left(x_{n+1}\right)}{g\left(x_{n+1}\right)}M=2{\eta }_{n+1}{\theta }_{n+1}M\hspace{1em}\hspace{1em}\forall n\ge 1.\end{array}$$

Thus, using these derived relationships, we obtain that inequality (9) becomes

$$\begin{array}{ccc}\hfill \parallel x_{n+1}{-x\parallel }^2+2{\eta }_{n+1}{\theta }_{n+1}M& \le & \parallel x_n{-x\parallel }^2+2{\eta }_n{\theta }_{n}M-2g\left(x\right){\eta }_n\left({\theta }_n-\frac{f\left(x\right)}{g\left(x\right)}\right)\hfill \\ & & -(1-a)\parallel x_{n+1}-x_n{\parallel }^2,\hfill \end{array}$$

and hence

$$\begin{array}{ccc}\hfill {\Gamma }_{n+1}\left(x\right)& \le & {\Gamma }_n\left(x\right)-2g\left(x\right){\eta }_n\left({\theta }_n-\frac{f\left(x\right)}{g\left(x\right)}\right)-(1-a){\parallel x_{n+1}-x_n\parallel }^2,\hfill \end{array}$$

as required. □

Thus, we prove that sequence ${\left\{x_n\right\}}_{n\ge 1}$ converges to a solution of the considered problem.

Theorem 1.

The sequence ${\left\{x_n\right\}}_{n\ge 1}$ that is generated by Algorithm 1 converges to an optimal solution $x^{*}$ within the solution set $\mathcal{S}$.

Proof. 

We let $x^{*}\in \mathcal{S}$. By plugging $x:=x^{*}$ into Lemma 2, we obtain

$$\begin{array}{ccc}\hfill {\Gamma }_{n+1}\left(x^{*}\right)& \le & {\Gamma }_n\left(x^{*}\right)-2g\left(x^{*}\right){\eta }_n\left({\theta }_n-{\theta }^{*}\right)-(1-a){\parallel x_{n+1}-x_n\parallel }^2,\hspace{1em}\hspace{1em}\forall n\ge 1.\hfill \end{array}$$

We note that ${\theta }_n\ge {\theta }^{*}$ for all $n\ge 1$. By applying Proposition 3 with $a_n:={\Gamma }_n\left(x^{*}\right)$, $b_n:=2g\left(x^{*}\right){\eta }_n\left({\theta }_n-{\theta }^{*}\right)+(1-a){\parallel x_{n+1}-x_n\parallel }^2$ and $c_n:=0$, we find that the limit ${lim}_{n\to \infty }{\Gamma }_n\left(x^{*}\right)$ exists and that the series

$$\sum _{n\ge 1}{\eta }_n\left({\theta }_n-{\theta }^{*}\right)<\infty \hspace{1em}\mathrm{and}\hspace{1em}\sum _{n\ge 1}{\parallel x_{n+1}-x_n\parallel }^2<\infty .$$

The last two results imply that

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\eta }_n\left({\theta }_n-{\theta }^{*}\right)=\underset{n\to \infty }{lim}(x_{n+1}-x_n)=0.\end{array}$$

Recalling that ${lim}_{n\to \infty }{\eta }_n>0$, we obtain

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}\left({\theta }_n-{\theta }^{*}\right)=\frac{{lim}_{n\to \infty }{\eta }_n\left({\theta }_n-{\theta }^{*}\right)}{{lim}_{n\to \infty }{\eta }_n}=0,\end{array}$$

and so

$$\begin{array}{c}\hfill \underset{n\to \infty }{lim}{\theta }_n={\theta }^{*}.\end{array}$$

(10)

Since the limits ${lim}_{n\to \infty }{\Gamma }_n\left(x^{*}\right)$ and ${lim}_{n\to \infty }{\eta }_n{\theta }_n$ both exist, we also note that sequence $\{\parallel x_n-x^{*}{\parallel \}}_{n\ge 1}$ is convergent and so, sequence ${\left\{x_n\right\}}_{n\ge 1}$ is bounded. This means that condition (i) of Proposition 4 holds true.

Now, let $\overline{x}$ be a cluster point of ${\left\{x_n\right\}}_{n\ge 1}$. Then, there exists a subsequence ${\left\{x_{n_k}\right\}}_{k\ge 1}$ of ${\left\{x_n\right\}}_{n\ge 1}$, such that $x_{n_k}\to \overline{x}\in {\mathbb{R}}^n$. We then prove that $\overline{x}\in \mathcal{S}$. We note from the relationship between the metric projection and the normal cone in Proposition 1 that

$$\begin{array}{c}\hfill x_{n_k}-{\eta }_{n_k}\nabla f\left(x_{n_k}\right)+{\eta }_{n_k}{\theta }_{n_k}\nabla g\left(x_{n_k}\right)-x_{n_k+1}\in N_C\left(x_{n_k+1}\right),\end{array}$$

and by using the property of the normal cone, we obtain

$$\begin{array}{c}\hfill \frac{(x_{n_k}-x_{n_k+1})}{{\eta }_{n_k}}-\nabla f\left(x_{n_k}\right)+{\theta }_{n_k}\nabla g\left(x_{n_k}\right)\in \frac{1}{{\eta }_{n_k}}{N}_C\left(x_{n_k+1}\right)\subset N_C\left(x_{n_k+1}\right).\end{array}$$

By adding the terms $\nabla f\left(x_{n_k+1}\right)$ and $-{\theta }^{*}\nabla g\left(x_{n_k+1}\right)$ into both sides of the above relationship, we obtain

$$\begin{array}{ccc}\hfill \frac{(x_{n_k}-x_{n_k+1})}{{\eta }_{n_k}}& +& (\nabla f\left(x_{n_k+1}\right)-\nabla f\left(x_{n_k}\right))\hfill \\ & +& ({\theta }_{n_k}-{\theta }^{*})\nabla g\left(x_{n_k}\right)+{\theta }^{*}(\nabla g\left(x_{n_k}\right)-\nabla g\left(x_{n_k+1}\right))\hfill \\ & \in & \nabla f\left(x_{n_k+1}\right)-{\theta }^{*}\nabla g\left(x_{n_k+1}\right)+N_C\left(x_{n_k+1}\right)\hfill \\ & =& \partial (f-{\theta }^{*}g+{\delta }_C)\left(x_{n_k+1}\right).\hfill \end{array}$$

(11)

We then consider the left-hand side of (11) as follows. We note that

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}∥\frac{x_{n_k}-x_{n_k+1}}{{\eta }_{n_k}}∥=\frac{{lim}_{k\to \infty }\parallel x_{n_k}-x_{n_k+1}\parallel }{{lim}_{k\to \infty }{\eta }_{n_k}}=0.\end{array}$$

(12)

Moreover, the $L_f$-smoothness of f and $L_g$-smoothness of g yields that

$$\begin{array}{ccc}\hfill 0& \le & \parallel (\nabla f\left(x_{n_k+1}\right)-\nabla f\left(x_{n_k}\right))+{\theta }^{*}(\nabla g\left(x_{n_k}\right)-\nabla g\left(x_{n_k+1}\right))\parallel \hfill \\ & \le & (L_f+{\theta }^{*}{L}_g)\parallel x_{n_k+1}-x_{n_k}\parallel \to 0.\hfill \end{array}$$

(13)

Furthermore, we note from the boundedness of sequence ${\{\nabla g\left(x_n\right)\}}_{n\ge 1}$ and (10) that

$$\begin{array}{c}\hfill 0\le ({\theta }_{n_k}-{\theta }^{*})\parallel \nabla g\left(x_{n_k}\right)\parallel \to 0.\end{array}$$

(14)

By invoking (12), (13) and (14), we obtain that the limit $k\to \infty$ of the left-hand side is equal to 0. Thus, by applying the closedness property of the convex subdifferential, we derive that

$$\begin{array}{c}\hfill 0\in \partial (f-{\theta }^{*}g+{\delta }_C)\left(\overline{x}\right),\end{array}$$

hence, under the necessary and sufficient optimality conditions for convex constrained optimization, we obtain that

$$\begin{array}{c}\hfill f\left(x\right)-{\theta }^{*}g\left(x\right)\ge f\left(\overline{x}\right)-{\theta }^{*}g\left(\overline{x}\right),\hspace{1em}\hspace{1em}\forall x\in C.\end{array}$$

Since $x^{*}\in \mathcal{S}\subset C$, we obtain

$$\begin{array}{c}\hfill 0=f\left(x^{*}\right)-{\theta }^{*}g\left(x^{*}\right)\ge f\left(\overline{x}\right)-{\theta }^{*}g\left(\overline{x}\right),\end{array}$$

that is

$$\begin{array}{c}\hfill {\theta }^{*}\ge \frac{f\left(\overline{x}\right)}{g\left(\overline{x}\right)},\end{array}$$

which implies that $\overline{x}\in \mathcal{S}$. This means that the cluster point $\overline{x}$ of the sequence ${\left\{x_n\right\}}_{n\ge 1}$ belongs to the solution set $\mathcal{S}$, in which condition (ii) of Proposition 4 is satisfied.

Finally, it is noted from Lemma 2 that

$$\begin{array}{ccc}\hfill \parallel x_{n+1}-x^{*}{\parallel }^2+2{\eta }_{n+1}{\theta }_{n+1}M& \le & \parallel x_n-x^{*}{\parallel }^2+2{\eta }_n{\theta }_{n}M,\hspace{1em}\hspace{1em}\forall x^{*}\in \mathcal{S},\forall n\ge 1,\hfill \end{array}$$

and by setting ${\beta }_n:=2{\eta }_n{\theta }_{n}M$ for all $n\ge 1$, we obtain that condition (iii) of Proposition 4 is also satisfied. Hence, we conclude that sequence ${\left\{x_n\right\}}_{n\ge 1}$ converges to a point in $\mathcal{S}$. □

Remark 3.

It is worth noting from Theorem 1 that the sequence of the function values ${\left\{{\theta }_n\right\}}_{n\ge 1}$ converges to the optimal value ${\theta }^{*}$ of the considered problem (1).

We can also estimate the rate of convergence of a sequence that is generated by Algorithm 1. In fact, we find that the distance from ${min}_{n=1,\dots ,N}{\theta }_n$ to ${\theta }^{*}$ is bounded above by a constant factor of $1/N$, as presented in the following theorem.

Theorem 2.

Let ${\left\{x_n\right\}}_{n\ge 1}$ be a sequence generated by Algorithm 1 and $x^{*}\in \mathcal{S}$ and ${\theta }^{*}$ be an optimal solution and the optimal value of problem (1), respectively. Then, for any $N=1,2,\dots$, the following estimates can be obtained:

If there is $n_0\ge 1$ such that ${\eta }_{n_0}\le \underline{\eta }$, then

$$\begin{array}{c}\hfill \underset{n=1,\dots ,N}{min}{\theta }_n-{\theta }^{*}\le \frac{1}{N}\left(\frac{\parallel x_1-x^{*}{\parallel }^2+2{\eta }_1{\theta }_{1}M}{2g\left(x^{*}\right){\eta }_{n_0}}\right).\end{array}$$

Otherwise

$$\begin{array}{c}\hfill \underset{n=1,\dots ,N}{min}{\theta }_n-{\theta }^{*}\le \frac{1}{N}\left(\frac{\parallel x_1-x^{*}{\parallel }^2+2{\eta }_1{\theta }_{1}M}{2g\left(x^{*}\right)\underline{\eta }}\right).\end{array}$$

Proof. 

We prove the upper bounds for the series of the different ${\theta }_n-{\theta }^{*}$ when ${\eta }_n\ge \underline{\eta }$ for all $n\ge 1$. The other case can also be obtained using the same lines of proof. Then, by taking $x:=x^{*}\in \mathcal{S}$ from Lemma 2, we obtain for all $n\ge 1$

$$\begin{array}{c}\hfill {\Gamma }_n\left(x^{*}\right)-{\Gamma }_{n+1}\left(x^{*}\right)\ge 2g\left(x^{*}\right){\eta }_n\left({\theta }_n-{\theta }^{*}\right)\ge 2g\left(x^{*}\right)\underline{\eta }\left({\theta }_n-{\theta }^{*}\right).\end{array}$$

By summing this inequality for all n from 1 to N and discarding the nonpositive term on the right-hand side, we obtain

$$\begin{array}{c}\hfill {\Gamma }_1\left(x^{*}\right)\ge 2g\left(x^{*}\right)\underline{\eta }\sum _{n=1}^N\left({\theta }_n-{\theta }^{*}\right),\end{array}$$

which yields that

$$\begin{array}{c}\hfill N\left(\underset{n=1,\dots ,N}{min}{\theta }_n-{\theta }^{*}\right)\le \sum _{n=1}^N({\theta }_n-{\theta }^{*})\le \frac{\parallel x_1-x^{*}{\parallel }^2+2{\eta }_1{\theta }_{1}M}{2g\left(x^{*}\right)\underline{\eta }},\end{array}$$

and hence

$$\begin{array}{c}\hfill \underset{n=1,\dots ,N}{min}{\theta }_n-{\theta }^{*}\le \frac{1}{N}\left(\frac{\parallel x_1-x^{*}{\parallel }^2+2{\eta }_1{\theta }_{1}M}{2g\left(x^{*}\right)\underline{\eta }}\right),\end{array}$$

as required. □

4. Numerical Examples

In this section, we present some numerical behaviors of the proposed APGM (Algorithm 1) in the nonlinear fractional programming problem. We implemented the numerical codes in MATLAB and performed all computations on a MacBook Pro 13-inch 2019 with a 2.4 GHz Intel Core i5 processor and an 8 GB/2133 MHz LPDDR3 memory.

4.1. Convex–Concave Fractional Programming

We start this subsection by considering the following one-dimensional convex–concave fractional programming

$${\displaystyle \underset{x\in [0,2]}{min}\frac{x^2+1}{1.1-{(x-1)}^2}.}$$

(15)

Note that the functions $f\left(x\right)=x^2+1$ and $g\left(x\right)=1.1-{(x-1)}^2$ are convex differentiable and concave differentiable, respectively, with a Lipschitz constant gradient of 2, i.e., $L_f=L_g=2>0$. Moreover, it can be seen that $f\left(x\right)\ge 1\ge 0$ and $0<0.1\le g\left(x\right)\le 1.1=:M$ for all $x\in [0,2]$. This means that the assumptions ($A_f$) and ($A_g$) are satisfied. Thus, this considered problem fits into the framework of the fractional programming problem (1).

In our numerical experiments, we explored the influence of APGM on various values of the initial point $x_1\in [0,2]$ and various values of the parameter $a\in (0,1)$, which were compared to the performance of the proximal gradient method (PGM) for convex–concave fractional programming that was proposed in [30] (Algorithm 6). We set the lower bound as $\underline{\eta }={10}^{-10}$. It is worth mentioning that the optimal solution and the optimal value for this problem are approximately 0.5945 and 1.4466, respectively. To demonstrate the performance of APGM and PGM, we explored the behavior of the iterate $x_n$, function value ${\theta }_n=\frac{f\left(x_n\right)}{g\left(x_n\right)}$ and step size ${\eta }_n$ for 50 iterations, as shown in Figure 1 and Figure 2, respectively.

In Figure 1, it can be seen that in spite of using different values of the initial point $x_1$, both PGM and APGM converged to their approximate solutions with various values of the parameter a. To be precise, it is worth observing that for any case of $x_1$, PGM and APGM both converged to their solutions within 10 iterations with $a=0.9$ and $a=0.99$. Moreover, it can be seen that, in some situations, APGM managed to obtain approximate solutions that were closer to the optimal solution than those obtained by PGM and that the best overall performance was delivered when using $a=0.99$. These findings underline the advantages of our proposed APGM.

According to Figure 2, the values of ${\eta }_n$ tended to be nondecreasing throughout the considered iterations when performed by PGM, meanwhile the values of ${\eta }_n$ rapidly decreased for all values of the parameter a when performed by APGM. This numerical behavior of APGM demonstrated its theoretical perspective that was stated in Lemma 1.

4.2. Convex–Concave Fractional Programming with Linear Constraints

In this subsection, we present our considerations of the following convex–concave fractional programming with linear constraints:

$$\begin{array}{ccc}\hfill minimize& & \frac{{0.5\parallel x\parallel }^2}{10-{0.5\parallel x\parallel }^2}\hfill \\ \hfill \mathrm{subject}\mathrm{to}& & \mathbf{A}x\le \mathbf{b}\hfill \\ & & x\in {[0,1]}^2\hfill \end{array}$$

(16)

where

$$\begin{array}{c}\hfill \mathbf{A}:=\left[\begin{array}{cc}-2& 1\\ 4& -3\\ -1& -2\\ -3& -1\\ 1& 1\end{array}\right]\hspace{1em}\mathrm{and}\hspace{1em}\mathbf{b}:=\left[\begin{array}{c}2\\ 4\\ 2\\ 4\\ 2\end{array}\right].\end{array}$$

By invoking the convex differentiable proximity function in which the weighted sum of the squared values of the constraints is minimal, we obtain

$$\begin{array}{c}\hfill h\left(x\right):=0.5\sum _{i=1}^5\frac{1}{5}{(max\{\langle \mathbf{A}(i,:),x\rangle -\mathbf{b}\left(i\right),0\})}^2,\hspace{1em}\hspace{1em}\forall x\in {\mathbb{R}}^2,\end{array}$$

so we can consider problem (16) as the following nonlinear convex–concave fractional programming:

$$\begin{array}{c}\hfill \underset{x\in {[0,1]}^2}{min}\frac{{0.5\parallel x\parallel }^2+0.5{\sum }_{i=1}^5\frac{1}{5}{(max\{\langle \mathbf{A}(i,:),x\rangle -\mathbf{b}\left(i\right),0\})}^2}{10-{0.5\parallel x\parallel }^2}.\end{array}$$

(17)

Note that function $f\left(x\right)={0.5\parallel x\parallel }^2+0.5{\sum }_{i=1}^5\frac{1}{5}{(max\{\langle \mathbf{A}(i,:),x\rangle -\mathbf{b}\left(i\right),0\})}^2$ is a convex differentiable with a gradient of the Lipschitz constant $L_f=46>0$. Moreover, we observe that $f\left(x\right)\ge 0$ for all $x\in {[0,1]}^2$, which shows that assumption ($A_f$) is satisfied. Furthermore, we observe that function $g\left(x\right)=10-{0.5\parallel x\parallel }^2$ is a concave differentiable with a gradient of the Lipschitz constant $L_g=1>0$ and we note that $0<9\le g\left(x\right)\le 10=:M$ for all $x\in {[0,1]}^2$. This means that assumption ($A_g$) is also satisfied. Thus, the considered problem fits into the framework of the fractional programming problem (1).

In the same fashion as the previous subsection, we now present the numerical experiment that illustrated the influence of APGM on various values of the initial point $x_1\in {[0,1]}^2$ and various values of the parameter $a\in (0,1)$. We set the the lower bound as $\underline{\eta }={10}^{-10}$. The behavior of the iterate $x_n$, function value ${\theta }_n$ and step-size ${\eta }_n$ for 500 iterations are presented in Figure 3, Figure 4 and Figure 5, respectively. It is worth mentioning that we did not include PGM ([30], Algorithm 6) in this experiment because this method was not appropriate for solving problem (17) as the computation of the proximal operator of the sum of the convex functions required the subproblem in each iteration (n) to be solved. Again, this demonstrates the advantages of our proposed APGM.

In Figure 3, it can be seen that for all different values of $x_1$, APGM converged to their approximate solutions for all values of a. Moreover, it can also be seen that $a=0.99$ yielded the fastest overall convergence for every value of $x_1$. In particular, by using $x_1={(0.1,0.1)}^{\top }$ with $a=0.9$, APGM converged within only 200 iterations.

Figure 4 shows that the function values ${\theta }_n$ of APGM for all values of $x_1$ and a tended to decrease to the optimal value. We noticed that the best overall performance was observed with $a=0.99$. Moreover, it can be seen that by using $x_1={(0.1,0.1)}^{\top }$ with $a=0.7,0.9$ and $0.99$, the function values of APGM achieved their lowest values within only 200 iterations.

According to Figure 5, it can be seen that the values of ${\eta }_n$ appeared to be nonincreasing throughout the 500 iterations for all values of the parameter a when performed by APGM. This numerical behavior of APGM again demonstrates its theoretical perspective that was stated in Lemma 1. Moreover, we observed that the greatest value of ${\eta }_n$ was obtained with $a=0.99$, while the lowest value of ${\eta }_n$ occurred with $a=0.1$.

4.3. Quadratic Fractional Programming

In this subsection, we present our considerations of quadratic fractional programming in the following form ([31], Problem (23)):

$$\begin{array}{c}\hfill \underset{x\in {[1,3]}^5}{min}\frac{\langle x,\mathbf{A}x\rangle +\langle \mathbf{b},x\rangle +c}{\langle \mathbf{d},x\rangle +r}\end{array}$$

(18)

where

$$\begin{array}{c}\hfill \mathbf{A}:=\left[\begin{array}{ccccc}5& -1& 2& 0& 2\\ -1& 6& -1& 3& 0\\ 2& -1& 3& 0& 1\\ 0& 3& 0& 5& 0\\ 2& 0& 1& 0& 4\end{array}\right],\hspace{1em}\hspace{1em}\mathbf{b}:=\left[\begin{array}{c}1\\ 2\\ -1\\ -2\\ 1\end{array}\right],\hspace{1em}\hspace{1em}\mathbf{d}:=\left[\begin{array}{c}1\\ 0\\ -1\\ 0\\ 1\end{array}\right],\end{array}$$

$c:=-2$ and $r:=20$. Note that function $f\left(x\right)=\langle x,\mathbf{A}x\rangle +\langle \mathbf{b},x\rangle +c$ is a convex differentiable with a gradient of the Lipschitz constant $L_f=2\parallel M\parallel >0$. We observe that $f\left(x\right)\ge 34>0$ for all $x\in {[1,3]}^5$. This means that assumption ($A_f$) is satisfied. Furthermore, function $g\left(x\right)=\langle \mathbf{d},x\rangle +r$ is an affine (i.e., concave) differentiable with a gradient of the Lipschitz constant $L_g=0$. We note that $0<21\le g\left(x\right)\le 23=:M$ for all $x\in {[1,3]}^5$, which means that assumption ($A_g$) is also satisfied. Thus, the considered problem fits into the framework of the fractional programming problem (1).

In this numerical experiment, we explored the influence of APGM on the initial point $x_1:={(3,1.5,2,1.5,2)}^{\top }$ with different values of parameter $a\in (0,1)$. We set the lower bound as $\underline{\eta }={10}^{-10}$. We compared the performance of APGM to that of the proximity gradient–sub-gradient algorithm (PGSA) for fractional programming, which was proposed in [8] (Algorithm 1). The behavior of the iterate $x_n$ and function value ${\theta }_n$ are illustrated in

Table 1. The behavior of the iterate $x_n$ and function value ${\theta }_n$ when performed by APGM and PGSA with various values for a.

a	n	APGM		PGSA ([8], Algorithm 1)
		${\mathit{x}}_{\mathit{n}}$	${\mathit{\theta }}_{\mathit{n}}$	${\mathit{x}}_{\mathit{n}}$	${\mathit{\theta }}_{\mathit{n}}$
$0.6$	1	${(3,1.5,2,1.5,2)}^{\top }$	6.6630	${(3,1.5,2,1.5,2)}^{\top }$	6.6630
	2	${(1.7758,1,1,1,1.1365)}^{\top }$	2.4070	${(1.7758,1,1,1,1.1365)}^{\top }$	2.4070
	3	${(1.0605,1,1,1,1)}^{\top }$	1.6641	${(1.0250,1,1,1,1)}^{\top }$	1.6375
	4	${(1,1,1,1,1)}^{\top }$	1.6190	${(1,1,1,1,1)}^{\top }$	1.6190
$0.7$	1	${(3,1.5,2,1.5,2)}^{\top }$	6.6630	${(3,1.5,2,1.5,2)}^{\top }$	6.6630
	2	${(1.5717,1,1,1,1)}^{\top }$	2.1025	${(1.5717,1,1,1,1)}^{\top }$	2.1025
	3	${(1,1,1,1,1)}^{\top }$	1.6190	${(1,1,1,1,1)}^{\top }$	1.6190
$0.9$	1	${(3,1.5,2,1.5,2)}^{\top }$	6.6630	${(3,1.5,2,1.5,2)}^{\top }$	6.6630
	2	${(1.1637,1,1,1,1)}^{\top }$	1.7443	${(1.1637,1,1,1,1)}^{\top }$	1.7443
	3	${(1,1,1,1,1)}^{\top }$	1.6190	${(1,1,1,1,1)}^{\top }$	1.6190
$0.99$	1	${(3,1.5,2,1.5,2)}^{\top }$	6.6630	${(3,1.5,2,1.5,2)}^{\top }$	6.6630
	2	${(1,1,1,1,1)}^{\top }$	1.6190	${(1,1,1,1,1)}^{\top }$	1.6190

.

It can be observed from Table 1 that both methods yielded almost the exact same results throughout the considered iterations. This was probably caused by the simplicity of the affine function g, which had a gradient of the Lipschitz constant $L_g=0$. However, note that APGM and PGSA were studied in different settings, yet both methods achieved the same results.

5. Conclusions

In this paper, we presented our so-called adaptive projection gradient method for solving the nonlinear fractional optimization problem in which the numerator is smooth convex and the denominator is smooth concave. An interesting property that distinguished the proposed method from some existing methods was the nonincreasing property of the step-size sequence, which was stated in Lemma 1. We proved that the sequence of iterates that was generated by the proposed method converged to a solution for the considered problem. Finally, we also presented numerical experiments to illustrate the efficiency of the proposed method by applying APGM to solve three different types of the nonlinear fractional programming problem.