A Subspace Derivative-Free Conjugate Gradient Method for Solving Nonlinear Monotone Equations with Convex Constraints

Abstract

We propose a novel subspace derivative-free conjugate gradient method for solving large-scale nonlinear monotone equations with convex constraints. At each iteration, the search direction is constructed by minimizing a quadratic model within a subspace spanned by the current negative function value vector and the two most recent search directions. The algorithm incorporates a hyperplane projection technique to generate feasible iterative points. Under reasonable assumptions, we establish the global convergence and R-linear convergence rate of the proposed method. Extensive numerical experiments on benchmark problems demonstrate that the new algorithm significantly outperforms state-of-the-art derivative-free methods in terms of number of iterations, function evaluations, and CPU time. The results confirm the efficiency and robustness of the proposed approach for solving large-scale monotone systems.

1. Introduction

Nonlinear equations have numerous practical applications, such as motion control problems [1], neural networks [2], compressive sensing [3], image restoration [4], etc. Many related issues can be transformed into

$$F\left(x\right)=0,x\in \Omega ,$$

(1)

where $\Omega \subset {\mathbb{R}}^n$ stands for a non-empty closed convex set, and $F:{\mathbb{R}}^n\to {\mathbb{R}}^n$ is a continuous mapping fulfilling the monotonicity condition

$$⟨F\left(x\right)-F\left(y\right),x-y⟩\ge 0,\forall x,y\in {\mathbb{R}}^n.$$

(2)

Among existing methods for problem (1), Newton methods [5,6], quasi-Newton methods [7,8], Levenberg–Marquardt methods and its variants [9,10,11], and trust region methods [12] have long attracted widespread attention from researchers due to their local superlinear convergence properties. Meanwhile, these methods require solving linear systems involving the Jacobian matrix or its approximations at each iteration, which can lead to high computational costs when dealing with large-scale nonlinear systems of equations. In practice, large-scale variants of these methods, including inexact or matrix-free Newton methods and limited-memory quasi-Newton approaches, effectively mitigate these computational costs, enabling their application to large-scale problems.

Among various derivative-free methods, subspace techniques have garnered significant interest as they can effectively balance computational cost and convergence performance. Porcelli and Toint [13] proposed a structured version of the derivative-free stochastic pattern search optimization algorithm. Hare et al. [14] provided a theoretical basis for the efficiency advantages of low-dimensional random sampling from the viewpoint of expected descent. Chen et al. [15] combined a more accurate Q-fully quadratic model with a random subspace trust region framework and analyzed its convergence rate analysis. Kimiaei et al. [16], focusing on nonlinear monotone equations, further proposed the subspace inertial line search algorithm, which improved the efficiency of solving such equations by dynamically updating the subspace structure. Xie and Yuan [17] proposed a two-dimensional-model-based subspace method, combining the two-dimensional quadratic interpolation model with the trust region technique to solve large-scale derivative-free optimization problems. In addition, subspace methods have also been widely applied to various scenarios [18,19,20].

In the research progress of derivative-free conjugate gradient methods, various innovative approaches have emerged, significantly expanding the technical pathways in this field [21,22]. As an important branch of such methods, the subspace minimization conjugate gradient method [23] has attracted considerable attention for its excellent numerical performance. It generates search directions by minimizing a quadratic approximation model of the objective function at the current point. In recent years, the application scope of this method has expanded. For example, Tang et al. [24] proposed a three-term subspace minimization conjugate gradient method, which generates the search direction by constructing a three-dimensional subspace and minimizing the quadratic approximation function, significantly improving the ability to solve large-scale nonlinear monotone systems of equations. Y, Arfat et al. [25,26] developed hybrid steepest-descent methods for convex minimization over split equilibrium problems and fixed point sets of multivalued mappings.

Projection methods, especially hyperplane projection, are widely used for convex-constrained monotone equations as they ensure feasible iterates and global convergence without derivative information. Conjugate gradient directions are efficient for large-scale problems due to low memory cost and fast convergence. Combining subspace CG directions with hyperplane projection inherits both merits: strong descent for acceleration and projection for feasibility and convergence. This motivates our work.

Inspired by the subspace conjugate gradient framework for large-scale unconstrained optimization in Yang et al. [27], we develop the subspace derivative-free conjugate gradient algorithm with well-justified modifications. We extend its subspace structure to convex-constrained monotone equations for feasible iterates, redesign the three-term search direction to enhance descent and stability, integrate hyperplane projection and a safeguard parameter to ensure sufficient descent and global convergence, and adopt derivative-free line search to avoid Jacobian calculation. These targeted improvements result in the proposed SDCG algorithm for convex-constrained nonlinear monotone equations. The main innovations of the algorithm include the following:

• The method constructs search directions by combining the current negative gradient vector with the two most recent search directions, solving a quadratic subproblem within a low-dimensional subspace. This balances computational efficiency and algorithmic performance.
• The coefficients for the subspace directions are derived explicitly through minimizing a quadratic approximation, with three specific cases based on the subspace dimension. A safeguard parameter ensures the sufficient descent condition is always satisfied. The algorithm incorporates a hyperplane projection technique to ensure global convergence and iterate feasibility, while preserving derivative-free properties.
• Under standard assumptions, the method is proven to be globally convergent and achieves an R-linear convergence rate. Extensive numerical experiments on large-scale test problems show that the proposed SDCG method significantly outperforms existing derivative-free conjugate gradient methods in terms of iterations, function evaluations, and CPU time, especially for high-dimensional systems.

The rest of this paper is organized as follows. In Section 2, we present the specific steps of the fundamentals of the proposed subspace derivative-free conjugate gradient algorithm and a sufficient descent proof. In Section 3, we give a proof of boundedness of the search direction and global convergence of the algorithm. In Section 4, we analyze the numerical performance of the algorithm, including numerical results for solving large-scale nonlinear systems of monotone equations and comparisons with other algorithms.

2. Algorithm

In this section, we propose a new method for (1) with the projection technique, in which the search directions are determined by minimizing the quadratic approximation of the objective function in the subspace ${\Omega }_k.$

In order to avoid zigzagging and accelerate convergence, we choose the subspace spanned by the current residual and previous two search directions, capturing both the first-order information and the memory of the iterative path, i.e., ${\Omega }_k\triangleq \mathrm{span}\{-g_{k+1},s_k,s_{k-1}\}$, and consider the approximation of $f\left(x\right)$ as

$${\mathsf{\Phi }}_{k+1}\left(d\right)=g_{k+1}^{T}d+{\displaystyle \frac{1}{2}}{d}^T{B}_{k+1}d,$$

(3)

where $g_{k+1}=g\left(x_{k+1}\right)$ is the gradient of $F\left(x\right)$ at $x_{k+1}$, $B_{k+1}$ is an approximation of the Hessian ${\nabla }^{2}f\left(x_{k+1}\right)$, and $d\in {\Omega }_k$. Suppose $B_{k+1}$ is always strictly positive definite. Since ${\nabla }^{2}f\left(x_{k+1}\right)s_k\approx y_k$, we would like to choose $B_{k+1}$ satisfying the quasi-Newton equation $B_{k+1}{s}_k=y_k$. For the sake of brevity, we denote $F\left(x_k\right)$ as $F_k$ throughout this paper.

Specifically, let

$$\overline{d}=-F_{k+1}+a{s}_k+b{s}_{k-1},$$

(4)

where $(a,b)$ is the solution of the following minimizing problem:

$$\underset{a,b\in \mathbb{R}}{min}{\mathsf{\Phi }}_{k+1}\left(\overline{d}\right),\overline{d}\in {\Omega }_k.$$

(5)

First, we show the solution for $a_k,b_k,$ which can be separated into the following three cases:

Case I. If $dim\left({\Omega }_k\right)=2$ and ${\Omega }_k\triangleq \mathrm{span}\{-F_{k+1},s_k\}$, then the solution can be expressed as

$$a_k={\displaystyle \frac{F_{k+1}^T{B}_{k+1}{s}_k-F_{k+1}^T{s}_k}{s_k^T{B}_{k+1}{s}_k}}={\displaystyle \frac{F_{k+1}^T{y}_k-F_{k+1}^T{s}_k}{y_k^T{s}_k}},b_k=0,$$

(6)

and

$$\overline{d}=-F_{k+1}+a_k{s}_k.$$

Case II. If $dim\left({\Omega }_k\right)=2$ and ${\Omega }_k\triangleq \mathrm{span}\{-F_{k+1},s_{k-1}\}$, then the solution can be expressed as

$$a_k=0,b_k={\displaystyle \frac{1}{2}}\left[{\displaystyle \frac{F_{k+1}^T{y}_k}{y_k^T{s}_{k-1}}}-{\displaystyle \frac{\left(F_{k+1}^T{s}_k\right)\left(y_k^T{s}_{k-1}\right)\left(y_k^T{s}_k\right)}{\left(s_k^T{s}_k\right){\left(y_k^T{s}_{k-1}\right)}^2}}\right],$$

(7)

and

$$\overline{d}=-F_{k+1}+b_k{s}_{k-1}.$$

Case III. If $dim\left({\Omega }_k\right)=3$ and ${\Omega }_k\triangleq \mathrm{span}\{-F_{k+1},s_k,s_{k-1}\}$, then the solution can be expressed as

$$a_k={\displaystyle \frac{1}{{\Delta }_k}}\left[{\eta }_k\left(F_{k+1}^T{y}_k-F_{k+1}^T{s}_k\right)-\left(y_k^T{s}_{k-1}\right)\left({\omega }_k-F_{k+1}^T{s}_{k-1}\right)\right],$$

(8)

$$b_k={\displaystyle \frac{1}{{\Delta }_k}}\left[\left(y_k^T{s}_k\right)\left({\omega }_k-F_{k+1}^T{s}_{k-1}\right)-\left(y_k^T{s}_{k-1}\right)\left(F_{k+1}^T{y}_k-F_{k+1}^T{s}_k\right)\right],$$

(9)

and

$$\overline{d}=-F_{k+1}+a_k{s}_k+b_k{s}_{k-1},$$

where ${\omega }_k=F_{k+1}^T{s}_{k-1}+{\displaystyle \frac{\left(F_{k+1}^T{y}_k\right)\left(y_k^T{s}_{k-1}\right)}{y_k^T{s}_k}}-{\displaystyle \frac{\left(F_{k+1}^T{s}_k\right)\left(s_k^T{s}_{k-1}\right)}{s_k^T{s}_k}},{\eta }_k=2{\displaystyle \frac{{\left(y_k^T{s}_{k-1}\right)}^2}{y_k^T{s}_k}}$, and ${\Delta }_k={\left(y_k^{\mathrm{T}}{s}_{k-1}\right)}^2.$

If $s_k^T{y}_k$ is small, it indicates that the inner product between the change in iteration point and in $F\left(x\right)$ during the previous iteration is small. By setting ${\xi }_1$, we ensure that the search direction satisfies the sufficient descent condition at all times. Therefore, based on (4) and selecting a parameter ${\xi }_1$ as a protective parameter, we have

$$d_{k+1}=\left\{\begin{array}{cc}\overline{d},\hfill & \mathrm{if}{s}_k^T{y}_k\ge {\xi }_1{\parallel y_k\parallel }^2,{\xi }_1>0,\hfill \\ -F_{k+1},\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(10)

where $\overline{d}$ is given by (4), $y_k={\overline{y}}_k+r{s}_k,{\overline{y}}_k=F_{k+1}-F_k,s_k=x_{k+1}-x_k$, and $s_{k-1}=x_k-x_{k-1}$.

Now, we select an appropriate line search technique to get the step length ${\alpha }_k$. Zhang and Zhou [28] presented the following derivative-free line search rule, which requires ${\alpha }_k$ to be the largest step length of ${\alpha }_k=max\{\xi {\rho }^i|i=0,1,2,\dots \}$ to satisfy the following condition

$$-F{(x_k+\alpha d_k)}^{\mathrm{T}}{d}_k\ge \sigma \alpha \parallel F(x_k+\alpha d_k)\parallel \parallel d_k{\parallel }^2,$$

(11)

where $\sigma >0$ and $0<\rho <1$.

After the search direction $d_k$ and step length ${\alpha }_k$ are obtained, we set auxiliary point

$$z_k=x_k+{\alpha }_k{d}_k,$$

(12)

which satisfies

$$F{\left(z_k\right)}^{\mathrm{T}}(x_k-z_k)>0.$$

(13)

For any $x^{*}$ such that $F\left(x^{*}\right)=0$, by the monotonicity of $F\left(x\right)$, we have

$$F{\left(z_k\right)}^{\mathrm{T}}(x^{*}-z_k)=-{\left(F\left(x^{*}\right)-F\left(z_k\right)\right)}^{\mathrm{T}}(x^{*}-z_k)\le 0.$$

Thus, the hyperplane

$$H_k=\left\{x\in {\mathbb{R}}^n\mid F{\left(z_k\right)}^{\mathrm{T}}(x-z_k)=0\right\},$$

strictly separates the current iterate $x_k$ from zeros of the equation system (1).

Based on the above conclusion, Solodov and Svaiter [29] projected $x_k$ onto $H_k$ to get the next iteration point $x_{k+1}$. Its iteration format is

$$x_{k+1}=x_k-{\displaystyle \frac{F{\left(z_k\right)}^{\mathrm{T}}(x_k-z_k)}{\parallel F\left(z_k\right){\parallel }^2}}F\left(z_k\right).$$

(14)

This projection technique ensures that the solution generated at each iteration always lies within the given convex constraint set $\Omega$ and verifies the boundedness of the search direction. Numerical experiments show it is superior to other similar algorithms in solving large-scale nonlinear monotone equations. Next, we present the specific steps of the novel subspace derivative-free conjugate gradient method (SDCG).

Lemma 1.

Suppose $\left\{x_k\right\}$ is generated by Algorithm 1. Then, the search direction fulfills the sufficient descent condition

$$d_{k+1}^T{F}_{k+1}\le -C{\parallel F_{k+1}\parallel }^2,$$

(15)

for all $k\ge 0$.

Algorithm 1 SDCG

Step 0. Let $x_0\in {\mathbb{R}}^n$, $x_{-1}=x_0$, $\epsilon >0$, $0<\sigma <1$, $0<\xi <1$, $0<{\xi }_1<1$, $0<\rho <1,0<\kappa <2$ and compute $F_0=F\left(x_0\right)$. Set $d_0:=-F_0$, $k:=0$.    Step 1. Stop if $\parallel F_k\parallel \le \epsilon$, else compute $d_{k+1}$ by (10).    Step 2. Set $z_k=x_k+{\alpha }_k{d}_k$, compute ${\alpha }_k=max\{\xi {\rho }^i|i=0,1,2,\dots \}$ satisfying $$-⟨F(x_k+\alpha d_k),d_k⟩\ge \sigma \alpha \parallel F(x_k+\alpha d_k)\parallel \parallel d_k{\parallel }^2.$$    Step 3. If $z_k\in \Omega$ and $\parallel F\left(z_k\right)\parallel \le \epsilon$, set $x_{k+1}=z_k$, and stop. Otherwise compute $${\lambda }_k={\displaystyle \frac{⟨F\left(z_k\right),x_k-z_k⟩}{\parallel F\left(z_k\right){\parallel }^2}}$$ and $$x_{k+1}=P_{\Omega }[x_k-\kappa {\lambda }_{k}F\left(z_k\right)].$$    Step 4. Set $k:=k+1$, go to Step 1.

Proof. 

We prove the conclusion by considering the three cases of subspace dimension $dim\left({\Omega }_k\right)$ separately, where ${\Omega }_k\triangleq \mathrm{span}\{-F_{k+1},s_k,s_{k-1}\}$, $s_k=x_{k+1}-x_k$, and $s_{k-1}=x_k-x_{k-1}$.

According to the case I, $a_k$ and $b_k$ are determined by (6), so we can obtain

$$\begin{array}{ccc}\hfill d_{k+1}^T{F}_{k+1}& =& {\left(-F_{k+1}+{\displaystyle \frac{F_{k+1}^T{y}_k-F_{k+1}^T{s}_k}{y_k^T{s}_k}}·s_k\right)}^T{F}_{k+1}\hfill \\ & \le & -\left|\right|F_{k+1}{\left|\right|}^2+\left(1-{\displaystyle \frac{s_k^T{y}_k}{{\parallel y_k\parallel }^2}}\right){\parallel F_{k+1}\parallel }^2\hfill \\ & \le & -{\xi }_1{\parallel F_{k+1}\parallel }^2.\hfill \end{array}$$

According to case II, $a_k$ and $b_k$ are determined by (7), so we can obtain

$$\begin{array}{ccc}\hfill d_{k+1}^T{F}_{k+1}& =& {\left(-F_{k+1}+{\displaystyle \frac{1}{2}}\left[{\displaystyle \frac{F_{k+1}^{\mathrm{T}}{y}_k}{y_k^{\mathrm{T}}{s}_{k-1}}}-{\displaystyle \frac{\left(F_{k+1}^{\mathrm{T}}{s}_k\right)\left(s_k^{\mathrm{T}}{s}_{k-1}\right)\left(y_k^{\mathrm{T}}{s}_k\right)}{\left(s_k^{\mathrm{T}}{s}_k\right){\left(y_k^{\mathrm{T}}{s}_{k-1}\right)}^2}}\right]·s_{k-1}\right)}^T{F}_{k+1}\hfill \\ & \le & -\left|\right|F_{k+1}{\left|\right|}^2+{\displaystyle \frac{1}{2}}\left(1-{\displaystyle \frac{s_k^T{y}_k}{{\parallel y_k\parallel }^2}}\right){\parallel F_{k+1}\parallel }^2\hfill \\ & \le & -{\displaystyle \frac{1}{2}}-{\displaystyle \frac{1}{2}}{\xi }_1{\parallel F_{k+1}\parallel }^2.\hfill \end{array}$$

According to case III, $a_k$ and $b_k$ are determined by (8) and (9), so we can obtain

$$\begin{array}{ccc}\hfill d_{k+1}^T{F}_{k+1}& =& {\left(-F_{k+1}+{\displaystyle \frac{1}{{\Delta }_k}}(a_k{s}_k+b_k{s}_{k-1})\right)}^T{F}_{k+1}\hfill \\ & \le & -{\displaystyle \frac{s_k^T{y}_k}{{\parallel y_k\parallel }^2}}{\parallel F_{k+1}\parallel }^2\hfill \\ & \le & -{\xi }_1{\parallel F_{k+1}\parallel }^2.\hfill \end{array}$$

Combining the three cases, let $C=min\left(1,{\xi }_1,{\displaystyle \frac{1}{2}}+{\displaystyle \frac{1}{2}}{\xi }_1\right)$. Then, $⟨d_{k+1},F_{k+1}⟩\le -C{\parallel F_{k+1}\parallel }^2$ for all $k\ge 0$, which completes the proof. □

Lemma 2.

Let $\left\{d_k\right\}$ and $\left\{x_k\right\}$ be generated by Algorithm 1. Then, a step size ${\alpha }_k$ satisfying (11) always exists.

Proof. 

We use contradiction to prove it. By assuming that inequality (11) does not hold, we obtain a result that contradicts (15). □

3. Convergence Analysis

In this section, we establish the convergence results of Algorithm 1. Some assumptions are listed.

Assumption 1.

The problem admits a solution $x^{*}\in \Omega$ such that $F\left(x^{*}\right)=0$.

Assumption 2.

The mapping $F\left(x\right)$ is continuous and monotone.

Lemma 3.

Let Assumptions 1 and 2 hold. For the sequences $\left\{x_k\right\}$ and $\left\{z_k\right\}$ generated by Algorithm 1, the following inequality holds

$${\parallel x_{k+1}-x^{*}\parallel }^2\le \parallel x_k-x^{*}{\parallel }^2-\kappa (2-\kappa ){\parallel x_k-z_k\parallel }^4,\forall x^{*}\in \Omega .$$

Furthermore, $\left\{x_k\right\}$ is bounded and

$${\displaystyle \sum _{k=0}^{\infty }}{\parallel x_k-z_k\parallel }^4<+\infty .$$

Proof. 

From (2), we have

$$⟨F\left(z_k\right),x_k-x^{*}⟩\ge ⟨F\left(z_k\right),x_k-z_k⟩,\forall x^{*}\in \Omega ,$$

which, together with (11), implies that

$$⟨F\left(z_k\right),x_k-z_k⟩\ge \sigma {\alpha }_k^2\parallel F\left(z_k\right)\parallel \parallel d_k{\parallel }^2\ge 0.$$

As a result, we have

$$\begin{array}{ccc}\hfill \parallel x_{k+1}-x^{*}{\parallel }^2& =& \parallel P_{\Omega }[x_k-\kappa {\lambda }_{k}F\left(z_k\right)]-x^{*}{\parallel }^2\hfill \\ & \le & \parallel x_k-\kappa {\lambda }_{k}F\left(z_k\right)-x^{*}{\parallel }^2\hfill \\ & =& \parallel x_k-x^{*}{\parallel }^2-2\kappa {\lambda }_k⟨F\left(z_k\right),x_k-x^{*}⟩+{\parallel \kappa {\lambda }_{k}F\left(z_k\right)\parallel }^2\hfill \\ & \le & \parallel x_k-x^{*}{\parallel }^2-2\kappa {\lambda }_k⟨F\left(z_k\right),x_k-z_k⟩+{\parallel \kappa {\lambda }_{k}F\left(z_k\right)\parallel }^2\hfill \\ & =& \parallel x_k-x^{*}{\parallel }^2-\kappa (2-\kappa ){\displaystyle \frac{{⟨F\left(z_k\right),x_k-z_k⟩}^2}{\parallel F\left(z_k\right){\parallel }^2}}\hfill \\ & \le & \parallel x_k-x^{*}{\parallel }^2-\kappa (2-\kappa ){\sigma }^2{\parallel x_k-z_k\parallel }^4.\hfill \end{array}$$

This completes the proof. □

Thus, $\{\parallel x_k-x^{*}\parallel \}$ is non-increasing, so $\left\{x_k\right\}$ is bounded. We have

$$\kappa (2-\kappa ){\sigma }^2{\displaystyle \sum _{k=0}^{\infty }}\parallel x_k-z_k{\parallel }^4<{\parallel x_0-x^{*}\parallel }^2<+\infty .$$

According to the definition of $\left\{z_k\right\}$,

$$\underset{k\to \infty }{lim}\parallel x_k-z_k\parallel =\underset{k\to \infty }{lim}{\alpha }_k\parallel d_k\parallel =0.$$

(16)

Lemma 4.

Let the sequence $\left\{d_k\right\}$ be generated by Algorithm 1. Then, for all $k\ge 0$, we have

$$C\parallel F_{k+1}\parallel \le \parallel d_{k+1}\parallel \le C_3\parallel F_{k+1}\parallel ,$$

(17)

where $0<C<C_3$.

Proof. 

By Lemma 1 and the Cauchy–Schwarz inequality, we obtain

$$\parallel d_{k+1}\parallel \ge C\parallel F_{k+1}\parallel .$$

(18)

By utilizing (2), we can obtain

$$s_k^T{y}_k=s_k^T(F_{k+1}-F_k)+r{s}_k^T{s}_k\ge r{\parallel s_k\parallel }^2>0.$$

(19)

In the following, we consider three cases:

According to case I, $a_k$ and $b_k$ are determined by (6), so we can obtain

$$\begin{array}{ccc}\hfill \parallel d_{k+1}\parallel & =& ∥-F_{k+1}+{\displaystyle \frac{F_{k+1}^{\mathrm{T}}{y}_k-F_{k+1}^{\mathrm{T}}{s}_k}{y_k^{\mathrm{T}}{s}_k}}·s_k∥\hfill \\ & \le & \parallel F_{k+1}\parallel +∥{\displaystyle \frac{F_{k+1}^{\mathrm{T}}{y}_k-F_{k+1}^{\mathrm{T}}{s}_k}{y_k^{\mathrm{T}}{s}_k}}·s_k∥\hfill \\ & \le & \left(2-{\displaystyle \frac{1}{r}}\right)\parallel F_{k+1}\parallel .\hfill \end{array}$$

According to case II, $a_k$ and $b_k$ are determined by (7), so we can obtain

$$\begin{array}{ccc}\hfill \parallel d_{k+1}\parallel & =& ∥-F_{k+1}+{\displaystyle \frac{1}{2}}\left[{\displaystyle \frac{F_{k+1}^{\mathrm{T}}{y}_k}{y_k^{\mathrm{T}}{s}_{k-1}}}-{\displaystyle \frac{\left(F_{k+1}^{\mathrm{T}}{s}_k\right)\left(s_k^{\mathrm{T}}{s}_{k-1}\right)\left(s_k^{\mathrm{T}}{y}_k\right)}{\left(s_k^{\mathrm{T}}{s}_k\right){\left(y_k^{\mathrm{T}}{s}_{k-1}\right)}^2}}\right]∥\hfill \\ & \le & \parallel F_{k+1}\parallel +{\displaystyle \frac{1}{2}}∥{\displaystyle \frac{F_{k+1}^{\mathrm{T}}{y}_k}{y_k^{\mathrm{T}}{s}_{k-1}}}-{\displaystyle \frac{\left(F_{k+1}^{\mathrm{T}}{s}_k\right)\left(s_k^{\mathrm{T}}{s}_{k-1}\right)\left(s_k^{\mathrm{T}}{y}_k\right)}{\left(s_k^{\mathrm{T}}{s}_k\right){\left(y_k^{\mathrm{T}}{s}_{k-1}\right)}^2}}{s}_{k-1}∥\hfill \\ & \le & \left({\displaystyle \frac{3}{2}}-{\displaystyle \frac{1}{2r}}\right)\parallel F_{k+1}\parallel .\hfill \end{array}$$

According to case III, $a_k$ and $b_k$ are determined by (8) and (9), so we can obtain

$$\begin{array}{ccc}\hfill \parallel d_{k+1}\parallel & =& ∥-F_{k+1}+a_k{s}_k+b_k{s}_{k-1}∥\hfill \\ & \le & \parallel F_{k+1}\parallel +\parallel a_k{s}_k\parallel +\parallel b_k{s}_{k-1}\parallel .\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \parallel a_k{s}_k\parallel & =& \parallel {\displaystyle \frac{s_k}{{\left(y_k^T{s}_{k-1}\right)}^2}}[2{\left(y_k^T{s}_{k-1}\right)}^2·{\displaystyle \frac{F_{k+1}^T{y}_k-F_{k+1}^T{s}_k}{y_k^T{s}_k}}\hfill \\ & & -y_k^T{s}_{k-1})\left({\displaystyle \frac{\left(F_{k+1}^T{y}_k\right)\left(y_k^T{s}_{k-1}\right)}{y_k^T{s}_k}}-{\displaystyle \frac{\left(F_{k+1}^T{s}_k\right)\left(s_k^T{s}_{k-1}\right)}{s_k^T{s}_k}}\right)]\parallel \hfill \\ & \le & \left(1+{\displaystyle \frac{1}{r}}\right)\parallel F_{k+1}\parallel .\hfill \end{array}$$

$$\begin{array}{ccc}\hfill \parallel b_k{s}_{k-1}\parallel & =& \parallel {\displaystyle \frac{1}{{\left(y_k^T{s}_{k-1}\right)}^2}}[\left(y_k^T{s}_k\right)\left({\displaystyle \frac{\left(F_{k+1}^T{y}_k\right)\left(y_k^T{s}_{k-1}\right)}{y_k^T{s}_k}}-{\displaystyle \frac{\left(F_{k+1}^T{s}_k\right)\left(s_k^T{s}_{k-1}\right)}{s_k^T{s}_k}}\right)\hfill \\ & & -\left(y_k^T{s}_{k-1}\right)·(F_{k+1}^T{y}_k-F_{k+1}^T{s}_k)]s_{k-1}\parallel \hfill \\ & \le & \left(2-{\displaystyle \frac{2}{r}}\right)\parallel F_{k+1}\parallel ,\hfill \\ \hfill \parallel d_{k+1}\parallel & \le & \left(4-{\displaystyle \frac{1}{r}}\right)\parallel F_{k+1}\parallel .\hfill \end{array}$$

So, let $C_3=max\left\{1,2-{\displaystyle \frac{1}{r}},{\displaystyle \frac{3}{2}}-{\displaystyle \frac{1}{2r}},4-{\displaystyle \frac{1}{r}}\right\}$, completing the proof. □

Theorem 1.

Let Assumption 2 hold, with the sequences $\left\{x_k\right\}$ and $\left\{z_k\right\}$ generated by Algorithm 1, then

$$\underset{k\to \infty }{lim}\parallel F_{k+1}\parallel =0.$$

(20)

Proof. 

We proceed by contradiction. Assume (20) is false, i.e., there exists $r>0$ such that $\parallel F_k\parallel \ge r$ for all $k\ge 0$. Together with (18), this gives

$$\parallel d_k\parallel \ge Cr,\forall k\ge 0.$$

(21)

By (16) and (21), $\underset{k\to \infty }{lim}{\alpha }_k=0$. Since ${\alpha }_k=\xi {\rho }^{i_k}$, the line search (11) implies for sufficiently large k that

$$-⟨F(x_k+\xi {\rho }^{i_k}{d}_k),d_k⟩<\sigma \xi {\rho }^{i_k}\parallel F(x_k+\xi {\rho }^{i_k}{d}_k)\parallel \parallel d_k{\parallel }^2.$$

(22)

By (17) and $\parallel F_k\parallel \ge r$, $\left\{d_k\right\}$ is bounded. We may assume $\left\{x_k\right\}$ and $\left\{d_k\right\}$ are convergent. Letting $k\to \infty$ in (22) yields

$$-⟨F\left(\overline{x}\right),\overline{d}⟩<0,$$

(23)

where $\overline{x}$, $\overline{d}$ are the limit points. Taking limits in (15) leads to

$$-⟨F\left(\overline{x}\right),\overline{d}⟩\ge C{\parallel F\left(\overline{x}\right)\parallel }^2.$$

(24)

Combining (23) and (24) gives $\parallel F\left(\overline{x}\right)\parallel =0$, which contradicts $\parallel F_k\parallel \ge r$. Hence, (20) holds. The proof is completed. □

The local convergence rate of Algorithm 1 is listed.

Assumption 3.

For any $x^{*}\in \Omega$, there exist constants $\omega \in (0,1)$ and $\delta >0$ satisfying

$$\omega \mathrm{dist}(x,\Omega )\le {\parallel F\left(x\right)\parallel }^2,\forall x\in N(x^{*},\delta ),$$

(25)

where $\mathrm{dist}(x,\Omega )$ stands for the distance from x to the solution set $\Omega$, and $N(x^{*},\delta )=\{x\in \Omega \mid \parallel x-x^{*}\parallel \le \delta \}$.

Theorem 2.

Holding Assumptions 2 and 3, let $\left\{x_k\right\}$ be generated by Algorithm 1. Then, $dist(x_k,\Omega )$ converges Q-linearly to 0, and $\left\{x_k\right\}$ converges R-linearly to $\overline{x}\in \Omega$.

Proof. 

Let $u_k:=argmin\{\parallel x_k-u\parallel \mid u\in \Omega \}$. We know clearly that $u_k$ is the nearest solution point from $x_k$, i.e.,

$$\parallel x_k-u_k\parallel =\mathrm{dist}(x_k,\Omega ).$$

According to (18) and (25), we can easily conclude that $\mathrm{dist}(x_k,\Omega )$ is Q-linearly convergent to 0. If $\mathrm{dist}(x_k,\Omega )$ possesses this property, $\left\{x_k\right\}$ is R-linearly convergent to $\overline{x}$. □

4. Numerical Experiments

In this section, we compare the performance of SDCG with that of the DFCG [30], DFPB1 [31], and MRMIL1 [32]. All test algorithms were coded in MATLAB R2018b and run on an HP desktop with an Intel(R) Core(TM) i5-10210 CPU (2.11 GHz), 8.00 GB RAM and Windows 10 OS.

In SDCG, we select the following parameters:

$$\rho =0.7,\sigma =0.3,\xi =0.1,{\xi }_1={10}^{-7},\kappa =1.9,r=0.0001.$$

All test methods are terminated if the iteration exceeds 10,000, or if the function value of the current iteration satisfies the condition $\parallel F_k\parallel \le {10}^{-4}$. We now list the following test problems.

Problem 1.

The function $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right),\dots ,f_n\left(x\right))}^T$, where

$$f_i\left(x\right)=2x_i-sin\left(x_i\right),1\le i\le n.$$

Problem 2.

The function $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right),\dots ,f_n\left(x\right))}^T$, where

$$f_i\left(x\right)=2x_i-sin\left|x_i\right|,1\le i\le n.$$

Problem 3.

The function $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right),\dots ,f_n\left(x\right))}^T$, where

$$\left\{\begin{array}{c}{f}_1\left(x\right)=x_1-exp\left(cos\left({\displaystyle \frac{x_1+x_2}{n+1}}\right)\right),\hfill \\ f_i\left(x\right)=x_i-exp\left(cos\left({\displaystyle \frac{x_{i-1}+x_i+x_{i+1}}{n+1}}\right)\right),\hfill & 2\le i\le n-1,\hfill \\ f_n\left(x\right)=x_n-exp\left(cos\left({\displaystyle \frac{x_{n-1}+x_n}{n+1}}\right)\right).\hfill \end{array}\right.$$

Problem 4.

The function $F\left(x\right)=Ax+b\left(x\right)-e$, where $b\left(x\right)={(e^{x_1},e^{x_2},\dots ,e^{x_n})}^T$, $e={(1,1,\dots ,1)}^T$ are the $n\times 1$ vectors, and

$$A=\left(\begin{array}{ccccc}2& -1& & & \\ -1& 2& -1& & \\ & \ddots & \ddots & \ddots & \\ & & \ddots & \ddots & -1\\ & & & -1& 2\end{array}\right)$$

is the $n\times n$ matrix.

Problem 5.

The function $F\left(x\right)=Ax-e$, where $e={(1,1,\dots ,1)}^T$ is the $n\times 1$ vector, and

$$A=\left(\begin{array}{ccccc}{\displaystyle \frac{5}{2}}& 1& & & \\ 1& {\displaystyle \frac{5}{2}}& 1& & \\ & \ddots & \ddots & \ddots & \\ & & \ddots & \ddots & 1\\ & & & 1& {\displaystyle \frac{5}{2}}\end{array}\right)$$

is the $n\times n$ matrix.

Problem 6.

The function $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right),\dots ,f_n\left(x\right))}^T$, where

$$F_i\left(x\right)=e^{x_i}-1,\mathrm{for}i=1,2,3,\dots ,n.$$

Problem 7.

The function $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right),\dots ,f_n\left(x\right))}^T$, where

$$\left\{\begin{array}{c}{F}_1\left(x\right)=2.5x_1+x_2-1,\hfill \\ F_i\left(x\right)=x_{i-1}+2.5x_i+x_{i+1}-1,\hfill & \mathrm{for}i=2,3,\dots ,n-1,\hfill \\ F_n\left(x\right)=x_{n-1}+2.5x_n-1.\hfill \end{array}\right.$$

Problem 8.

The function $F\left(x\right)={(f_1\left(x\right),f_2\left(x\right),\dots ,f_n\left(x\right))}^T$, where

$$f_l\left(x\right)=log(x_i+1)-{\displaystyle \frac{x_i}{n}},i=1,2,\dots ,n.$$

The above problems with different dimensions ($n=$ 500, 1000, 3000, 5000 and 10,000) are used to test the four methods, and all problems are initialized with the following 6 starting points: $x_0^1={(1,1,\dots ,1)}^T$, $x_0^2=-{(1,1,\dots ,1)}^T$, $x_0^3={(0.1,0.1,\dots ,0.1)}^T$, $x_0^4={(1,{\displaystyle \frac{1}{2}},\dots ,{\displaystyle \frac{1}{n}})}^T$, $x_0^5={({\displaystyle \frac{1}{n}},{\displaystyle \frac{2}{n}},\dots ,1)}^T$, $x_0^6={({\displaystyle \frac{n-1}{n}},{\displaystyle \frac{n-2}{n}},\dots ,0)}^T$.

We list partial numerical results with $x_0^2$ as the initial point in Table 1. $Pi$ ($i=1,2,\cdots ,8$) means the i-th test problem listed above, and $Ni$ and $NF$ mean the number of iterations and the number of function evaluations. Finally, T stands for the CPU time required for iterations.

Table 1. The numerical results with $x_0^2=-{(1,1,\dots ,1)}^T$.

		SDCG	DFCG	DFPB1	MRMIL1
Prob.	n	NF/Ni/T	NF/Ni/T	NF/Ni/T	NF/Ni/T
	500	60/8/0.047	74/10/0.017	46/14/0.036	46/15/0.036
	1000	63/10/0.054	99/13/0.050	70/17/0.047	69/18/0.045
P1	3000	75/11/0.167	275/22/0.350	142/25/0.172	145/27/0.178
	5000	84/13/0.266	235/27/0.450	200/31/0.427	197/32/0.407
	10,000	109/18/0.707	421/40/1.800	308/41/1.307	308/42/1.290
	500	25/7/0.016	117/14/0.034	82/13/0.024	91/15/0.028
	1000	30/10/0.017	148/17/0.066	116/17/0.056	372/49/0.201
P2	3000	57/15/0.100	243/25/0.283	217/25/0.254	218/26/0.26
	5000	76/18/0.183	317/31/0.600	287/31/0.584	294/32/0.597
	10,000	114/22/0.550	466/42/1.850	436/41/1.795	440/43/1.792
	500	92/15/0.048	251/33/0.100	245/36/0.110	242/38/0.100
	1000	125/24/0.100	413/46/0.200	377/48/0.101	372/49/0.200
P3	3000	220/30/0.452	844/78/1.100	711/80/1.100	769/81/1.201
	5000	297/39/0.864	1128/99/2.600	1058/101/2.501	1056/102/2.600
	10,000	462/60/2.584	1698/138/8.200	1618/140/7.900	1619/141/7.902
	500	58/15/0.034	198/33/0.095	79/20/0.054	172/33/0.118
	1000	62/17/0.134	213/38/0.265	91/20/0.174	196/37/0.362
P4	3000	77/23/1.233	267/45/2.740	192/37/2.115	259/45/2.693
	5000	88/28/2.984	315/47/7.075	242/42/5.861	299/50/6.893
	10,000	117/35/13.066	425/60/34.242	328/47/27.705	392/57/31.735
	500	72/14/0.066	195/26/0.012	166/23/0.017	177/27/0.015
	1000	75/16/0.183	248/29/0.142	210/26/0.250	230/32/0.132
P5	3000	85/20/1.467	369/39/3.052	351/37/3.104	377/43/3.181
	5000	96/26/3.380	495/50/9.569	452/47/9.283	479/50/9.802
	10,000	120/30/14.151	695/62/50.202	648/56/49.591	678/63/50.867
	500	64/10/0.050	81/12/0.023	39/15/0.015	34/11/0.020
	1000	65/13/0.067	82/14/0.053	61/18/0.033	59/17/0.033
P6	3000	69/15/0.149	159/23/0.196	127/26/0.176	124/26/0.158
	5000	76/18/0.284	210/29/0.446	184/32/0.405	176/28/0.370
	10,000	95/22/0.650	315/39/1.375	290/43/1.319	277/38/1.209
	500	73/12/0.045	195/26/0.049	166/23/0.100	177/27/0.048
	1000	76/15/0.056	230/27/0.093	625/108/0.310	225/31/0.098
P7	3000	93/17/0.183	364/38/0.384	897/145/1.201	362/40/0.395
	5000	108/20/0.317	486/46/0.862	1059/164/2.600	469/48/0.844
	10,000	139/24/0.783	677/57/2.470	1255/176/6.502	664/60/2.525
	500	2/1/0.001	2/1/0.032	2/1/0.002	2/1/0.001
	1000	2/1/0.003	2/1/0.002	2/1/0.003	2/1/0.003
P8	3000	2/1/0.006	2/1/0.005	2/1/0.006	2/1/0.006
	5000	2/1/0.017	2/1/0.008	2/1/0.007	2/1/0.008
	10,000	2/1/0.015	2/1/0.011	2/1/0.015	2/1/0.013

As shown in Table 1, SDCG exhibits excellent performance in solving high-dimensional problems. We can see that SDCG costs less than twice as much as other algorithms in solving test problems 2, 7, and 8. When handling most test problems, SDCG can achieve optimal solutions in less time with fewer iterations and fewer function evaluations.

We employ the performance profiles [21], which are defined by the following fraction:

$${\rho }_{\nu }\left(\tau \right)={\displaystyle \frac{1}{\left|P\right|}}\left|\left\{p\in P:{log}_2\left({\displaystyle \frac{t_{p,\nu }}{min\{t_{p,\nu }:\nu \in V\}}}\right)\le \tau \right\}\right|.$$

to describe the performance of the algorithms. Here, P is the test set, $\left|P\right|$ is the number of problems in the test set P, V is the set of optimization solvers, and $t_{p,\nu }$ is the CPU time (or the number of function evaluations, or the number of iterations) for $p\in P$ and $\nu \in V$.

We analyze the values of parameters $\xi$, $\kappa$, and r in SDCG, among which the value of parameter r directly affects the search direction and its descent property. Firstly, we investigate the numerical results corresponding to parameter values of $r=0.0001$, $0.05$, $0.1$, and 1 in terms of the number of function evaluations, the number of iterations, and CPU time. We focus on parameter r which affects the search direction and descent property, and test $r=0.0001,0.05,0.1$, and 1. Figure 1 shows that $r=0.0001,0.05$, and $0.1$ yield nearly the same function evaluation counts, with overlapping curves. In contrast, $r=1$ requires more function evaluations, reducing efficiency. Figure 2 confirms this trend: $r=0.0001,0.05$, and $0.1$ have similar iteration counts, while $r=1$ leads to more iterations. Together, these figures show that r in 0.0001–0.1 maintains algorithm efficiency, supporting our choice of $r=0.0001$ for subsequent experiments. Furthermore, based on the CPU time presented in Figure 3, we can see that $r=0.0001$ and $r=1$ perform better than $r=0.05$ and $r=0.1$. And the algorithm achieves the most stable overall performance when $r=0.0001$. Therefore, the value of $r=0.0001$ is adopted in subsequent experiments.

Figure 1. Performance profiles of SDCG with different r based on the number of function evaluations.

Figure 2. Performance profiles of SDCG with different r based on the number of iterations.

Figure 3. Performance profiles of SDCG with different r based on the CPU time.

Next, we investigate the impacts of the initial step size $\xi$ and parameter $\kappa$ on the algorithm performance. In SDCG, we adopt $d_k$ as the search direction, and solve test problems by setting the initial step size to $\xi =0.1$, $0.5$, and 1 and parameter $\kappa$ to 1 and $1.9$, respectively, with detailed results presented in Figure 4, Figure 5 and Figure 6. When $\xi =0.1$, $\kappa =1.9$, and $r=0.0001$, SDCG achieves superior performance in terms of the number of function evaluations and iterations, while its CPU time curve is slightly higher than those under other parameter configurations. This indicates that with such parameter settings, the algorithm exhibits favorable stability and requires a shorter average time to solve test problems.

Figure 4. Performance profiles of SDCG with different $\xi$ and $\kappa$ based on the number of function evaluations.

Figure 5. Performance profiles of SDCG with different $\xi$ and $\kappa$ based on the iterations.

Figure 6. Performance profiles of SDCG with different $\xi$ and $\kappa$ based on the CPU time.

Based on the above experimental results, we further evaluate the numerical performance of the search direction of SDCG under the parameter configuration of $\xi =0.1$, $\kappa =1.9$, and $r=0.0001$, and conduct a comparative analysis with the DFCG, DFPB1 and MRMIL1 algorithms.

Figure 7, Figure 8 and Figure 9 plot the performance curves of our proposed SDCG against those of the DFCG, MRMIL1, and DFPB1 algorithms, with a focus on three key metrics: the number of function evaluations, the number of iterations, and CPU time. Specifically, Figure 7 verifies that SDCG outperforms DFCG, DFPB1, and MRMIL1 in terms of the number of function evaluations. In Figure 8, it is clear that DFCG, MRMIL1, and DFPB1 yield better performance in iteration counts when $\tau <1$; however, once $\tau \ge 1$, our proposed algorithm regains and maintains its superior performance. Figure 9 further underscores the advantages of SDCG: it can solve large-scale nonlinear monotone systems more rapidly and consistently stands out among the four tested algorithms.

Figure 7. Performance profiles of different algorithms based on the number of function evaluations.

Figure 8. Performance profiles of different algorithms based on the number of iterations.

Figure 9. Performance profiles of different algorithms based on the CPU time.

Based on the above findings, we can conclude that the SDCG is significantly superior to the other three methods. This result demonstrates that our newly proposed subspace derivative-free conjugate gradient method exhibits strong stability when tackling large-scale nonlinear monotone systems: it reduces the required number of iterations and function evaluations, shortens CPU running time, and thus improves the overall efficiency.

5. Conclusions

In this research, we have successfully developed a novel and highly efficient algorithm. We initiate by meticulously analyzing the sufficient descent property of the search direction. This step was crucial as it provided a solid theoretical foundation for the subsequent convergence analysis. Through rigorous mathematical derivations and logical reasoning, we were able to establish both global convergence and convergence and the R-linear convergence rate. The numerical results obtained were highly encouraging. They clearly demonstrated that the proposed method outperforms many existing algorithms in terms of computational efficiency, accuracy, and robustness when dealing with nonlinear monotone equations with convex constraints.

However, we also recognize that there are still significant areas for further exploration. These models can potentially capture the complex characteristics of the equations more accurately, leading to more efficient and effective algorithms. This will be the primary focus of our future research, where we aim to push the boundaries of current knowledge and develop even more advanced techniques for solving these challenging problems.