Zeroth-Order Riemannian Adaptive Regularized Proximal Quasi-Newton Optimization Method

Abstract

Recently, the adaptive regularized proximal quasi-Newton (ARPQN) method has demonstrated a strong performance in solving composite optimization problems over the Stiefel manifold. However, its reliance on first-order information limits its applicability to scenarios where gradient and Hessian evaluations are unavailable or costly. In this paper, we propose a zeroth-order adaptive regularized proximal quasi-Newton method (ZO-ARPQN) for black-box composite optimization over Riemannian manifolds, particularly the Stiefel and symmetric positive definite (SPD) manifolds. The proposed method estimates the Riemannian gradient and curvature information through randomized one-point finite-difference approximations and adaptively updates a regularized quasi-Newton matrix to capture the local manifold geometry. Theoretically, we established global convergence and complex analyses under mild assumptions. More importantly, by incorporating curvature-aware regularization and random perturbations in the proximal quasi-Newton framework, we proved that ZO-ARPQN can escape strict saddle points with a high probability. This guarantees convergence to a stationary point, even in the absence of explicit gradients. Extensive numerical experiments were conducted on manifold-constrained problems, including sparse PCA and robot stiffness tuning. These demonstrated that ZO-ARPQN shows a competitive convergence behavior compared with other state-of-the-art Riemannian optimization methods, while requiring only function evaluations.

1. Introduction

In recent years, composite optimization problems on Riemannian manifolds have attracted widespread attention in various application fields. The objective function of such problems typically consists of the sum of a smooth function and a nonsmooth function, with optimization variables constrained on a manifold. The general form of a Riemannian composite optimization problem is summarized as follows:

$$\underset{X\in \mathcal{M}}{min}F\left(X\right)=f\left(X\right)+h\left(X\right).$$

(1)

where $f:{\mathbb{R}}^{n\times r}\to \mathbb{R}$ is a smooth function and $h:{\mathbb{R}}^{n\times r}\to \mathbb{R}$ is a convex, but nonsmooth, function. The aforementioned Riemannian composite optimization problems have found extensive applications, such as compressed sensing [1], sparse principal component analyses [2], and clusteringproblems [3]. For more applications of composite optimization on manifolds, readers are referred to [4,5,6].

Composite optimization problems with manifold constraints have been extensively studied in recent years. Currently, popular solution methods can be roughly categorized into the following types: operator-splitting methods, proximal gradient methods, and proximal Newton methods.

Operator-Splitting Methods. Riemannian ADMM (alternating-direction method of multipliers) provides a flexible decomposition strategy, especially suitable for composite problems with special structures. It usually transforms the original problem into an equivalent form via variable splitting and then solves it through alternating updates and dual ascent. Additionally, other operator-splitting ideas, such as Douglas–Rachford splitting, have been extended to manifolds [7]. Research on Riemannian ADMM focuses on its convergence analysis under nonconvex and nonsmooth settings [8] and efficient implementation in specific applications [3]. For example, in distributed or federated learning scenarios, the decomposable nature of ADMM enables the design of communication-efficient and privacy-preserving Riemannian optimization algorithms [9]. The advantages of Riemannian ADMM lie in its high flexibility and ability to decouple variables; however, it requires the introduction of additional variables when constructing the Lagrangian objective function, which slows down its convergence rate.

First-Order Methods: Proximal Gradient Methods. This is the most fundamental and straightforward approach for solving Riemannian composite problems. The Riemannian proximal gradient method iteratively decomposes the smooth and nonsmooth terms, i.e., [4,5]:

$$x_{k+1}={\mathrm{prox}}_{{\gamma }_{k}h}\left(R_{x_k}\left(-{\gamma }_k\nabla f\left(x_k\right)\right)\right)$$

(2)

which denotes the Riemannian proximal operator. This algorithm first performs one step of Riemannian gradient descent on the smooth part f, and then corrects the nonsmooth part h via the proximal operator, entirely within the manifold $\mathcal{M}$.

To accelerate convergence, researchers have proposed various accelerated Riemannian proximal gradient algorithms by leveraging Nesterov’s acceleration idea. These methods typically improve the convergence rate from $O(1/k)$ to $O(1/k^2)$ under geodesically convex or specific curvature conditions [10,11]. For large-scale data, computing the full gradient $\nabla f\left(x_k\right)$ is prohibitively expensive; thus, Riemannian stochastic proximal gradient methods and their variance-reduced variants (e.g., R-SVRG-PG [12]) have emerged. They approximate the gradient through random sampling, significantly reducing the cost per iteration while achieving fast convergence. Considering that $f\left(x\right)$ is nonconvex in many practical applications, recent works have analyzed the convergence of proximal gradient methods under nonconvex settings, proving that the algorithm converges to stationary points and providing convergence rate guarantees [13]. This method is simple to implement and theoretically mature, but suffers from slow convergence and reliance on the proximal operator.

Second-Order Methods: Proximal Newton Methods. To achieve faster convergence than first-order methods, second-order methods incorporate curvature (Hessian) information of the smooth term f to guide updates. The core idea of the regularized Riemannian proximal Newton method is to obtain the update direction ${\eta }_k$ at each tangent space $T_{x_k}\mathcal{M}$ by solving a quadratic approximation subproblem [4,5]:

$${\eta }_k=\underset{\eta \in T_{X_k}\mathcal{M}}{arg\; min}〈\nabla f\left(X_k\right),\eta 〉+{\displaystyle \frac{1}{2}}〈\eta ,{\mathcal{H}}_k\left[\eta \right]〉+h\left(R_{X_k}\left(\eta \right)\right)$$

(3)

where ${\mathcal{H}}_k$ is a positive definite approximation of the Riemannian Hessian operator $\mathbb{H}f\left(x_k\right)$.

Standard Newton methods face two major challenges: (1) how to ensure global convergence when the Hessian is non-positive definite, and (2) how to handle the enormous computational, storage, and inversion costs of the Hessian matrix. Recent research on quasi-Newton methods and adaptive regularization has focused on addressing these two issues. These are key techniques to address the computational bottleneck of second-order methods. Riemannian quasi-Newton methods (e.g., Riemannian BFGS or L-BFGS) no longer directly compute the true Hessian matrix; instead, they construct an increasingly accurate Hessian approximation matrix $B_k$ using first-order (gradient) information collected during iterations [14,15]. A representative method is the adaptive regularized proximal quasi-Newton (ARPQN) method [16], which solves a subproblem of the following form at each step:

$$\underset{V\in T_{X_k}\mathcal{M}}{min}〈\nabla f\left(X_k\right),V〉+{\displaystyle \frac{1}{2}}〈({\mathcal{B}}_k+{\sigma }_{k}I)\left[V\right],V〉+h\left(R_{X_k}\left(V\right)\right)$$

(4)

The core advantages of this method are Hessian approximation and adaptive regularization. This allows the algorithm to flexibly balance gradient and curvature information, thereby achieving faster convergence in practice.

It should be noted that the aforementioned methods all assume that the gradient and Hessian matrix of the objective function have analytical forms. However, many optimization problems in practical applications face the following challenges, rendering traditional gradient-based (first-order or second-order) methods inapplicable:

1.

Unavailable Gradient: In many practical problems, the objective function $f\left(X\right)$ is a black box: one can only query its value without access to analytical gradients or automatic differentiation. This often occurs in complex physical simulations such as fluid dynamics or structural mechanics, where each function evaluation is computationally expensive and differentiation is infeasible. Traditional approaches like Bayesian optimization mitigate the cost through surrogate models [17], but become inefficient as the problem dimension increases. In robotic stiffness control, the objective function describing the robot–environment interaction has no closed-form gradient and can only be evaluated through high-fidelity simulations or experiments. The mapping from stiffness parameters (an SPD matrix) to performance scores thus forms a typical black-box optimization problem.

2.

Prohibitive Gradient Computation Cost: Even if the gradient exists in theory, the cost of computing it may be unacceptably high. In large-scale reinforcement-learning scenarios, the reward function of an agent may depend on an extremely long sequence of decisions. Computing the gradient of policy parameters through backpropagation may consume enormous computational resources and memory. Directly evaluating the performance of the policy (zeroth-order information) may be much cheaper. Zeroth-order methods such as evolutionary strategies have been proven to be scalable alternatives for reinforcement learning [18].

Based on the above motivations, zeroth-order methods can be applied to any optimization problem where function evaluations are accessible, regardless of how complex its internal structure is or whether it is differentiable. This greatly expands the application boundary of optimization algorithms, enabling them to solve black-box problems that traditional methods cannot address.

First, we clarify the definition: Zeroth-order Riemannian methods, also known as derivative-free Riemannian optimization methods, refer to a class of methods that solve optimization problems on manifolds using only function evaluation values (zeroth-order information), without using explicit gradients (first-order information) or Hessian matrices (second-order information) of the objective function. By abandoning reliance on accurate gradient information, zeroth-order Riemannian methods gain the ability to handle complex problems such as black-box, noisy, and high-cost scenarios. They sacrifice a certain degree of convergence efficiency (compared to gradient-based methods) to significantly broaden the application scope of optimization algorithms on manifolds.

2. Theoretical Contributions

This work proposes the zeroth-order Riemannian adaptive quasi-Newton method (ZO-ARPQN) for composite optimization on Stiefel manifolds, establishing three fundamental advances:

Zeroth-Order Extension of Proximal Quasi-Newton on Riemannian Manifolds. We generalize the adaptive regularized proximal quasi-Newton (ARPQN) framework [16] to the zeroth-order setting over Riemannian manifolds such as the Stiefel and SPD manifolds. By constructing randomized finite-difference estimators that approximate both the Riemannian gradient and curvature information, ZO-ARPQN achieves first-order accuracy using only function evaluations, enabling black-box optimization in manifold-constrained problems.

Global Convergence Guarantees under Noisy Gradient Estimates. We establish global convergence to a stationary point under mild smoothness and regularization assumptions, even when the gradient and Hessian information are approximated by random perturbations. Theoretical bounds are derived on the bias and variance of the zeroth-order estimators, showing that ZO-ARPQN maintains the same global convergence as its first-order counterpart.

Saddle-Point Escaping. By incorporating curvature-aware regularization and adaptive random perturbation in the proximal quasi-Newton step, we prove that ZO-ARPQN can escape strict saddle points with a high probability, ensuring convergence to a stationary point. This result extends existing saddle-point escape theory to the manifold and zeroth-order settings.

3. Preliminaries on Riemannian Optimization

The core idea of Riemannian optimization is to extend classical optimization methods from Euclidean space to non-Euclidean spaces, i.e., Riemannian manifolds. Compared with the Euclidean case, manifold optimization requires constraining the geometric structure of iterates, such that search directions, gradients, and update operators are all defined within the framework of tangent spaces. This requires us to introduce tools such as tangent spaces, Riemannian metrics, retractions, and Riemannian Hessians.

3.1. Tangent Space and Tangent Bundle

Let $\mathcal{M}$ be a manifold. The tangent space at a point $X\in \mathcal{M}$ is denoted as $T_X\mathcal{M}$, which consists of all tangent vectors at X. The tangent bundle

$$\mathcal{T}\mathcal{M}:=\bigcup _{X\in \mathcal{M}}{T}_X\mathcal{M}$$

represents the collection of all tangent spaces of $\mathcal{M}$.

3.2. Riemannian Metric and Norm

If the tangent spaces are endowed with an inner product

$${\langle \xi ,\eta \rangle }_X,\xi ,\eta \in T_X\mathcal{M}$$

that varies smoothly with the point X, then $(\mathcal{M},\langle ·,·\rangle )$ is called a Riemannian manifold. The induced norm is

$${\parallel \xi \parallel }_X=\sqrt{{\langle \xi ,\xi \rangle }_X}.$$

For simplicity, we denote $\parallel \xi \parallel$ and $\langle \xi ,\xi \rangle$ if there is no ambiguity.

3.3. Riemannian Gradient

In [19], if $f:\mathcal{M}\to \mathbb{R}$ is a smooth function, its gradient $\nabla f\left(X\right)\in T_X\mathcal{M}$ at $X\in \mathcal{M}$ is defined as the unique tangent vector satisfying

$$\langle \nabla f\left(X\right),\xi \rangle =Df\left(X\right)\left[\xi \right],\forall \xi \in T_X\mathcal{M},$$

where $Df\left(X\right)\left[\xi \right]$ denotes the directional derivative of f at X along $\xi$. In this paper, we adopt the Riemannian metric induced by the Euclidean inner product, i.e.,

$${\langle ·,·\rangle }_x=\langle ·,·\rangle ,\forall x\in \mathcal{M}.$$

Based on this Riemannian metric, the Riemannian gradient of a function is defined as the projection of its Euclidean gradient onto the tangent space:

$$\nabla f\left(x\right)={\text{Proj}}_{T_x\mathcal{M}}(\nabla f\left(x\right))$$

(5)

Definition 1 

(Retraction [4]). A retraction on a manifold $\mathcal{M}$ is a smooth mapping $R:\mathcal{T}\mathcal{M}\to \mathcal{M}$ satisfying the following properties. Let $R_X$ denote the restriction of R to $T_X\mathcal{M}$: (1) $R_X\left(0_X\right)=X$, where $0_X$ is the zero element of $T_X\mathcal{M}$; (2) under the canonical isomorphism $T_{0_X}\left(T_X\mathcal{M}\right)\cong T_X\mathcal{M}$, we have

$$D{R}_X\left(0_X\right)={\mathit{id}}_{T_X\mathcal{M}},$$

where $D{R}_X\left(0_X\right)$ denotes the differential of $R_X$ at $0_X$, and ${\mathit{id}}_{T_X\mathcal{M}}$ denotes the identity mapping on $T_X\mathcal{M}$.

The following proposition serves as a key bridge connecting retractions and optimization analyses. It endows complex manifold optimization algorithms with strict mathematical certainty. Inequality (6) guarantees the boundedness and controllability of the retraction operation. It ensures that a tangent space displacement $\xi$ of a finite magnitude does not result in an infinitely distant move on the manifold, which is a prerequisite for any subsequent convergence analysis.

Proposition 1 

([19]). If $\mathcal{M}$ is a compact embedded submanifold of ${\mathbb{R}}^N$ and R is a retraction, then there exist constants $M_1,M_2>0$ such that, for all $X\in \mathcal{M}$ and $\xi \in T_X\mathcal{M}$,

$$\parallel R_X\left(\xi \right)-X\parallel \le M_1\parallel \xi \parallel$$

(6)

$$\parallel R_X\left(\xi \right)-X-\xi \parallel \le M_2{\parallel \xi \parallel }^2$$

(7)

The following assumption is the most natural and practical way to extend the well-known L-smooth (i.e., the function has a Lipschitz-continuous gradient) concept from Euclidean space to Riemannian manifolds.

Assumption 1 

(L-retraction smoothness [20]). There exists a constant $L_g\ge 0$ such that, for the objective function f of problem (1), the following inequality holds:

$$\left|f\left(R_x\left(\eta \right)\right)-f\left(x\right)-{\langle \nabla f\left(x\right),\eta \rangle }_x\right|\le {\displaystyle \frac{L_g}{2}}{\parallel \eta \parallel }_x^2,\forall x\in \mathcal{M},\eta \in T_x\mathcal{M}.$$

(8)

This assumption ensures that the behavior of function f along the retraction path is well-posed, i.e., as long as $\eta$ is sufficiently small, predicting the function value at the next point using the gradient information at the current point is relatively accurate, ensuring that the gradient-based linear approximation is locally effective. Thus, a convergence analysis from Euclidean space (gradient descent, trust region, Newton method) can be extended to manifolds.

4. Zeroth-Order Riemannian Adaptive Regularized Proximal Quasi-Newton Newton (ZO-ARPQN)

4.1. Zeroth-Order Riemannian Gradient and Hessian

In existing manifold optimization methods, gradients or their tangent space projections are assumed to be directly available. However, in many real-world problems such as black-box optimization, noisy or nonsmooth functions, and high-dimensional models, explicit gradient computation is either infeasible or too costly. Zeroth-order optimization offers an effective alternative by relying only on function evaluations.

While classical Euclidean zeroth-order estimators (e.g., Gaussian smoothing or coordinate finite differences) can approximate gradients accurately, they require many function calls and do not generalize well to manifolds. To overcome this, we employ a random direction difference method that samples one or several random tangent directions and uses one-sided finite differences to estimate gradients. This approach makes the function query cost independent of the dimension, achieves good scalability to high-dimensional Riemannian problems, and balances theoretical soundness with computational efficiency.

Definition 2 

(Stochastic Difference Zeroth-Order Riemannian Gradient [20]). Generate $u=P{u}_0\in T_x\mathcal{M}$, where $u_0\sim \mathcal{N}(0,I_n)\in {\mathbb{R}}^n$, and $P\in {\mathbb{R}}^{n\times n}$ is an orthogonal projection matrix that projects vectors onto the tangent space $T_x\mathcal{M}$. Thus, u follows the distribution $\mathcal{N}(0,P{P}^{\top })$, i.e., the standard normal distribution on the tangent space; all eigenvalues of $P{P}^{\top }$ are either 0 (corresponding to eigenvectors orthogonal to the tangent space) or 1 (corresponding to eigenvectors embedded in the tangent space). The zeroth-order Riemannian gradient is defined as:

$$g_{\mu }\left(x\right)={\displaystyle \frac{f\left(R_x\left(\mu u\right)\right)-f\left(x\right)}{\mu }}u={\displaystyle \frac{f\left(R_x\left(\mu P{u}_0\right)\right)-f\left(x\right)}{\mu }}P{u}_0.$$

(9)

It should be noted that the projection matrix P is easy to compute on common manifolds. For example, for the Stiefel manifold $\mathcal{M}$, its projection can be written as

$${\mathrm{Proj}}_{T_X\mathcal{M}}\left(Y\right)=(I-X{X}^{\top })Y+X\mathrm{skew}\left(X^{\top }Y\right),$$

where $\mathrm{skew}\left(A\right):=(A-A^{\top })/2$. In the stochastic case, performing multiple samplings in each iteration can improve the convergence speed:

$${\overline{g}}_{\mu ,\xi }\left(x\right)={\displaystyle \frac{1}{m}}\sum _{i=1}^m{g}_{\mu ,{\xi }_i}\left(x\right),\mathrm{where}{g}_{\mu ,{\xi }_i}\left(x\right)={\displaystyle \frac{F\left(R_x\left(\mu u_i\right),{\xi }_i\right)-F\left(x,{\xi }_i\right)}{\mu }}{u}_i,$$

(10)

where $u_i$ is a standard normal random vector on the tangent space $T_x\mathcal{M}$. We also have:

$${\mathbb{E}}_{\xi }\left[g_{\mu ,{\xi }_i}\left(x\right)\right]={\displaystyle \frac{f\left(R_x\left(\mu u\right)\right)-f\left(x\right)}{\mu }}u=g_{\mu }\left(x\right).$$

(11)

Lemma 1 

(Upper bound on the error of the averaged zeroth-order gradient [20]). For the zeroth-order Riemannian gradient estimation with multiple samplings, we have

$$\mathbb{E}{∥{\overline{g}}_{\mu ,\xi }\left(x\right)-\nabla f\left(x\right)∥}^2\le {\mu }^2{L}_g^2{(d+6)}^3+{\displaystyle \frac{8(d+4)}{m}}{\sigma }^2+{\displaystyle \frac{8(d+4)}{m}}{\parallel \nabla f\left(x\right)\parallel }^2,$$

(12)

where the expectation $\mathbb{E}$ is taken over by the Gaussian vectors $U=\{u_1,\dots ,u_m\}$ and ξ.

Definition 3 

(Zeroth-Order Riemannian Hessian [20]). The zeroth-order Riemannian Hessian estimation of function f at point x is defined as:

$$H_{\mu }\left(x\right)={\displaystyle \frac{1}{2{\mu }^2}}\left(u{u}^{\top }-P\right)\left[F\left(R_x\left(\mu u\right),\xi \right)+F\left(R_x(-\mu u),\xi \right)-2F(x,\xi )\right].$$

(13)

It should be noted that the Riemannian Hessian here is actually the projected Hessian estimation of the pullback function

$$F_x(\eta ,\xi ):=F\left(R_x\left(\eta \right),\xi \right),\forall x\in \mathcal{M},\eta \in T_x\mathcal{M}$$

on the tangent space $T_x\mathcal{M}$. Additionally, multi-sampling technology is adopted. For $i=1,\dots ,b$, each Hessian is defined as:

$$H_{\mu ,i}\left(x\right)={\displaystyle \frac{1}{2{\mu }^2}}\left(u_i{u}_i^{\top }-P\right)\left[F\left(R_x\left(\mu u_i\right),{\xi }_i\right)+F\left(R_x(-\mu u_i),{\xi }_i\right)-2F(x,{\xi }_i)\right].$$

(14)

Its averaged form is:

$${\overline{H}}_{\mu ,\xi }\left(x\right)={\displaystyle \frac{1}{b}}\sum _{i=1}^b{H}_{\mu ,i}\left(x\right).$$

(15)

Assumption 2 

(Lipschitz Hessian assumption). For any $x\in \mathcal{M}$ and $\eta \in T_x\mathcal{M}$, we have

$${∥P_{\eta }^{-1}\circ \mathbb{H}F\left(R_x\left(\eta \right),\xi \right)\circ P_{\eta }-\mathbb{H}F(x,\xi )∥}_{\mathit{op}}\le L_H\parallel \eta \parallel ,$$

(16)

which holds almost everywhere, where $P_{\eta }:T_x\mathcal{M}\to T_{R_x\left(\eta \right)}\mathcal{M}$ denotes the parallel transport and ∘ denotes the function composition. Here, ${\parallel ·\parallel }_{\mathit{op}}$ denotes the operator norm. This assumption constrains the rate of change in the Hessian on the manifold.

Assumption 2 is the Riemannian counterpart of the Lipschitz Hessian assumption in Euclidean space, whose equivalent conditions include (see [13]):

$$∥P_{\eta }^{-1}\nabla F\left(R_x\left(\eta \right),\xi \right)-\nabla f\left(x\right)-\mathbb{H}F(x,\xi )\left[\eta \right]∥\le {\displaystyle \frac{L_H}{2}}{\parallel \eta \parallel }^2,$$

(17)

$$\left|F\left(R_x\left(\eta \right),\xi \right)-\left[F(x,\xi )+\langle \eta ,\nabla F(x,\xi )\rangle +{\displaystyle \frac{1}{2}}\langle \eta ,\mathbb{H}F(x,\xi )\left[\eta \right]\rangle \right]\right|\le {\displaystyle \frac{L_H}{6}}{\parallel \eta \parallel }^3.$$

(18)

In the Euclidean case, $P_{\eta }$ degenerates to the identity mapping. In this section, we also assume that $F(·,\xi )$ satisfies Assumption 1 and introduce the following common assumption, which is often used in zeroth-order stochastic optimization.

Assumption 3 

(Bounded Variance Assumption [19,21]). Let $E={\mathbb{E}}_{\xi }$; then, we have: $\mathbb{E}\left[F\right(x,\xi \left)\right]=f\left(x\right)$, $\mathbb{E}[\nabla F(x,\xi \left)\right]=\nabla f\left(x\right)$, ${\mathbb{E}\parallel \nabla F(x,\xi )-\nabla f\left(x\right)\parallel }^2\le {\sigma }^2$ for all $x\in \mathcal{M}$.

The error between the approximate Hessian matrix constructed using the zeroth-order method and the true Riemannian $\mathbb{H}f\left(x\right)$ is bounded in expectation. By choosing a sufficiently large number of samples b and a sufficiently small step size $\mu$, the approximation error can be made arbitrarily small, thereby ensuring the effectiveness and controllability of the zeroth-order Hessian estimation.

Lemma 2 

([20]). Let ${\overline{H}}_{\mu ,\xi }\left(x\right)$ be computed according to Formula (15), and $H_{\mu }\left(x\right)$ be computed according to Formula (13). Then, for any $x\in \mathcal{M}$ and $\eta \in T_x\mathcal{M}$, we have:

$${\mathbb{E}}_{\mu ,\xi }{∥{\overline{H}}_{\mu ,\xi }\left(x\right)-\mathbb{H}f\left(x\right)∥}_{\mathit{op}}^2\le {\displaystyle \frac{{(d+16)}^4}{\sqrt{2b}}}{L}_g+{\displaystyle \frac{{\mu }^2{L}_H^2}{18}}{(d+6)}^5,$$

(19)

$${\mathbb{E}}_{u,\mathrm{\Xi }}{∥H_{\mu }\left(x\right)∥}_F^4\le {\displaystyle \frac{{(d+16)}^8}{8}}{L}_g^2.$$

(20)

This lemma provides a theoretical basis for the zeroth-order quasi-Newton method: even without an explicit Hessian, a high-quality Hessian approximation can be constructed using function value approximation; it provides an error upper bound for subsequent convergence analyses, ensuring the theoretical feasibility of the method.

4.2. Zeroth-Order Extension: Zeroth-Order Riemannian Adaptive Regularized Proximal Quasi-Newton Algorithm

To address the scenario where first-order or second-order derivative information cannot be directly obtained, based on the above groundwork, we propose a zeroth-order information-based adaptive Riemannian optimization algorithm. Our method utilizes the aforementioned zeroth-order gradient and Hessian estimators to construct the following regularized quadratic approximation model in each iteration:

$${\tilde{\varphi }}_k\left(V\right)=\langle {\overline{g}}_{\mu ,\xi }\left(x_k\right),V\rangle +{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x_k\right)+{\sigma }_{k}I)\left[V\right],V\rangle +h(x_k+V),$$

(21)

where ${\overline{g}}_{\mu ,\xi }\left(x_k\right)$ denotes the multi-sampling mean of the zeroth-order Riemannian gradient, ${\overline{H}}_{\mu ,\xi }\left(x_k\right)$ is the mean of the zeroth-order Riemannian Hessian estimation, and ${\sigma }_k$ is the adaptive regularization parameter. By minimizing Subproblem (21) on the tangent space $T_{x_k}\mathcal{M}$, we can obtain a reasonable descent direction using only function values (zeroth-order information). Furthermore, the adaptive adjustment mechanism of the regularization parameter ${\sigma }_k$ ensures the stability and convergence of the algorithm under nonconvex conditions, thereby enabling the method to exhibit an excellent theoretical and practical performance in zeroth-order Riemannian optimization problems.

The greatest advantage of the zeroth-order extended adaptive Riemannian proximal quasi-Newton method is that it breaks the dependence on explicit gradients and Hessians, but can still closely approximate the second-order curvature information in black-box, non-differentiable, or even noisy environments, maintaining the efficiency and stability of the algorithm. This significantly expands the application scope of traditional Riemannian proximal Newton methods [16,22]. The complete optimization process is shown in Algorithm 1 (ZO-ARPQN). Below, we briefly describe the main steps of the ZO-ARPQN algorithm.

Algorithm 1 ZO-ARPQN: Zeroth-Order Adaptive Regularized Proximal Quasi-Newton Algorithm

Require:  Initial point $X_0\in \mathcal{M}$; initial regularization parameter ${\sigma }_0>0$; line search parameters $\sigma ,\gamma \in (0,1)$; threshold parameters $0<{\eta }_1<{\eta }_2<1$; sample batch sizes $m,b$; scaling factors $0<{\gamma }_1<1<{\gamma }_2$   1: for $k=0,1,2,\dots$ do   2:     Generate the zeroth-order gradient ${\overline{g}}_{\mu ,\xi }\left(x_k\right)$ and zeroth-order Hessian ${\overline{H}}_{\mu ,\xi }\left(x_k\right)$ based on sample sizes $m,b$   3:     while true do   4:         Solve the subproblem $${\tilde{\varphi }}_k\left(V\right)=\langle {\overline{g}}_{\mu ,\xi }\left(x_k\right),V\rangle +{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x_k\right)+{\sigma }_{k}I)\left[V\right],V\rangle +h(x_k+V),$$ to obtain the search direction $V_k$   5:         Set initial step size ${\alpha }_k=1$   6:         while Condition (22) is not satisfied do   7:            ${\alpha }_k\leftarrow \gamma {\alpha }_k$   8:         end while   9:         Let $Z_k=R_{X_k}\left({\alpha }_k{V}_k\right)$ 10:         Compute the ratio $${\rho }_k={\displaystyle \frac{F\left(R_{X_k}\left({\alpha }_k{V}_k\right)\right)-F\left(X_{l\left(k\right)}\right)}{{\tilde{\varphi }}_k\left({\alpha }_k{V}_k\right)-{\tilde{\varphi }}_k\left(0\right)}}.$$ 11:         if ${\rho }_k\ge {\eta }_1$ then 12:            if ${\rho }_k\ge {\eta }_2$ then 13:                Update ${\sigma }_k\leftarrow {\gamma }_1{\sigma }_k$ 14:            end if 15:            Break 16:         else 17:            Update ${\sigma }_k\leftarrow {\gamma }_2{\sigma }_k$ 18:         end if 19:     end while 20:     Update $X_{k+1}=Z_k$, ${\sigma }_{k+1}={\sigma }_k$ 21: end for

Step size selection strategy. In each iteration, the algorithm first obtains the search direction $V_k$ by solving the subproblem. Then, a non-monotone line search with the Armijo condition is used to determine the appropriate step size ${\alpha }_k$. Specifically, it is defined as:

$${\alpha }_k={\gamma }^{N_k},N_k\in \mathbb{N},$$

where $N_k$ is the smallest integer satisfying the following condition:

$$F\left(R_{X_k}\left({\alpha }_k{V}_k\right)\right)\le \underset{max\{0,k-m\}\le j\le k}{max}F\left(X_j\right)-{\displaystyle \frac{1}{2}}\sigma {\alpha }_k{\parallel V_k\parallel }_{B_k}^2.$$

(22)

The scaling factor $\gamma \in (0,1)$ ensures that ${\alpha }_k$ decreases gradually during backtracking; $\sigma >0$ is a line search parameter. The non-monotone condition allows the function value at the current iterate to not strictly decrease, but instead be compared with the maximum value among the most recent m iterations, thereby improving the algorithm’s robustness in complex nonconvex problems.

Model consistency measure (ratio definition). Let

$$F\left(X_{l\left(k\right)}\right)=\underset{max\{0,k-m\}\le j\le k}{max}F\left(X_j\right).$$

This serves as the reference benchmark for the function value in the current iteration. To measure the degree of matching between the constructed quadratic approximation model and the true function decrease, we introduce the ratio:

$${\rho }_k:={\displaystyle \frac{F\left(R_{x_k}\left({\alpha }_k{V}_k\right)\right)-F\left(X_{l\left(k\right)}\right)}{{\tilde{\varphi }}_k\left({\alpha }_k{V}_k\right)-{\tilde{\varphi }}_k\left(0\right)}}.$$

(23)

To simplify the notation, we introduce

$$l\left(k\right):={\mathrm{argmax}}_{max\{0,k-m\}\le j\le k}F\left(X_j\right).$$

(24)

The ratio ${\rho }_k$ characterizes the consistency between the predicted and actual decrease, where the numerator is the actual function decrease under step size ${\alpha }_k$ and the denominator is the predicted decrease in the quadratic approximation model.

Update strategy for regularization parameter ${\sigma }_k$. To ensure the stability and convergence of the algorithm, the regularization parameter ${\sigma }_k$ is dynamically adjusted based on the ratio ${\rho }_k$:

1.

If ${\rho }_k<{\eta }_1$, it indicates an inaccurate prediction or an excessively large step size, and the model is too optimistic:

$${\sigma }_{k+1}={\gamma }_2{\sigma }_k,{\gamma }_2>1,$$

enhancing the regularization strength to make the model more conservative.

2.

If ${\rho }_k\ge {\eta }_1$, it indicates an acceptable decrease performance.

3.

If ${\rho }_k\ge {\eta }_2$, it indicates an extremely accurate model prediction and an ideal decrease performance:

$${\sigma }_{k+1}={\gamma }_1{\sigma }_k,{\gamma }_1\in (0,1),$$

weakening the regularization.

4.

If ${\eta }_1\le {\rho }_k<{\eta }_2$, keep the regularization unchanged:

$${\sigma }_{k+1}={\sigma }_k.$$

This corresponds to the scenario where the model prediction is roughly consistent with the true decrease. If ${\sigma }_k$ is reduced hastily at this point, the model may become overly optimistic, leading to instability in the next iteration; if ${\sigma }_k$ is increased hastily, the model may become overly conservative, sacrificing convergence speed. Therefore, the most reasonable choice is to keep the regularization level unchanged. This design ensures that the algorithm balances convergence and efficiency in long-term operation, without affecting stability due to frequent adjustments of ${\sigma }_k$.

Finally, the new iterate is given by:

$$X_{k+1}=R_{X_k}\left({\alpha }_k{V}_k\right).$$

This algorithm integrates mechanisms from three aspects: (1) a non-monotone line search, which improves the algorithm’s robustness in nonconvex scenarios; (2) the model consistency ratio, which dynamically evaluates the difference between the predicted and true decreases; and (3) an adaptive regularization update, which balances convergence and efficiency by adjusting ${\sigma }_k$.

Thus, the ZO-ARPQN algorithm can efficiently simulate the behavior of second-order methods using only function values, and it balances the convergence stability and computational efficiency both theoretically and practically.

5. Convergence Analysis

In this subsection, we prove the global convergence of Algorithm 1. First, we present some standard assumptions that will be used in subsequent analyses.

Assumption 4. 

Let $\left\{X_k\right\}$ be the iterate sequence generated by Algorithm 1. Then, (1) $f:{\mathbb{R}}^{n\times r}\to \mathbb{R}$ is a continuously differentiable function, and its gradient $\nabla f$ is Lipschitz continuous with constant L; (2) $h:{\mathbb{R}}^{n\times r}\to \mathbb{R}$ is a convex, but nonsmooth, function, and h is Lipschitz continuous with constant $L_h$; (3) there exist constants $0<{\kappa }_1<{\kappa }_2$ such that, for all $k\ge 0$,

$${\kappa }_1{\parallel V\parallel }^2\le \langle {\overline{H}}_k\left[V\right],V\rangle \le {\kappa }_2{\parallel V\parallel }^2,\forall V\in T_{X_k}\mathcal{M};$$

(25)

and (4) the optimal solution $V_k$ of Subproblem (4) satisfies $\parallel V_k\parallel \ne 0$ for all $k\ge 0$.

Remark 1. 

In Assumption (4), the first two items are standard assumptions in the convergence analysis of composite optimization. From Assumption (4), the objective function ${\varphi }_k$ of Subproblem (21) is strongly convex, and thus has a unique solution, denoted as $V_k$. According to the first-order optimality condition of the original problem (1)

$$0\in \nabla f\left(X^{*}\right)+{\mathit{Proj}}_{T_{X^{*}}\mathcal{M}}(\partial h\left(X^{*}\right))$$

(26)

we have

$$0\in {\mathit{Proj}}_{T_{X_k}\mathcal{M}}\partial {\tilde{\varphi }}_k\left(V_k\right)=\nabla f\left(X_k\right)+({\tilde{H}}_k+{\sigma }_{k}I)\left[V_k\right]+{\mathit{Proj}}_{T_{X_k}\mathcal{M}}\partial h(X_k+V_k).$$

(27)

Therefore, $V_k=0$ is equivalent to $X_k$ satisfying (26), i.e., $X_k$ is a stationary point of Problem (1). If $V_k\ne 0$, similar to the proof of Lemma 5.1 in [23], it can be shown that $V_k$ brings a sufficient decrease in ${\varphi }_k$. For completeness, we provide the proof below.

Lemma 3. 

Given iterate $X_k$, let

$$\tilde{\varphi }\left(V\right):=\langle {\overline{g}}_{\mu ,\xi }\left(x\right),V\rangle +{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\sigma }_{k}I)\left[V\right],V\rangle +h(x+V)$$

(28)

be the objective function in (21). Then, for any $\alpha \in [0,1]$, we have:

$$\tilde{\varphi }\left(\alpha V_k\right)-\tilde{\varphi }\left(0\right)\le {\displaystyle \frac{\alpha (\alpha -2)}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\sigma }_{k}I)\left[V_k\right],V_k\rangle .$$

(29)

Proof. 

Since $\tilde{\varphi }$ is ${\displaystyle \frac{1}{t}}$-strongly convex, we have

$$\tilde{\varphi }\left(\widehat{V}\right)\ge \tilde{\varphi }\left(V\right)+\langle \partial \tilde{\varphi }\left(V\right),\widehat{V}-V\rangle +{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\alpha }_{k}I)\left[V\right],V\rangle ,\forall V,\widehat{V}\in {\mathbb{R}}^{n\times r}.$$

(30)

In particular, if $V,\widehat{V}$ are feasible solutions (i.e., $V,\widehat{V}\in T_{X_k}\mathcal{M}$), then:

$$\langle \partial \tilde{\varphi }\left(V\right),\widehat{V}-V\rangle =\langle {\mathrm{Proj}}_{T_{X_k}\mathcal{M}}\partial \tilde{\varphi }\left(V\right),\widehat{V}-V\rangle .$$

From the optimality condition, $0\in {\mathrm{Proj}}_{T_{X_k}\mathcal{M}}\partial \tilde{\varphi }\left(V_k\right)$. Let $V=V_k$ and substitute $\widehat{V}=0$ into (30); then, we obtain:

$$\tilde{\varphi }\left(0\right)\ge \tilde{\varphi }\left(V_k\right)+{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\alpha }_{k}I)\left[V\right],V\rangle .$$

According to the definitions of $\tilde{\varphi }\left(0\right)=h\left(X_k\right)$ and $\tilde{\varphi }\left(V_k\right)$, further expansion gives:

$$h\left(X_k\right)\ge \langle \nabla f\left(X_k\right),V_k\rangle +{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\alpha }_{k}I)\left[V\right],V\rangle +h(X_k+V_k)+{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\alpha }_{k}I)\left[V\right],V\rangle .$$

Additionally, using the convexity of h, we have:

$$h(X_k+\alpha V_k)=h(\alpha (X_k+V_k)+(1-\alpha )X_k)\le \alpha h(X_k+V_k)+(1-\alpha )h\left(X_k\right).$$

According to $\tilde{\varphi }\left(0\right)=h\left(X_k\right)$, expand ${\tilde{\varphi }}_k\left(\alpha V_k\right)$ by definition, and combine the above inequalities:

$$\begin{array}{cc}\hfill {\tilde{\varphi }}_k\left(\alpha V_k\right)-{\tilde{\varphi }}_k\left(0\right)& =\langle \nabla f\left(X_k\right),\alpha V_k\rangle +{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\sigma }_{k}I)\left[\alpha V_k\right],\alpha V_k\rangle +h(X_k+\alpha V_k)-h\left(X_k\right)\hfill \\ \hfill & \le \alpha \left(\langle \nabla f\left(X_k\right),V_k\rangle +{\displaystyle \frac{\alpha }{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\sigma }_{k}I)\left[V_k\right],V_k\rangle +h(X_k+V_k)-h\left(X_k\right)\right)\hfill \\ \hfill & =\alpha \left({\tilde{\varphi }}_k\left(V_k\right)-{\tilde{\varphi }}_k\left(0\right)+{\displaystyle \frac{\alpha -1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\sigma }_{k}I)\left[V_k\right],V_k\rangle \right)\hfill \\ \hfill & \le {\displaystyle \frac{\alpha (\alpha -2)}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x\right)+{\sigma }_{k}I)\left[V_k\right],V_k\rangle .\hfill \end{array}$$

(31)

This completes the proof. □

Regarding the boundedness of ${\sigma }_k$: An important part of the convergence analysis of regularized Newton-type methods is to prove the boundedness of the sequence $\left\{{\sigma }_k\right\}$. We first present a preparatory lemma. According to the procedure of Algorithm 1, if ${\rho }_k<{\eta }_1$, then ${\sigma }_k$ is scaled up by a factor of ${\gamma }_2$. We aim to prove that, when ${\sigma }_k$ is sufficiently large, ${\rho }_k\ge {\eta }_1$ always holds. Thus, the inner loop (Steps 3–15) of Algorithm 1 will terminate in finite steps. Since the manifold $\mathcal{M}$ is compact, we can define

$$\varrho :=\underset{X\in \mathcal{M}}{sup}\parallel \nabla f\left(X\right)\parallel .$$

Recall that parameters $M_1$ and $M_2$ are given by (6) and (7), respectively. Our goal is to provide a sufficient condition that ensures $\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1$.

Lemma 4. 

(Sufficient condition for $\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1$) Under the assumptions that f is L-smooth (Assumption (4)) and h is $L_h$-Lipschitz, define constants

$$c_1:=\varrho M_2+{\displaystyle \frac{1}{2}}L{M}_1^2,c_2:=c_1+L_h{M}_2,\varrho :=\underset{x\in \mathcal{M}}{sup}\parallel \nabla f\left(x\right)\parallel .$$

Let $V_k\in T_{x_k}\mathcal{M}$ be the solution of the subproblem ${min}_V{\tilde{\varphi }}_k\left(V\right)$, take step size ${\alpha }_k\in (0,1]$, denote ${\eta }_k={\alpha }_k{V}_k$, and define

$${\rho }_k:={\displaystyle \frac{F\left(x_k\right)-F\left(R_{x_k}\left({\eta }_k\right)\right)}{-{\tilde{\varphi }}_k\left({\eta }_k\right)+{\tilde{\varphi }}_k\left(0\right)}}.$$

If there exists ${\kappa }_1>0$ such that (corresponding to parameter (25))

$$-{\tilde{\varphi }}_k\left(\alpha V\right)+{\tilde{\varphi }}_k\left(0\right)\ge {\displaystyle \frac{\alpha (2-\alpha )}{2}}{\kappa }_1{\parallel V\parallel }^2,\forall \alpha \in [0,1],$$

(32)

and the regularization parameter ${\sigma }_k$ and trust region radius ${\parallel V\parallel }_k\le \Delta$ satisfy

$${\sigma }_k\ge \overline{\sigma }:=-{\kappa }_1+{\displaystyle \frac{2c_2}{2-{\eta }_1}}+{\displaystyle \frac{2}{2-{\eta }_1}}\left({\displaystyle \frac{{\epsilon }_g}{\Delta }}+{\epsilon }_H\right),$$

(33)

then we have the conclusion

$$\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1.$$

Proof. 

According to the strong convexity of f,

$$f\left(R_x\left(\eta \right)\right)\le f\left(x\right)+\langle \nabla f\left(x\right),\eta \rangle +{\displaystyle \frac{L_g}{2}}{\parallel \eta \parallel }^2.$$

(34)

Since h is $L_h$-Lipschitz in the ambient space, we have the following second-order retraction error:

$$\parallel R_x{\left(\eta \right)-(X+\eta )\parallel }_F\le M_2{\parallel \eta \parallel }^2.$$

(35)

Thus,

$$h\left(R_x\left(\eta \right)\right)\le h(x+\eta )+L_h{M}_2{\parallel \eta \parallel }^2.$$

(36)

Define updated constants

$$c_1:={\displaystyle \frac{L_g}{2}},c_2:=c_1+L_h{M}_2={\displaystyle \frac{L_g}{2}}+L_h{M}_2.$$

(37)

From the operator norm error of the Hessian

$${\epsilon }_H:=\mathbb{E}{\parallel {\overline{H}}_{\mu ,\xi }\left(x\right)-\mathbb{H}f\left(x\right)\parallel }_{\mathrm{op}}\le \tilde{C}{\displaystyle \frac{{(d+16)}^6}{b^{3/2}}}{L}_g^{1.5}+{\displaystyle \frac{1}{27}}{\mu }^3{L}_H^3{(d+6)}^{7.5}.$$

(38)

From Lemma 1, using Jensen’s inequality to extract the square root

$${\epsilon }_g:=\sqrt{{\mu }^2{L}_g^2{(d+\delta )}^3+{\displaystyle \frac{8(d+4)}{n}}{\sigma }^2+{\displaystyle \frac{8(d+4)}{n}}{\parallel \nabla f\left(x_k\right)\parallel }^2},\mathbb{E}\parallel {\overline{g}}_{\mu ,\xi }\left(x_k\right)-\nabla f\left(x_k\right)\parallel \le {\epsilon }_g.$$

(39)

There exists ${\kappa }_1>0$ such that

$$\langle ({\tilde{H}}_{\mu ,\xi }\left(x\right)+{\kappa }_{1}I)\left[V\right],V\rangle \ge 0.$$

(40)

Thus, for any $\alpha \in [0,1]$, there is a “strongly convex” lower bound for the model

$$-{\tilde{\varphi }}_k\left(\alpha V\right)+{\tilde{\varphi }}_k\left(0\right)\ge {\displaystyle \frac{\alpha (2-\alpha )}{2}}{\kappa }_1{\parallel V\parallel }^2.$$

(41)

From (34)–(36), for any $V\in T_x\mathcal{M}$, $\alpha >0$, we have

$$\begin{array}{cc}\hfill F\left(R_x\left(\alpha V\right)\right)& \le f\left(x\right)+\langle \nabla f\left(x\right),\alpha V\rangle +c_1{\parallel \alpha V\parallel }^2+h(x+\alpha V)+L_h{M}_2{\parallel \alpha V\parallel }^2\hfill \\ \hfill & =F\left(x\right)+\langle \nabla f\left(x\right),\alpha V\rangle +c_2{\parallel \alpha V\parallel }^2+h(x+\alpha V)-h\left(x\right).\hfill \end{array}$$

(42)

Add and subtract ${\tilde{\varphi }}_k\left(\alpha V\right)$ on the right-hand side, then add and subtract $\nabla f$ and $\overline{g}$, and use $\langle (\mathbb{H}f\left(x_k\right)+{\kappa }_{1}I)\left[U\right],U\rangle \ge 0$ to obtain

$$\begin{array}{cc}\hfill F\left(R_{x_k}\left(\alpha V\right)\right)& \le F\left(x_k\right)+{\tilde{\varphi }}_k\left(\alpha V\right)+\left(c_2-{\displaystyle \frac{1}{2}}({\kappa }_1+{\sigma }_k)\right){\parallel \alpha V\parallel }^2\hfill \\ \hfill & +\langle \nabla f\left(x_k\right)-{\overline{g}}_{\mu ,\xi }\left(x_k\right),\alpha V\rangle -{\displaystyle \frac{1}{2}}\langle ({\overline{H}}_{\mu ,\xi }\left(x_k\right)-\mathbb{H}f\left(x_k\right))\left[\alpha V\right],\alpha V\rangle .\hfill \end{array}$$

(43)

Take the expectation over $(\mu ,\xi )$ and apply the Cauchy–Schwarz inequality and (38) and (39); then, we have

$$\begin{array}{cc}\hfill F\left(R_{x_k}\left(\alpha V\right)\right)& \le F\left(x_k\right)+\mathbb{E}{\tilde{\varphi }}_k\left(\alpha V\right)+\left(c_2-{\displaystyle \frac{1}{2}}({\kappa }_1+{\sigma }_k)\right){\parallel \alpha V\parallel }^2\hfill \\ \hfill & +\underset{\mathrm{Gradient}\mathrm{Estimation}\mathrm{Error}}{\underbrace{\langle \nabla f\left(x_k\right)-{\overline{g}}_k,\alpha V\rangle }}-{\displaystyle \frac{1}{2}}\underset{\mathrm{Hessian}\mathrm{Estimation}\mathrm{Error}}{\underbrace{\langle ({\overline{H}}_k-\mathbb{H}f\left(x_k\right))\left[\alpha V\right],\alpha V\rangle }}.\hfill \end{array}$$

(44)

Take the expectation over $(\mu ,\xi )$ and apply the Cauchy–Schwarz inequality and (38) and (39):

$$\mathbb{E}F\left(R_{x_k}\left(\alpha V\right)\right)\le F\left(x_k\right)+\mathbb{E}{\tilde{\varphi }}_k\left(\alpha V\right)+\left(c_2-{\displaystyle \frac{1}{2}}({\kappa }_1+{\sigma }_k)+{\displaystyle \frac{1}{2}}{\epsilon }_H\right){\parallel \alpha V\parallel }^2+\alpha {\epsilon }_g\parallel V\parallel .$$

(45)

Let $V_k$ be the solution of the subproblem ${min}_V{\tilde{\phi }}_k\left(V\right)$, and define

$${\rho }_k:={\displaystyle \frac{F\left(x_k\right)-F\left(R_{x_k}\left({\alpha }_k{V}_k\right)\right)}{-{\tilde{\varphi }}_k\left({\alpha }_k{V}_k\right)+{\tilde{\varphi }}_k\left(0\right)}}.$$

(46)

From (32) and (45), we have

$$\begin{array}{cc}\hfill 1-\mathbb{E}\left[{\rho }_k\right]& ={\displaystyle \frac{\mathbb{E}[F\left(R_{x_k}\left({\alpha }_k{V}_k\right)\right)-F\left(x_k\right)-{\tilde{\varphi }}_k\left({\alpha }_k{V}_k\right)+{\tilde{\varphi }}_k\left(0\right)]}{-{\tilde{\varphi }}_k\left({\alpha }_k{V}_k\right)+{\tilde{\varphi }}_k\left(0\right)}}\hfill \\ \hfill & \le {\displaystyle \frac{\left(c_2-\frac{1}{2}({\kappa }_1+{\sigma }_k)+\frac{1}{2}{\epsilon }_H\right)\parallel {\alpha }_k{V}_k{\parallel }^2+{\alpha }_k{\epsilon }_g\parallel V_k\parallel }{\frac{{\alpha }_k(2-{\alpha }_k)}{2}{\kappa }_1{\parallel V_k\parallel }^2}}\hfill \\ \hfill & ={\displaystyle \frac{2}{{\kappa }_1(2-{\alpha }_k)}}\left(c_2-{\displaystyle \frac{1}{2}}({\kappa }_1+{\sigma }_k)+{\displaystyle \frac{1}{2}}{\epsilon }_H+{\displaystyle \frac{{\epsilon }_g}{\parallel V_k\parallel }}\right).\hfill \end{array}$$

(47)

To ensure $\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1$, it is sufficient to set the last term ≤1$-{\eta }_1$ and solve for ${\sigma }_k$ to obtain the equivalent condition

$${\sigma }_k\ge -{\kappa }_1(2-{\alpha }_k)(1-{\eta }_1)+{\displaystyle \frac{2{\epsilon }_g}{\parallel V_k\parallel }}+2c_2+{\epsilon }_H-{\kappa }_1.$$

(48)

Additionally, $0\le {\alpha }_k\le 1$; thus, $1\le 2-{\alpha }_k\le 2$. Introduce the trust region radius $\parallel V_k \parallel \le \Delta$; thus, it is sufficient to have

$${\sigma }_k\ge 2c_2+{\epsilon }_H+{\displaystyle \frac{2{\epsilon }_g}{\Delta }}-{\kappa }_1(2-{\eta }_1)$$

(49)

to ensure $\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1$. □

Lemma 5 

(Boundedness of the Regularization Parameter ${\sigma }_k$). Assume that the solution to the subproblem satisfies a uniform radius upper bound $∥V_k∥ \le \Delta$ (attainable via the trust region radius or proximal regularization term). Let:

$${\sigma }_{\mathit{est}}:=2c_2+{\epsilon }_H+{\displaystyle \frac{2{\epsilon }_g}{\Delta }}-{\kappa }_1(2-{\eta }_1)$$

(50)

Then, for all $k\ge 0$, the following holds:

$${\sigma }_k\le max\left\{{\sigma }_0,{\gamma }_2{\sigma }_{\mathit{est}}\right\}$$

(51)

Proof. 

From Equation (49), we know that, if ${\sigma }_k\ge {\sigma }_{\mathrm{est}}$, then $\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1$ always holds. We prove the conclusion by mathematical induction:

Base Case ($k=0$): It is trivially true that ${\sigma }_0\le max\{{\sigma }_0,{\gamma }_2{\sigma }_{\mathrm{est}}\}$.

Inductive Hypothesis: Assume that the inequality holds for $k=j$, i.e., ${\sigma }_j\le max\{{\sigma }_0,{\gamma }_2{\sigma }_{\mathrm{est}}\}$.

Inductive Step ($k=j+1$): Consider two cases: (1) If ${\sigma }_j<{\sigma }_{\mathrm{est}}$: By Lemma 4, ${\sigma }_j$ fails to satisfy the acceptance condition in Step 10 of Algorithm 1. According to Step 15 of the algorithm, ${\sigma }_j$ is scaled up by a factor of ${\gamma }_2>1$, so ${\sigma }_{j+1}={\gamma }_2{\sigma }_j<{\gamma }_2{\sigma }_{\mathrm{est}}$. (2) If ${\sigma }_j\ge {\sigma }_{\mathrm{est}}$: by Lemma 4, $\mathbb{E}\left[{\rho }_j\right]\ge {\eta }_1$. According to Steps 10–13 of the algorithm, ${\sigma }_{j+1}$ is either unchanged or scaled down by ${\gamma }_1\in (0,1)$ (i.e., ${\sigma }_{j+1}={\gamma }_1{\sigma }_j$). Thus, ${\sigma }_{j+1}\le {\sigma }_j\le max\{{\sigma }_0,{\gamma }_2{\sigma }_{\mathrm{est}}\}$ (by the inductive hypothesis).

By combining both cases, we have ${\sigma }_{j+1}\le max\{{\sigma }_0,{\gamma }_2{\sigma }_{\mathrm{est}}\}$. By the principle of mathematical induction, the inequality holds for all $k\ge 0$. □

Theorem 1 

(Convergence to Stationary Points). Define:

$${\overline{\alpha }}_{\mathit{est}}:=min\left\{1,{\displaystyle \frac{(2-\sigma ){\kappa }_1}{2\left(c_2+\frac{1}{2}{\epsilon }_H+\frac{{\epsilon }_g}{\Delta }\right)}}\right\}$$

(52)

When ${\alpha }_k\ge \gamma {\overline{\alpha }}_{\mathrm{est}}$ for all $k\ge 0$ (where $\gamma \in (0,1)$ is the backtracking scaling factor in Step 7 of the algorithm), the backtracking line search (Steps 6–8 of the algorithm) terminates in finite steps. Furthermore, we have:

$$\underset{k\to \infty }{lim}\mathbb{E}\left[∥V_k∥\right]=0$$

(53)

and any accumulation point $X^{*}$ of the iterate sequence $\left\{X_k\right\}$ is a first-order stationary point of the problem $F\left(X\right)=f\left(X\right)+h\left(X\right)$ (i.e., $\nabla F\left(X^{*}\right)=0$).

Proof. 

Substitute Equation (54) into Equation (47). When $\alpha \le {\overline{\alpha }}_{\mathrm{est}}$, the following inequality holds:

$$\left(c_2-{\displaystyle \frac{1}{2}}({\kappa }_1+{\sigma }_k)+{\displaystyle \frac{1}{2}}{\epsilon }_H\right)\alpha +{\displaystyle \frac{{\epsilon }_g}{\Delta }}\le {\displaystyle \frac{1-{\eta }_1}{2}}{\kappa }_1(2-\alpha )$$

(54)

This implies that $\mathbb{E}\left[{\rho }_k\right]\ge {\eta }_1$, so the step size $\alpha$ is accepted. For the backtracking line search, we start with $\alpha =1$ and scale it down by $\gamma$ iteratively. Since $\alpha$ will eventually be reduced to $\le {\overline{\alpha }}_{\mathrm{est}}$ (or ${\sigma }_k$ will be increased to $\ge {\overline{\sigma }}_{\mathrm{est}}$), the line search terminates in finite steps, and ${\alpha }_k\ge \gamma {\overline{\alpha }}_{\mathrm{est}}$.

From Equation (32) and the lower bound of ${\alpha }_k$, we derive:

$$\mathbb{E}\left[F\left(x_k\right)-F\left(x_{k+1}\right)\right]\ge {\displaystyle \frac{{\eta }_1{\kappa }_1}{2}}{\alpha }_{\min }(2-{\alpha }_{\min })\mathbb{E}\left[{∥V_k∥}^2\right]$$

(55)

where ${\alpha }_{\min }=\gamma {\overline{\alpha }}_{\mathrm{est}}>0$. By summing both sides over k from 0 to ∞, the left-hand side becomes $\mathbb{E}\left[F\left(x_0\right)-{lim}_{k\to \infty }F\left(x_k\right)\right]$. Since $\left\{\mathbb{E}\left[F\left(x_{l\left(k\right)}\right)\right]\right\}$ is non-increasing and bounded below (as F is bounded from below), the sum converges:

$$\sum _{k=0}^{\infty }{\alpha }_k\mathbb{E}\left[{∥V_k∥}^2\right]<\infty$$

(56)

Because ${\alpha }_k\ge {\alpha }_{\min }>0$, the terms of the series ${\sum }_{k=0}^{\infty }\mathbb{E}\left[{∥V_k∥}^2\right]$ must tend to zero. Thus:

$$\underset{k\to \infty }{lim}\mathbb{E}\left[∥V_k∥\right]=0$$

To prove that accumulation points are stationary, let $X^{*}$ be an accumulation point of $\left\{X_k\right\}$, and let $\left\{X_{k_i}\right\}$ be a subsequence converging to $X^{*}$. From ${lim}_{i\to \infty }\mathbb{E}\left[∥V_{k_i}∥\right]=0$, we have ${lim}_{i\to \infty }{V}_{k_i}=0$ (by the continuity of the norm). Substitute $V_{k_i}\to 0$ into the first-order optimality condition of the subproblem:

$$0\in {\overline{g}}_{\mu ,\xi }\left(X_{k_i}\right)+({\overline{H}}_{\mu ,\xi }\left(X_{k_i}\right)+{\sigma }_{k_i}I)V_{k_i}+{\mathrm{Proj}}_{T_{X_{k_i}}\mathcal{M}}\partial h(X_{k_i}+V_{k_i})$$

By taking the limit as $i\to \infty$ and using the continuity of $\nabla f$, error, and $\partial h$, we obtain the first-order optimality condition of the original problem:

$$0\in \nabla f\left(X^{*}\right)+{\mathrm{Proj}}_{T_{X^{*}}\mathcal{M}}(\partial h\left(X^{*}\right))$$

This confirms that $X^{*}$ is a stationary point of Problem (1). □

5.1. Complexity Analysis

When analyzing the overall complexity of stochastic zeroth-order Riemannian optimization algorithms, we must consider both the computational cost of the outer iterations (Steps 1–16 of the algorithm) and the inner iterations (Steps 3–15 of the algorithm). The former characterizes the number of main iterations required to converge to an $\epsilon$-stationary point, while the latter reflects the number of calls to the approximate subproblem solver on the manifold (ASSN) within each outer iteration. Due to the non-monotone line search and trust region/regularization update mechanisms, there is a tight coupling between the complexities of the outer and inner iterations.

In this proof, the acceptance criterion is defined as:

$${\rho }_k\ge {\eta }_1$$

(57)

Definition 4 

($\epsilon$-Stationary Point). If the optimal solution $V_k$ of the subproblem in a certain outer iteration satisfies $∥V_k∥=\epsilon$, then $X_k$ is called an ε-stationary point.

Theorem 2 

(Outer and Inner Iteration Complexity). Let $F^{*}$ be the optimal value of Problem (1), and $\sigma \in (0,1)$ be the constant in the sufficient decrease criterion. Define:

$$\mathrm{\Theta }:={\displaystyle \frac{2\left(F\left(x_0\right)-F^{*}\right)}{{\eta }_1{\kappa }_1\gamma \overline{\alpha }(2-\gamma \overline{\alpha }){\epsilon }^2}}$$

(58)

The algorithm finds an ε-stationary point within, at most, $\lceil \mathrm{\Theta }\rceil$ outer iterations. Furthermore, the total number of inner ASSN calls satisfies:

$$\sum _{i=0}^{\lceil \mathrm{\Theta }\rceil -1}r\left(i\right)\le \lceil \mathrm{\Theta }\rceil \left(2-{log}_{{\gamma }_2}{\gamma }_1\right)+{log}_{{\gamma }_2}\left({\displaystyle \frac{max\left\{{\sigma }_0,{\gamma }_2\overline{\sigma }\right\}}{{\sigma }_0}}\right)$$

(59)

Proof. 

Step 1: Upper Bound on Outer Iterations From Equation (55) and ${\alpha }_k\ge {\alpha }_{\min }:=\gamma \overline{\alpha }$, we have:

$$\mathbb{E}\left[F\left(X_k\right)-F\left(X_{k+1}\right)\right]\ge {\displaystyle \frac{{\eta }_1{\kappa }_1}{2}}{\alpha }_{\min }(2-{\alpha }_{\min })\mathbb{E}\left[{∥V_k∥}^2\right]$$

(60)

Assume the K-th outer iteration yields $∥V_K∥=\epsilon$. By summing both sides over k from 0 to $K-1$, the left-hand side telescopes to $\mathbb{E}\left[F\left(X_0\right)-F\left(X_K\right)\right]$. Since $F\left(X_K\right)\ge F^{*}$, we get:

$$\mathbb{E}\left[F\left(X_0\right)-F^{*}\right]\ge K·{\displaystyle \frac{{\eta }_1{\kappa }_1}{2}}{\alpha }_{\min }(2-{\alpha }_{\min }){\epsilon }^2$$

(61)

Rearranging for K gives:

$$K\le {\displaystyle \frac{2\left(\mathbb{E}\left[F\left(X_0\right)-F^{*}\right]\right)}{{\eta }_1{\kappa }_1{\alpha }_{\min }(2-{\alpha }_{\min }){\epsilon }^2}}=\mathrm{\Theta }$$

(62)

Taking the ceiling function confirms that an $\epsilon$-stationary point is found within $\lceil \mathrm{\Theta }\rceil$ outer iterations.

Step 2: Upper Bound on Inner Iterations By the boundedness of ${\sigma }_k$ (Lemma 5):

$${\sigma }_k\le max\left\{{\sigma }_0,{\gamma }_2\overline{\sigma }\right\}$$

(63)

Suppose the inner loop calls ASSN $r\left(i\right)$ times in the i-th outer iteration. By the update rule of $\sigma$, we have ${\sigma }_{i+1}={\gamma }_2^{r\left(i\right)-1}{\sigma }_i$. Since ${\gamma }_1<1$, the following inequality holds:

$${\displaystyle \frac{{\sigma }_{i+1}}{{\sigma }_i}}\ge {\gamma }_1{\gamma }_2^{r\left(i\right)-1}\Rightarrow r\left(i\right)\le 1+{log}_{{\gamma }_2}\left({\displaystyle \frac{{\sigma }_{i+1}}{{\sigma }_i}}·{\displaystyle \frac{{\gamma }_2}{{\gamma }_1}}\right)$$

(64)

umming both sides from $i=0$ to $i=K-1$ (where $K=\lceil \mathrm{\Theta }\rceil$) results in the following:

$$\sum _{i=0}^{K-1}r\left(i\right)\le K+\sum _{i=0}^{K-1}{log}_{{\gamma }_2}\left({\displaystyle \frac{{\sigma }_{i+1}}{{\sigma }_i}}\right)+K{log}_{{\gamma }_2}\left({\displaystyle \frac{{\gamma }_2}{{\gamma }_1}}\right)$$

The sum of logarithms simplifies to the logarithm of the product (telescoping sum):

$$\sum _{i=0}^{K-1}{log}_{{\gamma }_2}\left({\displaystyle \frac{{\sigma }_{i+1}}{{\sigma }_i}}\right)={log}_{{\gamma }_2}\left({\displaystyle \frac{{\sigma }_K}{{\sigma }_0}}\right)$$

By substituting Equation (63) (i.e., ${\sigma }_K\le max\{{\sigma }_0,{\gamma }_2\overline{\sigma }\}$) and simplifying the logarithmic term, we obtain the following:

$$\sum _{i=0}^{K-1}r\left(i\right)\le K\left(2-{log}_{{\gamma }_2}{\gamma }_1\right)+{log}_{{\gamma }_2}\left({\displaystyle \frac{max\left\{{\sigma }_0,{\gamma }_2\overline{\sigma }\right\}}{{\sigma }_0}}\right)$$

(65)

This completes the proof. □

5.2. Saddle-Point Escape Theorem

In non-convex optimization problems, the presence of saddle points severely degrades the convergence efficiency of algorithms. Unlike local minima, saddle points have zero gradients while their Hessian matrices have negative eigenvalues. This causes first-order methods (e.g., stochastic gradient descent) to easily stagnate near saddle points, leading to prolonged slow convergence or even halting. This phenomenon is more prominent in high-dimensional manifold optimization, where the number of saddle points far exceeds that of local minima.

In recent years, second-order methods (e.g., Hessian-based algorithms) have been proven to escape saddle points more effectively. However, in the zeroth-order setting, we cannot directly access accurate gradient or Hessian information, and can only rely on function value approximations. This makes the identification and escape of saddle points more challenging. Traditional zeroth-order methods often require excessive sampling and computation, resulting in a high complexity that hinders practical application. The proposed method in this paper achieves saddle-point escape using only function value queries, without the need for explicit first-order or second-order information.

Lemma 6. 

Let $g_k=\nabla f\left(x_k\right)$, ${\overline{g}}_{\mu }\left(x_k\right)$ be the zeroth-order gradient estimate, and ${\overline{H}}_{\mu }\left(x_k\right)={\displaystyle \frac{1}{b}}{\sum }_{i=1}^b{H}_{\mu ,i}\left(x_k\right)$ be the mini-batch average of Hessian estimates. The following control inequality holds:

$$∥{\overline{g}}_{\mu }\left(x_k\right)+{\overline{H}}_{\mu }\left(x_k\right)\left[{\eta }_k\right]∥\le {\displaystyle \frac{1}{2}}{∥{\eta }_k∥}^2+\sqrt{{\delta }_g}+{\delta }_{\mu }$$

(66)

where ${\delta }_g$ and ${\delta }_{\mu }$ correspond to the zeroth-order gradient estimation error and Hessian mini-batch estimation error, respectively.

Proof. 

Expand using the triangle inequality:

$$∥{\overline{g}}_{\mu }\left(x_k\right)+{\overline{H}}_{\mu }\left(x_k\right)\left[{\eta }_k\right]∥\le ∥g_k-{\overline{g}}_{\mu }\left(x_k\right)∥+∥{\overline{H}}_{\mu }\left(x_k\right)\left[{\eta }_k\right]∥+∥g_k∥$$

Step 1: Bound on $∥{\overline{H}}_{\mu }\left(x_k\right)\left[{\eta }_k\right]∥$. Apply the Frobenius norm and Cauchy–Schwarz inequality:

$$∥{\overline{H}}_{\mu }\left(x_k\right)\left[{\eta }_k\right]∥\le ∥[∥F]{\overline{H}}_{\mu }\left(x_k\right)∥{\eta }_k∥$$

Step 2: Apply Young’s Inequality. For the product term, Young’s inequality ($ab\le {\displaystyle \frac{a^2}{2}}+{\displaystyle \frac{b^2}{2}}$) gives:

$$∥[∥F]{\overline{H}}_{\mu }\left(x_k\right)∥{\eta }_k∥\le {\displaystyle \frac{1}{2}}{∥{\eta }_k∥}^2+{\displaystyle \frac{1}{2}}∥[∥F]{{\overline{H}}_{\mu }\left(x_k\right)}^2$$

(67)

Step 3: Bound on $\mathbb{E}\left[∥[∥F]{{\overline{H}}_{\mu }\left(x_k\right)}^2\right]$. Note that ${\overline{H}}_{\mu }\left(x_k\right)$ is the average of b independent and identically distributed (i.i.d.) terms $H_{\mu ,i}\left(x_k\right)$. By Jensen’s inequality and Cauchy–Schwarz, we have:

$$\mathbb{E}\left[∥[∥F]{{\overline{H}}_{\mu }\left(x_k\right)}^2\right]\le {\displaystyle \frac{1}{b}}\sum _{i=1}^b\mathbb{E}\left[∥[∥F]{H_{\mu ,i}\left(x_k\right)}^2\right]\le {\displaystyle \frac{1}{b}}{\left(\mathbb{E}\left[∥[∥F]{H_{\mu }\left(x_k\right)}^4\right]\right)}^{1/2}$$

Using the fourth-moment bound from Equation (20) ($\mathbb{E}\left[∥[∥F]{H_{\mu }\left(x_k\right)}^4\right]\lesssim {\displaystyle \frac{{(d+16)}^8{L}_g^2}{8}}$), we define:

$${\displaystyle \frac{1}{2}}\mathbb{E}\left[∥[∥F]{{\overline{H}}_{\mu }\left(x_k\right)}^2\right]\le {\displaystyle \frac{{(d+16)}^4}{4\sqrt{2}}}{L}_g:={\delta }_{\mu }$$

(68)

Step 4: Combine Results. Substitute Equation (67) and the gradient estimation error bound (from Lemma 1, $\mathbb{E}\left[∥g_k-{\overline{g}}_{\mu }\left(x_k\right)∥\right]\le \sqrt{{\delta }_g}$) into the triangle inequality. Taking the expectation and simplifying gives:

$$\mathbb{E}\left[∥{\overline{g}}_{\mu }\left(x_k\right)+{\overline{H}}_{\mu }\left(x_k\right)\left[{\eta }_k\right]∥\right]\le {\displaystyle \frac{1}{2}}\mathbb{E}\left[{∥{\eta }_k∥}^2\right]+\sqrt{{\delta }_g}+{\delta }_{\mu }$$

This completes the proof. □

Lemma 7. 

Let $x_{k+1}=R_{x_k}\left({\eta }_k\right)$ and $g_k=\nabla f\left(x_k\right)$. The following holds:

$$\mathbb{E}\left[∥g_{k+1}∥\right]\le {\displaystyle \frac{L_H+1}{2}}\mathbb{E}\left[{∥{\eta }_k∥}^2\right]+\sqrt{{\delta }_g}+{\delta }_H+{\delta }_{\mu }$$

(69)

where ${\delta }_g$, ${\delta }_H$, and ${\delta }_{\mu }$ correspond to the zeroth-order gradient estimation error, zeroth-order Hessian estimation error, and mini-batch averaging error, respectively.

Proof. 

Since parallel transport $P_{{\eta }_k}$ is isometric, we have:

$$∥g_{k+1}∥=∥P_{{\eta }_k}^{-1}{g}_{k+1}∥$$

Add and subtract $g_k+\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]$ to decompose the term into four components:

$$\begin{array}{cc}\hfill P_{{\eta }_k}^{-1}{g}_{k+1}& =\underset{\mathrm{Taylor}\mathrm{Remainder}}{\underbrace{P_{{\eta }_k}^{-1}{g}_{k+1}-g_k-\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]}}+\underset{\mathrm{Gradient}\mathrm{Estimation}\mathrm{Error}}{\underbrace{g_k-{\overline{g}}_{\mu ,\xi }\left(x_k\right)}}\hfill \\ & +\underset{\mathrm{Hessian}\mathrm{Estimation}\mathrm{Error}}{\underbrace{\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]-{\overline{H}}_{\mu ,\xi }\left(x_k\right)\left[{\eta }_k\right]}}+\underset{\mathrm{Subproblem}\mathrm{Main}\mathrm{Term}}{\underbrace{{\overline{g}}_{\mu ,\xi }\left(x_k\right)+{\overline{H}}_{\mu ,\xi }\left(x_k\right)\left[{\eta }_k\right]}}\hfill \end{array}$$

Taking the norm and applying the triangle inequality yields the following:

$$∥g_{k+1}∥\le ∥P_{{\eta }_k}^{-1}{g}_{k+1}-g_k-\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]∥+∥g_k-{\overline{g}}_{\mu ,\xi }∥+∥\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]-{\overline{H}}_{\mu ,\xi }\left[{\eta }_k\right]∥+∥{\overline{g}}_{\mu ,\xi }+{\overline{H}}_{\mu ,\xi }\left[{\eta }_k\right]∥$$

(i) Bound on the Taylor Remainder. By the equivalent condition of Assumption 2 (Lipschitz Hessian, Equation (17)),

$$∥P_{{\eta }_k}^{-1}{g}_{k+1}-g_k-\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]∥\le {\displaystyle \frac{L_H}{2}}{∥{\eta }_k∥}^2$$

(ii) Bound on Gradient Estimation Error. From Lemma 1, the expectation of the gradient estimation error satisfies $\mathbb{E}\left[∥g_k-{\overline{g}}_{\mu ,\xi }\left(x_k\right)∥\right]\le \sqrt{{\delta }_g}$.

(iii) Bound on Hessian Estimation Error. Using the operator norm and Young’s inequality:

$$∥\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]-{\overline{H}}_{\mu ,\xi }\left[{\eta }_k\right]∥\le ∥[∥\mathrm{op}]\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }∥{\eta }_k∥\le {\displaystyle \frac{1}{2}}{∥{\eta }_k∥}^2+{\displaystyle \frac{1}{2}}∥[∥\mathrm{op}]{\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }}^2$$

Taking the expectation and using Lemma 2 ($\mathbb{E}\left[∥[∥\mathrm{op}]{\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }}^2\right]\le {\delta }_H$), we get:

$$\mathbb{E}\left[∥\mathbb{H}f\left(x_k\right)\left[{\eta }_k\right]-{\overline{H}}_{\mu ,\xi }\left[{\eta }_k\right]∥\right]\le {\displaystyle \frac{1}{2}}\mathbb{E}\left[{∥{\eta }_k∥}^2\right]+{\delta }_H$$

(iv) Bound on the Subproblem Main Term. By Lemma 6, we have $\mathbb{E}\left[∥{\overline{g}}_{\mu ,\xi }+{\overline{H}}_{\mu ,\xi }\left[{\eta }_k\right]∥\right]\le {\displaystyle \frac{1}{2}}\mathbb{E}\left[{∥{\eta }_k∥}^2\right]+\sqrt{{\delta }_g}+{\delta }_{\mu }$.

(v) Combine All Bounds. Substitute (i)–(iv) into the triangle inequality and simplify using algebraic manipulations:

$$\mathbb{E}\left[∥g_{k+1}∥\right]\le \left({\displaystyle \frac{L_H}{2}}+1\right)\mathbb{E}\left[{∥{\eta }_k∥}^2\right]+\sqrt{{\delta }_g}+{\delta }_H+{\delta }_{\mu }$$

This completes the proof. □

Lemma 8 

(Relationship Between Step Size and Minimum Eigenvalue). Let $x_{k+1}=R_{x_k}\left({\eta }_k\right)$, and $P_{{\eta }_k}:T_{x_k}\mathcal{M}\to T_{x_{k+1}}\mathcal{M}$ be the isometric parallel transport along ${\eta }_k$. Let ${\sigma }_k\ge 0$ be such that ${\overline{H}}_{\mu ,\xi }+{\sigma }_{k}I⪰0$. Then, the following holds:

$$\mathbb{E}\left[∥{\eta }_k∥\right]\ge -{\displaystyle \frac{\sqrt{{\delta }_H}+{\sigma }_k+\mathbb{E}\left[{\lambda }_{\mathit{min}}\left(\mathbb{H}f\left(x_{k+1}\right)\right)\right]}{L_H}}$$

(70)

Proof. 

Step 1: Invariance of Minimum Eigenvalue Under Parallel Transport. Since parallel transport is isometric, the minimum eigenvalue of the Hessian is invariant:

$${\lambda }_{\min }\left(\mathbb{H}f\left(x_{k+1}\right)\right)={\lambda }_{\min }\left(P_{{\eta }_k}^{-1}\mathbb{H}f\left(x_{k+1}\right)P_{{\eta }_k}\right)$$

Let ${\Delta }_k:=P_{{\eta }_k}^{-1}\mathbb{H}f\left(x_{k+1}\right)P_{{\eta }_k}-\mathbb{H}f\left(x_k\right)$. Then:

$${\lambda }_{\min }\left(\mathbb{H}f\left(x_{k+1}\right)\right)={\lambda }_{\min }(\mathbb{H}f\left(x_k\right)+{\Delta }_k)$$

Step 2: Apply Weyl’s Inequality. Weyl’s inequality states that, for symmetric matrices A and B, ${\lambda }_{\min }(A+B)\ge {\lambda }_{\min }\left(A\right)+{\lambda }_{\min }\left(B\right)$. For any symmetric matrix A, the minimum eigenvalue satisfies ${\lambda }_{\min }\left(A\right)\ge -∥[∥\mathrm{op}]A$ (by the Rayleigh quotient characterization). Thus:

$${\lambda }_{\min }\left({\Delta }_k\right)\ge -∥[∥\mathrm{op}]{\Delta }_k$$

By Assumption 2 (Equation (16)), $∥[∥\mathrm{op}]{\Delta }_k\le L_H∥{\eta }_k∥$. Substituting into Weyl’s inequality yields the following:

$${\lambda }_{\min }\left(\mathbb{H}f\left(x_{k+1}\right)\right)\ge {\lambda }_{\min }\left(\mathbb{H}f\left(x_k\right)\right)-L_H∥{\eta }_k∥$$

(71)

Step 3: Decompose the Hessian and Introduce Regularization. Rewrite the true Hessian as:

$$\mathbb{H}f\left(x_k\right)=(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }\left(x_k\right))+({\overline{H}}_{\mu ,\xi }\left(x_k\right)+{\sigma }_{k}I)-{\sigma }_{k}I$$

Substitute into Equation (71) and apply Weyl’s inequality again. Since ${\overline{H}}_{\mu ,\xi }+{\sigma }_{k}I⪰0$, its minimum eigenvalue is non-negative, so:

$${\lambda }_{\min }\left(\mathbb{H}f\left(x_{k+1}\right)\right)\ge {\lambda }_{\min }(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }\left(x_k\right))-{\sigma }_k-L_H∥{\eta }_k∥$$

(72)

Step 4: Take Expectation and Rearrange. For the symmetric matrix $\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }\left(x_k\right)$, we have ${\lambda }_{\min }(·)\ge -∥[∥\mathrm{op}]·$. Taking the expectation and using Lemma 2 ($\mathbb{E}\left[∥[∥\mathrm{op}]\mathbb{H}f\left(x_k\right)\right]$$-{\overline{H}}_{\mu ,\xi }\left(x_k\right)\le \sqrt{{\delta }_H}$) yields the following:

$$\mathbb{E}\left[{\lambda }_{\min }(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }\left(x_k\right))\right]\ge -\sqrt{{\delta }_H}$$

Taking the expectation of Equation (72) and rearranging terms gives:

$$\mathbb{E}\left[{\lambda }_{\min }\left(\mathbb{H}f\left(x_{k+1}\right)\right)\right]\ge -\sqrt{{\delta }_H}-{\sigma }_k-L_H\mathbb{E}\left[∥{\eta }_k∥\right]$$

Rearranging for $\mathbb{E}\left[∥{\eta }_k∥\right]$ completes the proof. □

Theorem 3. 

Let $\mathcal{M}$ be a manifold, and let $f:\mathcal{M}\to \mathbb{R}$ satisfy Assumptions 1–3. Define:

$$k_{min}:={argmin}_k{\mathbb{E}}_{u_k,{\mathrm{\Xi }}_k}∥{\eta }_k∥$$

If the step size in the update of Algorithm 1 satisfies $\alpha \ge L_H$, then:

$$\mathbb{E}\left[∥g_{k_{\min }+1}∥\right]\le O\left(\epsilon \right),\mathbb{E}\left[{\lambda }_{\min }\left(\mathbb{H}{f}_{k_{\min }+1}\right)\right]\ge -O\left(\sqrt{\epsilon }\right)$$

(73)

where ${\lambda }_{\min }(·)$ denotes the minimum eigenvalue. The parameters must satisfy:

$$N=O\left({\displaystyle \frac{1}{{\epsilon }^{3/2}}}\right),\mu =O\left(min\left\{{\displaystyle \frac{\epsilon }{d^{3/2}}},\sqrt{{\displaystyle \frac{\epsilon }{d^3}}}\right\}\right),m=O\left({\displaystyle \frac{d}{{\epsilon }^2}}\right),b=O\left({\displaystyle \frac{d^4}{\epsilon }}\right)$$

(74)

Thus, the complexity of the zeroth-order oracle is:

$$O(Nm+Nb)=O\left({\displaystyle \frac{d}{{\epsilon }^{7/2}}}+{\displaystyle \frac{d^4}{{\epsilon }^{5/2}}}\right)$$

Proof. 

From Lemma 6, we have:

$$\mathbb{E}\left[∥g_{k+1}∥\right]\le {\displaystyle \frac{1}{2}}(L_H+1)\mathbb{E}\left[{∥{\eta }_k∥}^2\right]+\sqrt{{\delta }_g}+{\delta }_H+{\delta }_{\mu }$$

(75)

where ${\delta }_g$, ${\delta }_H$, and ${\delta }_{\mu }$ correspond to the gradient and Hessian estimation errors. On the other hand, from Lemma 4.8 (relationship between step size and minimum eigenvalue),

$$\mathbb{E}\left[∥{\eta }_k∥\right]\ge -{\displaystyle \frac{\sqrt{{\delta }_H}+{\alpha }_k+\mathbb{E}\left[{\lambda }_{\min }\left(\mathbb{H}{f}_{k+1}\right)\right]}{L_H}}$$

(76)

Next, we analyze the upper bound of $\mathbb{E}\left[∥{\eta }_k∥\right]$.

Step 1: Third-Order Taylor Upper Bound. By the equivalent form of Assumption 2 (Equation (18)),

$$f\left(R_{x_k}\left({\eta }_k\right)\right)\le f\left(x_k\right)+〈g_k,{\eta }_k〉+{\displaystyle \frac{1}{2}}〈{\eta }_k,\mathbb{H}f\left(x_k\right){\eta }_k〉+{\displaystyle \frac{L_H}{6}}{∥{\eta }_k∥}^3$$

(77)

Step 2: Decompose Gradient and Hessian into Estimation + Error. Write the true gradient and Hessian as:

$$g_k={\overline{g}}_{\mu ,\xi }+(g_k-{\overline{g}}_{\mu ,\xi }),\mathbb{H}f\left(x_k\right)={\overline{H}}_{\mu ,\xi }+(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi })$$

Substitute into Equation (77) to decompose the function value into the main term, first-order error, second-order error, and third-order term:

$$f_{k+1}\le f_k+\underset{\mathrm{Main}\mathrm{Term}}{\underbrace{〈{\overline{g}}_{\mu ,\xi },{\eta }_k〉+{\displaystyle \frac{1}{2}}〈{\eta }_k,{\overline{H}}_{\mu ,\xi }{\eta }_k〉}}+\underset{\mathrm{First}\text{-}\mathrm{Order}\mathrm{Error}}{\underbrace{〈g_k-{\overline{g}}_{\mu ,\xi },{\eta }_k〉}}+\underset{\mathrm{Second}\text{-}\mathrm{Order}\mathrm{Error}}{\underbrace{{\displaystyle \frac{1}{2}}〈{\eta }_k,(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }){\eta }_k〉}}+{\displaystyle \frac{L_H}{6}}{∥{\eta }_k∥}^3$$

(78)

Step 3: First-Order Optimality Condition of the Subproblem. From the first-order optimality condition of Subproblem (21),

$$0\in {\overline{g}}_{\mu ,\xi }\left(x_k\right)+({\overline{H}}_{\mu ,\xi }\left(x_k\right)+\alpha I){\eta }_k+{\mathrm{Proj}}_{T_{X_k}\mathcal{M}}\partial h(x_k+{\eta }_k)$$

(79)

Take the inner product with ${\eta }_k$ and rearrange. Using the positive definiteness of ${\overline{H}}_{\mu ,\xi }+2\alpha I$ and the subgradient inequality for convex h ($〈\partial h(x_k+{\eta }_k),{\eta }_k〉\ge h(x_k+{\eta }_k)-h\left(x_k\right)$), we get:

$$\begin{array}{cc}\hfill 〈{\overline{g}}_{\mu ,\xi },{\eta }_k〉+{\displaystyle \frac{1}{2}}〈{\eta }_k,{\overline{H}}_{\mu ,\xi }{\eta }_k〉& =-{\displaystyle \frac{1}{2}}〈{\eta }_k,({\overline{H}}_{\mu ,\xi }+2\alpha I){\eta }_k〉-〈\partial h(x_k+{\eta }_k),{\eta }_k〉\hfill \\ \hfill & \le h\left(x_k\right)-h(x_k+{\eta }_k)\hfill \end{array}$$

(80)

Substitute this into Equation (78) and combine terms for $F=f+h$:

$$F_{k+1}\le F_k+\underset{\mathrm{First}\text{-}\mathrm{Order}\mathrm{Error}}{\underbrace{〈g_k-{\overline{g}}_{\mu ,\xi },{\eta }_k〉}}+\underset{\mathrm{Second}\text{-}\mathrm{Order}\mathrm{Error}}{\underbrace{{\displaystyle \frac{1}{2}}〈{\eta }_k,(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu ,\xi }){\eta }_k〉}}+{\displaystyle \frac{L_H}{6}}{∥{\eta }_k∥}^3$$

(81)

Step 4: Bound First-Order and Second-Order Errors. Using the Cauchy–Schwarz inequality and Young’s inequality,

$$\begin{array}{cc}\hfill \mathbb{E}\left[〈g_k-{\overline{g}}_{\mu }\left(x_k\right),{\eta }_k〉+{\displaystyle \frac{1}{2}}〈{\eta }_k,(\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu }\left(x_k\right)){\eta }_k〉\right]& \le \mathbb{E}\left[∥g_k-{\overline{g}}_{\mu }\left(x_k\right)∥∥{\eta }_k∥+{\displaystyle \frac{1}{2}}∥[∥\mathrm{op}]\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu }\left(x_k\right){∥{\eta }_k∥}^2\right]\hfill \\ \hfill & \le {\displaystyle \frac{32}{3\beta }}\mathbb{E}\left[{∥g_k-{\overline{g}}_{\mu }\left(x_k\right)∥}^{3/2}\right]+{\displaystyle \frac{12}{\beta }}\mathbb{E}\left[∥[∥\mathrm{op}]{\mathbb{H}f\left(x_k\right)-{\overline{H}}_{\mu }\left(x_k\right)}^3\right]\hfill \\ & +{\displaystyle \frac{\beta }{24}}\mathbb{E}\left[{∥{\eta }_k∥}^3\right]\hfill \end{array}$$

(82)

Substitute into Equation (82), take $\beta >L_H$, and use the lower bounds of gradient and Hessian errors. Summing over $k=0$ to $N-1$ and rearranging gives:

$${\displaystyle \frac{1}{N}}\sum _{k=0}^N\mathbb{E}\left[{∥{\eta }_k∥}^3\right]\le {\displaystyle \frac{24}{L_H}}\left({\displaystyle \frac{F_0-F^{*}}{N}}+{\displaystyle \frac{32}{3L_H}}{\delta }_g^{3/4}+{\displaystyle \frac{12}{L_H}}{\tilde{\delta }}_H\right)$$

(83)

where ${\tilde{\delta }}_H=\tilde{C}{\displaystyle \frac{{(d+16)}^6}{b^{3/2}}}{L}_g^{1.5}+{\displaystyle \frac{1}{27}}{\mu }^3{L}_H^3{(d+6)}^{7.5}$. By combining with the parameter choices in Equation (74), we obtain:

$$\mathbb{E}\left[{∥{\eta }_{k_{\min }}∥}^3\right]\le O\left({\epsilon }^{3/2}\right),\mathbb{E}\left[{∥{\eta }_{k_{\min }}∥}^2\right]\le O\left(\epsilon \right)$$

(84)

Step 5: Final Bounds. Substitute Equation (84) into Lemmas 6 and 7. Simplifying using the parameter choices gives:

$$\mathbb{E}\left[∥g_{k_{\min }+1}∥\right]\le O\left(\epsilon \right),\mathbb{E}\left[{\lambda }_{\min }\left(\mathbb{H}{f}_{k_{\min }+1}\right)\right]\ge -O\left(\sqrt{\epsilon }\right)$$

The complexity of the zeroth-order oracle is calculated by substituting N, m, and b from Equation (74) into $O(Nm+Nb)$, leading to:

$$O\left({\displaystyle \frac{d}{{\epsilon }^{7/2}}}+{\displaystyle \frac{d^4}{{\epsilon }^{5/2}}}\right)$$

This completes the proof. □

6. Application of ZO-ARPQN

6.1. Sparse Principal Component Analysis (SPCA) on the Stiefel Manifold

A traditional principal component analysis (PCA) extracts low-dimensional representations of data by maximizing the projection variance, but its loading vectors are usually dense, making it difficult to interpret the contribution of individual variables. A sparse principal component analysis (SPCA) introduces a sparsity penalty while maximizing the variance, thereby balancing variable selection and interpretability. Typical applications include gene expression analyses, spectral component separation, financial factor construction, and feature compression in high-dimensional regression. Compared with a PCA, the additional benefits of an SPCA are reflected in: a stronger interpretability (fewer non-zero variables can describe components), a better robustness (reducing the impact of noisy variables), and convenience for downstream modeling (sparse factors are more conducive to regression/classification tasks). This paper considers the SPCA problem on the Stiefel manifold $St(n,p)=\{X\in {\mathbb{R}}^{n\times p}\mid X^{\top }X=I_p\}$, formulated as:

$$\underset{X\in St(n,p)}{min}F\left(X\right):=f\left(X\right)+{\lambda \parallel X\parallel }_1=-{\displaystyle \frac{1}{2}}Tr\left(X^{\top }HX\right)+\lambda {\parallel X\parallel }_1,$$

(85)

where $H\in {\mathbb{R}}^{n\times n}$ is a symmetric positive semi-definite matrix, and ${\parallel ·\parallel }_1$ denotes the element-wise ${\mathsf{\ell }}_1$-norm to induce sparsity.

6.1.1. Experimental Parameter Settings

The following parameters were adopted in this experiment (consistent with the reproducibility requirements of SJTU doctoral theses):

• Data Generation: Set $n=100$ (number of variables), $p=10$ (number of principal components), and $K=5$ (rank of the true subspace). First, perform QR orthogonalization on a Gaussian matrix $A\in {\mathbb{R}}^{n\times K}$, construct $\Lambda =diag\left(\right|\mathcal{N}(0,1)\left|\right)$ (diagonal matrix of absolute values of standard normal samples), and then form $H=A\Lambda A^{\top }$ and normalize it by the spectral norm to ensure numerical stability. The sparsity coefficient is set to $\lambda =0.1$ (determined via 5-fold cross-validation to balance sparsity and variance retention).
• ZO-ManPG Algorithm: Refer to Algorithm 3 in Reference [24]. Set the finite difference radius to $\mu =0.05$, the gradient sampling counts to $m=\{5,20,50\}$ (to analyze the impact of the sampling density on zero-order estimation), and the maximum iterations to $N=150$ (sufficient to ensure the convergence of all the compared algorithms).
• ZO-ARPQN Algorithm: Adopt the same $\mu =0.05$ and $m=\{5,20,50\}$ as ZO-ManPG; set the Hessian sampling count to $b=15$ (compromising between the estimation accuracy and computational cost); the initial regularization parameter to ${\sigma }_0=5.0$; the line search shrinkage factor to $\gamma =0.5$; the ratio test thresholds to ${\eta }_1=0.25$ (lower bound for an acceptable model fidelity) and ${\eta }_2=0.75$ (threshold for reducing regularization); the adaptive regularization coefficients to ${\gamma }_1=0.8$ (reduction factor) and ${\gamma }_2=2.0$ (increase factor); and the maximum iterations to $N=150$.
• ARPQN Algorithm [16]: As the oracle baseline (with exact gradient/Hessian), use the same ${\sigma }_0$, $\gamma$, ${\eta }_1$, ${\eta }_2$, ${\gamma }_1$, ${\gamma }_2$, and N as ZO-ARPQN, except that the exact Riemannian gradient/Hessian (instead of zero-order estimation) is used to verify the performance upper bound of quasi-Newton methods.

6.1.2. Result Analysis

The vertical axis of the convergence curves plots the optimality gap $F\left(X_k\right)-F^{*}$ (in logarithmic scale) to clearly show the convergence trend. Since Problem (85) is non-convex, the analytical optimal value is difficult to obtain; thus, this paper takes the minimum function value among all the compared algorithms as the empirical optimal value:

$$F^{*}=\underset{a\in \{\mathrm{ZO}\text{-}\mathrm{ManPG},\mathrm{ZO}\text{-}\mathrm{ARPQN},\mathrm{ARPQN}\}}{min}{F}_a\left(X\right),$$

where $F_a\left(X\right)$ denotes the final function value of algorithm a. The sparsity metric is defined as:

$$\mathrm{sparsity}\left(X\right)={\displaystyle \frac{\#\{(i,j):|X_{ij}|<{10}^{-3}\}}{n·p}}\times 100\%,$$

which measures the proportion of zero elements in the solution matrix. A higher sparsity indicates that more variables are “compressed” to zero, enhancing the feature-selection ability of the model; however, excessive sparsity may lead to loss of principal component information.

To analyze the impact of the gradient sampling count m on the stability and accuracy of zero-order algorithms, this experiment set $m=5, 15, 30$ respectively for the SPCA problem on the Stiefel manifold, and compared three algorithms: the zeroth-order manifold proximal gradient (ZO-ManPG), the zeroth-order adaptive regularized proximal quasi-Newton (ZO-ARPQN), and ARPQN (with exact gradient/Hessian). All algorithms wererun for 150 iterations under the same initial point and parameters, and the convergence curves of the optimality gap $F\left(X_k\right)-F^{*}$ are plotted. As shown in the figures, all three algorithms exhibited a monotonically decreasing convergence trend, but their performance differed significantly under different sampling counts.

6.1.3. Case 1: Low Sampling Count ($M=5$) (Figure 1)

When the sampling count was small, the number of random perturbation directions was insufficient, leading to large variance in zero-order estimation of gradients and Hessians, resulting in obvious fluctuations in the convergence curves. Even so, ZO-ARPQN still outperformed ZO-ManPG, which indicates that the quasi-Newton direction and adaptive regularization can effectively suppress noise. ARPQN converged rapidly within approximately 20 iterations, serving as a performance upper bound. Due to the large noise in zero-order Hessian estimation of ZO-ARPQN, the ${\mathsf{\ell }}_1$-regularization term dominated, leading to excessive sparsity (60.0%); in contrast, ZO-ManPG barely produced a significant sparse structure (5.4%), indicating that its gradient approximation is sensitive to noise.

Figure 1. Convergence curves of three algorithms (gradient sampling count $m=5$).

Figure 1. Convergence curves of three algorithms (gradient sampling count $m=5$).

6.1.4. Case 2: Moderate Sampling Count ($M=15$) (Figure 2)

With an increase in the sampling count, zero-order estimation became more stable. The convergence speed of ZO-ARPQN significantly accelerated and almost overlapped with ARPQN after approximately 40 iterations; the oscillation of ZO-ManPG weakened, but its convergence speed remained slow. As the sampling increased, the sparsity of ZO-ARPQN decreased significantly (2.4%), indicating that gradient estimation became more accurate and the impact of sparse constraints weakened; the sparsity of ZO-ManPG increased slightly (20%), showing a more stable convergence trend.

Figure 2. Convergence curves of three algorithms (gradient sampling count $m=15$).

Figure 2. Convergence curves of three algorithms (gradient sampling count $m=15$).

6.1.5. Case 3: High Sampling Count ($M=30$) (Figure 3)

When the sampling count further increased, the convergence curve of ZO-ARPQN almost completely overlapped with that of ARPQN, indicating that its zero-order Hessian approximation is extremely accurate under high sampling. Although ZO-ManPG was still the slowest, its final accuracy improved significantly. The sparsity of ZO-ARPQN was close to that of ARPQN (18.2% vs. 24.2% (

Table 1. Sparsity comparison of three algorithms under different gradient sampling counts m.

Sampling Count m	Algorithm	Final Objective Value ${\mathit{F}}_{\mathbf{final}}$	Sparsity (%)
5	ZO-ManPG	1.6205	5.4
5	ZO-ARPQN	2.4447	60.0
5	ARPQN	3.3191	24.2
15	ZO-ManPG	1.3465	20.0
15	ZO-ARPQN	3.8697	2.4
15	ARPQN	3.3191	24.2
30	ZO-ManPG	1.2499	53.4
30	ZO-ARPQN	3.4940	18.2
30	ARPQN	3.3191	24.2

), indicating that high sampling can effectively recover the true sparse structure. The sparsity of ZO-ManPG increased to 53.4%, but its overall convergence remained slow.

Figure 3. Convergence curves of three algorithms (gradient sampling count $m=30$).

Figure 3. Convergence curves of three algorithms (gradient sampling count $m=30$).

In conclusion, increasing the sampling count m can significantly improve the convergence quality and stability of zero-order algorithms. Among them, ZO-ARPQN benefited the most from high sampling, as its quasi-Newton mechanism can fully utilize the accurate zero-order Hessian information to approach the performance of the oracle algorithm (ARPQN); ZO-ManPG, as a first-order method, is limited by the inherent low accuracy of zero-order gradient estimation, and even high sampling cannot compensate for the performance gap with second-order methods.

6.2. Black-Box Optimization in Robotic Stiffness Control (Figure 4)

The specific connection between zeroth-order Riemannian optimization and its application in robotic stiffness control [16] is elaborated as follows.

Problem Background: Adaptive force/stiffness control aims to address a classic challenge in robotics: how to enable robots to adjust the force they apply adaptively when performing delicate manipulation tasks (e.g., inserting a key into a lock hole, scraping along a complex surface). This is closely associated with stiffness control, as the robot’s impedance controller achieves precise force control by adjusting its own stiffness and damping. Therefore, learning an adaptive force controller essentially involves learning how to dynamically adjust stiffness parameters based on real-time conditions.

Motivation for Black-Box/Zeroth-Order Optimization: The performance of a controller (e.g., whether key insertion is smooth, whether jamming occurs) cannot be described by simple mathematical formulas. Its performance score can only be obtained through physical experiments or high-fidelity simulations. The process of converting controller parameters to task performance scores is a typical black-box system, and its gradient is completely unknown. Thus, zeroth-order optimization methods become indispensable. The Bayesian optimization adopted in this paper is precisely a zeroth-order method designed for expensive black-box problems.

Figure 4. Schematic diagram of robotic stiffness control.

Figure 4. Schematic diagram of robotic stiffness control.

Riemannian Manifold Constraints: In robotic force/stiffness control, key parameters often lie on specific manifolds:

• Stiffness Matrix (K): Physically valid stiffness matrices must be symmetric positive definite (SPD) matrices. The set of all SPD matrices forms a Riemannian manifold. During the optimization process, it is critical to ensure that the updated parameters remain SPD matrices after each iteration.
• Desired Pose (R): The controller may need to adjust the desired tool pose (e.g., the orientation of a key), which is represented by a rotation matrix belonging to the $SO\left(3\right)$ manifold.

In Cartesian space impedance/stiffness control, the compliance of the end-effector is characterized by a symmetric positive definite (SPD) matrix $K^P\in S_{++}^d$. Given the current end-effector position $\widehat{p}\in {\mathbb{R}}^d$, end-effector velocity $\dot{p}$, and external force $f^e\in {\mathbb{R}}^d$, the control law drives the system to reach a new equilibrium position $p\left(K^P\right)$, whose common approximation is:

$$p\left(K^P\right)=\widehat{p}-\dot{p}-{\left(K^P\right)}^{-1}{f}^e.$$

Intuitively, the stiffness matrix $K^P$ determines the “hardness/softness” of the robot end-effector in response to external disturbances. Given the current end-effector position $\widehat{p}\in {\mathbb{R}}^d$, desired equilibrium position P, external force $f^e$, and damping matrix $K^D=K^P$ (under critical damping), the equilibrium equation can be expressed as:

$$f^e=K^P(\widehat{p}-p)-K^D\dot{p},$$

(86)

from which the position p is solved.

To obtain a reasonable stiffness matrix $K^P$, the following objective function is defined [25]:

$$f\left(K^P\right)=w_p{∥\widehat{p}-p\left(K^P\right)∥}^2+w_{d}det\left(K^P\right)+w_c\mathrm{cond}\left(K^P\right),$$

(87)

where $w_p,w_d,w_c>0$ are weights, $p\left(K^P\right)$ denotes the equilibrium position, and $\mathrm{cond}(·)$ represents the condition number.

This objective function consists of three components:

• First term: squared position error ${∥\widehat{p}-p\left(K^P\right)∥}^2$, which ensures the end-effector returns as close as possible to the target position;
• Second term: penalty on $det\left(K^P\right)$, which prevents an excessively large stiffness (avoiding damage to the robot or target object);
• Third term: penalty on the condition number, which avoids ill-conditioned stiffness matrices (ensuring stable force control).

This is a black-box optimization problem defined on the SPD manifold $S_{++}^d$: the function f has no explicit gradient information, and only function value queries are available. This characteristic makes traditional first-order methods (e.g., Riemannian gradient descent) difficult to apply directly, necessitating the design of Riemannian optimization methods based on zeroth-order function values.

6.2.1. Manifold and Tangent Space

The SPD manifold is mathematically defined as:

$$\mathcal{M}={\mathcal{S}}_{++}^d=\left\{K\in {\mathbb{R}}^{d\times d}\mid K=K^{\top },K\succ 0\right\},$$

where $K\succ 0$ indicates that K is a positive definite matrix. The tangent space at any point $K\in \mathcal{M}$ (denoted as $T_K\mathcal{M}$) consists of all symmetric matrices, i.e.:

$$T_K\mathcal{M}=\left\{U\in {\mathbb{R}}^{d\times d}\mid U=U^{\top }\right\}.$$

6.2.2. Riemannian Metric

To ensure geometric consistency in stiffness control, the affine-invariant metric—a standard metric for SPD manifolds—is adopted:

$${\langle U,V\rangle }_K=\mathrm{tr}\left(K^{-1}U{K}^{-1}V\right),\forall U,V\in T_K\mathcal{M},$$

where $\mathrm{tr}(·)$ denotes the matrix trace. The norm induced by this metric is:

$${\parallel U\parallel }_K^2={\langle U,U\rangle }_K,$$

which characterizes the “magnitude” of tangent vectors in the context of SPD matrix updates.

6.2.3. Riemannian Gradient and Exponential Map

For a smooth objective function $f:\mathcal{M}\to \mathbb{R}$, let ${\nabla }_{E}f\left(K\right)$ denote its Euclidean gradient (computed as if K were an unconstrained matrix in ${\mathbb{R}}^{d\times d}$). The Riemannian gradient—projected onto the tangent space $T_K\mathcal{M}$—is:

$$\mathrm{grad}f\left(K\right)=K·\mathrm{sym}\left({\nabla }_{E}f\left(K\right)\right)·K,$$

where $\mathrm{sym}\left(A\right)={\displaystyle \frac{1}{2}}(A+A^{\top })$ denotes the symmetric part of matrix A.

The exponential map (used to map tangent vectors back to the SPD manifold) is:

$${\mathrm{Exp}}_K\left(U\right)=K^{1/2}exp\left(K^{-1/2}U{K}^{-1/2}\right)K^{1/2},\forall U\in T_K\mathcal{M},$$

where $K^{1/2}$ is the matrix square root of K, and $exp(·)$ denotes the matrix exponential. In numerical implementation, this exponential map is directly used as the retraction (a key operation in Riemannian optimization to maintain manifold constraints).

6.2.4. Experimental Setup

Since explicit calculation of gradients and Hessian matrices is impossible in black-box stiffness control, the standard ARPQN (proximal quasi-Newton method with exact gradients) cannot be used as a comparative baseline.

Basic Parameters

• Initial stiffness matrix: $K_0=I_d$ (identity matrix, ensuring initial feasibility on the SPD manifold);
• Task space dimension: $d=3$ (consistent with 3D Cartesian space control of robotic end-effectors);
• Maximum iterations: 100 (balancing the computational efficiency and convergence accuracy).

Algorithm-Specific Parameters

Two zeroth-order Riemannian optimization algorithms were compared, with parameters tuned to achieve an optimal performance:

• ZO-ManPG (Zeroth-Order Manifold Proximal Gradient): step size: $\alpha =0.1$; finite difference radius (for zero-order gradient estimation): $\mu ={10}^{-3}$; and mini-batch size (for noise reduction): $m=10$.
• ZO-ARPQN (Zeroth-Order Adaptive Regularized Proximal Quasi-Newton): smoothing parameter (for zero-order Hessian estimation): $\mu ={10}^{-4}$; mini-batch size (for gradient/Hessian estimation): $m=10$; initial regularization parameter: ${\sigma }_0=5.0$; ratio test thresholds (for adaptive regularization): ${\eta }_1=0.1$, ${\eta }_2=0.8$; and step size decay factor: $\gamma =0.6$.

Both algorithms use the exponential map ${\mathrm{Exp}}_K\left(U\right)$ for retraction. After each update, matrix eigenvalues are clipped to the range $[{10}^{-3},{10}^3]$ to ensure the stiffness matrix K remains symmetric positive definite (avoiding physical invalidity of the stiffness parameters).

6.2.5. Experimental Results and Analysis

Figure 5 shows the objective function descent process of ZO-ManPG and ZO-ARPQN on the SPD manifold for the 3D stiffness adjustment task. The vertical axis represents the objective function value $F\left(K_k\right)$ (logarithmic scale to highlight convergence trends), and the horizontal axis represents the number of iterations.

$$F_{\mathrm{final}}^{\mathrm{ManPG}}=0.2245, F_{\mathrm{final}}^{\mathrm{ARPQN}}=0.1622$$

Convergence Trend Analysis

(1) Early-stage convergence: Both algorithms reduced the objective function by approximately two orders of magnitude within the first 10 iterations, verifying that zero-order approximation is feasible for black-box stiffness tuning. (2) Late-stage convergence: The ZO-ARPQN curve remained consistently below ZO-ManPG, indicating a higher descent efficiency and more stable convergence. This was attributed to ZO-ARPQN’s adaptive regularization mechanism (dynamically adjusting ${\sigma }_k$ based on model fidelity) and quasi-Newton direction (using Hessian approximation to guide updates).

Quantitative Performance

After 100 iterations, the final objective function values were:

$$F_{\mathrm{final}}^{\mathrm{ManPG}}=0.2245, F_{\mathrm{final}}^{\mathrm{ARPQN}}=0.1622.$$

Compared to ZO-ManPG, ZO-ARPQN achieved a 27.7% reduction in the objective function value under the same iteration budget, confirming its advantage in balancing zero-order stability and the convergence accuracy.

Practical Significance

The experimental results validate that ZO-ARPQN is effective for non-differentiable black-box SPD optimization tasks (e.g., robotic stiffness control). Its ability to handle “no gradient + manifold constraints” makes it suitable for engineering scenarios where only experimental/simulation scores are available, providing a feasible solution for adaptive stiffness tuning in robotic manipulation.

7. Conclusions

We propose ZO-ARPQN, the first zeroth-order adaptive regularized proximal quasi-Newton framework for composite optimization on Riemannian manifolds, including the Stiefel and SPD manifolds. The method extends the classical ARPQN algorithm to black-box settings by constructing stochastic one-point finite-difference estimators for both the Riemannian gradient and curvature, enabling second-order optimization without explicit derivatives.

We established global convergence guarantees under mild smoothness assumptions, proved that the algorithm achieves convergence to first-order stationary points even with noisy zeroth-order information, and further demonstrated that the curvature-aware regularization enables escape from strict saddle points with a high probability. A complete iteration-complexity analysis is provided, showing that ZO-ARPQN maintains competitive theoretical efficiency compared to existing zeroth-order Riemannian methods.

Extensive numerical experiments on sparse PCA and robot stiffness tuning validated the practical effectiveness of the proposed method. ZO-ARPQN achieved a convergence behavior comparable to first-order ARPQN and other state-of-the-art Riemannian solvers, while requiring only function evaluations. These results highlight its potential for applications in black-box manifold optimization, robotics, machine learning, and simulation-based design.

Future work will include exploring variance-reduced zeroth-order strategies, developing limited-memory versions for large-scale manifolds, and extending the framework to Riemannian stochastic compositional optimization and constrained reinforcement learning.