Approximation Conjugate Gradient Method for Low-Rank Matrix Recovery

Abstract

Large-scale symmetric and asymmetric matrices have emerged in predicting the relationship between genes and diseases. The emergence of large-scale matrices increases the computational complexity of the problem. Therefore, using low-rank matrices instead of original symmetric and asymmetric matrices can greatly reduce computational complexity. In this paper, we propose an approximation conjugate gradient method for solving the low-rank matrix recovery problem, i.e., the low-rank matrix is obtained to replace the original symmetric and asymmetric matrices such that the approximation error is the smallest. The conjugate gradient search direction is given through matrix addition and matrix multiplication. The new conjugate gradient update parameter is given by the F-norm of matrix and the trace inner product of matrices. The conjugate gradient generated by the algorithm avoids SVD decomposition. The backtracking linear search is used so that the approximation conjugate gradient direction is computed only once, which ensures that the objective function decreases monotonically. The global convergence and local superlinear convergence of the algorithm are given. The numerical results are reported and show the effectiveness of the algorithm.

1. Introduction

Problem Description Motivation

The low-rank matrix plays an important role in a broad range of applications and multiple scientific fields. For example, many bioinformatics experts use the matrix restoration technique to predict the relationship between diseases and genes [1]. In many bioinformatics problems, the information between genes and diseases is presented in the form of a symmetric or asymmetric matrix, but the dimensions of symmetric and asymmetric matrices are extremely large, which makes it difficult to calculate effectively in practical situations and even leads to the inability to perform calculations. Hence, it is often possible to recover it due to the low-rank structure. In many cases, the rank r of the low-rank matrix is fixed and the matrix decomposition $M=U{V}^T$ is completed, where $M\in {\mathbb{R}}^{m\times n}$ is a known matrix and $U\in {\mathbb{R}}^{m\times r},V\in {\mathbb{R}}^{n\times r}$ is the variable matrices of the following problem:

$$\begin{array}{c}\hfill \underset{U\in {\mathbb{R}}^{m\times r},V\in {\mathbb{R}}^{n\times r}}{min}F(U,V)=\frac{1}{2}{\parallel U{V}^T-M\parallel }_F^2.\end{array}$$

(1)

In many cases, matrix M is a symmetric matrix, i.e., $M\in {\mathbb{R}}^{n\times n}$ and the matrix decomposition $M=U{U}^T$, $U\in {\mathbb{R}}^{n\times r}$, problem (1) is equivalent to solving the following low-rank symmetric matrix recovery problem:

$$\begin{array}{c}\hfill \underset{U\in {\mathbb{R}}^{n\times r}}{min}F(U,U)=\frac{1}{2}{\parallel U{U}^T-M\parallel }_F^2.\end{array}$$

(2)

As the factorized problem (1) or (2) involves only $(m+n)r$ or $2nr$ variables, these methods are, in general, more scalable and can cope with larger matrices. The solution to the matrix recovery problem has also attracted the interest of many scholars.

On the Euclidean manifold, Sun and Luo [2] established a theoretical guarantee for the factorization-based formulation to correctly recover the underlying low-rank matrix. Tu et al. [3] proposed a Procrustes Flow algorithm for solving the problem of recovering a low-rank matrix from linear measurements.

On other Riemannian manifolds, Keshavan et al. [4] introduced an efficient algorithm such that the method could reconstruct matrix $M\in {\mathbb{R}}^{m\alpha \times n}$ from $O\left(rn\right)$. Ngo and Saad [5] described gradient methods based on a scaled metric for low-rank matrix completion. Vandereycken [6] proposed a new algorithm for matrix completion. Mishra and Sepulchre [7] introduced a nonlinear conjugate gradient method for solving low-rank matrix completion. Mishra et al. [8] proposed a gradient descent and trust-region algorithm for low-rank matrix completion. Boumal and Absil [9] exploited the geometry of the low-rank constraint to recast the problem as an unconstrained optimization problem on a single Grassmann manifold. Wei et al. [10] introduced a family of Riemannian optimization algorithms for low-rank matrix recovery problems. In the past year, Najafi and Hajarian [11] proposed an improved Riemannian conjugate gradient method to solve robust matrix completion problems. Duan et al. [12] proposed the Riemannian conjugate gradient method for solving the low-rank tensor completion problem.

In addition to the aforementioned work, there are also some algorithms for solving matrix completion problems (see [13,14,15,16,17,18]). For solving matrix recovery problems, some algorithms and related research have also been widely studied (see [19,20,21,22,23,24]).

Inspired by the algorithm ideas in [25,26], this paper proposes an approximation conjugate gradient method to solve the low-rank matrix recovery problem. The approximation conjugate gradient direction is given such that the search direction is the descent direction of the objective $F(U,V)$. The algorithm avoids the singular value decomposition of the matrix and only obtains the search direction of the problem through the multiplication of the matrix. The backtracking line search technique ensures that the objective function of problem (1) is monotonic descent.

In this paper, the approximation conjugate gradient direction is introduced in Section 2. In Section 3, the detailed steps of the approximation conjugate gradient algorithm are proposed. The global convergence of the algorithm is given in Section 4. In Section 5, the local superlinear convergence of the algorithm is given. Numerical results are presented in Section 6, and in Section 7, some conclusions are given.

Notation

In this paper, the low-rank matrix recovery problem is solved by the approximation Newton algorithm. In problem (1), let $Z\in {\mathbb{R}}^{m\times n}$ denote the variables, and $Z^{*}$ be the optimal solution of objective function $f\left(Z\right)$. The F-norm of matrix Z is defined as ${\parallel Z\parallel }_F^2={\displaystyle \sum _{i=1,j=1}^{m,n}}{Z}_{ij}^2$. A symmetric and positive semidefinite matrix $B-A$ is represented as $A⪯B$. Given any matrix $A,B\in {\mathbb{R}}^{m\times n}$, $A•B$ denotes the usual trace inner product of A and B.

2. The Approximation Conjugate Gradient Step

In this paper, we propose a new approximation conjugate gradient algorithm for solving problem (1). Next, we first introduce the traditional conjugate gradient algorithm for solving the unconstrained nonlinear programming problem.

In the Euclidean case, the conjugate gradient methods are line search algorithms to solve the unconstrained nonlinear programming problem, where the objective function is $f:{\mathbb{R}}^n\to \mathbb{R}$. The sequence $\left\{x_k\right\}$ in ${\mathbb{R}}^n$ is generated by

$$\begin{array}{c}\hfill x_{k+1}=x_k+{\alpha }_k{\widehat{\eta }}_k,\end{array}$$

with ${\alpha }_k>0$ for all $k\ge 0$ from an initial point $x_0\in {\mathbb{R}}^n$, and the search directions ${\widehat{\eta }}_k\in {\mathbb{R}}^n$ are computed using the gradient ${\widehat{g}}_k=\nabla f\left(x_k\right)$ as ${\widehat{\eta }}_0=-{\widehat{g}}_0$ and

$$\begin{array}{c}\hfill {\widehat{\eta }}_{k+1}=-{\widehat{g}}_{k+1}+{\beta }_{k+1}{\widehat{\eta }}_k=-\nabla f\left(x_{k+1}\right)+{\beta }_{k+1}{\widehat{\eta }}_k\end{array}$$

for all $k\ge 0$. The computation of real values ${\beta }_{k+1}$ is crucial for the performance of conjugate gradient methods. A famous real value of ${\beta }_{k+1}$ is

$$\begin{array}{c}\hfill \mathrm{Conjugate}\mathrm{descent}:{\beta }_{k+1}^{CD}=\frac{\parallel {\widehat{g}}_{k+1}{\parallel }_2^2}{-{\widehat{g}}_k^T{\widehat{\eta }}_k}.\end{array}$$

It was proposed by Fletcher [27].

To solve problem (1), we let

$$\begin{array}{cc}\hfill \nabla F_U\left(Z\right)=& \hfill \nabla F_V(U,V)=(U{V}^T-M)V,\hfill \\ \hfill \nabla F_V\left(Z\right)=& \hfill \nabla F_U(U,V)={(U{V}^T-M)}^{T}U.\hfill \end{array}$$

Hence,

$$\nabla F\left(Z\right)=\nabla F(U,V)=\left[\begin{array}{c}\nabla F_U(U,V)\\ \nabla F_V(U,V)\end{array}\right]=\left[\begin{array}{c}(U{V}^T-M)V\\ {(U{V}^T-M)}^{T}U\end{array}\right],$$

and

$$g_k=\nabla F\left(Z_k\right)=\nabla F(U_k,V_k)=\left[\begin{array}{c}\nabla F_U(U_k,V_k)\\ \nabla F_V(U_k,V_k)\end{array}\right]=\left[\begin{array}{c}(U_k{V}_k^T-M)V_k\\ {(U_k{V}_k^T-M)}^T{U}_k\end{array}\right],$$

on iteration point $Z_k=\left[\begin{array}{c}{U}_k\\ V_k\end{array}\right]$.

The sequence $\left\{Z_k\right\}$ is generated by

$$\begin{array}{c}\hfill Z_{k+1}=Z_k+{\alpha }_k{\eta }_k,\end{array}$$

from an initial point $Z_0\in {\mathbb{R}}^{(m+n)\times r}$ with ${\alpha }_k>0$ for all $k\ge 0$. The search direction ${\eta }_k\in {\mathbb{R}}^{(m+n)\times r}$ are computed using the gradient $g_k=\nabla F\left(Z_k\right)$ as ${\eta }_0=-g_0=-\nabla F\left(Z_0\right)$ and

$$\begin{array}{cc}\hfill {\eta }_{k+1}=& -g_{k+1}+{\beta }_{k+1}{\eta }_k\hfill \\ \hfill =& \hfill -\nabla F\left(Z_{k+1}\right)+{\beta }_{k+1}{\eta }_k,\hfill \end{array}$$

(3)

for all $k\ge 0$. To obtain better search directions, we constructed the following real value ${\beta }_{k+1}$ of Algorithm 1 based on the structure of the problem itself,

$${\beta }_{k+1}=\left\{\begin{array}{cc}\frac{\parallel \nabla F\left(Z_{k+1}\right){\parallel }_F^2}{-\nabla F\left(Z_{k+1}\right)•{\eta }_k},& \mathrm{if}|\nabla F\left(Z_{k+1}\right)•{\eta }_k|>\frac{{ϵ}^2}{2},\hfill \\ \frac{\parallel \nabla F\left(Z_{k+1}\right){\parallel }_F^2}{2{ϵ}^2},& \mathrm{otherwise},\hfill \end{array}\right.$$

(4)

where $\nabla F\left(Z_{k+1}\right)•{\eta }_k$ denotes the usual trace inner product of $\nabla F\left(Z_{k+1}\right),{\eta }_k\in {\mathbb{R}}^{(m+n)\times r}$ and $ϵ\in (0,1)$.

Algorithm 1 (ACGA)

Input: Choose an initial $Z_0=\left[\begin{array}{c}{U}_0\\ V_0\end{array}\right]$, where $U_0\in {\mathbb{R}}^{m\times r}$, $V_0\in {\mathbb{R}}^{n\times r}$. Let $0<\beta <1$, $ϵ>0$ and set $k=0$. Main step: 1. If $k=0$, then compute $\nabla F\left(Z_k\right)$ and let ${\eta }_k=-\nabla F\left(Z_k\right)$. 2. If $k=0$ and $\parallel \nabla F\left(Z_k\right){\parallel }_F\le ϵ$, then stop, $Z_k$ is the optimal solution of problem (1). 3. Compute ${\alpha }_k\in (0,1]$ such that $$\begin{array}{c}\hfill F(Z_k+{\alpha }_k{\eta }_k)\le F\left(Z_k\right)+{\alpha }_k\beta \nabla F\left(Z_k\right)•{\eta }_k.\end{array}$$ (5) 4. Let $Z_{k+1}=Z_k+{\alpha }_k{\eta }_k$. 5. Calculate $\nabla F\left(Z_{k+1}\right)$, if $\parallel \nabla F\left(Z_{k+1}\right){\parallel }_F\le ϵ$, then stop, $Z_{k+1}$ is the optimal solution of problem (1). 6. Compute ${\beta }_{k+1}$ as (2) and $$\begin{array}{c}\hfill {\eta }_{k+1}=-\nabla F\left(Z_{k+1}\right)+{\beta }_{k+1}{\eta }_k,\end{array}$$ let $k=k+1$, go to 3.

3. The Approximation Conjugate Gradient Algorithm (ACGA)

In Section 2, we introduce the approximation conjugate gradient step with updated real value ${\beta }_k$.

Remark 1. 

We obtain the step size ${\alpha }_k$ by the backtracking line search based on objective function $F\left(Z\right)$, i.e., given $\omega \in (0,1)$, let ${\alpha }_k=1,\omega ,{\omega }^2,\cdots$ until (5) holds.

Remark 2. 

We can randomly generate initial $m\times r$ matrix $U_0$ and initial $n\times r$ matrix $V_0$ with rank r. We can generate the matrices $U_0\in {\mathbb{R}}^{m\times r}$ and $V_0\in {\mathbb{R}}^{n\times r}$ as follow,

$$\begin{array}{c}\hfill U_0={\left[\begin{array}{cccccc}1& 0& \cdots & 0& \cdots & 0\\ 0& 1& \cdots & 0& \cdots & 0\\ \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\ 0& 0& \cdots & 1& \cdots & 0\end{array}\right]}_{m\times r}\mathrm{and}{V}_0={\left[\begin{array}{cccccc}1& 0& \cdots & 0& \cdots & 0\\ 0& 1& \cdots & 0& \cdots & 0\\ \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\ 0& 0& \cdots & 1& \cdots & 0\end{array}\right]}_{n\times r}.\end{array}$$

4. Convergence Analysis

In this section, to prove the convergence of the algorithm (ACGA), we require the model to satisfy the following assumption:

Assumption 1. 

Assume that the objective functions of problem (1) are twice continuous differentiable, i.e., for any $F\left(Z\right)$, it is twice continuous differentiable. For any point $Z\in {\mathbb{R}}^{(m+n)\times r}$ in our method, we define the level set

$$\begin{array}{c}\hfill L\left(Z_0\right)=\{Z\in {\mathbb{R}}^{(m+n)\times r},F\left(Z\right)\le F\left(Z_0\right)\}.\end{array}$$

Assume the level set $L\left(Z_0\right)$ is bounded.

Assumption 2. 

Assume the gradient of $F\left(Z\right)$ is Lipschitz-continuous, i.e., there exists ${\kappa }_H>0$ such that

$$\begin{array}{cc}\hfill {\parallel \nabla F\left(X\right)-\nabla F\left(Y\right)\parallel }_F\le & {\kappa }_H{\parallel X-Y\parallel }_F,\hfill \end{array}$$

for any $X,Y\in {\mathbb{R}}^{m\times n}$.

The bounded function $F\left(Z\right)$ is important for the proof of convergence of Algorithm 1 and the following lemma presents the boundedness of the objective function $F\left(Z\right)$.

Lemma 1. 

Under Assumption 1, there exists constant ${\kappa }_f>0$ and ${\kappa }_g>0$ such that

$$\begin{array}{cc}\hfill \parallel F\left(Z_k\right){\parallel }_F\le \hfill & \hfill {\kappa }_f,\hfill \\ \hfill \parallel \nabla F\left(Z_k\right){\parallel }_F\le \hfill & \hfill {\kappa }_g.\hfill \end{array}$$

Proof. 

The proof is similar to Lemma 3.2 in [28]. □

The purpose of step 3 of Algorithm 1 is to monotonically decrease the objective function of problem (1). Hence, we need to prove that $\nabla F\left(Z_k\right)•{\eta }_k<0$ when Algorithm 1 does not terminate.

Lemma 2. 

If Algorithm 1 dose not terminate, i.e., for constant $ϵ>0$, we have $\parallel \nabla F\left(Z_k\right){\parallel }_F>ϵ$, then

$$\begin{array}{c}\hfill \nabla F\left(Z_k\right)•{\eta }_k<0.\end{array}$$

Proof. 

If Algorithm 1 dose not terminate, i.e., $\parallel \nabla F\left(Z_k\right)\parallel >ϵ$, then according to the definition of ${\eta }_k$, we have that

$$\begin{array}{cc}\hfill \nabla F\left(Z_k\right)•{\eta }_k=& \nabla F\left(Z_k\right)•(-\nabla F\left(Z_k\right)+{\beta }_k{\eta }_{k-1})\hfill \\ \hfill =& -\nabla F\left(Z_k\right)•\nabla F\left(Z_k\right)+{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill =& -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}.\hfill \end{array}$$

(6)

If $|\nabla F\left(Z_k\right)•{\eta }_{k-1}|>\frac{{ϵ}^2}{2}$, by (4) and (6), we have that

$$\begin{array}{cc}\hfill \nabla F\left(Z_k\right)•{\eta }_k=& -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill =& -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{-\nabla F\left(Z_k\right)•{\eta }_{k-1}}\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill =& -2\parallel \nabla F\left(Z_k\right){\parallel }_F^2\hfill \\ \hfill \le & -2{ϵ}^2<0.\hfill \end{array}$$

If $|\nabla F\left(Z_k\right)•{\eta }_{k-1}|\le \frac{{ϵ}^2}{2}$, then we have that

$$\begin{array}{cc}\hfill \nabla F\left(Z_k\right)•{\eta }_k=& -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill \le & -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+{\beta }_k|\nabla F\left(Z_k\right)•{\eta }_{k-1}|\hfill \\ \hfill \le & -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+{\beta }_k\frac{{ϵ}^2}{2}\hfill \\ \hfill \le & -\parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{2}\hfill \\ \hfill =& -\frac{{ϵ}^2}{2}<0.\hfill \end{array}$$

□

Step 3 is important for Algorithm 1 because it ensures that the objective function is monotonically decreasing. But we must prove that step 3 is to terminate in a finite step, i.e., there exists ${\alpha }_k>0$ such that (5) holds.

Lemma 3. 

Under Assumptions 1 and 2, let $\left\{Z_k\right\}$ be the sequence generated by Algorithm 1, then step 3 is to terminate in a finite step, i.e., there exists ${\alpha }_k>0$ such that (5) holds.

Proof. 

Suppose Algorithm 1 dose not terminate on iteration k, i.e., there exists positive constant $ϵ>0$ such that $\parallel \nabla F\left(Z_k\right){\parallel }_F\ge ϵ$. First, we prove that $\parallel {\eta }_k{\parallel }_F$ is bounded for all $k\ge 0$. We use mathematical induction to prove this conclusion. When $k=0$, we have ${\eta }_0=-\nabla F\left(Z_0\right)$, according to Lemma 1, we have ${\parallel {\eta }_0\parallel }_F={\parallel \nabla F\left(Z_0\right)\parallel }_F\le {\kappa }_g$, suppose ${\eta }_k$ is bounded, then by (3), we have that

$$\begin{array}{cc}\hfill {\parallel {\eta }_{k+1}\parallel }_F=& \parallel -\nabla F\left(Z_{k+1}\right)+{\beta }_{k+1}{\eta }_k{\parallel }_F\hfill \\ \hfill \le & \parallel \nabla F\left(Z_{k+1}\right){\parallel }_F+|{\beta }_{k+1}|\parallel {\eta }_k{\parallel }_F.\hfill \end{array}$$

(7)

By (4), if $|\nabla F\left(Z_{k+1}\right)•{\eta }_k|>\frac{{ϵ}^2}{2}$, then

$$\begin{array}{c}\hfill {\beta }_{k+1}=\frac{\parallel \nabla F\left(Z_{k+1}\right){\parallel }_F^2}{-\nabla F\left(Z_{k+1}\right)•{\eta }_k}.\end{array}$$

By the bounded of $\nabla F\left(Z_{k+1}\right)$ and ${\eta }_k$, we know that

$$\begin{array}{cc}\hfill |{\beta }_{k+1}|=& \hfill \frac{\parallel \nabla F\left(Z_{k+1}\right){\parallel }_F^2}{|\nabla F\left(Z_{k+1}\right)•{\eta }_k|}\le \frac{2{\kappa }_g^2}{{ϵ}^2}.\hfill \end{array}$$

If $|\nabla F\left(Z_{k+1}\right)•{\eta }_k|\le \frac{{ϵ}^2}{2}$, then by (4), we know that

$$\begin{array}{c}\hfill |{\beta }_{k+1}|=\frac{\parallel \nabla F\left(Z_{k+1}\right){\parallel }_F^2}{2ϵ}\le \frac{{\kappa }_g^2}{2{ϵ}^2}.\end{array}$$

Hence, let ${\kappa }_{\beta }=\frac{{\kappa }_g^2}{2{ϵ}^2}$, we have that $|{\beta }_{k+1}|\le {\kappa }_{\beta }$. Hence, by (7), we have that

$$\begin{array}{c}\hfill {\parallel {\eta }_{k+1}\parallel }_F\le {\kappa }_g+{\kappa }_{\beta }{\parallel {\eta }_k\parallel }_F.\end{array}$$

According to the assumption of the bounded of ${\eta }_k$, we obtain that there exists a positive constant ${\kappa }_{\eta }$ such that

$$\begin{array}{c}\hfill \parallel {\eta }_{k+1}\parallel \le {\kappa }_{\eta }.\end{array}$$

According to Assumption 2, the gradient of $F\left(Z\right)$ is Lipschitz-continuous, then

$$\begin{array}{c}\hfill F\left(Z_{k+1}\right)\le F\left(Z_k\right)+{\alpha }_k\nabla F\left(Z_k\right)•{\eta }_k+\frac{{\kappa }_H{\alpha }_k^2}{2}{\parallel {\eta }_k\parallel }_F^2.\end{array}$$

(8)

If $|\nabla F\left(Z_k\right)•{\eta }_{k-1}|>\frac{{ϵ}^2}{2}$, then by (5) and (8), we have that

$$\begin{array}{cc}\hfill & F\left(Z_{k+1}\right)-F\left(Z_k\right)-{\alpha }_k\beta \nabla F\left(Z_k\right)•{\eta }_k\hfill \\ \hfill \le & {\alpha }_k\nabla F\left(Z_k\right)•{\eta }_k+\frac{{\alpha }_k^2}{2}{\kappa }_H{\parallel {\eta }_k\parallel }_F^2-{\alpha }_k\beta \nabla F\left(Z_k\right)•{\eta }_k\hfill \\ \hfill =& (1-\beta ){\alpha }_k\nabla F\left(Z_k\right)•{\eta }_k+\frac{{\alpha }_k^2{\kappa }_H}{2}{\parallel {\eta }_k\parallel }_F^2\hfill \\ \hfill \le & (1-\beta ){\alpha }_k\nabla F\left(Z_k\right)•(-\nabla F\left(Z_k\right)+{\beta }_k{\eta }_{k-1})+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\hfill \\ \hfill \le & (1-\beta ){\alpha }_k\left(-\parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{-\nabla F\left(Z_k\right)•{\eta }_{k-1}}\nabla F\left(Z_k\right)•{\eta }_{k-1}\right)+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\hfill \\ \hfill \le & -2(1-\beta ){\alpha }_k{ϵ}^2+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2.\hfill \end{array}$$

(9)

By (9), we want $F\left(Z_{k+1}\right)-F\left(Z_k\right)-{\alpha }_k\beta \nabla F\left(Z_k\right)•{\eta }_k\le 0$, then

$$\begin{array}{c}\hfill -2(1-\beta ){\alpha }_k{ϵ}^2+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\le 0.\end{array}$$

(10)

By (10), we have that

$$\begin{array}{c}\hfill {\alpha }_k\le \frac{4(1-\beta ){ϵ}^2}{{\kappa }_H^2{\kappa }_{\eta }^2}.\end{array}$$

Hence, we let ${\alpha }_k=\frac{2(1-\beta ){ϵ}^2}{{\kappa }_H^2{\kappa }_{\eta }^2}>0$, then the conclusion holds.

If $|\nabla F\left(Z_k\right)•{\eta }_{k-1}|\le \frac{{ϵ}^2}{2}$, similar to the proof of (9), we have that

$$\begin{array}{cc}\hfill & F\left(Z_{k+1}\right)-F\left(Z_k\right)-{\alpha }_k\beta \nabla F\left(Z_k\right)•{\eta }_k\hfill \\ \hfill \le & (1-\beta ){\alpha }_k\left(-\parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{2{ϵ}^2}\nabla F\left(Z_k\right)•{\eta }_{k-1}\right)+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\hfill \\ \hfill \le & (1-\beta ){\alpha }_k\left(-\parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{2}\right)+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\hfill \\ \hfill \le & -\frac{(1-\beta ){\alpha }_k{ϵ}^2}{2}+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2.\hfill \end{array}$$

Hence, to ensure $F\left(Z_{k+1}\right)-F\left(Z_k\right)-{\alpha }_k\beta \nabla F\left(Z_k\right)•{\eta }_k\le 0$, we want $-\frac{(1-\beta ){\alpha }_k{ϵ}^2}{2}+\frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\le 0$, then

$$\begin{array}{c}\hfill \frac{{\alpha }_k^2{\kappa }_H^2}{2}{\kappa }_{\eta }^2\le \frac{(1-\beta ){\alpha }_k{ϵ}^2}{2},\end{array}$$

which means that

$$\begin{array}{c}\hfill {\alpha }_k\le \frac{(1-\beta ){ϵ}^2}{4{\kappa }_H^2{\kappa }_{\eta }^2}.\end{array}$$

Let ${\alpha }_k=\frac{(1-\beta ){ϵ}^2}{8{\kappa }_H^2{\kappa }_{\eta }^2}>0$, then the conclusion holds. □

The following theorem shows that the sequence $\left\{Z_k\right\}$ generated by Algorithm 1 converges to the critical point of problem (1).

Theorem 1. 

Under Assumptions 1 and 2, the sequence $\left\{Z_k\right\}$ generated by Algorithm 1 satisfy

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}{\parallel \nabla f\left(Z_k\right)\parallel }_F=0.\end{array}$$

It means that the iteration sequence $\left\{Z_k\right\}$ converges to the critical point of problem (1).

Proof. 

By (6), we have that

$$\begin{array}{c}\hfill \nabla F\left(Z_k\right)•{\eta }_k=-{\parallel \nabla F\left(Z_k\right)\parallel }_F^2+{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}.\end{array}$$

By (4), if $|\nabla F\left(Z_k\right)•{\eta }_{k-1}|>\frac{{ϵ}^2}{2}$, we have that

$$\begin{array}{cc}-\nabla F\left(Z_k\right)•{\eta }_k=\hfill & \parallel \nabla F\left(Z_k\right){\parallel }_F^2-{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill =& \parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{\nabla F\left(Z_k\right)•{\eta }_{k-1}}\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill =& 2\parallel \nabla F\left(Z_k\right){\parallel }_F^2.\hfill \end{array}$$

(11)

If $|\nabla F\left(Z_k\right)•{\eta }_{k-1}|\le \frac{{ϵ}^2}{2}$, then

$$\begin{array}{cc}\hfill -\nabla F\left(Z_k\right)•{\eta }_k=& \parallel \nabla F\left(Z_k\right){\parallel }_F^2-{\beta }_k\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill =& \parallel \nabla F\left(Z_k\right){\parallel }_F^2-\frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{2{ϵ}^2}\nabla F\left(Z_k\right)•{\eta }_{k-1}\hfill \\ \hfill \ge & \frac{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}{2}.\hfill \end{array}$$

(12)

Combining (5), (11) and (12), we can see that

$$\begin{array}{cc}\hfill F\left(Z_k\right)-F\left(Z_{k+1}\right)\ge & -{\alpha }_k\beta \nabla f\left(Z_k\right)•{\eta }_k\hfill \\ \hfill \ge & \frac{{\alpha }_k\beta }{2}{\parallel \nabla f\left(Z_k\right)\parallel }_F^2,\hfill \end{array}$$

hence, for any large enough $N>0$, we have that

$$\begin{array}{cc}\hfill F\left(Z_0\right)-F\left(Z_N\right)=& {\displaystyle \sum _{k=0}^N}F\left(Z_k\right)-F\left(Z_{k+1}\right)\hfill \\ \hfill \ge & {\displaystyle \sum _{k=0}^N}\frac{1}{2}{\alpha }_k\beta {\parallel \nabla F\left(Z_k\right)\parallel }_F^2\hfill \\ \hfill =& \frac{1}{2}\beta {\displaystyle \sum _{k=0}^N}{\alpha }_k{\parallel \nabla F\left(Z_k\right)\parallel }_F^2.\hfill \end{array}$$

(13)

According to Assumption 1, we know that $F\left(Z\right)$ is blow bounded, hence, by (13), we have that

$$\begin{array}{c}\hfill \underset{N\to \infty }{lim}{\alpha }_N{\parallel \nabla F\left(Z_N\right)\parallel }_F=0.\end{array}$$

(14)

According to Lemma 3, we know that ${\alpha }_N\ge \frac{(1-\beta ){ϵ}^2}{8{\kappa }_H^2{\kappa }_{\eta }^2}>0$ for any index N if $\parallel \nabla F\left(Z_N\right){\parallel }_F>ϵ$, and

$$\begin{array}{c}\hfill {\alpha }_N\parallel \nabla F\left(Z_N\right){\parallel }_F\ge \frac{(1-\beta ){ϵ}^2}{8{\kappa }_H^2{\kappa }_{\eta }^2}{\parallel \nabla F\left(Z_N\right)\parallel }_F.\end{array}$$

(15)

Combining (14) and (15), we have that

$$\begin{array}{c}\hfill \underset{k\to \infty }{lim}{\parallel \nabla F\left(Z_k\right)\parallel }_F=0.\end{array}$$

□

5. Local Convergence

In Section 4, we have shown that the sequence $\{\parallel \nabla F\left(Z_k\right){\parallel }_F\}$ generated by the conjugate gradient algorithm converges globally to zero. We now discuss the superlinear convergence in a neighborhood of the solution $Z^{*}$.

First, we show that if $\frac{\parallel {\eta }_k{\parallel }_F^2}{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}\le \frac{1-\beta }{{\kappa }_H}$, then ${\alpha }_k=1$ for any iteration index k.

Lemma 4. 

Under Assumptions 1 and 2, let $\left\{Z_k\right\}$ be the sequence generated by Algorithm 1, if $\frac{\parallel {\eta }_k{\parallel }_F^2}{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}\le \frac{(1-\beta )}{{\kappa }_H}$, then ${\alpha }_k=1$.

Proof. 

First, we consider the follow inequality

$$\begin{array}{cc}\hfill & F\left(Z_{k+1}\right)-F\left(Z_k\right)-\beta \nabla F\left(Z_k\right)•{\eta }_k\hfill \\ \hfill \le & (1-\beta )\nabla f\left(Z_k\right)•{\eta }_k+\frac{{\kappa }_H}{2}{\parallel {\eta }_k\parallel }_F^2\hfill \\ \hfill \le & -\frac{1-\beta }{2}\parallel \nabla F\left(Z_k\right){\parallel }_F^2+\frac{{\kappa }_H}{2}{\parallel {\eta }_k\parallel }_F^2.\hfill \end{array}$$

We want $f\left(Z_{k+1}\right)-f\left(Z_k\right)-\beta \nabla f\left(Z_k\right)•{\eta }_k\le 0$, then

$$\begin{array}{c}\hfill \frac{{\kappa }_H}{2}\parallel {\eta }_k{\parallel }_F^2\le \frac{1-\beta }{2}{\parallel \nabla F\left(Z_k\right)\parallel }_F^2.\end{array}$$

(16)

By (16) and $\frac{\parallel {\eta }_k{\parallel }_F^2}{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}\le \frac{1-\beta }{{\kappa }_H}$, we have that $f\left(Z_{k+1}\right)-f\left(Z_k\right)-\beta \nabla f\left(Z_k\right)•{\eta }_k\le 0$ holds, which means that ${\alpha }_k=1$. □

Finally, we show that the sequence $\left\{Z_k\right\}$ generated by Algorithm 1 converges to critical point $Z^{*}$ superlinearly.

Theorem 2. 

Under Assumptions 1 and 2, if $\frac{\parallel {\eta }_k{\parallel }_F^2}{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}\le \frac{1-\beta }{{\kappa }_H}$ and the objective function $F\left(Z\right)$ satisfies the local error bound condition, i.e., there exists ${\kappa }_1>0$ such that ${\parallel F\left(X\right)-F\left(Y\right)\parallel }_F\ge {\kappa }_1{\parallel X-Y\parallel }_F$ for all $X,Y\in R^{m\times n}$, then the sequence $\left\{Z_k\right\}$ generated by Algorithm 1 converges to critical point $Z^{*}$ superlinearly.

Proof. 

Because $\frac{\parallel {\eta }_k{\parallel }_F^2}{\parallel \nabla F\left(Z_k\right){\parallel }_F^2}\le \frac{1-\beta }{{\kappa }_H}$, according to Lemma 4, we have ${\alpha }_k=1$. Then,

$$\begin{array}{cc}\hfill {\kappa }_1{\parallel Z_{k+1}-Z^{*}\parallel }_F\le & \parallel F\left(Z_{k+1}\right)-F\left(Z^{*}\right){\parallel }_F\hfill \\ \hfill =& \parallel \nabla F\left(Z_{k+1}\right)•(Z_{k+1}-Z^{*})+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F{)\parallel }_F\hfill \\ \hfill =& \parallel (\nabla F\left(Z_{k+1}\right)-\nabla F\left(Z_k\right))•(Z_{k+1}-Z^{*})\hfill \\ \hfill & +\nabla F\left(Z_k\right)•(Z_{k+1}-Z^{*})+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F{)\parallel }_F\hfill \\ \hfill \le & \parallel \nabla F\left(Z_{k+1}\right)-\nabla F\left(Z_k\right){\parallel }_F{\parallel Z_{k+1}-Z^{*}\parallel }_F\hfill \\ \hfill & +\parallel \nabla F\left(Z_k\right)•(Z_k+{\eta }_k-Z^{*}){\parallel }_F+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F)\hfill \\ \hfill =& \parallel \nabla F\left(Z_{k+1}\right)-\nabla F\left(Z^{*}\right)+\nabla F\left(Z^{*}\right)-\nabla F\left(Z_k\right){\parallel }_F{\parallel Z_{k+1}-Z^{*}\parallel }_F\hfill \\ \hfill & +\parallel \nabla F\left(Z_k\right)•(Z_k+{\eta }_k-Z^{*}){\parallel }_F+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F)\hfill \\ \hfill \le & \parallel \nabla F\left(Z_{k+1}\right)-\nabla F\left(Z^{*}\right){\parallel }_F\parallel Z_{k+1}-Z^{*}{\parallel }_F+\parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F{\parallel Z_{k+1}-Z^{*}\parallel }_F\hfill \\ \hfill & +\parallel \nabla F\left(Z_k\right)•(Z_k+{\eta }_k-Z^{*}){\parallel }_F+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F)\hfill \\ \hfill \le & \parallel \nabla F\left(Z_{k+1}\right)-\nabla F\left(Z^{*}\right){\parallel }_F\parallel Z_{k+1}-Z^{*}{\parallel }_F+\parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F{\parallel Z_{k+1}-Z^{*}\parallel }_F\hfill \\ \hfill & +\parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F\parallel Z_k-Z^{*}{\parallel }_F+\parallel \nabla F\left(Z_k\right)•{\eta }_k{\parallel }_F+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F).\hfill \end{array}$$

(17)

According to Assumption 2,

$$\begin{array}{cc}\hfill \parallel \nabla F\left(Z_{k+1}\right)-\nabla F\left(Z^{*}\right){\parallel }_F{\parallel Z_{k+1}-Z^{*}\parallel }_F\le & {\kappa }_H{\parallel Z_{k+1}-Z^{*}\parallel }_F^2\hfill \\ \hfill \parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F{\parallel Z_{k+1}-Z^{*}\parallel }_F\le & {\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F{\parallel Z_k-Z^{*}\parallel }_F\hfill \\ \hfill \parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F{\parallel Z_k-Z^{*}\parallel }_F\le & {\kappa }_H{\parallel Z_k-Z^{*}\parallel }_F^2.\hfill \end{array}$$

(18)

By (11) and (12), we have that

$$\begin{array}{cc}\hfill \parallel \nabla F\left(Z_k\right)•{\eta }_k{\parallel }_F\le & 3\parallel \nabla F\left(Z_k\right){\parallel }_F^2\hfill \\ \hfill =& 3\parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F^2\hfill \\ \hfill \le & 3{\kappa }_H^2{\parallel Z_k-Z^{*}\parallel }^2.\hfill \end{array}$$

(19)

Combining (17), (18) and (19), we have that

$$\begin{array}{cc}\hfill {\kappa }_1{\parallel Z_{k+1}-Z^{*}\parallel }_F\le & {\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F^2+{\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F{\parallel Z_k-Z^{*}\parallel }_F\hfill \\ \hfill & +{\kappa }_H\parallel Z_k-Z^{*}{\parallel }_F^2+\frac{1}{2}\parallel \nabla F\left(Z_k\right)-\nabla F\left(Z^{*}\right){\parallel }_F^2+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F)\hfill \\ \hfill \le & {\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F^2+{\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F{\parallel Z_k-Z^{*}\parallel }_F\hfill \\ \hfill & +3{\kappa }_H^2\parallel Z_k-Z^{*}{\parallel }_F^2+o(\parallel Z_{k+1}-Z^{*}{\parallel }_F).\hfill \end{array}$$

(20)

Since $Z_k\to Z^{*}$, we know that

$$\begin{array}{c}\hfill max\left\{o\right(\parallel Z_{k+1}-Z^{*}{\parallel }_F),{\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F^2\}\le \frac{{\kappa }_1}{4}{\parallel Z_{k+1}-Z^{*}\parallel }_F,\end{array}$$

for large enough k.

Hence, by (20), we obtain that

$$\begin{array}{c}\hfill {\kappa }_1\parallel Z_{k+1}-Z^{*}{\parallel }_F\le {\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F\parallel Z_k-Z^{*}{\parallel }_F+3{\kappa }_H^2\parallel Z_k-Z^{*}{\parallel }_F^2+\frac{{\kappa }_1}{2}{\parallel Z_{k+1}-Z^{*}\parallel }_F,\end{array}$$

which means that

$$\begin{array}{c}\hfill \frac{{\kappa }_1}{2}\parallel Z_{k+1}-Z^{*}{\parallel }_F\le {\kappa }_H\parallel Z_{k+1}-Z^{*}{\parallel }_F\parallel Z_k-Z^{*}{\parallel }_F+3{\kappa }_H^2{\parallel Z_k-Z^{*}\parallel }_F^2.\end{array}$$

(21)

Divide by $\parallel Z_k-Z^{*}{\parallel }_F$ on both sides of the inequality (21), we have that

$$\begin{array}{c}\hfill \frac{\parallel Z_{k+1}-Z^{*}{\parallel }_F}{\parallel Z_k-Z^{*}{\parallel }_F}\le \frac{2{\kappa }_H}{{\kappa }_1}\parallel Z_{k+1}-Z^{*}{\parallel }_F+\frac{6{\kappa }_H^2}{{\kappa }_1}{\parallel Z_k-Z^{*}\parallel }_F.\end{array}$$

As $Z_{k+1}\to Z^{*}$ and $Z_k\to Z^{*}$ when $k\to +\infty$, we can obtain that

$$\begin{array}{c}\hfill \frac{\parallel Z_{k+1}-Z^{*}{\parallel }_F}{\parallel Z_k-Z^{*}{\parallel }_F}\to 0,\end{array}$$

(22)

as $k\to +\infty$, i.e.,

$$\begin{array}{c}\hfill \parallel Z_{k+1}-Z^{*}{\parallel }_F=o(\parallel Z_k-Z^{*}{\parallel }_F),\end{array}$$

which means that the sequence $\left\{Z_k\right\}$ generated by Algorithm 1 converges to critical point $Z^{*}$ superlinearly. □

6. Numerical Results

In Section 3, the approximation conjugate gradient method is given, and the global and local superlinear convergence of this algorithm are given in Section 4 and Section 5, respectively. Next, we give the numerical experiments to illustrate the performance of the method proposed in this paper. We apply ACGA to synthetic problems. The synthetic data of the experiments are increased as they are in [29]. First, the matrices $U\in {\mathbb{R}}^{m\times r}$ and $V\in {\mathbb{R}}^{n\times r}$ with independent and identically distributed Gaussian entries such that $M=U{V}^T$ is filled with zero-mean and unite-variance nonindependent Gaussian entries are given. Then, the sample $K=\rho r(m+n-r)$ is given at random, where $\rho$ is the oversampling factor. To test the effectiveness of the algorithm, we compare ACGA to RMC [29], AOPMC [30], GRASTA [31], and $l_2$ method RTRMC [9].

RMC was the method for solving the low-rank matrix completion, in which the fraction of the entries was corrupted by non-Gaussian noise and typically outliers. This method smoothed the $l\infty$ norm, and the low-rank constraint was dealt with by Riemannian optimization.

AOPMC was a robust method for recovering the low-rank matrix using the adaptive outlier pursuit technique and Riemannian trust-region method when part of the measurements was damaged by outliers.

GRASTA was a Grassmannian robust adaptive subspace tracking algorithm. This method needed the rank-r SVD decomposition for the matrix and could improve computation speed while still allowing for accurate subspace learning.

RTRMC exploited the geometry of the low-rank constraint to recast the problem as an unconstrained optimization problem on a single Grassmann manifold. The second-order Riemannian trust-region methods and Riemannian conjugate gradient methods are used to solve the unconstrained optimization problem.

Before solving the low-rank recovery problem, we will introduce the parameters selected in the actual calculation of the algorithm proposed in this paper. The important parameter of this paper is $\beta \in (0,1)$. To obtain a larger search step size and the objective function of problem $\left(P\right)$ descent more quickly, the backtracking line search parameter $\beta$ in step 3 of the algorithm selects a smaller value close to 0, so $\beta =0.15$ is selected in the algorithm proposed in this paper.

To solve the low-rank recovery problem using the algorithm in this paper, we use MATLAB (2014a) to write a computer program, and the computer is a ThinkPad T480 (CPU is i7-8550U, main frequency is 1.99 Hz, memory is 8 G). The termination accuracy of the algorithm is chosen by $ϵ={10}^{-2}$, $ϵ={10}^{-3}$ and $ϵ={10}^{-5}$, respectively.

In this paper, we selected 12 problems to test the effectiveness of the algorithm. The number of rows and columns of the selected questions, as well as the rank of matrices U and V, are shown in the following

Table 1. The information of test problems.

Problem	m	n	r	Problem	m	n	r
1	100	100	20	7	100	300	20
2	300	300	20	8	100	500	20
3	500	500	20	9	100	800	20
4	800	800	20	10	100	1000	20
5	1000	1000	20	11	300	1000	20
6	50	100	20	12	500	1000	20

. In Table 1, m and n denote the number of rows and columns of matrix M, respectively. r denotes the rank of matrices U and V. To test our algorithm, we randomly generated 50 different scenarios for each problem in Table 1 and obtained the final number of iterations by a weighted average of all random problem results. We have set three different termination accuracies for our algorithm ($ϵ={10}^{-2},{10}^{-3},{10}^{-5}$) and used our algorithm to calculate the test problems in Table 1 for each accuracy. We also used the four classic methods mentioned above to solve every test problem in Table 1 and recorded the number of iterations of each method to solve every test problem in Table 1. The comparison results of our algorithm with the other four algorithms under different termination accuracies are shown in Figure 1, Figure 2 and Figure 3.

To draw a comparison diagram of the results of five algorithms, we use the performance comparison formula of the algorithm proposed by Dolan and More [32] to calculate the computational efficiency between different algorithms. Here are the specific formulas:

$$\begin{array}{c}\hfill r_{p,s}=\frac{{\tau }_{p,s}}{min\{{\tau }_{p,u}:u\in S\}},\end{array}$$

(23)

where S denotes the set of algorithms. Let P denote problems set, $n_s=\left|S\right|$ and $n_p=\left|P\right|$. For $\forall t\ge 1$, let

$$\begin{array}{c}\hfill {\rho }_s\left(t\right)=\frac{1}{n_p}size\{p\in P:r_{p,s}\le t\},\end{array}$$

(24)

where ${\rho }_s\left(t\right)$ represent the efficiency of each solver.

We solve the test problems in Table 1 using ACGA, RMC, AOPMC, GRASTA and $l_2$ method RTRMC, respectively. First, the terminate parameter is chosen by $ϵ={10}^{-2}$. The test results are reported in the following Figure 1.

From Figure 1, we can see that our algorithm has significant effectiveness compared to the other four algorithms for calculating low-rank matrices. Especially when $t=1$, our algorithm has reached 0.83 and even reached 0.9 in comparison with RTRMC, but other algorithms only reach between 0.1 and 0.2. This indicates that at an accuracy level of $ϵ={10}^{-2}$, our algorithm can solve 83–90% of test problems faster than the other four algorithms. By (23) and (24), the red curve representing our algorithm tends to 1 faster than the blue curve representing other algorithms, indicating that our algorithm is more effective in solving the test problems in Table 1 using all five algorithms. From the abscissa of the four subgraphs mentioned above, it can be seen that our algorithm spends much fewer iterations solving most testing problems than the other four algorithms. In summary, our algorithm is more effective in solving the test problems in Table 1.

Next, we use five algorithms to solve the test problems when the terminate parameter is chosen by $ϵ={10}^{-3}$. The test results are reported in Figure 2.

From Figure 2, we can see that when the termination parameter is set to $ϵ={10}^{-3}$, it indicates that our algorithm’s red curve tends to 1 much faster than the other four algorithms. When $t=1$, the red curve has already reached 0.8, or even 0.9, while other algorithms only reach around 0.2, with a worst-case value of 0.1. This indicates that even with improved termination accuracy, our algorithm still has high computational efficiency, i.e., our algorithm spends fewer total iterations when solving more than $80\%$ of problems. According to the abscissa of each subgraph, it can be seen that our algorithm spends much fewer iterations in solving many testing problems in Table 1 than the other four algorithms, indicating that our algorithm has higher efficiency in solving testing problems.

From Figure 3, we clearly see that with the continuous improvement of termination accuracy, the five algorithms listed in this article have shown a decrease in efficiency when solving test problems compared to those with accuracies of $ϵ={10}^{-2}$ and $ϵ={10}^{-3}$. In other words, the number of iterations required to solve all test problems has greatly increased. However, when the termination accuracy is $ϵ={10}^{-5}$, our algorithm still has higher efficiency compared to the other four algorithms. The main manifestation is that the red curve representing our algorithm has reached a range of 0.8 to 0.9 when $t=1$, but other algorithms only reach around 0.4 or 0.5. This indicates that our algorithm spends less iteration time solving most test problems than the other four algorithms. In addition, the speed at which the red curve tends to 1 is also faster than the blue curve representing other algorithms, and it can be clearly seen from the abscissa that although the overall solving efficiency has decreased, our algorithm still spends far fewer iterations on solving some problems than the other four algorithms.

Finally, we changed the termination parameter to $ϵ={10}^{-5}$ and still used five different algorithms to solve the test problem. The results are reported in Figure 3.

We choose the rank $r=15,r=25,r=30$ and $r=35$ for solving the test problems in Table 1 using Algorithm 3, RMC, AOPMC, GRASTA and RTRMC, respectively. The number of iterations of every method for solving every problem of Table 1 is reported. Using (23) and (24), we obtain a comparison graph of the number of iterations when we solve every problem in Table 1 using four algorithms. The comparison results are reported in Figure 4.

By (23) and (24), we can see that the red curve tends to 1 and is also faster than other curves representing other algorithms. This means that our algorithm costs fewer iterations than the other four algorithms when we solve all test problems in the table, with $r=15,r=25,r=30$, and $r=35$, respectively.

Next, we will test our algorithm on the Netflix dataset, which is a real-world dataset in the Netflix Prize [33]. Netflix is a movie rental company that recommends movies to its users. Hence, they have much information from users. This information is translated into a $480,189\times 17,770$ matrix (this matrix is denoted as M). To test our algorithm, we randomly selected 10 matrices of $500\times 1000$ in M, and the rank of U and V are chosen by $r=20$. We will choose $ϵ={10}^{-2},ϵ={10}^{-3},ϵ={10}^{-4}$ and $ϵ={10}^{-5}$ to test five algorithms. The number of iterations for each algorithm at different accuracies and ranks is recorded in Figure 5.

From Figure 5, we can see that the speed at which the red curve tends towards 1 is the fastest at different accuracies. This means that for different termination accuracies, the number of iterations of our algorithm is less than the other four algorithms in most cases when solving all real-test problems, indicating that our algorithm is effective in solving practical problems.

We determine the complexity of the algorithm by recording the $CPU$ time when the algorithm solves the problems in Table 1. In other words, the more $CPU$ time is spent, the higher its complexity; conversely, the lower its computational complexity. Next, we calculate the test problems in Table 1 using five different algorithms and record the $CPU$ time by each algorithm in Figure 6.

From Figure 6, we can see that the speed at which the red curve tends towards 1 is the fastest, indicating that our algorithm spends less $CPU$ time than other algorithms when solving most of the test problems in Table 1. In particular, from Figure 6, we can see that the yellow curve representing the SVD decomposition algorithm is much lower than the red curve representing our algorithm, which means that our algorithm has a lower computational complexity than the SVD algorithm.

7. Concluding Remarks

The main purpose of this paper is to introduce the approximation conjugate gradient method, which avoids the complex SVD decomposition for solving the low-rank matrix recovery. The idea is to extend the approximation conjugate gradient method and the conjugate gradient parameter update technique to the matrix calculation and obtain the search direction. The backtracking linear search technique ensures the global convergence of the algorithm. At the same time, this technique ensures that the objective function decreases monotonously. The local superlinear convergence of the algorithm is also given, and the numerical results are reported to show the effectiveness of the algorithm.