Fast, Efficient, and Viable Compressed Sensing, Low-Rank, and Robust Principle Component Analysis Algorithms for Radar Signal Processing

Abstract

Modern radar signal processing techniques make strong use of compressed sensing, affine rank minimization, and robust principle component analysis. The corresponding reconstruction algorithms should fulfill the following desired properties: complex valued, viable in the sense of not requiring parameters that are unknown in practice, fast convergence, low computational complexity, and high reconstruction performance. Although a plethora of reconstruction algorithms are available in the literature, these generally do not meet all of the aforementioned desired properties together. In this paper, a set of algorithms fulfilling these conditions is presented. The desired requirements are met by a combination of turbo-message-passing algorithms and smoothed ${\ell }_0$-refinements. Their performance is evaluated by use of extensive numerical simulations and compared with popular conventional algorithms.

1. Introduction

The compressive sensing (CS), affine rank minimization (ARM), and compressed robust principal component analysis (CRPCA) methods are enjoying great popularity in terms of their application in modern radar signal processing. At present, they are being applied in almost every possible field of application ranging from basic radar signal processing [1,2,3,4] to more sophisticated applications such as multiple-input multiple-output (MIMO) radar [5,6], ground moving target indication (GMTI) [7,8,9,10,11], synthetic aperture radar (SAR) [12,13,14], inverse synthetic aperture radar (ISAR) [15,16,17,18], interference and clutter mitigation [19,20,21,22], or SAR-GMTI [23,24,25], imaging [26], and passive radar [27], to name a few—this list is far from being exhaustive. Many more examples of CS applied to radar signal processing can be found e.g., in [28]. Alongside the specific properties of the application at hand, the success of these methods depends also on the algorithms for solving the emerging linear inverse problems. Many CS, ARM, and CRPCA algorithms found in the literature do not consider the practical requirements of radar signal processing. They either suffer from restrictions to real numbers, a slow convergence rate, or low reconstruction performance. Furthermore, many fast converging algorithms assume knowledge of generally unknown parameters, such as the precise number of sparse entries or the exact rank of a low-rank matrix. An overview of available algorithms is given in [29,30]. In this paper, a complete set of algorithms to solve general CS, ARM, and CRPCA problems is introduced that have the aforementioned properties. Due to the structural similarity of CS and ARM problems, a unified framework to solve them as well as the combined CRPCA problem can be formulated. The desirable properties are achieved by combining, augmenting, and extending turbo-message-passing algorithms and smoothed ${\ell }_0$-refinements. The turbo-message-passing framework provides a fast converging approach to obtain an initial ${\ell }_1$- or convex solution. The smoothed ${\ell }_0$-approach further improves upon the convex solution by enforcing a stricter sparsity or rank measure and as such improves the reconstruction performance. The presented algorithms are termed

• Turbo shrinkage-thresholding (TST)
• Complex successive concave sparsity approximation (CSCSA)
• Turbo singular value thresholding (TSVT)
• Complex smoothed rank approximation (CSRA)
• Turbo compressed robust principal component analysis (TCRPCA)

where TST and CSCSA apply to CS problems, TSVT and CSRA to ARM problems, and TCRPCA allows for solving combined CS and ARM problems. These algorithms are designed such that no parameters that are unknown in practice are required, and for unavoidable parameters, equations for their determination are given. The only parameter assumed to be known is noise power $P_{\mathrm{n}}$, which is a reasonable assumption in the field of radar applications. Furthermore, these algorithms offer a very high convergence rate alongside low computational complexity due to the use of closed solutions of subsequent optimization problems.

In this paper, the presented algorithms are evaluated in terms of extensive numerical simulations. These focus on comparing the algorithms, as generally as possible, to popular algorithms from the literature. For this purpose, phase transition plots, convergence and computation speed, and reconstruction performance in terms of signal to noise ratio (SNR) are suitable. The application of the presented algorithms to radar applications with real measurement data is a subject for future publications.

1.1. Background

Since CS, ARM, and CRPCA frameworks are very similar in structure, it is convenient to present them in a combined way. Their corresponding objectives are to recover solutions from limited noisy observations of the respective forms [31,32,33]

$$\begin{array}{cc}\hfill \mathit{y}& =\mathit{A}\tilde{\mathit{s}}+\mathit{n}\hfill \\ \hfill \mathit{y}& =\mathcal{A}(\tilde{\mathit{L}})+\mathit{n}\hfill \\ \hfill \mathit{y}& =\mathcal{A}(\tilde{\mathit{S}}+\tilde{\mathit{L}})+\mathit{n},\hfill \end{array}$$

where $\tilde{\mathit{s}}\in {\mathbb{C}}^n$ is an unknown sparse vector with $\mathsf{\kappa }\ll n$ entries, $\tilde{\mathit{S}}\in {\mathbb{C}}^{N_1\times N_2}$ is a corresponding sparse matrix with $\mathsf{\kappa }\ll n=N_1{N}_2$ entries, $\tilde{\mathit{L}}\in {\mathbb{C}}^{N_1\times N_2}$ is an unknown matrix whose $rank(\tilde{\mathit{L}})=$$\rho \ll n_{\min }=min\left(N_1,N_2\right)$, $\mathit{A}\in {\mathbb{C}}^{m\times n}$ ($m<n$) and $\mathcal{A}:{\mathbb{C}}^{N_1\times N_2}\to {\mathbb{C}}^m$ ($m<N_1{N}_2=n$) are known affine transformations, $y\in {\mathbb{C}}^m$ is a measurement vector, and $n\in {\mathbb{C}}^m$ is additive noise with complex normal i. i. d. coefficients of zero mean and variance $P_{\mathrm{n}}$. To find a solution to these under-determined linear systems, the closest sparse and low-rank solutions consistent with the measurements are sought via

$$\begin{array}{cc}\hfill \underset{\mathit{s}}{min}{∥\mathit{s}∥}_0\mathrm{subject}\mathrm{to}h\left(\mathit{s}\right)& \le {\epsilon }^2\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill \underset{\mathit{L}}{min}rank\left(\mathit{L}\right)\mathrm{subject}\mathrm{to}h\left(\mathit{L}\right)& \le {\epsilon }^2\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill \underset{\mathit{S},\mathit{L}}{min}\mathsf{\lambda }{∥\mathit{S}∥}_0+rank\left(\mathit{L}\right)\mathrm{subject}\mathrm{to}h(\mathit{S}+\mathit{L})& \le {\epsilon }^2,\hfill \end{array}$$

(3)

where

$$h\left(\mathit{s}\right)={∥\mathit{A}\mathit{s}-\mathit{y}∥}_2^2$$

(4)

and

$$h\left(\mathit{X}\right)={∥\mathcal{A}\left(\mathit{X}\right)-\mathit{y}∥}_2^2$$

(5)

are the data fidelity terms with ${∥·∥}_p$ denoting the ${\ell }_p$-norm, and ${\epsilon }^2\ge {∥\mathit{n}∥}_2^2$ is some constant error energy. It is well known that, in general, (1)–(3) are NP-hard to solve [33]. This has given rise to the plethora of algorithms that seek approximate formulations of (1) to (3) in order to find solutions in a computationally tractable manner.

1.2. State of the Art

The most popular relaxations to the respective problems (1) to (3) can be categorized into the families of convex relaxation and greedy algorithms [34]. Further approaches comprise hard thresholding (HT), smoothed-${\ell }_0$, and approximated message passing (AMP) algorithms.

All of the aforementioned approaches require the restricted isometry property (RIP) and restricted rank isometry property (RRIP) conditions

$$\begin{array}{cc}\hfill \left(1-{\mathsf{\delta }}_K\left(\mathit{A}\right)\right){∥\mathit{s}∥}_2^2\le {∥\mathit{A}\mathit{s}∥}_2^2& \le \left(1+{\mathsf{\delta }}_K\left(\mathit{A}\right)\right){∥\mathit{s}∥}_2^2\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill \left(1-{\mathsf{\delta }}_R\left(\mathcal{A}\right)\right){∥\mathit{L}∥}_{\mathrm{F}}^2\le {∥\mathcal{A}\left(\mathit{L}\right)∥}_{\mathrm{F}}^2& \le \left(1+{\mathsf{\delta }}_R\left(\mathcal{A}\right)\right){∥\mathit{L}∥}_{\mathrm{F}}^2,\hfill \end{array}$$

(7)

To be fulfilled for all ${∥s∥}_0\le K$ and $rank\left(\mathit{L}\right)\le R$ by the sensing operators A and $\mathcal{A}$ for some constants ${\mathsf{\delta }}_K$ and ${\mathsf{\delta }}_R$ [33,35]. Basically, (6) and (7) ensure that the sensing operators A and $\mathcal{A}$ keep different sparse vectors with at most K entries and different matrices of rank $\rho \le R$ distinguishable. Furthermore, for CRPCA, the sparse and low-rank matrices $\tilde{\mathit{S}}$ and $\tilde{\mathit{L}}$ must be distinguishable to allow for (3) to have a unique solution. This means that the sparse matrix $\tilde{\mathit{S}}$ must not be low ranking and, likewise, the low-rank matrix $\tilde{\mathit{L}}$ must not be of sparse nature. This condition is known as the rank-sparsity incoherence condition, which is fulfilled if the maximum number of nonzero entries per row and column in $\tilde{\mathit{S}}$ and the incoherence of L with respect to the standard basis are sufficiently small [36]. The first condition not only limits the total number of sparse entries in $\tilde{\mathit{S}}$ but also demands that its support is not clustered. The second condition demands that the singular vectors of $\tilde{\mathit{L}}$ are not sparse, i.e., are reasonably spread out. This entails the consequence that the maximum entry in magnitude of the dyadic ${\mathit{U}\mathit{V}}^{\mathrm{H}}$ for a given upper $\mathsf{\mu }$-incoherence is bounded as [32]

$${∥\mathit{U}{\mathit{V}}^{\mathrm{H}}∥}_{\infty }\le \frac{\mathsf{\mu }\rho }{\sqrt{N_1{N}_2}},$$

(8)

where $\tilde{\mathit{L}}={\mathit{U}\mathsf{\Sigma }\mathit{V}}^{\mathrm{H}}$ is the singular value decomposition (SVD) of $\tilde{\mathit{L}}$. This means $\tilde{\mathit{L}}$ must not contain spiky entries. In the following, the individual approaches and their advantages and disadvantages are briefly discussed.

1.2.1. Greedy Algorithms

The idea of greedy algorithms is to apply some heuristic approach that locally optimizes the objective functions (1) to (3) in an iterative procedure. Famous examples of CS greedy algorithms are orthogonal matching pursuit (OMP) [37], stagewise OMP (StOMP) [38], regularized OMP (ROMP) [39], its improved version compressive sampling matching pursuit (CoSaMP) [40], and subspace pursuit (SP) [41], though this list is far from exhaustive. The famous OMP algorithm and its improved version StOMP have a very simple structure that, however, goes along with the disadvantage that some incorrect sample index, once added to the support (which may happen in general), cannot be removed any more [42]. The CoSaMP and SP algorithms circumvent this problem by adding a backward step to prune wrongly selected support. This, however, requires knowledge of the sparsity level $\mathsf{\kappa }$ and rank $\rho$. Famous ARM and CRPCA greedy algorithms are atomic decomposition for minimum rank approximation (ADMiRA) and SpaRCS, respectively, where ADMiRA was inspired by CoSaMP and SpaRCS combines CoSaMP and ADMiRA [43,44]. Depending on the problem size, greedy algorithms are fast and of low computational complexity. However, they are known to be sensitive to noise [45,46]. Another issue arises in the context of closely spaced radar targets, for which greedy algorithms tend to merge such targets into a single one.

1.2.2. Hard Thresholding Algorithms

The objective functions of HT algorithms are [47,48,49]

$$\begin{array}{cc}\hfill \underset{\mathit{s}}{min}h\left(\mathit{s}\right)\mathrm{subject}\mathrm{to}{∥\mathit{s}∥}_0& \le K\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill \underset{\mathit{L}}{min}h\left(\mathit{L}\right)\mathrm{subject}\mathrm{to}rank\left(\mathit{L}\right)& \le R\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill \underset{\mathit{L}}{min}h\left(\mathit{S}+\mathit{L}\right)\mathrm{subject}\mathrm{to}rank\left(\mathit{L}\right)& \le R,{∥\mathit{S}∥}_0\le K,\hfill \end{array}$$

(11)

where $K\in \mathbb{N}$ and $R\in \mathbb{N}$ are some chosen constants. Famous examples are, e.g., normalized iterative hard thresholding (NIHT) for CS tasks [47], singular value projection (SVP) for ARM problems [48], and an extended combination of the aforementioned algorithm termed nonconvex free lunch (NFL) for CRPCA problems [49]. These algorithms have a very simple structure. They consist of a gradient update step followed by a HT operation as

$$\begin{array}{cc}\hfill {\mathit{s}}_{i+1}& ={\mathcal{H}}_{\mathrm{s},K}\left({\mathit{s}}_i-{\mathsf{\mu }}_i{\nabla }_{\mathit{s}}h\left({\mathit{s}}_i\right)\right)\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\mathit{L}}_{i+1}& ={\mathcal{H}}_{\mathrm{l},R}\left({\mathit{L}}_i-{\mathsf{\mu }}_i{\nabla }_{\mathit{L}}h\left({\mathit{L}}_i\right)\right).\hfill \end{array}$$

(13)

Within (12),

$$\begin{array}{cc}\hfill {\left[{\mathcal{H}}_{\mathrm{s},K}\left(\mathit{x}\right)\right]}_i& =\left\{\begin{array}{cc}{x}_i,\hfill & \mathrm{if}\left|x_i\right|\ge {\mathit{x}}^{\left[K\right]}\hfill \\ 0,\hfill & \mathrm{else}\hfill \end{array}\right.\hfill \end{array}$$

(14)

denotes the sparsity HT operator with ${\left[\mathit{x}\right]}_i$ being the $i\mathrm{th}$ entry and ${\mathit{x}}^{\left[K\right]}$ the $K\mathrm{th}$ biggest entry in magnitude in vector $\mathit{x}$. As such, ${\mathcal{H}}_{\mathrm{s},K}\left(\mathit{x}\right)$ sets all but the K largest elements in magnitude of $\mathit{x}$ to zero. Likewise, in (13),

$${\mathcal{H}}_{\mathrm{l},R}\left(\mathit{X}\right)=\sum _{i=1}^R{\mathsf{\sigma }}_i{\mathit{u}}_i{\mathit{v}}_i^{\mathrm{H}}$$

(15)

denotes the low-rank HT operator, where $\mathit{X}=\mathit{U}diag\left(\mathsf{\sigma }\right){\mathit{V}}^{\mathrm{H}}$ is the SVD of $\mathit{X}$ with ${\mathit{u}}_i$ and ${\mathit{v}}_i$ denoting the $i\mathrm{th}$ column vector of $\mathit{U}$ and $\mathit{V}$, and $\mathsf{\sigma }={\left[\begin{array}{ccc}{\mathsf{\sigma }}_1& \cdots & {\mathsf{\sigma }}_{n_{\min }}\end{array}\right]}^{\mathrm{T}}$ is a vector holding the singular values of $\mathit{X}$ in decreasing order. As such, ${\mathcal{H}}_{\mathrm{l},R}\left(\mathit{X}\right)$ sets all but the R largest singular values of $\mathit{X}$ to zero. The step sizes ${\mathsf{\mu }}_i$ in the update steps need to be updated per iteration, which is crucial for reconstruction success [47]. This approach shows a high convergence rate and offers guaranteed reconstruction performance provided $K\approx \mathsf{\kappa }$ and $R\approx \rho$. In the case of wrongly chosen parameters K and R, however, the reconstruction performance deteriorates significantly, as will be illustrated further below. The convergence rate was found to be higher than for the abovementioned greedy algorithms, although its reconstruction performance is slightly worse [50].

1.2.3. Convex Relaxations Algorithms

Another famous family comprises convex relaxations via the basis pursuit denoise (BPDN) and stable principal component pursuit (SPCP) approach [51,52]. The objective Functions (1) to (3) are relaxed to

$$\begin{array}{cc}\hfill \underset{\mathit{s}}{min}{∥\mathit{s}∥}_1\mathrm{subject}\mathrm{to}h\left(\mathit{s}\right)& \le {\epsilon }^2\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill \underset{\mathit{L}}{min}{∥\mathit{L}∥}_{\ast }\mathrm{subject}\mathrm{to}h\left(\mathit{L}\right)& \le {\epsilon }^2\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill \underset{\mathit{L}}{min}{\mathsf{\lambda }}_s{∥\mathit{S}∥}_1+{∥\mathit{L}∥}_{\ast }\mathrm{subject}\mathrm{to}h\left(\mathit{S}+\mathit{L}\right)& \le {\epsilon }^2,\hfill \end{array}$$

(18)

where ${∥·∥}_{\ast }$ denotes the nuclear norm. Famous examples that solve the regularized unconstrained formulations of (16) to (18)

$$\begin{array}{cc}\hfill \widehat{\mathit{s}}& =\underset{\mathit{s}}{arg\; min}{\mathsf{\lambda }}_s{∥\mathit{s}∥}_1+h\left(\mathit{s}\right)\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill \widehat{\mathit{L}}& =\underset{\mathit{L}}{arg\; min}{\mathsf{\lambda }}_l{∥\mathit{L}∥}_{\ast }+h\left(\mathit{L}\right)\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill \widehat{\mathit{S}},\widehat{\mathit{L}}& =\underset{\mathit{S},\mathit{L}}{arg\; min}{\mathsf{\lambda }}_l{∥\mathit{L}∥}_{\ast }+{\mathsf{\lambda }}_s{∥\mathit{S}∥}_1+h\left(\mathit{S}+\mathit{L}\right)\hfill \end{array}$$

(21)

are fast iterative shrinkage-thresholding algorithm (FISTA) and spectral projected gradient for ${\ell }_1$ (SPGL1) for CS [51,53], singular value thresholding (SVT) for ARM [31], and the variational approaches from [52] termed as SPCP. The respective underlying algorithm to FISTA, iterative shrinkage-thresholding algorithm (ISTA), and SVT show a similar structure as the HT algorithms, where a solution is obtained via the iterative procedures

$$\begin{array}{cc}\hfill {\mathit{s}}_{i+1}& ={\mathcal{S}}_{\mathrm{s},\mathsf{\mu }\mathsf{\lambda }}\left(\mathit{s}{}_i-{\mathsf{\mu }}_i{\nabla }_{\mathit{s}}h\left(\mathit{s}{}_i\right)\right)\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill {\mathit{L}}_{i+1}& ={\mathcal{S}}_{\mathrm{l},\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{L}}_i-{\mathsf{\mu }}_i{\nabla }_{\mathit{L}}h\left({\mathit{L}}_i\right)\right).\hfill \end{array}$$

(23)

Within (22), ${\mathcal{S}}_{\mathrm{s},a}\left(\mathit{x}\right)$ denotes the sparsity soft thresholding (ST) operator, which in this work is defined for complex numbers as

$${\left[{\mathcal{S}}_{\mathrm{s},a}\left(\mathit{x}\right)\right]}_i={\left(\left|x_i\right|-a\right)}_{+}sgn\left(x_i\right),$$

(24)

where ${\left(·\right)}_{+}=max\left(0,·\right)$ and

$$sgn\left(z\right)=\left\{\begin{array}{cc}0\hfill & \mathrm{if}z=0\hfill \\ \frac{z}{\left|z\right|}\hfill & \mathrm{else}\hfill \end{array}\right.$$

is the complex sign function [54]. Likewise, in (23),

$${\mathcal{S}}_{\mathrm{l},a}\left(\mathit{X}\right)=\sum _{i=1}^{n_{\min }}{\left({\mathsf{\sigma }}_i-a\right)}_{+}{\mathit{u}}_i{\mathit{v}}_i^{\mathrm{H}}$$

(25)

denotes the low-rank ST operator. Similar to the HT operators, the ST operators also possess an unknown parameter, namely the shrinkage parameter a. This shrinkage parameter, however, can be heuristically determined from the noise power $P_{\mathrm{n}}$, as will be shown further below. The convex relaxation approach thus does not suffer from requiring parameters that are hard to determine but, however, are known for their slow convergence and low reconstruction performance. A famous acceleration approach offers FISTA, which boosts the convergence of ISTA, $\mathcal{O}(\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$i$}\right.)$, up to $\mathcal{O}(\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$i^2$}\right.)$ without increasing the computational complexity notably [53].

1.2.4. Approximated Message-Passing Algorithms

In recent years, another form of acceleration technique has gained huge interest, namely AMP, from which the so-called turbo algorithms emerged [55,56,57,58,59]. This technique further improves the convergence rate of ST and HT approaches for suitable sensing operators $\mathit{A}$ and $\mathcal{A}$. The turbo algorithms presented in this work are inspired by these. To the best of the author’s knowledge, [55,56,57,58,59] presented their algorithms for generic and in particular HT operators but do not elaborate on ST operators, which are of major interest in this work. Utilizing the turbo approach in combination with a ST operator, a convex relaxed solution for CS and ARM problems can be found. Closed form solutions for required parameters of the turbo approach are presented in this work.

1.2.5. Smoothed ${\ell }_0$-Algorithms

The ${\ell }_1$- and nuclear norms are the tightest convex approximations of the ${\ell }_0$-norm and rank function, respectively [45,60]. However, it was found that the reconstruction error of solutions derived from convex relaxed approaches can be further reduced by applying smoothed ${\ell }_0$- and smoothed rank approximation frameworks [45,46,61,62,63,64]. Their purpose is to enforce a stricter sparsity or rank measure compared to the ${\ell }_1$- and nuclear norm. Doing so yields a refinement algorithm which allows for an improved reconstruction performance. Among the smoothed ${\ell }_0$-approaches, successive concave sparsity approximation (SCSA) appeals through its simplicity and efficiency due to the availability of closed form solutions of subsequent optimization steps. This algorithm was extended to the complex case named CSCSA [54] and serves, in the algorithm collection presented in this work, as the final refinement approach of CS applications. In the work presented here, a similar approach is applied to the smoothed rank approximation (SRA) algorithm from [64] to extend it into the complex case and to improve its efficiency due to the use of closed form solutions of subsequent minimization steps. The result called CSRA serves as the final refinement algorithm of ARM.

1.3. Contribution

In this paper, a set of algorithms applicable to radar signal processing tasks is presented. This is achieved by combining turbo-message-passing principles and smoothed ${\ell }_0$-refinements. Available approaches from the literature are extended to the complex case and closed form solutions for required parameters and subsequent minimization problems are derived. The resulting algorithms have a high convergence rate and low computational complexity, offer high reconstruction performance, and are easy to implement and free of unknown parameters. The performance in terms of reconstruction error, convergence rate, computation time, etc., is compared with classical standard algorithms.

1.4. Outline of the Paper

In Section 2, the ${\ell }_1$-norm minimization algorithm TST and its smoothed ${\ell }_0$-refinement algorithm CSCSA are presented. Section 3 provides the nuclear norm minimization algorithm TSVT and the corresponding refinement algorithm CSRA. In Section 4, the CRPCA algorithm as a combination of the aforementioned algorithm is presented. Within all aforementioned sections, numerical simulations are presented to evaluate the proposed algorithms and to compare them with popular alternative algorithms. Finally, concluding remarks are drawn in Section 5.

2. Compressed Sensing

In the following section, the TST and CSCSA algorithms are introduced. The TST algorithm obtains an initial sparse ${\ell }_1$- or convex solution, which is further refined by the CSCSA algorithm. Their combination yields all the desired properties of fast convergence, high reconstruction performance, no knowledge of generally unknown parameters etc.

2.1. Turbo Shrinkage Thresholding

The TST algorithm is inspired by the turbo algorithms presented in [55,56,57,59], which describe general reconstruction algorithms for CS and ARM problems applying the message-passing principle. This principle allows for a drastic improvement in the convergence speed of right-orthogonally invariant linear (ROIL) sensing operators to which random and discrete Fourier transform (DFT) sensing operators belong (consider a linear operator $\mathcal{A}$ with matrix form A, the SVD of A is $\mathit{A}={\mathit{U}}_A{\mathsf{\Sigma }}_A{\mathit{V}}_A^{{H}}$. If ${\mathit{V}}_A$ is a Haar-distributed random matrix independent of ${\mathsf{\Sigma }}_A$, then $\mathcal{A}$ is a right-orthogonally invariant linear (ROIL) operator [57]). The two main contributions presented in this paper are (1) the provision of required formulas for the complex case and (2) a closed form solution of the required divergence of the ST denoiser, hence the name TST algorithm.

The TST algorithm attempts to find a solution to (1) by solving the convex relaxed regularized optimization problem (19). This is achieved using the turbo-framework iterative procedure [57]

$$\begin{array}{ccc}\hfill {\mathit{r}}_i& =& {\mathit{s}}_{i-1}-\mathsf{\mu }{\nabla }_{\mathit{s}}h\left({\mathit{s}}_{i-1}\right)\hfill \\ \hfill {\mathit{z}}_i& =& {\mathcal{S}}_{S,\mathsf{\mu }{\lambda }}\left({\mathit{r}}_i\right)\hfill \\ \hfill {\mathit{s}}_i& =& c_i\left({\mathit{z}}_i-{\mathsf{\alpha }}_i{\mathit{r}}_i\right),\hfill \end{array}$$

(26)

where $\mathsf{\mu }$ is some step size, i is the iteration index, and ${\mathcal{S}}_{\mathrm{s},a}$ is the soft thresholding operator given by (24). The required parameters $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$ are chosen according to the turbo principle such that

$$\begin{array}{cc}\hfill {\left({\mathit{r}}_i-\tilde{\mathit{s}}\right)}^{\mathrm{H}}\left({\mathit{s}}_{i-1}-\tilde{\mathit{s}}\right)& =0,\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill {\left({\mathit{r}}_i-\tilde{\mathit{s}}\right)}^{\mathrm{H}}\left({\mathit{s}}_i-\tilde{\mathit{s}}\right)& =0,\hfill \end{array}$$

(28)

and further that for a given ${\mathit{s}}_{i-1}$, ${∥{\mathit{s}}_i-\tilde{\mathit{s}}∥}_2^2$ is minimized under (27) and (28) [57]. In the above, (27) ensures that the input and output error of the gradient update step are uncorrelated. Equally, (28) ensures a decorrelation of the input and output error of the denoising or ST step. This strategy allows for an improved convergence rate compared with classical gradient approaches. In order to fulfill (27) and (28), the true solution $\tilde{s}$ is required to determine exact values for $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$. Since $\tilde{\mathit{s}}$ is unknown, asymptotic formulations for $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$ for $n\to \infty$ were derived in [57] for the real valued case. In the general complex case, these parameters can be adapted to

$$\begin{array}{cc}\hfill \mathsf{\mu }& =\frac{n}{m{∥\mathit{A}∥}_2^2}\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill {\mathsf{\alpha }}_i& =\left\{\begin{array}{cc}\frac{1}{n}\mathrm{div}\left({\mathcal{S}}_{S,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{r}}_i\right)\right)\hfill & \mathrm{if}\mathit{s}\in {\mathbb{R}}^n\hfill \\ \frac{1}{2n}\mathrm{div}\left({\mathcal{S}}_{S,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{r}}_i\right)\right)\hfill & \mathrm{if}\mathit{s}\in {\mathbb{C}}^n\hfill \end{array}\right.\hfill \end{array}$$

(30)

$$\begin{array}{cc}\hfill c_i& =\frac{{\left({\mathit{z}}_i-{\mathsf{\alpha }}_i{\mathit{r}}_i\right)}^{\mathrm{H}}{\mathit{r}}_i}{{∥{\mathit{z}}_i-{\mathsf{\alpha }}_i{\mathit{r}}_i∥}_2^2},\hfill \end{array}$$

(31)

where $\mathrm{div}\left(·\right)$ is the weak divergence operator. For small scene sizes n, it was found from simulations that the step size $\mathsf{\mu }$ required for convergence has to be reduced to $\mathsf{\mu }<1/{∥\mathit{A}∥}_2^2$, which corresponds to the stable step size of the FISTA algorithm. This phenomena was not discussed in [55,56,57,59]; however, since the parameter Equations (29) to (31) are asymptotic approximations, their validity requires the scene size to be sufficiently large. Nevertheless, TST still shows superior convergence speed even in the reduced step size case. The required weak divergence is derived in Appendix A in closed form as

$$\mathrm{div}\left({\mathcal{S}}_{\mathit{S},a}\left(\mathit{r}\right)\right)=\sum _{i=1}^n\left(2-\frac{a}{\left|r_i\right|}\right)\mathbb{I}\left(\left|r_i\right|>a\right),$$

(32)

where $\mathbb{I}\left(·\right)$ denotes the indicator function. The next question of course is how to choose the regularization parameter $\mathsf{\lambda }$ and thus which shrinkage $a=\mathsf{\mu }\mathsf{\lambda }$ should be applied. The usage of a ST operator causes the optimal solution ${\mathit{s}}_{\mathrm{opt}}$ generated by the turbo framework (26) to possess an offset of a for every sparse entry compared with the true solution $\tilde{\mathit{s}}$. Too much shrinkage results therefore in a large bias, while too little results in a slow convergence rate. As such, an optimal constant $\mathsf{\lambda }$ does not exist; rather, $\mathsf{\lambda }$ would need to be adjusted in every iteration. Unfortunately, to the best of the author’s knowledge, a closed form solution to determine such an optimal $\mathsf{\lambda }$ is not known to exist. A reasonable choice for a constant parameter $\mathsf{\lambda }$ is a scaled version of the formula of [65]

$$\mathsf{\lambda }=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$4c_r$}\right.\sqrt{P_{\mathrm{n}}}{\mathsf{\Phi }}^{-1}\left(1-\frac{{\mathsf{\alpha }}_r}{2n}\right),$$

(33)

which was found from simulations to perform well as illustrated further below. The parameter $c_r>1$ in (33) is some constant, $\mathsf{\Phi }$ is the cumulative density function (CDF) of $\mathcal{N}\left(0,1\right)$, and ${\mathsf{\alpha }}_r\in \left[0,1\right]$ is some parameter. With this choice, a least absolute shrinkage and selection operator (LASSO) estimator achieve a so-called near-oracle performance with a probability of at least $1-{\mathsf{\alpha }}_r$ [53]. For further details, e.g., proof of convergence, interested readers may refer to the proofs given in [55,56,57,59], which also hold for the TST version presented here.

Putting all the above steps together, the final TST algorithm is listed in Algorithm 1. It is aborted either after a maximum number of iterations I, or if the relative improvement from iteration to iteration

$$d_i={∥\mathit{s}{}_i-\mathit{s}{}_{i-1}∥}_2/{∥\mathit{s}{}_{i-1}∥}_2$$

(34)

drops below a certain threshold $\mathsf{ϵ}$. The particular values to determine $\mathsf{ϵ}$ in Algorithm 1 are taken from [63] and proven to work well in every case.

Algorithm 1 The TST algorithm.

Input: A, y, λ, I Initialization:   1:       $\mathsf{ϵ}\leftarrow min\left({10}^{-4},5·{10}^{-3}\mathsf{\lambda }\right)$, $i\leftarrow 0$, $d\leftarrow \infty$, ${\mathit{s}}_0\leftarrow \mathbf{0}$   2:       $\mathsf{\mu }\leftarrow (0.99/{∥\mathit{A}∥}_2^2,n/(m{∥\mathit{A}∥}_2^2))$ Body:   1:  while $d>\mathsf{ϵ}$ and $i<I$ do   2:       $i\leftarrow i+1$   3:       ${\mathit{r}}_i\leftarrow {\mathit{s}}_{i-1}-\mathsf{\mu }{\nabla }_{\mathit{s}}h\left({\mathit{s}}_{i-1}\right)$   4:       ${\mathit{z}}_i\leftarrow {\mathcal{S}}_{S,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{r}}_i\right)$   5:       ${\mathsf{\alpha }}_i\leftarrow \left\{\begin{array}{cc}\frac{1}{n}\mathrm{div}\left({\mathcal{S}}_{S,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{r}}_i\right)\right)\hfill & \mathrm{if}\mathit{s}\in {\mathbb{R}}^n\hfill \\ \frac{1}{2n}\mathrm{div}\left({\mathcal{S}}_{S,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{r}}_i\right)\right)& \mathrm{if}\mathit{s}\in {\mathbb{C}}^n\end{array}\right.$   6:       $c_i\leftarrow {\left({\mathit{z}}_i-{\mathsf{\alpha }}_i{\mathit{r}}_i\right)}^{\mathrm{H}}{\mathit{r}}_i/{∥{\mathit{z}}_i-{\mathsf{\alpha }}_i{\mathit{r}}_i∥}_2^2$   7:       ${\mathit{s}}_i\leftarrow c_i\left({\mathit{z}}_i-{\mathsf{\alpha }}_i{\mathit{r}}_i\right)$   8:       $d\leftarrow {∥{\mathit{s}}_i-{\mathit{s}}_{i-1}∥}_2/{∥{\mathit{s}}_{i-1}∥}_2$ 9: end while Output: $\widehat{\mathit{s}}\leftarrow {\mathit{r}}_i$

In the following, simulation results are shown to illustrate the performance of TST. For all the following simulations presented, the SNR is defined as

$$SNR={∥\mathit{A}\tilde{\mathit{s}}∥}_2^2/{∥\mathit{n}∥}_2^2$$

(35)

and the recovery quality is measured as squared reconstruction error (SRE)

$$SRE={∥\tilde{\mathit{s}}-\widehat{\mathit{s}}∥}_2^2/{∥\tilde{\mathit{s}}∥}_2^2.$$

(36)

The reconstruction performance is evaluated by use of phase transition plots [66]. For a given type of sensing operator A (random or DFT), a number of sparse vectors $\tilde{\mathit{s}}$ are to be reconstructed for different measurement ratios $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$n$}\right.\in (0,1]$ and sparsity ratios $\raisebox{1ex}{$\mathsf{\kappa }$}\!\left/ \!\raisebox{-1ex}{$m$}\right.\in (0,1]$. The resulting SRE is averaged over $N_{\mathrm{mc}}=100$ Monte Carlo runs as $\frac{1}{N_{\mathrm{mc}}}{\sum }_{i=1}^{N_{\mathrm{mc}}}{SRE}_i$ with ${SRE}_i$ denoting the SRE of the $i\mathrm{th}$ reconstruction. In general, the higher the measurement rate $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$n$}\right.$ is, the easier it is to find a solution to (1). Likewise, the higher the sparsity ratio $\raisebox{1ex}{$\mathsf{\kappa }$}\!\left/ \!\raisebox{-1ex}{$m$}\right.$ is, the more difficult the reconstruction problem becomes. Hence, the phase transition plot is rendered into a collection of reconstruction problems of varying difficulty. Various reconstruction algorithms now compete on how many problems of varying difficulty they can successfully reconstruct. Usually, a sharp transition of reconstructable from non-reconstructable problems arise for a given algorithm, hence the name phase transition diagram. For a single reconstruction problem, the sparse vector $\tilde{\mathit{s}}$ is set up by first determining $\mathsf{\kappa }$ and m from the given measurement and sparsity ratios. Then, $\mathsf{\kappa }$ support indices are drawn from an i. i. d. $\mathcal{U}\left(1,n\right)$ distribution. The entries at the determined indices are drawn from an i. i. d. complex Gaussian distribution. Next, the sensing operators A of size $m\times n$ are set up, where for random sensing operators $\mathit{A}\sim \mathcal{CN}\left(0,1\right)$ or for DFT operators A is set up from m randomly selected rows of an $n\times n$ DFT matrix. Finally, the elements of the noise vector n are drawn from an i. i. d. standard complex Gaussian distribution and scaled to the given SNR according to (35). For the simulation results shown in the following $n=1250$, $SNR=40\mathrm{dB}$, and $I=300$ is used. It should be noted that random sensing matrices are of limited interest in radar engineering and serve here merely as a comparison benchmark, since many algorithms in the literature are stated for random sensing matrices alone. The DFT sensing operator servers as a representative for radar applications, since it often occurs there. Naturally, many more exist, a thorough evaluation of the TST algorithm for the CS applications mentioned in Section 1 is, however, beyond the scope of this text.

As comparison benchmarks, the aforementioned simulations are also conducted for SPGL1, FISTA, and NIHT. The SPGL1 algorithm is set up to solve the BPDN problem and as such is equipped with the true noise power $P_{\mathrm{n}}$. Its implementation is taken from its official Github repository (https://github.com/mpf/spgl1, accessed on 20 December 2022, version v1.9). The maximum number of iterations is set to $I=300$ iterations, and for all remaining parameters, the default setting is used. The FISTA algorithm is implemented according to [53] and its regularization parameter $\mathsf{\lambda }$ is set to (33). The NIHT algorithm is likewise implemented according to [47] and equipped once with the true number of sparse entries $\mathsf{\kappa }$ and once with twice the number of sparse entries $2\mathsf{\kappa }$. Both algorithms are aborted after a maximum number of $I=300$ iterations or if the relative improvement $d_{\mathrm{i}}$, as defined in (34), is below a threshold similar to the TST algorithm.

The phase transition diagrams of the TST algorithm for random and DFT sensing operators are shown in Figure 1a,b. Its overall reconstruction performance is higher than for SPGL1 shown in Figure 1c,d. Only for a very low sparsity ratio does SPGL1 achieve better performance. The reconstruction performance of FISTA is shown in Figure 1e,f. Its performance is inferior to that of the aforementioned algorithms. Figure 1g,h show the reconstruction performance of NIHT, where the parameter K was set to the true number of sparse entries $\mathsf{\kappa }$. The result in the case of a wrongly chosen parameter $K=2\mathsf{\kappa }$ is shown in Figure 2. As can be seen, the reconstruction success depends heavily on K. Compared with FISTA and NIHT, TST exhibits better performance overall. A comparison of the convergence speed is shown in Figure 3, which shows the intermediate SREs. The convergence speed of NIHT in the case of $K=\mathsf{\kappa }$ outperforms that of the TST algorithm; however, it drops significantly in the case of $K=2\mathsf{\kappa }$. A comparison of the computation time is shown in Figure 4 for a region where all algorithms perform equally well, except for FISTA. The simulations are conducted using Matlab^® R2022b on an Ubuntu 20.04 LTS OS equipped with an Intel^® Xeon(R) CPU E5-2687W v3 @ 3.10 GHz × 10. The fastest algorithm is TST, followed by FISTA and NIHT, provided $K=\mathsf{\kappa }$ is used. The reason why NIHT is not the fastest algorithm, despite its superior convergence rate, is its costly line search procedure. The slowest algorithm is SPGL1; however, it should be noted that no effort in optimizing its performance was made. Finally, the SRE for various SNRs is shown in Figure 5 for all tested algorithms. As can be seen, the best SRE offers NIHT in the case of $K=\mathsf{\kappa }$. For $SNR=0\mathrm{dB}$, FISTA outperforms TST, but for higher SNR, however, TST achieves better performance. SPGL1 closely follows the available SNR level. In summary, TST is very easy to implement (in contrast to SPGL1), shows a state-of-the-art convergence rate, has low computational complexity as there are closed form solutions available for all required parameters, and finally does not generally require any unknown parameters such as NIHT. Hence, it is well suited for practical applications.

The reconstruction performance of ${\ell }_1$-relaxed minimization approaches can be further improved by the use of smoothed ${\ell }_0$-techniques. A suitable and convenient version thereof is presented in the following section.

2.2. Complex Successive Concave Sparsity Approximation

The reconstruction error of ${\ell }_1$-relaxed minimization approaches can be further reduced by smoothed ${\ell }_0$-frameworks. The key idea is to approximate the ${\ell }_0$-quasi-norm in the original objective Function (1) by a smooth function. The CSCSA algorithm does so, which is the author’s extension of the SCSA algorithm from [63], capable of handling complex numbers. The CSCSA algorithm was published in [54] and is summarized in the following in brevity. Simulation results illustrating the performance of CSCSA in combination with the TST, SPGL1, and FISTA algorithms are presented in the following. Due to the structural similarity between CS and ARM problems, the idea of CSCSA can also be applied on ARM problems as presented in Section 3.2.

The idea of CSCSA is to substitute the ${\ell }_0$-quasi-norm of $\mathit{s}={[s_1\cdots s_n]}^{\mathrm{T}}$ by a more tractable approximation. In general, the ${\ell }_0$-quasi-norm is defined as the number of nonzero elements in s. Let

$$\mathsf{\delta }\left(x\right)=\left\{\begin{array}{cc}1\hfill & \mathrm{if}x=0\hfill \\ 0\hfill & \mathrm{else}\hfill \end{array}\right.$$

(37)

be the Kronecker delta function, then the ${\ell }_0$-quasi-norm of s can be defined as

$${∥\mathit{s}∥}_0=\sum _{i=1}^n\left[1-\mathsf{\delta }\left(\left|s_i\right|\right)\right],$$

(38)

where $\left|s_i\right|$ denotes the magnitude of $s_i$. To make (38) smooth, it may be approximated by

$$1-\mathsf{\delta }\left(\left|s\right|\right)\approx f_{\mathsf{\gamma }}\left(\left|s\right|\right)=1-exp\left(-\frac{\left|s\right|}{\mathsf{\gamma }}\right),$$

(39)

where $\mathsf{\gamma }$ determines how accurately the Kronecker function is approximated. An illustration of this approximation and other common approximations to the ${\ell }_0$-quasi-norm are shown in Figure 6. As shown in [63], the series $\left\{f_{\mathsf{\gamma }}\left(\left|s\right|\right)\right\}$ converges pointwise to $1-\mathsf{\delta }\left(\left|s\right|\right)$ as

$$\underset{\mathsf{\gamma }\to 0^{+}}{lim}{f}_{\mathsf{\gamma }}\left(\left|s\right|\right)=\left\{\begin{array}{cc}0\hfill & \mathrm{if}\left|s\right|=0\hfill \\ 1\hfill & \mathrm{else}.\hfill \end{array}\right.$$

An approximation to the ${\ell }_0$-quasi-norm is therefore

$${∥\mathit{s}∥}_0\approx \sum _{i=1}^n{f}_{\mathsf{\gamma }}\left(\left|s_i\right|\right)=F_{\mathsf{\gamma }}\left(\left|\mathit{s}\right|\right),$$

(40)

where $\left|\mathit{s}\right|$ denotes a vector, which holds the magnitudes of the elements of s. The optimization Problem (1) may now be relaxed to

$$\underset{\mathit{s}}{min}{F}_{\mathsf{\gamma }}\left(\left|\mathit{s}\right|\right)\mathrm{subject}\mathrm{to}h\left(\mathit{s}\right)\le {\mathsf{ϵ}}^2,$$

(41)

where the residual term $h\left(\mathit{s}\right)$ was defined in (4) and ${\mathsf{ϵ}}^2\ge {∥\mathit{n}∥}_2^2$ is some constant noise energy. The constrained optimization Problem (41) can, for a fixed $\mathsf{\gamma }$, be converted to an unconstrained optimization problem by use of regularization as

$$\widehat{\mathit{s}}=\underset{\mathit{s}}{arg\; min}{\mathsf{\lambda }}_{\mathsf{\gamma }}{F}_{\mathsf{\gamma }}\left(\left|\mathit{s}\right|\right)+h\left(\mathit{s}\right).$$

(42)

At this point, it should be noted that the Program (42) is not convex anymore but, rather, a sum of a concave and a convex function. As such, (42) does not possess a unique minimum, and it is possible to get stuck in local minimum. To circumvent this problem, the graduated non-convexity (GNC) approach is used. The idea of GNC is to start the Program (42) with a solution ${\widehat{\mathit{s}}}_0$ that is somewhat close to the true solution $\tilde{\mathit{s}}$ obtained from a convex algorithm, e.g., FISTA or TST. Then, (42) is minimized for a $\mathsf{\gamma }$ large enough such that the solution $\widehat{\mathit{s}}$ is closer to $\tilde{\mathit{s}}$ yet does not get stuck in a wrong local minimum. Subsequently, $\mathsf{\gamma }$ is reduced by a constant factor $\mathsf{\gamma }\leftarrow c\mathsf{\gamma }$ to further approximate the ${\ell }_0$-quasi-norm, and (42) is minimized using the solution from the previous iteration. The procedure is conducted until a stop criterion is met.

For a fixed $\mathsf{\gamma }$, the optimization Problem (42) can be solved by use of the iterative thresholding (IT) approach, similar to ISTA (22), by iteratively conducting

$$\begin{array}{cc}\hfill {\mathit{s}}_{i+1}& ={\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}\left(\mathit{s}{}_i-\mathsf{\mu }{\nabla }_{\mathit{s}}h\left(\mathit{s}{}_i\right)\right),\hfill \end{array}$$

(43)

where $\mathsf{\mu }$ is some step size and

$${\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}\left(x\right)=\left\{\begin{array}{cc}0\hfill & \left|x\right|<\mathsf{\gamma }\left(1+ln\left(\frac{2{\mathsf{\lambda }}_{\mathsf{\gamma }}\mathsf{\mu }}{{\mathsf{\gamma }}^2}\right)\right)\hfill \\ 0\hfill & L^{\prime }\left(0,\left|x\right|\right)<L^{\prime }\left(\left|s_1\right|,\left|x\right|\right),\hfill \\ s_1\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(44)

is again an elementwise thresholding operator. Furthermore,

$$\begin{array}{cc}\hfill L^{\prime }\left(p,q\right)& =\frac{1}{2\mathsf{\mu }}{\left(p-q\right)}^2+{\mathsf{\lambda }}_{\mathsf{\gamma }}{f}_{\mathsf{\gamma }}\left(p\right)\hfill \\ \hfill s_1& =\mathsf{\gamma }{W}_0\left(z\right)sgn\left(x\right)+x\hfill \\ \hfill z& =-\frac{2{\mathsf{\lambda }}_{\mathsf{\gamma }}\mathsf{\mu }}{{\mathsf{\gamma }}^2}exp\left(-\frac{\left|x\right|}{\mathsf{\gamma }}\right)\hfill \end{array}$$

and $W_0\left(·\right)$ denotes the upper branch of the multi-valued Lambert W function [54]. It should be noted that ${\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}$ is a closed form solution of a subsequent minimization problem emerging from the IT approach. This closed form is possible only through the special choice of the ${\ell }_0$-quasi-norm approximation (39), which leads to the possibility of applying the Lambert W function. For different choices, the subsequent minimization problem would require an additional minimization loop increasing the computational complexity. The regularization parameter is set to ${\mathsf{\lambda }}_{\mathsf{\gamma }}=2\mathsf{\gamma }\mathsf{\lambda }$, where (33) is used for $\mathsf{\lambda }$. Furthermore, the initial value ${\mathsf{\gamma }}_0$ is set to ${\mathsf{\gamma }}_0=max\left(\left|{\mathit{s}}_0\right|\right)/10$, where ${\mathit{s}}_0$ denotes the convex relaxed initial solution. More details regarding the CSCSA algorithm and the special selections of ${\mathsf{\lambda }}_{\mathsf{\gamma }}$ and ${\mathsf{\gamma }}_0$ are given in [54]. Finally, to further improve the convergence speed of the applied IT approach, a FISTA-like technique can be used as in [53], which does not increase the computational complexity but boosts the convergence rate from $\mathcal{O}(\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$i$}\right.)$ up to $\mathcal{O}\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$i^2$}\right.$. Putting all the above steps together, the final CSCSA algorithm is listed in Algorithm 2. It consists of two loops, an inner and an outer one. In the outer loop $\mathsf{\gamma }$ is decreased gradually according to the GNC technique. The inner loop solves (42) by using a FISTA-like technique. The loops are aborted after a maximum number of iterations J and P or if the solution change, measured by the relative distance between consecutive solutions, drops below certain thresholds ${\mathsf{ϵ}}_{\mathrm{o}}$ and ${\mathsf{ϵ}}_{\mathrm{i}}$. The threshold selections given in Algorithm 2 are taken from [63] and shown to work well in the following simulations.

Algorithm 2 The CSCSA algorithm.

Input: A, y, λ, I, J Initialization:   1:       $c\leftarrow \left(0,0.5\right)$, $\mathsf{\mu }\leftarrow 0.99/{∥\mathit{A}∥}_2^2$   2:       ${\mathsf{ϵ}}_{\mathrm{o}}\leftarrow min\left({10}^{-4},5·{10}^{-3}\mathsf{\lambda }\right)$   3:       ${\mathsf{ϵ}}_{\mathrm{i}}\leftarrow min\left({10}^{-3},5·{10}^{-3}\mathsf{\lambda }\right)$   4:       $\widehat{\mathit{s}}\leftarrow \mathrm{TST}\left(\mathit{A},\mathit{y},\mathsf{\lambda },{\mathsf{ϵ}}_{\mathrm{i}},J\right)$, $\mathsf{\gamma }\leftarrow max\left(\left|\widehat{\mathit{s}}\right|\right)/10$ Body:   1: $i\leftarrow 0$, $d_{\mathrm{o}}\leftarrow \infty$   2: while $d_{\mathrm{o}}>{\mathsf{ϵ}}_{\mathrm{o}}$ and $i<I$ do   3:       $i\leftarrow i+1$, $j\leftarrow 0$, $d_{\mathrm{i}}\leftarrow \infty$   4:       $t_1\leftarrow 1$, ${\mathit{z}}_1\leftarrow \widehat{\mathit{s}}$, ${\mathit{s}}_0\leftarrow \widehat{\mathit{s}}$, ${\mathsf{\lambda }}_{\mathsf{\gamma }}\leftarrow 2\mathsf{\lambda }\mathsf{\gamma }$   5:       while $d_{\mathrm{i}}>{\mathsf{ϵ}}_{\mathrm{i}}$ and $j<J$ do   6:           $j\leftarrow j+1$   7:           ${\mathit{s}}_j\leftarrow {\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}\left({\mathit{z}}_j-\mathsf{\mu }{\mathit{A}}^{\mathrm{H}}\left(\mathit{A}{\mathit{z}}_j-\mathit{y}\right)\right)$   8:           $t_{j+1}\leftarrow \left(1+\sqrt{1+4t_j^2}\right)/2$   9:           ${\mathit{z}}_{j+1}\leftarrow {\mathit{s}}_j+\left(t_j-1\right)\left({\mathit{s}}_j-{\mathit{s}}_{j-1}\right)/t_{j+1}$   10:         $d_{\mathrm{i}}\leftarrow {∥{\mathit{s}}_j-{\mathit{s}}_{j-1}∥}_2/{∥{\mathit{s}}_{j-1}∥}_2$   11:       end while   12:       $d_{\mathrm{o}}\leftarrow {∥{\mathit{s}}_j-\widehat{\mathit{s}}∥}_2/{∥\widehat{\mathit{s}}∥}_2$   13:       $\widehat{\mathit{s}}\leftarrow {\mathit{s}}_j$   14:       $\mathsf{\gamma }\leftarrow c\mathsf{\gamma }$   15: end while Output: $\widehat{\mathit{s}}$

Simulation results of a CSCSA update onto FISTA, SPGL1, and TST for random and DFT sensing operators are shown in Figure 7. For $\mathsf{\lambda }$, (33) was used and the decreasing factor for $\mathsf{\gamma }$ was set to $c=0.1$. As can be seen, CSCSA significantly improves upon the convex results shown in Figure 1. A comparison of the reconstruction performances is shown in Figure 8, in which the curves indicate the $50\%$ success rate with respect to the Monte Carlo runs. Success is defined as $SRE\le -SNR$. In addition, the $25\%$ and $75\%$ success rate confidence intervals are indicated as shaded areas. The CSCSA refinement algorithm basically obtains similar reconstruction performance for every initialization algorithm as shown in Figure 8.

3. Affine Rank Minimization

In the following section, the TSVT and CSRA algorithms are introduced. Due to the structural similarity of CS and ARM algorithms, the TSVT and CSRA algorithms are also similar in structure to their CS counterparts from the former section. The TSVT algorithm obtains an initial low-rank convex solution, which is further refined by the CSRA algorithm. Their combination, likewise, yields all the desired properties of fast convergence and high reconstruction performance with no knowledge of generally unknown parameters, e.g., the true rank $\rho$ of $\tilde{\mathit{L}}$ etc.

3.1. Turbo Singular Value Thresholding

The TSVT algorithm is inspired by turbo affine rank minimization (TARM) presented in [57], which is solved for ARM problems applying the turbo principle. This principle allows for a drastic improvement in convergence speed for ROIL sensing operators to which random and DFT sensing operators belong. Within TARM, a HT operator is applied. Therefore, TARM was reworked here to use a ST operator. After finalizing TSVT, it came to the author’s attention, that a similar approach was formerly published in [56] called singular value thresholding-turbo-compressive sensing (SVT-Turbo-CS), conducting the same reconstruction as TSVT. The contributions of this work beyond SVT-Turbo-CS are twofold: (1) expansion into the complex case and (2) providing an equation to select the regularization parameter $\mathsf{\lambda }$.

The TSVT algorithm attempts to find a solution to (2) by solving the convex relaxed regularized optimization Problem (20). This is achieved using the iterative procedure

$$\begin{array}{cc}\hfill {\mathit{R}}_i& ={\mathit{L}}_{i-1}-\mathsf{\mu }{\nabla }_{\mathit{L}}h\left({\mathit{L}}_{i-1}\right)\hfill \\ \hfill {\mathit{Z}}_i& ={\mathcal{S}}_{\mathrm{l},\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{R}}_i\right)\hfill \\ \hfill {\mathit{L}}_i& =c_i\left({\mathit{Z}}_i-{\mathsf{\alpha }}_i{\mathit{R}}_i\right)\hfill \end{array}$$

(45)

where $\mathsf{\mu }$ is some step size, i is the iteration index, and ${\mathcal{S}}_{\mathrm{l},a}\left(\mathit{X}\right)$ is the ST operator given by (25). The required parameters $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$ are chosen according to the turbo principle such that [57]

$$\begin{array}{cc}\hfill {〈{\mathit{R}}_i-\tilde{\mathit{L}},{\mathit{L}}_{i-1}-\tilde{\mathit{L}}〉}_{\mathrm{F}}& =0\hfill \end{array}$$

(46)

$$\begin{array}{cc}\hfill {〈{\mathit{R}}_i-\tilde{\mathit{L}},{\mathit{L}}_i-\tilde{\mathit{L}}〉}_{\mathrm{F}}& =0\hfill \end{array}$$

(47)

and that, furthermore, for a given ${\mathit{L}}_{i-1}$, ${∥{\mathit{L}}_i-\tilde{\mathit{L}}∥}_{\mathrm{F}}^2$ is minimized under (46) and (47). In the above equations, ${〈\mathit{X},\mathit{Y}〉}_{\mathrm{F}}$ denotes the Frobenius product. Equation (46) ensures that the input and output error of the gradient update step are uncorrelated. Equally, (47) ensures a decorrelation of the input and output error of the denoising step. This strategy allows for an improved convergence rate compared to classical gradient approaches. In order to fulfill (46) and (47), the true solution $\tilde{\mathit{L}}$ is required to determine exact values for $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$. Since $\tilde{\mathit{L}}$ is unknown, approximate formulations for $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$ were derived in [57] for the real valued case. In the general case these parameters are

$$\begin{array}{cc}\hfill \mathsf{\mu }& =\frac{n}{m{∥\mathcal{A}∥}_2^2}\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill {\mathsf{\alpha }}_i& =\left\{\begin{array}{cc}\frac{1}{n}\mathrm{div}\left({\mathcal{S}}_{l,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{R}}_i\right)\right)\hfill & \mathrm{if}\mathit{X}\in {\mathbb{R}}^{N_1\times N_2}\hfill \\ \frac{1}{2n}\mathrm{div}\left({\mathcal{S}}_{l,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{R}}_i\right)\right)\hfill & \mathrm{if}\mathit{X}\in {\mathbb{C}}^{N_1\times N_2}\hfill \end{array}\right.\hfill \end{array}$$

(49)

$$\begin{array}{cc}\hfill c_i& =\frac{{〈{\mathit{Z}}_i-{\mathsf{\alpha }}_i{\mathit{R}}_i,{\mathit{R}}_i〉}_{\mathrm{F}}}{{∥{\mathit{Z}}_i-{\mathsf{\alpha }}_i{\mathit{R}}_i∥}_{\mathrm{F}}^2}.\hfill \end{array}$$

(50)

For small scene sizes $N_1\times N_2$, it was found from simulations that the step size $\mathsf{\mu }$ required for convergence has to be reduced to $\mathsf{\mu }<1/{∥\mathcal{A}∥}_2^2$, which corresponds to the stable step size of the SVT algorithm. This phenomena was not discussed in [57,59], however, since the parameter Equations (48) to (50) are asymptotic approximations, and their validity requires the scene size to be sufficiently large. Nevertheless, TSVT still shows superior convergence speed even in the case of reduced step size. For the required divergence operator in (49), which is to be interpreted in the weak sense, i.e., it can fail to exist on negligible sets, a closed form solution exists [67] as

$$\mathrm{div}\left({\mathcal{S}}_{l,a}\left(\mathit{X}\right)\right)=\sum _{j=1}^{n_{\min }}\left[\mathbb{I}\left({\mathsf{\sigma }}_j>a\right)+A_j\right]+2B$$

(51)

for $\mathit{X}\in {\mathbb{R}}^{N_1\times N_2}$ or

$$\mathrm{div}\left({\mathcal{S}}_{l,a}\left(\mathit{X}\right)\right)=\sum _{i=j}^{n_{\min }}\left[\mathbb{I}\left({\mathsf{\sigma }}_j>a\right)+A_j\right]+4B$$

(52)

for $\mathit{X}\in {\mathbb{C}}^{N_1\times N_2}$, when $\mathit{X}$ is simple, i.e., $\mathit{X}$ has no repeated singular values, and 0 otherwise. In (51) and (52), $\mathbb{I}\left(·\right)$ denotes the indicator function and

$$\begin{array}{cc}\hfill A_j& =\left\{\begin{array}{cc}|N_1-N_2|{\left(1-\frac{a}{{\mathsf{\sigma }}_i}\right)}_{+}\hfill & \mathrm{if}\mathit{X}\in {\mathbb{R}}^{N_1\times N_2}\hfill \\ \left(2|N_1-N_2|+1\right){\left(1-\frac{a}{{\mathsf{\sigma }}_i}\right)}_{+}\hfill & \mathrm{if}\mathit{X}\in {\mathbb{C}}^{N_1\times N_2}\hfill \end{array}\right.\hfill \\ \hfill B& =\sum _{i\ne j,i,j=1}^{n_{\min }}\frac{{\mathsf{\sigma }}_i{\left({\mathsf{\sigma }}_i-a\right)}_{+}}{{\mathsf{\sigma }}_i^2-{\mathsf{\sigma }}_j^2}.\hfill \end{array}$$

For more details on the derivation of $\left\{\mathsf{\mu },{\mathsf{\alpha }}_i,c_i\right\}$, the reader is referred to [57]. The next question of course is how to choose the regularization parameter $\mathsf{\lambda }$ and thus which shrinkage $a=\mathsf{\mu }\mathsf{\lambda }$ should be applied. The usage of a ST operator causes the singular values of the optimal solution ${\mathit{L}}_{\mathrm{opt}}$ generated by the turbo framework (45) to possess an offset of a for every singular value compared with the true solution $\tilde{\mathit{L}}$. Too much shrinkage results in a large bias, while too little results in a slow convergence rate. As such, an optimal constant $\mathsf{\lambda }$ does not exists, rather $\mathsf{\lambda }$ would need to be adjusted in every iteration. Unfortunately, to the best of the author’s knowledge, a closed form solution to determine such a $\mathsf{\lambda }$ is known to exist only for the signal model $\mathit{X}=\tilde{\mathit{L}}+\mathit{E}$ with the entries of $\mathit{E}$ being i. i. d. normally distributed [67]. However, this does not apply here, since the gradient update step as input to ${\mathcal{S}}_{\mathit{L},a}\left(·\right)$ is not of such form. We therefore follow another approach and set the regularization parameter $\mathsf{\lambda }$ in a similar manner to (33) as

$$\mathsf{\lambda }=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$4$}\right.c_{\mathrm{r}}{\mathsf{\Phi }}^{-1}\left(1-\frac{{\mathsf{\alpha }}_{\mathrm{r}}}{2n}\right)\sqrt{P_{\mathrm{n}}max\left(N_1,N_2\right)},$$

(53)

where $c_{\mathrm{r}}>1$ is some constant, $\mathsf{\Phi }$ is the CDF of $\mathcal{N}\left(0,1\right)$, and ${\mathsf{\alpha }}_r\in \left[0,1\right]$ is some parameter. The additional factor of $\sqrt{max\left(N_1,N_2\right)}$ adjusts the threshold from (33) for singular values. For proofs of convergence, unique solutions, etc., of TARM and thus TSVT, the interested reader may have a look into [57].

Putting all the above steps together, the final TSVT algorithm is listed in Algorithm 3. It is aborted if an upper limit of iterations I is reached or if the relative change of the intermediate solutions

$$d_i={∥\mathit{L}{}_i-\mathit{L}{}_{i-1}∥}_{\mathrm{F}}/{∥\mathit{L}{}_{i-1}∥}_{\mathrm{F}}$$

(54)

is below a predefined threshold $\mathsf{ϵ}$. The threshold selection given in Algorithm 3 was evaluated numerically and proven to work well in every case.

In the following, simulation results are shown to illustrate the performance of TSVT. For all the following ARM algorithms presented, the SNR is defined as

$$SNR={∥\mathcal{A}\left(\tilde{\mathit{L}}\right)∥}_2^2/{∥\mathit{n}∥}_2^2$$

(55)

and the recovery success is measured as SRE

$$SRE={∥\tilde{\mathit{L}}-\widehat{\mathit{L}}∥}_{\mathrm{F}}^2/{∥\tilde{\mathit{L}}∥}_{\mathrm{F}}^2.$$

(56)

The reconstruction performance is evaluated by use of phase transition plots, where for a given type of sensing operator $\mathcal{A}$ (random or DFT) a number of low-rank matrices $\tilde{\mathit{L}}$ are to be reconstructed for different measurement ratios $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$n$}\right.\in (0,1]$ and degree of freedom ratios $\raisebox{1ex}{$d_{\rho }$}\!\left/ \!\raisebox{-1ex}{$m$}\right.\in (0,1]$, where $d_{\rho }=\rho \left(N_1+N_2-\rho \right)$ denotes the number of degrees of freedom in a rank-$\rho$ matrix. The low-rank matrices $\tilde{\mathit{L}}$ are set up by first determining $\rho$ and m from the given ratios. Here, it should be mentioned that since the rank $\rho$ has to be a whole number, it is not possible to clearly find a suitable integer $\rho$ for every possible $d_{\rho }$. Instead, $\rho$ is determined by the closest integer to give the desired $d_{\rho }$. Obviously, this only approximates the desired $d_{\rho }$, where the approximation becomes better for larger dimensions $N_1\times N_2$. This, however, results in very high computation time. With this approach, the resulting $\rho$ may also result to zero, especially for small $d_{\rho }$. In this case, no simulations were conducted, which are indicated by blank white entries in the following phase transition plots. A true low-rank matrix is set up as $\tilde{\mathit{L}}={\mathit{X}}_r{\mathit{X}}_l^{\mathrm{H}}$, where ${\mathit{X}}_r\in {\mathbb{C}}^{N_1\times \rho }$ and ${\mathit{X}}_l\in {\mathbb{C}}^{N_2\times \rho }$ are two orthonormalized matrices with elements drawn from an i. i. d. complex Gaussian distribution. Next, the sensing operators $\mathcal{A}$ are set up, where for random sensing operators $\mathcal{A}\sim \mathcal{CN}\left(0,1\right)$ or for DFT operators $\mathcal{A}$ is set up from m randomly selected rows of an $N_1{N}_2\times N_1{N}_2$ DFT matrix following the identity $\mathit{y}=\mathcal{A}\left(\mathit{L}\right)=\mathit{A}vec\left(\mathit{L}\right)$. Finally, the elements of the noise vector n are drawn from an i. i. d. standard complex Gaussian distribution and scaled the given SNR according to (55). For every combination of measurement and rank ratio, 100 Monte Carlo runs were conducted and averaged to generate the phase transition plot. For the simulation results shown in the following $N_1=80$, $N_2=24$, $SNR=40\mathrm{dB}$, and $I=300$ is used.

Algorithm 3 The TSVT algorithm.

Input: $\mathcal{A}$, y, λ, I Initialization:   1:       $\mathsf{ϵ}\leftarrow min\left({10}^{-4},5·{10}^{-3}\mathsf{\lambda }\right)$, $i\leftarrow 0$, $d\leftarrow \infty$, ${\mathit{L}}_0\leftarrow \mathbf{0}$   2:       $\mathsf{\mu }\leftarrow (0.99/{∥\mathcal{A}∥}_2^2,n/(m{∥\mathcal{A}∥}_2^2))$ Body:   1: while $d>\mathsf{ϵ}$ and $i<I$ do   2:       $i\leftarrow i+1$   3:       ${\mathit{R}}_i\leftarrow {\mathit{L}}_{i-1}-\mathsf{\mu }{\nabla }_{\mathit{L}}h\left({\mathit{L}}_{i-1}\right)$   4:       ${\mathit{Z}}_i\leftarrow {\mathcal{S}}_{\mathrm{l},\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{R}}_i\right)$   5:       ${\mathsf{\alpha }}_i\leftarrow \left\{\begin{array}{cc}\frac{1}{n}\mathrm{div}\left({\mathcal{S}}_{l,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{R}}_i\right)\right)\hfill & \mathrm{if}\mathit{X}\in {\mathbb{R}}^{N_1\times N_2}\hfill \\ \frac{1}{2n}\mathrm{div}\left({\mathcal{S}}_{l,\mathsf{\mu }\mathsf{\lambda }}\left({\mathit{R}}_i\right)\right)& \mathrm{if}\mathit{X}\in {\mathbb{C}}^{N_1\times N_2}\end{array}\right.$   6:       $c_i\leftarrow {〈{\mathit{Z}}_i-{\mathsf{\alpha }}_i{\mathit{R}}_i,{\mathit{R}}_i〉}_{\mathrm{F}}/{∥{\mathit{Z}}_i-{\mathsf{\alpha }}_i{\mathit{R}}_i∥}_{\mathrm{F}}^2$   7:       ${\mathit{L}}_i\leftarrow c_i\left({\mathit{Z}}_i-{\mathsf{\alpha }}_i{\mathit{R}}_i\right)$   8:       $d\leftarrow {∥{\mathit{L}}_i-{\mathit{L}}_{i-1}∥}_{\mathrm{F}}/{∥{\mathit{L}}_{i-1}∥}_{\mathrm{F}}$   9: end while Output: $\widehat{\mathit{L}}\leftarrow {\mathit{R}}_i$

As comparison benchmarks, the aforementioned simulations are also conducted for SVT, SVP, and TARM. The SVT algorithm is implemented according to [31], and its regularization parameter $\mathsf{\lambda }$ is set to (53). The SVP and TARM algorithms are likewise implemented according to [48,57] and equipped once with the true rank $\rho$ and once with $2\rho$. Both algorithms are aborted after a maximum number of $I=300$ iterations or if the relative improvement $d_{\mathrm{i}}$ as defined in (54) is below a threshold similar to the TSVT algorithm.

The phase transition diagrams of the TSVT algorithm for random and DFT sensing operators are shown in Figure 9a,b. Its reconstruction performance is higher than SVT and SVP as shown in Figure 9c–f. The reconstruction performance of TARM in the case of the correctly chosen parameter $R=\rho$ is comparable to TSVT, as can be seen in Figure 9g,h. Interestingly, the DFT sensing operator reveals unfavorable properties for TARM in the upper right part of the phase transition plot shown in Figure 9h. This effect is also visible for TSVT as shown in Figure 9b, though less pronounced. The reason therefore is the rather structured nature of the DFT sensing operator, especially in the case of only a few randomly removed rows for a high measurement ratio $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$n$}\right.$. In [56], a similar effect is shown, but for CS and for correlated sparse signals. To fix this issue, the sensing operator was “improved” in [56] by adding more “randomness” to the DFT transformation by applying random signs to the columns of the glsdft matrix. In [57], such an augmented DFT matrix was also directly chosen. For radar, however, this approach is usually not possible, since the Fourier transform is inherently contained in the signal model. The reconstruction performance of TARM in the case of wrongly chosen parameter $R=2\rho$ is shown in Figure 10. As can be seen, it depends heavily on R. The same holds for the convergence speed shown in Figure 11, which shows the intermediate SREs. The convergence speed of SVT is inferior and merely servers as a comparison benchmark. The convergence speed of SVP and TARM depend on the parameter R, where TARM outperforms SVP in any case. The convergence speed of TARM in the case of $R=\rho$ outperforms the TSVT algorithm, although it drops significantly in the case of $R=2\rho$. Compared with SVP, TSVT shows a faster or comparable convergence speed, depending on the sensing operator. For DFT sensing operators, the TSVT algorithm shows slower convergence speed than TARM and SVP. This is again due to the structured nature of the DFT sensing operator. While TARM and SVP can leverage the knowledge of $R=\rho$ to achieve fast convergence, TSVT requires more iterations. However, TARM and SVP are very sensitive to the selected value of R as illustrated by the dashed lines in Figure 11. A comparison of the computation time is shown in Figure 12 for a region where all algorithms perform equally well, except for SVT. The system specifications are the same as given in Section 2. The fastest algorithm is TARM provided $R=\rho$ is used. The second fastest is TSVT, followed by SVT and SVP. The computation time of TARM and SVP obviously increase in the case of $R>\rho$, rendering TSVT a good choice in the case $\rho$ is unknown. Finally, the SRE for various SNRs is shown in Figure 13 for all tested algorithms. As can be seen, the best SREs offer TARM and SVP provided $R=\rho$. Interestingly, the higher the available SNR, the worse is the achievable SRE in the case of $R=2\rho$. The TSVT algorithm follows the SRE of SVP and NIHT with an average loss of $2\mathrm{dB}$. The SVT algorithm provides a comparable SRE only for $SNR=0\mathrm{dB}$. In summary, TSVT is very easy to implement, shows a state-of-the-art convergence rate, has low computational complexity as there are closed form solutions available for all required parameters, and finally it does not require any unknown parameters, e.g., as the true rank $\rho$ of $\tilde{\mathit{L}}$, in general. Hence, it is well suited for practical applications.

The reconstruction performance of the nuclear norm relaxed approaches can be further improved by use of smoothed rank techniques. A suitable and convenient version thereof is presented in the following section.

3.2. Complex Smoothed Rank Approximation

Although the performance of TSVT is already good, it can be improved in a similar manner as CSCSA does for TST results. The CSRA algorithm does so by enforcing a stricter rank measure than the nuclear norm used in (20). The CSRA algorithm is our extension based on the smoothed rank function (SRF) algorithm [61] and the SRA approach in [64]. CSRA is applicable to complex valued problems and in contrast to [64], has a closed form solution to a subsequent optimization problem, hence, reduced computational complexity. In the following, an overview of CSRA is given, while additional details can be found in the Appendix B.

To enforce a stricter rank measure, a different replacement for the rank function is proposed. The rank of $\mathit{L}={\mathit{U}\mathsf{\Sigma }\mathit{V}}^{\mathrm{H}}$, where $\mathsf{\Sigma }=diag\left(\mathsf{\sigma }\left(\mathit{L}\right)\right)$, is defined as the number of nonzero elements in $\mathsf{\sigma }$, where the vector $\mathsf{\sigma }\left(\mathit{L}\right)={\left[\begin{array}{ccc}{\mathsf{\sigma }}_1& \cdots & {\mathsf{\sigma }}_{n_{\min }}\end{array}\right]}^{\mathrm{T}}$ holds the singular values of L. The rank of L can thus be defined as

$$rank\left(\mathit{L}\right)=\sum _{i=1}^{n_{\min }}\left[1-\mathsf{\delta }\left({\mathsf{\sigma }}_i\left(\mathit{L}\right)\right)\right],$$

(57)

where ${\mathsf{\sigma }}_i\left(\mathit{L}\right)$ is the $i\mathrm{th}$ largest singular value of L and $\mathsf{\delta }\left(x\right)$ is the Kronecker delta function. Similar to CSCSA, (57) can be relaxed as

$$f_{\mathsf{\gamma }}\left(x\right)=1-\mathsf{\delta }\left(x\right)\approx 1-exp\left(-\frac{\left|x\right|}{\mathsf{\gamma }}\right).$$

(58)

As can be seen, $\mathsf{\gamma }$ determines how close the rank function is approximated. Thus, an approximation to the rank function can be define as

$$rank\left(\mathit{L}\right)\approx \sum _{i=1}^{n_{\min }}{f}_{\mathsf{\gamma }}\left({\mathsf{\sigma }}_i\left(\mathit{L}\right)\right)=F_{\mathsf{\gamma }}^{\prime }\left(\mathsf{\sigma }\left(\mathit{L}\right)\right)=F_{\mathsf{\gamma }}\left(\mathit{L}\right).$$

(59)

The optimization problem (2) may now be relaxed to

$$\underset{\mathit{L}}{min}{F}_{\mathsf{\gamma }}\left(\mathit{L}\right)\mathrm{subject}\mathrm{to}h\left(\mathit{L}\right)\le \mathsf{ϵ}{}^2,$$

(60)

where the data fidelity term $h\left(\mathit{L}\right)$ was defined in (5). The constrained optimization problem (60) can be converted to an unconstrained one by use of regularization, which yields

$$\underset{\mathit{L}}{min}{\mathsf{\lambda }}_{\mathsf{\gamma }}{F}_{\mathsf{\gamma }}\left(\mathit{L}\right)+h\left(\mathit{L}\right),$$

(61)

where $\mathsf{\lambda }$ again is some regularization parameter. The minimization in (61) constitutes an alternative to the original ${\ell }_1$ problem given by (20). In this approach, $F_{\mathsf{\gamma }}\left(\mathit{L}\right)$ is not concave nor convex (since $f_{\mathsf{\gamma }}\left(x\right)$ is defined also for negative numbers due to evidential requirements) and not smooth, i.e., not differentiable at the origin. In order to avoid getting stuck in local minimum, the GNC approach is again applied similar to CSCSA. At first an initial solution $\mathit{L}{}_0$ is obtained from a convex optimization algorithm such as TSVT and $\mathsf{\gamma }$ is chosen big enough such that (62) does not get stuck in a local minimum. After convergence, $\mathsf{\gamma }$ is subsequently reduced until a stopping criterion is met. For a fixed $\mathsf{\gamma }$, the optimization Problem (61) can be solved by the IT method, by iteratively solving

$$\begin{array}{cc}\hfill {\mathit{L}}_{0i}& =\mathit{L}{}_i-\mathsf{\mu }{\nabla }_{\mathit{L}}h\left(\mathit{L}{}_i\right)\hfill \\ \hfill {\mathit{L}}_{i+1}& ={\mathit{U}}_{0i}diag\left({\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}\left({\mathsf{\sigma }}_{0i}\right)\right){\mathit{V}}_{0i}^{\mathrm{H}}\hfill \end{array}$$

(62)

where ${\mathit{L}}_{0i}={\mathit{U}}_{0i}diag\left({\mathsf{\sigma }}_{0i}\right){\mathit{V}}_{0i}^{\mathrm{H}}$ is the SVD of the output of the gradient update step ${\mathit{L}}_{0i}$ and ${\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}$ is again a thresholding operator, which is given by (A25). In contrast to [64], a difference of convex (D.C.) optimization strategy to solve (61) is not needed; rather, the closed form solution ${\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}\left(·\right)$ is available. The regularization parameter is set to ${\mathsf{\lambda }}_{\mathsf{\gamma }}=16\mathsf{\gamma }\mathsf{\lambda }$, where (53) is used for $\mathsf{\lambda }$. This value was found from extensive numerical simulations to nicely balance the data fidelity error $h\left(\mathit{L}\right)$ and the approximated rank function $F_{\mathsf{\gamma }}\left(\mathit{L}\right)$ and perform well in any case. Furthermore, the initial value ${\mathsf{\gamma }}_0$ is set to ${\mathsf{\gamma }}_0={∥{\mathit{L}}_0∥}_2/10$, where ${\mathit{L}}_0$ denotes the convex relaxed initial solution. More details regarding the special selection of ${\mathsf{\gamma }}_0$ are given in the Appendix B. Finally, the convergence rate of the CSRA algorithm is accelerated using a FISTA-like technique [53]. Putting all the above steps together, the final CSRA algorithm is listed in Algorithm 4. The algorithm consists of two loops, an inner and an outer one. In the outer loop $\mathsf{\gamma }$ is decreased gradually according to the GNC technique. The inner loop solves (62) by using a FISTA-like technique. The loops are aborted after a maximum number of iterations J and P or if the solution changes, measured by the relative distance between consecutive solutions, drop below certain thresholds ${\mathsf{ϵ}}_{\mathrm{o}}$ and ${\mathsf{ϵ}}_{\mathrm{i}}$. The threshold selections given in Algorithm 4 were found numerically as in [63] and were proven to work well in every case.

Simulation results for random and DFT sensing operators are shown in Figure 14. The decreasing factor for $\mathsf{\gamma }$ was set to $c=0.5$. As can be seen, CSRA improves upon the convex results shown in Figure 9. If SVT was used as initialization algorithm for CSRA, the additional gain is dramatic for the random as well as the DFT sensing operator. If TSVT was used as an initialization algorithm, the improvements are not as high, because TSVT already achieves high reconstruction performance especially for random sensing operators. Nevertheless, especially for DFT sensing operators, CSRA helps to boost the reconstruction performance.

A comparison of the reconstruction performance of TSVT + CSRA to TARM is shown in Figure 15, in which the curves indicate a 50% success rate with respect to the Monte Carlo runs. Success is defined twofold as either $SRE\le -SNR$ for a strict success definition and $SRE\le -\left(SNR-5\mathrm{dB}\right)$ for a less strict success definition. In addition, the $25\%$ and $75\%$ success rate confidence intervals are indicated as shaded areas. As can be seen, the combination of TSVT + CSRA follows and even outperforms TARM despite its non-awareness of the true rank $\rho$.

Algorithm 4 The CSRA algorithm.

Input: $\mathcal{A}$, y, λ, J, P Initialization:   1:       $c\leftarrow \left(0,0.5\right)$, $\mathsf{\mu }\leftarrow 0.99/{∥\mathcal{A}∥}_2^2$   2:       ${\mathsf{ϵ}}_{\mathrm{o}}\leftarrow min\left({10}^{-4},5·{10}^{-3}\mathsf{\lambda }\right)$   3:       ${\mathsf{ϵ}}_{\mathrm{i}}\leftarrow min\left({10}^{-3},5·{10}^{-3}\mathsf{\lambda }\right)$   4:       $\widehat{\mathit{L}}\leftarrow \mathrm{TSVT}\left(\mathcal{A},\mathit{y},\mathsf{\lambda },{\mathsf{ϵ}}_{\mathrm{i}},J\right)$, $\mathsf{\gamma }\leftarrow {∥\widehat{\mathit{L}}∥}_2/10$ Body:   1: $p\leftarrow 0$, $d_{\mathrm{o}}\leftarrow \infty$   2: while $d_{\mathrm{o}}>{\mathsf{ϵ}}_{\mathrm{o}}$ and $p<P$ do   3:       $p\leftarrow p+1$, $j\leftarrow 0$, $d_{\mathrm{i}}\leftarrow \infty$   4:       $t_1\leftarrow 1$, ${\mathit{Z}}_1\leftarrow \widehat{\mathit{L}}$, ${\mathit{L}}_0\leftarrow \widehat{\mathit{L}}$, ${\mathsf{\lambda }}_{\mathsf{\gamma }}\leftarrow 16\mathsf{\lambda }\mathsf{\gamma }$   5:       while $d_{\mathrm{i}}>{\mathsf{ϵ}}_{\mathrm{i}}$ and $j<J$ do   6:             $j\leftarrow j+1$   7:             ${\mathit{L}}_j\leftarrow {\mathcal{T}}_{\mathsf{\mu }{\mathsf{\lambda }}_{\mathsf{\gamma }}}^{\left(\mathsf{\gamma }\right)}\left({\mathit{Z}}_j-\mathsf{\mu }{\mathcal{A}}^{\mathrm{H}}\left(\mathcal{A}\left({\mathit{Z}}_j\right)-\mathit{y}\right)\right)$   8:             $t_{j+1}\leftarrow (1+\sqrt{1+4t_j^2})/2$   9:             ${\mathit{Z}}_{j+1}\leftarrow {\mathit{L}}_j+\left(t_j-1\right)\left({\mathit{L}}_j-{\mathit{L}}_{j-1}\right)/t_{j+1}$   10:           $d_{\mathrm{i}}\leftarrow {∥{\mathit{L}}_j-{\mathit{L}}_{j-1}∥}_{\mathrm{F}}/{∥{\mathit{L}}_{j-1}∥}_{\mathrm{F}}$   11:       end while   12:       $d_{\mathrm{o}}\leftarrow {∥{\mathit{L}}_j-\widehat{\mathit{L}}∥}_{\mathrm{F}}/{∥\widehat{\mathit{L}}∥}_{\mathrm{F}}$   13:       $\widehat{\mathit{L}}\leftarrow {\mathit{L}}_j$   14:       $\mathsf{\gamma }\leftarrow c\mathsf{\gamma }$   15: end while Output: $\widehat{\mathit{L}}$

In summary, CSRA in combination with TSVT is very easy to implement, shows a similar convergence rate and reconstruction performance as TARM equipped with the true rank of the unknown low-rank matrix in estimation, and has low computational complexity, as there are closed form solutions available for all required parameters and subsequent optimization problems. Finally, both do not generally require any unknown parameters. Hence, they are well suited for practical applications.

4. Compressed Robust Principle Component Analysis

Turbo Compressed Robust Principle Component Analysis

For TCRPCA, all of the aforementioned reconstruction algorithms for CS and ARM problems are combined together. Following the GNC approach described in Section 2.2, the corresponding relaxed convex Problem (21) is solved via a combination of TST and TSVT by iteratively updating

$$\begin{array}{ccc}\hfill {\mathit{R}}_{S,i}& =& {\mathit{S}}_{i-1}-{\mathsf{\mu }}_S{\nabla }_{\mathit{S}}h\left({\mathit{S}}_{i-1},{\mathit{L}}_{i-1}\right)\hfill \\ \hfill {\mathit{Z}}_{S,i}& =& {\mathcal{S}}_{S,{\mathsf{\mu }}_S{\mathsf{\lambda }}_{S0}}\left({\mathit{R}}_{S,i}\right)\hfill \\ \hfill {\mathit{S}}_i& =& {\mathcal{Q}}_{{\mathsf{\kappa }}_S}\left(c_{S,i}\left({\mathit{Z}}_{S,i}-{\mathsf{\alpha }}_{S,i}{\mathit{R}}_{S,i}\right)\right)\hfill \\ \\ \hfill {\mathit{R}}_{l,i}& =& {\mathit{L}}_{i-1}-{\mathsf{\mu }}_l{\nabla }_{\mathit{L}}h\left({\mathit{S}}_{i-1},{\mathit{L}}_{i-1}\right)\hfill \\ \hfill {\mathit{Z}}_{l,i}& =& {\mathcal{S}}_{l,{\mathsf{\mu }}_l{\mathsf{\lambda }}_l}\left({\mathit{R}}_{l,i}\right)\hfill \\ \hfill {\mathit{L}}_i& =& {\mathcal{P}}_{{\mathsf{\phi }}_l}\left(c_{l,i}\left({\mathit{Z}}_{l,i}-{\mathsf{\alpha }}_{l,i}{\mathbf{R}}_{l,i}\right)\right).\hfill \end{array}$$

The required parameters ${\mathsf{\mu }}_S$, $c_{S,i}$, and ${\mathsf{\alpha }}_{S,i}$ are determined as for the TST algorithm explained in Section 2.1 and ${\mathsf{\mu }}_l$, ${\mathsf{\lambda }}_l$, $c_{l,i}$, and ${\mathsf{\alpha }}_{l,i}$ as for the TSVT algorithm explained in Section 3.1. The regularization parameter ${\mathsf{\lambda }}_{S0}$ is chosen as defined in (33). The combination of TST and TSVT is inspired by the turbo algorithms presented in [58,59]. In contrast to those works, only ST operators are used in the TCRPCA algorithm, whose parameters are not learned or determined via excessive grid search. Furthermore, in order to support the incoherence condition of the sparse and low-rank matrices $\mathit{S}$ and $\mathit{L}$, the sparsity ratio operator ${\mathcal{Q}}_{{\mathsf{\kappa }}_S}:{\mathbb{C}}^{N_1\times N_2}\to {\mathbb{C}}^{N_1\times N_2}$ and infinity norm operator ${\mathcal{P}}_{{\mathsf{\phi }}_l}:{\mathbb{C}}^{N_1\times N_2}\to {\mathbb{C}}^{N_1\times N_2}$ may be applied [49]. The usage of ${\mathcal{Q}}_{{\mathsf{\kappa }}_{\mathrm{s}}}$ prevents clustering of sparse entries and is defined element-wise as

$${\left[{\mathcal{Q}}_{{\mathsf{\kappa }}_{\mathrm{s}}}\left(\mathit{S}\right)\right]}_{ij}=\left\{\begin{array}{cc}{S}_{ij}\hfill & \mathrm{if}\left|{\mathit{S}}_{ij}\right|\ge {\mathit{S}}_{i:}^{\left[⌊{\mathsf{\kappa }}_{\mathrm{s}}{N}_2⌋\right]}\&\left|{\mathit{S}}_{ij}\right|\ge {\mathit{S}}_{:j}^{\left[⌊{\mathsf{\kappa }}_{\mathrm{s}}{N}_1⌋\right]}\hfill \\ 0\hfill & \mathrm{else},\hfill \end{array}\right.$$

(63)

where ${\mathsf{\kappa }}_{\mathrm{s}}\in \left(0,1\right)$, $S_{ij}$ denotes the $(i,j)$-$\mathrm{th}$ entry of S, ${\mathit{S}}_{i:}$ the i-$\mathrm{th}$ row, ${\mathit{S}}_{:j}$ the j-$\mathrm{th}$ column of S in Matlab notation, and $x^{\left[a\right]}$ the a-$\mathrm{th}$ biggest entry in magnitude in $x$. The usage of ${\mathcal{P}}_{\mathsf{\phi }}$ prevents spikiness in the low-rank reconstruction and is defined element-wise as

$${\left[{\mathcal{P}}_{\mathsf{\phi }}\left(\mathit{L}\right)\right]}_{ij}=\left\{\begin{array}{cc}\mathsf{\phi }exp\left(\mathrm{j}arg\left(l{}_{ij}\right)\right),\hfill & \mathrm{if}\left|l{}_{ij}\right|\ge \mathsf{\phi }\hfill \\ l{}_{ij},\hfill & \mathrm{else}\hfill \end{array}\right..$$

(64)

Unfortunately, the required parameters ${\mathsf{\kappa }}_S$ and ${\mathsf{\phi }}_l$ are unknown in general. For ${\mathsf{\kappa }}_S$, a reasonable guess is required and ${\mathsf{\phi }}_l$ can be determined from

$${\mathsf{\phi }}_l=\frac{\mathsf{\mu }\rho c_{\mathsf{\phi }}{∥\tilde{\mathit{L}}∥}_2}{\sqrt{N_1{N}_2}},$$

(65)

where $c_{\mathsf{\phi }}$ is a parameter set manually usually $<1$ [32,49]. Most parameters in (65) are unknown, since knowledge of the true low-rank matrix $\tilde{\mathit{L}}$ is required. As a rough estimate, we may use ${∥\tilde{\mathit{L}}∥}_2\approx {∥{\mathcal{A}}^{\mathrm{H}}\left(\mathit{y}\right)∥}_2$, $\mathsf{\mu }=1$, and $\rho \approx min\left(N_1,N_2\right)/2$. Certainly, this estimate is not justified to be anywhere close to an optimal value, but it was found from simulations that it is sufficient to apply ${\mathcal{Q}}_{{\mathsf{\kappa }}_S}$ and ${\mathcal{P}}_{{\mathsf{\phi }}_l}$ only for a limited number of iterations, e.g., the first 10 iterations. The intermediate results ${\mathit{S}}_i$ and ${\mathit{L}}_i$ then lie in a surrounding of a diffuse sparse and low-rank solution, and subsequent iterations do not need any further “guidance” by the projection operators. This also circumvents the problem of not knowing the optimal parameters ${\mathsf{\kappa }}_S$ and ${\mathsf{\phi }}_l$.

Once a suitable convex solution was obtained, a refinement is conducted by solving

$$\widehat{\mathit{S}},\widehat{\mathit{L}}=\underset{\mathit{S},\mathit{L}}{arg\; min}{\mathsf{\lambda }}_S{F}_{0,{\mathsf{\gamma }}_S}\left(\mathit{S}\right)+{\mathsf{\lambda }}_l{F}_{\mathrm{r},{\mathsf{\gamma }}_l}\left(\mathit{L}\right)+h(\mathit{S},\mathit{L}),$$

(66)

where $F_{0,{\mathsf{\gamma }}_S}\left(·\right)$ is the ${\ell }_0$-approximation function (40) and $F_{\mathrm{r},{\mathsf{\gamma }}_l}\left(·\right)$ is the rank approximation Function (59). Program (66) is solved via a combination of CSCSA and CSRA by iteratively updating

$$\begin{array}{ccc}\hfill {\mathit{S}}_i& =& {\mathcal{T}}_{0,{\mathsf{\mu }}_S{\mathsf{\lambda }}_{S,{\mathsf{\gamma }}_S}}^{\left({\mathsf{\gamma }}_S\right)}\left({\mathit{Z}}_{S,i}-{\mathsf{\mu }}_S{\nabla }_{\mathit{S}}h\left({\mathit{Z}}_{S,i},{\mathit{Z}}_{l,i}\right)\right)\hfill \\ \hfill {\mathit{L}}_i& =& {\mathcal{T}}_{\mathrm{r},{\mathsf{\mu }}_l{\mathsf{\lambda }}_l}^{\left({\mathsf{\gamma }}_l\right)}\left({\mathit{Z}}_{l,i}-{\mathsf{\mu }}_l{\nabla }_{\mathit{L}}h\left({\mathit{Z}}_{S,i},{\mathit{Z}}_{l,i}\right)\right)\hfill \\ \hfill t_{i+1}& =& \frac{1+\sqrt{1+4t_i^2}}{2}\hfill \\ \hfill {\mathit{Z}}_{S,i+1}& =& \mathit{S}{}_i+\frac{\left(t_i-1\right)}{t_{i+1}}\left(\mathit{S}{}_i-\mathit{S}{}_{i-1}\right)\hfill \\ \hfill {\mathit{Z}}_{l,i+1}& =& \mathit{L}{}_i+\frac{\left(t_i-1\right)}{t_{i+1}}\left(\mathit{L}{}_i-\mathit{L}{}_{i-1}\right),\hfill \end{array}$$

where ${\mathcal{T}}_{0,b}^{\left(a\right)}\left(·\right)$ is the thresholding operator as defined in CSCSA in Section 2.2 and ${\mathcal{T}}_{\mathrm{r},b}^{\left(a\right)}\left(·\right)$ is the thresholding operator as defined in CSRA in Section 3.2. The required parameters ${\mathsf{\lambda }}_S$, ${\mathsf{\gamma }}_S$, and ${\mathsf{\gamma }}_l$ are also the same as defined in CSCSA and CSRA.

Putting all the above steps together, the final TCRPCA algorithm is listed in Algorithm 5, which delivers a solution to program (21), and Algorithm 6 which solves for Program (66).

Algorithm 5 Part 1 of TCRPCA algorithm delivering convex solution.

Input: $\mathcal{A}$, y, λs, λl, κs, φl, I Initialization:   1:       ${\mathsf{ϵ}}_s\leftarrow min\left({10}^{-4},5·{10}^{-3}{\mathsf{\lambda }}_s\right)$   2:       ${\mathsf{ϵ}}_l\leftarrow min\left({10}^{-4},5·{10}^{-3}{\mathsf{\lambda }}_l\right)$   3:       ${\mathit{S}}_0\leftarrow \mathbf{0}$, ${\mathit{L}}_0\leftarrow \mathbf{0}$, ${\mathsf{\mu }}_s\leftarrow n/\left(m{∥\mathcal{A}∥}_2^2\right)$   4:       ${\mathsf{\mu }}_l\leftarrow n/\left(m{∥\mathcal{A}∥}_2^2\right)$, $i\leftarrow 0$, $d_s\leftarrow \infty$, $d_l\leftarrow \infty$ Body:   1: while ($d_s>{\mathsf{ϵ}}_s$ or $d_l>{\mathsf{ϵ}}_l$) and $i<I$ do   2:       $i\leftarrow i+1$   3:       ${\mathit{R}}_{s,i}\leftarrow {\mathit{S}}_{i-1}-{\mathsf{\mu }}_s{\nabla }_{\mathit{S}}h\left({\mathit{S}}_{i-1},{\mathit{L}}_{i-1}\right)$   4:       ${\mathit{Z}}_{s,i}\leftarrow {\mathcal{S}}_{s,{\mathsf{\mu }}_s{\mathsf{\lambda }}_s}\left({\mathit{R}}_{s,i}\right)$   5:       ${\mathsf{\alpha }}_{s,i}\leftarrow$(30)   6:       $c_{s,i}\leftarrow$(31)   7:       ${\mathit{S}}_i\leftarrow {\mathcal{Q}}_{{\mathsf{\kappa }}_s}\left(c_{s,i}\left({\mathit{Z}}_{s,i}-{\mathsf{\alpha }}_{s,i}{\mathit{R}}_{s,i}\right)\right)$   8:       ${\mathit{R}}_{l,i}\leftarrow {\mathit{L}}_{i-1}-{\mathsf{\mu }}_l{\nabla }_{\mathit{L}}h\left({\mathit{S}}_{i-1},{\mathit{L}}_{i-1}\right)$   9:       ${\mathit{Z}}_{l,i}\leftarrow {\mathcal{S}}_{l,{\mathsf{\mu }}_l{\mathsf{\lambda }}_l}\left({\mathit{R}}_{l,i}\right)$   10:      ${\mathsf{\alpha }}_{l,i}\leftarrow$(49)   11:      $c_{l,i}\leftarrow$(50)   12:      ${\mathit{L}}_i\leftarrow {\mathcal{P}}_{{\mathsf{\phi }}_l}\left(c_{l,i}\left({\mathit{Z}}_{l,i}-{\mathsf{\alpha }}_{l,i}{\mathit{R}}_{l,i}\right)\right)$   13:      $d_s\leftarrow {∥{\mathit{S}}_i-{\mathit{S}}_{i-1}∥}_{\mathrm{F}}/{∥{\mathit{S}}_{i-1}∥}_{\mathrm{F}}$   14:      $d_l\leftarrow {∥{\mathit{L}}_i-{\mathit{L}}_{i-1}∥}_{\mathrm{F}}/{∥{\mathit{L}}_{i-1}∥}_{\mathrm{F}}$   15: end while Output: $\widehat{\mathit{S}}\leftarrow {\mathit{R}}_{s,i}$, $\widehat{\mathit{L}}\leftarrow {\mathit{R}}_{l,i}$

In the following, simulation results are shown to illustrate the performance of TCRPCA for a DFT and a noiselet sensing operator instead of a random sensing operator for reasons of computational load. The sparse and low-rank matrices $\tilde{\mathit{S}}$ and $\tilde{\mathit{L}}$ are generated similar as for the aforementioned algorithms in Section 2 and Section 3, however, for a size of $N_1=N_2=128$. The performance evaluation follows an approach similar as in [44]. The low-rank matrices $\tilde{\mathit{L}}$ are set up for fixed ranks $\rho =\{3,5,7,9\}$ and the phase transition plots are evaluated over the sparsity rate $\raisebox{1ex}{$\mathsf{\kappa }$}\!\left/ \!\raisebox{-1ex}{$m$}\right.>0$; hence, the sparse matrices $\tilde{\mathit{S}}$ are accordingly set up. The resulting phase transition plots illustrated in Figure 16 indicate the $50\%$ success rate with respect to the Monte Carlo runs, where 20 runs are conducted. Success is defined as $SRE\le -SNR$, where the SRE of the reconstructed low-rank matrices $\widehat{\mathit{L}}$ is used. These transition plots thus provide information of up to which sparsity rate or “corruption rate” the low-rank matrices can still be reconstructed.

Algorithm 6 Part 2 of TCRPCA algorithm delivering refined solution.

Input: $\mathcal{A},\mathit{y},{\mathsf{\lambda }}_s,{\mathsf{\lambda }}_l,{\widehat{\mathit{S}}}_0,{\widehat{\mathit{L}}}_0,I,J$ Initialization:   1:       $c\leftarrow \left(0,0.5\right)$, ${\mathsf{\mu }}_s\leftarrow 0.99/{∥\mathcal{A}∥}_2^2$, ${\mathsf{\mu }}_l\leftarrow 0.99/{∥\mathcal{A}∥}_2^2$   2:       ${\mathsf{\gamma }}_s\leftarrow max\left(\right|\widehat{\mathit{S}}\left|\right)/10$, ${\mathsf{\gamma }}_l\leftarrow {∥\widehat{\mathit{L}}∥}_2/10$   3:       ${\mathsf{ϵ}}_{s\mathrm{i}}\leftarrow min\left({10}^{-3},5·{10}^{-3}{\mathsf{\lambda }}_s\right)$   4:       ${\mathsf{ϵ}}_{s\mathrm{o}}\leftarrow min\left({10}^{-4},5·{10}^{-3}{\mathsf{\lambda }}_s\right)$   5:       ${\mathsf{ϵ}}_{l\mathrm{i}}\leftarrow min\left({10}^{-3},5·{10}^{-3}{\mathsf{\lambda }}_l\right)$   6:       ${\mathsf{ϵ}}_{l\mathrm{o}}\leftarrow min\left({10}^{-4},5·{10}^{-3}{\mathsf{\lambda }}_l\right)$ Body:   1: $i\leftarrow 0$, $d_{\mathrm{s}\mathrm{o}}\leftarrow \infty$, $d_{\mathrm{l}\mathrm{o}}\leftarrow \infty$   2: while ($d_{s\mathrm{o}}>{\mathsf{ϵ}}_{s\mathrm{o}}$ or $d_{l\mathrm{o}}>{\mathsf{ϵ}}_{l\mathrm{o}}$) and $i<I$ do   3:       $i\leftarrow i+1$, $j\leftarrow 0$, $d_{s\mathrm{i}}\leftarrow \infty$, $d_{l\mathrm{i}}\leftarrow \infty$   4:       $t_1\leftarrow 1$, ${\mathit{Z}}_{s,1}\leftarrow \widehat{\mathit{S}}$, ${\mathit{Z}}_{l,1}\leftarrow \widehat{\mathit{L}}$, ${\mathit{S}}_0\leftarrow \widehat{\mathit{S}}$, ${\mathit{L}}_0\leftarrow \widehat{\mathit{L}}$   5:       ${\mathsf{\lambda }}_{s,{\mathsf{\gamma }}_s}\leftarrow 2{\mathsf{\lambda }}_s{\mathsf{\gamma }}_s$, ${\mathsf{\lambda }}_{l,{\mathsf{\gamma }}_l}\leftarrow 16{\mathsf{\lambda }}_l{\mathsf{\gamma }}_l$   6:       while ($d_{s\mathrm{i}}>{\mathsf{ϵ}}_{s\mathrm{i}}$ or $d_{l\mathrm{i}}>{\mathsf{ϵ}}_{l\mathrm{i}}$) and $j<J$ do   7:             $j\leftarrow j+1$   8:             ${\mathit{S}}_j\leftarrow {\mathcal{T}}_{0,{\mathsf{\mu }}_s{\mathsf{\lambda }}_{s,{\mathsf{\gamma }}_s}}^{\left({\mathsf{\gamma }}_s\right)}\left({\mathit{Z}}_{s,j}-{\mathsf{\mu }}_s{\nabla }_{S}h\left({\mathit{Z}}_{s,j},{\mathit{Z}}_{l,j}\right)\right)$   9:             ${\mathit{L}}_j\leftarrow {\mathcal{T}}_{\mathrm{r},{\mathsf{\mu }}_l{\mathsf{\lambda }}_{l,{\mathsf{\gamma }}_l}}^{\left({\mathsf{\gamma }}_l\right)}\left({\mathit{Z}}_{l,j}-{\mathsf{\mu }}_l{\nabla }_{L}h\left({\mathit{Z}}_{s,j},{\mathit{Z}}_{l,j}\right)\right)$   10:           $t_{j+1}\leftarrow (1+\sqrt{1+4t_j^2})/2$   11:           ${\mathit{Z}}_{s,j+1}\leftarrow \mathit{S}{}_j+\left(t_j-1\right)\left(\mathit{S}{}_j-\mathit{S}{}_{j-1}\right)/t_{j+1}$   12:           ${\mathit{Z}}_{l,j+1}\leftarrow \mathit{L}{}_j+\left(t_j-1\right)\left(\mathit{L}{}_j-\mathit{L}{}_{j-1}\right)/t_{j+1}$   13:           $d_{s\mathrm{i}}\leftarrow {∥{\mathit{S}}_j-{\mathit{S}}_{j-1}∥}_{\mathrm{F}}/{∥{\mathit{S}}_{j-1}∥}_{\mathrm{F}}$   14:           $d_{l\mathrm{i}}\leftarrow {∥{\mathit{L}}_j-{\mathit{L}}_{j-1}∥}_{\mathrm{F}}/{∥{\mathit{L}}_{j-1}∥}_{\mathrm{F}}$   15:       end while   16:       $d_{\mathrm{s}\mathrm{o}}\leftarrow {∥{\mathit{S}}_j-\widehat{\mathit{S}}∥}_{\mathrm{F}}/{∥\widehat{\mathit{S}}∥}_{\mathrm{F}}$   17:       $d_{\mathrm{l}\mathrm{o}}\leftarrow {∥{\mathit{L}}_j-\widehat{\mathit{L}}∥}_{\mathrm{F}}/{∥\widehat{\mathit{L}}∥}_{\mathrm{F}}$   18:       $\widehat{\mathit{S}}\leftarrow {\mathit{S}}_j$, $\widehat{\mathit{L}}\leftarrow {\mathit{L}}_j$   19:       ${\mathsf{\gamma }}_s\leftarrow c{\mathsf{\gamma }}_s$, ${\mathsf{\gamma }}_l\leftarrow c{\mathsf{\gamma }}_l$   20: end while Output: $\widehat{\mathit{S}}$,$\widehat{\mathit{L}}$

As comparison benchmarks, the aforementioned simulations are also conducted for SpaRCS, NFL, and TMP-CRPCA algorithm, which uses HT instead of ST denoisers. Additional comparisons of CRPCA algorithms are given in [68]. The SpaRCS algorithm is equipped with the true number of sparse entries $\mathsf{\kappa }$ and rank $\rho$. Its implementation is taken from its official Github repository (https://github.com/image-science-lab/SpaRCS, accessed on 20 December 2022). The maximum number of iterations is set to $I=300$, and for all remaining parameters the default setting is used. The NFL and TMP-CRPCA algorithms are implemented according to [44,59] and, likewise, equipped with the true number of sparse entries $\mathsf{\kappa }$ and rank $\rho$. For the NFL algorithm, only the initial phase algorithm is used as the gradient descent phase algorithm merely serves as a final fast refinement step. For the initial phase algorithm, the maximum number of iterations of the Dykstra projection step are set to $I=5$, and its corresponding infinity threshold value ${\mathsf{\phi }}_l$, as defined in (65), is chosen similar to the TCRPCA algorithm. The abortion criteria of NFL and TMP-CRPCA are also set as for the TCRPCA algorithm.

As can be seen in Figure 16a,b, the reconstruction performance of TCRPCA gracefully degrades with increasing rank $\rho$. For the generation of these phase transition plots, no sparsity ratio operator ${\mathcal{Q}}_{{\mathsf{\kappa }}_S}$ and infinity norm operator ${\mathcal{P}}_{{\mathsf{\phi }}_l}$ are applied, since these have turned out to be unnecessary for $\mathsf{\kappa }>0$, $\rho >0$, and $SNR\ge 10\mathrm{dB}$. The special cases of $\mathsf{\kappa }=0$ or $\rho =0$ are treated further below. The performance gain of the refinement step achieved by Part 2 of the TCRPCA algorithm appears moderate compared with the gains achieved in the pure CS and ARM applications. The reason therefore is that Part 1 of the TCRPCA algorithm shows a rather sharp transition in SRE, and thus Part 2 of TCRPCA lacks a sufficiently good initial solution to improve upon (not shown here). Nevertheless, the refinement step constitutes a computationally efficient and fast procedure offering increased reconstruction performance. In comparison, TMP-CRPCA and NFL shown in Figure 16c–f offer higher reconstruction performance for lower measurement rates $\raisebox{1ex}{$m$}\!\left/ \!\raisebox{-1ex}{$n$}\right.$, with NFL offering the highest. The results for SpaRCS illustrated in Figure 16g,h show a comparable performance in the case the rank of the low-rank matrix is as low as $\rho =3$. Its performance, however, rapidly degrades with increasing rank $\rho$. The reason therefore is that SpaRCS, in contrast to the remaining algorithms, lacks a sharp phase transition as illustrated in Figure 17. In this simulation, TMP-CRPCA, NFL, and SpaRCS are equipped with the true sparsity and rank parameters of the unknown matrices to reconstruct, namely $K=\mathsf{\kappa }$ and $R=\rho$. In the case the parameters are set to $K=2\mathsf{\kappa }$ and $R=2\rho$, all algorithms completely fail and, as such, are not shown here. A comparison of the convergence speed is shown in Figure 18, which shows the intermediate SREs. In this comparison, only Part 1 of the TCRPCA algorithm is shown. As can be seen, TMP-CRPCA, NFL, and SpaRCS show altogether a superior convergence rate compared with Part 1 of TCRPCA. This is in stark contrast to the pure CS and ARM counterparts shown in Section 2.1 and Section 3.1. Further evaluations revealed that Part 1 of TCRPCA requires $\sim 1/4$ of its iterations to correctly identify $\mathsf{\kappa }$ and $\rho$, and the remaining iterations are used to minimize the reconstruction error. A comparison of the computation time is shown in Figure 19 for a region where all algorithms perform equally well. The system specifications are the same as given in Section 2. The overall fastest algorithm is TMP-CRPCA. In cases where the sensing operator can be implemented in an efficient manner, as is the case for the DFT sensing operator, the Part 1 TCRPCA algorithm offers the slowest computation time. In cases where the sensing operator is computationally more expansive to evaluate, Part 1 TCRPCA becomes more efficient and is faster than SpaRCS. The NFL algorithm is overall faster than in Part 1 TCRPCA. It should be noted that Part 1 TCRPCA can be aborted and Part 2 TCRPCA started sooner in order to reduce its overall computation time. This, however, is not done here for sake of clear evaluation. Finally, the SRE for various SNRs is shown in Figure 20 for all tested algorithms. For Part 1 of TCRPCA, the performance is shown with and without the spikiness operator ${\mathcal{P}}_{\mathsf{\phi }}\left(\mathit{L}\right)$ defined in (65) for $c_{\mathsf{\phi }}=1/4$. As can be seen, for $SNR=0\mathrm{dB}$, the spikiness operator is required to support the reconstruction, especially for higher sparsity ratios $\raisebox{1ex}{$\mathsf{\kappa }$}\!\left/ \!\raisebox{-1ex}{$m$}\right.$. In the case of higher SNRs, no supporting operator is required anymore; in fact, it is obstructive at high SNR. For these simulations, no sparsity ratio operator ${\mathcal{Q}}_{{\mathsf{\kappa }}_S}$ is required. The best SRE offer TMP-CRPCA and NFL provided $K=\mathsf{\kappa }$ and $R=\rho$.

For CRPCA, $\tilde{\mathit{S}}=\mathbf{0}$ and $\tilde{\mathit{L}}=\mathbf{0}$ represent special cases. In order to allow for successful reconstructions in the case of $\tilde{\mathit{S}}=\mathbf{0}$, it is found from simulations that the sparsity ratio operator ${\mathcal{Q}}_{{\mathsf{\kappa }}_S}$ is required for all iterations with a setting of ${\mathsf{\kappa }}_S\le 0.25$ to allow for a successful identification of $\widehat{\mathit{S}}=\mathbf{0}$. Likewise, the infinity norm operator ${\mathcal{P}}_{{\mathsf{\phi }}_l}$ with a setting of $c_{\mathsf{\phi }}\le 0.5$ is found to be required for all iterations to successfully identify the special case of $\widehat{\mathit{L}}=\mathbf{0}$. The use of the sparsity ratio and infinity norm operators incur restrictions regarding the possible reconstruction performance of the TCRPCA algorithm. Obviously, the use of ${\mathcal{Q}}_{{\mathsf{\kappa }}_S}$ with ${\mathsf{\kappa }}_S=0.25$ prohibits successful reconstructions in the case of $\mathsf{\kappa }/min(N_1,N_2)>0.25$. In a similar manner, the use of ${\mathcal{P}}_{{\mathsf{\phi }}_l}$ with $c_{\mathsf{\phi }}=0.5$ prohibits successful reconstructions in the case $\rho /min(N_1,N_2)>0.25$. How to treat the special cases of either $\tilde{\mathit{S}}=\mathbf{0}$ or $\tilde{\mathit{L}}=\mathbf{0}$ in a satisfactory manner, i.e., how to relax ${\mathsf{\kappa }}_s$ and $c_{\mathsf{\phi }}$ conveniently, is an open question and subject to further investigation. The obvious approach of conducting a CS, an ARM, and an CRPCA reconstruction separately and comparing the resulting residual errors to determine if either $\tilde{\mathit{S}}=\mathbf{0}$ or $\tilde{\mathit{L}}=\mathbf{0}$ does not work. Particularly in the case of low $n/m$, the CRPCA approach always yields the lowest residual error regardless if the scene is strictly sparse or low-rank due to its larger degree of freedom.

In summary, TCRPCA offers comparable reconstruction performance to its greedy and HT counterparts despite its unawareness of the true sparsity and rank values. It is very easy to implement and has low computational complexity, as there are closed form solutions available for all required parameters and subsequent optimization problems. In the case of $\tilde{\mathit{S}}\ne \mathbf{0}$ and $\tilde{\mathit{L}}\ne \mathbf{0}$, no generally unknown parameters are required for successful reconstruction. Hence, TCRPCA is well suited for practical applications. In the special cases $\tilde{\mathit{S}}=\mathbf{0}$ or $\tilde{\mathit{L}}=\mathbf{0}$, TCRPCA is capable of a successful reconstruction if $\mathsf{\kappa }/min(N_1,N_2)\le 0.25$ or $\rho /min(N_1,N_2)\le 0.25$, respectively.

5. Conclusions

In this paper, fast, efficient, and viable CS, ARM, and CRPCA algorithms suitable for radar signal processing are proposed. They are designed such that no parameters unknown in practice, e.g., the number of sparse entries or the rank of the unknown low-rank matrix, are required. The only parameter that is needed to be known is the noise power, which in the field of radar signal processing is usually available. For all remaining parameters, either suitable heuristic formulas or closed form solutions are given. The general reconstruction scheme comprises two steps: First, a convex solution is calculated for which the turbo-message-passing framework is utilized. This initial solution is in a second step refined by use of smoothed ${\ell }_0$-refinements. The proposed algorithms for CS are termed TST and CSCSA, for ARM problems TSVT and CSRA, and TCRPCA for the combined CS and ARM problems. All algorithms show state-of-the-art reconstruction performance and are of high computational efficiency, as closed form solutions are available for subsequent optimization tasks.