A Semismooth Newton-Based Augmented Lagrangian Algorithm for the Generalized Convex Nearly Isotonic Regression Problem

Abstract

The generalized convex nearly isotonic regression problem addresses a least squares regression model that incorporates both sparsity and monotonicity constraints on the regression coefficients. In this paper, we introduce an efficient semismooth Newton-based augmented Lagrangian (Ssnal) algorithm to solve this problem. We demonstrate that, under reasonable assumptions, the Ssnal algorithm achieves global convergence and exhibits a linear convergence rate. Computationally, we derive the generalized Jacobian matrix associated with the proximal mapping of the generalized convex nearly isotonic regression regularizer and leverage the second-order sparsity when applying the semismooth Newton method to the subproblems in the Ssnal algorithm. Numerical experiments conducted on both synthetic and real datasets clearly demonstrate that our algorithm significantly outperforms first-order methods in terms of efficiency and robustness.

1. Introduction

Data generated in various fields often exhibit clear monotonicity, as seen in meteorological climate data [1,2], economic demand/supply curves [3], and biological growth curves [4]. Thus, this paper focuses on statistical models under order constraints. Specifically, suppose that we have m observations $({\mathit{a}}_{\mathit{i}},b_i)$ for $i=1,\dots ,m$, where ${\mathit{a}}_{\mathit{i}}=(a_{i1},\dots ,a_{in})$ is a vector with n features and $b_i$ is a response value. We concentrate on addressing the following optimization problem:

$$\underset{z\in {\mathbb{R}}^n}{min}\left\{{\displaystyle \frac{1}{2}}{\parallel Az-b\parallel }^2+\lambda {\parallel z\parallel }_1+\tau \sum _{i=1}^{n-1}{(z_i-z_{i+1})}_{+}\right\},$$

(1)

where $A=({\mathit{a}}_{\mathbf{1}},\dots ,{\mathit{a}}_{\mathit{m}})$ is an $m\times n$ data matrix, $b={(b_1,\dots ,b_m)}^{\top }$ is the response vector, $\lambda ,\tau \ge 0$ are given regularization parameters, and ${\left(z\right)}_{+}=\max (z,0)$. In high-dimensional statistical regression, it is common for the number of features to exceed the number of samples. Therefore, in our paper, we assume that $m\le n$. The penalty term is composed of two components: the first enforces sparsity in the coefficient estimates by incorporating prior knowledge, and the second penalizes violations of monotonicity among adjacent pairs.

Problem (1) is a generalization of a wide range of ordered convex problems, including an isotonic regression model [5], nearly isotonic regression model [6], and ordered lasso problem [7]. The isotonic regression problem involves determining a vector $z\in {\mathbb{R}}^n$ that approximates a given vector $y\in {\mathbb{R}}^n$ while ensuring that z exhibits a non-decreasing (or non-increasing) sequence, i.e.,

$$\underset{z\in {\mathbb{R}}^n}{\min }\left\{\sum _{i=1}^n{(y_i-z_i)}^2|z_1\le z_2\le \dots \le z_n\right\},$$

(2)

where $z={(z_1,z_2,\dots ,z_n)}^{\top }$ and $y={(y_1,y_2,\dots ,y_n)}^{\top }$. Since the restricted monotonicity constraint may lead to a model too rigid and make it difficult to adapt to complex data structures, Tibshirani et al. [6] relaxed this monotonicity constraint and considered the following nearly isotonic regression model:

$$\underset{\mathit{z}\in {\mathbb{R}}^n}{min}\left\{{\displaystyle \frac{1}{2}}\sum _{i=1}^n{(y_i-z_i)}^2+\tau \sum _{i=1}^{n-1}{(z_i-z_{i+1})}_{+}\right\},$$

(3)

where $\tau \ge 0$ is a given parameter. It is evident that problem (1) is a generalization of problem (3), as it addresses general regression issues and incorporates sparsity constraints on the coefficients. We refer to problem (1) as the generalized convex nearly isotonic regression (GCNIR) problem.

In addition, problem (1) can also be regarded as a generalization of the following ordered lasso problem [7]:

$$\begin{array}{cc}\hfill \underset{z\in {\mathbb{R}}^n}{min}& {\displaystyle \frac{1}{2}}{\parallel Az-b\parallel }^2+\lambda {\parallel z\parallel }_1\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& |z_1|\ge |z_2| \ge \cdots \ge |z_n|.\hfill \end{array}$$

(4)

Clearly, problem (4) extends the lasso problem by incorporating a monotonicity constraint on the absolute values of the coefficients. Like problem (2), this approach can lead to an overly rigid model (4). However, the GCNIR problem (1) mitigates the stringent monotonicity requirement of the ordered lasso, transforming it into a convex problem that is more flexible and tractable.

The GCNIR problem (1) can be reformulated into a convex quadratic programming (QP) problem by introducing new variables:

$$\begin{array}{cc}\hfill \underset{s\in {\mathbb{R}}^{4n}}{min}& {\displaystyle \frac{1}{2}}{s}^{\top }Hs+⟨v,s⟩\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& Ws=0,\hfill \\ \hfill & s\ge 0,\hfill \end{array}$$

(5)

where

$$\begin{array}{c}\hfill v=\left[\begin{array}{c}\lambda {\mathbf{1}}_n-A^{\top }b\\ \lambda {\mathbf{1}}_n+A^{\top }b\\ \tau {\mathbf{1}}_n\\ 0_{n\times 1}\end{array}\right],H={\left[\begin{array}{cccc}{A}^{\top }A& -A^{\top }A& 0& 0\\ -A^{\top }A& A^{\top }A& 0& 0\\ 0& 0& 0& 0\\ 0& 0& 0& 0\end{array}\right]}_{4n\times 4n},\end{array}$$

$$\begin{array}{c}\hfill W=\left[\begin{array}{c}S,-S,-I_n,I_n\end{array}\right],S={\left[\begin{array}{ccccc}1& -1& 0& \cdots & 0\\ 0& 1& -1& \cdots & 0\\ ⋮& ⋮& \ddots & \ddots & ⋮\\ 0& 0& \cdots & 1& -1\\ 0& 0& \cdots & 0& 0\end{array}\right]}_{n\times n},\end{array}$$

${\mathbf{1}}_n$ and $I_n$ denote the all-ones column vector and $n\times n$ identity matrix, respectively. This implies that one can utilize the QP function “quadprog” provided by MATLAB or well-developed QP solvers, such as Gurobi and CPLEX [8], to compute reformulation (5) and thus solve problem (1). However, since $A^{\top }A$ has a size of $n\times n$, the computational cost of solving and storing $A^{\top }A$ becomes prohibitive, making it challenging to apply the aforementioned methods to large-scale problems.

Due to the challenges in solving the QP reformulation (5), it is logical to adapt the methods used for the previously discussed problems to address problem (1). The pool adjacent violators algorithm (PAVA) [9] is a cornerstone method for tackling shape-constrained statistical regression problems, as discussed in [10]. Initially developed for the isotonic regression model (2), PAVA has been extended to accommodate the nearly isotonic regression model (3), with adaptations such as the modified PAVA (MPAVA) [6] and the generalized PAVA (GPAVA) [2]. Despite its broad application, there is no theoretical guarantee that PAVA can be modified to tackle convex nonseparable minimization problems. Additionally, other approaches, such as the Generalized Proximal Gradient algorithm [7] and the alternating direction method of multipliers (ADMM) [11], have been proposed for solving the ordered lasso problem (4). To our knowledge, most current techniques for dealing with ordered models rely primarily on first-order information from the associated nonsmooth optimization framework. Consequently, we aim to develop a customized algorithm that utilizes second-order information to address the GCNIR problem more effectively.

This paper aims to develop a semismooth Newton-based augmented Lagrangian (Ssnal) algorithm to address the GCNIR problem (1) from a dual viewpoint. The Ssnal algorithm’s primary benefits include its superior convergence characteristics and reduced computational demands, which are achieved by exploiting second-order sparsity and employing efficient strategies within the semismooth Newton (Ssn) algorithm. Furthermore, the Ssnal algorithm has demonstrated its effectiveness in handling large-scale sparse convex models, as evidenced by its performance in applications such as Lasso [12], group Lasso [13], fused Lasso [14], clustered Lasso [15,16], multi-task Lasso [17], ${\ell }_1$ trend filtering [18], density matrix least squares problems [19], the Dantzig selector [20], and others [21,22,23,24]. Building on these successes, we propose to apply the Ssnal algorithm to solve problem (1).

The primary contributions of this paper are as follows. First, we calculate the proximal mapping related to the GCNIR regularizer and its generalized Jacobian. Second, we utilize the Ssnal algorithm to address the GCNIR problem from a dual perspective. Furthermore, by capitalizing on the low-rank properties and second-order sparsity inherent in the GCNIR problem, we significantly reduce the computational cost associated with the Ssn algorithm when solving the subproblems. Lastly, we perform a numerical analysis comparing our algorithm with first-order methods, including ADMM and the Accelerated Proximal Gradient (APG) method, demonstrating the efficiency and robustness of our approach.

The remaining sections of this paper are organized as follows. Section 2 delves into the analysis of the proximal mapping associated with the GCNIR regularizer and its generalized Jacobian. Section 3 outlines the framework of the Ssnal algorithm and discusses its convergence properties when applied to the dual formulation of the GCNIR problem (1). In Section 4, we evaluate the performance of the Ssnal algorithm through numerical experiments. Finally, we conclude the paper in Section 5.

Notation. For any $z\in {\mathbb{R}}^n$, “$\mathrm{Diag}\left(z\right)$” represents a diagonal matrix with $z_i$ in its i-th diagonal component. “$\left|z\right|$” refers to an absolute vector, where each entry i is $|z_i|$. “$\mathrm{sign}\left(z\right)$” indicates the sign vector, i.e., $\mathrm{sign}\left(z_i\right)=1$ when $z_i>0$, $\mathrm{sign}\left(z_i\right)=-1$ when $z_i<0$, and $\mathrm{sign}\left(z_i\right)=0$ when $z_i$ is equal to zero. Additionally, the notation “$\mathrm{Supp}\left(z\right)$” refers to the support of the element z, specifically the collection of indices for which $z_i$ is not equal to zero. For any positive integer n, $e_1={(1,0,0,\dots ,0)}^{\top }$ and $e_n={(0,0,\dots ,0,1)}^{\top }$ are the $n\times 1$ unit column vectors. $I_n=(e_1,e_2,\dots ,e_n)\in {\mathbb{R}}^{n\times n}$, while ${\mathbf{1}}_n=(1,1,\dots ,1)\in {\mathbb{R}}^n$. $D^{†}$ denotes the Moore–Penrose pseudoinverse of the matrix $D\in {\mathbb{R}}^{m\times n}$. Typically, $h^{*}$ denotes the Fenchel conjugate of a given function h.

2. The Proximal Mapping of the GCNIR Regularizer and Its Generalized Jacobian

In this section, we shall present some results concerning the proximal mapping linked to the GCNIR regularizer along with its generalized Jacobian, which are necessary for later analysis.

Given any scalar $\kappa >0$, for any proper closed convex function $p:{\mathbb{R}}^n\to (-\infty ,\infty ]$, the proximal mapping and Moreau envelope [25] of p is defined by

$$\begin{array}{cc}\hfill {\mathrm{Prox}}_p\left(w\right)& :=\underset{\mathit{z}}{argmin}\left\{{\displaystyle \frac{1}{2}}{\parallel z-w\parallel }^2+p\left(z\right)\right\},\forall w\in {\mathbb{R}}^n,\hfill \\ \hfill M_p^{\kappa }\left(w\right)& :=\underset{\mathit{z}}{min}\left\{{\displaystyle \frac{1}{2\kappa }}{\parallel z-w\parallel }^2+p\left(z\right)\right\},\forall w\in {\mathbb{R}}^n.\hfill \end{array}$$

The Moreau identity [26] holds, i.e.,

$${\mathrm{Prox}}_{\kappa p}\left(w\right)+\kappa {\mathrm{Prox}}_{p*/\kappa }(w/\kappa )=w,\forall w\in {\mathbb{R}}^n.$$

According to [27], $M_p^{\kappa }(·)$ is convex and continuously differentiable, and 

$$\nabla M_p^{\kappa }\left(w\right)=(w-{\mathrm{Prox}}_{\kappa p}\left(w\right))/\kappa ,\forall w\in {\mathbb{R}}^n.$$

Let $\phi$ be the GCNIR regularizer in (1), i.e.,

$$\phi \left(z\right)={\lambda \parallel z\parallel }_1+g\left(z\right),\forall z\in {\mathbb{R}}^n,$$

where $g\left(z\right)=\tau {\sum }_{i=1}^{n-1}{(z_i-z_{i+1})}_{+},\forall z\in {\mathbb{R}}^n$.

Before diving into the proximal mapping associated with the GCNIR regularizer $\phi$, we briefly introduce ${\sum }_{i=1}^{n-1}|z_i-z_{i+1}|$ and relevant results, which are discussed in [14].

Define $q\left(z\right)={\sum }_{i=1}^{n-1}|z_i-z_{i+1}{|=\parallel Bz\parallel }_1$, where B is defined by

$$\begin{array}{c}\hfill B={\left(\begin{array}{ccccc}1& -1& 0& \cdots & 0\\ 0& 1& -1& \cdots & 0\\ ⋮& ⋮& \ddots & \ddots & ⋮\\ 0& 0& \cdots & 1& -1\end{array}\right)}_{(n-1)\times n}.\end{array}$$

The proximal mapping $Z_{\tau }(·)$ with respect to $\tau q$ is given as

$$Z_{\tau }\left(w\right):=\underset{\mathit{z}}{argmin}\left\{{\displaystyle \frac{1}{2}}{\parallel z-w\parallel }^2+\tau q\left(z\right)\right\},\forall w\in {\mathbb{R}}^n.$$

Lemma 1.

(See [14], Lemma 1). For any given $\tau \ge 0$, we have that $Z_{\tau }\left(w\right)=w-B^{\top }{S}_{\tau }\left(Bw\right)$, where

$$S_{\tau }\left(w\right):=\underset{\mathit{s}}{argmin}\left\{{\displaystyle \frac{1}{2}}\parallel B^{\top }{s\parallel }^2-⟨s,w⟩\mid {\parallel s\parallel }_{\infty }\le \tau \right\},\forall w\in {\mathbb{R}}^{n-1}.$$

(6)

On the basis of the above lemma, we can now explicitly calculate ${\mathrm{Prox}}_p(·)$. For later convenience, we define $w_{\tau }=w-\tau (e_1-e_n)/2$.

Proposition 1.

For any given $\lambda ,\tau \ge 0$, it follows that for any $w\in {\mathbb{R}}^n$,

$${\mathrm{Prox}}_{\phi }\left(w\right)={\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1}\left(Z_{\tau /2}\left(w_{\tau }\right)\right)=\mathrm{sign}\left(Z_{\tau /2}\left(w_{\tau }\right)\right)\circ \max \left(\left|Z_{\tau /2}\left(w_{\tau }\right)\right|-\lambda ,0\right).$$

Proof. 

According to the definition of the proximal mapping, it holds that for any $w\in {\mathbb{R}}^n$,

$$\begin{array}{cc}\hfill {\mathrm{Prox}}_{\phi }\left(w\right)& =\underset{\mathit{z}}{argmin}\left\{{\displaystyle \frac{1}{2}}{\parallel z-w\parallel }^2+\phi \left(z\right)\right\}\hfill \\ \hfill & =\underset{\mathit{z}}{argmin}\left\{{\displaystyle \frac{1}{2}}{\parallel z-w\parallel }^2+\lambda {\parallel z\parallel }_1+\tau \sum _{i=1}^{n-1}{(z_i-z_{i+1})}_{+}\right\}\hfill \\ \hfill & =\underset{\mathit{z}}{argmin}\left\{{\displaystyle \frac{1}{2}}{\parallel z-w\parallel }^2+{\lambda \parallel z\parallel }_1+{\displaystyle \frac{\tau }{2}}\sum _{i=1}^{n-1}|z_i-z_{i+1}|+{\displaystyle \frac{\tau }{2}}⟨z,e_1-e_n⟩\right\}\hfill \\ \hfill & =\underset{\mathit{z}}{argmin}\left\{{\displaystyle \frac{1}{2}}{∥z-\left(w-{\displaystyle \frac{\tau }{2}}(e_1-e_n)\right)∥}^2+{\lambda \parallel z\parallel }_1+{\displaystyle \frac{\tau }{2}}\sum _{i=1}^{n-1}|z_i-z_{i+1}|\right\}\hfill \\ \hfill & ={\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1+{\displaystyle \frac{\tau }{2}}q}\left(w-{\displaystyle \frac{\tau }{2}}(e_1-e_n)\right).\hfill \end{array}$$

It follows from ([28], Corollary 4) that for any $w\in {\mathbb{R}}^n$,

$$\begin{array}{cc}\hfill {\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1+{\displaystyle \frac{\tau }{2}}q}\left(w-{\displaystyle \frac{\tau }{2}}(e_1-e_n)\right)& ={\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1}\left(Z_{\tau /2}\left(w_{\tau }\right)\right)\hfill \\ \hfill & =\mathrm{sign}\left(Z_{\tau /2}\left(w_{\tau }\right)\right)\circ \max \left(\left|Z_{\tau /2}\left(w_{\tau }\right)\right|-\lambda ,0\right).\hfill \end{array}$$

This completes the proof.    □

Next, we analyze the generalized Jacobian of ${\mathrm{Prox}}_{\phi }(·)$, which is crucial for leveraging computational efficiency. We begin with presenting some findings concerning the generalized HS-Jacobian for $S_{\tau }(·)$, according to [14,29].

As noted in [14], the generalized HS-Jacobian for $S_{\tau /2}$ at $B{w}_{\tau }$ is given by

$$\mathcal{Q}\left(w_{\tau }\right)=\left\{Q\in {\mathbb{R}}^{(n-1)\times (n-1)}\mid Q={\left({\mathsf{\Sigma }}_{\gamma }B{B}^{\top }{\mathsf{\Sigma }}_{\gamma }\right)}^{†},\gamma \in \mathsf{\Gamma }\left(w_{\tau }\right)\right\},$$

where ${\mathsf{\Sigma }}_{\gamma }=\mathrm{Diag}\left({\sigma }_{\gamma }\right)\in {\mathbb{R}}^{(n-1)\times (n-1)}$ with

$${\left({\sigma }_{\gamma }\right)}_i=\left\{\begin{array}{cc}0\hfill & \mathrm{if}i\in \mathsf{\Gamma },\hfill \\ 1\hfill & \mathrm{otherwise},i=1,2,\dots ,n-1,\hfill \end{array}\right.$$

and $\mathsf{\Gamma }\left(w_{\tau }\right)=\left\{\gamma \subseteq \{1,2,\dots ,n-1\}\mid \mathrm{Supp}\left(\xi \right)\subseteq \gamma \subseteq \mathcal{I}\left(w_{\tau }\right)\right\},$ where $\xi =B\left(Z_{\tau /2}\left(w_{\tau }\right)\right)$ is an optimal Lagrangian multiplier for the constraint ${\parallel s\parallel }_{\infty }\le \tau /2$ and 

$$\mathcal{I}\left(w_{\tau }\right)=\left\{i\subseteq \{1,2,\dots ,n-1\}|{\left|S_{\tau /2}\left(B\left(w_{\tau }\right)\right)\right|}_i=\tau /2\right\}$$

(7)

is an active index set.

The multifunction $\mathcal{H}:{\mathbb{R}}^n\rightrightarrows {\mathbb{R}}^{n\times n}$ is given by

$$\mathcal{H}\left(w_{\tau }\right)=\left\{H\in {\mathbb{R}}^{n\times n}\mid H=I_n-B^{\top }QB,Q\in \mathcal{Q}\left(w_{\tau }\right)\right\}.$$

The subsequent proposition demonstrates that $\mathcal{Q}\left(w_{\tau }\right)$ and $\mathcal{H}\left(w_{\tau }\right)$ may be regarded as the generalized HS-Jacobian for $S_{\tau /2}$ at $B{w}_{\tau }$, $Z_{\tau /2}$ at $w_{\tau }$, respectively.

Proposition 2.

For all $w\in {\mathbb{R}}^n$, there exists a neighborhood $\mathcal{X}$ of w such that for any $x\in \mathcal{X}$,

$$\begin{array}{c}\hfill \left\{\begin{array}{c}\mathsf{\Gamma }\left(x_{\tau }\right)\subseteq \mathsf{\Gamma }\left(w_{\tau }\right),\mathcal{Q}\left(x_{\tau }\right)\subseteq \mathcal{Q}\left(w_{\tau }\right),\mathcal{H}\left(x_{\tau }\right)\subseteq \mathcal{H}\left(w_{\tau }\right),\hfill \\ S_{\tau /2}\left(B{x}_{\tau }\right)=S_{\tau /2}\left(B{w}_{\tau }\right)+QB(x-w),\forall Q\in \mathcal{Q}\left(x_{\tau }\right),\hfill \\ Z_{\tau /2}\left(x_{\tau }\right)=Z_{\tau /2}\left(w_{\tau }\right)+H(x-w),\forall H\in \mathcal{H}\left(x_{\tau }\right),\hfill \end{array}\right.\end{array}$$

where $x_{\tau }=x-\tau (e_1-e_n)/2$.

Proof. 

The results are derived from [14] (Proposition 2) and [29] (Lemma 2.1) with minor revisions.    □

The multifunction $\mathcal{V}:{\mathbb{R}}^n\rightrightarrows {\mathbb{R}}^{n\times n}$ is defined as

$$\begin{array}{c}\hfill \mathcal{V}\left(w\right):=\left\{V\in {\mathbb{R}}^{n\times n}|V=\mathsf{\Theta }H,\begin{array}{cc}\hfill & \mathsf{\Theta }\in {\partial }_B{\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1}\left(Z_{\tau /2}\left(w_{\tau }\right)\right),\hfill \\ \hfill & H\in \mathcal{H}\left(w_{\tau }\right)\hfill \end{array}\right\},\end{array}$$

(8)

where

$$\begin{array}{c}\hfill {\partial }_B{\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1}\left(\eta \right)=\left\{\mathrm{Diag}\left(\omega \right)|\begin{array}{cc}{\omega }_i=0\hfill & |{\eta }_i| < \lambda ,\hfill \\ {\omega }_i\in \{0,1\}\hfill & |{\eta }_i| = \lambda ,\hfill \\ {\omega }_i=1\hfill & |{\eta }_i| >\lambda ,\hfill \end{array}\right\}.\end{array}$$

The mapping $\mathcal{V}\left(w\right)$ essentially acts as the generalized Jacobian for ${\mathrm{Prox}}_{\phi }$ at w, which can be derived using the change-of-variables technique from previous work in [14] (Theorem 2).

Theorem 4.

Let λ and τ be non-negative real numbers, and let w be an element of ${\mathbb{R}}^n$. The set-valued function $\mathcal{V}$ has the following properties: it is compact-valued, nonempty, and upper semicontinuous. For each $V\in \mathcal{V}\left(w\right)$, it can be concluded that the matrix V is symmetric and positive semidefinite. Furthermore, there exists a neighborhood $\mathcal{X}$ of w that for any $x\in \mathcal{X}$,

$${\mathrm{Prox}}_{\phi }\left(x\right)-{\mathrm{Prox}}_{\phi }\left(w\right)-V(x-w)=0,\forall V\in \mathcal{V}\left(x\right).$$

3. A Semismooth Newton-Based Augmented Lagrangian Algorithm

This section introduces the semismooth Newton-based augmented Lagrangian (Ssnal) algorithm, an efficient approach for solving the GCNIR problem in the high-dimension low-sample setting, i.e., the case of $m\ll n$. Directly applying the Ssnal algorithm to the primal problem would require solving a linear system of $n\times n$ dimension, leading to significant computational costs, particularly for large-scale problems. To overcome this, we solve the GCNIR problem from the dual perspective.

We can reformulate the GCNIR problem (1) as

$$\underset{\mathit{z}\in {\mathbb{R}}^n,v\in {\mathbb{R}}^m}{min}\left\{{\displaystyle \frac{1}{2}}{\parallel v\parallel }^2+\phi \left(z\right)|Az-v=b\right\}.$$

(P)

The dual problem of (P) takes the following minimization form:

$$\underset{\mathit{s}\in {\mathbb{R}}^m,\zeta \in {\mathbb{R}}^n}{min}\left\{{\displaystyle \frac{1}{2}}{\parallel s\parallel }^2+⟨b,s⟩+{\phi }^{*}\left(\zeta \right)|A^{\top }s+\zeta =0\right\}.$$

(D)

The Lagrangian function of (D) is

$$l(s,\zeta ;z)={\displaystyle \frac{1}{2}}{\parallel s\parallel }^2+⟨b,s⟩+{\phi }^{*}\left(\zeta \right)-⟨z,A^{\top }s+\zeta ⟩.$$

Additionally, given $\sigma >0$, the augmented Lagrangian function of (D) takes the form:

$${\mathcal{L}}_{\sigma }(s,\zeta ;z)=l(s,\zeta ;z)+{\displaystyle \frac{\sigma }{2}}{\parallel A^{\top }s+\zeta \parallel }^2.$$

3.1. The Framework of the  Ssnal Algorithm

Below is the outline of the framework for an Ssnal algorithm designed to solve problem (D) (Algorithm 1).

Algorithm 1 (Ssnal) A semismooth Newton-based augmented Lagrangian algorithm for (D)

Initialization: ${\sigma }_0>0$, $s^0\in {\mathbb{R}}^m$, ${\zeta }^0\in {\mathbb{R}}^n$, $z^0\in {\mathbb{R}}^n$, $j=0.$ 1: Compute $$s^{j+1}\approx \underset{s\in {\mathbb{R}}^m}{\mathrm{argmin}}\left\{{\mathsf{\Phi }}_j\left(s\right)\right\}.$$ (9) 2: Compute ${\zeta }^{j+1}=\left(z^j-{\sigma }_j{A}^{\top }{s}^{j+1}-{\mathrm{Prox}}_{{\sigma }_j\phi }\left(z^j-{\sigma }_j{A}^{\top }{s}^{j+1}\right)\right)/{\sigma }_j.$ 3: Update $z^{j+1}=z^j-{\sigma }_j(A^{\top }{s}^{j+1}+{\zeta }^{j+1}).$ 4: Update ${\sigma }_{j+1}\uparrow {\sigma }_{\infty }\le \infty$, set $j\leftarrow j+1$, and go to Step 1.

The stopping criteria, which have been studied in [30,31] for approximately solving the solution to (9), can be stated as:

$$\begin{array}{cc}\hfill \parallel \nabla {\mathsf{\Phi }}_j\left(s^{j+1}\right)\parallel & \le {ϵ}_j/\sqrt{{\sigma }_j},\sum _{j=0}^{\infty }{ϵ}_j<\infty ,{ϵ}_j>0,\hfill \end{array}$$

(C1)

$$\begin{array}{cc}\hfill \parallel \nabla {\mathsf{\Phi }}_j\left(s^{j+1}\right)\parallel & \le {\rho }_j\sqrt{{\sigma }_j}\parallel A^{\top }{s}^{j+1}+{\zeta }^{j+1}\parallel ,\sum _{j=0}^{\infty }{\rho }_j<\infty ,{\rho }_j>0,\hfill \end{array}$$

(C2)

$$\begin{array}{cc}\hfill \parallel \nabla {\mathsf{\Phi }}_j\left(s^{j+1}\right)\parallel & \le {\rho }_j^{\prime }\parallel A^{\top }{s}^{j+1}+{\zeta }^{j+1}\parallel ,0\le {\rho }_j^{\prime }\to 0.\hfill \end{array}$$

(C3)

Next, we present the convergence results of the Ssnal algorithm, addressing both global and local convergence. Since problem (P) is feasible, ref. [30] (Theorem 4) establishes that satisfying the stopping criterion (C1) guarantees the global convergence of the Ssnal algorithm for problem (D).

Theorem 5.

(Global convergence.) The infinite sequence $(s^j,{\zeta }^j;z^j)$ generated by the Algorithm 1, in accordance with the stopping criteria specified in (C1), ensures that the sequence $z^j$ converges to an optimal solution of problem (P) and the sequence $(s^j,{\zeta }^j)$ converges to the unique optimal solution of problem (D).

Let the objective function of problem (P) be denoted by $f\left(z\right):={\displaystyle \frac{1}{2}}{\parallel Az-b\parallel }^2+\phi \left(z\right)$. It is clear that $T_f\left(z\right):=\partial f\left(z\right)$ and $T_l(s,\zeta ;z):=\{(s^{\prime },{\zeta }^{\prime };z^{\prime })\mid (s^{\prime },{\zeta }^{\prime };-z^{\prime })\in \partial l(s,\zeta ;z)\}$ are piecewise polyhedral multifunctions as described in [32]. This implies that both ${\mathcal{T}}_f$ and ${\mathcal{T}}_l$ satisfy error bound conditions at the original point with positive moduli $c_f$ and $c_l$, respectively, as characterized in [33].

Following the analysis presented, we establish the local convergence of Algorithm 1, supported by the results in [12,30,31,34]. The proofs are analogous to those in [12] (Theorems 3.3). Therefore, we omit the detailed proofs here.

Theorem 3.

(Local convergence.) Given the conditions specified in (C1) and (C2), the sequence $(s^j,{\zeta }^j,z^j)$ generated by the Algorithm 1 converges to $(s^{*},{\zeta }^{*},z^{*})$, and for sufficiently large values of j, the following holds:

$$\parallel z^{j+1}-z^{*}\parallel \le {\delta }_j\parallel z^j-z^{*}\parallel ,$$

where ${\delta }_j=(2{\rho }_j+c_f/\sqrt{c_f^2+{\sigma }_j^2})/(1-{\rho }_j)$ and ${lim}_{j\to \infty }{\delta }_j=c_f/\sqrt{c_f^2+{\sigma }_{\infty }^2}<1$. Furthermore, if the condition (C3) is also applied, then for sufficiently large values of j,

$$\parallel (s^{j+1},{\zeta }^{j+1})-(s^{*},{\zeta }^{*})\parallel \le {\delta }_j^{\prime }\parallel z^{j+1}-z^j\parallel ,$$

where ${\delta }_j^{\prime }=c_l(1+{\rho }_j^{\prime })/{\sigma }_j$→$c_l/{\sigma }_{\infty }$ when $j\to \infty$.

3.2.  Ssn Algorithm for Subproblem (9)

We shall present a highly effective semismooth Newton (Ssn) algorithm designed to tackle problem (9) in this subsection.

For any given $\sigma >0$ and a fixed $\tilde{z}\in {\mathbb{R}}^n$, our goal is to solve the following problem:

$$\underset{s\in {\mathbb{R}}^m}{\min }\left\{\mathsf{\Phi }\left(s\right):=\underset{\zeta }{\inf }{\mathcal{L}}_{\sigma }(s,\zeta ;\tilde{z})\right\},$$

(11)

where the objective function of problem (11) is given by

$$\mathsf{\Phi }\left(s\right)=M_{{\phi }^{*}}^{1/\sigma }\left({\displaystyle \frac{\tilde{z}}{\sigma }}-A^{\top }s\right)-{\displaystyle \frac{1}{2\sigma }}\parallel \tilde{z}{\parallel }^2+⟨s,b⟩+{\displaystyle \frac{1}{2}}{\parallel s\parallel }^2.$$

It is well known that $\mathsf{\Phi }(·)$ is continuously differentiable with

$$\nabla \mathsf{\Phi }\left(s\right)=-A{\mathrm{Prox}}_{\sigma \phi }(\tilde{z}-\sigma A^{\top }s)+b+s=-\sigma A{\mathrm{Prox}}_{\phi }\left({\displaystyle \frac{\tilde{z}}{\sigma }}-A^{\top }s\right)+b+s.$$

Since $\mathsf{\Phi }(·)$ is strongly convex, we can obtain the unique solution of problem (11) via solving the following nonsmooth equations:

$$\nabla \mathsf{\Phi }\left(s\right)=0.$$

(12)

For any $s\in {\mathbb{R}}^m$, the multifunction is well defined and can be expressed as

$$\mathcal{N}\left(s\right):=\left\{N\in {\mathbb{R}}^{m\times m}|N=I_m+\sigma AV{A}^{\top },V\in \mathcal{V}\left({\displaystyle \frac{\tilde{z}}{\sigma }}-A^{\top }s\right)\right\},$$

where the operator $\mathcal{V}$ is defined in (8).

Remark 1.

Based on Theorem 4, we can deduce that $\mathcal{V}$ is not only nonempty but also has the properties of being compact-valued and upper semicontinuous. In addition, each component of the function $\mathcal{N}(·)$ is symmetric and positive definite.

Before delving into the Ssn algorithm, we introduce the following lemma, the proof of which is analogous to the one presented in [15] (Remark 2.12).

Lemma 2.

For any positive constant r, the proximal mapping ${\mathrm{Prox}}_{\phi }(·)$ is r-order semismooth at $w\in {\mathbb{R}}^n$ with respect to the multifunction $\mathcal{V}$. Similarly, the gradient $\nabla \mathsf{\Phi }$ is r-order semismooth at $s\in {\mathbb{R}}^m$ with respect to the multifunction $\mathcal{N}$.

Proof. 

Given that $S_{\tau /2}(·)$ and ${\mathrm{Prox}}_{{\lambda \parallel ·\parallel }_1}(·)$ are piecewise affine functions as shown in [14], it follows that $Z_{\tau /2}(·)$ is Lipschitz continuous. Consequently, the proximal mapping ${\mathrm{Prox}}_{\phi }(·)$ is also a piecewise affine function that maintains Lipschitz continuity. As established in [35], ${\mathrm{Prox}}_{\phi }(·)$ is directionally differentiable at every point. Combining this with Theorem 4 and the definition of semismoothness [36,37,38,39], we can conclude that ${\mathrm{Prox}}_{\phi }(·)$ exhibits r-order semismoothness on ${\mathbb{R}}^n$. Similarly, it can be inferred that $\nabla \mathsf{\Phi }$ also demonstrates r-order semismoothness on ${\mathbb{R}}^m$. This completes the proof.    □

Given that the gradient $\nabla \mathsf{\Phi }(·)$ is nonsmooth, it is appropriate to employ the semismooth Newton (Ssn) algorithm instead of the standard Newton method to solve Equations (12). Building on the analysis provided, we are now ready to proceed with the development of an Ssn algorithm to solve (12) (Algorithm 2).

Algorithm 2 (Ssn) A semismooth Newton algorithm for (12)

Initialization: $s^0\in {\mathbb{R}}^n$, $\alpha \in (0,1/2)$, $\beta \in (0,1)$. Set $t=0$. 1: Select $N_t\in \mathcal{N}\left(s^t\right)$. Solve the following linear system $$N_{t}d=-\nabla \mathsf{\Phi }\left(s^t\right).$$ (13) 2: Set ${\mu }_t={\beta }^{m_t}$, where $m_t$ is the first nonnegative integer m such that $$\mathsf{\Phi }(s^t+{\beta }^m{d}^t)\le \mathsf{\Phi }\left(s^t\right)+\alpha {\beta }^m⟨\nabla \mathsf{\Phi }\left(s^t\right),d^t⟩.$$ 3: Update $s^{t+1}=s^t+{\mu }_t{d}^t$, $t\leftarrow t+1$, and go to step 1.

For problem (13), we can employ the conjugate gradient algorithm to obtain $d^t$ such that

$$\parallel N_t{d}^t+\nabla \mathsf{\Phi }\left(s^t\right)\parallel \le \min \{\varrho ,\parallel \nabla \mathsf{\Phi }\left(s^t\right){\parallel }^{1+\xi }\},$$

(14)

where $\varrho \in (0,1)$, $\xi \in (0,1]$. Regarding the convergence of Algorithm 2, it is confirmed in the work of Li [14] (Theorem 3), and we now present their result directly as follows.

Theorem 4.

The infinite sequence $s^t$, generated by the Algorithm 2, converges to the unique optimal solution $\overline{s}$ to problem (11). Additionally, it follows that

$$\parallel s^{t+1}-\overline{s}\parallel =O(\parallel s^t-\overline{s}{\parallel }^{1+\xi }),$$

where ξ is given in solving problem (14).

When applying Algorithm 2 to solve problem (12), the most computationally intensive step involves determining the search direction, which is derived from solving the linear system (13), i.e.,

$$(I_m+\sigma AV{A}^{\top })d=-\nabla \mathsf{\Phi }\left(s\right).$$

(15)

Therefore, we aim to investigate the second-order information of the matrix V to reduce computation time. Firstly, let $\mathsf{\Theta }=\mathrm{Diag}\left(\theta \right)$ with

$$\begin{array}{cc}\hfill & {\theta }_i=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{\left|Z_{\tau /2}\left(w_{\tau }\right)\right|}_i\le \lambda \hfill \\ 1\hfill & \mathrm{otherwise}\hfill \end{array}\right.,i=1,2,\dots ,n,\hfill \end{array}$$

and denote $\mathsf{\Sigma }=\mathrm{Diag}\left(\sigma \right)\in {\mathbb{R}}^{(n-1)\times (n-1)}$ with

$$\begin{array}{cc}\hfill & {\sigma }_i=\left\{\begin{array}{cc}0\hfill & \mathrm{if}i\in \mathcal{I}\left(w_{\tau }\right)\hfill \\ 1\hfill & \mathrm{otherwise}\hfill \end{array}\right.,i=1,2,\dots ,n-1,\hfill \end{array}$$

where $\mathcal{I}\left(w_{\tau }\right)$ is given as in (7). It is derived from [14] (Propositions 2 and 3) that $V_0\in \mathcal{V}\left(w\right)$, i.e.,

$$V_0=\mathsf{\Theta }H=\mathsf{\Theta }(I_n-B^{\top }{\left(\mathsf{\Phi }B{B}^{\top }\mathsf{\Phi }\right)}^{†}B),$$

where $H\in \mathcal{H}\left(w_{\tau }\right)$. Additionally, we can restructure H as a block diagonal matrix and exploit the generalized Jacobian’s sparse low-rank structure to substantially reduce computation time. Moreover, we have several options for solving the linear system (15), such as the Cholesky factorization, Sherman–Morrison–Woodbury (SMW) formula, or preconditioned conjugate gradient (PCG) method. These techniques further contribute to improving computational efficiency. Specific details can be found in [14,40], which we do not repeat here.

4. Numerical Experiments

In this section, we evaluate the efficiency of the Ssnal algorithm for solving the GCNIR problem by comparing it with the ADMM and APG algorithms using both synthetic and real datasets. The computational results were achieved by using MATLAB R2022b on a Dell desktop equipped with an Intel(R) Core(TM) i7-11700 CPU running at 2.50 GHz, along with 8 GB of RAM.

4.1. Some First-Order Methods for the GCNIR Problem

We provide a brief overview of the framework of ADMM and APG.

ADMM is recognized as a representative algorithm for addressing convex optimization problems [41], including the problem presented in (D). Here is a summary of the framework for the ADMM algorithm (see Algorithm 3):

Algorithm 3 ADMM for the dual problem (D)

Initialization: Set $\sigma >0$, $c\in (0,(1+\sqrt{5})/2)$, ${\zeta }^0\in {\mathbb{R}}^n$, $z^0\in {\mathbb{R}}^n$, and initialize $j=0$. 1: Address $$s^{j+1}\approx \underset{s\in {\mathbb{R}}^m}{\mathrm{argmin}}{\mathcal{L}}_{\sigma }(s,{\zeta }^j;z^j).$$ (16) by utilizing either direct solvers or iterative methods like preconditioned conjugate gradient (PCG) method for solving this linear system to obtain $s^{j+1}$: $$(I_m+\sigma A{A}^{\top })s=A(z^j-\sigma {\zeta }^j)-b.$$ 2: Calculate ${\zeta }^{j+1}=\underset{\zeta \in {\mathbb{R}}^n}{\mathrm{argmin}}{\mathcal{L}}_{\sigma }(s^{j+1},\zeta ;z^j)={\mathrm{Prox}}_{{\phi }^{*}/\sigma }\left({\displaystyle \frac{z^j}{\sigma }}-A^{\top }{s}^{j+1}\right).$ 3: Update $z^{j+1}=z^j-c\sigma (A^{\top }{s}^{j+1}+{\zeta }^{j+1}).$ 4: Set $j\leftarrow j+1$, and go to Step 1.

Let $L={\lambda }_{\max }\left(A^{\top }A\right)$, representing the Lipschitz constant for function ${\parallel Az-b\parallel }^2/2$ in the primal problem (P). Here is a summary of the framework for the APG algorithm (see Algorithm 4):

Algorithm 4 APG for the primal problem (P)

Initialization: Choose $ϵ>0$, set $x^0=z^0\in {\mathbb{R}}^n$, $r_0=1$, and initialize $j=0.$ 1: Calculate $$z^{j+1}={\mathrm{Prox}}_{\phi /L}\left(x^j-{\displaystyle \frac{A^{\top }(A{x}^j-b))}{L}}\right).$$ 2: Compute $r_{j+1}=(1+\sqrt{1+4r_j^2})/2.$ 3: Update $$x^{j+1}=z^{j+1}+{\displaystyle \frac{r_j-1}{r_{j+1}}}(z^{j+1}-z^j).$$ 4: Set $j\leftarrow j+1$, and go to Step 1.

4.2. Stopping Criteria

Utilizing the KKT conditions of problem (P) and (D), we can obtain the following relative KKT residual:

$$\begin{array}{c}\hfill E_P:={\displaystyle \frac{\parallel Az-b-s\parallel }{1+\parallel b\parallel }},E_D:={\displaystyle \frac{\parallel A^{\top }s+\zeta \parallel }{1+\parallel \zeta \parallel }},E_K:={\displaystyle \frac{\parallel z-{\mathrm{Prox}}_{\phi }(z-A^{\top }(Az-b))\parallel }{1+\parallel z\parallel +\parallel A^{\top }(Az-b)\parallel }}.\end{array}$$

In addition, let ${\mathrm{obj}}_{\mathrm{P}}$ and ${\mathrm{obj}}_{\mathrm{D}}$ represent the objective values of (P) and (D), respectively, i.e.,

$$\begin{array}{c}\hfill {\mathrm{obj}}_{\mathrm{P}}:={\displaystyle \frac{1}{2}}{\parallel Az-b\parallel }^2+{\lambda \parallel z\parallel }_1+\tau \sum _{i=1}^{n-1}{(z_i-z_{i+1})}_{+},{\mathrm{obj}}_{\mathrm{D}}:=-{\displaystyle \frac{1}{2}}{\parallel s\parallel }^2-⟨b,s⟩,\end{array}$$

Then, the relative dual gap is defined by

$$\begin{array}{c}\hfill E_G:={\displaystyle \frac{|{\mathrm{obj}}_{\mathrm{P}}-{\mathrm{obj}}_{\mathrm{D}}|}{1+|{\mathrm{obj}}_{\mathrm{P}}|+|{\mathrm{obj}}_{\mathrm{D}}|}}.\end{array}$$

In later experiments, we start the Ssnal algorithm with the parameters $(s^0,{\zeta }^0,z^0)=(0,0,0)$ and terminate it when $\mathrm{Res}:=\max \{E_P,E_D,E_K,E_G\}\le \mathrm{tol}$ with a given error tolerance “to”. To enhance convergence speed, it is essential to dynamically modify the penalty parameter $\sigma$ in the Ssnal algorithm. Specifically, we initially set ${\sigma }_0=\min (1,1/\sqrt{{\lambda }_{\max }\left(A{A}^{\top }\right)})$ and tune ${\sigma }_{j+1}$ every three steps, i.e.,

$$\begin{array}{c}\hfill \begin{array}{c}\hfill {\sigma }_{j+1}=\left\{\begin{array}{cc}\max \{0.001,0.002{\sigma }_j\},\hfill & E_D^j<0.1E_P^j,\hfill \\ \min \{5000,50{\sigma }_j\},\hfill & E_D^j>10E_P^j,\hfill \\ {\sigma }_j,\hfill & \mathrm{otherwise},\hfill \end{array}\right.\end{array}\end{array}$$

where ${\sigma }_j$, $E_P^j$, and $E_D^j$ represent the values of $\sigma$, $E_P$, and $E_D$ during the j-th iteration, respectively.

In the case of the ADMM algorithm, we set the initial point $(s^0,{\zeta }^0,z^0)=(0,0,0)$ and terminate the algorithm when $\mathrm{Res}=E_K\le \mathrm{tol}$. For the APG algorithm, we start the initial point $(z^0,x^0)=(0,0)$ and terminate the algorithm when $\mathrm{Res}=E_K\le \mathrm{tol}$.

Additionally, we set a tolerance level of $\mathrm{tol}={10}^{-6}$. The tested algorithms will terminate under two conditions: either upon reaching their maximum number of iterations (100 for Ssnal; 30,000 for ADMM and APG) or if their running time exceeds 3 h.

In the following tables, “nnz” and “mon” represent the counts of non-zero elements in z and $Bz$, as derived from Ssnal through these estimations:

$$\begin{array}{cc}\hfill \mathrm{nnz}& :=\min \left\{j\mid \sum _{k=1}^j{\left|z\right|}_{\left(k\right)}\ge 0.999{\parallel z\parallel }_1\right\},\hfill \\ \hfill \mathrm{mon}& :=\min \left\{j\mid \sum _{k=1}^j{|\left(Bz\right)}_{+}{|}_{\left(k\right)}\ge 0.999{\parallel {\left(Bz\right)}_{+}\parallel }_1\right\},\hfill \end{array}$$

where ${\left|z\right|}_{\left(k\right)}$ represents the k-th largest component in $\left|z\right|$, ordered as ${\left|z\right|}_{\left(1\right)}\ge {\left|z\right|}_{\left(2\right)}\ge \cdots \ge {\left|z\right|}_{\left(n\right)}$. Time is shown in seconds.

4.3. Results on Synthetic Data

In this subsection, we evaluate the performance of three algorithms: Ssnal, ADMM, and APG on synthetic data.

In later experiments, we test five instances: $(m;n)=(300k;8000k),k=1,\dots ,5$. To create synthetic data, we employ the model

$$b=Az+\tilde{\xi }ϵ,ϵ\sim N(0,I),$$

where $A\in {\mathbb{R}}^{m\times n}$ is drawn from a Gaussian distribution $N(0,I_n)$ and $z\in {\mathbb{R}}^n$ is generated by a distributed random number. Following the way provided in [15], we set $\tilde{\xi }$ to be $0.1\parallel Az\parallel /\parallel ϵ\parallel$ for each case. The regularization parameters for the GCNIR problem (1) are specified as follows:

$$\lambda =0.5\varpi \parallel A^{\top }{b\parallel }_{\infty },\tau =\lambda ,$$

(P1)

where $0<\varpi <2$. In total, we test 20 instances.

Table 1 presents a comparative analysis of the performance of three algorithms on synthetic datasets ranging from small to large scales. It is evident that the Ssnal algorithm exhibits both efficiency and robustness across a variety of parameter settings. Although all algorithms are capable of solving the problem with the required level of accuracy, Ssnal consistently outperforms the other two methods in terms of computational time. For instance, in the case of instance 5 with $\varpi =0.6$, Ssnal completes the task in a mere 3 s, whereas ADMM and APG take over 60 and 80 s, respectively.

Table 1. The performances of Ssnal, ADMM, and APG when applied to synthetic data.

i	${\mathit{\lambda }}_{\mathbf{max}}\left({\mathit{AA}}^{\mathit{\top }}\right)$	$\mathit{\varpi }$	nnz; mon	Time	Res
				$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$	$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$
1	$1.13\times {10}^4$	0.6	93; 69	0.26 | 0.40 | 1.56	$1.85\times {10}^{-7}$ | $9.42\times {10}^{-7}$ | $9.69\times {10}^{-7}$
		0.7	57; 45	0.19 | 0.32 | 1.29	$7.75\times {10}^{-8}$ | $9.21\times {10}^{-7}$ | $9.99\times {10}^{-7}$
		0.8	25; 20	0.11 | 0.29 | 1.12	$2.02\times {10}^{-8}$ | $8.89\times {10}^{-7}$ | $9.40\times {10}^{-7}$
		0.9	9; 8	0.07 | 0.29 | 0.66	$5.60\times {10}^{-7}$ | $8.38\times {10}^{-7}$ | $9.41\times {10}^{-7}$
2	$2.28\times {10}^4$	0.6	144; 100	0.53 | 4.25 | 13.38	$3.13\times {10}^{-8}$ | $9.97\times {10}^{-7}$ | $9.90\times {10}^{-7}$
		0.7	73; 53	0.48 | 4.20 | 9.85	$7.98\times {10}^{-9}$ | $9.30\times {10}^{-7}$ | $9.69\times {10}^{-7}$
		0.8	23; 16	0.29 | 3.85 | 7.11	$3.87\times {10}^{-7}$ | $9.36\times {10}^{-7}$ | $9.57\times {10}^{-7}$
		0.9	8; 5	0.16 | 3.72 | 3.64	$1.07\times {10}^{-7}$ | $9.72\times {10}^{-7}$ | $9.77\times {10}^{-7}$
3	$3.43\times {10}^4$	0.6	251; 170	1.57 | 16.54 | 41.17	$2.56\times {10}^{-8}$ | $9.91\times {10}^{-7}$ | $9.92\times {10}^{-7}$
		0.7	134; 93	1.02 | 16.24 | 30.42	$6.80\times {10}^{-7}$ | $9.82\times {10}^{-7}$ | $9.66\times {10}^{-7}$
		0.8	54; 37	0.78 | 15.50 | 23.71	$3.08\times {10}^{-7}$ | $9.29\times {10}^{-7}$ | $9.98\times {10}^{-7}$
		0.9	14; 8	0.42 | 14.51 | 19.73	$1.09\times {10}^{-7}$ | $9.21\times {10}^{-7}$ | $9.83\times {10}^{-7}$
4	$4.55\times {10}^4$	0.6	286; 205	2.01 | 35.65 | 63.67	$9.47\times {10}^{-7}$ | $9.52\times {10}^{-7}$ | $9.94\times {10}^{-7}$
		0.7	145; 102	1.77 | 33.64 | 49.07	$3.31\times {10}^{-7}$ | $9.77\times {10}^{-7}$ | $9.94\times {10}^{-7}$
		0.8	58; 40	1.12 | 31.12 | 39.93	$1.46\times {10}^{-7}$ | $9.60\times {10}^{-7}$ | $9.50\times {10}^{-7}$
		0.9	12; 9	0.64 | 28.96 | 34.60	$7.32\times {10}^{-8}$ | $9.66\times {10}^{-7}$ | $9.47\times {10}^{-7}$
5	$5.69\times {10}^4$	0.6	189; 133	2.62 | 61.07 | 83.39	$3.50\times {10}^{-7}$ | $9.68\times {10}^{-7}$ | $9.98\times {10}^{-7}$
		0.7	69; 47	1.50 | 58.16 | 63.74	$1.24\times {10}^{-7}$ | $9.44\times {10}^{-7}$ | $9.59\times {10}^{-7}$
		0.8	10; 9	1.10 | 52.05 | 53.83	$6.25\times {10}^{-8}$ | $9.16\times {10}^{-7}$ | $9.41\times {10}^{-7}$
		0.9	4; 3	0.83 | 50.61 | 42.12	$2.10\times {10}^{-8}$ | $8.82\times {10}^{-7}$ | $9.61\times {10}^{-7}$

4.4. Results on Real Datasets

We evaluate the performances of three methods on three datasets: 10-K Corpus, StatLib, and UCI, which have been sourced from the LIBSVM datasets [42].

In our experiments with real datasets, we follow the methodology in [12] and utilize polynomial basis functions [43] to expand the original features across twelve datasets. For example, the number “10” in “mgscale10” indicates that we use a tenth-order polynomial to generate the basis functions. Furthermore, Table 2 provides statistical details for the twelve datasets under consideration, where “m” refers to the sample size and “n” denotes the number of features. For the GCNIR problem, the regularization parameters are set as the following three different strategies [13,14]: (P1) given in the previous section and

$$\begin{array}{c}\hfill \lambda =0.5\varpi \parallel A^{\top }{b\parallel }_{\infty },\tau =9\lambda ;\end{array}$$

(P2)

$$\begin{array}{c}\hfill \lambda =0.5\varpi \parallel A^{\top }{b\parallel }_{\infty },\tau =\lambda /\sqrt{n}.\end{array}$$

(P3)

In our experiments, we use two values of $\varpi$ for all datasets.

Table 2. Summary of tested data sets.

No.	Proname	$\mathit{m};\mathit{n}$	${\mathit{\lambda }}_{\mathbf{max}}\left({\mathit{AA}}^{\mathit{\top }}\right)$
N1	E2006.train	16,087;150,360	$1.91\times {10}^5$
N2	E2006.test	3308; 150,358	$4.79\times {10}^4$
N3	pyrim5	74; 201,376	$1.22\times {10}^6$
N4	triazines4	186; 635,376	$2.07\times {10}^7$
N5	abalone7	4177; 6435	$5.21\times {10}^5$
N6	bodyfat7	252; 116,280	$5.29\times {10}^4$
N7	housing7	506; 77,520	$3.28\times {10}^5$
N8	mpg7	392; 3432	$1.28\times {10}^4$
N9	space9	3107; 5005	$4.01\times {10}^3$
N10	mg10	1385; 8008	$5.11\times {10}^3$
N11	eunite2001_train6	336; 74,613	$3.54\times {10}^4$
N12	eunite2001_test6	31; 74,613	$2.14\times {10}^3$

Table 3, Table 4 and Table 5 display the comparative results of the three algorithms across parameter sets (P1), (P2), and (P3), respectively. An analysis of these tables reveals that the Ssnal algorithm was able to solve all 72 test cases within a 70 s timeframe, with the majority of these cases being resolved in under 20 s. In comparison, ADMM encountered failures in 23 cases, while APG failed in 50 instances. These results underscore the superior performance of Ssnal, which not only consistently outperformed ADMM and APG in terms of speed but also demonstrated a higher success rate.

Table 3. The performances of Ssnal, ADMM, and APG when applied to real datasets with the regularization parameters defined as in (P1).

No.	$\mathit{\varpi }$	nnz; mon	Time	Res
			$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathrm{APG}$	$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$
N1	$1\times {10}^{-5}$	3; 2	1.48 | 918.82 | 394.89	$2.11\times {10}^{-7}$ | $9.97\times {10}^{-7}$ | $5.89\times {10}^{-7}$
	$1\times {10}^{-6}$	30; 27	2.95 | 10,425.34 | 3574.08	$3.13\times {10}^{-7}$ |$\mathbf{1.64}*\times {\mathbf{10}}^{-\mathbf{7}}$ |$\mathbf{6.47}\times {\mathbf{10}}^{-\mathbf{7}}$
N2	$1\times {10}^{-5}$	1; 1	0.87 | 5.35 | 2.69	$1.78\times {10}^{-7}$ | $7.19\times {10}^{-7}$ | $1.44\times {10}^{-10}$
	$1\times {10}^{-6}$	53; 51	1.40 | 1415.89 | 1147.08	$3.39\times {10}^{-7}$ |$\mathbf{4.12}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{2.79}\times {\mathbf{10}}^{-\mathbf{4}}$
N3	$1\times {10}^{-3}$	188; 62	9.35 | 1218.89 | 1769.23	$2.19\times {10}^{-9}$ | $9.99\times {10}^{-7}$ |$\mathbf{1.14}\times {\mathbf{10}}^{-\mathbf{3}}$
	$1\times {10}^{-4}$	229; 72	12.64 | 1844.98 | 1759.28	$1.99\times {10}^{-8}$ |$\mathbf{1.19}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{3.58}\times {\mathbf{10}}^{-\mathbf{3}}$
N4	$1\times {10}^{-3}$	813; 144	44.33 | 10,800.13 | 10,800.24	$7.25\times {10}^{-7}$ |$\mathbf{8.61}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{2.67}\times {\mathbf{10}}^{-\mathbf{3}}$
	$1\times {10}^{-4}$	932; 191	66.75 | 10,800.39 | 10,800.20	$5.07\times {10}^{-8}$ |$\mathbf{4.20}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{1.51}\times {\mathbf{10}}^{-\mathbf{2}}$
N5	$1\times {10}^{-3}$	31; 16	3.74 | 203.73 | 3056.66	$2.16\times {10}^{-8}$ | $9.95\times {10}^{-7}$ |$\mathbf{2.08}\times {\mathbf{10}}^{-\mathbf{5}}$
	$1\times {10}^{-4}$	76; 57	8.13 | 2050.13 | 3028.76	$6.65\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{8.40}\times {\mathbf{10}}^{-\mathbf{4}}$
N6	$1\times {10}^{-3}$	2; 2	4.03 | 1718.95 | 613.07	$1.01\times {10}^{-9}$ | $1.00\times {10}^{-6}$ | $6.60\times {10}^{-7}$
	$1\times {10}^{-4}$	4; 3	4.78 | 1087.70 | 2832.63	$1.10\times {10}^{-9}$ | $1.00\times {10}^{-6}$ | $9.89\times {10}^{-7}$
N7	$1\times {10}^{-3}$	192; 121	8.07 | 1391.03 | 3812.60	$2.83\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{5.54}\times {\mathbf{10}}^{-\mathbf{5}}$
	$1\times {10}^{-4}$	337; 232	10.96 | 4666.00 | 4022.00	$9.20\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{2.20}\times {\mathbf{10}}^{-\mathbf{3}}$
N8	$1\times {10}^{-3}$	57; 33	0.35 | 0.96 | 15.52	$4.17\times {10}^{-9}$ | $1.00\times {10}^{-6}$ |$\mathbf{1.68}\times {\mathbf{10}}^{-\mathbf{6}}$
	$1\times {10}^{-4}$	169; 99	0.48 | 27.31 | 20.70	$9.73\times {10}^{-7}$ |$\mathbf{1.59}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{5.51}\times {\mathbf{10}}^{-\mathbf{5}}$
N9	$1\times {10}^{-3}$	23; 12	0.92 | 105.33 | 272.03	$3.45\times {10}^{-7}$ | $9.97\times {10}^{-7}$ | $8.64\times {10}^{-7}$
	$1\times {10}^{-4}$	47; 31	1.38 | 1057.48 | 1602.03	$1.26\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{5.78}\times {\mathbf{10}}^{-\mathbf{6}}$
N10	$1\times {10}^{-3}$	28; 21	0.66 | 37.79 | 268.66	$1.26\times {10}^{-7}$ | $9.98\times {10}^{-7}$ | $9.92\times {10}^{-7}$
	$1\times {10}^{-4}$	87; 60	1.44 | 505.11 | 1062.37	$2.73\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{2.98}\times {\mathbf{10}}^{-\mathbf{6}}$
N11	$1\times {10}^{-3}$	22; 7	3.17 | 204.05 | 2806.48	$2.80\times {10}^{-8}$ | $9.96\times {10}^{-7}$ |$\mathbf{1.94}\times {\mathbf{10}}^{-\mathbf{5}}$
	$1\times {10}^{-4}$	142; 26	3.27 | 321.88 | 2704.28	$9.52\times {10}^{-8}$ | $9.99\times {10}^{-7}$ |$\mathbf{4.34}\times {\mathbf{10}}^{-\mathbf{5}}$
N12	$1\times {10}^{-3}$	19; 5	0.75 | 27.07 | 241.41	$2.25\times {10}^{-8}$ | $9.98\times {10}^{-7}$ |$\mathbf{5.09}\times {\mathbf{10}}^{-\mathbf{6}}$
	$1\times {10}^{-4}$	102; 27	0.88 | 252.12 | 248.56	$3.96\times {10}^{-7}$ |$\mathbf{9.87}\times {\mathbf{10}}^{-\mathbf{6}}$ |$\mathbf{7.27}\times {\mathbf{10}}^{-\mathbf{6}}$

* The bolded data indicate that the corresponding value did not meet the precision requirements.

Table 4. The performances of Ssnal, ADMM, and APG when applied to real datasets with the regularization parameters defined as in (P2).

No.	$\mathit{\varpi }$	nnz; mon	Time	Res
			$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$	$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$
N1	$1\times {10}^{-5}$	1; 1	1.03 | 289.40 | 2.51	$9.13\times {10}^{-9}$ | $9.63\times {10}^{-7}$ | $5.50\times {10}^{-9}$
	$1\times {10}^{-6}$	3; 2	2.16 | 1142.71 | 2192.27	$5.88\times {10}^{-8}$ | $9.99\times {10}^{-7}$ | $2.36\times {10}^{-7}$
N2	$1\times {10}^{-5}$	1; 1	0.33 | 5.97 | 0.70	$1.12\times {10}^{-7}$ | $7.18\times {10}^{-7}$ | $2.78\times {10}^{-9}$
	$1\times {10}^{-6}$	5; 4	0.64 | 274.56 | 1128.41	$1.92\times {10}^{-7}$ | $9.99\times {10}^{-7}$ |$\mathbf{2.16}*\times {\mathbf{10}}^{-\mathbf{5}}$
N3	$1\times {10}^{-3}$	716; 46	5.70 | 574.10 | 1760.94	$1.00\times {10}^{-8}$ | $9.99\times {10}^{-7}$ |$\mathbf{5.91}\times {\mathbf{10}}^{-\mathbf{4}}$
	$1\times {10}^{-4}$	776; 61	7.90 | 1785.58 | 1728.24	$8.79\times {10}^{-7}$ |$\mathbf{9.31}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{5.62}\times {\mathbf{10}}^{-\mathbf{3}}$
N4	$1\times {10}^{-3}$	2475; 119	26.30 | 10,800.38 | 10,800.40	$4.09\times {10}^{-8}$ |$\mathbf{2.41}\times {\mathbf{10}}^{-\mathbf{3}}$ |$\mathbf{9.35}\times {\mathbf{10}}^{-\mathbf{4}}$
	$1\times {10}^{-4}$	3525; 161	50.08 | 10,800.09 | 10,800.13	$2.56\times {10}^{-7}$ |$\mathbf{4.60}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{8.83}\times {\mathbf{10}}^{-\mathbf{3}}$
N5	$1\times {10}^{-3}$	96; 9	2.18 | 232.40 | 2917.46	$6.92\times {10}^{-9}$ | $9.95\times {10}^{-7}$ |$\mathbf{4.89}\times {\mathbf{10}}^{-\mathbf{6}}$
	$1\times {10}^{-4}$	82; 21	4.39 | 202.29 | 2925.83	$6.72\times {10}^{-9}$ | $9.97\times {10}^{-7}$ |$\mathbf{1.76}\times {\mathbf{10}}^{-\mathbf{4}}$
N6	$1\times {10}^{-3}$	9; 3	1.98 | 1187.52 | 1422.26	$6.39\times {10}^{-7}$ | $1.00\times {10}^{-6}$ | $9.71\times {10}^{-7}$
	$1\times {10}^{-4}$	8; 3	3.30 | 1064.66 | 2015.64	$3.20\times {10}^{-7}$ | $1.00\times {10}^{-6}$ | $9.62\times {10}^{-7}$
N7	$1\times {10}^{-3}$	347; 52	5.07 | 711.36 | 4618.18	$5.92\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{1.75}\times {\mathbf{10}}^{-\mathbf{5}}$
	$1\times {10}^{-4}$	711; 125	8.65 | 3054.37 | 4570.98	$1.61\times {10}^{-7}$ | $1.00\times {10}^{-6}$ |$\mathbf{2.74}\times {\mathbf{10}}^{-\mathbf{4}}$
N8	$1\times {10}^{-3}$	53; 13	0.24 | 1.15 | 13.51	$4.70\times {10}^{-7}$ | $1.00\times {10}^{-6}$ | $9.96\times {10}^{-7}$
	$1\times {10}^{-4}$	214; 51	0.52 | 11.98 | 19.47	$7.79\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{1.09}\times {\mathbf{10}}^{-\mathbf{5}}$
N9	$1\times {10}^{-3}$	16; 8	0.74 | 6.79 | 44.64	$1.10\times {10}^{-9}$ | $8.36\times {10}^{-7}$ | $9.60\times {10}^{-7}$
	$1\times {10}^{-4}$	70; 18	1.27 | 553.37 | 1597.02	$3.19\times {10}^{-8}$ | $1.00\times {10}^{-6}$ | $9.91\times {10}^{-7}$
N10	$1\times {10}^{-3}$	34; 12	0.73 | 5.65 | 100.67	$3.00\times {10}^{-7}$ | $9.32\times {10}^{-7}$ | $9.97\times {10}^{-7}$
	$1\times {10}^{-4}$	151; 33	0.96 | 219.02 | 701.33	$6.53\times {10}^{-8}$ | $9.99\times {10}^{-7}$ | $1.00\times {10}^{-6}$
N11	$1\times {10}^{-3}$	110; 3	1.89 | 304.06 | 874.43	$9.99\times {10}^{-9}$ | $9.99\times {10}^{-7}$ | $9.53\times {10}^{-7}$
	$1\times {10}^{-4}$	811; 19	3.24 | 180.17 | 2900.86	$8.66\times {10}^{-8}$ | $9.99\times {10}^{-7}$ |$\mathbf{4.28}\times {\mathbf{10}}^{-\mathbf{5}}$
N12	$1\times {10}^{-3}$	218; 3	0.56 | 10.26 | 52.18	$1.81\times {10}^{-7}$ | $1.00\times {10}^{-6}$ | $9.41\times {10}^{-7}$
	$1\times {10}^{-4}$	491; 15	0.84 | 266.41 | 254.19	$3.78\times {10}^{-7}$ |$\mathbf{2.06}\times {\mathbf{10}}^{-\mathbf{6}}$ |$\mathbf{3.66}\times {\mathbf{10}}^{-\mathbf{6}}$

* The bolded data indicate that the corresponding value did not meet the precision requirements.

Table 5. The performances of Ssnal, ADMM, and APG when applied to real datasets with the regularization parameters defined as in (P3).

No.	$\mathit{\varpi }$	nnz; mon	Time	Res
			$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$	$\mathbf{S}\mathbf{SNAL}\mathbf{\mid }\mathbf{ADMM}\mathbf{\mid }\mathbf{APG}$
N1	$1\times {10}^{-5}$	2; 2	3.09 | 1863.19 | 2538.36	$5.75\times {10}^{-7}$ | $9.99\times {10}^{-7}$ | $9.56\times {10}^{-7}$
	$1\times {10}^{-6}$	95; 93	4.55 | 9785.88 | 3499.10	$3.32\times {10}^{-7}$ |$\mathbf{4.03}*\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{1.02}\times {\mathbf{10}}^{-\mathbf{3}}$
N2	$1\times {10}^{-5}$	3; 3	1.97 | 180.83 | 1142.03	$3.34\times {10}^{-8}$ | $9.99\times {10}^{-7}$ |$\mathbf{1.98}\times {\mathbf{10}}^{-\mathbf{5}}$
	$1\times {10}^{-6}$	170; 168	7.19 | 1411.02 | 1103.22	$6.32\times {10}^{-7}$ |$\mathbf{4.20}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{5.08}\times {\mathbf{10}}^{-\mathbf{4}}$
N3	$1\times {10}^{-3}$	84; 83	4.39 | 1615.80 | 1548.70	$3.38\times {10}^{-7}$ |$\mathbf{3.10}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{9.85}\times {\mathbf{10}}^{-\mathbf{4}}$
	$1\times {10}^{-4}$	79; 78	4.62 | 1571.90 | 1546.56	$9.00\times {10}^{-7}$ |$\mathbf{9.94}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{3.05}\times {\mathbf{10}}^{-\mathbf{3}}$
N4	$1\times {10}^{-3}$	231; 168	33.43 | 10,800.08 | 10,800.05	$7.56\times {10}^{-7}$ |$\mathbf{3.58}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{4.47}\times {\mathbf{10}}^{-\mathbf{3}}$
	$1\times {10}^{-4}$	230; 192	52.89 | 10,800.22 | 10,800.35	$3.57\times {10}^{-7}$ |$\mathbf{3.72}\times {\mathbf{10}}^{-\mathbf{2}}$ |$\mathbf{1.33}\times {\mathbf{10}}^{-\mathbf{2}}$
N5	$1\times {10}^{-3}$	29; 28	4.48 | 972.66 | 3166.79	$1.33\times {10}^{-7}$ | $1.00\times {10}^{-6}$ |$\mathbf{1.04}\times {\mathbf{10}}^{-\mathbf{4}}$
	$1\times {10}^{-4}$	92; 86	16.39 | 3582.72 | 3013.39	$1.35\times {10}^{-7}$ |$\mathbf{1.37}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{2.21}\times {\mathbf{10}}^{-\mathbf{3}}$
N6	$1\times {10}^{-3}$	2; 2	3.31 | 1644.35 | 694.08	$1.87\times {10}^{-8}$ | $1.00\times {10}^{-6}$ | $3.33\times {10}^{-7}$
	$1\times {10}^{-4}$	8; 7	3.83 | 846.32 | 2765.49	$5.32\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{4.51}\times {\mathbf{10}}^{-\mathbf{5}}$
N7	$1\times {10}^{-3}$	195; 187	7.29 | 3279.67 | 4712.21	$5.63\times {10}^{-7}$ | $9.99\times {10}^{-7}$ |$\mathbf{1.71}\times {\mathbf{10}}^{-\mathbf{4}}$
	$1\times {10}^{-4}$	337; 331	7.50 | 4727.07 | 5025.99	$9.51\times {10}^{-7}$ |$\mathbf{1.94}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{4.61}\times {\mathbf{10}}^{-\mathbf{3}}$
N8	$1\times {10}^{-3}$	56; 54	0.28 | 5.28 | 16.26	$2.66\times {10}^{-8}$ | $1.00\times {10}^{-6}$ |$\mathbf{6.37}\times {\mathbf{10}}^{-\mathbf{6}}$
	$1\times {10}^{-4}$	169; 140	0.41 | 22.06 | 16.95	$1.06\times {10}^{-7}$ |$\mathbf{1.65}\times {\mathbf{10}}^{-\mathbf{4}}$ |$\mathbf{1.28}\times {\mathbf{10}}^{-\mathbf{4}}$
N9	$1\times {10}^{-3}$	18; 15	0.99 | 133.89 | 531.25	$1.13\times {10}^{-9}$ | $9.97\times {10}^{-7}$ | $5.77\times {10}^{-7}$
	$1\times {10}^{-4}$	50; 39	1.71 | 1936.72 | 1632.34	$1.02\times {10}^{-7}$ |$\mathbf{2.45}\times {\mathbf{10}}^{-\mathbf{6}}$ |$\mathbf{6.39}\times {\mathbf{10}}^{-\mathbf{6}}$
N10	$1\times {10}^{-3}$	32; 28	0.96 | 39.57 | 471.36	$1.72\times {10}^{-7}$ | $9.93\times {10}^{-7}$ | $9.93\times {10}^{-7}$
	$1\times {10}^{-4}$	92; 86	1.41 | 1105.05 | 1022.00	$2.44\times {10}^{-7}$ |$\mathbf{2.31}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{1.10}\times {\mathbf{10}}^{-\mathbf{5}}$
N11	$1\times {10}^{-3}$	18; 13	2.49 | 290.45 | 2284.04	$1.04\times {10}^{-8}$ | $9.97\times {10}^{-7}$ |$\mathbf{7.13}\times {\mathbf{10}}^{-\mathbf{6}}$
	$1\times {10}^{-4}$	100; 95	3.19 | 1436.32 | 2290.59	$6.20\times {10}^{-9}$ | $9.88\times {10}^{-7}$ |$\mathbf{1.05}\times {\mathbf{10}}^{-\mathbf{4}}$
N12	$1\times {10}^{-3}$	23; 21	0.63 | 110.50 | 235.08	$1.74\times {10}^{-7}$ | $9.97\times {10}^{-7}$ |$\mathbf{1.17}\times {\mathbf{10}}^{-\mathbf{5}}$
	$1\times {10}^{-4}$	47; 44	0.67 | 243.67 | 236.79	$6.24\times {10}^{-7}$ |$\mathbf{3.30}\times {\mathbf{10}}^{-\mathbf{5}}$ |$\mathbf{1.41}\times {\mathbf{10}}^{-\mathbf{5}}$

* The bolded data indicate that the corresponding value did not meet the precision requirements.

From Table 3, Ssnal not only succeeds in achieving the required accuracy but also takes less time than ADMM and APG. For instance, in the case of N7 with $\varpi =1\times {10}^{-4}$, the Ssnal algorithm takes $10.96$ s to reach the high accuracy of $\mathrm{Res}=9.20\times {10}^{-8}$, while the ADMM algorithm needs 4666 s to achieve the lower accuracy of $\mathrm{Res}=1.00\times {10}^{-6}$, and the APG algorithm even requires 4022 s to achieve a relatively large error of $\mathrm{Res}=2.20\times {10}^{-3}$. Therefore, the Ssnal algorithm outperforms the other two algorithms in addressing the GCNIR problem.

Table 4 shows that the number of reversed order coefficients in z is almost the fewest among all tables. This is because the regularization parameter $\tau$, which enforces monotonicity, is larger than the regularization parameter $\lambda$, which enforces sparsity in (P2). From Table 4, it is also evident that Ssnal demands considerably less time than the other two methods on twelve cases. Moreover, for more challenging tests, such as N3 with $\varpi =1\times {10}^{-4}$, only Ssnal successfully solved this problem, while the other two algorithms did not meet the accuracy requirements. The results strongly indicate that our Ssnal algorithm can efficiently and reliably solve the GCNIR problem.

Table 5 further illustrates that Ssnal continues to outperform ADMM and APG by a significant margin. This advantage is particularly pronounced for large-scale problems. In particular, for the case N4 with $\varpi =1\times {10}^{-4}$, Ssnal solves it to the desired accuracy within 53 s, while ADMM and APG fail to solve it within 3 h.

Consequently, we can confidently state that our Ssnal algorithm can efficiently and robustly solve the GCNIR problem (1) on real datasets with high accuracy.

5. Conclusions

In this paper, we proposed a highly efficient semismooth Newton-based augmented Lagrangian method for solving the GCNIR problem from the dual perspective. The proximal mapping associated with the GCNIR regularizer and its generalized Jacobian have been derived, and we have utilized the second-order sparsity structure to achieve superior performance in solving the subproblem of the Ssnal algorithm. Numerical results have demonstrated the efficiency and robustness of our proposed algorithm compared to the widely used ADMM and APG methods on both the synthetic and real datasets. Looking ahead, we anticipate our algorithm to play a significant role in solving convex problems with the GCNIR regularizer, thereby facilitating data analysis in statistical learning.