An Approximate Algorithm for Sparse Distributionally Robust Optimization

Abstract

In this paper, we propose a sparse distributionally robust optimization (DRO) model incorporating the Conditional Value-at-Risk (CVaR) measure to control tail risks in uncertain environments. The model utilizes sparsity to reduce transaction costs and enhance operational efficiency. We reformulate the problem as a Min-Max-Min optimization and convert it into an equivalent non-smooth minimization problem. To address this computational challenge, we develop an approximate discretization (AD) scheme for the underlying continuous random vector and prove its convergence to the original non-smooth formulation under mild conditions. The resulting problem can be efficiently solved using a subgradient method. While our analysis focuses on CVaR penalty, this approach is applicable to a broader class of non-smooth convex regularizers. The experimental results on the portfolio selection problem confirm the effectiveness and scalability of the proposed AD algorithm.

1. Introduction

The problem of robust optimization has gained significant attention in various fields due to its ability to handle uncertainty in optimization problems (see [1,2,3]). In particular, DRO has emerged as an effective approach for modeling uncertain parameters, where the objective is to optimize a performance criterion under the worst-case distribution of uncertain data (see [4,5,6,7,8]). While the CVaR measure has been widely used in DRO to account for tail risk, the application of such methods often faces computational challenges, especially when dealing with non-smooth optimization problems. CVaR is widely considered to be a more effective measurement of extreme losses, particularly in the face of large market fluctuations or black swan events (e.g., [9,10,11,12,13,14]), as it better captures the potential risks of a portfolio, and it can be determined from the formula

$$\begin{array}{c}\hfill {\varphi }_{\beta }\left(x\right)=\underset{\alpha \in \mathbb{R}}{min}\{\alpha +{(1-\beta )}^{-1}\mathbb{E}{\left[\mathcal{l}(x,\xi )-\alpha \right]}_{+}\},\end{array}$$

(1)

where $\mathcal{l}(x,\xi )$ is a convex loss function in $x\in X$ that depends on some vector of parameters $\xi \in \Xi$.

We propose a sparse DRO model that leverages the CVaR risk measure to manage uncertainty in the decision process. By incorporating sparsity into the model, we aim to improve both the computational efficiency and practical applicability of the solution, especially in high-dimensional settings where traditional methods may be computationally expensive. For example, traditional portfolio optimization methods often lead to a full investment solution, where all assets have non-zero weights. However, in practice, investors generally seek a sparse portfolio (e.g., [15,16,17]), where most asset weights are zero, to reduce transaction costs and management complexity. When considering DRO, a challenge remains as to how to ensure the robustness of the model while achieving a sparse solution. Therefore, achieving a sparse portfolio while maintaining robustness has become a core problem that needs to be addressed.

The proposed model can be formulated as a Min-Max-Min optimization problem, which is then transformed into an equivalent non-smooth minimization problem. A key contribution of this paper is the introduction of an approximate discretization scheme to solve the resulting non-smooth minimization problem. The proposed scheme involves discretizing the continuous random vector and allows for efficient computation through subgradient methods or smoothing-based algorithms. Moreover, we demonstrate that the discretization scheme converges to the equivalent non-smooth formulation under mild conditions. Although we use the CVaR penalty for illustration, the methodology extends to general non-smooth convex penalty functions, which are broadly applicable to various optimization problems that arise in practice. The proposed approximate discretization (AD) algorithm in Algorithm 1 belongs to the class of data-driven optimization frameworks, which have gained increasing attention due to their flexibility and scalability in handling complex and uncertain environments. Unlike physics-based computational approaches [18,19,20], data-driven approaches rely on observed or simulated data to model uncertainties and guide decision-making [4].

In Section 2, we propose the sparse DRO with CVaR (SDRPC) model and establish an equivalent tractable reformulation for SDRPC using the Lagrangian dual problem. Meanwhile, we prove that the dual problem is convex and the set of optimal solutions is nonempty. The objective of the dual problem includes the maximization of an infinite number of convex functions. We propose a discretization scheme for this problem and study its convergence in Section 3.

Throughout the paper, we use the following notation: We use ${\mathbb{S}}_{+}^d$ to denote a cone of symmetric positive semi-definite matrices, and ${\Delta }_d=\left\{x\in {\mathbb{R}}^d\mid \sum _{i=1}^d{x}_i=1,x\ge 0\right\}$. For a number $r\in \mathbb{R}$, we denote ${\left[r\right]}_{+}=max\left\{r,0\right\}$. Given two square matrices M and N, we write $M⪯N$ to indicate $N-M$ being positive semi-definite. The notation $〈M,N〉=\sum _{i,j}{M}_{i,j}{N}_{i,j}$. For the sake of simplicity, we denote $\nu =(x,\alpha ,q,\Lambda )$ and $\mathcal{V}={\Delta }_d\times \mathbb{R}\times {\mathbb{R}}^m\times {\mathbb{S}}_{+}^m$.

2. The Sparse DRO with CVaR (SDRPC) Model

In order to find a sparse and robust optimal solution, we propose the SDRPC model as follows:

$$\begin{array}{c}\hfill (Min-Max-Minproblem)\underset{x\in {\Delta }_d}{min}\underset{P\in \mathcal{P}}{max}{\mathbb{E}}_P\left[\mathbf{F}(x,\xi )\right]+{\tau }_1{\parallel x\parallel }_1+{\tau }_2{\varphi }_{\beta }\left(x\right),\end{array}$$

(2)

where ${\tau }_1,{\tau }_2>0$ are given parameters, $\mathbf{F}(x,\xi )$ is convex and continuous, and the CVaR ${\varphi }_{\beta }\left(x\right)$ is defined in (1). Note that $\xi$ is a random variable with distribution $P\in \mathcal{P}$. The general formulation of $\mathbf{F}(x,\xi )$ makes the model scalable and adaptable to a wide range of economic problems. Whether applied to portfolio optimization, production planning, or supply chain management, the framework can accommodate large, complex systems and diverse economic environments, making it a versatile tool in practical economic decision-making. Although problem (2) appears to be a typical minimax formulation at first glance—minimizing over the decision variable $x\in {\Delta }_d$ and maximizing over the worst-case distribution $P\in \mathcal{P}$—it further involves an inner minimization over the scalar threshold $\alpha \in \mathbb{R}$ due to the definition of the CVaR term in (1). Therefore, problem (2) can be regarded as a Min-Max-Min problem.

To describe the set of probability measures $\mathcal{P}$, let us denote by $\widehat{\mu }$ and $\widehat{\Sigma }$ the reference values of the mean vector and covariance matrix of the historical data $({\widehat{\xi }}^1,\dots ,{\widehat{\xi }}^N)$. We assume that $\widehat{\Sigma }$ is a symmetric and positive definite matrix. The ambiguity set $\mathcal{P}$ in (2) is constructed by moment constraints, as intensively studied by Delage and Ye in [4] and by Xu et al. [21]:

$$\begin{array}{c}\hfill \mathcal{P}=\left\{\begin{array}{cc}P\in \mathcal{M}\hfill & |\begin{array}{c} {\left({\mathbb{E}}_P\left[\xi \right]-\widehat{\mu }\right)}^{\top }{\widehat{\Sigma }}^{-1}\left({\mathbb{E}}_P\left[\xi \right]-\widehat{\mu }\right)\le {\kappa }_1\hfill \\ {\mathbb{E}}_P\left[(\xi -\widehat{\mu }){(\xi -\widehat{\mu })}^{\top }\right]⪯{\kappa }_2\widehat{\Sigma }\hfill \end{array}\hfill \end{array}\right\},\end{array}$$

(3)

where ${\kappa }_1\ge 0$, ${\kappa }_2\ge 1$ are two given numbers and $\mathcal{M}$ is the convex set of all probability measures in the measurable space $(\Xi ,\mathbb{B})$, with $\Xi \subseteq {\mathbb{R}}^m$ denoting a convex compact set known to contain the support of P and $\mathbb{B}$ denoting the Borel $\sigma$-algebra on $\Xi$. It is easy to observe that the first constraint in (3), through the Schur complement theorem (Section A.5.5 of [22]), can be equivalently written as

$$\begin{array}{c}\hfill {\mathbb{E}}_P\left[\begin{array}{cc}-\widehat{\Sigma }\hfill & \hfill \widehat{\mu }-\xi \\ {(\widehat{\mu }-\xi )}^{\top }\hfill & \hfill -{\kappa }_1\end{array}\right]⪯0.\end{array}$$

We denote

$$\begin{array}{ccc}\hfill h_1\left(\nu \right)& :=& 〈{\kappa }_2\widehat{\Sigma },\Lambda 〉+{({\Lambda }^{\top }\widehat{\mu }+q)}^{\top }\widehat{\mu }+{\tau }_1{\parallel x\parallel }_1+{\tau }_2\alpha ,\hfill \\ \hfill h_2(\nu ,\xi )& :=& \mathbf{F}(x,\xi )-{({\Lambda }^{\top }\xi +q)}^{\top }\xi +{\displaystyle \frac{{\tau }_2}{1-\beta }}{\left[\mathcal{l}(x,\xi )-\alpha \right]}_{+},\hfill \\ \hfill \widehat{K}(x,\alpha ,\xi )& :=& \mathbf{F}(x,\xi )+{\displaystyle \frac{{\tau }_2}{1-\beta }}{\left[\mathcal{l}(x,\xi )-\alpha \right]}_{+}+{\tau }_1{\parallel x\parallel }_1+{\tau }_2\alpha .\hfill \end{array}$$

(4)

Below we will reformulate the “Min-Max-Min” problem (2) as the following non-smooth SDP:

$$\begin{array}{c}\hfill \underset{\nu \in \mathcal{V}}{min}\{\phi \left(\nu \right):=\underset{\xi \in \Xi }{max}h(\nu ,\xi )\},\mathrm{where}h(\nu ,\xi ):=h_1\left(\nu \right)+\sqrt{{\kappa }_1}\parallel {\widehat{\Sigma }}^{{\displaystyle \frac{1}{2}}}(q+2\Lambda \widehat{\mu })\parallel +h_2(\nu ,\xi ).\end{array}$$

(5)

Theorem 1

(Non-smooth SDP reformulation of SDRPC). Consider the SDRPC problem (2) and its associated non-smooth SDP problem (5). Then the optimal values for (2) and (5) are equal, in the sense that x is an optimal solution of (2) if and only if there exists $\nu \in \mathcal{V}$, which is an optimal solution of (5).

Proof. 

Based on (4), problem (2) is equivalent to the following problem:

$$\begin{array}{c}\hfill \underset{x\in {\Delta }_d}{min}\underset{P\in \mathcal{P}}{max}\underset{\alpha \in \mathbb{R}}{min}{\int }_{\Xi }\widehat{K}(x,\alpha ,\xi )P\left(d\xi \right).\end{array}$$

(6)

Since $\Xi$ is a compact set in (3), according to Section 2.2 in [21] and the boundedness of probability measures on a compact set, the set of all probability measures on $(\Xi ,\mathbb{B})$ is compact under the topology of weak convergence. Then the minimax theorem, Theorem 4.2 in [23], is applicable to the $\underset{P\in \mathcal{P}}{max}\underset{\alpha \in \mathbb{R}}{min}$ expression in (6). This is due to the convexity of the function $\widehat{K}(x,\alpha ,P)$ with respect to $\alpha$ (stemming from the convexity of linear and plus functions) and the concavity (indeed, linearity) with respect to P, along with the compactness of $\mathcal{P}$. Then, interchanging the $\underset{P\in \mathcal{P}}{max}$ and $\underset{\alpha \in \mathbb{R}}{min}$ operators in (6) leads to an equivalent form:

$$\begin{array}{c}\hfill \underset{x\in {\Delta }_d}{min}\underset{\alpha \in \mathbb{R}}{min}\underset{P\in \mathcal{P}}{max}{\int }_{\Xi }\widehat{K}(x,\alpha ,\xi )P\left(d\xi \right).\end{array}$$

(7)

We start by transforming the inner maximization problem of (7):

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{P\in \mathcal{M}}{max}& {\mathbb{E}}_P\left[\widehat{K}(x,\alpha ,\xi )\right]\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathbb{E}}_P\left[\begin{array}{cc}-\widehat{\Sigma }\hfill & \hfill \widehat{\mu }-\xi \\ {(\widehat{\mu }-\xi )}^{\top }\hfill & \hfill -{\kappa }_1\end{array}\right]⪯0,\hfill \\ \hfill & {\mathbb{E}}_P\left[(\xi -\widehat{\mu }){(\xi -\widehat{\mu })}^{\top }\right]⪯{\kappa }_2\widehat{\Sigma },\hfill \\ \hfill & {\mathbb{E}}_P\left[1\right]=1,\hfill \end{array}\end{array}$$

(8)

which uses the ambiguity set (3). From the definition (4) of $\widehat{K}(x,\alpha ,\xi )$, the Lagrange function of (8) is

$$\begin{array}{ccc}& & \int \mathbf{F}(x,\xi )-{({\Lambda }^{\top }\xi -2\widehat{q})}^{\top }\xi +2{\widehat{\mu }}^{\top }\Lambda \xi +{\displaystyle \frac{{\tau }_2}{1-\beta }}{\left[\mathcal{l}(x,\xi )-\alpha \right]}_{+}-rdP\left(\xi \right)\hfill \\ & & +〈{\kappa }_2\widehat{\Sigma },\Lambda 〉-{({\Lambda }^{\top }\widehat{\mu }+2\widehat{q})}^{\top }\widehat{\mu }+{\tau }_1{\parallel x\parallel }_1+{\tau }_2\alpha +r+〈\widehat{\Sigma },Q〉+{\kappa }_{1}s.\hfill \end{array}$$

Here, the parameters $r\in \mathbb{R}$ and $\Lambda \in {\mathbb{R}}^{m\times m}$ are the dual variables for the last and second constraints of (8), respectively. And $Q\in {\mathbb{R}}^{m\times m},\widehat{q}\in {\mathbb{R}}^m$ and $s\in \mathbb{R}$ together form a matrix consisting of the dual variables for the first constraint of (8). Then the Lagrange dual variable of (8) takes the following form:

$$\begin{array}{ccc}\hfill \underset{r,\Lambda ,Q,\widehat{q},s}{min}& & 〈{\kappa }_2\widehat{\Sigma },\Lambda 〉-{({\Lambda }^{\top }\widehat{\mu }+2\widehat{q})}^{\top }\widehat{\mu }+{\tau }_1{\parallel x\parallel }_1+{\tau }_2\alpha +r+〈\widehat{\Sigma },Q〉+{\kappa }_{1}s\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& & \mathbf{F}(x,\xi )-{({\Lambda }^{\top }\xi -2\widehat{q})}^{\top }\xi +2{\widehat{\mu }}^{\top }\Lambda \xi +{\displaystyle \frac{{\tau }_2}{1-\beta }}{\left[\mathcal{l}(x,\xi )-\alpha \right]}_{+}-r\le 0,\forall \xi \in \Xi ,\hfill \end{array}$$

(9)

$$\begin{array}{ccc}& & r\in \mathbb{R},\Lambda ⪰0,\left[\begin{array}{cc}Q& \widehat{q}\\ {\widehat{q}}^{\top }& s\end{array}\right]⪰0.\hfill \end{array}$$

(10)

Similarly to the procedure used in the proof of Lemma 1 in [4], we can simplify the dual problem by analytically solving for the variables $(Q,\widehat{q},s)$ while keeping $(\Lambda ,r)$ fixed. For the sake of completeness, we briefly outline these steps below. In view of the semi-definite constraint (10), we consider two cases for the variable $s^{*}$: either $s^{*}=0$ or $s^{*}>0$. Let us first consider the case of $s^{*}=0$. In this scenario, if ${\widehat{q}}^{*}\ne 0$, this would lead to ${\widehat{q}}^{*\top }{\widehat{q}}^{*}>0$ and

$${\left[\begin{array}{c}{\widehat{q}}^{*}\\ y\end{array}\right]}^{\top }\left[\begin{array}{cc}{Q}^{*}& {\widehat{q}}^{*}\\ {\widehat{q}}^{*\top }& s^{*}\end{array}\right]\left[\begin{array}{c}{\widehat{q}}^{*}\\ y\end{array}\right]={\widehat{q}}^{*\top }{Q}^{*}{\widehat{q}}^{*}-2{\widehat{q}}^{*\top }{\widehat{q}}^{*}y<0,\mathrm{for}y>{\displaystyle \frac{{\widehat{q}}^{*\top }{Q}^{*}{\widehat{q}}^{*}}{2{\widehat{q}}^{*\top }{\widehat{q}}^{*}}},$$

which contradict Equation (10). Therefore, it must be the case that ${\widehat{q}}^{*}=0$. Furthermore, given that $\widehat{\Sigma }\succ 0$, $Q⪰0$ and the objective is to minimize $〈\widehat{\Sigma },Q〉$, we can conclude that $Q^{*}=0$. Let us consider the case when $s^{*}>0$. According to the Schur complement of Section A.5.5 in [22], (10) is equivalent to $Q⪰{\displaystyle \frac{1}{s}}\widehat{q}{\widehat{q}}^{\top }$. Since $\widehat{\Sigma }\succ 0$ and the objective is to minimize $〈\widehat{\Sigma },Q〉$, we can deduce that $Q^{*}={\displaystyle \frac{1}{s}}\widehat{q}{\widehat{q}}^{\top }$, $s^{*}=arg\underset{s>0}{min}{\displaystyle \frac{1}{s}}{\widehat{q}}^{\top }\widehat{\Sigma }\widehat{q}+{\kappa }_{1}s={\displaystyle \frac{\parallel {\widehat{\Sigma }}^{\frac{1}{2}}\widehat{q}\parallel }{\sqrt{{\kappa }_1}}}$, and $〈\widehat{\Sigma },Q^{*}〉+{\kappa }_1{s}^{*}=\sqrt{{\kappa }_1}∥{\widehat{\Sigma }}^{1/2}\widehat{q}∥$. For the above two cases, after substituting $q=-2\left(\widehat{q}+\Lambda \widehat{\mu }\right)$, the Lagrange dual Formulas (9) and (10) of (8) simplify to

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{r,\Lambda ,q}{min}& r+〈{\kappa }_2\widehat{\Sigma },\Lambda 〉+{({\Lambda }^{\top }\widehat{\mu }+q)}^{\top }\widehat{\mu }+{\tau }_1{\parallel x\parallel }_1+{\tau }_2\alpha +\sqrt{{\kappa }_1}∥{\widehat{\Sigma }}^{1/2}\left(q+2\Lambda \widehat{\mu }\right)∥\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \mathbf{F}(x,\xi )-{({\Lambda }^{\top }\xi +q)}^{\top }\xi +{\displaystyle \frac{{\tau }_2}{1-\beta }}{\left[\mathcal{l}(x,\xi )-\alpha \right]}_{+}-r\le 0,\forall \xi \in \Xi ,\hfill \\ & r\in \mathbb{R},q\in {\mathbb{R}}^m,\Lambda \in {\mathbb{S}}_{+}^m.\hfill \end{array}\end{array}$$

(11)

Following Shapiro’s duality theory for moment problems, Proposition 3.4 in [24] and Equation (2.3) in [21], the Slater-type condition of (8) can be written as

$$\begin{array}{c}\hfill (1,0,0)\in \mathrm{int}\{\left(〈P,1〉,〈P,\left(\begin{array}{cc}-\widehat{\Sigma }\hfill & \hfill \widehat{\mu }-\xi \\ {(\widehat{\mu }-\xi )}^{\top }\hfill & \hfill -{\kappa }_1\end{array}\right)〉,〈P,(\xi -\widehat{\mu }){(\xi -\widehat{\mu })}^{\top }-{\kappa }_2\widehat{\Sigma }〉\right)\\ \hfill +\left\{0\right\}\times {\mathbb{S}}_{+}^{m+1}\times {\mathbb{S}}_{+}^m|P\in \mathcal{M}\},\end{array}$$

(12)

where $〈P,\Psi \left(\xi \right)〉={\int }_{\Xi }\Psi \left(\xi \right)P\left(d\xi \right)$. From Example 2.3 in [21], the moment constraints (3) satisfy the Slater-type condition (12). Then the equivalence between problems (8) and (11) holds. Since the support set $\Xi$ is compact, we know that the optimal value of (8) is finite. According to Proposition 3.4 in [24], if the common optimal value of the primal problem and the dual problem is finite, then the set of optimal solutions to the dual problem is nonempty. Consequently, we deduce that the set of optimal solutions to the dual problem (11) is nonempty and bounded.

Given any fixed x and $\alpha$, we obtain the equivalence between (8) and its dual problem (11) from the paragraph above. We now return to discuss semi-infinite programming (11). The main difficulty in solving a semi-infinite programming problem comes from infinitely many constraints since there are infinitely many values of $\xi$ in the sample space $\Xi$. We rewrite $h_2(\nu ,\xi )\le r,\forall \xi \in \Xi$, for $\underset{\xi \in \Xi }{max}\left\{h_2(\nu ,\xi )\right\}\le r$. Then the Lagrange dual problem (11) is equivalent to

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{q,\Lambda }{min}& \underset{\xi \in \Xi }{max}h(\nu ,\xi )\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& q\in {\mathbb{R}}^m,\Lambda \in {\mathbb{S}}_{+}^m.\hfill \end{array}\end{array}$$

We use the fact that Min-Min operators can be performed jointly. This leads to an equivalent formulation (5) for SDRPC.    □

Using similar arguments to those used for tractable reformulation in Lemma 1 of [4] by Delage and Ye, we equivalently transform (8) to a semi-infinite problem (11) through the Lagrange dual equation and then use a strategy that further simplifies this dual problem by solving analytically for the ${(m+1)}^2$ dual variables corresponding to the first constraint in (8) using an auxiliary vector q of m variables while keeping the other dual variables. In fact, this strategy reduces $m^2+m+1$ unknown variables, at the expense of adding a non-smooth term $\parallel ·\parallel$ to the objective function.

So far, we have transformed the SDRPC index tracking model (2) and equivalently reformulated it as the SDP in (5), as shown in Theorem 1. Since ${\Delta }_d$ and $\mathcal{P}$ are bounded and the objective function in (2) is continuous, we know that the solution set of the original problem (2) is nonempty and bounded. Using Theorem 1, the existence of optimal solutions to the SDP (5) is guaranteed.

3. The Discretization Scheme

For the purpose of computation, we provide a discretization model for the equivalent tractable reformulation. We show the existence of solutions for the discretization model under mild conditions. The convergence results of the optimal values and solutions of the discretization scheme to those of the original equivalent reformulation are also given under mild assumptions.

In what follows, we consider the discrete approximation of (5). Our first step is to develop a discrete approximation of the continuous support set $\Xi$. Let ${\Xi }_{\left[N\right]}=\{{\xi }^1,\dots ,{\xi }^N\}$ be the independent and identically distributed (i.i.d.) samples of $\xi$ drawn by Monte Carlo sampling from the set $\Xi$. We consider the discretization scheme of (5):

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \underset{\nu \in \mathcal{V}}{min}\{{\phi }^N\left(\nu \right):=\underset{\xi \in {\Xi }_{\left[N\right]}}{max}h(\nu ,\xi )\},\end{array}\end{array}$$

(13)

where h is defined in (5). Then the corresponding approximation to ${\varphi }_{\beta }\left(x\right)$ in (1) is

$$\begin{array}{c}\hfill {\varphi }_{\beta }^N\left(x\right)=\underset{\alpha \in \mathbb{R}}{min}\left\{\alpha +{\displaystyle \frac{1}{(1-\beta )N}}\sum _{i=1}^N{\left[\mathcal{l}(x,{\xi }^i)-\alpha \right]}_{+}\right\}.\end{array}$$

(14)

Let $\mathbb{A}$ be a compact set consisting of the values of $\alpha$ for which the minimum in ${\varphi }_{\beta }^N\left(x\right)$ is attained. We respectively denote the optimal values of (13) and (5) as ${\widehat{\vartheta }}_N$ and $\widehat{\vartheta }$. In Theorem 2, we state the convergence of problem (13) to problem (5) in terms of the optimal value.

Theorem 2.

The non-smooth model in (13) is convex and the solution set of (13) is nonempty. Suppose that the optimal value of (8) is finite, and let ${\xi }^1,\dots ,{\xi }^N$ be i.i.d. samples of ξ, with a continuous probability distribution P over Ξ such that

$$\begin{array}{c}\hfill P\left(∥\xi -{\xi }^0∥<\delta \right)\ge C_2{\delta }^{{\gamma }_2}\end{array}$$

(15)

for any fixed point ${\xi }^0\in \Xi$ and $\delta \in \left(0,{\delta }_0\right)$, where $C_2$, ${\gamma }_2$ and ${\delta }_0$ are some positive constants. When N is sufficiently large, for any positive number ε, there exist positive constants $\widehat{C}\left(\epsilon \right)$ and $\widehat{\beta }\left(\epsilon \right)$ such that

$$\begin{array}{c}\hfill \mathrm{Prob}\left(|{\widehat{\vartheta }}_N-\widehat{\vartheta }|\ge \epsilon \right)\le \widehat{C}\left(\epsilon \right)e^{-\widehat{\beta }\left(\epsilon \right)N}.\end{array}$$

(16)

Proof. 

For any $\lambda \in (0,1)$, $q^1,q^{\left(2\right)}\in {\mathbb{R}}^m$, and ${\Lambda }^{\left(1\right)},{\Lambda }^{\left(2\right)}\in {\mathbb{S}}_{+}^m$, we have, through direct computation and the convexity of $\parallel ·\parallel$, that

$$\begin{array}{cc}& \sqrt{{\kappa }_1}∥{\widehat{\Sigma }}^{{\displaystyle \frac{1}{2}}}\left((\lambda q^{\left(1\right)}+(1-\lambda )q^{\left(2\right)})+2({\lambda }_1{\Lambda }^{\left(1\right)}+(1-\lambda ){\Lambda }^{\left(2\right)})\widehat{\mu }\right)∥\\ & \le \lambda \sqrt{{\kappa }_1}∥{\widehat{\Sigma }}^{{\displaystyle \frac{1}{2}}}(q^1+2{\Lambda }^{\left(1\right)}\widehat{\mu })\parallel +(1-\lambda )\sqrt{{\kappa }_1}\parallel {\widehat{\Sigma }}^{{\displaystyle \frac{1}{2}}}(q^{\left(2\right)}+2{\Lambda }^{\left(2\right)}\widehat{\mu })∥.\end{array}$$

Therefore, using the definition of a convex function, we can show without difficulty that $h(\nu ,\xi )$ in (5) with respect to $\nu$ is a convex function. From Proposition 1.38 in [25], the maximum of a finite number of convex functions is also convex. In other words, ${\phi }^N\left(\nu \right)$ is convex in $\nu$. Moreover, the feasible set $\mathcal{V}$ is a convex set. Hence, the non-smooth model (13) is convex.

From (14), we see that if the sampling generates a collection of vectors ${\xi }^1,\dots ,{\xi }^N$, then the maximum with respect to $\alpha$ in (14) is achieved at a finite $\alpha$. In other words, we may restrict the maximum with respect to $\alpha$ to be taken within a closed interval $[-c,c]$ for some sufficiently large positive number c; see [9,21]. Let $\mathbb{A}$ be a compact set consisting of the values of $\alpha$ for which the minimum in ${\varphi }_{\beta }^N\left(x\right)$ is attained. From the compactness of ${\Delta }_d$ and ${\Xi }_{\left[N\right]}$, there exist $x^{*}\in {\Delta }_d$ and ${\xi }^{*}\in {\Xi }_{\left[N\right]}$ as the optimal solutions of (13). Then, the solution set of (13) is nonempty. We notice the following points:

(i) Denote by

$$M_{\nu }\left(t\right):=\mathbb{E}\left[e^{t\left(h\right(\nu ,\xi )-\mathbb{E}[h(\nu ,\xi )\left]\right)}\right]$$

the moment-generating function of the random variable $h(\nu ,\xi )-\mathbb{E}\left[h\right(\nu ,\xi \left)\right]$. Since $\Xi$ is a compact set and from Section 3.1 in [21], for each $\nu \in \mathcal{V}$, $\underset{\xi \in \Xi }{sup}h(\nu ,\xi )<\infty$ and the moment-generating function $M_{\nu }\left(t\right)$ is finitely valued for all t in the neighborhood of zero.

(ii) Through the continuity of the function h, we can establish the existence of a nonnegative measurable function $\kappa :\Xi \to {\mathbb{R}}_{+}$ and a constant $\gamma >0$ such that for all $\xi \in \Xi$,

$$\begin{array}{c}\hfill \left|h\left({\nu }^{\prime },\xi \right)-h\left({\nu }^{\prime \prime },\xi \right)\right|\le \kappa \left(\xi \right){∥{\nu }^{\prime }-{\nu }^{\prime \prime }∥}^{\gamma },\forall {\nu }^{\prime },{\nu }^{\prime \prime }\in \mathcal{V}.\end{array}$$

(iii) Considering the boundedness of the support set $\Xi$ and Section 5 in [26], we can conclude that the moment-generating function $M_{\kappa }\left(t\right)$ of $\kappa \left(\xi \right)$ is finite for all t in the neighborhood of zero.

The above facts, (i)–(iii), along with (15) and the continuity of $h(\nu ,·)$ over $\Xi$ enable us to conclude that the relationship between the optimal values of (13) and (5), i.e., (16), holds true, as indicated by Lemma 3.1 (i) in [21].    □

Remark 1.

From Proposition 1 in [12], condition (15) is very weak: it can be guaranteed whenever the density of ξ is bounded below by a positive real-valued analytic function. It is easily seen to hold when the density is bounded away from zero around ${\xi }^0$, and so it is an effective condition.

4. The Approximate Discretization Algorithm

The approximate discretization scheme in Section 3 can be solved using either a subgradient algorithm [27,28,29] or a smoothing projected-gradient algorithm [30,31,32]. In this section, we use the subgradient algorithm to solve (13), i.e., the discretization scheme of the equivalent reformulation. To do this, the subgradient algorithm uses the simple iteration

$$\begin{array}{c}\hfill {\nu }_{k+1}=P_{\mathcal{V}}({\nu }_k-{\alpha }_k{g}_k).\end{array}$$

(17)

Here ${\nu }_k$ is the k-th iterate, $g_k$ is any subgradient of ${\phi }^N$ at ${\nu }_k$, $P_{\mathcal{V}}$ is the projected operator, and ${\alpha }_k>0$ is the k-th step size.

Now we are ready to provide a basic scheme for the approximate discretization (AD) algorithm in Algorithm 1.

Algorithm 1 The approximate discretization (AD) algorithm for DRO

Step 0.1. Set $\mathbf{F}(x,\xi )$ in (2) and $\mathcal{l}(x,\xi )$ in (1). Step 0.2. Interchange the $\underset{P\in \mathcal{P}}{max}$ and $\underset{\alpha \in \mathbb{R}}{min}$ operators in (6), leading to an equivalent form (7). Step 0.3. Calculate the Lagrange dual variable (11) of the inner maximization problem (8) and obtain the non-smooth SDP (5). Step 0.4. Set ${\Xi }_{\left[N\right]}=\{{\xi }^1,\dots ,{\xi }^N\}$ as i.i.d. samples of $\xi$ drawn by Monte Carlo sampling from the set $\Xi$. Step 0.5. Set the discretization scheme of (5), i.e., (13). Step 0.6. Set the initial point ${\nu }_0\in \mathcal{V}$ and the step size $\{{\alpha }_k>0\}$ for any $k\ge 1$. Step 1.   For $k\ge 1$, call the subgradient algorithm (17) m times to obtain $${\nu }_{k+1}=P_{\mathcal{V}}({\nu }_k-{\alpha }_k{g}_k),k=1,\dots ,m-1.$$ Output: ${\nu }_m$

Compared to the smoothed projected-gradient method, the subgradient method offers several distinct advantages. First, it eliminates the need to update a sequence of smoothing parameters, thereby reducing the number of parameters in the algorithm. Second, for complex non-smooth functions, constructing a suitable smoothed approximation often results in intricate formulations, leading to significantly increased computational costs in the smoothed projected-gradient method. In contrast, the subgradient method avoids such overhead. Third, the subgradient method allows for a variety of step size strategies, including both adaptive step size rules and fixed step size schemes, providing greater flexibility in practical implementation.

Remark 2.

In this paper, we focus on a flexible framework by requiring the loss function $\mathcal{l}(x,\xi )$ in CVaR formulation to be convex, but without imposing any specific form or structure on $\mathcal{l}(x,\xi )$. This relaxed assumption allows us to consider a wide range of loss functions, making our model more adaptable to various applications where the exact form of the loss function may not be known in advance. Similarly, we do not place any restrictive conditions on the convex continuous function $\mathbf{F}(x,\xi )$ in the optimization problem (2). By allowing this general form of $\mathbf{F}(x,\xi )$, we further extend the applicability of our SDRPC model. This flexibility ensures that the model can accommodate various types of objective functions and constraints that may arise in different settings, such as portfolio optimization, resource allocation, and risk management.

Remark 3.

Although the proposed AD algorithm is developed and illustrated in the context of the CVaR penalty, its applicability is not limited to this specific regularizer. In fact, the reformulation and discretization techniques presented in this work are valid for a broad class of non-smooth convex regularization terms, such as the ${\mathcal{l}}_2$-norm, quantile loss, and Huber loss. As long as the involved penalty term remains convex (possibly non-smooth), the equivalence transformation and convergence guarantees established in this paper remain applicable. This generality significantly improves the scope and versatility of the AD algorithm.

5. Applications and Numerical Results

5.1. Applications

The SDRPC model (2) is broadly applicable across various domains due to the versatility in selecting $\mathbf{F}(x,\xi )$ and $\mathcal{l}(x,\xi )$ in ${\varphi }_{\beta }\left(x\right)$ (1). Below, we illustrate three practical examples that arise from different choices of these functions.

Example 1

(Lasso sparse index tracking problem). Index tracking aims to replicate the index of a financial market by constructing a portfolio consisting of assets in that market that minimizes the tracking error, which measures how closely the portfolio mimics the performance of the benchmark. Let ${\xi }_B=({\xi }_{B,1},\dots ,{\xi }_{B,d})\in {\mathbb{R}}^d$ be the observed historical individual return vector of the d assets and ${\xi }_a\in \mathbb{R}$ be the observed corresponding random market index return for $j=1,\dots ,N$. We denote this by $\xi ={({{\xi }_B}^T,{\xi }_a)}^T\in {\mathbb{R}}^{d+1}$. Let $x={(x_1,\dots ,x_d)}^T\in {\mathbb{R}}^d$ be the tracking portfolio, with $x_i$ being the investment weight in the ith component stock. The risk-averse variant of the Lasso sparse index tracking problem in [33] can be formulated as

$$\begin{array}{c}\hfill \underset{x\in {\Delta }_d}{min}\underset{P\in \mathcal{P}}{max}{\mathbb{E}}_P\left[{\left({\xi }_a-{{\xi }_B}^{\top }x\right)}^2\right]+{\tau }_1{\parallel x\parallel }_1+{\tau }_2{\varphi }_{\beta }\left(x\right),\end{array}$$

where $\mathcal{l}(x,\xi )=-{{\xi }_B}^{\top }x$ in ${\varphi }_{\beta }\left(x\right)$, ${\tau }_1$ and ${\tau }_2$ are given regularization parameters, and the ${\mathcal{l}}_1$-norm penalty aims to enhance the sparsity of the portfolio. Since short selling is prohibited, nonnegativity constraints are enforced on the decision variables.

Example 2

(Multiproduct newsvendor problem [34]). Assume that a newsvendor trades in $i=1,\dots ,d$ products. Before observing the uncertain demands ${\xi }_i$, the newsvendor orders $x_i$ units of product i at the wholesale price $c_i$. Once ${\xi }_i$ is observed, she can sell the quantity $min\left(x_i,{\xi }_i\right)$ at the retail price $v_i$. Any unsold stock ${\left(x_i-{\xi }_i\right)}_{+}$ is cleared at the salvage price $g_i$, and any unsatisfied demand ${\left({\xi }_i-x_i\right)}_{+}$ is lost. We study the risk-averse variant of the multiproduct newsvendor problem with $\mathbf{F}(x,\xi )=U\left(\mathcal{l}\right(x,\xi \left)\right)$, where $U\left(y\right):=e^{y/10}$ is an exponential disutility function and

$$\mathcal{l}(x,\xi )={(c-v)}^{T}x+{(v-g)}^T{(x-\xi )}_{+}.$$

Example 3

(Portfolio optimization problem). We consider a portfolio optimization problem [21] where the investor makes an optimal decision using historical return rates from the National Association of Securities Dealers Automated Quotations (NASDAQ) index: https://cn.investing.com. To simplify the discussions, we ignore the transaction fee; therefore, the total value of the portfolio is $\mathbf{F}(x,\xi )=-{\xi }^{T}x$ in (2) and $\mathcal{l}(x,\xi )=-{\xi }^{T}x$ in (1).

5.2. Numerical Results

We consider the portfolio optimization problem in Example 3, where the investor makes an optimal decision using a historical return rate of $d=40$ between January 2005 and July 2023 from NASDAQ, which contains $N_{\mathrm{tol}}=4675$ samples; that is, $q=4675$ in (13). We denote the daily return rates of n assets on the i-th day as ${\xi }_{B,i}={({\xi }_{B_1,i},{\xi }_{B_2,i},\dots ,{\xi }_{B_n,i})}^T$, where ${\xi }_{B_j,i}$ represents the natural logarithm of the closing price divided by the opening price of the j-th asset on the i-th day for $i=1,\dots ,N_{\mathrm{tol}}$. Meanwhile, we set ${\kappa }_1=0.1$ and ${\kappa }_2=1.1$ in (3). All experiments are performed in Windows 11 on an AMD Ryzen 9 7900X 12-Core CPU at 4.70 GHz with 32 GB of RAM using MATLAB R2024b.

We systematically investigate the impact of parameters ${\tau }_1$ and ${\tau }_2$ on the performance of the proposed model (2) by conducting experiments over a grid of values $\{{10}^{-3},{10}^{-2},{10}^{-1}\}$.

Table 1. Obj and CPU time under different parameter settings.

${\mathit{\tau }}_1$	${\mathit{\tau }}_2$	Obj	CPU Time
${10}^{-3}$	${10}^{-3}$	0.2082	59.9984
${10}^{-3}$	${10}^{-2}$	0.3031	74.0531
${10}^{-3}$	${10}^{-1}$	0.5183	78.4453
${10}^{-2}$	${10}^{-3}$	0.2882	68.1875
${10}^{-2}$	${10}^{-2}$	0.3799	76.6968
${10}^{-2}$	${10}^{-1}$	0.5275	79.2125
${10}^{-1}$	${10}^{-3}$	0.4013	69.6593
${10}^{-1}$	${10}^{-2}$	0.4225	77.2062
${10}^{-1}$	${10}^{-1}$	0.6086	80.0687

indicates that increasing either ${\tau }_1$ or ${\tau }_2$ leads to a significant rise in the objective value (Obj). This trend suggests that both regularization terms, while enhancing sparsity and controlling risk, exert a suppressive effect on the optimal solution—especially when their magnitudes are large, resulting in strong penalization.

The minimum Obj is achieved at ${\tau }_1={10}^{-3}$ and ${\tau }_2={10}^{-3}$, while the maximum value occurs at ${\tau }_1={10}^{-1}$ and ${\tau }_2={10}^{-1}$, with an increase of nearly threefold in the objective function. This substantial degradation in performance confirms that the overly aggressive enforcement of sparsity and CVaR risk control can severely affect solution quality. This trend is visually corroborated by the bivariate heat map in Figure 1. Each grid corresponds to a specific $({\tau }_1,{\tau }_2)$ combination, with its associated Obj mapped to a color using a continuous colormap. The colormap is chosen such that blue corresponds to the lowest (best) Obj, while yellow represents the highest (worse) Obj.

A closer examination of Table 1 reveals that the effects of ${\tau }_1$ and ${\tau }_2$ are not symmetric. When ${\tau }_1$ is held fixed, increasing ${\tau }_2$ causes more drastic changes in the objective function, suggesting that the CVaR-related regularization term (${\tau }_2$) has a more sensitive and dominant influence on the optimization outcome in this context. To further analyze the effect of ${\tau }_1$, Figure 2 presents the objective trends as ${\tau }_1$ varies while ${\tau }_2$ is fixed.

Moreover, Figure 3 depicts the convergence trajectory of Obj with respect to CPU time under the settings ${\tau }_1={10}^{-3}$ and ${\tau }_2={10}^{-3}$. The curve indicates that the AD algorithm converges rapidly, with Obj reaching a relatively stable value within approximately 1.6 s. The algorithm continues running up to 60 s only due to the preset stopping criterion of 1000 iterations. Similar convergence behavior is observed under other parameter configurations, further confirming the stability and computational efficiency of our AD algorithm.

6. Conclusions

In this paper, we propose a sparse model that combines DRO and the CVaR penalty, where the CVaR penalty can be replaced by other convex functions. We transform the model to an equivalent non-smooth positive semi-definite type of programming, where the non-smoothness arises from the maximum of numerous non-smooth functions. An approximate discretization algorithm of the non-smooth positive semi-definite programming is provided, which is convergent under mild conditions. This discretization scheme of the equivalent reformulation can be solved directly by a subgradient method or a smoothing projected-gradient method. We propose an approximate discretization (AD) algorithm by combining a low-complexity subgradient method with the preceding transformation. Furthermore, numerical experiments on a portfolio optimization problem validate the effectiveness and efficiency of the AD algorithm in real-world scenarios.