Convergence Analysis on Data-Driven Fortet-Mourier Metrics with Applications in Stochastic Optimization

Abstract

Fortet-Mourier (FM) probability metrics are important probability metrics, which have been widely adopted in the quantitative stability analysis of stochastic programming problems. In this study, we contribute to different types of convergence assertions between a probability distribution and its empirical distribution when the deviation is measured by FM metrics and consider their applications in stochastic optimization. We first establish the quantitative relation between FM metrics and Wasserstein metrics. After that, we derive the non-asymptotic moment estimate, asymptotic convergence, and non-asymptotic concentration estimate for FM metrics, which supplement the existing results. Finally, we apply the derived results to four kinds of stochastic optimization problems, which either extend the present results to more general cases or provide alternative avenues. All these discussions demonstrate the motivation as well as the significance of our study.

1. Introduction

The estimation of the distance between a distribution and its empirical approximation obtained from some independent and identically distributed (iid) samples is an important subject in probability theory, mathematical statistics, and information theory. It has vast applications in many fields, such as quantization, optimal matching, density estimation, clustering, and so on (see [1] and the references therein for more details). To quantify the distance between two probability distributions, some rules have been adopted to generate probability metrics, such as the commonly used $\zeta$-structure metric. By selecting different generators of the $\zeta$-structure metric, we obtain a number of well-known probability metrics, such as the Wasserstein metric (in this study, when we refer to Wasserstein metric, it means the 1-Wasserstein metric, which is also called the Kantorovich–Rubinstein metric or Kantorovich metric), FM metric, and total variation metric.

Among probability metrics with $\zeta$-structure, the Wasserstein metric is the most popular one which has been widely applied in statistics, probability, and machine learning [2]. It originates from the optimal transportation problem and thus can be interpreted as an optimal mass transportation plan. Except for its practical meaning in transportation, the Wasserstein metric has some good properties. For example, convergence in the Wasserstein metric is equivalent to weak convergence plus the convergence of the first order absolute moment [2].

There is some literature concentrated on the convergence analysis under Wasserstein metrics between a distribution and its empirical approximation. From now on, we refer to this as the data-driven Wasserstein metric for simplicity, and other probability metrics do the same. These convergence analyses can be mainly divided into two parts: moment estimates which aim at providing the rate of convergence for the expectation of the Wasserstein distance between a distribution and its empirical approximation and concentration estimates which focus on the violation probability under a given tolerance. As for moment estimates, some earlier results can be found in [3,4], which provide a relatively loose convergence rate. More recently, Weed and Bach [5] focused on the compact support set case and obtained a sharp convergence rate. Dereich et al. [6] conducted the almost optimal convergence analysis. However, they put some restrictions on the range of parameters. An interesting result was given in [1], which extends some results in [6] from a limited range of parameters to the general case. As for concentration estimates, only a few results are available. The corresponding results can be found in [7,8] under some strong assumptions. Moreover, it requires that the violation parameter is large enough. In [9], Zhao and Guan investigated the case with a discrete and bounded support set. Particularly, an elaborate result on the rate of convergence of data-driven Wasserstein distance was presented in [1].

As pointed out in [2] (p. 110), the Wasserstein metric is a rather strong probability metric. Intuitively, it needs harsh conditions to establish the strong Wasserstein type upper bound estimate. Actually, we know from the definition of the Wasserstein metric that its generator is the set of Lipschitz continuous functions with Lipschitz modulus being one.

Compared with the Wasserstein metric, FM metrics are more general; their generator is a class of locally Lipschitz continuous functions. Therefore, it is more friendly to obtain some upper bounds by adopting FM metrics. In view of this, FM metrics have been widely used in the quantitative stability analysis of stochastic programming problems when the underlying probability distribution is perturbed and approximated, see for example [10,11,12,13]. Moreover, FM metrics have a close relationship with Wasserstein metrics through dual representation (Kantorovich–Rubinstein theorem). Generally, the pth order FM metric degrades into the Wasserstein metric when $p=1$. From this point of view, the FM metric can be viewed as an extension of the Wasserstein metric. Nevertheless, there are few results concerning the convergence analysis for data-driven FM metrics. To the best of our knowledge, only Strugarek [14] examined the asymptotic convergence analysis under the FM distance.

In view of the above situations, in this article we study the data-driven FM metric. The main contributions of this study can be summarized as follows:

• We establish the quantitative connection between the Wasserstein metric and the FM metric. Based on this connection, we investigate the non-asymptotic moment estimate, asymptotic convergence, and non-asymptotic concentration estimate for data-driven FM metrics.
• We provide an alternative avenue for the convergence analysis of discrete approximations for two-stage stochastic programming problems. Different from the convergence or exponential rate of convergence analysis in [15,16], where some complex conditions are required, our approach is straightforward and brief.
• We reestablish the quantitative stability results for stochastic optimization problems with stochastic dominance constraints through FM metrics. Compared with that in [17], our conditions are weaker and different probability metrics are adopted. More importantly, we can apply the convergence conclusion to examine the discrete approximation method which is crucial for numerical solution.
• We consider data-driven distributionally robust optimization (DRO) problems with FM ball, which extends the results in [18] from the ambiguity set constructed by Wasserstein ball to the FM ball case. We prove the finite sample guarantee and asymptotic consistency, which lay the theoretical foundation for the data-driven approach for the DRO model.
• We analyze the discrete approximation of the DRO problem whose ambiguity set is constructed with the general moment information. Compared with the existing work [19] under the bounded support set, we weaken their conditions and extend their results to the case with an unbounded support set.

The remainder of this study is organized as follows. In Section 2, we give some prerequisites for further discussion. In Section 3, we discuss different kinds of convergence results for data-driven FM metrics. We consider four applications to verify our convergence results and to further demonstrate the motivation and significance of this study in Section 4. Finally, we have some concluding remarks in Section 5.

2. Prerequisites

Let $\xi :\Omega \to \Xi \subseteq {\mathbb{R}}^s$ be a random vector defined on the probability space $(\Omega ,\mathcal{F},\mathbb{P})$. Then, its induced probability distribution (sometimes it is called probability measure) on $\Xi$ is $P:=\mathbb{P}◦{\xi }^{-1}$. We use $\mathcal{P}(\Xi )$ to denote all the probability distributions on $\Xi$. The set of probability distributions having finite pth order absolute moments is denoted by ${\mathcal{P}}_p(\Xi ):=\{P\in \mathcal{P}(\Xi ):{\int }_{\Xi }{∥\xi ∥}^{p}P\left(d\xi \right)<+\infty \}$.

Probability metrics measure the distance between two probability distributions. Generally, they do not satisfy the three axioms of usual distance in metric space. A commonly used class of probability metrics is the probability metric with $\zeta$-structure, whose definition is as follows.

Definition 1.

Let $\mathcal{G}$ be a set of measurable functions from Ξ to $\mathbb{R}$. Then, for any $P,Q\in \mathcal{P}(\Xi )$,

$${\mathbb{D}}_{\mathcal{G}}(P,Q):=\underset{g\in \mathcal{G}}{sup}\left|{\mathbb{E}}_P\left[g\left(\xi \right)\right]-{\mathbb{E}}_Q\left[g\left(\xi \right)\right]\right|$$

is called the ζ-structure probability metric induced by $\mathcal{G}$.

The $\mathcal{G}$ in Definition 1 totally determines the resulting $\zeta$-structure probability metric, so it is called the generator of the $\zeta$-structure probability metric. FM metrics and Wasserstein metrics can be deduced from the $\zeta$-structure probability metric by choosing specific generators. Particularly, we have the following definitions.

Definition 2.

Let $P,Q\in {\mathcal{P}}_p(\Xi )$ for some $p\ge 1$ and ${\mathcal{G}}_{{\mathrm{FM}}_p}$ denote a set of locally Lipschitz continuous functions given by

$$\left\{g:\Xi \to \mathbb{R}:|g\left({\xi }_1\right)-g\left({\xi }_2\right)|\le max{\{1,∥{\xi }_1∥,∥{\xi }_2∥\}}^{p-1}∥{\xi }_1-{\xi }_2∥,\forall {\xi }_1,{\xi }_2\in \Xi \right\}.$$

Then, the pth order FM metric between P and Q is

$${\zeta }_p(P,Q):=\underset{g\in {\mathcal{G}}_{{\mathrm{FM}}_p}}{sup}\left|{\mathbb{E}}_P\left[g\left(\xi \right)\right]-{\mathbb{E}}_Q\left[g\left(\xi \right)\right]\right|.$$

Definition 3.

Let $P,Q\in {\mathcal{P}}_1(\Xi )$ and

$${\mathcal{G}}_W:=\left\{g:\Xi \to \mathbb{R}:|g\left({\xi }_1\right)-g\left({\xi }_2\right)|\le ∥{\xi }_1-{\xi }_2∥,\forall {\xi }_1,{\xi }_2\in \Xi \right\}.$$

Then, the Wasserstein metric between P and Q is

$${\mathbb{D}}_W(P,Q):=\underset{g\in {\mathcal{G}}_W}{sup}\left|{\mathbb{E}}_P\left[g\left(\xi \right)\right]-{\mathbb{E}}_Q\left[g\left(\xi \right)\right]\right|.$$

It is easy to see from the above definitions that ${\zeta }_1(P,Q)={\mathbb{D}}_W(P,Q)$ for any $P,Q\in \mathcal{P}(\Xi )$. Moreover, we have that: if $g\in {\mathcal{G}}_{{\mathrm{FM}}_p}$, then, $-g\in {\mathcal{G}}_{{\mathrm{FM}}_p}$, so does ${\mathcal{G}}_W$. Therefore, we can ignore the absolute value operator in Definitions 2 and 3 when we take supremum. Moreover, both FM metrics and Wasserstein metrics have a close relationship with weak convergence. One can refer to [10] (p. 490) and [2] (Theorem 6.9) for more details.

The Wasserstein metric has an alternative definition which corresponds to the coupling marginal distributions. Specifically, the Wasserstein metric between P and Q is defined as (see [2], Definition 6.1):

$${\mathbb{D}}_W(P,Q)=inf\left\{{\int }_{\Xi \times \Xi }∥{\xi }_1-{\xi }_2∥\pi (d{\xi }_1,d{\xi }_2):\pi \in \Pi \right\},$$

(1)

where $\Pi$ is the collection of all joint distributions of ${\xi }_1$ and ${\xi }_2$ with marginal distributions P and Q, respectively. It is known from Kantorovich–Rubinstein theorem [20] that Definition 3 is the dual representation of (1).

We have the following extension theorem for Lipschitz functions in Hilbert space (see [21], Theorems 4 and 5).

Lemma 1.

Let X and Y be Hilbert spaces and $g:B\subseteq X\to Y$ be a Lipschitz function with Lipschitz modulus $L_g$. Then, there exists a Lipschitz function $\widehat{g}:X\to Y$ such that $\widehat{g}\left(x\right)=g\left(x\right)$ for any $x\in B$ and $L_g$ is also the Lipschitz modulus of $\widehat{g}$.

Lemma 1 is important for the following discussion. In [1], the authors assumed that the support set $\Xi$ is the whole space ${\mathbb{R}}^s$. They obtained the non-asymptotic moment estimate [1] (Theorem 1) and concentration estimate [1] (Theorem 2) for the Wasserstein metric. For any $P,Q\in \mathcal{P}(\Xi )$, we can view them as probability distributions $\tilde{P},\tilde{Q}\in \mathcal{P}\left({\mathbb{R}}^s\right)$ through the following correspondence:

$$\tilde{P}\left(A\right):=P(A\cap \Xi )\mathrm{and}\tilde{Q}\left(A\right):=P(Q\cap \Xi )$$

for all $A\subseteq {\mathbb{R}}^s$. That is, we set the probability of the area ${\mathbb{R}}^s\setminus \Xi$ to be zero. Generally, we have ${\mathbb{D}}_W(P,Q)={\mathbb{D}}_W(\tilde{P},\tilde{Q})$. The details are as follows:

$$\begin{array}{cc}\hfill {\mathbb{D}}_W(P,Q)& =\underset{g\in {\mathcal{G}}_W(\Xi )}{sup}\left|{\int }_{\Xi }g\left(\xi \right)(P-Q)\left(d\xi \right)\right|\hfill \\ \hfill & =\underset{\widehat{g}\in {\widehat{\mathcal{G}}}_W\left({\mathbb{R}}^s\right)}{sup}\left|{\int }_{{\mathbb{R}}^s}\widehat{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)\right|,\hfill \end{array}$$

where ${\mathcal{G}}_W(\Xi )$ denotes the collection of all the Lipschitz continuous functions with Lipschitz modulus 1 on $\Xi$, and ${\widehat{\mathcal{G}}}_W\left({\mathbb{R}}^s\right)$ is the extension of ${\mathcal{G}}_W(\Xi )$ according to Lemma 1. Obviously, ${\widehat{\mathcal{G}}}_W\left({\mathbb{R}}^s\right)\subseteq {\mathcal{G}}_W\left({\mathbb{R}}^s\right)$ which is the set of Lipschitz continuous functions with Lipschitz modulus 1 over ${\mathbb{R}}^s$. Thus, we have the estimation

$$\begin{array}{cc}\hfill \underset{\widehat{g}\in {\widehat{\mathcal{G}}}_W\left({\mathbb{R}}^s\right)}{sup}\left|{\int }_{{\mathbb{R}}^s}\widehat{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)\right|& \le \underset{\overline{g}\in {\mathcal{G}}_W\left({\mathbb{R}}^s\right)}{sup}\left|{\int }_{{\mathbb{R}}^s}\overline{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)\right|\hfill \\ \hfill & ={\mathbb{D}}_W(\tilde{P},\tilde{Q}).\hfill \end{array}$$

That is, ${\mathbb{D}}_W(P,Q)\le {\mathbb{D}}_W(\tilde{P},\tilde{Q})$.

On the other hand, for any $\overline{g}\in {\mathcal{G}}_W\left({\mathbb{R}}^s\right)$, its restriction on $\Xi$ is Lipschitz continuous with Lipschitz modulus 1. Thus,

$$\begin{array}{cc}\hfill \underset{\overline{g}\in {\mathcal{G}}_W\left({\mathbb{R}}^s\right)}{sup}\left|{\int }_{{\mathbb{R}}^s}\overline{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)\right|& =\underset{\overline{g}\in {\mathcal{G}}_W\left({\mathbb{R}}^s\right)}{sup}\left|{\int }_{\Xi }\overline{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)+{\int }_{{\Xi }^c}\overline{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)\right|\hfill \\ \hfill & =\underset{\overline{g}\in {\mathcal{G}}_W\left({\mathbb{R}}^s\right)}{sup}\left|{\int }_{\Xi }\overline{g}\left(\xi \right)(\tilde{P}-\tilde{Q})\left(d\xi \right)\right|\hfill \\ \hfill & \le {\mathbb{D}}_W(P,Q).\hfill \end{array}$$

Finally, we have ${\mathbb{D}}_W(P,Q)={\mathbb{D}}_W(\tilde{P},\tilde{Q})$. Therefore, although all the convergence results in [1] were derived under ${\mathbb{R}}^s$, we can extend them to any support set $\Xi \subseteq {\mathbb{R}}^s$.

Lemma 2

([1], Theorem 1). Let $P\in {\mathcal{P}}_p(\Xi )$ for some $p>1$. Then, there exists a constant C depending only on s (the dimension of Ξ) and p such that, for all $N\ge 1$,

$$\begin{array}{cc}\hfill & \mathbb{E}\left[{\mathbb{D}}_W(P_N,P)\right]\le C{\left({\mathbb{E}}_P\left[{∥\xi ∥}^p\right]\right)}^{1/p}\hfill \\ \hfill & \times \left\{\begin{array}{cc}{N}^{-1/2}+N^{-(p-1)/p}\hfill & ifs=1andp\ne 2,\hfill \\ N^{-1/2}log(1+N)+N^{-(p-1)/p}\hfill & ifs=2andp\ne 2,\hfill \\ N^{-1/s}+N^{-(p-1)/p}\hfill & ifs>2andp\ne s/(s-1),\hfill \end{array}\right.\hfill \end{array}$$

where log is the natural logarithm.

Lemma 2 cannot cover all the pairs $(s,p)$, for example, $(s,p)=(1,2)$ or $(s,p)=(2,2)$. However, we can always reset p such that Lemma 2 holds by the following procedures. If $s=1$ or 2 and $p=2$, P must belong to ${\mathcal{P}}_q(\Xi )$ for any $q\in (1,2)$. If $s>2$ and $p=2$, we can select $q\in (1,2)$ such that $q\ne s/(s-1)$. If $s>2$ and $p=s/(s-1)$, we can choose any $q\in (1,s/(s-1\left)\right)$. Then, we let $p=q$ and have that Lemma 2 holds with $s=1$ or 2 and $p\in (1,2)$ or $s>2$ and $p\ne s/(s-1)$. Therefore, Lemma 2 is applicable for any $s\in \mathbb{N}$ through carefully prepared p. In the following discussion, without loss of generality, we always assume that Lemma 2 holds for any pair $(s,p)\in \mathbb{N}\times [1,+\infty )$. Further, we can, according to Lemma 2, obtain the following uniform upper bound:

$$\begin{array}{c}\hfill \mathbb{E}\left[{\mathbb{D}}_W(P_N,P)\right]\le C{\left({\mathbb{E}}_P\left[{∥\xi ∥}^p\right]\right)}^{1/p}\left(N^{-1/max\{2,s\}}log(1+N)+N^{-(p-1)/p}\right)\end{array}$$

for any $s\in \mathbb{N}$ and $N\ge 1$.

Assumption 1.

Let $P\in \mathcal{P}(\Xi )$ satisfy

$$A:={\mathbb{E}}_P\left[exp\left({\parallel \xi \parallel }^b\right)\right]={\int }_{\Xi }exp\left({\parallel \xi \parallel }^b\right)P\left(\mathrm{d}\xi \right)<\infty$$

(2)

for some constant b.

Lemma 3.

Suppose that Assumption 1 holds for some $b>1$. Then, we have for $ϵ\in (0,1]$ that

$$\begin{array}{cc}\hfill & \mathbb{P}\left\{{\mathbb{D}}_{\mathrm{W}}\left(P,P_N\right)\ge ϵ\right\}\le \alpha \times \left\{\begin{array}{cc}exp\left(-\beta N{ϵ}^2\right)\hfill & ifs=1,\hfill \\ exp\left(-\beta N{(ϵ/log(2+1/ϵ))}^2\right)\hfill & ifs=2,\hfill \\ exp\left(-\beta N{ϵ}^s\right)\hfill & ifs>2\hfill \end{array}\right.\hfill \end{array}$$

for all $N\ge 1$, where α and β are two positive constants depending only on P, b, and s.

Proof. 

Based on Assumption 1, we know that Condition (1) in [1] holds. Then, due to $ϵ\in (0,1]$, Lemma 3 directly follows from [1] (Theorem 2). □

For a more comprehensive version of Lemma 3, one can refer to [1] (Theorem 2). Here, we focus on the case $ϵ\in (0,1]$ because it is more interesting for us to investigate a smaller violation rather than a bigger one. A simplified version can also be found in [18] (Theorem 3.4) where the assumption $s\ne 2$ is imposed.

To simplify the following discussion, we derive a uniform upper bound for the right-hand side in Lemma 3. Note the fact that $1+\delta \le e^{\delta }$ for any $\delta \in \mathbb{R}$. We have

$$\begin{array}{cc}\hfill log\left(2+\frac{1}{ϵ}\right)& =1+log\left(2+\frac{1}{ϵ}\right)-log\left(e\right)\hfill \\ \hfill & =1+log\left(\frac{2}{e}+\frac{1}{eϵ}\right)\le \frac{2}{e}+\frac{1}{eϵ}.\hfill \end{array}$$

Letting $ϵ\in (0,1/2]$ gives us that

$$\begin{array}{c}\hfill {\left(\frac{ϵ}{log(2+1/ϵ)}\right)}^2\ge \frac{e^2{ϵ}^4}{4}.\end{array}$$

When $s=2$, we have

$$\mathbb{P}\left\{{\mathbb{D}}_{\mathrm{W}}\left(P,P_N\right)\ge ϵ\right\}\le \alpha exp\left(-\beta N{(ϵ/log(2+1/ϵ))}^2\right)\le \alpha exp\left(-\frac{e^2\beta N{ϵ}^4}{4}\right).$$

Moreover, for $ϵ\in (0,1/2]$,

$$exp\left(-\beta N{ϵ}^4\right)\ge exp\left(-\frac{e^2\beta N{ϵ}^4}{4}\right)\ge exp\left(-\beta N{ϵ}^2\right).$$

Therefore, we can obtain a loose but uniform upper bound estimation

$$\begin{array}{c}\hfill \mathbb{P}\left\{{\mathbb{D}}_{\mathrm{W}}\left(P,P_N\right)\ge ϵ\right\}\le \alpha exp\left(-\beta N{ϵ}^{max\{4,s\}}\right)\end{array}$$

(3)

for any $ϵ\in (0,1/2]$ and $s\in \mathbb{N}$.

3. Convergence Analyses of Data-Driven FM Metrics

In this section, we will investigate different kinds of convergence for data-driven FM metrics. To this end, let ${\xi }^1,{\xi }^2,\cdots ,{\xi }^N$ be N iid samples generated according to P. These samples are viewed here as the random sample ${\xi }^i:\Omega \to \Xi$, $1\le i\le N$, on the probability space $(\Omega ,\mathcal{F},\mathbb{P})$. Then, we obtain the empirical distribution $P_N$ defined as

$$P_N=\frac{1}{N}\sum _{i=1}^N{\mathbf{1}}_{{\xi }^i},$$

where ${\mathbf{1}}_{{\xi }^i}(·)$ is the indicator function, that is, ${\mathbf{1}}_{{\xi }^i}\left(\xi \right)=1$ for $\xi ={\xi }^i$ and ${\mathbf{1}}_{{\xi }^i}\left(\xi \right)=0$ otherwise.

We first give the following vital lemma.

Lemma 4.

Let $P,Q\in {\mathcal{P}}_p(\Xi )$ for some $p\ge 1$. Then,

$$\begin{array}{c}\hfill {\zeta }_p(P,Q)\le R^{p-1}{\mathbb{D}}_W(P,Q)+4{\int }_{\{\xi \in \Xi :∥\xi ∥>R\}}{∥\xi ∥}^p(P+Q)\left(d\xi \right)\end{array}$$

for any R satisfying $R\ge 1$ and $\mathbb{B}(\mathbf{0},R)\cap \Xi \ne \varnothing$. Here $\mathbf{0}$ is the original point in ${\mathbb{R}}^s$ and $\mathbb{B}(\mathbf{0},R)$ is the closed ball centered at $\mathbf{0}$ with radius R.

The proof of Lemma 4 can be found in Appendix A.

If we define

$${\varphi }_P(P,Q):=\underset{R}{inf}\left\{R^{p-1}{\mathbb{D}}_W(P,Q)+4{\int }_{M_R^c}{∥\xi ∥}^p(P+Q)\left(d\xi \right):R\ge 1,\mathbb{B}(\mathbf{0},R)\cap \Xi \ne \varnothing \right\},$$

then, we can obtain a tighter upper bound estimation of ${\zeta }_p(P,Q)$, that is

$${\zeta }_p(P,Q)\le {\varphi }_P(P,Q).$$

The first convergence result is about the non-asymptotic moment estimate. It provides an upper bound for the expectation of the FM distance between P and its empirical approximation distribution.

Theorem 1

(Non-asymptotic moment estimates for FM metrics). Suppose that $P\in {\mathcal{P}}_p(\Xi )$ for some $p>1$. Then, for sufficiently large N, we have

$$\mathbb{E}\left[{\zeta }_p(P,P_N)\right]\le {\beta }_N,$$

where $\left\{{\beta }_N\right\}$ is a sequence of positive numbers satisfying ${\beta }_N\to 0$ as $N\to \infty$.

The proof of Theorem 1 can be found in Appendix A.

Theorem 1 establishes the convergence in the sense of expectation. However, it fails to tell us the sample-wise convergence. The following theorem states the asymptotic convergence under FM metrics for almost every sample.

Theorem 2

(Asymptotic convergence of FM metrics). Suppose that $P\in {\mathcal{P}}_p(\Xi )$. Then,

$${\zeta }_p(P,P_N)\to 0$$

with probability 1, as $N\to \infty$. Here $P_N$ is defined at the beginning of this section.

The proof of Theorem 2 can be found in Appendix A.

Theorems 1 and 2 claim the convergence. As we know, the rate of convergence is quite important for guiding the solution process in practice. The following theorem gives the estimate of the convergence rate under certain assumptions.

Theorem 3

(Non-asymptotic concentration estimates for FM metrics). Suppose that $P\in {\mathcal{P}}_p(\Xi )$$(p\ge 1)$ and Assumption 1 holds with $b>p$. Then, for any $ϵ\in (0,1/2]$, we have

$$\mathbb{P}({\zeta }_p(P,P_N)\ge ϵ)\le \widehat{\alpha }exp(-\widehat{\beta }N)$$

for some constants $\widehat{\alpha }>0$ depending on P, b, and s, and $\widehat{\beta }>0$ depending on P, b, s, and ϵ.

The proof of Theorem 3 can be found in Appendix A.

Remark 1.

Here we assume that $ϵ\in (0,1/2]$. The main reason is that we want to give a relatively simple proof. Fortunately, it is more interesting for us to consider a small violation rather than a large one.

Under certain assumptions, we can obtain an estimation for $I\left(\frac{ϵ}{16}\right)$. For example, if $M\left(t\right)\le exp\left({\sigma }^2{t}^2/2\right)$ for $t\in \mathbb{R}$, here σ is a positive constant, we have $logM\left(t\right)\le {\sigma }^2{t}^2/2$. Then, according to the properties of convex quadratic functions, the rate function has the lower bound

$$I\left(\frac{ϵ}{16}\right)\ge \frac{{ϵ}^2}{512{\sigma }^2}.$$

Thus, we can further obtain a concrete estimate for $\widehat{\beta }$. For more details in this aspect, one can refer to [16].

4. Applications

In this section, we consider four applications of convergence conclusions about FM metrics obtained in Section 3. Specifically, we study the discrete approximation of two-stage stochastic programming problems, stochastic optimization problems with dominance constraints, data-driven distributionally robust optimization problems with FM ball, and the discrete approximation for distributionally robust optimization problems with general moment ambiguity set. They will not only further illustrate the motivations of this study but also provide alternative avenues or extensions for the current results.

4.1. Two-Stage Stochastic Linear Programming Problems

Discrete approximation is an important issue in stochastic optimization, which is crucial for its numerical solution. In this subsection, by employing the convergence results in Section 3, we give an alternative avenue for analyzing the discrete approximation of two-stage stochastic programming problems.

Consider the two-stage stochastic programming problem:

$$\underset{x\in X}{min}{c}^{\top }x+{\mathbb{E}}_P[\Phi (x,\xi )],$$

(4)

where $c\in {\mathbb{R}}^n$; $X\subseteq {\mathbb{R}}^n$ is a polyhedron; the probability measure P is supported on $\Xi \subseteq {\mathbb{R}}^s$, which is a polyhedron; and

$$\Phi (x,\xi ):=inf\left\{q{\left(\xi \right)}^{\top }y\left(\xi \right):Wy\left(\xi \right)+T\left(\xi \right)x=h\left(\xi \right),y\left(\xi \right)\ge 0\right\}.$$

(5)

Here $W\in {\mathbb{R}}^{r\times m}$, $T\left(\xi \right)\in {\mathbb{R}}^{r\times n}$, $q\left(\xi \right)\in {\mathbb{R}}^m$, $h\left(\xi \right)\in {\mathbb{R}}^r$. $q\left(\xi \right),T\left(\xi \right)$ and $h\left(\xi \right)$ depend affine linearly on $\xi$.

Denote $f(x,\xi )=c^{\top }x+\Phi (x,\xi )$, and let $v\left(P\right)$ and $S\left(P\right)$ denote the optimal value and optimal solution set of Problem (4). Moreover, we use $\mathrm{Pos}W$ to denote the set $\{W\widehat{y}:\widehat{y}\in {\mathbb{R}}_{+}^m\}$. Denote $D=\{u\in {\mathbb{R}}^m:\{z\in {\mathbb{R}}^r:W^{\top }z\le u\}\ne \varnothing \}$.

To quantify the upper semicontinuity or the deviation distance of the optimal solution set, we define the growth function ${\psi }_P:{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ as

$${\psi }_P\left(\tau \right):=min\left\{{\mathbb{E}}_P\left[f(x,\xi )\right]-v\left(P\right):d(x,S\left(P\right))\ge \tau ,x\in X\right\}.$$

Its inverse function ${\psi }_P^{-1}$ is given by

$${\psi }_P^{-1}\left(t\right):=sup\{\tau \in {\mathbb{R}}_{+}:{\psi }_P\left(\tau \right)\le t\}.$$

Thus, we can define the associated conditioning function ${\Psi }_P:{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ as

$${\Psi }_P\left(\eta \right)=\eta +{\psi }_P^{-1}\left(2\eta \right).$$

It is easy to verify that ${\psi }_P$ is nondecreasing and ${\Psi }_P$ is increasing. Both ${\psi }_P$ and ${\Psi }_P$ are lower semicontinuous on ${\mathbb{R}}_{+}$ and vanish at 0. One can refer to [10] for more details.

Moreover, we have ${\psi }_P^{-1}\left(t\right)\to 0_{+}$ as $t\to 0_{+}$. We illustrate this fact by contradiction. Suppose that there exists a sequence $\left\{t_n\right\}$ satisfying $t_n\to 0_{+}$ as $n\to \infty$, such that ${\psi }_P^{-1}\left(t_n\right)\nrightarrow 0_{+}$. Denote ${\tau }_n={\psi }_P^{-1}\left(t_n\right)$. The lower semicontinuity of ${\psi }_P$ means that $\{\tau \in {\mathbb{R}}_{+}:{\psi }_P\left(\tau \right)\le t_n\}$ is closed. Thus, ${\psi }_P\left({\tau }_n\right)\le t_n$. Due to the nondecreasing property of ${\psi }_P$ and $t_n\to 0_{+}$ as $n\to \infty$, $\left\{{\tau }_n\right\}$ must be bounded. Without loss of generality, we assume that ${\tau }_n\to {\tau }^{*}$ as $n\to \infty$, where ${\tau }^{*}$ is a positive constant. According to the lower semicontinuity of ${\psi }_P$, we have

$$0=\underset{n\to \infty }{lim\; inf}{t}_n\ge \underset{n\to \infty }{lim\; inf}{\psi }_P\left({\tau }_n\right)\ge {\psi }_P\left({\tau }^{*}\right)>0,$$

which leads to a contradiction.

According to the definition of ${\Psi }_P$, we can immediately deduce that ${\Psi }_P\left(\eta \right)\to 0_{+}$ as $\eta \to 0_{+}$.

To introduce the following discussion, we make some standard assumptions (see [11]).

Assumption 2.

Let the following assertions hold:

(1) 

For each pair $(x,\xi )\in X\times \Xi$, $h\left(\xi \right)-T\left(\xi \right)x\in \mathrm{Pos}W$ and $q\left(\xi \right)\in D$;

(2) 

$P\in {\mathcal{P}}_2(\Xi )$.

Under the above assumptions, we have the following quantitative stability results about the optimal value and optimal solution set of Problem (4).

Lemma 5

([11], Theorem 3.3). Suppose that Assumption 2 holds and $S\left(P\right)$ is nonempty and bounded. Then, there exist constants $L>0$ and $\delta >0$ such that

$$\begin{array}{cc}\hfill \left|v\left(P\right)-v\left(Q\right)\right|& \le L{\zeta }_2(P,Q),\hfill \\ \hfill \varnothing \ne S\left(Q\right)& \subseteq S\left(P\right)+{\Psi }_P\left(L{\zeta }_2(P,Q)\right)\mathbb{B}\hfill \end{array}$$

when $Q\in {\mathcal{P}}_2(\Xi )$ and ${\zeta }_2(P,Q)<\delta$, where $\mathbb{B}$ is the closed unit ball in ${\mathbb{R}}^n$.

Based on Lemma 5 and the convergence results in Section 3, we have the following convergence conclusions between the two-stage stochastic programming problem (4) and its empirical approximation.

Theorem 4.

Suppose that: (i) Assumption 2 holds; (ii) $S\left(P\right)$ is nonempty and bounded. Then,

$$\begin{array}{cc}\hfill |v\left(P\right)-v\left(P_N\right)|& \to 0,\hfill \\ \hfill d(S\left(P_N\right),S\left(P\right))& \to 0\hfill \end{array}$$

with probability 1, as $N\to \infty$.

Proof. 

For the first assertion, we have from Theorem 2 that ${\zeta }_2(P,P_N)\to 0$ with probability 1. This means that: for the $\delta$ defined in Lemma 5, there exists a positive number $N_0=N_0(\delta ,\omega )$ such that for any $N\ge N_0$, ${\zeta }_2(P,P_N)\le \delta$ for almost every $\omega \in \Omega$. Then, by Lemma 5, we have that

$$\begin{array}{cc}\hfill |v\left(P\right)-v\left(P_N\right)|& \le L{\zeta }_2(P,P_N),\hfill \\ \hfill d(S\left(P_N\right),S\left(P\right))& \le {\Psi }_P\left(L{\zeta }_2(P,P_N)\right)\hfill \end{array}$$

hold almost surely as $N\ge N_0$, here L is defined in Lemma 5. According to Theorem 2 and the property of ${\Psi }_P$, we have

$$\begin{array}{c}\hfill {\zeta }_2(P,P_N)\to 0\end{array}$$

and thus

$$\begin{array}{c}\hfill {\Psi }_P\left(L{\zeta }_2(P,P_N)\right)\to 0\end{array}$$

with probability 1, as $N\to \infty$. These facts imply that

$$\begin{array}{cc}\hfill |v\left(P\right)-v\left(P_N\right)|& \to 0,\hfill \\ \hfill d(S\left(P_N\right),S\left(P\right))& \to 0\hfill \end{array}$$

with probability 1, as $N\to \infty$. □

Theorem 5.

Suppose that: (i) Assumption 1 holds with $b>2$; (ii) Assumption 2 holds; (iii) $S\left(P\right)$ is nonempty and bounded. Then, for any $ϵ\in (0,1/2]$, there exist $\overline{\alpha }>0$ depending on P and s, and $\overline{\beta }>0$ depending on P, s and ϵ, such that

$$\begin{array}{cc}\hfill \mathbb{P}\left(|v\left(P\right)-v\left(P_N\right)|\ge Lϵ\right)& \le \overline{\alpha }exp(-\overline{\beta }N),\hfill \\ \hfill \mathbb{P}\left(d(S\left(P_N\right),S\left(P\right))\ge {\Psi }_P\left(Lϵ\right)\right)& \le \overline{\alpha }exp(-\overline{\beta }N).\hfill \end{array}$$

Proof. 

If $|v\left(P\right)-v\left(P_N\right)|\le L{\zeta }_2(P,P_N)$ for L defined in Lemma 5, we have from Theorem 3 that

$$\begin{array}{cc}\hfill \mathbb{P}\left(|v\left(P\right)-v\left(P_N\right)|\ge Lϵ\right)& \le \mathbb{P}\left({\zeta }_2(P,P_N)\ge ϵ\right)\le \overline{\alpha }exp(-\overline{\beta }\left(ϵ\right)N)\hfill \end{array}$$

for any $ϵ\in (0,1/2]$, where $\overline{\alpha }>0$ depends on P and s, and $\overline{\beta }>0$ depends on P, s and $ϵ$. Here we use $\overline{\beta }\left(ϵ\right)$ to stress its dependence on $ϵ$.

As shown in Theorem 4, a sufficient condition for

$$|v\left(P\right)-v\left(P_N\right)|\le L{\zeta }_2(P,P_N)$$

is ${\zeta }_2(P,P_N)<\delta$, where $\delta$ is defined in Lemma 5. Without loss of generality, we assume that $\delta \in (0,1/2]$. Analogously, we have that

$$\begin{array}{cc}\hfill \mathbb{P}\left({\zeta }_2(P,P_N)<\delta \right)& \ge 1-\overline{\alpha }exp(-\overline{\beta }\left(\delta \right)N).\hfill \end{array}$$

Then, we obtain

$$\begin{array}{cc}\hfill \mathbb{P}\left(|v\left(P\right)-v\left(P_N\right)|\ge Lϵ\right)& \le \overline{\alpha }exp(-\overline{\beta }\left(ϵ\right)N)·\left(1-\overline{\alpha }exp(-\overline{\beta }\left(\delta \right)N)\right)\hfill \\ \hfill & \le \overline{\alpha }exp(-\overline{\beta }\left(ϵ\right)N).\hfill \end{array}$$

Similarly, we have

$$\begin{array}{cc}\hfill \mathbb{P}\left(d(S\left(P_N\right),S\left(P\right))\ge {\Psi }_P\left(Lϵ\right)\right)& \le \mathbb{P}\left({\Psi }_P\left(L{\zeta }_2(P,P_N)\right)\ge {\Psi }_P\left(Lϵ\right)\right)\hfill \\ \hfill & =\mathbb{P}\left({\zeta }_2(P,P_N)\ge ϵ\right),\hfill \end{array}$$

where the equality follows from the strictly increasing property of ${\Psi }_P(·)$. By the same procedure, we can derive the second assertion. □

Remark 2.

The convergence analysis about two-stage stochastic programming problems can also be found in [11] (Section 4), where the covering and bracketing numbers are introduced. However, it seems difficult to verify the growth rate of the covering or bracketing number in the general case (see [11], Proposition 4.2). Our convergence results are more straightforward. Compared with [11] (Proposition 4.2), instead of the growth rate of the covering or bracketing number, we use the light-tailed distribution assumption. This assumption is commonly used in the literature, see for example [1,18].

4.2. Stochastic Optimization Problems with Stochastic Dominance Constraints

In this part, we consider stochastic optimization problems with stochastic dominance constraints. Stochastic dominance is an important ingredient in economics, decision theory, statistics, and nowadays in modern optimization. It has been widely studied in the last two decades, see for example [17,22,23,24,25,26] and their references therein. Different from classical stochastic optimization models which cope with random variables by taking expectation, stochastic dominance can better reflect the relationship between two random variables. It is known that expected utility theory can also provide the comparison of two random variables. However, it is hardly possible for us to explicitly express the utility functions of decision makers [27]. From this point of view, stochastic dominance is more friendly in practice. Actually, stochastic dominance has a close relationship with expected utility theory. Generally, a random variable $\mathcal{X}$ dominates another random variable $\mathcal{Y}$ in the kth $(k\ge 1)$ order, denoted by $\mathcal{X}{⪰}_{\left(k\right)}\mathcal{Y}$, if $\mathbb{E}\left[u\right(\mathcal{X}\left)\right]\ge \mathbb{E}\left[u\right(\mathcal{Y}\left)\right]$ for every nondecreasing function $u(·)$ from a certain set of utility functions [17]. Specially, $\mathcal{X}{⪰}_{\left(1\right)}\mathcal{Y}$ if and only if $\mathbb{E}\left[u\right(\mathcal{X}\left)\right]\ge \mathbb{E}\left[u\right(\mathcal{Y}\left)\right]$ for every nondecreasing utility function $u(·)$. $\mathcal{X}{⪰}_{\left(2\right)}\mathcal{Y}$ if and only if $\mathbb{E}\left[u\right(\mathcal{X}\left)\right]\ge \mathbb{E}\left[u\right(\mathcal{Y}\left)\right]$ for every nondecreasing and concave utility function $u(·)$ [27].

The convex stochastic optimization model with the kth order stochastic dominance constraint can be described as (see [22,27]):

$$min\{f\left(x\right):x\in D,G(x,\xi ){⪰}_{\left(k\right)}Y\},$$

(6)

where D is a nonempty closed and convex subset of ${\mathbb{R}}^n$; $f:{\mathbb{R}}^n\to \mathbb{R}$ is a convex function; Y is a random variable supported on $\mathcal{Y}\subseteq \mathbb{R}$, which can be treated as the random benchmark; and $G:{\mathbb{R}}^n\times \Xi \to \mathbb{R}$. Moreover, we assume that G is locally Lipschitz continuous with respect to $\xi$ in the following sense:

$$\begin{array}{c}\hfill |G(x,{\xi }^{\prime })-G(x,\xi )|\le L_{G}max{\left\{1,∥{\xi }^{\prime }∥,∥\xi ∥\right\}}^{p-1}∥{\xi }^{\prime }-\xi ∥\end{array}$$

(7)

for any ${\xi }^{\prime },\xi \in \Xi$, where $p\ge 1$ and $L_G>0$. G satisfies the linear growth condition:

$$\begin{array}{c}\hfill \left|G(x,\xi )\right|\le C_G\left(B\right)max\{1,∥\xi ∥\}\end{array}$$

(8)

for every $x\in B$ and $\xi \in \Xi$, where B is any bounded subset of ${\mathbb{R}}^n$, and $C_G\left(B\right)>0$ depends on B.

Actually, we can impose a more general growth condition on G, for example,

$$\begin{array}{c}\hfill \left|G(x,\xi )\right|\le C_G\left(B\right)max{\{1,∥\xi ∥\}}^q\end{array}$$

for $q\ge 1$ and the following discussion still holds. Here a linear growth condition simplifies the demonstration. The above requirements for $G(x,\xi )$ can be met easily. For instance, the objective function of the two-stage stochastic programming problem with fixed recourse satisfies the above conditions (see [11], Proposition 3.2).

Due to its attractive modeling technique, the quantitative stability analysis of stochastic optimization models with dominance constraints has been recently investigated in several works. Dencheva et al. first studied in [22] stochastic optimization problems with first order stochastic dominance constraints, which was extended by Dencheva and Römisch in [17] to the problem with general kth ($k\ge 2$) order stochastic dominance constraints. In [24], Chen and Jiang weaken the assumptions of the quantitative stability analysis in [17] by considering the case that $G(x,\xi )$ is generated by the two-stage fully random stochastic programming problem.

To establish the convergence results, we first investigate the quantitative stability of model (6). By convention, we consider its relaxed problem (see also [17,24]):

$$min\left\{f\left(x\right):x\in D,\mathbb{E}\left[{(\eta -G(x,\xi ))}_{+}^{k-1}\right]\le \mathbb{E}\left[{(\eta -Y)}_{+}^{k-1}\right]\mathrm{for}\forall \eta \in I\right\},$$

(9)

where $I\subseteq \mathbb{R}$ is a compact interval: ${(·)}_{+}:=max\{0,·\}$.

In view of our focus in this study, we reestablish the quantitative stability conclusions of Problem (9) in what follows. We use $P_{\xi }$ and $P_Y$ to denote the probability distributions of $\xi$ and Y, respectively. We denote the feasible solution set of Problem (9) by

$$\mathcal{X}(P_{\xi },P_Y)=\left\{x\in D:{\mathbb{E}}_{P_{\xi }}\left[{(\eta -G(x,\xi ))}_{+}^{k-1}\right]\le {\mathbb{E}}_{P_Y}\left[{(\eta -Y)}_{+}^{k-1}\right]\mathrm{for}\forall \eta \in I\right\}$$

and its perturbed feasible solution set under $(Q_{\xi },Q_Y)$ by

$$\mathcal{X}(Q_{\xi },Q_Y)=\left\{x\in D:{\mathbb{E}}_{Q_{\xi }}\left[{(\eta -G(x,\xi ))}_{+}^{k-1}\right]\le {\mathbb{E}}_{Q_Y}\left[{(\eta -Y)}_{+}^{k-1}\right]\mathrm{for}\forall \eta \in I\right\}.$$

First we examine the quantitative stability of the feasible solution set.

Proposition 1.

Let D be compact, $P_{\xi }\in {\mathcal{P}}_{k+p-2}(\Xi )$, $P_Y\in {\mathcal{P}}_{k-1}\left(\mathcal{Y}\right)$, and G satisfy the locally Lipschitz continuity condition (7) and the linear growth condition (8). Then, there exist constants $L>0$ and $\delta >0$ such that

$$d_H(\mathcal{X}(P_{\xi },P_Y),\mathcal{X}(Q_{\xi },Q_Y))\le L({\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y))$$

whenever $Q_{\xi }\in {\mathcal{P}}_{k+p-2}(\Xi )$, $Q_Y\in {\mathcal{P}}_{k-1}\left(\mathcal{Y}\right)$ and the pair $(Q_{\xi },Q_Y)$ satisfies ${\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y)<\delta$, where $d_H(·,·)$ denotes the Pompeiu–Hausdorff distance.

Proof. 

We know from the proof of [17] (Proposition 3.2) that

$$\begin{array}{cc}\hfill & d_H(\mathcal{X}(P_{\xi },P_Y),\mathcal{X}(Q_{\xi },Q_Y))\hfill \\ \hfill \le & \frac{1}{(k-1)!}\underset{\eta \in I}{max}\left|{\mathbb{E}}_{P_{\xi }}\left[{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}\right]-{\mathbb{E}}_{Q_{\xi }}\left[{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}\right]\right|\hfill \\ \hfill & +\frac{1}{(k-1)!}\underset{\eta \in I}{max}\left|{\mathbb{E}}_{P_Y}\left[{\left(\eta -Y\right)}_{+}^{k-1}\right]-{\mathbb{E}}_{Q_Y}\left[{\left(\eta -Y\right)}_{+}^{k-1}\right]\right|\hfill \end{array}$$

whenever the right-hand side is less than or equal to some positive scalar $\overline{\delta }$.

In view of this, we estimate

$$\underset{\eta \in I}{max}\left|{\mathbb{E}}_{P_{\xi }}\left[{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}\right]-{\mathbb{E}}_{Q_{\xi }}\left[{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}\right]\right|$$

and

$$\underset{\eta \in I}{max}\left|{\mathbb{E}}_{P_Y}\left[{\left(\eta -Y\right)}_{+}^{k-1}\right]-{\mathbb{E}}_{Q_Y}\left[{\left(\eta -Y\right)}_{+}^{k-1}\right]\right|,$$

respectively.

Note the fact that (see [17], (3.9))

$$|{(\eta -t)}_{+}^{k-1}-{(\eta -\widehat{t})}_{+}^{k-1}|\le K_I(k-1)max\{1,|t|,|\widehat{t}{\left|\right\}}^{k-2}\left|t-\widehat{t}\right|$$

for some positive constant $K_I$ and any $\eta \in I$. Then, we have

$$\begin{array}{cc}\hfill & |{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}-{\left(\eta -G(x,\widehat{\xi })\right)}_{+}^{k-1}|\hfill \\ \hfill \le & K_I(k-1)max{\left\{1,\left|G(x,\xi )\right|,|G(x,\widehat{\xi })|\right\}}^{k-2}\left|G(x,\xi )-G(x,\widehat{\xi })\right|\hfill \\ \hfill \le & K_I(k-1)max{\left\{1,C_G\left(D\right)max\{1,∥\xi ∥\},C_G\left(D\right)max\left\{1,∥\widehat{\xi }∥\right\}\right\}}^{k-2}\hfill \\ \hfill & ·L_{G}max{\{1,∥\xi ∥,∥\widehat{\xi }∥\}}^{p-1}∥\xi -\widehat{\xi }∥\hfill \\ \hfill \le & K_I(k-1)(C_G\left(D\right)+1)L_{G}max{\left\{1,∥\xi ∥,∥\widehat{\xi }∥\right\}}^{k+p-3}∥\xi -\widehat{\xi }∥.\hfill \end{array}$$

This means that

$$\begin{array}{c}\hfill \underset{\eta \in I}{max}\left|{\mathbb{E}}_{P_{\xi }}\left[{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}\right]-{\mathbb{E}}_{Q_{\xi }}\left[{\left(\eta -G(x,\xi )\right)}_{+}^{k-1}\right]\right|\le L_1{\zeta }_{k+p-2}(P_{\xi },Q_{\xi }),\end{array}$$

where $L_1:=K_I(k-1)(C_G\left(D\right)+1)L_G$.

Similarly, we have

$$\begin{array}{c}\hfill |{\left(\eta -Y\right)}_{+}^{k-1}-{\left(\eta -\widehat{Y}\right)}_{+}^{k-1}|\le K_I(k-1)max{\left\{1,\left|Y\right|,|\widehat{Y}|\right\}}^{k-2}\left|Y-\widehat{Y}\right|,\end{array}$$

which means that

$$\begin{array}{c}\hfill \underset{\eta \in I}{max}\left|{\mathbb{E}}_{P_Y}\left[{\left(\eta -Y\right)}_{+}^{k-1}\right]-{\mathbb{E}}_{Q_Y}\left[{\left(\eta -Y\right)}_{+}^{k-1}\right]\right|\le K_I(k-1){\zeta }_{k-1}(P_Y,Q_Y).\end{array}$$

Taking $L:=\frac{1}{(k-1)!}max\{L_1,K_I(k-1)\}$, we have

$$\begin{array}{c}\hfill d_H(\mathcal{X}(P_{\xi },P_Y),\mathcal{X}(Q_{\xi },Q_Y))\le L({\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y)),\end{array}$$

whenever ${\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y)\le \delta :=\frac{\overline{\delta }}{L}$. □

The quantitative stability result in Proposition 1 differs in two perspectives from the corresponding results in [17]. One is the locally Lipschitz continuity of G; the other is the probability metric we choose. In [17], the authors assumed that G is Lipschitz continuous, and adopted Rachev metrics and the $(k-1)$th order Wasserstein metric. As far as we know, there does not exist data-driven results under Rachev metrics.

Let $v(P_{\xi },P_Y)$ and $S(P_{\xi },P_Y)$ denote the optimal value and optimal solution set of Problem (9), respectively. Similar to that in Section 4.1, we can define the growth function of Problem (9) as

$${\psi }_{P_{\xi },P_Y}\left(\tau \right)=inf\{f\left(x\right)-v(P_{\xi },P_Y):d(x,S(P_{\xi },P_Y))\ge \tau ,x\in \mathcal{X}(P_{\xi },P_Y)\}.$$

Then, its inverse function and the associated conditioning function are

$${\psi }_{P_{\xi },P_Y}^{-1}\left(t\right):=sup\{\tau \in {\mathbb{R}}_{+}:{\psi }_{P_{\xi },P_Y}\left(\tau \right)\le t\}$$

and

$${\Psi }_{P_{\xi },P_Y}\left(\eta \right):=\eta +{\psi }_{P_{\xi },P_Y}^{-1}\left(2\eta \right).$$

Proposition 2.

Under the conditions of Proposition 1, there exist constants $\widehat{L}>0$ and $\widehat{\delta }>0$ such that

$$\begin{array}{cc}\hfill |v(P_{\xi },P_Y)-v(Q_{\xi },Q_Y)|& \le \widehat{L}({\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y)),\hfill \\ \hfill d(S(Q_{\xi },Q_Y),S(P_{\xi },P_Y))& \le {\Psi }_{P_{\xi },P_Y}\left(\widehat{L}({\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y))\right)\hfill \end{array}$$

whenever ${\zeta }_{k+p-2}(P_{\xi },Q_{\xi })+{\zeta }_{k-1}(P_Y,Q_Y)<\widehat{\delta }$.

Proof. 

Since f is convex, f is locally Lipschitz continuous. Since D is compact, f is in fact Lipschitz continuous over D. Then, the assertions follow from a similar proof as that for [17] (Theorem 3.3). □

Now we consider the iid samples of $\xi$ and Y. For convenience, we assume that the samples drawn from $\xi$ and Y have the same sample size N. The N iid samples of $\xi$ are ${\xi }^1,{\xi }^2,\cdots ,{\xi }^N$ and the N iid samples of Y are $Y^1,Y^2,\cdots ,Y^N$. Then, we have the following empirical distributions:

$$P_{\xi ,N}=\frac{1}{N}\sum _{i=1}^N{\mathbf{1}}_{{\xi }^i},$$

and

$$P_{Y,N}=\frac{1}{N}\sum _{i=1}^N{\mathbf{1}}_{Y^i}.$$

With these preparations, we can establish the following convergence results.

Theorem 6.

Let D be compact, $P_{\xi }\in {\mathcal{P}}_{k+p-2}(\Xi )$, $P_Y\in {\mathcal{P}}_{k-1}\left(\mathcal{Y}\right)$, and G satisfy the locally Lipschitz continuity condition (7) and the linear growth condition (8).

(i) 

We have

$$d_H(\mathcal{X}(P_{\xi },P_Y),\mathcal{X}(P_{\xi ,N},Q_{Y,N}))\to 0$$

and

$$\begin{array}{c}\hfill |v(P_{\xi },P_Y)-v(P_{\xi ,N},Q_{Y,N})|\to 0,\\ \hfill d(S(P_{\xi ,N},Q_{Y,N}),S(P_{\xi },P_Y))\to 0\end{array}$$

with probability 1, as $N\to \infty$.

(ii) 

If, moreover,

$${\mathbb{E}}_{P_{\xi }}\left[exp\left({\parallel \xi \parallel }^b\right)\right]={\int }_{\Xi }exp\left({\parallel \xi \parallel }^b\right)P_{\xi }\left(\mathrm{d}\xi \right)<+\infty$$

and

$${\mathbb{E}}_{P_Y}\left[exp\left({\left|Y\right|}^c\right)\right]={\int }_{\mathcal{Y}}exp\left({\left|y\right|}^c\right)P_Y\left(\mathrm{d}y\right)<+\infty$$

for some $b>k+p-2$ and $c>k-1$, then, for $ϵ\in (0,1/2]$, there exist positive scalers ${\alpha }_1$ depending on $P_{\xi }$, b and s; ${\beta }_1$ depending on $P_{\xi }$, b, s and ϵ; ${\alpha }_2$ depending on $P_Y$ and c, and ${\beta }_2$ depending on $P_Y$, c and ϵ, such that

$$\begin{array}{cc}\hfill \mathbb{P}\left(d_H(\mathcal{X}(P_{\xi },P_Y),\mathcal{X}(P_{\xi ,N},Q_{Y,N}))\ge Lϵ\right)& \le {\alpha }_{1}exp(-{\beta }_{1}N)+{\alpha }_{2}exp(-{\beta }_{2}N)\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \mathbb{P}\left(|v(P_{\xi },P_Y)-v(P_{\xi ,N},Q_{Y,N})|\ge \widehat{L}ϵ\right)& \le {\alpha }_{1}exp(-{\beta }_{1}N)+{\alpha }_{2}exp(-{\beta }_{2}N),\hfill \\ \hfill \mathbb{P}\left(d(S(P_{\xi ,N},Q_{Y,N}),S(P_{\xi },P_Y))\ge {\Psi }_{P_{\xi },P_Y}\left(\widehat{L}ϵ\right)\right)& \le {\alpha }_{1}exp(-{\beta }_{1}N)+{\alpha }_{2}exp(-{\beta }_{2}N),\hfill \end{array}$$

where L and $\widehat{L}$ are defined in Propositions 1 and 2, respectively.

Proof. 

Part (i) can be similarly proved as that in Theorem 4 by utilizing Theorem 2 and Proposition 1.

For Part (ii), we have

$$\begin{array}{cc}\hfill & \mathbb{P}\left(d_H(\mathcal{X}(P_{\xi },P_Y),\mathcal{X}(P_{\xi ,N},P_{Y,N}))\ge Lϵ\right)\hfill \\ \hfill \le & \mathbb{P}\left({\zeta }_{k+p-2}(P_{\xi },P_{\xi ,N})+{\zeta }_{k-1}(P_Y,P_{Y,N})\ge ϵ\right)\hfill \\ \hfill \le & \mathbb{P}\left({\zeta }_{k+p-2}(P_{\xi },P_{\xi ,N})\ge \frac{ϵ}{2}\right)+\mathbb{P}\left({\zeta }_{k-1}(P_Y,P_{Y,N})\ge \frac{ϵ}{2}\right)\hfill \\ \hfill \le & {\alpha }_{1}exp(-{\beta }_{1}N)+{\alpha }_{2}exp(-{\beta }_{2}N),\hfill \end{array}$$

where the last inequality follows from Theorem 3; ${\alpha }_1$ depends on $P_{\xi }$, b, and s; ${\beta }_1$ depends on $P_{\xi }$, b, s, and $ϵ$; ${\alpha }_2$ depends on $P_Y$ and c; and ${\beta }_2$ depends on $P_Y$, c, and $ϵ$.

The second and third probability inequalities can be analogously verified and thus we omit the proof here. □

4.3. Data-Driven DRO Problems with FM Ball

A general stochastic optimization model can be formulated as

$$\underset{x\in X}{min}{\mathbb{E}}_P\left[g(x,\xi )\right],$$

(10)

where $g:X\times \Xi \to \mathbb{R}$, $X\subseteq {\mathbb{R}}^n$, and $\Xi \subseteq {\mathbb{R}}^s$ is the support set of $\xi$. The sample average approximation (SAA) is usually used to solve Problem (10) numerically. The SAA method acquiescently assumes that we can generate any number of samples based on P. To better approximate Problem (10), a large sample size is needed [28]. However, in practice, the true probability distribution P cannot be known exactly, and thus we cannot generate a sufficiently large number of samples to make the SAA method well-defined, due to the expensive cost for more samples. However, it is possible for us to obtain a limited number of samples or scenarios, such as historical data. Under these settings, the data-driven DRO model is proposed [18,29,30]. The natural idea is to use the partial information to construct an ambiguity set such that the true probability distribution is included in the ambiguity set. As pointed out in [18], under certain conditions, it offers powerful out-of-sample performance guarantees.

For further discussion, we denote the limited finite samples by ${\xi }^1,{\xi }^2,\cdots ,{\xi }^N$ and the corresponding empirical distribution by $P_N$. Since the number of samples N is limited, we cannot adopt the classical SAA method, which requires that the sample size tends to infinity. However, we can use the limited information to construct a set of probability measures which contains the true one, that is, the ambiguity set. In this subsection, we consider the following FM ball-based ambiguity set:

$${\mathbb{B}}_r\left(P_N\right):=\{Q\in \mathcal{P}(\Xi ):{\zeta }_p(Q,P_N)\le r\},$$

where $p\ge 1$, the positive constant r stands for the confidence parameter determined by the decision maker. Then, we have the data-driven DRO problem with the FM ball-based ambiguity set of Problem (10) as follows:

$$\underset{x\in X}{min}\underset{Q\in {\mathbb{B}}_r\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x,\xi )\right].$$

(11)

It is common for us to see that the Wasserstein ball is used to build the ambiguity set, for example, [18]. To further explain the reasonability and motivations for us to adopt the FM metric, we have the following comments.

Remark 3.

As we know, a key issue for DRO problems is how to build the ambiguity set. Different kinds of ambiguity sets have been proposed, such as moment information [31], ζ-ball [32], and so on. Of course, the FM metric, as a specific case of the ζ-structure probability metric, can be employed to construct the ambiguity set.

More importantly, the decision maker can utilize the limited empirical distribution $P_N$ to obtain an approximate optimal value, say $v\left(P_N\right)$. By prior experience, the decision maker usually has some confidence, measured by the derivation constant $\widehat{r}>0$, that the true optimal value, denoted by $v\left(P\right)$, locates in the interval $[v\left(P_N\right)-\widehat{r},v\left(P_N\right)+\widehat{r}]$. Frequently, $g(x,\xi )$ is locally Lipschitz continuous in the following sense:

$$|g(x,\xi )-g(x,{\xi }^{\prime })|\le Lmax{\left\{1,∥\xi ∥,∥{\xi }^{\prime }∥\right\}}^{p-1}∥{\xi }^{\prime }-\xi ∥$$

for some positive constant L. A typical example is that $g(x,\xi )$ is the objective function of the two-stage stochastic programming problem, here $p=2$ (see [11], Proposition 3.2). Then, we have the quantitative relationship:

$$|v\left(P\right)-v\left(P_N\right)|\le L{\zeta }_p(P,P_N).$$

Therefore, it is reasonable for the decision maker to consider the ambiguity set

$$\{Q\in \mathcal{P}(\Xi ):{\zeta }_p(Q,P_N)\le r:=\widehat{r}/L\}.$$

Moreover, since ${\zeta }_p(P,P_N)\to 0$ with probability 1, as $N\to \infty$ (see Theorem 2), P must be included in $\{Q\in \mathcal{P}(\Xi ):{\zeta }_p(Q,P_N)\le r\}$ for suitable N and r.

Finally, we have ${\zeta }_p(P,Q)\ge {\mathbb{D}}_W(P,Q)$. The equality holds if $p=1$. Thus,

$$\{Q\in \mathcal{P}(\Xi ):{\zeta }_p(Q,P_N)\le r\}\subseteq \{Q\in \mathcal{P}(\Xi ):{\mathbb{D}}_W(Q,P_N)\le r\}.$$

This tells us that the ambiguity set constructed by the FM ball is tighter than that constructed with the Wasserstein ball.

All these arguments motivate us to consider the data-driven DRO problem with the FM ball-based ambiguity set.

We use $v^{*}$ and $v_N$ to denote the optimal values of Problems (10) and (11), respectively.

To quantify the out-of-sample performance of the data-driven DRO problem (11), we examine the following probability

$$\mathbb{P}\left({\mathbb{E}}_P\left[g(x_N,\xi )\right]\le v_N\right),$$

where $x_N$ is any optimal solution of Problem (11). Of course, we hope that, for sufficiently small $ϵ>0$, there exists a finite positive integer $N_0$ such that

$$\mathbb{P}\left({\mathbb{E}}_P\left[g(x_N,\xi )\right]\le v_N\right)\ge 1-ϵ$$

(12)

for any $N\ge N_0$.

If P satisfies ${\zeta }_p(P,P_N)<r$, we have $P\in {\mathbb{B}}_r\left(P_N\right)$. Thus,

$${\mathbb{E}}_P\left[g(x,\xi )\right]\le \underset{Q\in {\mathbb{B}}_r\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x,\xi )\right]$$

for any $x\in X$, which of course implies that

$${\mathbb{E}}_P\left[g(x_N,\xi )\right]\le \underset{Q\in {\mathbb{B}}_r\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x_N,\xi )\right]=v_N.$$

From Theorem 3, we have for any $r\in (0,1/2]$ that

$$\mathbb{P}({\zeta }_p(P,P_N)\ge r)\le \widehat{\alpha }exp(-\widehat{\beta }\left(r\right)N),$$

here we use the notation $\widehat{\beta }\left(r\right)>0$ to stress the dependence of $\widehat{\beta }$ on r. Consequently,

$$\begin{array}{cc}\hfill \mathbb{P}({\zeta }_p(P,P_N)\le r)& \ge \mathbb{P}({\zeta }_p(P,P_N)<r)\hfill \\ \hfill & \ge 1-\widehat{\alpha }exp(-\widehat{\beta }\left(r\right)N).\hfill \end{array}$$

A sufficient condition to satisfy (12) is

$$1-\widehat{\alpha }exp(-\widehat{\beta }\left(r\right)N)\ge 1-ϵ,$$

which is equivalent to

$$N\ge \frac{1}{\widehat{\beta }\left(r\right)}log\left(\frac{\widehat{\alpha }}{ϵ}\right).$$

Denote

$$N_0=⌈\frac{1}{\widehat{\beta }\left(r\right)}log\left(\frac{\widehat{\alpha }}{ϵ}\right)⌉,$$

(13)

where $\lceil ·\rceil$ stands for rounding up to an integer. Sometimes, we use the notation $N_0(ϵ,r)$ to stress the dependence of $N_0$ on $ϵ$ and r.

Summarizing the above discussions, we obtain the following so-called finite sample guarantee property (see also [18,30]).

Proposition 3

(Finite sample guarantee). Let $P\in {\mathcal{P}}_p(\Xi )$$(p\ge 1)$ and Assumption 1 hold for some $b>p$. For any $r\in (0,1/2]$, $ϵ\in (0,1)$ and $N_0$ defined in (13), we have that (12) holds for every $N\ge N_0$.

Proposition 3 tells us, for the fixed confidence parameter r, at least how large the sample size should be to ensure the significance level $ϵ$. Now we slightly modify model (11) and consider the following data-driven DRO problem:

$$\underset{x\in X}{min}\underset{Q\in {\mathbb{B}}_{r_N}\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x,\xi )\right],$$

(14)

where $r_N>0$ and $r_N\to 0$ as $N\to \infty$. It reflects the natural fact that the decision maker becomes more confident with more information. Meanwhile, the model (11) emphasizes the fixed limited information. We use ${\widehat{v}}_N$ to denote the optimal value of Problem (14). In what follows, we investigate the asymptotic consistency whenever N tends to infinity. To this end, we need the following lemma.

Lemma 6.

Let $\left\{A_N\right\}$ and $\left\{B_N\right\}$ be two sequences of random variables defined on the probability space $(\Omega ,\mathcal{F},\mathbb{P})$. If $\left\{B_N\right\}$ converges almost surely and

$$\mathbb{P}\left\{A_N\le B_N\right\}\ge 1-{\kappa }_N$$

(15)

for $N\in \mathbb{N}$, where ${\kappa }_N\in (0,1)$ with ${\sum }_{N=1}^{\infty }{\kappa }_N<+\infty$, we have

$$\mathbb{P}\left\{\underset{N\to \infty }{lim\; inf}{A}_N\le \underset{N\to \infty }{lim}{B}_N\right\}=1.$$

Proof. 

We prove by contradiction. That is, we assume that

$$\mathbb{P}\left\{\underset{N\to \infty }{lim\; inf}{A}_N\le \underset{N\to \infty }{lim}{B}_N\right\}<1.$$

This implies that there exists a subset $\widehat{\Omega }\subseteq \Omega$ with $\mathbb{P}\left(\widehat{\Omega }\right)>0$ such that

$$\underset{N\to \infty }{lim\; inf}{A}_N\left(\widehat{\omega }\right)>\underset{N\to \infty }{lim}{B}_N\left(\widehat{\omega }\right)$$

for every $\widehat{\omega }\in \widehat{\Omega }$. Define the sequence $\left\{{\Omega }_N\right\}$ as

$${\Omega }_N=\left\{\omega \in \Omega :A_N\left(\omega \right)>B_N\left(\omega \right)\right\}$$

for $N\in \mathbb{N}$. Obviously, according to (15), we have $\mathbb{P}\left({\Omega }_N\right)\le {\kappa }_N$, which implies that

$$\sum _{N=1}^{\infty }\mathbb{P}\left({\Omega }_N\right)\le \sum _{N=1}^{\infty }{\kappa }_N<+\infty .$$

Then, we can always choose a sufficiently large $\widehat{N}\in \mathbb{N}$, such that

$$\sum _{N=\widehat{N}}^{\infty }\mathbb{P}\left({\Omega }_N\right)<\mathbb{P}\left(\widehat{\Omega }\right).$$

Choose

$$\omega \in \widehat{\Omega }\setminus \left({\cup }_{N=\widehat{N}}^{\infty }{\Omega }_N\right),$$

and we have

$$A_N\left(\omega \right)\le B_N\left(\omega \right)$$

for all $N\ge \widehat{N}$, which implies that

$$\underset{N\to \infty }{lim\; inf}{A}_N\left(\omega \right)\le \underset{N\to \infty }{lim}{B}_N\left(\omega \right).$$

This contradicts the definition of $\widehat{\Omega }$. We complete the proof. □

The following proposition states that the optimal value and optimal solution set of the data-driven DRO problem (14) converge to those of the original Problem (10), which verifies the reasonability of our data-driven DRO model (14).

Proposition 4

(Asymptotic consistency). Suppose that $g(x,\xi )$ is locally Lipschitz continuous in the following sense:

$$|g(x,\xi )-g(x,{\xi }^{\prime })|\le Lmax{\{1,∥\xi ∥,∥{\xi }^{\prime }∥\}}^{p-1}∥{\xi }^{\prime }-\xi ∥$$

for every $x\in X$, where $L>0$ and $p\ge 1$. Let $x_N$ be any optimal solution of Problem (14). Then, the following assertions hold:

(i) 

${\widehat{v}}_N\to v^{*}$ with probability 1, as $N\to \infty$;

(ii) 

If, moreover, X is closed, $g(·,\xi )$ is lower semicontinuous for every $\xi \in \Xi$ and $g(x,\xi )$ dominates some P-integrable function uniformly with respect to $x\in X$, then, any accumulation point of $\left\{x_N\right\}$ is an optimizer of Problem (10) almost surely.

Proof. 

Part (i): Notice that

$$\begin{array}{cc}\hfill |{\widehat{v}}_N-v^{*}|& =|\underset{x\in X}{min}\underset{Q\in {\mathbb{B}}_{r_N}\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x,\xi )\right]-\underset{x\in X}{min}{\mathbb{E}}_P\left[g(x,\xi )\right]|\hfill \\ \hfill & \le max\{\underset{Q\in {\mathbb{B}}_{r_N}\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x^{*},\xi )\right]-{\mathbb{E}}_P\left[g(x^{*},\xi )\right],\hfill \\ \hfill & {\mathbb{E}}_P\left[g(x_N,\xi )\right]-\underset{Q\in {\mathbb{B}}_{r_N}\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x_N,\xi )\right]\},\hfill \end{array}$$

where $x^{*}$ and $x_N$ are any optimal solutions of Problems (10) and (14), respectively. For the first term on the right-hand side, we have

$$\begin{array}{cc}\hfill \underset{Q\in {\mathbb{B}}_{r_N}\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x^{*},\xi )\right]-{\mathbb{E}}_P\left[g(x^{*},\xi )\right]& \le {\mathbb{E}}_{Q_N}\left[g(x^{*},\xi )\right]-{\mathbb{E}}_P\left[g(x^{*},\xi )\right]+{\delta }_N\hfill \\ \hfill & \le L{\zeta }_p(Q_N,P)+{\delta }_N\hfill \\ \hfill & \le L{r}_N+{\delta }_N\to 0,\hfill \end{array}$$

almost surely, as $N\to \infty$, where the first inequality is due to the definition of supremum for some ${\delta }_N>0$ with ${\delta }_N\to 0$ almost surely and $Q_N\in {\mathbb{B}}_{r_N}\left(P_N\right)$; the second inequality follows from the definition of FM metric. Similarly, we can derive

$${\mathbb{E}}_P\left[g(x_N,\xi )\right]-\underset{Q\in {\mathbb{B}}_{r_N}\left(P_N\right)}{sup}{\mathbb{E}}_Q\left[g(x_N,\xi )\right]\to 0$$

almost surely, as $N\to \infty$. Thus, we obtain that

$$|{\widehat{v}}_N-v^{*}|\to 0\mathrm{almost}\mathrm{surely},\mathrm{as}N\to \infty .$$

Part (ii): Without loss of generality, in the following discussion, we assume $x_N\to \widehat{x}$ with probability 1 as $N\to \infty$. Moreover, we select a sequence $\left\{{ϵ}_k\right\}$ with ${ϵ}_k\in (0,1)$ and ${\sum }_{k=1}^{\infty }{ϵ}_k<\infty$. According to (12), for each pair $({ϵ}_k,r_k)$ with $r_k$ defined in (14), we can select an $N_k\ge N_0({ϵ}_k,r_k)$ ($N_0({ϵ}_k,r_k)$ is defined in (13)) such that

$$\mathbb{P}\left({\mathbb{E}}_P\left[g(x_{N_k},\xi )\right]\le {\widehat{v}}_{N_k}\right)\ge 1-{ϵ}_k.$$

We know from Lemma 6 and assertion (i) that

$$\mathbb{P}\left(\underset{k\to \infty }{lim\; inf}{\mathbb{E}}_P\left[g(x_{N_k},\xi )\right]\le \underset{k\to \infty }{lim}{\widehat{v}}_{N_k}=v^{*}\right)=1.$$

(16)

Then, the following inequalities hold almost surely:

$$v^{*}\stackrel{\left(a\right)}{\le }{\mathbb{E}}_P\left[g(\widehat{x},\xi )\right]\stackrel{\left(b\right)}{\le }{\mathbb{E}}_P\left[\underset{k\to \infty }{lim\; inf}g(x_{N_k},\xi )\right]\stackrel{\left(c\right)}{\le }\underset{k\to \infty }{lim\; inf}{\mathbb{E}}_P\left[g(x_{N_k},\xi )\right]\stackrel{\left(d\right)}{\le }{v}^{*},$$

where (a) follows from $\widehat{x}\in X$ due to the closedness of X; (b) follows from the lower semicontinuity of $g(·,\xi )$ for every $\xi \in \Xi$; (c) is due to Fatou’s lemma; (d) follows from (16). □

Remark 4.

Propositions 3 and 4 establish the finite sample guarantee and the asymptotic consistency, which are two desirable properties of the data-driven DRO problem [18,30]. Different from the existing results in [18] where the Wasserstein ball is used to construct the ambiguity set, we adopt the FM ball. Due to the feature of Wasserstein metric, to ensure the existence of the significance parameter ϵ, they explicitly derived the radius $r(ϵ,N)$ depending on ϵ and N and the finite sample size $N_0\left(ϵ\right)$ depending only on ϵ. In Proposition 3, we view both r and ϵ as parameters because $\widehat{\beta }$ couples with ϵ implicitly in Theorem 3. Moreover, the assumptions for the asymptotic consistency (Proposition 4) are different from those in [18] (Theorem 3.6), where the upper semicontinuity and linear growth were employed. Here we use the locally Lipschitz continuity but a weaker assumption of the lower bound. Specially, Ref. [18] (Theorem 3.6) employs Borel–Cantelli lemma to obtain

$$\mathbb{P}\left\{{\mathbb{E}}_P\left[g(x_N,\xi )\right]\le {\widehat{v}}_{N}forsufficientlylargeN\right\}=1.$$

This is not applicable for our case, so we need Lemma 6.

4.4. Discrete Approximation for DRO Problems with General Moment Information

We consider the following general DRO problem:

$$\underset{x\in X}{min}\underset{P\in \mathcal{P}}{sup}{\mathbb{E}}_P\left[h(x,\xi )\right],$$

(17)

where $X\subseteq {\mathbb{R}}^n$ is a compact set, $h:X\times \Xi \to \mathbb{R}$, $\mathcal{P}:=\{P\in \mathcal{P}(\Xi ):{\mathbb{E}}_P[\Gamma \left(\xi \right)]\in \mathcal{K}\}$, $\mathcal{K}$ is a closed and convex set in the Cartesian product of some finite dimensional vector and/or matrix spaces, and $\Gamma$ is a general mapping on $\Xi$. We implicitly assume that, for each $x\in X$, ${\mathbb{E}}_P\left[h(x,\xi )\right]<+\infty$ for all $P\in \mathcal{P}$.

The above ambiguity set $\mathcal{P}$ is very general, and it covers almost all the available ambiguity sets with moment information (see, e.g., [19], Examples 3–5). Zhang et al. discussed in [33] the quantitative stability of the DRO problem with a general moment information ambiguity set. There are usually two ways to numerically solve Problem (17): One is to use some kind of duality argument to reformulate Problem (17) as a solvable Problem [18,31]; the other is to discretize the ambiguity set, which leads to a saddle point problem in the finite dimensional space [19]. For instance, the discrete approximation in [19] is conducted under a bounded support set. In this part, by employing our results in Section 3, we consider the discrete approximation for problem (17) under weaker conditions.

Denote by ${\widehat{\mathcal{P}}}_N$ the collection of all discrete distributions which have at most N supporting elements, that is,

$${\widehat{\mathcal{P}}}_N=\left\{\sum _{i=1}^N{p}_i{\mathbf{1}}_{{\xi }^i}:\sum _{i=1}^N{p}_i=1,p_j\ge 0,{\xi }^j\in \Xi ,j=1,2,\cdots ,N\right\}.$$

We define the discrete approximation of $\mathcal{P}$ as

$${\mathcal{P}}_N=\{Q\in {\widehat{\mathcal{P}}}_N:{\mathbb{E}}_Q[\Gamma \left(\xi \right)]\in \mathcal{K}\}.$$

Obviously, ${\mathcal{P}}_N\subseteq \mathcal{P}$. Then, the discrete approximation of Problem (17) can be written as

$$\underset{x\in X}{min}\underset{P\in {\mathcal{P}}_N}{sup}{\mathbb{E}}_P\left[h(x,\xi )\right].$$

(18)

We use $v\left(\mathcal{P}\right)$ and $S\left(\mathcal{P}\right)$ to denote the optimal value and optimal solution set of Problem (17). $v\left({\mathcal{P}}_N\right)$ and $S\left({\mathcal{P}}_N\right)$ are the optimal value and optimal solution set of Problem (18). To make sense of the discrete approximation, we hope that Problem (18) can approximately solve Problem (17) when N is sufficiently large.

To continue the following discussion, we define the growth function of Problem (17) ${\psi }_{\mathcal{P}}:{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ as

$${\psi }_{\mathcal{P}}\left(\tau \right)=inf\left\{\underset{P\in \mathcal{P}}{sup}{\mathbb{E}}_P\left[h(x,\xi )\right]-v\left(\mathcal{P}\right):d(x,S\left(\mathcal{P}\right))\ge \tau ,x\in X\right\}$$

and its inverse function is

$${\psi }_{\mathcal{P}}^{-1}\left(t\right):=sup\{\tau \in {\mathbb{R}}_{+}:{\psi }_{\mathcal{P}}\left(\tau \right)\le t\}.$$

Thus, the associated conditioning function ${\Psi }_{\mathcal{P}}:{\mathbb{R}}_{+}\to {\mathbb{R}}_{+}$ is defined as

$${\Psi }_{\mathcal{P}}\left(\eta \right):=\eta +{\psi }_{\mathcal{P}}^{-1}\left(2\eta \right).$$

Immediately, we have the following quantitative stability results:

Proposition 5.

Suppose that: (i) $\mathcal{P}\subseteq {\mathcal{P}}_p(\Xi )$; (ii) $h(x,\xi )\ge g\left(\xi \right)$ for each $\xi \in \Xi$ and a measurable function $g:\Xi \to \mathbb{R}$ with ${\mathbb{E}}_P\left[g\left(\xi \right)\right]>-\infty$ for any $P\in \mathcal{P}$; (iii) $h(·,\xi )$ is lower semicontinuous for each $\xi \in \Xi$; (iv)

$$|h(x,\xi )-h(x,{\xi }^{\prime })|\le L_{h}max{\left\{1,∥\xi ∥,∥{\xi }^{\prime }∥\right\}}^{p-1}∥{\xi }^{\prime }-\xi ∥$$

for each $x\in X$. Then, $S\left(\mathcal{P}\right)\ne \varnothing$ and

$$\begin{array}{cc}\hfill |v\left(\mathcal{P}\right)-v\left({\mathcal{P}}_N\right)|& \le L_h{\zeta }_p(\mathcal{P},{\mathcal{P}}_N),\hfill \\ \hfill \varnothing \ne S\left({\mathcal{P}}_N\right)& \subseteq S\left(\mathcal{P}\right)+{\Psi }_{\mathcal{P}}\left(L_h{\zeta }_p(\mathcal{P},{\mathcal{P}}_N)\right).\hfill \end{array}$$

Proof. 

Since $h(x,\xi )\ge g\left(\xi \right)$ and $h(·,\xi )$ is lower semicontinuous for each $\xi \in \Xi$, we have from Fatou’s lemma that

$$\underset{k\to \infty }{lim\; inf}{\mathbb{E}}_P\left[h(x_k,\xi )\right]\ge {\mathbb{E}}_P\left[\underset{k\to \infty }{lim\; inf}h(x_k,\xi )\right]\ge {\mathbb{E}}_P\left[h(\overline{x},\xi )\right]$$

holds for any $\left\{x_k\right\}\subseteq X$ such that ${lim}_{k\to \infty }{x}_k=\overline{x}$. This implies that ${\mathbb{E}}_P\left[h(·,\xi )\right]$ is lower semicontinuous. According to [34] (Lemma 4.1), ${sup}_{P\in \mathcal{P}}{\mathbb{E}}_P\left[h(·,\xi )\right]$ is lower semicontinuous too. This, together with the compactness of X, ensures that $S\left(\mathcal{P}\right)\ne \varnothing$. Similarly, we can prove that $S\left({\mathcal{P}}_N\right)\ne \varnothing$.

Note that

$$\begin{array}{cc}\hfill |v\left(\mathcal{P}\right)-v\left({\mathcal{P}}_N\right)|& \stackrel{\left(a\right)}{=}\underset{x\in X}{min}\underset{P\in \mathcal{P}}{sup}{\mathbb{E}}_P\left[h(x,\xi )\right]-\underset{x\in X}{min}\underset{Q\in {\mathcal{P}}_N}{sup}{\mathbb{E}}_Q\left[h(x,\xi )\right]\hfill \\ \hfill & \le \underset{x\in X}{max}\left(\underset{P\in \mathcal{P}}{sup}{\mathbb{E}}_P\left[h(x,\xi )\right]-\underset{Q\in {\mathcal{P}}_N}{sup}{\mathbb{E}}_Q\left[h(x,\xi )\right]\right)\hfill \\ \hfill & =\underset{x\in X}{max}\left(\underset{P\in \mathcal{P}}{sup}\underset{Q\in {\mathcal{P}}_N}{inf}\left({\mathbb{E}}_P\left[h(x,\xi )\right]-{\mathbb{E}}_Q\left[h(x,\xi )\right]\right)\right)\hfill \\ \hfill & \stackrel{\left(b\right)}{\le }\underset{x\in X}{max}\left(\underset{P\in \mathcal{P}}{sup}\underset{Q\in {\mathcal{P}}_N}{inf}{L}_h{\zeta }_p(P,Q)\right)\hfill \\ \hfill & =L_h{\zeta }_p(\mathcal{P},{\mathcal{P}}_N),\hfill \end{array}$$

where (a) follows from the fact ${\mathcal{P}}_N\subseteq \mathcal{P}$; (b) is due to the definition of the pth order FM metric.

Finally, based on the first assertion, the inclusion for the optimal solution sets can be analogously derived as that in [11]. □

For simplicity as well as to show the linear relationship more clearly, we write ${\mathbb{E}}_P[\Gamma \left(\xi \right)]$ as $⟨P,\Gamma \left(\xi \right)⟩$ in what follows. We need the following technical assumption to proceed.

Assumption 3

(see [19]). The system $\mathcal{P}:=\{P:{\mathbb{E}}_P[\Gamma \left(\xi \right)]\in \mathcal{K}\}$ satisfies the following Slater condition:

$$⟨\tilde{P},\Gamma \left(\xi \right)⟩+\delta \mathbb{B}\subseteq \mathcal{K}$$

for some $\tilde{P}\in \mathcal{P}(\Xi )$ and $\delta >0$.

Proposition 6.

Suppose that Assumption 3 holds and $\mathcal{P}\subseteq {\mathcal{P}}_p(\Xi )$. Then, there exists an ${\Omega }_0\subseteq \Omega$ with $\mathbb{P}\left({\Omega }_0\right)=0$, such that for any $\widehat{\delta }<\delta$ and $\omega \in \Omega \setminus {\Omega }_0$, we have

$${\zeta }_p(Q,{\mathcal{P}}_{2N})\le \frac{{\zeta }_p(Q,\tilde{P})+1}{\widehat{\delta }}\underset{K\in \mathcal{K}}{inf}∥K-⟨Q,\Gamma \left(\xi \right)⟩∥$$

for any $Q\in {\widehat{\mathcal{P}}}_N$ and $N\ge \widehat{N}(\widehat{\delta },\omega )$, where $\widehat{N}(\widehat{\delta },\omega )$ is a positive integer depending on $\widehat{\delta }$ and ω.

Proof. 

Let the empirical approximation of $\tilde{P}$ be ${\tilde{P}}_N$. Then, we have from the law of large numbers that

$$⟨{\tilde{P}}_N,\Gamma \left(\xi \right)⟩\to ⟨\tilde{P},\Gamma \left(\xi \right)⟩$$

with probability 1, as $N\to \infty$. Equivalently, there exists an ${\Omega }_1\subseteq \Omega$ with $\mathbb{P}\left({\Omega }_1\right)=0$, such that for any $\widehat{\delta }<\delta$ and $\omega \in \Omega \setminus {\Omega }_1$, we have

$$∥⟨{\tilde{P}}_N\left(\omega \right),\Gamma \left(\xi \right)⟩-⟨\tilde{P},\Gamma \left(\xi \right)⟩∥\le \delta -\widehat{\delta }$$

for $N\ge N_1(\widehat{\delta },\omega )$. This implies that

$$⟨{\tilde{P}}_N\left(\omega \right),\Gamma \left(\xi \right)⟩\in ⟨\tilde{P},\Gamma \left(\xi \right)⟩+(\delta -\widehat{\delta })\mathbb{B},$$

or equivalently,

$$⟨{\tilde{P}}_N\left(\omega \right),\Gamma \left(\xi \right)⟩+\widehat{\delta }\mathbb{B}\subseteq ⟨\tilde{P},\Gamma \left(\xi \right)⟩+\delta \mathbb{B}\subseteq \mathcal{K}$$

for $N\ge N_1(\widehat{\delta },\omega )$, where $\mathbb{B}$ is the unit closed ball in the space of $\mathcal{K}$.

Notice that ${\tilde{P}}_N\in {\mathcal{P}}_N$, and hence, for $N\ge N_1(\widehat{\delta },\omega )$, the Slater condition holds with respect to $\widehat{\delta }$ for the system

$${\mathcal{P}}_N:=\{P\in {\widehat{\mathcal{P}}}_N:{\mathbb{E}}_P[\Gamma \left(\xi \right)]\in \mathcal{K}\}.$$

Now we define, for any $Q\in {\widehat{\mathcal{P}}}_N$, ${\rho }_Q={inf}_{K\in \mathcal{K}}∥K-⟨Q,\Gamma \left(\xi \right)⟩∥$ and

$$\overline{Q}=\left(1-\frac{{\rho }_Q}{{\rho }_Q+\widehat{\delta }}\right)Q+\frac{{\rho }_Q}{{\rho }_Q+\widehat{\delta }}{\tilde{P}}_N.$$

Obviously, we have $\overline{Q}\in {\widehat{\mathcal{P}}}_{2N}$. Similar to that proof of [19] (Theorem 2), we again obtain $\overline{Q}\in {\mathcal{P}}_{2N}$. Then, we have

$$\begin{array}{cc}\hfill {\zeta }_p(Q,{\mathcal{P}}_{2N})& \le {\zeta }_p(Q,\overline{Q})=\underset{g\in {\mathcal{G}}_{{\mathrm{FM}}_p}}{sup}\left|⟨Q,g⟩-⟨\overline{Q},g⟩\right|\hfill \\ \hfill & =\frac{{\rho }_Q}{{\rho }_Q+\widehat{\delta }}\underset{g\in {\mathcal{G}}_{{\mathrm{FM}}_p}}{sup}\left|⟨Q,g⟩-⟨{\tilde{P}}_N\left(\omega \right),g⟩\right|\hfill \\ \hfill & \le \frac{{\zeta }_p(Q,{\tilde{P}}_N\left(\omega \right))}{\widehat{\delta }}\underset{K\in \mathcal{K}}{inf}∥K-⟨Q,\Gamma \left(\xi \right)⟩∥\hfill \\ \hfill & \le \frac{{\zeta }_p(Q,\tilde{P})+{\zeta }_p(\tilde{P},{\tilde{P}}_N\left(\omega \right))}{\widehat{\delta }}\underset{K\in \mathcal{K}}{inf}∥K-⟨Q,\Gamma \left(\xi \right)⟩∥\hfill \\ \hfill & \le \frac{{\zeta }_p(Q,\tilde{P})+1}{\widehat{\delta }}\underset{K\in \mathcal{K}}{inf}∥K-⟨Q,\Gamma \left(\xi \right)⟩∥\hfill \end{array}$$

for $N\ge N_2\left(\omega \right)$ and $\omega \in \Omega \setminus {\Omega }_2$ with $\mathbb{P}\left({\Omega }_2\right)=0$, where the last inequality follows from Theorem 2.

Finally, letting ${\Omega }_0:={\Omega }_1\cup {\Omega }_2$ and $\widehat{N}(\widehat{\delta },\omega ):=max\{N_1(\widehat{\delta },\omega ),N_2\left(\omega \right)\}$ completes the proof. □

The following theorem states that the discrete approximation ambiguity set ${\mathcal{P}}_N$ converges to $\mathcal{P}$ as $N\to \infty$ in the sense of FM metrics.

Theorem 7.

Suppose that: (i) Assumption 3 holds; (ii) $\mathcal{P}\subseteq {\mathcal{P}}_p(\Xi )$; (iii)

$$\underset{P\in \mathcal{P}}{sup}∥{\mathbb{E}}_P[\Gamma \left(\xi \right)]∥<+\infty and{C}_{\mathcal{P}}:=\underset{P,Q\in \mathcal{P}}{sup}{\zeta }_p(P,Q)<+\infty .$$

Then,

$$\underset{N\to \infty }{lim}{\zeta }_p(\mathcal{P},{\mathcal{P}}_N)=0$$

with probability 1.

Proof. 

For any $P\in \mathcal{P}$, by the triangle inequality, we have

$$\begin{array}{cc}\hfill {\zeta }_p(P,{\mathcal{P}}_N)& \le {\zeta }_p(P,P_N)+{\zeta }_p(P_N,{\mathcal{P}}_N)\hfill \end{array}$$

where $P_N$ is the empirical distribution of P with N samples. Since $P_N\in {\widehat{\mathcal{P}}}_N\subseteq \mathcal{P}$, we know from Proposition 6 that

$$\begin{array}{cc}\hfill {\zeta }_p(P_N,{\mathcal{P}}_N)& \le \frac{{\zeta }_p(P_N,\tilde{P})+1}{\widehat{\delta }}\underset{K\in \mathcal{K}}{inf}∥K-⟨P_N,\Gamma \left(\xi \right)⟩∥\hfill \\ \hfill & \le \frac{C_{\mathcal{P}}+1}{\widehat{\delta }}∥⟨P,\Gamma \left(\xi \right)⟩-⟨P_N,\Gamma \left(\xi \right)⟩∥\hfill \end{array}$$

for $N\ge \widehat{N}(\widehat{\delta },\omega )$ and almost every $\omega \in \Omega$. Thus, we have

$$\begin{array}{cc}\hfill {\zeta }_p(P,{\mathcal{P}}_N)& \le {\zeta }_p(P,P_N)+\frac{C_{\mathcal{P}}+1}{\widehat{\delta }}∥⟨P,\Gamma \left(\xi \right)⟩-⟨P_N,\Gamma \left(\xi \right)⟩∥.\hfill \end{array}$$

Subsequently,

$$\begin{array}{cc}\hfill {\zeta }_p(\mathcal{P},{\mathcal{P}}_N)& =\underset{P\in \mathcal{P}}{sup}{\zeta }_p(P,{\mathcal{P}}_N)\hfill \\ \hfill & \le \underset{P\in \mathcal{P}}{sup}{\zeta }_p(P,P_N)+\frac{C_{\mathcal{P}}+1}{\widehat{\delta }}\underset{P\in \mathcal{P}}{sup}∥⟨P,\Gamma \left(\xi \right)⟩-⟨P_N,\Gamma \left(\xi \right)⟩∥.\hfill \end{array}$$

For the first term on the right-hand side, the definition of supremum, the boundedness of $\mathcal{P}$, and Theorem 2 give rise to

$$\begin{array}{cc}\hfill \underset{N\to \infty }{lim}\underset{P\in \mathcal{P}}{sup}{\zeta }_p(P,P_N)& \le \underset{N\to \infty }{lim}{\zeta }_p(P^k,P_N^k)+{ϵ}_k={ϵ}_k\hfill \end{array}$$

with probability 1, where $\left\{P^k\right\}$ is a sequence included in $\mathcal{P}$ such that

$$\underset{P\in \mathcal{P}}{sup}{\zeta }_p(P,P_N)\le {\zeta }_p(P^k,P_N^k)+{ϵ}_k$$

and $\left\{{ϵ}_k\right\}$ is a positive sequence with ${ϵ}_k\to 0$ as $N\to \infty$. Thus, we obtain

$$\underset{N\to \infty }{lim}\underset{P\in \mathcal{P}}{sup}{\zeta }_p(P,P_N)=0$$

with probability 1.

Analogously, by the law of large numbers, we can derive that

$$\underset{N\to \infty }{lim}\underset{P\in \mathcal{P}}{sup}∥⟨P,\Gamma \left(\xi \right)⟩-⟨P_N,\Gamma \left(\xi \right)⟩∥=0$$

with probability 1.

Then, we complete the proof. □

The following corollary shows the reasonability for the approximation of Problem (18) to Problem (17).

Corollary 1.

Under the conditions of Proposition 5 and Theorem 7, we have

$$\begin{array}{cc}\hfill |v\left(\mathcal{P}\right)-v\left({\mathcal{P}}_N\right)|& \to 0,\hfill \\ \hfill d\left(S\left({\mathcal{P}}_N\right),S\left(\mathcal{P}\right)\right)& \to 0.\hfill \end{array}$$

with probability 1, as $N\to \infty$.

Remark 5.

In this subsection, we investigated the discrete approximation of the DRO problem with the general moment information ambiguity set. Compared with the existing work [19], we have further weakened the necessary assumptions and extended them to a more general case. Firstly, the Lipschitz continuity of the objective function is required in [19] (Theorem 14) due to the adoption of the Wasserstein metric, so that the upper bound between the discrete approximation of the DRO problem and the original DRO problem can be derived [19] (Proposition 7). We only call for the locally Lipschitz continuity. More importantly, they restricted their discussion to the bounded support set case because the upper bound in [19] (Proposition 7) would be infinity when the support set is unbounded, which is not well defined in this case. However, our support set can be unbounded by employing our convergence results in Section 3.

5. Concluding Remarks

In this study, we investigated different kinds of convergence assertions about data-driven FM metrics and their possible applications. In view of the rich results about Wasserstein metrics (Lemmas 2 and 3), we first established the relationship between the FM metric and the Wasserstein metric (Lemma 4). Based on these results, the non-asymptotic moment estimate (Theorem 1), asymptotic convergence estimate (Theorem 2), and non-asymptotic concentration estimate (Theorem 3) for FM metrics were presented. These convergence assertions for FM metrics were applied to the asymptotic analyses of the empirical approximations of four kinds of stochastic optimization problems. The results sufficiently show the motivations of this study and its importance.

There are still some topics to settle in the future. For example, we leave the numerical tractability for the results in Section 4.3 and Section 4.4 for future work.