Sinkhorn Distributionally Robust Conditional Quantile Prediction with Fixed Design

Abstract

This paper proposes a novel data-driven distributionally robust framework for conditional quantile prediction under the fixed design setting of the covariates, which we refer to as Sinkhorn distributionally robust conditional quantile prediction. We derive a convex programming dual reformulation of the proposed problem and further develop a conic optimization reformulation for the case with finite support. Our method’s superior performance is demonstrated through several numerical experiments, highlighting its effectiveness in practical applications.

1. Introduction

Quantile regression [1] aims to estimate the value of a specific quantile of a given random variable. During the past few decades, quantile regression has been widely applied in various fields of academic research, including economics [2], clinical medicine [3], and environmental science [4], among others. In the financial field, for example, the 1% or 5% quantile describing the left tail of the distribution of the profit and loss account (the so-called Value-at-Risk (VaR)) is of interest to risk managers [5].

Under the random design framework [1,6], theoretical performance bounds are established by assuming that the data samples are independent and identically distributed (i.i.d.). Although this assumption holds in some cases, it is often unrealistic in practice, as external features may be predetermined or designed, preventing data samples from being treated as realizations of a random variable from an underlying distribution. This phenomenon of predetermined or designed external features is known as fixed design in regression analysis [7].

Common methods for solving conditional quantile prediction problems in the random design include sample average approximation (SAA) [8]. SAA has the advantage of strong applicability, but it is limited by increasing computational and storage costs as the sample size grows. However, when using the fixed design setting, the solution obtained using the SAA method may not be robust [9]. To achieve a robust yet not overly conservative solution, one can consider the distributionally robust optimization (DRO) approach [10,11]. DRO, which serves as a modeling paradigm for decision-making under model uncertainty, seeks to identify an optimal decision by minimizing the worst-case (or adversarial) outcome over a set of plausible probability distributions, commonly referred to as the ambiguity set.

The construction of the ambiguity set plays a crucial role in the performance of the DRO method. In the literature, there are two main approaches to constructing ambiguity sets. The first approach involves defining the ambiguity sets based on descriptive statistics, such as moment conditions [11,12,13], shape constraints [14,15], marginal distributions [16,17], etc. The second approach, which has become popular in recent years, considers distributions within a pre-specified statistical distance from a nominal distribution. Usually, the nominal distribution is the empirical distribution of the data samples. Commonly used statistical distances in the literature include $\varphi$-divergence [18,19,20], Wasserstein distance [21,22,23], maximum mean discrepancy [24], and Sinkhorn distance [25,26,27,28].

Among all metric-based ambiguity sets, the Kullback–Leibler (KL) ambiguity set received relatively early attention. However, as argued in [21], when the ambiguity set is centered at the discrete empirical distribution and the unknown true distribution is absolutely continuous with respect to the Lebesgue measure, any KL ambiguity set will necessarily exclude the true distribution. This occurs because the unknown true distribution is a continuous distribution that cannot assign positive probability mass to each training sample. In contrast, the Wasserstein ambiguity set naturally contains both discrete and continuous distributions, thereby overcoming the limitation of the KL ambiguity set. Subsequently, although Wasserstein distance is relatively popular in DRO research, it still has certain limitations. Wasserstein DRO is typically tractable only under stringent conditions on the loss function (see Table 1 in [27]), and for data-driven Wasserstein DRO, where the nominal distribution is finitely supported (usually the empirical distribution), the worst-case distribution is discrete and supported on, at most, $N+1$ points, although the underlying true distribution in many practical applications may be continuous [23]. This raises concerns about whether Wasserstein DRO hedges the correct family of distributions and whether it may lead to overly conservative performance.

To address these potential issues while maintaining the advantages of Wasserstein DRO, this paper adopts the Sinkhorn distance [25,26] to construct the ambiguity set. Sinkhorn distance, a smoothed variant of the Wasserstein distance, is defined as the minimal transport cost between two distributions in an entropy-regularized optimal transport problem. This design improves the computational complexity of Wasserstein distance, with demonstrated efficiency gains enabling widespread applications in domain adaptation [29,30], generative modeling [31,32], and dimensionality reduction [33,34,35].

Sinkhorn DRO refers to constructing an ambiguity set of distributions using Sinkhorn distance, finding the worst-case distribution that maximizes the loss within this ambiguity set, and subsequently finding the optimal decision within the decision set that minimizes the loss corresponding to the worst-case distribution. As argued by [27], computing the Sinkhorn distance and solving the Sinkhorn DRO problem are two different computational tasks. From the perspective of dual formulations, the former is a standard stochastic optimization problem, whereas the latter is a conditional stochastic optimization [36]. In the latter case, computing unbiased gradients becomes extremely challenging. Thus, the Sinkhorn DRO problem is computationally non-trivial.

To the best of our knowledge, research on Sinkhorn DRO is still in its early stages, with representative theoretical works including [27] and [28]. Compared to the Wasserstein DRO, its advantages mainly lie in the fact that the dual problem of Sinkhorn DRO is computationally tractable for a broader class of loss functions, cost functions, nominal distributions, and probability supports [27], and its worst-case distribution is absolutely continuous with respect to a pre-specified reference measure, such as the Lebesgue or counting measure. This characteristic of Sinkhorn DRO highlights its flexibility as a modeling choice and provides a more realistic representation of uncertainty that better aligns with the underlying true distribution in practical scenarios [27].

Currently, under the fixed design setting, the authors of [9] proposed a distributionally robust conditional quantile prediction problem based on a type-1 Wasserstein ambiguity set, establishing the finite sample guarantee and asymptotic consistency. Building on this work, Ref. [37] employed the type-p Wasserstein ambiguity set, thereby generalizing the original problem. Recognizing the advantages of Sinkhorn DRO over Wasserstein DRO, in this paper, we propose a Sinkhorn distributionally robust conditional quantile prediction (DRCQP) problem under the fixed design setting and establish its strong dual reformulation to ensure computational tractability. When the support is finite, the dual form reduces to an exponential cone program. We further investigate the theoretical relationships between our proposed problem and some classic DRCQP problems. Finally, through empirical analysis, we evaluate the out-of-sample performance of our proposed method and conduct sensitivity analyses of the key parameter $\eta$, thereby providing practical references for real-world applications.

The remainder of the paper is structured as follows. Section 2 presents the preliminary knowledge, including the standard linear regression model and the conditional quantile regression model. Section 3 introduces the definition of Sinkhorn distance and establishes the Sinkhorn DRCQP problem under the fixed design setting. Section 4 aims to tackle the proposed problem by providing an equivalent tractable reformulation under some mild assumptions, along with a conic optimization reformulation developed for finite support cases. Section 5 evaluates the performance of our DRO method through numerical experiments. Finally, Section 6 concludes the paper with a summary of key findings.

Notations: Let ${\mathbb{R}}^n$ denote the n-dimensional Euclidean space. For a positive integer N, let $\left[N\right]$ represent the set $\{1,2,\dots ,N\}$. For a measurable set $\mathcal{Z}$, denote the set of measures (not necessarily probability measures) as $\mathcal{M}\left(\mathcal{Z}\right)$ on $\mathcal{Z}$, and let $\mathcal{P}\left(\mathcal{Z}\right)$ denote the set of probability measures on $\mathcal{Z}$. Given a probability distribution $\mathbb{P}$ and a measure $\mu$, we denote $\mathbb{P}\ll \mu$ if $\mathbb{P}$ is absolutely continuous with respect to $\mu$. $\mathbb{P}\otimes \mathbb{Q}$ denotes the product measure of two probability distributions $\mathbb{P}$ and $\mathbb{Q}$. Given a measure $\mu \in \mathcal{M}\left(\mathcal{Z}\right)$ and a measurable variable $f:\mathcal{Z}\to \mathbb{R}$, we write ${\mathbb{E}}_{z\sim \mu }\left[f\left(z\right)\right]$ for ${\int }_{\mathbb{R}}f\left(z\right)\mathrm{d}\mu \left(z\right)$. For a given element x, denote the one-point probability distribution as ${\delta }_z$ supported on $\left\{z\right\}$.

2. Preliminaries

In this paper, we consider $Y\in \mathbb{R}$ as the response variable of interest and $(X_1,\dots ,X_p)\in {\mathbb{R}}^p$ as the vector of explanatory variables. The standard linear regression model without an intercept term is

$$Y={\beta }_1{X}_1+{\beta }_2{X}_2+\cdots +{\beta }_p{X}_p+\epsilon ,$$

(1)

where $\mathit{\beta }={({\beta }_1,\dots ,{\beta }_p)}^{\top }\in {\mathbb{R}}^p$ is the vector of unknown regression parameters, and $\epsilon \in \mathbb{R}$ is a random variable of residuals. We assume that $\epsilon$ is independent of $(X_1,\cdots ,X_p)$ with zero mean and unknown variance ${\sigma }^2$.

The ordinary least squares (OLSs) method is the classic approach to estimating the unknown $\mathit{\beta }$, but it is highly sensitive to outliers and performs poorly when $\epsilon$ follows the non-Gaussian distribution [1]. To overcome these limitations, Koenker and Basset [1] proposed quantile regression, which minimizes the asymmetric absolute deviations to characterize how explanatory variables affect different quantiles of the conditional distribution of the response variable Y. This research originated from the definition of the $\alpha$-quantile $Q_{\alpha }\left(Y\right)$ of the response variable Y:

$$Q_{\alpha }\left(Y\right)=inf\{q\in \mathbb{R}:\mathbb{P}(Y\le q)\ge \alpha \},0<\alpha <1.$$

(2)

In addition to (2), $Q_{\alpha }\left(Y\right)$ can also be expressed as the solution to the following stochastic optimization problem:

$$Q_{\alpha }\left(Y\right)=\underset{q\in \mathbb{R}}{argmin}\mathbb{E}\left[{\rho }_{\alpha }(Y-q)\right],$$

(3)

where ${\rho }_{\alpha }$ is the quantile loss function (also called the check function), defined as

$${\rho }_{\alpha }\left(u\right)=u(\alpha -{\mathbf{1}}_{(u\le 0)})=\left\{\begin{array}{cc}\alpha u,\hfill & \mathrm{if}u>0,\hfill \\ (\alpha -1)u,\hfill & \mathrm{if}u\le 0.\hfill \end{array}\right.$$

which assigns different weights to positive and negative residuals.

In the fixed design setting, we focus on the out-of-sample performance with certain feature vectors $(X_1,\cdots ,X_p)={(x_1,\cdots ,x_p)}^f:={\mathit{x}}^f$ (Here, the superscript “f” in ${\mathit{x}}^f$ represents the fixed design.). According to the linear problem (1) and the independence between $(X_1,\cdots ,X_p)$ and $\epsilon$, the $\alpha$-th conditional quantile $Q_{\alpha }(Y|{\mathit{x}}^f)$ of the response variable Y takes the following form:

$$Q_{\alpha }(Y|{\mathit{x}}^f)=Q_{\alpha }({\mathit{\beta }}^{\top }{\mathit{x}}^f+\epsilon |{\mathit{x}}^f)={\mathit{\beta }}^{\top }{\mathit{x}}^f+Q_{\alpha }\left(\epsilon \right)={\mathit{\beta }}^{\top }{\mathit{x}}^f+s_{\alpha },$$

where $s_{\alpha }:=Q_{\alpha }\left(\epsilon \right)$ is defined as the $\alpha$-th quantile of $\epsilon$. Consider N observed samples ${\left\{({\mathit{x}}_i^f,y_i)\right\}}_{i=1}^N$ generated from model (1). According to [1], for $\alpha \in (0,1)$, the $\alpha$-th regression quantile is defined as the solution to the following problem:

$$\underset{\tilde{\mathit{\beta }}\in {\mathbb{R}}^p,s\in \mathbb{R}}{min}\sum _{i=1}^N{\rho }_{\alpha }\left(y_i-{\tilde{\mathit{\beta }}}^{\top }{\mathit{x}}_i^f-s\right).$$

The aforementioned quantile regression problem can be viewed as applying the sample average approximation (SAA) method to solve the conditional stochastic optimization problem in (3). Under the random design setting, the SAA method performs well since the empirical distribution converges to the true distribution as the sample size N increases. However, under the fixed design setting, this convergence cannot be guaranteed, resulting in the deterioration of estimation efficiency. This motivates us to find more effective methods to solve the conditional quantile regression problem under the fixed design setting.

3. Problem Setup

3.1. Sinkhorn Distance

The core of distributionally robust optimization (DRO) lies in selecting a suitable ambiguity set. Such a set must balance computational tractability with practical interpretability. It should be sufficiently rich to cover relevant distributions yet exclude unnecessary ones that may lead to overly conservative decisions. Recently, the Sinkhorn distance has gained significant attention for constructing ambiguity sets. Its definition is as follows:

Definition 1

(Sinkhorn distance, [25,26,27]). Let $\mathcal{Z}$ be a measurable set. Consider distributions $\mathbb{P},\mathbb{Q}\in \mathcal{P}\left(\mathcal{Z}\right)$, and let $\mu ,\nu \in \mathcal{M}\left(\mathcal{Z}\right)$ be two reference measures such that $\mathbb{P}\ll \mu ,\mathbb{Q}\ll \nu$. For the entropic regularization parameter $\eta \ge 0$, the Sinkhorn distance between two distributions $\mathbb{P}$ and $\mathbb{Q}$ is defined as

$${\mathcal{W}}_{\eta }(\mathbb{P},\mathbb{Q})=\underset{\gamma \in \Gamma (\mathbb{P},\mathbb{Q})}{inf}\{{\mathbb{E}}_{(z,z^{\prime })\sim \gamma }\left[d{(z,z^{\prime })}^p\right]+\eta H(\gamma |\mu \otimes \nu )\},$$

where $\Gamma (\mathbb{P},\mathbb{Q})$ is defined as the set of all Borel probability distributions on $\mathcal{Z}\times \mathcal{Z}$ with marginal distributions $\mathbb{P}$ and $\mathbb{Q}$, $d{(z,z^{\prime })}^p$ denoting the cost function, and $H(\gamma |\mu \otimes \nu )$ denotes the relative entropy (also called Kullback–Leibler divergence, KL divergence) of γ with respect to the product measure $\mu \otimes \nu$:

$$H(\gamma |\mu \otimes \nu )={\mathbb{E}}_{(z,z^{\prime })\sim \gamma }\left[log\left({\displaystyle \frac{\mathrm{d}\gamma (z,z^{\prime })}{\mathrm{d}\mu \left(z\right)\mathrm{d}\nu \left(z^{\prime }\right)}}\right)\right].$$

When $\eta =0$, the Sinkhorn distance ${\mathcal{W}}_{\eta }(\mathbb{P},\mathbb{Q})$ reduces to the classic Wasserstein metric, and as $\eta$ goes to infinity, the entropic regularization term dominates; the distance increasingly emphasizes the entropy of the coupling rather than the transport cost. In this paper, we study a distributionally robust conditional quantile prediction problem, which is formulated as a one-dimensional minmax optimization problem. This leads us to focus on $\mathcal{Z}\subseteq \mathbb{R}$, and we take the standard distance $d(z,z^{\prime })=|z-z^{\prime }|$.

Based on the definition of the Sinkhorn distance, we introduce the Sinkhorn ball centered on a reference distribution to effectively capture and characterize the distributional ambiguity in our analysis.

Definition 2.

For a fixed distribution $\mathbb{P}\in \mathcal{P}\left(\mathcal{Z}\right)$ and $r>0$, the Sinkhorn ball ${\mathbb{B}}_{r,\eta }\left(\mathbb{P}\right)$ of radius r centered on $\mathbb{P}$ is defined as

$${\mathbb{B}}_{r,\eta }\left(\mathbb{P}\right):=\left\{\mathbb{Q}\in \mathcal{P}\left(\mathcal{Z}\right)|{\mathcal{W}}_{\eta }(\mathbb{P},\mathbb{Q})\le r\right\}.$$

(4)

Due to the convex entropic regularizer in ${\mathcal{W}}_{\eta }(\mathbb{P},\mathbb{Q})$ with respect to $\mathbb{P}$[38], the Sinkhorn distance ${\mathcal{W}}_{\eta }(\mathbb{P},\mathbb{Q})$ is convex in $\mathbb{P}$, i.e., ${\mathcal{W}}_{\eta }(\mathbb{P},\lambda {\mathbb{Q}}_1+(1-\lambda ){\mathbb{Q}}_2)\le \lambda {\mathcal{W}}_{\eta }(\mathbb{P},{\mathbb{Q}}_1)+(1-\lambda ){\mathcal{W}}_{\eta }(\mathbb{P},{\mathbb{Q}}_2)$ for all probability distributions ${\mathbb{Q}}_1,{\mathbb{Q}}_2\in \mathcal{P}\left(\mathcal{Z}\right)$ and all $0\le \lambda \le 1$. Therefore, the Sinkhorn ball ${\mathbb{B}}_{r,\eta }\left(\mathbb{P}\right)$ is a convex set.

3.2. Sinkhorn Distributionally Robust Conditional Quantile Prediction with Fixed Design

Under the fixed design setting, the optimal solution obtained by the SAA method for conditional quantile prediction is not robust [9]. To obtain a robust yet not overly conservative solution, we employ the DRO approach for modeling conditional quantile prediction, building upon similar methodologies proposed in [9]. The corresponding mathematical formulation is presented as follows:

$$\underset{\tilde{\mathit{\beta }}\in {\mathbb{R}}^p,s\in \mathbb{R}}{min}\underset{{\mathbb{P}}_Y\in {\mathcal{P}}_Y}{sup}{\mathbb{E}}_{{\mathbb{P}}_Y}\left[{\rho }_{\alpha }\left(Y-{\tilde{\mathit{\beta }}}^{\top }{\mathit{x}}^f-s\right)\right],$$

(5)

where ${\mathbb{P}}_Y$ is a distribution of Y conditional on ${\mathit{x}}^f$, and ${\mathcal{P}}_Y$ is an ambiguity set constructed from the observed data points ${\left\{({\mathit{x}}_i^f,y_i)\right\}}_{i=1}^N$ that are generated by model (1).

To transfer the distributional uncertainty from the response variable Y to the residual term $\tilde{\epsilon }$ and thereby simplify problem (5), we define $\tilde{\epsilon }=Y-{\tilde{\mathit{\beta }}}^{\top }{\mathit{x}}^f=\epsilon +{(\mathit{\beta }-\tilde{\mathit{\beta }})}^{\top }{\mathit{x}}^f$. Thus, problem (5) can be rewritten as

$$\underset{\tilde{\mathit{\beta }}\in {\mathbb{R}}^p,s\in \mathbb{R}}{min}\underset{{\mathbb{P}}_{\tilde{\epsilon }}\in {\mathcal{P}}_{\tilde{\epsilon }}}{sup}{\mathbb{E}}_{{\mathbb{P}}_{\tilde{\epsilon }}}\left[{\rho }_{\alpha }(\tilde{\epsilon }-s)\right],$$

where ${\mathbb{P}}_{\tilde{\epsilon }}$ and ${\mathcal{P}}_{\tilde{\epsilon }}$ are, respectively, the candidate distribution and ambiguity set of $\tilde{\epsilon }$.

Note that ${\mathcal{P}}_{\tilde{\epsilon }}$ depends on the unknown $\tilde{\mathit{\beta }}$. Following [9], we first estimate $\tilde{\mathit{\beta }}$ using the OLS estimator ${\widehat{\mathit{\beta }}}_N^{\mathrm{OLS}}$, which can be expressed as

$${\widehat{\mathit{\beta }}}_N^{\mathrm{OLS}}={\left({\left({\mathit{X}}^f\right)}^{\top }{\mathit{X}}^f\right)}^{-1}{\left({\mathit{X}}^f\right)}^{\top }\mathit{y},$$

where N samples ${\left\{({\mathit{x}}_i,y_i)\right\}}_{i=1}^N$ are observed from model (1), and ${\mathit{X}}^f$ is the fixed design matrix with a sequence of (row) vectors $\{{\left({\mathit{x}}_1^f\right)}^{\top },\cdots ,{\left({\mathit{x}}_N^f\right)}^{\top }\}$ and $\mathit{y}={(y_1,\cdots ,y_N)}^{\top }$. Notably, although the row vectors $\{{\left({\mathit{x}}_1^f\right)}^{\top },\cdots ,{\left({\mathit{x}}_N^f\right)}^{\top }\}$ are not i.i.d., we still assume the fixed design matrix ${\mathit{X}}^f$ satisfies Assumptions 1 and 2 in [9]; thus, ${\widehat{\mathit{\beta }}}_N^{\mathrm{OLS}}$ is a consistent estimator of $\tilde{\mathit{\beta }}$.

We define ${\epsilon }^{\prime }=\epsilon +{(\mathit{\beta }-{\widehat{\mathit{\beta }}}_N^{\mathrm{OLS}})}^{\top }{\mathit{x}}^f$ with an unknown distribution ${\mathbb{P}}_{{\epsilon }^{\prime }}$, which has the corresponding samples ${\left\{{\epsilon }_i^{\mathrm{OLS}}\right\}}_{i=1}^N$ with ${\epsilon }_i^{\mathrm{OLS}}=y_i-{\left({\widehat{\mathit{\beta }}}_N^{\mathrm{OLS}}\right)}^{\top }{\mathit{x}}_i^f$. Its empirical distribution ${\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\delta }_{{\epsilon }_i^{\mathrm{OLS}}}$ leads to the Sinkhorn ball ${\mathbb{B}}_{r,\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\right)$, where ${\mathbb{B}}_{r,\eta }\left(\mathbb{P}\right)$ is defined by (4). With this clarification, we now formally introduce our distributionally robust conditional quantile prediction problem based on the Sinkhorn ball:

$$\underset{s\in \mathbb{R}}{min}\underset{{\mathbb{P}}_{{\epsilon }^{\prime }}\in {\mathbb{B}}_{r,\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\right)}{sup}{\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }({\epsilon }^{\prime }-s)\right],$$

(6)

where ${\mathbb{P}}_{{\epsilon }^{\prime }}$ and ${\mathbb{B}}_{r,\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\right)$ denote the candidate distribution and the Sinkhorn uncertainty set of ${\epsilon }^{\prime }$, respectively. We refer to problem (6) as a Sinkhorn distributionally robust conditional quantile prediction (DRCQP) problem. In the remainder of this paper, we will continue to study this problem and present its tractable reformulation.

4. Tractable Reformulation

In this section, we focus on the Sinkhorn DRCQP problem (6), which is intractable due to the inner maximization problem involving uncountable candidate distributions within the Sinkhorn ambiguity set. Thus, we provide a strong dual reformulation that effectively transforms this problem into a finite-dimensional optimization problem. Moreover, under the assumption of a finite support, problem (6) can be reformulated as an equivalent conic optimization problem, which facilitates its analysis and solution. Finally, we provide the theoretical connections between problem (6) and other formulations studied in the machine learning literature.

Following the discussion in Remark 2 of [27], we take the reference measure $\mu ={\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)$ and let $\nu$ be the Lebesgue measure. Other options for reference measures are discussed in Remark 2 of [27]. For notational clarity, we define

$$\overline{r}:=r+{\displaystyle \frac{\eta }{N}}\sum _{i=1}^{N}log\left({\int }_{\mathbb{R}}{e}^{-|{\epsilon }_i^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p/\eta }\mathrm{d}{\epsilon }^{\prime }\right).$$

(7)

To reformulate problem (6), we make the following assumption:

Assumption 1.

(i)    $\nu \left\{z:0\le |{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p<\infty \right\}=1$ for ${\widehat{\mathbb{P}}}_N$-almost every ${\epsilon }^{\mathrm{OLS}};$

(ii) 

${\int }_{\mathbb{R}}{e}^{-|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p/\eta }\mathrm{d}{\epsilon }^{\prime }<\infty$ for ${\widehat{\mathbb{P}}}_N$ almost every ${\epsilon }^{\mathrm{OLS}};$

(iii) 

$\mathcal{Z}$ is a measurable space, and the objective function in problem (6) is measurable;

(iv) 

For every joint distribution γ on $\mathcal{Z}\times \mathcal{Z}$ with a first marginal distribution ${\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)$, it has a regular conditional distribution ${\gamma }_{{\epsilon }^{\mathrm{OLS}}}$, given that the value of the first marginal equals ${\epsilon }^{\mathrm{OLS}}.$

Assumption 1 is commonly imposed to ensure both the Sinkhorn distance ${\mathcal{W}}_{\eta }(\mathbb{P},\mathbb{Q})$ and the expected loss ${\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }({ϵ}^{\prime }-s)\right]$ are well-defined. For a more detailed discussion of this assumption, we refer to [27]. Given Assumption 1, we can derive a convex dual reformulation of problem (6).

Theorem 1.

Let the reference measure $\mu ={\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)$, ν be the Lebesgue measure in $\mathbb{R}$ and $\overline{r}\ge 0$. Under Assumption 1, problem (6) admits the following equivalent reformulation:

$$\underset{s\in \mathbb{R},\lambda \ge 0}{min}\left\{F(s,\lambda )\triangleq \lambda \overline{r}+{\displaystyle \frac{\lambda \eta }{N}}\sum _{i=1}^{N}log\left({\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{{\rho }_{\alpha }({\epsilon }^{\prime }-s)/\left(\lambda \eta \right)}\right]\right)\right\},$$

(8)

where $\overline{r}$ is given by (7), and the kernel probability distribution ${\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)$ is given by

$${\displaystyle \frac{\mathrm{d}{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\epsilon }^{\prime }}}={\displaystyle \frac{e^{-|{\epsilon }_i^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p/\eta }}{{\int }_{\mathbb{R}}{e}^{-|{\epsilon }_i^{\mathrm{OLS}}{-\zeta |}^p/\eta }\mathrm{d}\zeta }},i\in \left[N\right].$$

(9)

Proof of Theorem 1.

We begin by addressing the inner maximization problem in (6),

$$\underset{{\mathbb{P}}_{{\epsilon }^{\prime }}\in {\mathbb{B}}_{r,\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\right)}{sup}{\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }({\epsilon }^{\prime }-s)\right].$$

(10)

For a fixed $s\in \mathbb{R}$, problem (10) is equivalent to

$$\begin{array}{cc}\hfill \underset{{\mathbb{P}}_{{\epsilon }^{\prime }}\in \mathcal{P}\left(\mathbb{R}\right)}{sup}& {\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)\right]\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathcal{W}}_{\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right),{\mathbb{P}}_{{\epsilon }^{\prime }}\right)\le r.\hfill \end{array}$$

(11)

According to the definition of the Sinkhorn distance ${\mathcal{W}}_{\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right),{\mathbb{P}}_{{\epsilon }^{\prime }}\right)$ with $\nu$ taken as the Lebesgue measure on $\mathbb{R}$ (i.e., $\mathrm{d}\nu \left({\epsilon }^{\prime }\right)=\mathrm{d}{\epsilon }^{\prime }$), problem (11) admits the following equivalent formulation:

$$\begin{array}{cc}\hfill \underset{\gamma \in \mathcal{P}(\mathbb{R}\times \mathbb{R})}{sup}& {\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)\right]\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& {\mathbb{E}}_{({\epsilon }^{\mathrm{OLS}},{\epsilon }^{\prime })\sim \gamma }\left[|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p+\eta log\left({\displaystyle \frac{\mathrm{d}\gamma ({\epsilon }^{\mathrm{OLS}},{\epsilon }^{\prime })}{\mathrm{d}{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\mathrm{d}{\epsilon }^{\prime }}}\right)\right]\le r,\hfill \end{array}$$

(12)

where ${\epsilon }^{\mathrm{OLS}}\sim {\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)$ and ${\epsilon }^{\prime }\sim {\mathbb{P}}_{{\epsilon }^{\prime }}$. The joint distribution $\gamma ({\epsilon }^{\mathrm{OLS}},{\epsilon }^{\prime })$ has marginal distributions ${\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)$ and ${\mathbb{P}}_{{\epsilon }^{\prime }}$, respectively. According to the disintegration theorem in [39], we represent the joint distribution $\gamma$ such that $\mathrm{d}\gamma ({\epsilon }^{\mathrm{OLS}},{\epsilon }^{\prime })=\mathrm{d}{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)$ holds for any $({\epsilon }^{\mathrm{OLS}},{\epsilon }^{\prime })$, where ${\gamma }_{{\epsilon }^{\mathrm{OLS}}}$ is the conditional distribution of ${\epsilon }^{\prime }$, given the first marginal of $\gamma$ equals ${\epsilon }^{\mathrm{OLS}}$. Thereby, the constraint in problem (12) is equivalent to

$$\begin{array}{cc}\hfill & {\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}{\mathbb{E}}_{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}\left[|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p+\eta log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\epsilon }^{\prime }}}\right)\right]\le r.\hfill \end{array}$$

(13)

Note that any feasible solution $\gamma$ satisfies $\gamma \ll {\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\otimes \nu$, and hence, ${\gamma }_{{\epsilon }^{\mathrm{OLS}}}\ll \nu$. Since $\nu$ is the Lebesgue measure in $\mathbb{R}$, the term $log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\epsilon }^{\prime }}}\right)$ is well-defined. By applying the change-in-measure identity, we have

$$log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\epsilon }^{\prime }}}\right)=log\left({\displaystyle \frac{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\epsilon }^{\prime }}}\right)+log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}}\right),$$

in which

$${\displaystyle \frac{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\epsilon }^{\prime }}}={\displaystyle \frac{e^{-|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p/\eta }}{{\int }_{\mathbb{R}}{e}^{-|{\epsilon }^{\mathrm{OLS}}{-\zeta |}^p/\eta }\mathrm{d}\zeta }}.$$

Thus, constraint (13) can be reformulated as

$${\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}{\mathbb{E}}_{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}\left[|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p+\eta log\left({\displaystyle \frac{e^{-|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p/\eta }}{{\int }_{\mathbb{R}}{e}^{-|{\epsilon }^{\mathrm{OLS}}{-\zeta |}^p/\eta }\mathrm{d}\zeta }}\right)+\eta log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}}\right)\right]\le r,$$

(14)

where ${\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)\in \mathcal{P}\left(\mathbb{R}\right)$. For the integrand in the expectation term on the left-hand side of constraint (14), we can write

$$\eta log\left({\displaystyle \frac{e^{-|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p/\eta }}{{\int }_{\mathbb{R}}{e}^{-|{\epsilon }^{\mathrm{OLS}}{-\zeta |}^p/\eta }\mathrm{d}\zeta }}\right)=-{|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }|}^p-\eta log\left({\int }_{\mathbb{R}}{e}^{-|{\epsilon }^{\mathrm{OLS}}{-\zeta |}^p/\eta }\mathrm{d}\zeta \right).$$

By combining it with the first term $|{\epsilon }^{\mathrm{OLS}}-{\epsilon }^{\prime }{|}^p$, we have (14) being equivalent to

$$\eta {\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}{\mathbb{E}}_{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}\left[log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}}\right)\right]\le \overline{r}.$$

(15)

Similarly, the objective function of problem (10) can be written as

$${\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}{\mathbb{E}}_{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}\left[{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)\right].$$

By introducing the Lagrange multiplier $\lambda$ associated with constraint (15), we can reformulate problem (10) as

$$\underset{{\gamma }_{{\epsilon }^{\mathrm{OLS}}\left({\epsilon }^{\prime }\right)}\in \mathcal{P}\left(\mathbb{R}\right)}{sup}\underset{\lambda \ge 0}{inf}\left\{{\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}{\mathbb{E}}_{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}\left[{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)-\lambda \eta log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}}\right)\right]+\lambda \overline{r}\right\}.$$

(16)

Note that its objective function ${\rho }_{\alpha }({\epsilon }^{\prime }-s)$ is convex in ${\epsilon }^{\prime }$, and the feasible region ${\mathbb{B}}_{r,\eta }\left({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)\right)$ is a convex set; we have that problem (10) is a convex problem, and thus, the strong duality holds. Therefore, under Assumption 1, according to Theorem 1 of [27], problem (16) is equivalent to

$$\begin{array}{cc}\hfill & \underset{\lambda \ge 0}{inf}\underset{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)\in \mathcal{P}\left(\mathbb{R}\right)}{sup}\left\{{\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}{\mathbb{E}}_{{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}\left[{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)-\lambda \eta log\left({\displaystyle \frac{\mathrm{d}{\gamma }_{{\epsilon }^{\mathrm{OLS}}}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}}\right)\right]+\lambda \overline{r}\right\}\hfill \\ \hfill =& \underset{\lambda \ge 0}{inf}\left\{\lambda \overline{r}+\lambda \eta {\mathbb{E}}_{{\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right)}\left[log{\mathbb{E}}_{{\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }\left({\epsilon }^{\prime }\right)}\left(e^{{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)/\left(\lambda \eta \right)}\right)\right]\right\}.\hfill \end{array}$$

(17)

For $i=1,\cdots ,N$, given ${\epsilon }^{\mathrm{OLS}}={\epsilon }_i^{\mathrm{OLS}}$, denote ${\mathbb{Q}}_i:={\mathbb{Q}}_{{\epsilon }^{\mathrm{OLS}},\eta }$, $i\in \left[N\right]$. One can verify that for any $({\epsilon }^{\mathrm{OLS}},{\epsilon }^{\prime })$, there exist distributions ${\left\{{\mathbb{Q}}_i\right\}}_{i=1}^N$ such that the joint distribution $\gamma :={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{\delta }_{{\epsilon }_i^{\mathrm{OLS}}}\otimes {\mathbb{Q}}_i$. As a result, problem (17) is equivalent to

$$\underset{\lambda \ge 0}{inf}\left\{\lambda \overline{r}+{\displaystyle \frac{\lambda \eta }{N}}\sum _{i=1}^{N}log\left({\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{{\rho }_{\alpha }({\epsilon }^{\prime }-s)/\left(\lambda \eta \right)}\right]\right)\right\}.$$

(18)

Therefore, problem (6) is equivalent to problem (8). This completes the proof. □

From a computational perspective, the dual reformulation of Sinkhorn DRCQP (8) is non-trivial, as it can be viewed as a conditional stochastic optimization problem [36]. To solve it efficiently, ref. [27] proposed an efficient static mirror descent algorithm with biased subgradient estimators for Sinkhorn DRO problems, and ref. [40] developed a Nested-SGD algorithm designed for large-scale regularized Sinkhorn DRO problems. Detailed discussions of sample complexity and convergence guarantees are provided in these works.

Following Wang et al. [27], to distinguish the two cases $F(s,\lambda )<\infty$ and $F(s,\lambda )=\infty$, we introduce the light-tail condition on the quantile loss function ${\rho }_{\alpha }({\epsilon }^{\prime }-s)$.

Assumption 2.

$\tilde{\lambda }>0$ exists such that ${\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{|{\epsilon }^{\prime }-s|/\left(\tilde{\lambda }\eta \right)}\right]<\infty$ for ${\widehat{\mathbb{P}}}_N$ almost every ${\epsilon }^{\mathrm{OLS}}$.

Moreover, when the support $\mathcal{Z}$ is finite, the dual formulation (8) can be reformulated as the following conic programming problem under Assumption 2 (above). Therefore, the new formulation can be solved efficiently using off-the-shelf solvers such as CVX [41], which employ the interior point method.

Theorem 2

(Conic reformulation for finite support). Suppose that the support $\mathcal{Z}$ contains $L_{max}$ elements, i.e., $\mathcal{Z}={\left\{{\epsilon }_j^{\prime }\right\}}_{j=1}^{L_{max}}$. If Assumption 2 holds and $\overline{r}\ge 0$, problem (8) can be formulated as the following conic optimization:

$$\begin{array}{cc}\hfill \underset{\begin{array}{c}\lambda ,s,g,a\end{array}}{min}& \lambda \overline{r}+{\displaystyle \frac{1}{N}}\sum _{i=1}^N{g}_i\hfill \\ \hfill \mathrm{s}.\mathrm{t}.& \lambda \eta \ge \sum _{j=1}^{L_{max}}{q}_{i,j}{a}_{i,j},i\in \left[N\right],\hfill \\ \hfill & (\lambda \eta ,a_{i,j},{\rho }_{\alpha }({\epsilon }_j^{\prime }-s)-g_i)\in {\mathcal{K}}_{exp},i\in \left[N\right],j\in \left[L_{max}\right],\hfill \\ \hfill & \lambda \ge 0,s\in \mathbb{R},g\in {\mathbb{R}}^N,a\in {\mathbb{R}}^{N\times L_{max}}.\hfill \end{array}$$

(19)

where $q_{i,j}:={\mathbb{P}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\{{\epsilon }^{\prime }={\epsilon }_j^{\prime }\}$ with the distribution ${\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)$ defined in (9), and ${\mathcal{K}}_{exp}$ denotes the exponential cone ${\mathcal{K}}_{exp}=\left\{(\nu ,\lambda ,\delta )\in {\mathbb{R}}_{+}\times {\mathbb{R}}_{+}\times \mathbb{R}:exp(\delta /\nu )\le \lambda /\nu \right\}$.

Proof of Theorem 2.

We now introduce the epi-graphical variables $g_i,i=1,\cdots ,N$ to reformulate the dual objective (8) as

$$\begin{array}{cc}\hfill \underset{s,\lambda ,g_i}{min}& \lambda \overline{r}+{\displaystyle \frac{1}{N}}\sum _{i=1}^N{g}_i\hfill \\ \hfill s.t.& \lambda \eta log\left({\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{{\rho }_{\alpha }({\epsilon }^{\prime }-s)/\left(\lambda \eta \right)}\right]\right)\le g_i,\forall i\hfill \\ \hfill & s\in \mathbb{R},\lambda >0,g_i\in \mathbb{R}.\hfill \end{array}$$

For a fixed i, the i-th constraint can be reformulated as

$$\begin{array}{cc}\hfill & \left\{exp\left({\displaystyle \frac{g_i}{\lambda \eta }}\right)\ge {\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{{\rho }_{\alpha }({\epsilon }^{\prime }-s)/\left(\lambda \eta \right)}\right]\right\}\hfill \\ \hfill =& \left\{1\ge {\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{({\rho }_{\alpha }({\epsilon }^{\prime }-s)-g_i)/\left(\lambda \eta \right)}\right]\right\}\hfill \\ \hfill =& \left\{\lambda \eta \ge {\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[\lambda \eta e^{({\rho }_{\alpha }({\epsilon }^{\prime }-s)-g_i)/\left(\lambda \eta \right)}\right]\right\}\hfill \\ \hfill =& \left\{\lambda \eta \ge \sum _{j=1}^{L_{max}}{q}_{i,j}{a}_{i,j}\right\}\bigcap \left\{a_{i,j}\ge \lambda \eta exp\left({\displaystyle \frac{{\rho }_{\alpha }({\epsilon }_j^{\prime }-s)-g_i}{\lambda \eta }}\right),\forall j\right\},\hfill \end{array}$$

where the second constraint set can be formulated as

$$(\lambda \eta ,a_{i,j},{\rho }_{\alpha }({\epsilon }_j^{\prime }-s)-g_i)\in {\mathcal{K}}_{exp}.$$

Substituting this expression into the dual objective completes the proof. □

The following remarks elucidate the connection between the Sinkhorn DRCQP problem (8) and the DRCQP problems based on other ambiguity sets.

Remark 1

(Connection with Wasserstein DRCQP). As the regularization parameter $\eta \to 0$, formulation (6) reduces to

$$\underset{s}{min}\underset{{\mathbb{P}}_{{\epsilon }^{\prime }}\in \mathcal{P}\left({\epsilon }^{\prime }\right)}{sup}{\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }({\epsilon }^{\prime }-s)\right],$$

where $\mathcal{P}\left({\epsilon }^{\prime }\right)=\{{\mathbb{P}}_{{\epsilon }^{\prime }}:\mathcal{W}{({\widehat{\mathbb{P}}}_N\left({\epsilon }^{\mathrm{OLS}}\right),{\mathbb{P}}_{{\epsilon }^{\prime }})}^p\le r\}$ is the ambiguity set with Wasserstein distance $\mathcal{W}{(\mathbb{P},\mathbb{Q})}^p=\underset{\gamma \in \Gamma (\mathbb{P},\mathbb{Q})}{inf}\left\{{\mathbb{E}}_{(z,z^{\prime })\sim \gamma }\left[d{(z,z^{\prime })}^p\right]\right\}$. Furthermore, the dual objective of the Sinkhorn DRCQP problem (8) converges to

$$\lambda r+{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left[\underset{{\epsilon }^{\prime }\in \mathbb{R}}{sup}\left\{{\rho }_{\alpha }({\epsilon }^{\prime }-s)-\lambda {|{\epsilon }_i^{\mathrm{OLS}}-{\epsilon }^{\prime }|}^p\right\}\right].$$

For the complete proof, we refer to Remark 3 in [27]. The derivation and simplified results for the cases $p=1$ and $p>1$ are provided in [9] and [37], respectively.

Remark 2

(Connection with KL DRCQP). Applying Jensen’s inequality, we obtain an upper bound for the dual objective function of the Sinkhorn DRCQP

$$\lambda \overline{r}+\lambda \eta log\left({\displaystyle \frac{1}{N}}\sum _{i=1}^N{\mathbb{E}}_{{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)}\left[e^{{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)/\left(\lambda \eta \right)}\right]\right),$$

(20)

which corresponds to the dual objective for the following KL DRO [19]

$$\underset{s}{min}\underset{{\mathbb{P}}_{{\epsilon }^{\prime }}\in \mathcal{P}\left({\epsilon }^{\prime }\right)}{sup}{\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[{\rho }_{\alpha }\left({\epsilon }^{\prime }-s\right)\right],$$

(21)

where $\mathcal{P}\left({\epsilon }^{\prime }\right)=\{{\mathbb{P}}_{{\epsilon }^{\prime }}:{\mathcal{D}}_{KL}({\mathbb{P}}_{{\epsilon }^{\prime }}\parallel {\mathbb{P}}^0)\le \overline{r}/\eta \}$ is the ambiguity set with KL divergence ${\mathcal{D}}_{KL}({\mathbb{P}}_{{\epsilon }^{\prime }}\parallel {\mathbb{P}}^0)={\mathbb{E}}_{{\mathbb{P}}_{{\epsilon }^{\prime }}}\left[log\left({\displaystyle \frac{\mathrm{d}{\mathbb{P}}_{{\epsilon }^{\prime }}\left({\epsilon }^{\prime }\right)}{\mathrm{d}{\mathbb{P}}^0\left({\epsilon }^{\prime }\right)}}\right)\right]$, and ${\mathbb{P}}^0$ satisfies $\mathrm{d}{\mathbb{P}}^0\left({\epsilon }^{\prime }\right)={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N\left[\mathrm{d}{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)\right]$.

Proof of Remark 2.

Recall that Assumption 2 requires that the quantile loss ${\rho }_{\alpha }({\epsilon }^{\prime }-s)$ has a light tail under the distribution ${\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)$ for ${\widehat{\mathbb{P}}}_N$ almost every ${\epsilon }^{\mathrm{OLS}}$, which satisfies Assumption 1 in [18]. Consequently, by applying Theorem 1 in [18], problem (21) is equivalent to the following one-layer optimization problem:

$$\begin{array}{c}\hfill \underset{{\lambda }^{\prime }\ge 0}{min}\left\{{\displaystyle \frac{{\lambda }^{\prime }\overline{r}}{\eta }}+{\lambda }^{\prime }log\left({\mathbb{E}}_{{\mathbb{P}}^0\left({\epsilon }^{\prime }\right)}\left[e^{{\rho }_{\alpha }({\epsilon }^{\prime }-s)/{\lambda }^{\prime }}\right]\right)\right\}.\end{array}$$

(22)

Substituting $\lambda ={\displaystyle \frac{{\lambda }^{\prime }}{\eta }}$ and $\mathrm{d}{\mathbb{P}}^0\left({\epsilon }^{\prime }\right)={\displaystyle \frac{1}{N}}{\sum }_{i=1}^N\left[\mathrm{d}{\mathbb{Q}}_i\left({\epsilon }^{\prime }\right)\right]$, we can obtain problem (20). This completes the proof. □

5. Numerical Experiments

In Section 5.1, our numerical experiments on the simulated dataset are presented, and the real-world dataset is addressed in Section 5.2. First, for the conditional quantile prediction problem under the fixed design setting, we compare the out-of-sample performance and computational time of the Sinkhorn DRO method used in this paper with the SAA method [8], other DRO methods (such as Wasserstein DRO [9,37], KL DRO [19]), and other machine learning techniques, like robust quantile regression with Catoni’s log-truncated loss [42,43] and Cauchy-truncated loss [44]. Second, we conduct experiments to examine the impact of varying the entropy regularization parameter $\eta$ on the out-of-sample performance of our Sinkhorn DRCQP problem.

In this paper, we adopt two metrics to evaluate the out-of-sample performance of all models. The first metric is the out-of-sample expected cost achieved by the optimal solution ${\widehat{s}}_N$ to the conditional quantile prediction problem, obtained via the SAA method, the DRO method, and the machine learning methods, denoted as $J_{saa}$, $J_{dro}$, and $J_{ml}$, respectively. Additionally, for different DRO methods, their corresponding out-of-sample expected costs are denoted as $J_{{sink}_p}$ (Sinkhorn DRO), $J_{{wass}_p}$ (Wasserstein DRO), and $J_{kl}$ (KL DRO), where p represents the order of the cost function in the Sinkhorn distance and Wasserstein distance. For different machine learning methods, their corresponding out-of-sample expected costs are denoted as $J_{cat}$ (Catoni’s log-truncated robust quantile regression) and $J_{cau}$ (Cauchy-truncated robust quantile regression), respectively. Furthermore, the optimal solution ${\widehat{s}}_N$ for each model can be expressed as follows.

• Conditional quantile prediction problem based on the SAA method [8]:

  $${\widehat{\mathit{\beta }}}_N,{\widehat{s}}_N=\underset{\mathit{\beta },s}{argmin}{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\rho }_{\alpha }\left(y_i-{\mathit{\beta }}^{\top }{x}_i-s\right).$$
• KL DRCQP problem [19]:

  $${\widehat{s}}_N,{\widehat{\lambda }}_N=\underset{s,\lambda }{argmin}\lambda log{\displaystyle \frac{1}{N}}\sum _{i=1}^N\left[e^{{\rho }_{\alpha }({\epsilon }_i^{\mathrm{OLS}}-s)/\lambda }\right]+\lambda r.$$
• Type 1-Wasserstein DRCQP problem [9]:

  $${\widehat{s}}_N,{\widehat{\lambda }}_N=\underset{s,\lambda \ge max\{\alpha ,1-\alpha \}}{argmin}\lambda r+{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\rho }_{\alpha }({\epsilon }_i^{\mathrm{OLS}}-s).$$
• Type p-Wasserstein ($p\ge 2$) DRCQP problem [37]:
  • ${\widehat{s}}_N,{\widehat{\lambda }}_N=\underset{s,\lambda \ge 0}{argmin}\lambda r+{\displaystyle \frac{1}{N}}\sum _{i=1}^N{\rho }_{\alpha }^{\ast }({\epsilon }_i^{\mathrm{OLS}}-s)+{\lambda }^{-{\displaystyle \frac{1}{p-1}}}(p-1)\left[{\left({\displaystyle \frac{\alpha }{p}}\right)}^{{\displaystyle \frac{p}{p-1}}}{I}_1+{\left({\displaystyle \frac{1-\alpha }{p}}\right)}^{{\displaystyle \frac{p}{p-1}}}{I}_2\right]$
  • where $I_1={\displaystyle \frac{1}{N}}\sum _{i=1}^N{\mathbf{1}}_{\{{\epsilon }_i^{\mathrm{OLS}}\ge s+a\}}$, $I_2=1-I_1$, $a={\left(\lambda p\right)}^{-{\displaystyle \frac{1}{p-1}}}\left(1-{\displaystyle \frac{1}{p}}\right)\left[{(1-\alpha )}^{{\displaystyle \frac{p}{p-1}}}-{\alpha }^{{\displaystyle \frac{p}{p-1}}}\right]$
  • and ${\displaystyle {\rho }_{\alpha }^{\ast }\left(u\right)=u\left(\alpha -{\mathbf{1}}_{(u\le a)}\right)=\left\{\begin{array}{cc}\alpha u\hfill & \mathrm{if}u>a,\hfill \\ (\alpha -1)u\hfill & \mathrm{if}u\le a.\hfill \end{array}\right.}$
• Catoni’s log-truncated robust quantile regression [43]:

  $${\widehat{\mathit{\beta }}}_N,{\widehat{s}}_N=\underset{\mathit{\beta },s}{argmin}{\displaystyle \frac{1}{N\kappa }}\sum _{i=1}^N\Psi \left[\kappa {\rho }_{\alpha }\left(y_i-{\mathit{\beta }}^{\top }{x}_i-s\right)\right],$$

  where $\kappa >0$ is a robustification parameter to be tuned, and the non-decreasing influence function $\Psi :\mathbb{R}\to \mathbb{R}$ is

  $${\displaystyle \Psi \left(u\right)=\left\{\begin{array}{cc}log(1+u+\frac{u^2}{2})\hfill & \mathrm{if}u>0,\hfill \\ -log(1-u+\frac{u^2}{2})\hfill & \mathrm{if}u\le 0.\hfill \end{array}\right.}$$
• Cauchy-truncated robust quantile regression [44]:

  $${\widehat{\mathit{\beta }}}_N,{\widehat{s}}_N=\underset{\mathit{\beta },s}{argmin}{\displaystyle \frac{1}{N}}\sum _{i=1}^N\Phi \left[{\rho }_{\alpha }\left(y_i-{\mathit{\beta }}^{\top }{x}_i-s\right)\right],$$

  where $\kappa >0$, and the truncation function $\Phi :\mathbb{R}\to \mathbb{R}$ is ${\displaystyle {\Phi }_{\kappa }\left(u\right)=\kappa log\left(1+\frac{u}{\kappa }\right)}$.

The second metric is the improvement achieved by the DRO strategy compared to the SAA approach, calculated as ${\displaystyle \frac{J_{dro}-J_{saa}}{J_{saa}}}$ [9].

5.1. Simulation

In this section, we evaluate the performance of our proposed method on the simulated dataset. We generate the dataset under the fixed design setting, following a similar setup to [45]. Define $m_i:[0,1]\to \mathbb{R}$, $i=1,2,3$ as follows:

$$m_1\left(x\right)=sin\left(5x\right),m_2\left(x\right)={\mathrm{e}}^{5x}-{\left(5x\right)}^3,\mathrm{and}{m}_3\left(x\right)={\displaystyle \frac{1}{5x+1}}+sin\left(5x\right).$$

and the response variable is generated according to the model

$$y_i=m_1\left(x_i\right)+2m_2\left(x_i\right)+3m_3\left(x_i\right)+{\epsilon }_i,i=1,2,\dots ,N,$$

where $x_i=i/n$, $\mathit{\beta }=(1,2,3)$ and ${\epsilon }_i$ are i.i.d. samples drawn from a heavy-tailed distribution (e.g., Pareto distribution and Student’s t-distribution). According to the above model, a total of 2000 groups of sample data were generated. The first 1500 groups of samples were used as the training set, and retraining was performed every 50 data points, with a total of 30 training sessions. The last 500 groups of samples served as the test set.

Our simulation experiment consists of two parts: Section 5.1.1 presents a comprehensive performance comparison between the Sinkhorn DRCQP problem and the other five problems. The comparison includes the out-of-sample expected cost and the computational time for solving each problem. Section 5.1.2 evaluates the impact of the entropy regularization parameter $\eta$ on the final results of the Sinkhorn DRO.

5.1.1. Comparison of Out-of-Sample Performance and Computational Time

In this section, we compare the out-of-sample performance of six methods on the simulated dataset under different configurations of the distribution of $\epsilon$, the quantile $\alpha$, the order of the cost function p, and the entropy regularization parameter $\eta$, with a fixed radius of the ambiguity set $r=0.2$. Figure 1 and Figure 2 illustrate the out-of-sample performance at intervals of 50 data points.

Table 1. Out-of-sample performance of six methods.

Noise	$\mathit{\alpha }$	N	${\mathit{J}}_{saa}$	${\mathit{J}}_{cat}$	${\mathit{J}}_{cau}$	${\mathit{J}}_{{\mathit{wass}}_1}$	${\mathit{J}}_{{\mathit{wass}}_2}$	${\mathit{J}}_{{\mathit{sink}}_1}$	${\mathit{J}}_{{\mathit{sink}}_2}$	${\mathit{J}}_{\mathit{kl}}$
t(2)	0.2	100	7.78e+04	0.630	0.612	0.574 (−99.99%)	0.574 (−99.99%)	0.559 (−99.99%)	0.559 (−99.99%)	0.603 (−99.99%)
		800	1.060	0.630	0.612	0.587 (−44.63%)	0.586 (−44.68%)	0.564 (−46.76%)	0.559 (−47.31%)	0.603 (−43.12%)
		1500	0.664	0.630	0.612	0.587 (−11.61%)	0.581 (−12.55%)	0.564 (−14.99%)	0.559 (−15.87%)	0.603 (−9.18%)
	0.7	100	1.75e+04	0.688	0.660	0.667 (−99.99%)	0.657 (−99.99%)	0.648 (−99.99%)	0.635 (−99.99%)	0.676 (−99.99%)
		800	3.842	0.688	0.660	0.664 (−82.71%)	0.661 (−82.80%)	0.648 (−83.14%)	0.635 (−83.48%)	0.676 (−82.41%)
		1500	0.680	0.688	0.660	0.665 (−2.23%)	0.659 (−3.14%)	0.648 (−4.75%)	0.635 (−6.65%)	0.676 (−0.62%)
t(5)	0.2	100	8.50e+03	0.389	0.384	0.363 (−99.99%)	0.371 (−99.99%)	0.348 (−99.99%)	0.348 (−99.99%)	0.362 (−99.99%)
		800	8.755	0.389	0.384	0.353 (−95.97%)	0.355 (−95.94%)	0.348 (−96.03%)	0.348 (−96.03%)	0.362 (−95.86%)
		1500	0.364	0.389	0.384	0.351 (−3.42%)	0.358 (−1.53%)	0.348 (−4.40%)	0.348 (−4.40%)	0.362 (−0.42%)
	0.7	100	2.86e+03	0.433	0.425	0.428 (−99.99%)	0.423 (−99.99%)	0.418 (−99.99%)	0.418 (−99.99%)	0.427 (−99.99%)
		800	6.885	0.433	0.425	0.421 (−93.88%)	0.422 (−93.87%)	0.418 (−93.93%)	0.418 (−93.93%)	0.427 (−93.79%)
		1500	0.436	0.433	0.425	0.421 (−3.59%)	0.424 (−2.94%)	0.418 (−4.33%)	0.418 (−4.31%)	0.427 (−2.09%)
P(2,1)	0.2	100	2.16e+03	0.196	0.194	0.193 (−99.99%)	0.193 (−99.99%)	0.183 (−99.99%)	0.183 (−99.99%)	0.194 (−99.99%)
		800	0.673	0.196	0.194	0.193 (−71.38%)	0.193 (−71.38%)	0.183 (−72.82%)	0.183 (−72.82%)	0.194 (−71.11%)
		1500	0.194	0.196	0.194	0.193 (−0.97%)	0.193 (−0.97%)	0.183 (−5.95%)	0.183 (−5.95%)	0.194 (−0.01%)
	0.7	100	4.38e+04	0.519	0.511	0.508 (−99.99%)	0.508 (−99.99%)	0.476 (−99.99%)	0.476 (−99.99%)	0.513 (−99.99%)
		800	0.542	0.519	0.511	0.508 (−6.32%)	0.508 (−6.31%)	0.476 (−12.20%)	0.476 (−12.20%)	0.513 (−5.33%)
		1500	0.525	0.519	0.511	0.508 (−3.33%)	0.508 (−3.31%)	0.476 (−9.40%)	0.476 (−9.40%)	0.513 (−2.30%)
P(5,1)	0.2	100	1.06e+03	0.050	0.049	0.047 (−99.99%)	0.049 (−99.99%)	0.046 (−99.99%)	0.046 (−99.99%)	0.050 (−99.99%)
		800	0.225	0.050	0.049	0.047 (−78.85%)	0.049 (−78.28%)	0.046 (−79.56%)	0.046 (−79.56%)	0.050 (−77.66%)
		1500	0.050	0.050	0.049	0.047 (−5.89%)	0.049 (−3.36%)	0.046 (−9.04%)	0.046 (−9.04%)	0.050 (−0.57%)
	0.7	100	1.43e+03	0.121	0.126	0.107 (−99.99%)	0.109 (−99.99%)	0.102 (−99.99%)	0.102 (−99.99%)	0.108 (−99.99%)
		800	0.130	0.121	0.126	0.107 (−17.92%)	0.105 (−19.66%)	0.102 (−21.46%)	0.102 (−21.53%)	0.108 (−16.95%)
		1500	0.116	0.121	0.126	0.107 (−7.60%)	0.105 (−9.62%)	0.102 (−11.58%)	0.102 (−11.67%)	0.108 (−6.51%)

Here, $t\left(\zeta \right)$ denotes the Student’s t-distribution with $\zeta$ degrees of freedom, and $P(a,b)$ denotes the Pareto distribution with shape parameter a and scale parameter b. The values in parentheses in our tables are all calculated using the formula $(J_{\mathrm{dro}}-J_{\mathrm{saa}})/J_{\mathrm{saa}}$.

and

Table 2. Total computational time (in seconds) of six methods.

Noise	$\mathit{\alpha }$	SAA	CAT	CAU	1-WDRO	2-WDRO	1-SDRO	2-SDRO	KL-DRO
t(2)	0.2	2.11e+02	3.35e+02	5.39e+02	1.80e+02	1.05e+02	4.76e+00	3.23e+00	1.90e-01
	0.7	2.48e+02	3.17e+02	2.27e+02	1.53e+02	9.85e+01	5.39e+00	2.14e+00	2.00e-01
t(5)	0.2	4.18e+02	3.40e+02	5.13e+02	9.43e+01	8.89e+01	4.58e+00	2.12e+00	1.70e-01
	0.7	3.90e+02	3.35e+02	4.00e+02	9.96e+01	9.18e+01	4.75e+00	2.59e+00	1.90e-01
P(2,1)	0.2	3.26e+02	2.91e+02	3.16e+02	1.29e+02	1.38e+02	5.08e+00	3.17e+00	1.50e-01
	0.7	3.81e+02	1.99e+02	2.44e+02	9.82e+01	1.01e+02	4.34e+00	1.95e+00	2.00e-01
P(5,1)	0.2	2.88e+02	3.38e+02	5.05e+02	9.67e+01	1.03e+02	6.62e+00	2.37e+00	1.50e-01
	0.7	3.62e+02	3.62e+02	3.44e+02	8.92e+01	9.19e+01	5.85e+00	3.97e+00	1.70e-01

CAT, CAU, 1-WDRO, 2-WDRO, 1-SDRO, and 2-WDRO represent Catoni’s log-truncated robust quantile regression, Cauchy-truncated robust quantile regression, type-1 Wasserstein DRO, type-2 Wasserstein DRO, type-1 Sinkhorn DRO, and type-2 Sinkhorn DRO, respectively.

present the comparative out-of-sample performance and the computational time of six methods for the training sample sizes $N=100$, 800, and 1500, respectively.

Our results show that the proposed method in this paper outperforms the five traditional methods in most cases. As the sample size used for training increases, the out-of-sample expected costs of the SAA method gradually converge, while the out-of-sample costs of the DRO methods remain consistently stable and exhibit excellent performance. Although the out-of-sample performance of the two machine learning methods (denoted as $J_{cat}$ and $J_{cau}$; $\kappa =1$) is stable, their out-of-sample expected costs are higher than those of the DRO method in most cases. Additionally, the performance improvement of the DRO methods over the SAA method decreases as the training sample size expands. Among the three DRO methods, Sinkhorn DRO demonstrates the most remarkable performance improvement over the SAA method, which can also be significantly compared. For example, when taking $\epsilon \sim P(2,1)$, $\alpha =0.7$ and $N=1500$, the improvement in $J_{sin{k}_1}$ over $J_{saa}$ is $9.40\%$, whereas the improvement of $J_{was{s}_1}$ over $J_{saa}$ is only $3.33\%$.

In Table 2, the total computation time required for each method is recorded as the sample size increases from 1 to 1500 at intervals of 50 data points. As shown in Table 2, although the Sinkhorn DRO ranks second in terms of the total computation time among the six methods, its out-of-sample performance is the best in most cases. Therefore, it combines computational efficiency and prediction accuracy. In contrast, while the KL DRO has a shorter computation time, its out-of-sample performance is inferior to that of the Sinkhorn DRO method. Meanwhile, although the Wasserstein DRO delivers comparable predictive accuracy to Sinkhorn DRO, its computational time ranges from 14 to 53 times longer, rendering it less efficient computationally. Notably, the SAA method shows both the poorest predictive performance and the lowest computational efficiency among all methods in most cases, indicating that it is not well-suited for solving conditional quantile prediction problems under the fixed design settings.

5.1.2. Comparison of the Impacts of Parameter Settings

In this section, we analyze the effect of the entropy regularization parameter $\eta$ on the out-of-sample performance of the Sinkhorn DRO method under the fixed design settings.

As shown in Figure 3, regardless of whether $\epsilon$ follows a heavy-tailed distribution (e.g., $t\left(5\right)$ or $P(5,1)$), the Sinkhorn DRO method can achieve better predictive performance than both Wasserstein DRO and KL DRO through scientific adjustment of the entropy regularization parameter $\eta$. Specifically, when $\epsilon \sim t\left(5\right)$, $J_{sin{k}_1}$ reaches its minimum value of 0.4199 at $\eta =0.1$, while $J_{sin{k}_2}$ achieves its minimum of 0.4243 at $\eta =0.08$. When $\epsilon \sim P(5,1)$, both $J_{sin{k}_1}$ and $J_{sin{k}_2}$ attain their respective minimum of 0.1025 and 0.1023 at $\eta =0.3$. Furthermore, when $\eta \to 0$, the results of $J_{sink}$ approximate those of $J_{wass}$, which aligns with the theoretical results in Remark 1. As $\eta$ continues to increase, $J_{sink}$ first reaches its optimal value before gradually deteriorating, though it still maintains a noticeable advantage over $J_{kl}$.

5.2. Real-World Applications

In this section, we validate the performance of our proposed method using the real-world dataset. Our goal was to predict the total demand for public bicycles in Seoul, South Korea, per hour, taking into account external factors such as temperature and humidity. By predicting the quantiles of the stochastic demand process, we can better manage bicycle inventory, ensure a stable supply of rental bicycles, and improve the comfort of public transportation. Section 5.2.1 provides an overview of the real-world dataset and its reconstruction steps. In Section 5.2.2, we evaluate the out-of-sample performance of Sinkhorn DRO against the following: (i) the SAA baseline, (ii) two DRO benchmarks (Wasserstein DRO and KL DRO), and (iii) two robust quantile regression methods (with Catoni’s log-truncated loss and Cauchy-truncated loss). In addition, Section 5.2.3 evaluates the impact of relevant parameters on the final results, providing guidance for effective parameter tuning.

5.2.1. Data Selection and Reconstruction

The data used in the real-world applications were sourced from the UCI Machine Learning Repository, established in [46]. It includes a real dataset of the hourly rental count of public bicycles in Seoul, South Korea (which serves as the response variable, denoted as Y), along with corresponding weather data and information about holidays. In total, there are 8760 valid instances (valid means that the hourly rental count of public bicycles is not zero). To account for the influence of multicollinearity, we ultimately retained seven covariates to construct the design matrix ${\mathit{X}}^f$. The specific information about these covariates is shown in

Table 3. Summary of covariates.

Feature	Type	Value Range	Unit
Temperature	Numeric	(−17.8, 39.4)	°C
Humidity	Numeric	(0, 98)	%
Wind speed	Numeric	(0, 7.4)	m/s
Visibility	Numeric	(270, 20000)	m
Dew point temperature	Numeric	(−30.6, 27.2)	°C
Solar radiation	Numeric	(0, 3.52)	MJ/m2
Rainfall	Numeric	(0, 35)	mm

.

Since the observations are time series data, and covariates, such as temperature, humidity, and wind speed, tend to be closely correlated within a few consecutive days, the observations should not be treated as i.i.d. data. This means we need to consider the fixed design setting. The specific steps we take to process the dataset are as follows.

• Step 1: Construct a linear regression problem on the preprocessed dataset and use the coefficients estimated using the OLS method as the coefficients for potential true linear relationships, namely ${\mathit{\beta }}_0$;
• Step 2: Discard the demand column of the dataset and retain only the columns of the covariates. Then, generate new demand observations $y_i$ by adding i.i.d. simulated noise ${\epsilon }_i$ following the normal distribution. Specifically, $y_i={{\mathit{\beta }}_0}^{\top }{\mathit{x}}_i+{\epsilon }_i;$
• Step 3: Divide the observations into training and testing sets based on time periods. Starting from the 8000-th observation, the next 500 time periods are used as the test set. For any given sample size N, the training set consists of the N observations immediately preceding the start of the test set;
• Step 4: For each observation in the test set, i.i.d. noises are simulated 50 times, resulting in a total of $500\times 50=\mathrm{25,000}$ data points for the test set. These data points are used to evaluate the predictive performance of each method.

5.2.2. Comparison of Out-of-Sample Performance and Computational Time

In this section, we compare the out-of-sample performance of six methods under different configurations of the quantile $\alpha$, the standard deviation of the error term $\sigma$, the order of the cost function p, and the entropy regularization parameter $\eta$, with a fixed radius of the ambiguity set $r=0.05$. Figure 4 and Figure 5 illustrate the out-of-sample expected costs at intervals of 50 data points.

Table 4. Out-of-sample performance of six methods.

$\mathit{\sigma }$	$\mathit{\alpha }$	N	${\mathit{J}}_{\mathit{saa}}$	${\mathit{J}}_{\mathit{cat}}$	${\mathit{J}}_{\mathit{cau}}$	${\mathit{J}}_{{\mathit{wass}}_1}$	${\mathit{J}}_{{\mathit{wass}}_2}$	${\mathit{J}}_{{\mathit{sink}}_1}$	${\mathit{J}}_{{\mathit{sink}}_2}$	${\mathit{J}}_{\mathit{kl}}$
0.02	0.2	100	0.031	0.008	0.008	0.006 (−81.91%)	0.006 (−81.91%)	0.006 (−82.13%)	0.006 (−82.12%)	0.007 (−76.73%)
		550	0.014	0.008	0.008	0.006 (−60.53%)	0.006 (−60.53%)	0.006 (−60.99%)	0.006 (−60.97%)	0.007 (−49.20%)
		1500	0.009	0.008	0.008	0.006 (−38.39%)	0.006 (−38.39%)	0.006 (−39.11%)	0.006 (−39.07%)	0.007 (−20.71%)
	0.7	100	0.041	0.008	0.008	0.007 (−83.03%)	0.007 (−83.03%)	0.007 (−83.26%)	0.007 (−83.26%)	0.007 (−81.91%)
		550	0.007	0.008	0.008	0.007 (0.41%)	0.007 (0.41%)	0.007 (−0.59%)	0.007 (−0.43%)	0.007 (7.45%)
		1500	0.008	0.008	0.008	0.007 (−7.97%)	0.007 (−7.97%)	0.007 (−8.77%)	0.007 (−8.73%)	0.007 (−1.40%)
0.2	0.2	100	0.312	0.080	0.080	0.057 (−81.85%)	0.073 (−76.72%)	0.057 (−81.92%)	0.058 (−81.57%)	0.082 (−73.89%)
		550	0.143	0.080	0.080	0.057 (−60.39%)	0.075 (−47.31%)	0.056 (−60.93%)	0.058 (−59.76%)	0.082 (−32.75%)
		1500	0.092	0.080	0.080	0.057 (−38.18%)	0.071 (−22.19%)	0.056 (−39.04%)	0.058 (−37.18%)	0.082 (−18.73%)
	0.7	100	0.385	0.080	0.080	0.071 (−81.67%)	0.090 (−76.71%)	0.070 (−82.00%)	0.070 (−81.97%)	0.089 (−76.97%)
		550	0.178	0.080	0.080	0.071 (−60.20%)	0.081 (−54.40%)	0.070 (−60.87%)	0.070 (−60.81%)	0.089 (−49.93%)
		1500	0.097	0.080	0.080	0.070 (−27.51%)	0.081 (−16.63%)	0.070 (−28.61%)	0.070 (−28.49%)	0.089 (−8.65%)
2	0.2	100	1.715	0.677	0.687	0.567 (−66.94%)	0.565 (−67.08%)	0.558 (−67.48%)	0.558 (−67.48%)	0.601 (−64.93%)
		550	0.758	0.677	0.687	0.567 (−25.19%)	0.568 (−25.04%)	0.558 (−26.38%)	0.558 (−26.35%)	0.601 (−20.65%)
		1500	0.603	0.677	0.687	0.567 (−6.01%)	0.567 (−5.97%)	0.558 (−7.48%)	0.558 (−7.48%)	0.601 (−0.27%)
	0.7	100	4.153	0.696	0.696	0.704 (−83.06%)	0.707 (−82.98%)	0.695 (−83.27%)	0.694 (−83.29%)	0.745 (−82.06%)
		550	1.663	0.696	0.696	0.701 (−57.85%)	0.705 (−57.61%)	0.693 (−58.30%)	0.694 (−58.26%)	0.745 (−55.19%)
		1500	0.857	0.696	0.696	0.700 (−18.27%)	0.702 (−18.04%)	0.694 (−18.98%)	0.694 (−18.98%)	0.745 (−13.01%)

presents the comparative out-of-sample performance of the six methods for sample sizes $N=100, 550$, and 1500, with ${\displaystyle \frac{J_{dro}-J_{saa}}{J_{saa}}}$ shown in parentheses. Our results show that the proposed method outperforms the SAA method, two DRO benchmarks (Wasserstein DRO and KL DRO), and two robust quantile regression methods (with Catoni’s log-truncated loss and Cauchy-truncated loss) in most cases.

Specifically, we can observe that while the SAA method maintains some convergence for large samples, its out-of-sample performance is poor in the case of small samples. This is mainly because the data points in the training and testing sets are not randomly generated, failing to meet the assumption of i.i.d. conditions. Therefore, the performance guarantee of the SAA method under the fixed design setting is no longer applicable [9]. Moreover, while both Wasserstein DRO and KL DRO can be employed to formulate conditional quantile problems under the fixed design setting and demonstrate relatively stable performance, in most scenarios, their out-of-sample performance is slightly inferior to the Sinkhorn DRO method. In addition, compared to robust quantile regression using Catoni’s log-truncated loss and Cauchy-truncated loss when $\kappa =1$, it is found that in most cases, Sinkhorn DRO is outperformed by neither method, and both robust regression techniques are shown to outperform traditional quantile regression methods.

5.2.3. Comparison of the Impacts of Parameter Settings

In this section, we analyze the effect of the entropy regularization parameter $\eta$ on the out-of-sample performance of our Sinkhorn DRCQP problem.

Overall, for the fixed $\alpha ,\sigma ,N,p,r$, we compute the corresponding out-of-sample expected cost $J_{sink}$ and compare it with $J_{wass}$ and $J_{kl}$ under the same settings. Figure 6a and Figure 7a show that when $p=1$, as $\eta$ increases, $J_{sink}$ exhibits an increasing trend. Figure 6b and Figure 7b show that when $p=2$, as $\eta$ increases, $J_{sink}$ first decreases and subsequently increases.

Specifically, for the case $p=2$, when $\eta =0.005$, the results of $J_{sink}$ (0.0997 in Figure 6b and 0.0947 in Figure 7b) are remarkably close to those of $J_{wass}$ (0.0999 in Figure 6b and 0.0948 in Figure 7b), which validates Remark 1 in this paper. As $\eta$ increases to 0.045, $J_{sink}$ achieves its minimum values of $0.0948$ in Figure 6b and $0.0916$ in Figure 7b. When $\eta >0.045$, $J_{sink}$ deteriorates rapidly but consistently remains lower than $J_{kl}$ (0.1394 in Figure 6b and 0.1440 in Figure 7b). Generally speaking, Figure 6b and Figure 7b clearly demonstrate that by selecting appropriate values of $\eta$, the Sinkhorn DRCQP can outperform both the classic Wasserstein- and the KL divergence-based DRCQP approaches.

6. Conclusions

This paper proposes a Sinkhorn DRCQP problem under the fixed design setting. We demonstrate the tractability of the proposed model by deriving its strong dual formulation, which takes the form of an exponential cone programming problem for finite support cases. Finally, we conduct numerical experiments to evaluate the performance and computational time of the proposed method in comparison with five conventional methods, including SAA, Wasserstein DRO, KL DRO, Catoni’s log-truncated robust quantile regression, and Cauchy-truncated robust quantile regression. The experimental results demonstrate the superior performance of our approach by choosing suitable values of the entropy regularization parameter $\eta$.