PAC–Bayes Guarantees for Data-Adaptive Pairwise Learning

Abstract

We study the generalization properties of stochastic optimization methods under adaptive data sampling schemes, focusing on the setting of pairwise learning, which is central to tasks like ranking, metric learning, and AUC maximization. Unlike pointwise learning, pairwise methods must address statistical dependencies between input pairs—a challenge that existing analyses do not adequately handle when sampling is adaptive. In this work, we extend a general framework that integrates two algorithm-dependent approaches—algorithmic stability and PAC–Bayes analysis for this purpose. Specifically, we examine (1) Pairwise Stochastic Gradient Descent (Pairwise SGD), widely used across machine learning applications, and (2) Pairwise Stochastic Gradient Descent Ascent (Pairwise SGDA), common in adversarial training. Our analysis avoids artificial randomization and leverages the inherent stochasticity of gradient updates instead. Our results yield generalization guarantees of order $n^{-1/2}$ under non-uniform adaptive sampling strategies, covering both smooth and non-smooth convex settings. We believe these findings address a significant gap in the theory of pairwise learning with adaptive sampling.

1. Introduction

The increasing availability of data makes it feasible to use increasingly large models in principle. However, this comes at the expense of an increasing computational cost of training these models in large pairwise learning applications. Some notable examples of pairwise learning problems include ranking and preference prediction, AUC maximization, and metric learning [1,2,3,4,5]. For instance, in metric learning we aim to learn an appropriate distance or similarity to compare pairs of examples, which has numerous applications such as face verification, person re-identification (Re-ID) [6,7,8,9,10], and bioactivity prediction [11]. Pairwise learning has also been applied to positive-unlabeled (PU) learning problems [12], where only positive and unlabeled examples are available. Such problems arise in one-class classification settings, with practical applications in areas such as fault detection and diagnosis in advanced engineering systems [13]. Given the broad relevance of pairwise learning, there is a pressing need to deepen our theoretical understanding of its generalization properties. This in turn can inform the design of algorithms that generalize reliably to unseen pairs and offer interpretability and trustworthiness to end users.

In both pointwise and pairwise learning settings, Stochastic Gradient Descent (SGD) and Stochastic Gradient Descent Ascent (SGDA) are widely used for large-scale minimization and min-max optimization problems in machine learning due to their favorable computational efficiency. These methods rely on stochastic sampling strategies to approximate the true gradients, and several works have explored data-dependent sampling techniques to accelerate convergence to the optimum [14,15,16,17,18,19]. SGDA, in particular, is a standard approach in solving min-max problems, finding notable applications in generative adversarial networks (GANs) [20] and adversarial training [21,22].

Adversarial perturbations are subtle, often imperceptible modifications to input data designed to deceive models and cause incorrect predictions [23]. Recent studies in pairwise learning have explored strategies to enhance adversarial robustness, applying adversarial pairwise learning methods to min-max problems across various domains, such as metric learning [24,25], ranking [25,26], and kinship verification [27]. These developments illustrate the need for robust, theoretically grounded pairwise methods that can withstand adversarial attacks while maintaining generalization performance.

Under the assumption of i.i.d. data points in classic pointwise learning, the empirical risk of a fixed hypothesis is an average of i.i.d. random variables. However, in pairwise learning the pairs derived from i.i.d. points are no longer i.i.d. Instead, when the loss is symmetric and computed over all unordered pairs, the empirical risk takes the form of a second-order U-statistic. Therefore, results on U-processes may be used to investigate the generalization analysis of pairwise learning [3,28]. While there is much research on the generalization analysis of pairwise learning, the effect of non-uniform, data-dependent sampling schemes has not been rigorously studied.

Non-uniform sampling can be beneficial in noisy data situations where the training examples may not be equally reliable or equally informative. Some examples may be less important than others, or even misleading—e.g., mislabeled examples or examples situated in an ambiguous class-overlap region. In rare cases when the usefulness or importance of individual training examples is known, then the sampling distribution can be designed and fixed before training, and this may improve the representativeness of the sample. For instance, in the case of infrequent observations [29], inverse frequency sampling prioritizes rare examples that may be underrepresented in the training set, ensuring their proper influence. However, in most cases the relative importance of training examples is not known a priori; hence, it is desirable to learn the sampling distribution together with training the model.

The idea of adaptive sampling refers to a sampling distribution that depends on the training sample. Such non-uniform and data-dependent sampling shows great potential in the literature of randomized algorithms for both SGD- [14] and SGDA-based [30] optimizers, in both pointwise and pairwise settings. Importance sampling [14] is one of the widely used of such strategies, and a few others will be reviewed shortly in Section 2. Therefore, recent work [31,32] has begun to develop a better understanding of the generalization behavior of such algorithms, which we continue here in the setting of pairwise learning. The main bottlenecks in the analysis of adaptive sampling-based stochastic optimizers are that (i) a correction factor is often used to ensure the unbiasedness of the gradient [14], which also depends on training data points complicating the analysis, and (ii) in the pairwise setting we also need to cater to statistical dependencies between data pairs, which are due to the fact that each point participates in multiple pairs.

To tackle these problems, we develop a PAC–Bayesian analysis of the generalization of pairwise stochastic optimization methods, which removes the need for a correction factor, and we use U-statistics to capture the statistical structure of pairwise loss functions. The PAC–Bayes framework allows us to obtain generalization bounds that hold uniformly for all posterior sampling schemes, under a mild condition required on a pre-specified prior sampling scheme (chosen as the uniform sampling). For randomized methods, such as Pairwise SGD and Pairwise SGDA, the sampling index pairs will be treated as hyperparameters that follow a sampling distribution.

Our main contributions are listed in

Table 1. Summary of generalization rates obtained for two pairwise stochastic optimization algorithms (Pairwise SGD, Pairwise SGDA) under two sets of assumptions (Lipschitz (L), smooth (S), convex (C)) on the pairwise loss function, together with the chosen number of iterations T and step size $\eta$. The sample size is n. According to this summary, we notice that smaller step sizes and more iterations are needed if the smoothness assumption is removed (more details in Section 4).

Algo.	Asm.	Time T and Step Size $\mathit{\eta }$		Rates
Pairwise SGD	L, C	$T=\mathrm{\Theta }\left(n^2\right)$	$\eta =\mathrm{\Theta }(T^{-{\displaystyle \frac{3}{4}}})$	$\tilde{O}(1/\sqrt{n})$
				Theorem 1 (1)
	L, S, C	$T=\mathrm{\Theta }\left(n\right)$	$\eta =\mathrm{\Theta }(T^{-{\displaystyle \frac{1}{2}}})$	$\tilde{O}(1/\sqrt{n})$
				Theorem 1 (2)
Pairwise SGDA	L, C	$T=O\left(n^2\right)$	$\eta =O(T^{-{\displaystyle \frac{3}{4}}})$	$\tilde{O}(1/\sqrt{n})$
				Theorem 2 (1)
	L, S, C	$T=O\left(n\right)$	$\eta =O(T^{-{\displaystyle \frac{1}{2}}})$	$\tilde{O}(1/\sqrt{n})$
				Theorem 2 (2)

, summarizing the generalization bounds of the order $\tilde{O}(1/\sqrt{n})$ for these randomized algorithms under different assumptions, where n is the sample size.

Our technical contributions are summarized as follows:

• We bound the generalization gap of randomized pairwise learning algorithms that operate with an arbitrary data-dependent sampling, in a PAC–Bayesian framework, under a sub-exponential stability condition.
• We apply the above general result to Pairwise SGD and Pairwise SGDA with arbitrary sampling. For both of these algorithms, we verify the sub-exponential stability in both smooth and non-smooth problems.
• We exemplify how our bounds can inform algorithm design, and we demonstrate how to extract meaningful information from the resulting algorithms.

Our work builds on well-established tools, including a specific flavor of PAC–Bayesian analysis [31], U-statistic decomposition, and a moment bound for uniformly stable pairwise learning algorithms [33], aimed at bringing theoretical insight into an important, yet relatively underexplored, setting: pairwise learning with adaptive sampling. To the best of our knowledge, our analysis is the first to derive explicit generalization bounds for this setting.

The remainder of the paper is organized as follows. We survey the related work on the generalization analysis and non-uniform sampling in Section 2. We give a brief background on U-statistics and algorithmic stability analysis in Section 3. Our general result and its applications to Pairwise SGD and Pairwise SGDA are presented in Section 4.

2. Related Work

Adaptive Sampling in Stochastic Optimization. Importance Sampling for Stochastic Gradient Descent was proposed in [14]. To compute the stochastic gradient, a training example $z_i$ ($i\in \left[n\right]$) is sampled with probability proportional to the gradient norm $p_i\propto \parallel {\nabla }_{\mathbf{w}}\ell (\mathbf{w};z_i)\parallel$, where $\mathbf{w}$ are the model parameters and ℓ is the loss function. This prioritizes high-impact updates from the perspective of optimization—the authors proved that this can significantly reduce the variance of the stochastic gradient and accelerate convergence to the optimum. A related idea, loss-based sampling, proposed in [16], assigns sampling probabilities proportional to the loss evaluated on training points, that is, $p_i\propto \ell (\mathbf{w};z_i)$, thereby focusing on hard-to-fit examples. The authors show faster convergence to the optimum. While these works do not consider pairwise learning, they represent landmarks on adaptive sampling in stochastic optimization.

Furthermore, there are variants of these ideas aimed at lightening the computational demand. These include upper bounds to the gradient norm, shown to exhibit better performance in comparison with the loss-based sampling [34]. The work in [18] proposes to sample the training points based on their relative distance to each other. Another more recent data-dependent sampling approach called group sampling appears in [19] and has been applied to a person re-identification (Re-ID) application.

Adaptive sampling is an umbrella term referring to sampling distributions that depend on the training sample. Here, we mentioned a few of the most prominent existing examples. However, the appropriate sampling distribution is task-dependent. The works discussed, and most of the previous work on adaptive sampling in stochastic optimization, aim at accelerating convergence to the optimum. Therefore, they have no explicit cost for data-dependent sampling; instead, they have a multiplicative correction factor to ensure an unbiased gradient. However, the goal of learning is generalization, which is different from achieving the global optimum on training data. There must be a cost for data-dependent sampling to avoid over-reliance on a biased subset of points. Our forthcoming generalization bounds will quantify this rigorously and provide guidance for algorithm design.

With the exception of [32] and our previous conference paper [31], results on the generalization analysis of the resulting randomized algorithms are very scarce, which is our goal to advance in this paper specifically for the pairwise learning setting.

Generalization through Algorithmic Stability. Stability was popularized in the seminal work of [35], to formalize the intuition that algorithms whose output is resilient to changing an example in its input data will generalize. The stability framework subsequently motivated a chain of analysis of randomized iterative algorithms, such as SGD [36] and SGDA [37,38]. While the stability framework in the previous work is well suited for SGD-type algorithms that operate a uniform sampling scheme [36], this framework alone is unable to tackle arbitrary data-dependent sampling schemes.

Generalization through PAC–Bayes. The PAC–Bayes theory of generalization is another algorithm-dependent framework in statistical learning, the gist of which is to leverage a pre-specified prior distribution on the parameters of interest to obtain generalization bounds that hold uniformly for all posterior distributions [39,40]. Its complementarity with the algorithmic stability framework sparked ideas for combining them [41,42,43,44], some of which are also applicable to randomized learning algorithms such as SGD and SGLD [32,45,46,47]. While insightful, these works assume i.i.d. examples and cannot be applied to non-i.i.d. settings that arise in pairwise learning.

In non-i.i.d. settings, ref. [48] gave PAC–Bayes bounds using fractional covers, which allows for handling the dependencies within the inputs. This gives rise to generalization bounds for pairwise learning, with predictors following a distribution induced by a prior distribution on the model’s parameters. However, with SGD-type methods in mind, which have a randomization already built into the algorithm, the classic PAC–Bayes approach of placing a prior on a model’s parameters would be somewhat artificial. Indeed, considerable research effort has been spent to reverse such randomization [49]. Another issue concerns the prior specification—recent research [50] reveals that placing sufficient prior mass on good predictors is a condition for meaningful PAC–Bayes guarantees. These are difficult to set without a strong prior knowledge. Instead, the construction proposed in [31,32] (albeit restricted to the i.i.d. setting) is to exploit this built-in stochasticity of modern gradient-based optimization algorithms directly, by interpreting it as a PAC–Bayes prior placed on a hyperparameter. We will build on this idea further in this work.

3. Preliminaries

3.1. Pairwise Learning and U-Statistics

Let $\mathcal{D}$ be an unknown distribution on sample space $\mathcal{Z}$. We denote by $\mathcal{W}\subseteq {\mathbb{R}}^d$ the parameter space, and $\mathrm{\Phi }$ will be a hyperparameter space. Given a training set $S=\{z_1,\dots ,z_n\}$ drawn i.i.d. from $\mathcal{D}$ and a hyperparameter $\varphi \in \mathrm{\Phi }$, a learning algorithm A returns a model parameterized by $A(S;\varphi )\in \mathcal{W}$.

We are interested in pairwise learning problems and will use a pairwise loss function $\ell :\mathcal{W}\times (\mathcal{Z}\times \mathcal{Z})\mapsto {\mathbb{R}}_{+}$ to measure the mismatch between the prediction of the model that acts on example pairs. The generalization error, or true risk, is defined as the expected loss of the learned predictor applied on an unseen pair of inputs drawn from ${\mathcal{D}}^2$, that is,

$$R\left(A(S;\varphi )\right):={\mathbb{E}}_{z,\tilde{z}\sim \mathcal{D}}[\ell (A(S;\varphi ),z,\tilde{z})].$$

(1)

Since $\mathcal{D}$ is unknown, we consider the empirical risk,

$$R_S\left(A(S;\varphi )\right):={\displaystyle \frac{1}{n(n-1)}}\sum _{i,j\in \left[n\right]:i\ne j}\ell (A(S;\varphi ),z_i,z_j),$$

(2)

where $\left[n\right]:=\{1,\dots ,n\}$. The generalization error is a random quantity as a function of the sample S, which does not consider the randomization used when selecting the data or feature index for the update rule of A at each iteration.

To take advantage of the built-in stochasticity of the type of algorithms we consider, we further define two distributions on the hyperparameter space $\mathrm{\Phi }$: a sample-independent distribution $\mathrm{P}$ and a sample-dependent distribution $\mathrm{Q}$. In this stochastic or randomized learning algorithm setting, the expected risk and the expected empirical risk (both with respect to $\mathrm{Q}$) are defined as

$$R\left(\mathrm{Q}\right)=\underset{\varphi \sim \mathrm{Q}}{\mathbb{E}}\left[R\left(A(S;\varphi )\right)\right],R_S\left(\mathrm{Q}\right)=\underset{\varphi \sim \mathrm{Q}}{\mathbb{E}}\left[R_S\left(A(S;\varphi )\right)\right].$$

We denote the difference between the risk and the empirical risk (i.e., the generalization gap) by $G(S,\varphi ):=R\left(A(S;\varphi )\right)-R_S\left(A(S;\varphi )\right)$.

The difficulty with the pairwise empirical loss (2) is that, even with S consisting of i.i.d. instances, the pairs from S are dependent of each other. Instead, $R_S\left(A(S;\varphi )\right)$ is a second-order U-statistic. A powerful technique to handle the U-statistic is the representation as an average of “sums-of-i.i.d.” blocks [28]. That is, for a symmetric kernel $q:\mathcal{Z}\times \mathcal{Z}\mapsto \mathbb{R}$, we can represent the U-statistic $U_n:={\displaystyle \frac{1}{n(n-1)}}{\sum }_{i,j\in \left[n\right]:i\ne j}q(z_i,z_j)$ as

$$\begin{array}{c}\hfill U_n={\displaystyle \frac{1}{n!}}\sum _{\sigma }{\displaystyle \frac{1}{\lfloor n/2\rfloor }}\sum _{i=1}^{\lfloor n/2\rfloor }q(z_{\sigma \left(i\right)},z_{\sigma (\lfloor {\displaystyle \frac{n}{2}}\rfloor +i)}),\end{array}$$

(3)

where $\sigma$ ranges over all permutations of $\{1,\dots ,n\}$.

3.2. Connection with the PAC–Bayesian Framework

As described above, we consider two probability distributions on the hyperparameter space $\mathrm{\Phi }$, to account for the stochasticity in stochastic optimization algorithms, such as Pairwise SGD and Pairwise SGDA, where the hyperparameter $\varphi \in \mathrm{\Phi }$ is a sequence of pairs of indices that follow a discrete distribution. For instance, in Pairwise SGD, in every iteration $t\in \left[T\right]$, we have ${\varphi }_t=(i_t,j_t)$, that is, a pair of independently sampled sample indices, drawn from $\{(i_t,j_t):i_t,j_t\in \left[n\right],i_t\ne j_t\}$ with replacement (more details in Section 4.1). We define two distributions over $\mathrm{\Phi }$, namely the PAC–Bayes prior $\mathrm{P}$, which needs to be specified before seeing the training data, and the PAC–Bayes posterior $\mathrm{Q}$, which is allowed to depend on the training sample. This setting is different from the classic use of PAC–Bayes, which defines the two distributions directly on the trainable parameter space $\mathcal{W}$. Our distributions defined on $\mathrm{\Phi }$ indirectly induce distributions on the parameter estimates, without the need to know their parametric form. This setting of PAC–Bayes was formerly introduced in London [32] in combination with algorithmic stability and further improved in our previous work [31], both restricted to the i.i.d. pointwise setting.

3.3. Connection with the Algorithmic Stability Framework

A more recent framework for the generalization problem considers algorithmic stability [35], which measures the sensitivity of a learning algorithm to small changes in the training data. The concept considered in our work among several notions of algorithmic stability is uniform stability.

Definition 1

(Uniform Stability). For $\forall \varphi$, we say an algorithm $A:S\mapsto A(S;\varphi )$ is ${\beta }_{\varphi }$-uniformly stable if

$$|\ell (A(S;\varphi ),z,\tilde{z})-\ell (A(S^{\prime },\varphi ),z,\tilde{z})|\le {\beta }_{\varphi },\forall z,\tilde{z}\in \mathcal{Z},$$

(4)

where $S,S^{\prime }\in {\mathcal{Z}}^n$ differs by at most a single example.

The algorithmic stability framework is suitable for analyzing certain deterministic learning algorithms, or randomized algorithms with a pre-defined randomization. In turn, here we are concerned with inherently stochastic algorithms where we wish to allow any data-dependent stochasticity, such as the variants of importance sampling and other recent practical methods mentioned in the related works, e.g., [14,15,18,19,34]. Moreover, in principle our framework and results are applicable even if the sampling distribution is learned from the training data itself.

Sub-exponential Stability. A useful definition of stability that captures the stochastic nature of the algorithms we are interested in is the sub-exponential stability introduced in Zhou et al. [31]. Recall that $\varphi$ is a random variable following a distribution defined on $\mathrm{\Phi }$. Therefore, the stability parameter ${\beta }_{\varphi }$ is also a random variable as a function of $\varphi$. We want to control the tail behavior of ${\beta }_{\varphi }$ around a value that decays with the sample size n, and we define the sub-exponential stability as the following.

Definition 2

(Sub-exponential stability). Fix any prior distribution $\mathrm{P}$ on $\mathrm{\Phi }={\prod }_{t=1}^T{\mathrm{\Phi }}_t$. We say that a stochastic algorithm is sub-exponentially ${\beta }_{\varphi }$-stable (with respect to $\mathrm{P}$) if, given any fixed instance of $\varphi \sim \mathrm{P}$, it is ${\beta }_{\varphi }$-uniformly stable and there exist $c_1,c_2\in \mathbb{R}$ such that for any $\delta \in (0,1/n]$, the following holds with probability of at least $1-\delta$:

$${\beta }_{\varphi }\le c_1+c_{2}log(1/\delta ).$$

(5)

4. Main Results

In this section, we will give generalization bounds for Pairwise SGD and Pairwise SGDA in pairwise learning. To this aim, we first give a general result (Lemma 1) to show the connection between the sub-exponential stability condition (Assumption 2) and the generalization gap in the case of pairwise learning. We then derive stability bounds to show that this assumption holds for Pairwise SGD and Pairwise SGDA, in both smooth convex and non-smooth convex cases. Based on these, we apply the stability bounds to Lemma 1 to derive the corresponding generalization bounds. We use $K\lesssim K^{\prime }$ if there exists a universal constant $a>0$ such that $K\le a{K}^{\prime }$. The proof is given in Appendix A.

Lemma 1

(Generalization of randomized pairwise learning). Given distribution $\mathrm{P}$, $c_1,c_2>0$, and M-bounded loss for a sub-exponentially stable algorithm A, $\forall \delta \in (0,1/n)$, with probability of at least $1-\delta$, the following holds uniformly for all $\mathrm{Q}$ absolutely continuous with respect to $\mathrm{P}$:

$${\mathbb{E}}_{\varphi \sim \mathrm{Q}}\left[G(S,\varphi )\right]\lesssim \left(\mathrm{KL}(\mathrm{Q}\parallel \mathrm{P})+log{\displaystyle \frac{1}{\delta }}\right)max\left\{c_{1}logn+c_2{log}^{2}n,{\displaystyle \frac{M}{\sqrt{n}}}\right\},$$

where $\mathrm{KL}(\mathrm{Q}\parallel \mathrm{P})$ is the KL divergence between $\mathrm{Q}$ and $\mathrm{P}$, $\mathrm{KL}(\mathrm{Q}\parallel \mathrm{P})={\int }_{\varphi \in \mathrm{\Phi }}log{\displaystyle \frac{d\mathrm{Q}}{d\mathrm{P}}}d\mathrm{Q}$.

A strength of Lemma 1 is that we only need to check the sub-exponential stability condition under a prior distribution $\mathrm{P}$, and  Lemma 1 automatically translates it to generalization bounds for any posterior distribution $\mathrm{Q}$.

In the forthcoming applications both $\mathrm{Q}$ and P are discrete distributions, so we have $\mathrm{KL}(\mathrm{Q}\parallel \mathrm{P})={\sum }_{\varphi \in \mathrm{\Phi }}\mathrm{Q}\left(\varphi \right)log{\displaystyle \frac{\mathrm{Q}\left(\varphi \right)}{\mathrm{P}\left(\varphi \right)}}$. In particular, the prior $\mathrm{P}$ will be most naturally chosen as the discrete uniform distribution in the context of applications to stochastic optimization in Section 4.2. Let $\mathrm{P}=\mathcal{U}$ with $\mathcal{U}$ denoting the uniform distribution on ${(\left[n\right]\times \left[n\right])}^T$. Hence, the absolute continuity condition is satisfied, ensuring that $\mathrm{KL}(\mathrm{Q}\parallel \mathrm{P})<\infty$ for all distributions $\mathrm{Q}$ over the set ${(\left[n\right]\times \left[n\right])}^T$. Furthermore, in this setting, we have $\mathrm{KL}(\mathrm{Q}\parallel P)=-\mathrm{H}\left(\mathrm{Q}\right)+2Tlogn$, where H denotes the Shannon entropy.

We introduce some classic assumptions that are frequently employed in the analysis of randomized algorithms. Let ${\parallel ·\parallel }_2$ denote the Euclidean norm. Let S and $S^{\prime }$ be neighboring datasets (i.e., they differ in only one example, which we denote as the k-th example, $k\in \left[n\right]$). For brevity, we write $\ell \left(\mathbf{w}\right)$ for $\ell (\mathbf{w};z,\tilde{z})$, where we mean a property that holds for all $z,\tilde{z}\in \mathcal{Z}$.

Assumption 1.

Let $L>0$. We say ℓ is L-Lipschitz if for any ${\mathbf{w}}_1$, ${\mathbf{w}}_2$$\in \mathcal{W}$, we have $|\ell \left({\mathbf{w}}_1\right)-\ell \left({\mathbf{w}}_2\right)|\le L\parallel {\mathbf{w}}_1-{\mathbf{w}}_2{\parallel }_2.$

Assumption 2 (Convexity).

We say ℓ is convex if the following holds $\forall {\mathbf{w}}_1,{\mathbf{w}}_2\in \mathcal{W}$:

$$\ell \left({\mathbf{w}}_1\right)\ge \ell \left({\mathbf{w}}_2\right)+〈\nabla \ell \left({\mathbf{w}}_2\right),{\mathbf{w}}_1-{\mathbf{w}}_2〉,$$

where $\langle ·,·\rangle$ represents the inner product.

Assumption 3.

Let $\alpha \ge 0$. We say a differentiable function ℓ is α-smooth, if for any ${\mathbf{w}}_1$, ${\mathbf{w}}_2$$\in \mathcal{W}$, $\parallel \nabla \ell \left({\mathbf{w}}_1\right)-\nabla \ell \left({\mathbf{w}}_2\right){\parallel }_2\le \alpha {\parallel {\mathbf{w}}_1-{\mathbf{w}}_2\parallel }_2,$ where $\nabla \ell$ represents the gradient of ℓ.

4.1. Stability and Generalization of Pairwise SGD

We now consider Pairwise SGD, which, as we will show, also satisfies the sub-exponential stability condition in both smooth and non-smooth cases.

We denote ${\mathbf{w}}_1$ an initial point and a uniform distribution over ${\left(\left[n\right]\times \left[n\right]\right)}^T$. At the t-th iteration for Pairwise SGD, a pair of sample indices ${\varphi }_t=(i_t,j_t)$ is uniformly randomly selected from the set $\{(i_t,j_t):i_t,j_t\in \left[n\right],i_t\ne j_t\}$. This forms a sequence of index pairs $\varphi =({\varphi }_1,\dots ,{\varphi }_T)$. For step size ${\eta }_t$, the model is updated by ${\mathbf{w}}_{t+1}={\mathbf{w}}_t-{\eta }_t\nabla \ell ({\mathbf{w}}_t;z_{i_t},z_{j_t}).$

The following lemma shows that Pairwise SGD with uniform sampling applied to both smooth and non-smooth problems enjoys sub-exponential stability. The proof is given in Appendix B.1.

Lemma 2 (Sub-exponential stability of Pairwise SGD).

Let $\left\{{\mathbf{w}}_t\right\},\left\{{\mathbf{w}}_t^{\prime }\right\}$ be two parameter sequences produced by Pairwise SGD with fixed step sizes and uniform sampling $\mathrm{P}$, while being trained on neighboring training samples S and $S^{\prime }$. Suppose there is a loss in Lipschitzness and convexity (i.e., Assumptions 1 and 2 hold). Then, we have the following:

(1) 

At the t-th iteration, we have sub-exponential stability (Definition 2) with

$$c_1=2\sqrt{e}{L}^2\eta (\sqrt{t}+2t/n)\mathit{and}{c}_2=4\sqrt{e}{L}^2\eta \left(1+2{(t/n)}^{{\displaystyle \frac{1}{2}}}\right).$$

(2) 

If in addition the loss is also smooth (Assumption 3 holds), then with step size $\eta \le 2/\alpha$, at the t-th iteration, we have sub-exponential stability (Definition 2) with

$$c_1=4L^2\eta t/n\mathit{and}{c}_2=4L^2\eta \left(1+2{(t/n)}^{{\displaystyle \frac{1}{2}}}\right).$$

Using Lemmas 1 and 2, we obtain the following generalization bound for Pairwise SGD with general sampling.

Theorem 1 (Generalization bounds for Pairwise SGD).

Assume ℓ is M-bounded, Lipschitz, and convex (cf. Assumptions 1 and 2). For any $\delta \in (0,1)$, Pairwise SGD with fixed step sizes satisfies the following generalization guarantees with probability of at least $1-\delta$ over S, $S\sim {\mathcal{D}}^n$, for all posterior sampling distributions $\mathrm{Q}$ on ${\left(\left[n\right]\times \left[n\right]\right)}^T$:

(1) 

After T iterations, we have

$${\mathbb{E}}_{\mathrm{Q}}\left[G(S,\varphi )\right]\lesssim \left(Tlogn-\mathrm{H}\left(\mathrm{Q}\right)+log{\displaystyle \frac{1}{\delta }}\right)max\left\{L\eta \left(\sqrt{T}+{\displaystyle \frac{T}{n}}+\sqrt{{\displaystyle \frac{T}{n}}}\right){log}^{2}n,{\displaystyle \frac{M}{\sqrt{n}}}\right\}.$$

(2) 

If in addition the loss is also smooth (Assumption 3 holds), then with step size $\eta \le 2/\alpha$, we have

$${\mathbb{E}}_{\mathrm{Q}}\left[G(S,\varphi )\right]\lesssim \left(Tlogn-\mathrm{H}\left(\mathrm{Q}\right)+log{\displaystyle \frac{1}{\delta }}\right)max\left\{L\eta \left({\displaystyle \frac{T}{n}}+1+\sqrt{{\displaystyle \frac{T}{n}}}\right){log}^{2}n,{\displaystyle \frac{M}{\sqrt{n}}}\right\}.$$

Remark 1.

Suppose $\mathrm{KL}(\mathrm{Q}\parallel \mathcal{U})\in \tilde{O}\left(1\right)$, as it has been tacitly assumed also in previous work [32] when quantifying the generalization convergence rate. Taking the choice of parameters suggested by [51], if $\eta =\mathrm{\Theta }(T^{-{\displaystyle \frac{3}{4}}})$ and $T=\mathrm{\Theta }\left(n^2\right)$ in the non-smooth case (part 1), then the above theorem implies bounds of the order $\tilde{O}(1/\sqrt{n})$. In the smooth case (part 2), an analysis of the trade-off between optimization and generalization, Lei et al. [33] suggested setting $T=\mathrm{\Theta }\left(n\right)$ and $\eta =\mathrm{\Theta }(1/\sqrt{T})$ to get a Pairwise SGD to iterate with a good generalization performance. With these choices, our bounds in Theorem 1 are of order $\tilde{O}(1/\sqrt{n})$, which are not improvable in general.

Remark 2

(Implication of the $\mathrm{KL}(\mathrm{Q}\parallel \mathcal{U})=\tilde{O}\left(1\right)$ assumption). Let $\mathrm{supp}\left(\mathrm{Q}\right)\subseteq \mathrm{\Phi }$ denote the support of $\mathrm{Q}$, where $\mathrm{\Phi }={(\left[n\right]\times \left[n\right])}^T$ in pairwise learning.

Since $\mathcal{U}\left(\varphi \right)=1/n^{2T}$ for all ϕ, the KL divergence is

$$\mathrm{KL}(\mathrm{Q}\parallel \mathcal{U})=\sum _{\varphi \in \mathrm{\Phi }}\mathrm{Q}\left(\varphi \right)log\left({\displaystyle \frac{\mathrm{Q}\left(\varphi \right)}{1/n^{2T}}}\right)=-H\left(\mathrm{Q}\right)+2Tlogn.$$

To ensure this is of order $\tilde{O}\left(1\right)$, we need $-H\left(\mathrm{Q}\right)+2Tlogn\le \tilde{O}\left(1\right)$; hence,

$$\begin{array}{c}\hfill H\left(\mathrm{Q}\right)\ge 2Tlogn-\tilde{O}\left(1\right)=H\left(\mathcal{U}\right)-\tilde{O}\left(1\right).\end{array}$$

(6)

To give more intuition, consider $\mathrm{Q}$ on a restricted support. This is very much a worst-case scenario, as it would imply completely discarding (rather than down-weighting) some of the training points. For such $\mathrm{Q}$, the maximum entropy occurs when $\mathrm{Q}$ is uniform over its support, so $\mathrm{Q}\left(\varphi \right)={\displaystyle \frac{1}{\left|\mathrm{supp}\right(\mathrm{Q}\left)\right|}}$, and $\mathrm{KL}(\mathrm{Q}\parallel \mathcal{U})=log\left({\displaystyle \frac{n^{2T}}{\left|\mathrm{supp}\right(\mathrm{Q}\left)\right|}}\right)$. In this case, having $\mathrm{KL}(\mathrm{Q}\parallel \mathcal{U})=\tilde{O}\left(1\right)$ requires

$$\begin{array}{c}\hfill \left|\mathit{supp}\left(\mathrm{Q}\right)\right|=\mathrm{\Omega }\left({\displaystyle \frac{n^{2T}}{\mathit{poly}(T,n)}}\right).\end{array}$$

(7)

To sum up, $\mathrm{Q}$ must satisfy the entropy lower bound (6), and to achieve that entropy on a restricted support, it must have a large enough support, i.e., at least a $\mathrm{\Omega }(1/\mathit{poly}(T,n\left)\right)$ fraction of the entire Φ.

In Pairwise SGD with non-uniform data-dependent sampling, this result tells us that in order to keep generalization rates that compare against the uniform baseline, $\mathrm{Q}$ cannot discard a large subset of the index sequences. This limits how aggressively one can compress or “distill” a dataset (as in core-set selection or dataset distillation) without paying a KL penalty that slows down the rate—at least as long as the prior is uniform.

Non-uniform sampling alone is insufficient for robust learning. While it may mitigate the effect of a small fraction of bad examples (e.g., out-of-distribution or mislabeled training examples), achieving robustness also requires modeling choices such as robust loss functions. In the next section we approach this via Pairwise SGDA, the type of optimization required in adversarially robust training.

4.2. Stability and Generalization of Pairwise SGDA

In this subsection, we discuss Pairwise SGDA for solving minimax problems in the convex-concave case. We will abuse the notations to apply them to the minimax case. We receive a model $A(S;\varphi ):=\left(A_{\mathrm{w}}(S;\varphi ),A_{\mathrm{v}}(S;\varphi )\right)\in \mathcal{W}\times \mathcal{V}$ by applying a learning algorithm A on training set S and measure the performance with respect to a loss function $\ell :(\mathbf{w},\mathbf{v})\mapsto \ell (\mathbf{w},\mathbf{v};z,\tilde{z})$. For any $\varphi \in \mathrm{\Phi }$, we consider the risk defined as

$$\underset{\mathbf{w}\in \mathcal{W}}{min}\underset{\mathbf{v}\in \mathcal{V}}{max}R(A_{\mathbf{w}}(S;\varphi ),A_{\mathbf{v}}(S;\varphi )):={\mathbb{E}}_{z,\tilde{z}\sim \mathbb{D}}[\ell \left(A_{\mathbf{w}}(S;\varphi ),A_{\mathbf{v}}(S;\varphi );z,\tilde{z}\right)].$$

We consider the following empirical risk as the approximation:

$$R_S(A_{\mathbf{w}}(S;\varphi ),A_{\mathbf{v}}(S;\varphi )):={\displaystyle \frac{1}{n(n-1)}}\sum _{i,j\in \left[n\right]:i\ne j}\ell \left(A_{\mathbf{w}}(S;\varphi ),A_{\mathbf{v}}(S;\varphi );z_i,z_j\right).$$

We consider Pairwise SGDA with a general sampling scheme, where the random index pairs follow from a general distribution.

We denote ${\mathbf{w}}_1$ and ${\mathbf{v}}_1$ the initial points. Let ${\nabla }_{\mathbf{w}}\ell$ and ${\nabla }_{\mathbf{v}}\ell$ be the gradients with respect to $\mathbf{w}$ and $\mathbf{v}$, respectively. Let $\mathrm{P}$ be a uniform distribution over ${\left(\left[n\right]\times \left[n\right]\right)}^T$ and S be a training dataset with n samples. Let $(i_t,j_t)$ from set $\{(i_t,j_t):i_t,j_t\in \left[n\right],i_t\ne j_t\}$ be drawn uniformly at random. At the t-th iteration, with step size sequence $\left\{{\eta }_t\right\}$, Pairwise SGDA updates the model as follows:

$$\left\{\begin{array}{c}{\mathbf{w}}_{t+1}={\mathbf{w}}_t-{\eta }_t{\nabla }_{\mathbf{w}}\ell ({\mathbf{w}}_t,{\mathbf{v}}_t;z_{i_t},z_{j_t}),\hfill \\ {\mathbf{v}}_{t+1}={\mathbf{v}}_t+{\eta }_t{\nabla }_{\mathbf{v}}\ell ({\mathbf{w}}_t,{\mathbf{v}}_t;z_{i_t},z_{j_t}).\hfill \end{array}\right.$$

Before giving the results for Pairwise SGDA, we restate the assumptions we need, adapted to the new setting with two distinct parameter vectors $\mathbf{w}$ and $\mathbf{v}$ [38,52].

Assumption 4.

Let $L\ge 0$. We say a differentiable function ℓ is L-Lipschitz with respect to $\mathbf{w}$ and $\mathbf{v}$ if the following holds: For any $z,\tilde{z}\in \mathcal{Z}$, $\mathbf{w}\in \mathcal{W}$, $\mathbf{v}\in \mathcal{V}$, we have

$${∥{\nabla }_{\mathbf{w}}\ell (\mathbf{w},\mathbf{v};z,\tilde{z})∥}_2\le L\mathit{and}{∥{\nabla }_{\mathbf{v}}\ell (\mathbf{w},\mathbf{v};z,\tilde{z})∥}_2\le L.$$

Assumption 5.

Let $\alpha >0$. We say a differentiable function ℓ is α-smooth if the following inequality holds for any ${\mathbf{w}}_1$, ${\mathbf{w}}_2\in \mathcal{W}$, ${\mathbf{v}}_1$, ${\mathbf{v}}_2\in \mathcal{V}$ and $z,\tilde{z}\in \mathcal{Z}$:

$${∥\left(\begin{array}{c}{\nabla }_{\mathbf{w}}f({\mathbf{w}}_1,{\mathbf{v}}_1;z,\tilde{z})-{\nabla }_{\mathbf{w}}f\left({\mathbf{w}}_2,{\mathbf{v}}_2;z,\tilde{z}\right)\\ {\nabla }_{\mathbf{v}}f({\mathbf{w}}_1,{\mathbf{v}}_1;z,\tilde{z})-{\nabla }_{\mathbf{v}}f\left({\mathbf{w}}_2,{\mathbf{v}}_2;z,\tilde{z}\right)\end{array}\right)∥}_2\le \alpha {∥\left(\begin{array}{c}{\mathbf{w}}_1-{\mathbf{w}}_2\\ {\mathbf{v}}_1-{\mathbf{v}}_2\end{array}\right)∥}_2.$$

Assumption 6 (Convexity-Concavity).

We say ℓ is concave if $-\ell$ is convex. We say ℓ is convex-concave if $\ell (·,\mathbf{v})$ is convex for every $\mathbf{v}\in \mathcal{V}$ and $\ell (\mathbf{w},·)$ is concave for every $\mathbf{w}\in \mathcal{W}$.

Now, we apply Lemma 1 to develop bounds for Pairwise SGDA in both smooth and non-smooth cases. In the following lemma, proved in Appendix B.2, we establish the sub-exponential stability of Pairwise SGDA.

Lemma 3 (Sub-exponential stability of Pairwise SGDA).

Let $\{{\mathbf{w}}_t,{\mathbf{v}}_t\},\{{\mathbf{w}}_t^{\prime },{\mathbf{v}}_t^{\prime }\}$ be two parameter sequences produced by Pairwise SGDA with fixed step sizes and uniform sampling $\mathrm{P}$ while being trained on neighboring samples S and $S^{\prime }$. Suppose the loss is Lipschitz and convex (cf. Assumptions 4–6). Then, we have the following:

(1) 

At the t-th iteration, we have sub-exponential stability (Definition 2) with

$$c_1=2\sqrt{2e}{L}^2\eta (\sqrt{t}+2t/n)\mathit{and}{c}_2=4\sqrt{2e}{L}^2\eta (1+\sqrt{2t/n}).$$

(2) 

If in addition the loss is also smooth (cf. Assumption 5), then at the t-th iteration, sub-exponential stability (Definition 2) holds with

$$c_1=4\sqrt{e}{L}^2\eta exp\left({\displaystyle \frac{1}{2}}{\alpha }^{2}t{\eta }^2\right)(1+2t/n)\mathit{and}{c}_2=8\sqrt{e}{L}^2\eta exp\left({\displaystyle \frac{1}{2}}{\alpha }^{2}t{\eta }^2\right)(1+\sqrt{2t/n}).$$

We combine the above lemma with Lemma 1 to obtain bounds for Pairwise SGDA with a general sampling distribution.

Theorem 2 (Generalization bounds for Pairwise SGDA).

Assume ℓ is M-bounded, Lipschitz, and convex (cf. Assumptions 4 and 6). For  any $\delta \in (0,1)$, Pairwise SGDA with fixed step sizes satisfies the following generalization guarantees with probability of at least $1-\delta$ over draws of $S\sim {\mathcal{D}}^n$, for all posterior sampling distributions $\mathrm{Q}$ on ${\left(\left[n\right]\times \left[n\right]\right)}^T$:

(1) 

After T iterations, we have

$${\mathbb{E}}_{\varphi \sim \mathrm{Q}}\left[G(S,\varphi )\right]\lesssim \left(Tlogn-\mathrm{H}\left(\mathrm{Q}\right)+log{\displaystyle \frac{1}{\delta }}\right)max\left\{L^2\eta (\sqrt{T}+T/n){log}^{2}n,{\displaystyle \frac{M}{\sqrt{n}}}\right\}.$$

(2) 

If in addition the loss is also smooth (cf. Assumption 5), we have

$$\begin{array}{c}{\mathbb{E}}_{\varphi \sim \mathrm{Q}}\left[G(S,\varphi )\right]\lesssim \left(Tlogn-\mathrm{H}\left(\mathrm{Q}\right)+log{\displaystyle \frac{1}{\delta }}\right)\hfill \\ \hfill max\left\{L^2\eta exp\left({\alpha }^{2}t{\eta }^2\right)\left({\displaystyle \frac{T}{n}}+1+\sqrt{{\displaystyle \frac{T}{n}}}\right){log}^{2}n,{\displaystyle \frac{M}{\sqrt{n}}}\right\}.\end{array}$$

Let us assume again that $\mathrm{KL}(\mathrm{Q}\parallel \mathcal{U})\in \tilde{O}\left(1\right)$. As Theorem 2 deals with min-max optimization applicable to minimizing an adversarially robust loss function, this assumption still allows some extra flexibility to account for a few outliers while having the following rates. For part 1, if we choose $T=O\left(n^2\right)$ and $\eta =O\left(T^{-3/4}\right)$, this gives a rate of the order $\tilde{O}(1/\sqrt{n})$. For part 2, if we choose $T=O\left(n\right)$ and $\eta =O(1/\sqrt{n})$, this gives the bounds of the order $\tilde{O}(1/\sqrt{n})$.

5. Algorithmic Implications and Illustrative Experiments

Our theoretical guarantees in the previous section are given up to constant factors. This is common in theoretical analyses, as such results still give useful information about the behavior of bounds with quantities of interest, such as the sample size n. To further verify that our bounds are informative, in this section we show how one can convert them into learning algorithms by minimizing the terms on the r.h.s. of our bounds. We will then illustrate the working of the resulting algorithms in numerical experiments and demonstrate examples of extracting meaningful information from these new algorithms. The goal of this section is to empirically corroborate our theoretical guarantees and demonstrate their potential use for algorithm design.

In line with our theory, we use uniform sampling as the PAC–Bayes prior, and we learn the posterior sampling along with the model’s parameters, by alternating minimization of our bounds. We choose $\mathrm{Q}=q^T$ of a factorized form, which corresponds to sampling from training indices $\left[n\right]$ with replacement T times during the training trajectory. Minimizing the terms on the r.h.s. of the bounds in Theorem 1 yields an adaptive Pairwise SGD algorithm that we refer to as Pairwise SGD-Q, and likewise minimizing the r.h.s. of the bounds in Theorem 2 yields an adaptive Pairwise SGDA algorithm SGDA-Q. The pseudo-codes of both of these resulting algorithms are given in Algorithm 1 (with the options of Pairwise SGD-Q or Pairwise SGDA-Q).

Algorithm 1 Pairwise SGD-Q/Pairwise SGDA-Q

1: Inputs: $\{(i_t,j_t):i_t,j_t\in \left[n\right],i_t\ne j_t\}$, ℓ, $\nu$, $T_{iter}$, $Epochs$ 2: Initialize: $q\leftarrow$ uniform, ${\mathbf{w}}_0=0$, ${\mathbf{v}}_0=0$, $t\leftarrow 1$ 3: for $epoch=1\mathbf{to}Epochs$ do 4:    for $t=1\mathbf{to}{T}_{iter}$ do 5:      Sample $(i_t,j_t)\sim q$ 6:      Pairwise SGD-Q: 7:      ${\mathbf{w}}_{t+1}={\mathbf{w}}_t-\eta {\nabla }_{\mathbf{w}}\ell ({\mathbf{w}}_t;z_{i_t},z_{j_t})$; 8:      Pairwise SGDA-Q: 9:      ${\mathbf{v}}_{t+1}={\mathbf{v}}_t+\eta {\nabla }_{\mathbf{v}}\ell (({\mathbf{w}}_t,{\mathbf{v}}_t);z_{i_t},z_{j_t})$; 10:      ${\mathbf{w}}_{t+1}={\mathbf{w}}_t-\eta {\nabla }_{\mathbf{w}}\ell (({\mathbf{w}}_t,{\mathbf{v}}_t);z_{i_t},z_{j_t})$; 11:      Let $t\leftarrow t+1$ 12:    end for 13:    Update all q as $q(i,j)\propto exp\left(-{\displaystyle \frac{1}{\nu }}\ell (\mathbf{w};z_i,z_j)\right)$ for each pair 14: end for 15: Return $\mathbf{w}$, $\mathbf{v}$, q

Before moving on to exemplify our algorithms at work, we should note that there are two in-built guardrails that prevent sampling bias by design, as follows. The r.h.s. of all bounds that we minimize contain two key terms, each acting as a guardrail: (i) Minimizing the $D_{\mathrm{KL}(Q\parallel P)}$ term is equivalent to maximizing the entropy of Q, i.e., the sampling distribution that we learn (since P is the uniform sampling)—hence, Q will only deviate from uniformity for a good reason (e.g., when encountering misleading or mislabeled training pairs). (ii) Minimizing the expected empirical risk (rather than the empirical risk itself) has the consequence that the correction term that usually appears in existing non-uniform sampling-based SGD-type algorithms simply cancels out, automatically ensuring unbiased gradients by construction and without an external correction. Overall, this illustrates the advantage of learning algorithms obtained by minimizing generalization bounds, as the two terms together minimize an upper bound on the quantity we care about, i.e., the true risk.

The forthcoming numerical experiments are meant to showcase the way in which our adaptive sampling methods enhance robustness by learning to down-weight misleading or mislabeled training examples, thereby avoiding being misled on them. As a byproduct, these weights provide explanatory information about the data pairs.

Numerical Results in Pairwise Preference Learning

We illustrate the working of our bound-based pairwise learning algorithms on a toy problem involving pairwise ranking on synthetic 2D data. Pairwise ranking is the task of inferring relative preferences by comparing items in pairs. It has broad applicability, including information retrieval, recommendation systems, preference modeling [53], and positive-unlabeled learning [12]. In this section, we present experiments using a simple linear preference model to demonstrate how our theoretical findings manifest in practice. The aim of this section is not to compete with state-of-the-art empirical methods, but rather to provide insight into the behavior of the algorithms and the use of our bounds under controlled conditions.

We generate $n=50$ i.i.d. points from a 2D standard Gaussian and assign “preference scores” to each using a hidden linear function $s_{true}\left(x_i\right)={\mathbf{w}}_{true}^T{x}_i,i\in \left[n\right]$, where ${\mathbf{w}}_{\mathrm{true}}={(1.5,-1)}^T$ is fixed. We then sample 1000 pairs from this data and assign binary labels $\mathbb{1}({\mathbf{w}}_{true}{x}_i\le {\mathbf{w}}_{true}{x}_j)$ indicating which of two items is preferred in each sampled pair.

We use the resulting labeled pairs in the form of difference vectors $x_i-x_j$ (where $i,j\in \left[n\right],i\ne j$) as inputs to our Pairwise SGD-Q algorithm to train a linear model with cross-entropy loss for 15 epochs of $T_{iter}$ iterations, each set equal to the number of pairs (so $T=T_{iter}·Epochs$), with step size $\eta =0.1$. We aim to learn a scoring function so that for any two items $x_i$ and $x_j$ the model can say which one ranks higher. The model learns a weight vector $\mathbf{w}$ so that the score of item x is $s\left(x\right)={\mathbf{w}}^{T}x$, and we want ${\mathbf{w}}^T{x}_i>{\mathbf{w}}^T{x}_j$ if item i is preferred to j. That is, the model projects all points onto the direction $\mathbf{w}$ and ranks them by how far they fall along that line.

The results are plotted in Figure 1. The top-left figure shows the 2D data colored by their preference scores. A red arrow shows the ranking direction learned by our Pairwise SGD-Q, i.e., the direction the trained model uses to order points. Its associated “decision boundary” is shown by the dashed line. In this context, the decision boundary represents the level set ${\mathbf{w}}^{T}x=0$ that is the dividing line (more generally, hyperplane) orthogonal to the learned ranking direction. It shows how $\mathbf{w}$ splits the space into higher- and lower-scoring regions.

The top-right figure is a scatter plot showing the relationship between the learned pairwise margins ${\mathbf{w}}^T(x_i-x_j)$ and the learned sampling probabilities $q(i,j)$ (q-scores for each pair). Ambiguous pairs with small margins, i.e., those the model is less confident about, tend to have higher losses and thus get down-weighted by a lower sampling probability $q(i,j)$. Hence, more confident (larger-margin) pairs are sampled more often.

The bottom-left plot overlays an edge for the 20 lowest q-scored pairs. These are the pairs that have the same preference score but differing feature coordinates—indeed the least helpful pairs for learning the preference change direction. The 20 highest q-scored pairs are shown on the bottom-right plot; these are most informative of the direction of change.

We repeated the experiment using a noisy preference model, where the observed scores are given by $s_{true}\left(x_i\right)={\mathbf{w}}_{true}^{\top }{x}_i+{\upsilon }_i$, with ${\upsilon }_i{\sim }_{\mathrm{i}.\mathrm{i}.\mathrm{d}.}\mathcal{N}(0,1)$ for $i\in \left[n\right]$. Figure 2 shows the results of training a model of the same form as previously, using our Pairwise SGD-Q. The top-left plot depicts the noisy data overlaid with the learned direction $\mathbf{w}$. The top-right plot shows the pairwise margins ${\mathbf{w}}^{\top }(x_i-x_j)$ against the corresponding sampling probabilities $q(i,j)$. While q still decreases near the pairwise margin, the noise now causes mislabeled or ambiguous pairs to receive even lower q values. This becomes evident in the bottom-left plot: the lowest q-scored pairs have similar preference scores but differing features and are no longer orthogonal to $\mathbf{w}$ due to noise. The bottom-right plot shows the highest q-scored pairs, which continue to reflect the most informative preference differences aligned on average with the learned direction.

Next, we plot learning curves to see how the generalization performance of the pairwise ranking model trained with our Pairwise SGD-Q and Pairwise SGDA-Q algorithms varies with the number of i.i.d. items, under both clean and noisy settings, while we keep the number of pairs used for training constant. The Pairwise SGDA-Q experiments represent an instance of adversarial training. This can model, for instance, malicious users, bots, or strategic agents in applications like recommendation systems, crowd-sourced ranking, sports, or election ranking.

We vary $n\in \{5,10,20,30,40,50,60,70,80,90,100\}$ and repeat all experiments on both clean and noisy preference score generation models. This totals 44 different experiment settings, and we perform 50 independent trials for each. The data generation process is the same as before. While the sample size n varies, the number of pairs sampled from these n points for training is set to 1000 throughout, and these pairs are labeled in the same way as before.

Training with Pairwise SGD-Q is performed in the same way as previously described, but this time we set $T_{iter}=300$ to run for 30 epochs. When training with Pairwise SGDA-Q, at each training epoch the algorithm first computes the pairwise losses after adversarial maximization over weight vector $\mathbf{v}$ constrained to an ${\ell }_2$-ball of radius $ϵ=0.05$ around the current model $\mathbf{w}$, using 6 gradient ascent steps with step size $\eta =0.1$. The resulting adversarially induced losses are used to compute the sampling distribution $q(i,j)\propto exp(-{\ell }_{ij})$, from which $T_{iter}=300$ pairs are drawn to iteratively update $\mathbf{w}$, for 30 epochs.

For both algorithms, evaluation is performed using an independent test set of 500 unseen items, drawn from the clean preference scoring model. For Pairwise SGD-Q, the out-of-sample errors are computed. For Pairwise SGDA-Q, three different out-of-sample error metrics are computed: (1) standard pairwise error, (2) error under adversarial perturbation of the model’s weights $\mathbf{w}$, and (3) error under adversarial perturbation of the test pairwise inputs $x_i-x_j$, all within the same ${\ell }_2$ radius and using the same Pairwise SGDA-Q ascent procedure.

Figure 3 reports the obtained learning curves for both algorithms: the error bars show the average and standard error across 50 independent runs. From these figures we can see, as expected, that both natural noise and adversarial perturbations make the problem harder. However, all out-of-sample errors display a decreasing trajectory as the sample size grows.

6. Conclusions

We obtained stability-based PAC–Bayes bounds for randomized pairwise learning, applicable to general sampling. These bounds are applicable to analyzing the generalization of stochastic optimization algorithms, and we demonstrated this in the case of Pairwise SGD and Pairwise SGDA. Our generalization analysis of these methods is suggestive of new stochastic optimizers that allow non-uniform and data-dependent sampling distributions to be updated during the training process. We believe this is a theoretically grounded step that connects two important ideas and may support future work on more complex or application-specific methods. The practical use of this idea is explored in a companion paper [22].

Limitations: Our analysis of Pairwise SGD and Pairwise SGDA is built on a set of classic assumptions regarding the loss function, with convexity being perhaps the most restrictive among them. Nonetheless, insights gained from the convex setting remain a valuable stepping stone to tackling more general, non-convex problems in future work. Indeed, a worthwhile avenue will be to obtain bounds and associated algorithms under relaxed assumptions. Furthermore, here we demonstrated numerical results with our bound-based algorithm under its intended conditions. It will also be interesting to explore experimentally to what extent such algorithms remain functional and potentially useful outside the theoretical conditions in which they were obtained.