Moment Estimation in Paired Comparison Models with a Growing Number of Subjects

Abstract

When the number of subjects, n, is large, paired comparisons are often sparse. Here, we study statistical inference in a class of paired comparison models parameterized by a set of merit parameters, under an Erdös–Rényi comparison graph, where the sparsity is measured by a probability $p_n$ tending to zero. We use the moment estimation base on the scores of subjects to infer the merit parameters. We establish a unified theoretical framework in which the uniform consistency and asymptotic normality of the moment estimator hold as the number of subjects goes to infinity. A key idea for the proof of the consistency is that we obtain the convergence rate of the Newton iterative sequence for solving the estimator. We use the Thurstone model to illustrate the unified theoretical results. Further extensions to a fixed sparse comparison graph are also provided. Numerical studies and real data analysis illustrate our theoretical findings.

1. Introduction

Subjects are repeatedly compared in pairs in a wide spectrum of situations, including sports games [1], ranking of scientific journals [2,3], the quality of product brands [4] and crowdsourcing [5]. For instance, one team plays with another team in basketball; papers in one journal cite papers in another journal; one consumer chooses one product over another; workers in a crowdsourcing setup are asked to compare pairs of items.

One of the fundamental problems in paired comparison analysis is to derive a fair and reliable ranking of all subjects based on observed comparison data. In round-robin tournaments—where every pair of subjects competes sufficiently many times—a natural ranking can be directly obtained from the number of wins, as the full pairwise comparisons eliminate biases from incomplete matchups. However, in most practical scenarios (e.g., sports leagues, crowdsourcing evaluations, or journal rankings), comparisons are often sparse (not all pairs interact) and stochastic (outcomes contain random noise), leading to unreliable direct rankings based solely on raw win counts. To address this issue, paired comparison models have been developed to statistically infer the underlying merit parameters of subjects and generate objective rankings; see the classic monograph by [6] for a comprehensive overview of such models and their theoretical foundations. Statistical models not only provide a method of ranking all subjects but are also tools for making inferences on the merits of subjects (e.g., testing whether two subjects have the same merit).

Here, we are concerned with a class of paired comparison models that assign one merit parameter to each subject and assume that the win–loss probability of any pair only depends on the difference between their merit parameters. Specifically, the probability of subject i winning j is

$$\mathbb{P}\left(i\mathrm{wins}j\right)=F({\beta }_i-{\beta }_j),i,j=0,1,\dots ,n;i\ne j,$$

(1)

where F is a known cumulative distribution function satisfying $F\left(x\right)=1-F(-x)$, ${\beta }_i$ is the merit parameter of subject i and $n+1$ is the total number of subjects. The well-known Bradley–Terry model [7], which dates back to at least 1929 [8], and the Thurstone model [9], are two special cases of Model (1). The former postulates the logistic distribution of $F\left(x\right)$, while the latter postulates the normal distribution.

In the standard setting that n is fixed and the number of comparisons in each pair goes to infinity, the theoretical properties of Model (1) have been widely investigated in Chapter 4 of [6]. In the opposite scenario that n goes to infinity and each pair has a fixed number of comparisons, ref. [10] proved the uniform consistency and asymptotic normality of the maximum likelihood estimator (MLE) in the Bradley–Terry model.

When the number of subjects is large, paired comparisons are often sparse. Taking the NCAA Division I FBS (Football Bowl Subdivision) regular season, for example, a team plays with at most 14 other teams among a total of 120 teams. The observed comparisons can be represented in a comparison graph with $n+1$ nodes denoting subjects and a weighted edge between two nodes denoting the number of comparisons. The Erdös–Rényi comparison graph has been widely considered in the literature, e.g., [1,11,12,13], where the number of comparisons between any two subjects follows a binomial distribution $(T,p_n)$, and $p_n$ measures the sparsity. Under a very weak condition on the sparsity on $p_n$, ref. [13] established the uniform consistency and asymptotic normality of the MLE in the Bradley–Terry model by extending the proof strategies in [10].

Moreover, ref. [14] considered a fixed sparse comparison graph by controlling the length from one subject to another subject with 2 or 3, in which the consistency and asymptotic normality of the MLE also hold. Inference in the high-dimensional setting under the Bradley–Terry model and some generalized versions has also attracted great interest in the machine learning literature; the upper bounds of various errors are established under different conditions [e.g., the ${\ell }_1$ error $\parallel \widehat{\beta }{-\beta \parallel }_1$ in [15,16], the mean square error in [17], the bias $\mathbb{E}\parallel \widehat{\beta }{-\beta \parallel }_{\infty }$ in [18]. Under the assumption that the log-likelihood function is strictly convex, ref. [1] establish the uniform consistency of the MLE in general paired comparison models. However, the asymptotic theory of moment estimation under sparse paired comparison models remains largely underdeveloped in the literature. Existing theoretical developments focus almost exclusively on maximum likelihood estimation (MLE), which relies on strict distributional assumptions and is computationally demanding in high-dimensional settings. In contrast, this paper develops the method of moments (MOM), which avoids full distribution assumptions and maintains computational simplicity while achieving comparable asymptotic properties. The primary novelty of this work is to extend the asymptotic theory of high-dimensional sparse paired comparison models from MLE to the method of moments, establishing a parallel and complementary theoretical framework.

We further elaborate on the advantages of the method of moments (MOM) for high-dimensional sparse paired comparison models. Beyond computational efficiency, MOM has two key strengths over maximum likelihood estimation (MLE) for practical inference: (1) Robustness—the moment estimator is much less sensitive to outliers (e.g., upsets in our NFL data) in sparse settings, where MLE can be biased by extreme observations; (2) Quasi-likelihood compatibility—MOM relies only on moment conditions and avoids strict distributional assumptions, making it robust to model misspecification in high-dimensional sparse scenarios. These merits justify the use of MOM in this study, and our subsequent analysis establishes its asymptotic properties as the core theoretical contribution.

The main contributions of this paper are as follows. First, we develop the moment estimation, instead of the maximum likelihood estimation (MLE) or Bayesian estimation, based on the scores of subjects (i.e., the number of wins) to estimate the merit parameters in Model (1). The reason why we prioritize moment estimation over MLE is that it is natural to rank subjects according to their scores and the computation based on moment equations is simpler, especially in high-dimensional sparse settings where MLE may suffer from numerical instability due to nonlinear optimization. When $F(·)$ belongs to the exponential family distribution, both estimations are identical. Second, under an Erdös–Rényi comparison graph, we establish a unified theoretical framework in which the uniform consistency and asymptotic normality of the moment estimator hold when n goes to infinity and $p_n$ tends to zero. A key idea for the proof of the consistency is that we obtain the convergence rate of the Newton iterative sequence for solving the estimator. The asymptotic normality is proved by applying Taylor expansions to a series of functions constructed from estimating equations and showing that remainder terms in the expansions are asymptotically neglected. Although each pair of subjects is assumed to have a comparison with the same probability $p_n$, our proof strategy can be easily extended to the case with different comparison probabilities at the order of $p_n$. Third, we use the Thurstone model to illustrate the unified theoretical results. Further extensions to a fixed sparse comparison graph in [14] are also derived. Numerical studies and real data analysis illustrate our theoretical findings.

The rest of this paper is organized as follows. In Section 2, we present the moment estimation. In Section 3, we present the consistency and asymptotic normality of the moment estimator. We illustrate our unified results with one application in Section 4. We extend the asymptotic results to a fixed comparison graph in Section 5. In Section 6, we carry out simulations and give real data analysis. We give a summary and further discussion in Section 7. The proofs of the main results are relegated to Appendix A. The proofs of supported lemmas are relegated to Appendix B.

2. Moment Estimation

Assume that $n+1$ subjects that are labeled as “$0,\dots ,n$”, are compared in pairs repeatedly. Let $t_{ij}$ be the times that subject i compares with subject j and $a_{ij}$ be the times that subject i wins subject j out of $t_{ij}$ comparisons. As a result, $a_{ij}+a_{ji}=t_{ij}$. By convention, define $t_{ii}=0$ and $a_{ii}=0$. The comparison matrix ${\left(t_{ij}\right)}_{(n+1)\times (n+1)}$ is generated from an Erdös–Rényi comparison graph, where $t_{ij}$ follows a binomial distribution $Bin(T,p_n)$ with $p_n$ measuring the sparsity of comparisons. More generally, $t_{ij}\sim Bin(T_{ij},p_n)$. We set $T_{ij}$ to be the same for ease of exposition. Recall that ${\beta }_0,\dots ,{\beta }_n$ are the merit parameters of subjects $0,\dots ,n$. The probability in Model (1) implies that the winning probability only depends on the difference in merit between two subjects. For the identification of models, we normalize ${\beta }_i,i=0,1,\dots ,n$ by setting ${\beta }_0=0$ as in [10]. We assume that all paired comparisons are independent and $a_{ij}$ follows a binomial distribution $Bin(t_{ij},p_{ij})$ conditional on $t_{ij}$.

Let $a_i={\sum }_{j=0}^n{a}_{ij}$ be the total wins of subject i and $a={(a_1,\dots ,a_n)}^{\top }$. To motivate the estimating equations, we compare the maximum likelihood equation and the moment equation under the Thurstone model described in Section 4. The maximum likelihood equations are

$$\sum _{j\ne i}\left[{\displaystyle \frac{a_{ij}\varphi ({\beta }_i-{\beta }_j)}{\mathrm{\Phi }({\beta }_i-{\beta }_j)}}-{\displaystyle \frac{(t_{ij}-a_{ij})\varphi ({\beta }_i-{\beta }_j)}{1-\mathrm{\Phi }({\beta }_i-{\beta }_j)}}\right]=0,i=1,\dots ,n.$$

where $\varphi (·)$ is the density function of the standard normality and $\mathrm{\Phi }(·)$ is its distribution function. The corresponding moment equations are

$$a_i=\sum _{j\ne i}{t}_{ij}\mathrm{\Phi }({\beta }_i-{\beta }_j),i=1,\dots ,n.$$

We can see that the latter is simpler and easier to compute. On the other hand, it is natural to rank subjects according to their scores. Thus, we use the moment estimation here. When $F(·)$ in Model (1) belongs to the exponential family distributions, both are the same.

Write $\mu (·)$ as the expectation of $F(·)$ and ${\mu }_{ij}\left(\beta \right)=\mu ({\beta }_i-{\beta }_j)$. Then, the estimating equations are

$$a_i=\sum _{j\ne i}{t}_{ij}{\mu }_{ij}\left(\beta \right),i=1,\dots ,n.$$

(2)

The solution to the above equations is the moment estimator denoted by $\widehat{\beta }={({\widehat{\beta }}_1,\dots ,{\widehat{\beta }}_n)}^{\top }$ and ${\widehat{\beta }}_0=0$. Let

$$\phi \left(\beta \right)={(\sum _{j\ne 1}{t}_{ij}{\mu }_{1j}\left(\beta \right),\dots ,\sum _{j\ne n}{t}_{nj}{\mu }_{nj}\left(\beta \right))}^{\top }.$$

If $\phi \left(\beta \right):{\mathbb{R}}^n\to (0,\infty )$ is a one-to-one mapping, then $\widehat{\beta }$ exists and is unique, i.e., $\widehat{\beta }={\phi }^{-1}\left(a\right)$. When ${\phi }^{-1}$ does not exist (i.e., $\phi$ is not one-to-one), any solution $\widehat{\beta }$ of Equation (2) is a moment estimator of $\beta$. The Newton–Raphson algorithm can be used to solve Equation (2). Moreover, the R language provides the package “BradleyTerry2” to solve the estimator in the Bradley–Terry model.

We discuss the existence of $\widehat{\beta }$ from the viewpoint of graph connection. If the comparison graph with the matrix ${\left(t_{ij}\right)}_{i,j=0,\dots ,n}$ as its adjacency matrix is not connected, then there are two empty sets such that there are no comparisons between subjects in the first set and those in the second. In this case, there is no basis for ranking subjects in the first set and those in the second set. Further, a necessary condition for the existence of $\widehat{\beta }$ is that the directed graph ${\mathcal{G}}_n$ with the win–loss matrix $A=\left(a_{ij}\right)$ as its adjacency matrix is strongly connected. In other words, for every partition of the subjects into two nonempty sets, a subject in the second set beats a subject in the first at least once. To see this, assume that there are two empty sets $B_1$ and $B_2$ such that all subjects in $B_1$ win all comparisons with subjects in $B_2$. Without loss of generality, we set $B_1=\{0,\dots ,m\}$ and $B_2=\{m+1,\dots ,n\}$ with $0\le m<n$, where $a_{ij}=t_{ij}$ for $i\in B_1$ and $j\in B_2$. By summing $a_i$ over $i=0,\dots ,m$, we have

$$\sum _{i=0}^m{a}_i=\sum _{i=0}^m\sum _{j=0}^m{t}_{ij}\mu ({\beta }_i-{\beta }_j)+\sum _{i=0}^m\sum _{j=m+1}^n{t}_{ij}\mu ({\beta }_i-{\beta }_j).$$

Because $a_i$ is a sum of $a_{ij}$, $j=0,\dots ,n$, and $\mu ({\beta }_i-{\beta }_j)+\mu ({\beta }_j-{\beta }_i)=1$, we have

$$\sum _{i=0}^m\sum _{j=m+1}^n{a}_{ij}=\sum _{i=0}^m\sum _{j=m+1}^n{t}_{ij}\mu ({\beta }_i-{\beta }_j).$$

Because $a_{ij}=t_{ij}$ for $i=0,\dots ,m$ and $j=m+1,\dots ,n$ and at least such one $t_{ij}>0$, it must be $\mu ({\beta }_i-{\beta }_j)=1$ when $t_{ij}>0$ in order to guarantee both sides in the above equation to be equal. In this case, at least one such difference ${\beta }_i-{\beta }_j$ must go to infinity such that the moment estimate does not exist. The strong connection of ${\mathcal{G}}_n$ is also sufficient for guaranteeing the existence of the MLE in the Bradley–Terry model [19] in which the moment estimator is equal to the MLE. It is interesting to see whether the strong connection of ${\mathcal{G}}_n$ is sufficient to guarantee the existence of $\widehat{\beta }$ in a general model. In the next section, we will show that $\widehat{\beta }$ exists with probability approaching one under some mild conditions.

3. Asymptotic Properties

In this section, we present the consistency and asymptotic normality of the moment estimator. We first introduce some notations. For a subset $C\subset {\mathbb{R}}^n$, let $C^0$ and $\overline{C}$ denote the interior and closure of C, respectively. For a vector $x={(x_1,\dots ,x_n)}^{\top }\in {\mathbb{R}}^n$, denote $\parallel x\parallel$ by a vector norm with the ${\ell }_{\infty }$-norm, ${\parallel x\parallel }_{\infty }={max}_{1\le i\le n}|x_i|$, and the ${\ell }_1$-norm, ${\parallel x\parallel }_1={\sum }_i|x_i|$. Let $B(x,ϵ)=\{y:\parallel x-y{\parallel }_{\infty }\le ϵ\}$ be an $ϵ$-neighborhood of x. For an $n\times n$ matrix $J=\left(J_{ij}\right)$, let ${\parallel J\parallel }_{\infty }$ denote the matrix norm induced by the ${\ell }_{\infty }$-norm on vectors in ${\mathbb{R}}^n$, i.e.,

$${\parallel J\parallel }_{\infty }=\underset{x\ne 0}{max}{\displaystyle \frac{{\parallel Jx\parallel }_{\infty }}{{\parallel x\parallel }_{\infty }}}=\underset{1\le i\le n}{max}\sum _{j=1}^n|J_{ij}|,$$

and let $\parallel J\parallel$ be a general matrix norm. Define the matrix maximum norm: ${\parallel J\parallel }_{max}={max}_{i,j}|J_{ij}|$. We use the superscript “*” to denote the true parameter under which the data are generated. When there is no ambiguity, we omit the superscript “*”.

Recall that $\mu (·)$ is the expectation of $F(·)$. We assume that $\mu (·)$ is a continuous function with the third derivative. Write ${\mu }^{\prime }$ and ${\mu }^{″}$ as the first and second derivatives of $\mu \left(\pi \right)$ on $\pi$, respectively. Let ${ϵ}_n$ be a small positive number. When $\beta \in B({\beta }^{\ast },{ϵ}_n)$, we assume that there are three positive numbers, $b_{n0},b_{n1},b_{n2}$, such that

$$\begin{array}{c}[\underset{i,j}{min}{\mu }^{\prime }\left({\pi }_{ij}\right)]·[\underset{i,j}{max}{\mu }^{\prime }\left({\pi }_{ij}\right)]>0,\end{array}$$

(3a)

$$\begin{array}{c}{b}_{n0}\le \underset{i,j}{min}|{\mu }^{\prime }\left({\pi }_{ij}\right)|\le \underset{i,j}{max}|{\mu }^{\prime }\left({\pi }_{ij}\right)|\le b_{n1},\end{array}$$

(3b)

$$\begin{array}{c}\underset{i,j}{max}|{\mu }^{″}\left({\pi }_{ij}\right)|\le b_{n2},\end{array}$$

(3c)

where ${\pi }_{ij}:={\beta }_i-{\beta }_j$.

We use the Bradley–Terry model to illustrate the above inequalities, where $\mu \left(x\right)=e^x/(1+e^x)$. A direct calculation gives that

$${\mu }^{\prime }\left(x\right)={\displaystyle \frac{e^x}{{(1+e^x)}^2}},{\mu }^{″}\left(x\right)={\displaystyle \frac{e^x(1-e^x)}{{(1+e^x)}^3}}.$$

It is easy to show that

$$b_{n0}={\displaystyle \frac{e^{2\parallel {\beta }^{\ast }{\parallel }_{\infty }+2{ϵ}_n}}{{(1+e^{2\parallel {\beta }^{\ast }{\parallel }_{\infty }+2{ϵ}_n})}^2}}\le |{\mu }^{\prime }\left(x\right)|\le b_{n1}={\displaystyle \frac{1}{4}},|{\mu }^{″}\left(x\right)|\le b_{n2}={\displaystyle \frac{1}{4}}.$$

(4)

If ${ϵ}_n=o\left(1\right)$, then $1/b_{n0}=O\left(e^{2\parallel {\beta }^{\ast }{\parallel }_{\infty }}\right)$.

3.1. Consistency

To establish the consistency of $\widehat{\beta }$, let us first define a system of functions:

$$H_i\left(\beta \right)=\sum _{j=0}^n{t}_{ij}{\mu }_{ij}\left(\beta \right)-a_i,i=0,\dots ,n,$$

(5)

and $H\left(\beta \right)={(H_1\left(\beta \right),\dots ,H_n\left(\beta \right))}^{\top }$. It is clear that $H(\widehat{\beta })=0$. Let $H^{\prime }\left(\beta \right)$ be the Jacobian matrix of $H\left(\beta \right)$ on the parameter $\beta$. The asymptotic behavior of $\widehat{\beta }$ depends crucially on the inverse of $H^{\prime }\left(\beta \right)$. For convenience, denote $H^{\prime }\left(\beta \right)$ as $V={\left(v_{ij}\right)}_{i,j=1,\dots ,n}$, where

$$v_{ij}=-t_{ij}{\mu }^{\prime }\left({\pi }_{ij}\right),i\ne j,v_{ii}=\sum _{j=0,j\ne i}^n{t}_{ij}{\mu }^{\prime }\left({\pi }_{ij}\right).$$

Define

$$v_{i0}=v_{0i}:=\sum _{j=0,j\ne i}^n{v}_{ij}-v_{ii}=-t_{0i}{\mu }^{\prime }\left({\pi }_{0i}\right),i=1,\dots ,n,v_{00}=\sum _{j=1}^n{t}_{0j}{\mu }^{\prime }\left({\pi }_{0j}\right).$$

When $\beta \in B({\beta }^{\ast },{ϵ}_n)$ and ${min}_{i,j}{\mu }^{\prime }\left({\pi }_{ij}\right)>0$, in view of inequality (3b), the entries of V satisfy the following inequalities:

$$\begin{array}{cc}\hfill \mathrm{if}{t}_{i0}>0,& t_{i0}{b}_{n0}\le v_{ii}+\sum _{j=1,j\ne i}^n{v}_{ij}\le t_{i0}{b}_{n1},i=1,\dots ,n,\hfill \\ \hfill \mathrm{if}{t}_{ij}>0,& t_{ij}{b}_{n0}\le -v_{ij}\le t_{ij}{b}_{n1},i,j=1,\dots ,n;i\ne j.\hfill \end{array}$$

(6)

Without loss of generality, we assume that ${min}_{i\ne j}{\mu }^{\prime }\left({\pi }_{ij}\right)>0$ when $\beta \in B({\beta }^{\ast },{ϵ}_n)$ hereafter (otherwise, we redefine $H_i\left(\beta \right)=a_i-{\sum }_{j\ne i}{\mu }_{ij}\left(\beta \right)$ and repeat a similar process). Our strategy for the proof of consistency crucially depends on the existence of the inverse of V, which requires that V is a full rank matrix. It is easy to show that V is positively semi-definite. Thus, if V has a full rank, then V must be positively definite. The following lemma assures the existence of the inverse of V.

Lemma 1.

Assume that ${min}_{i,j}{\mu }_{ij}^{\prime }\left(\beta \right)>0$. With probability at least $1-{(1-p_n)}^{nT}$, $H^{\prime }\left(\beta \right)$ is positively definite.

Because $\log (1-x)\le -x$ when $x\in (0,1)$, we have

$$e^{nT\log (1-p_n)}\le e^{-p_{n}Tn}.$$

The probability of the nonexistence of $V^{-1}$ is less than $e^{-p_{n}Tn}$, going exponentially fast to zero. Generally, the inverse of V does not have a closed form. Ref. [20] proposed to approximate the inverse of V, $V^{-1}$, by the matrix $S={\left(s_{ij}\right)}_{n\times n}$, where

$$s_{ij}={\displaystyle \frac{{\delta }_{ij}}{v_{ii}}}+{\displaystyle \frac{1}{v_{00}}}.$$

(7)

In the above equation, ${\delta }_{ij}=1$ if $i=j$; otherwise, ${\delta }_{ij}=0$. By extending the proof of [20] to the sparse case, the upper bound of the approximate error $\parallel V^{-1}{-S\parallel }_{max}$ is given in Lemma A2.

Recall that the main idea of the proof of the consistency in the Bradley–Terry model [10,14] contains two parts. Let ${\widehat{u}}_i=e^{{\widehat{\beta }}_i}$, $u_i=e^{{\beta }_i}$, $i_0=arg{max}_i{\widehat{u}}_i/u_i$ and $i_1=arg{min}_i{\widehat{u}}_i/u_i$. Since ${\widehat{u}}_0/u_0=1$, it suffices to show that the ratio of subject $i_0$, ${\widehat{u}}_{i_0}/u_{i_0}$, and the ratio of $i_1$, ${\widehat{u}}_{i_1}/u_{i_1}$ are very close. With the nice mathematical properties of the logistic function $\mu \left(x\right)=e^x/(1+e^x)$, the first part is to show that there are a number of subjects satisfying the following inequalities:

$$b\sum _{\{j:t_{i_0,j}>0\}}({\widehat{u}}_{i_0}/u_{i_0}-{\widehat{u}}_j/u_j)\le c,b\sum _{\{j:t_{i_1,j}>0\}}({\widehat{u}}_j/u_j-{\widehat{u}}_{i_0}/u_{i_0})\le c,$$

(8)

where b and c are certain numbers. The second part is to eliminate common terms ${\widehat{u}}_j/u_j$ based on the condition that the number of the common neighbors between any two subjects, ${min}_{i,j}\#\{k:t_{ik}>0,t_{jk}>0\}$, is at least $\tau n$, where $\tau =1$ in [10] and $\tau \in (0,1)$ in [14]. In the Erdös–Rényi comparison graph, [13] further showed that there is at least one subject with its ratio close to both ${\widehat{u}}_{i_0}/u_{i_0}$ and ${\widehat{u}}_{i_1}/u_{i_1}$.

The aforementioned strategies for the proof of consistency are built on the the premise of the existence of the MLE, which is guaranteed by the necessary and sufficient condition that the directed graph with the win–loss matrix as its adjacency matrix is strongly connected [19]. As discussed before, it may be difficult to find the minimal sufficient condition to guarantee the existence of $\widehat{\beta }$ in general paired comparison models. To overcome this difficulty, we aim to obtain the convergence rate of the Newton iterative sequence for solving Equation (2). Under the well-known Newton–Kantorich conditions, the Newton iterative sequence converges, and its limiting point is the solution. We apply an adjusted version of the Newton–Kantorich theorem in [21] to this end, which not only guarantees the existence of the solution but also gives an optimal error bound for the Newton iterative sequence.

Now, we formally state the consistency result.

Theorem 1.

Assume that Conditions (3a), (3b) and (3c) hold. If $b_{n1}^4{b}_{n2}/\left(b_{n0}^6{p}_n^4\right)=o\left({(n/\log n)}^{1/2}\right)$, then $\widehat{\beta }$ exists with probability approaching one and is uniformly consistent in the sense that

$$\parallel \widehat{\beta }-{\beta }^{\ast }{\parallel }_{\infty }=O_p\left({\displaystyle \frac{b_{n1}^2}{b_{n0}^3{p}_n^2}}\sqrt{{\displaystyle \frac{\log n}{n}}}\right)=o_p\left(1\right).$$

(9)

To see how small $p_n$ could be, we consider a special case that ${\beta }^{\ast }$ is a constant vector, in which $b_{n0},b_{n1}$ and $b_{n2}$ are also constants. According to the above theorem, if $p_n>O\left({(\log n/n)}^{1/8}\right)$, then $\parallel \widehat{\beta }-{\beta }^{\ast }{\parallel }_{\infty }=O_p\left(p_n^{-2}{(\log n/n)}^{1/2}\right)$.

3.2. Asymptotic Normality of $\widehat{\beta }$

We establish the asymptotic distribution of $\widehat{\beta }$ by characterizing its asymptotical representation. In detail, we apply a second-order Taylor expansion to $H(\widehat{\beta })$ and find that $\widehat{\beta }-{\beta }^{\ast }$ can be represented as the sum of a main term $V^{-1}(a-{\mathbb{E}}^{\ast }a)$ and an asymptotically neglected remainder term, where ${\mathbb{E}}^{\ast }$ denotes the conditional expectation conditional on $\{t_{ij}:i,j=0,\dots ,n\}$. Because $V^{-1}$ does not have a closed form, we use the matrix S defined in (7) to approximate it. We formally state the asymptotic normality of $\widehat{\beta }$ as follows.

Theorem 2.

Let $V=\partial H\left({\beta }^{\ast }\right)/\partial \beta$ and $U=\left(u_{ij}\right):=\mathrm{Var}(a|t_{ij},0\le i,j\le n)$. If $b_{n2}{b}_{n1}^6{b}_{n0}^{-9}{p}_n^{-6}=o(n^{1/2}/\log n)$, then for fixed k, the vector $(({\widehat{\beta }}_1-{\beta }^{\ast }),\dots ,({\widehat{\beta }}_k-{\beta }_k^{\ast }))$ follows a k-dimensional multivariate normal distribution with mean zero and the covariance matrix $\mathrm{\Sigma }={\left({\sigma }_{ij}\right)}_{k\times k}$, where

$${\sigma }_{ij}={\displaystyle \frac{{\delta }_{ij}{u}_{ii}}{v_{ii}^2}}+{\displaystyle \frac{u_{00}}{v_{00}^2}}.$$

(10)

Remark 1.

If $U=V$, then ${\sigma }_{ij}$ is equal to ${\delta }_{ij}/v_{ii}+1/v_{00}$. When $F(·)$ belongs to the exponential family distribution (e.g., the Bradley–Terry model), U is identical to V. If $U\ne V$, then the asymptotic variance of ${\widehat{\beta }}_i$ is involved with an additional factor $u_{ii}$. The asymptotic variance of ${\widehat{\beta }}_i$ is on the order of ${\left(n{p}_n\right)}^{-1/2}$ if ${\beta }^{\ast }$ is bounded above by a constant.

4. Application to the Thurston Model

In this section, we illustrate the unified theoretical result by the application to the Thurston model.

The original Thurston model has a variance ${\sigma }^2$ in the normal distribution, i.e., the probability that subject i is preferred over j is $F(({\beta }_i-{\beta }_j)/\sigma )$. Since the merit parameters are scale invariable, we simply set $\sigma =1$ hereafter. Recall that $\varphi \left(x\right)={\left(2\pi \right)}^{1/2}{e}^{-x^2/2}$ is the standard normal density function and $\mathrm{\Phi }\left(x\right)={\int }_{-\infty }^x\varphi \left(x\right)dx$ is the distribution function of the standard normality. In the Thurston model, $\mu \left(x\right)=\mathrm{\Phi }\left(x\right)$. Then,

$${\mu }^{\prime }\left(x\right)=\varphi \left(x\right),{\mu }^{″}\left(x\right)={\displaystyle \frac{x}{\sqrt{2\pi }}}{e}^{-x^2/2}.$$

Since $\varphi \left(x\right)={\left(2\pi \right)}^{1/2}{e}^{-x^2/2}$ is an decreasing function on $|x|$, we have when $|x|\le Q_n$,

$${\displaystyle \frac{1}{2\pi }}{e}^{-Q_n^2/2}\le \varphi \left(x\right)\le {\displaystyle \frac{1}{2\pi }}.$$

Let $h\left(x\right)=x{e}^{-x^2/2}$. Then, $h^{\prime }\left(x\right)=(1-x^2)e^{-x^2/2}$. Therefore, when $x\in (0,1)$, $h\left(x\right)$ is an increasing function on its argument x; when $x\in (1,\infty )$, $h\left(x\right)$ is an decreasing function on x. As a result, $h\left(x\right)$ attains its maximum value at $x=1$ when $x>0$. Since $h\left(x\right)$ is a symmetric function, we have $|h\left(x\right)|\le e^{-1/2}\approx 0.6$. Therefore,

$$b_{n0}={\displaystyle \frac{1}{2\pi }}{e}^{-(\parallel {\beta }^{\ast }{{\parallel }_{\infty }+{ϵ}_n)}^2/2},b_{n1}={\displaystyle \frac{1}{2\pi }},b_{n2}={\left(2\pi e\right)}^{-1/2}.$$

In view of Theorems 1 and 2, we have the following corollary.

Corollary 1.

If $b_{n1}^4{b}_{n2}/\left(b_{n0}^6{p}_n^2\right)=o\left({(n/\log n)}^{1/2}\right)$ and $p_n>24\log n/n$, then $\widehat{\beta }$ exists with probability approaching one and is uniformly consistent in the sense that

$$\parallel \widehat{\beta }-{\beta }^{\ast }{\parallel }_{\infty }=O_p\left({\displaystyle \frac{b_{n1}^2}{b_{n0}^3{p}_n}}\sqrt{{\displaystyle \frac{\log n}{n}}}\right)=o_p\left(1\right).$$

(11)

Let $V=\partial H\left({\beta }^{\ast }\right)/\partial \beta$. If $b_{n2}{b}_{n1}^6{b}_{n0}^{-9}=o(n^{1/2}/\log n)$, then for fixed k, the vector $(({\widehat{\beta }}_1-{\beta }^{\ast }),\dots ,({\widehat{\beta }}_k-{\beta }_k^{\ast }))$ follows a k-dimensional multivariate normal distribution with mean zero and the covariance matrix $\mathsf{\Sigma }={\left({\sigma }_{ij}\right)}_{k\times k}$ defined at (10).

5. Extension to a Fixed Sparse Design

In some applications such as sports, the comparison graph may be fixed, not random. For example, in the regular season of the National Football League (NFL), games are scheduled in advance. More specially, there are 32 teams in the 2 conferences of the NFL that are divided into 8 divisions, each consisting of 4 teams. In the regular season, each team plays 16 matches, 6 within the division and 10 between the divisions. Motivated by the design, ref. [14] proposed a sparse condition to control the length from one subject to another subject with 2 or 3:

$${\tau }_n:=\underset{0\le i<j\le n}{min}{\displaystyle \frac{\#\{k:t_{ik}>0,t_{jk}>0\}}{t}}.$$

That is, ${\tau }_n$ is the minimum ratio of the total number of paths between any i and j with length 2 or 3. Under the Erdös–Rényi comparison graph, there are similar sparsities. Specifically, the set of common neighbors of any two subjects i and j has at least the following size:

$$\#\{k:t_{ik}>0,t_{jk}>0\}\ge \frac{1}{2}(n-1)p_n^2,$$

with a probability of at least $1-O(1/n)$ if $n{p}_n^2\ge 24\log n$; see (A12) in the proof of A2.

We assume that if two subjects have comparisons, they are compared T times, in accordance with the aforementioned setting for easy of exposition. Similar to Lemma A2, the approximate error of using S to approximate $V^{-1}$ is

$$\parallel V^{-1}{-S\parallel }_{max}\le {\displaystyle \frac{2T^2{b}_{n1}^2{\rho }_{max}}{b_{n0}^3{\tau }_n^3{n}^2}},$$

where ${\rho }_{max}=t_{max}/n$ and $t_{max}={max}_i{t}_i$. With similar lines of argument as in the proofs of Theorems 1 and 2, we have the following theorem, whose proof is omitted.

Theorem 3.

Assume that conditions (3a), (3b) and (3c) hold. If $b_{n1}^4{b}_{n2}/\left(b_{n0}^6{\tau }_n^2\right)=o\left({(n/\log n)}^{1/2}\right)$, then $\widehat{\beta }$ exists with probability approaching one and is uniformly consistent in the sense that

$$\parallel \widehat{\beta }-{\beta }^{\ast }{\parallel }_{\infty }=O_p\left({\displaystyle \frac{b_{n1}^2}{b_{n0}^3{\tau }_n}}\sqrt{{\displaystyle \frac{\log n}{n}}}\right)=o_p\left(1\right).$$

(12)

Let $V=\partial H\left({\beta }^{\ast }\right)/\partial \beta$ and $U=\left(u_{ij}\right):=\mathrm{Var}(a|t_{ij},0\le i,j\le n)$. If $b_{n2}{b}_{n1}^6{b}_{n0}^{-9}{\tau }_n^{-3}=o(n^{1/2}/\log n)$, then for fixed k, the vector $(({\widehat{\beta }}_1-{\beta }^{\ast }),\dots ,({\widehat{\beta }}_k-{\beta }_k^{\ast }))$ follows a k-dimensional multivariate normal distribution with mean zero and the covariance matrix $\mathrm{\Sigma }={\left({\sigma }_{ij}\right)}_{k\times k}$, where ${\sigma }_{ij}$ is given in (7).

6. Numerical Studies

In this section, we evaluate the asymptotic results of the moment estimator in the Thurston model through simulation studies and a real data example.

6.1. Simulation Studies

We carry out simulations to evaluate the finite sample performance of the moment estimator in the Thurston model. We set $T=1$, which means that any pair has one comparison with probability $p_n$ and no comparison with probability $1-p_n$. Let c be a constant. We set the merit parameters to be a linear form, i.e., ${\beta }_i^{\ast }=ic\log n/n$ for $i=1,\dots ,n$, where ${\beta }_0^{\ast }=0$. We considered four different values for c as $c=0.3,0.5,0.8$. By allowing $\parallel {\beta }^{\ast }{\parallel }_{\infty }$ to grow with n, we intended to assess the asymptotic properties under different asymptotic regimes.

In order to see how small $p_n$ could be, we first evaluate the fail frequency that the “win–loss” graph $G_n$ is strongly connected, which is the necessary condition to guarantee the moment estimator exists. We set c to be fixed with $c=0.4$. The results are shown in

Table 1. The fail frequency ($\times 100\%$).

${\mathit{p}}_{\mathit{n}}$	$\mathit{n}=100$	$\mathit{n}=500$	$\mathit{n}=1000$
${(\log n/n)}^{1/2}$	$0.5$	0	0
${(\log n/n)}^{2/3}$	$25.8$	$0.2$	0
$\log n/n$	100	100	100

with 1000. We can see that the necessary condition did not hold in each simulation when $p_n=\log n/n$, while it holds with almost $100\%$ frequency when $p_n={(\log n/n)}^{1/2}$. This shows that it is necessary to control the rate of $p_n$ tending to zero.

Based on Theorem 2, ${\widehat{\xi }}_{ij}=[{\widehat{\beta }}_i-{\widehat{\beta }}_j-({\beta }_i^{\ast }-{\beta }_j^{\ast })]/{({\widehat{u}}_{ii}/{\widehat{v}}_{ii}^2+{\widehat{u}}_{00}/{\widehat{v}}_{00}^2)}^{1/2}$ converges in distribution to the standard normality, where ${\widehat{u}}_{ii}$ and ${\widehat{v}}_{i,i}$ are the estimates of $u_{ii}$ and $v_{ii}$ by replacing ${\beta }^{\ast }$ with $\widehat{\beta }$. Therefore, we assessed the asymptotic normality of ${\widehat{\xi }}_{ij}$ via the coverage probability of the $95\%$ confidence interval and the length of the confidence interval. The times that $\widehat{\beta }$ failed to exist were also recorded. Two values, $n=100$ and $n=200$, were considered for the number of subjects. Each simulation was repeated 10,000 times.

The simulation results are shown in

Table 2. The reported values are the coverage frequency ($\times 100\%$) for ${\beta }_i-{\beta }_j$ for a pair $(i,j)$/length of the confidence interval/fail probabilities ($\times 100\%$).

${\mathit{p}}_{\mathit{n}}={(\log \mathit{n}/\mathit{n})}^{1/4}$
n	$\mathit{i}$	$\mathit{c}=\mathbf{0.2}$	$\mathit{c}=\mathbf{0.5}$	$\mathit{c}=\mathbf{0.8}$
100	$(1,2)$	$94.5/1.04/0$	$94.62/1.06/0$	$94.62/1.06/0$
	$(49,50)$	$94.97/1.04/0$	$94.34/1.04/0$	$94.34/1.04/0$
	$(99,100)$	$95.12/1.04/0$	$94.99/1.06/0$	$94.99/1.06/0$
200	$(1,2)$	$95.18/0.78/0$	$94.71/0.79/0$	$94.71/0.79/0$
	$(99,100)$	$95.14/0.78/0$	$94.49/0.78/0$	$94.49/0.78/0$
	$(199,200)$	$94.82/0.78/0$	$94.64/0.79/0$	$94.64/0.79/0$
${\mathit{p}}_{\mathit{n}}={(\mathbf{log}\mathit{n}/\mathit{n})}^{\mathbf{1}/\mathbf{2}}$
n	$\mathit{i}$	$\mathit{c}=\mathbf{0.2}$	$\mathit{c}=\mathbf{0.4}$	$\mathit{c}=\mathbf{0.6}$
100	$(1,2)$	$93.28/1.58/0.19$	$93.48/1.60/0.49$	$93.48/1.60/1.15$
	$(49,50)$	$93.85/1.58/0.19$	$93.94/1.58/0.49$	$93.94/1.58/1.15$
	$(99,100)$	$93.80/1.58/0.19$	$93.96/1.61/0.49$	$93.96/1.61/1.15$
200	$(1,2)$	$94.04/1.26/0$	$93.89/1.28/0$	$93.89/1.28/0$
	$(99,100)$	$94.14/1.26/0$	$94.22/1.26/0$	$94.22/1.26/0$
	$(199,200)$	$94.38/1.26/0$	$93.98/1.28/0$	$93.98/1.28/0$

. When $p_n={(\log n/n)}^{1/4}$, all simulated coverage probabilities are very close to the target level $95\%$. On the other hand, when $p_n={(\log n/n)}^{1/2}$, they are a little lower than the normal level in the case $n=100$ and are very close to $95\%$ in the case $n=200$. The length of the confidence interval decreases as n increases, which qualitatively agrees with the theory. It is also expected that the length of the confidence interval increases as $p_n$ decreases when n is fixed. Another phenomenon is that the length of the confidence interval under three distinct c has little difference when n and $p_n$ are fixed.

6.2. A Real Data Example

We use the 2018 NFL regular season data as an illustrative example, which is available from https://www.espn.com/nfl/schedule/_/year/2018 (accessed on 20 August 2025). The NFL league consists of thirty-two teams that are divided evenly into two conferences, and each conference has four divisions that have four teams each. In the regular season, each team plays with three intra-division teams (each twice) and ten games with ten inter-division teams (each once). As discussed in [14], the design of the NFL regular season satisfies the sparsity condition of the fixed comparison graph, where ${\tau }_n=1/16$. We removed two ties before our analysis. The fitted merits that were obtained from fitting the Thurstone model for the remaining data are given in

Table 3. The fitted merit ${\widehat{\beta }}_i$, the number of wins $a_i$, and the standard error ${\widehat{\sigma }}_i$.

American Football Conference					National Football Conference
Division	Team	${\widehat{\mathit{\beta }}}_{\mathit{i}}$	${\mathit{a}}_{\mathit{i}}$	${\widehat{\mathit{\sigma }}}_{\mathit{i}}$	Division	Team	${\widehat{\mathit{\beta }}}_{\mathit{i}}$	${\mathit{a}}_{\mathit{i}}$	${\widehat{\mathit{\sigma }}}_{\mathit{i}}$
East	New England Patriots	$1.452$	11	$0.519$	East	Dallas Cow boys	$1.284$	10	$0.512$
	New York Jets	$0.338$	4	$0.530$		Philadelphia Eagles	$1.193$	9	$0.511$
	Miami Dophins	$0.718$	7	$0.509$		New York Giants	$0.423$	5	$0.514$
	Buffalo Bills	$0.673$	6	$0.514$		Washington Redskins	$0.756$	7	$0.511$
North	Baltimore Ravens	$1.382$	10	$0.514$	North	Chicago Bears	$1.494$	12	$0.532$
	Cincinnati Bengals	$0.769$	6	$0.514$		Green Bay Packers	$0.595$	6	$0.522$
	Pittsburgh Steelers	$1.337$	9	$0.52$		Minnesota Vikings	$1.059$	8	$0.523$
	Cleveland Browns	$0.968$	7	$0.519$		Detroit Lions	$0.579$	6	$0.516$
South	Indianapolis Colts	$1.244$	10	$0.512$	South	New Orleans Saints	$1.908$	13	$0.544$
	Houston Texans	$1.430$	11	$0.516$		Atlanta Falcons	$0.767$	7	$0.512$
	Tennessee Titans	$1.205$	9	$0.51$		Carolina Panthers	$0.840$	7	$0.511$
	Jacksonville Jaguars	$0.591$	5	$0.517$		Tampa Bay Buccaneers	$0.506$	5	$0.52$
West	Kansas City Chiefs	$1.762$	12	$0.537$	West	Los Angeles Rams	$1.963$	13	$0.559$
	Denver Broncos	$0.713$	6	$0.523$		San Francisco 49ers	$0.166$	4	$0.54$
	Oakland Raiders	$0.365$	4	$0.537$		Seattle Seahawks	$1.305$	10	$0.525$
	Los Angeles Chargers	$1.748$	12	$0.536$		Arizona Cardinals	0	3	$0.555$

, where we used “Arizona Cardinals” with the smallest number of wins as the baseline (with ${\widehat{\beta }}_0=0$).

It is interesting to compare the ordering of six playoff seeds of the two conferences with the NFL rules with the ordering by their fitted merits in Table 3. The NFL rules are based on the regular season won–lost percentage record (PCT) and can be briefly summarized as follows: the teams in each division with the best PCT are seeded one through four; another two teams from each conference are seeded five and six based on their PCT. The six playoff seeds in the American Football Conference from No. 1 to No. 6 based on the PCT are Kansas City Chiefs, New England Patriots, Pittsburgh Steelers, Houston Texans, Los Angeles Chargers, and Indianapolis Colts, while the selected teams based on the fitted merits are Kansas City Chiefs, New England Patriots, Houston Texans, Baltimore Ravens, Los Angeles Chargers, and Pittsburgh Steelers. The corresponding six playoff seeds in the National Football Conference based on the PCT are Los Angeles Rams, New Orleans Saints, Chicago Bears, Washington Redskins, Seattle Seahawks, and Carolina Panthers, while the selected teams base on the fitted merits are Los Angeles Rams, New Orleans Saints, Chicago Bears, Dallas Cow boys, Seattle Seahawks, and Philadelphia Eagles. As we can see, the selected top three teams based on the PCT and the fitted merits in each conference are the same, and the selected teams from No. 4, No. 5 and No. 6 are not all the same.

7. Summary and Discussion

We have presented the moment estimation based on the scores of subjects in the paired comparison model under sparse comparison graphs. We have established the uniform consistency and asymptotic normality of the moment estimator. The consistency is shown by obtaining the convergence rate of the Newton iterative sequence. This leads to a condition on the sparsity parameter $p_n$ requiring that $p_n\ge O\left({(\log n/n)}^{1/8}\right)$ if ${\beta }^{\ast }$ is a constant vector. We note that this condition looks much stronger than that in the Bradley–Terry model in [13]. Since we consider a general model, it would seem to be suitable that a more severe condition is imposed. On the other hand, the condition imposed on $b_{n0}$ may not be the best possible condition. In particular, the conditions for guaranteeing the asymptotic normality seem stronger than those needed for the consistency. Note that the asymptotic behavior of the moment estimator depends not only on $b_{n0}$ but also on the configuration of all parameters. It would be of interest to investigate whether these conditions could be relaxed.

In this paper, we assume that given the comparison graph, all paired comparisons are independent. Note that the moment equation holds regardless of whether comparisons are independent.

When comparisons are not independent, the moment estimation still works. The consistency result in Theorem 1 still holds as long as there is the same order of the upper bound of $a_i-{\mathbb{E}}^{\ast }{a}_i$ in Lemma A4. In fact, the independence assumption is not directly used when checking our proofs. It is only used in Lemma A4 to derive the upper bound of $a_i-{\mathbb{E}}^{\ast }{a}_i$ using the Hoeffding inequality. Analogously, the independence assumption is used to derive the central limit theorem of $a_i-{\mathbb{E}}^{\ast }{a}_i$. In the dependence case, there are also many Hoeffding-type exponential tail inequalities (e.g., [22,23,24]) and cental limit theorems for sums of a sequence of random variables (e.g., [25,26]) to apply.

Building on the theoretical and empirical findings of this study, we identify two promising avenues for further exploration, which not only address the current limitations but also extend the moment estimation framework to more complex and practical scenarios: 1. Moment estimation for paired comparison models with dependent outcomes. This paper assumes that all paired comparison outcomes are independent, which is a standard but restrictive assumption in many real-world settings (e.g., crowdsourcing evaluations where raters may have consistent biases, or sports leagues where team performance is serially correlated). A natural extension is to develop moment estimation methods for models with dependent outcomes, such as incorporating Markovian dependence or exchangeable correlation structures. Key challenges include deriving unbiased moment equations under dependence and establishing asymptotic properties (consistency, asymptotic normality) using tools from dependent random variable theory (e.g., Hoeffding-type inequalities for associated variables [22,23,24]). 2. High-dimensional sparse paired comparison models with structured merit parameters. This study focuses on unstructured merit parameters (i.e., ${\beta }_i$ are independent). However, in many applications, merit parameters often exhibit inherent structures, such as group-level homogeneity (e.g., teams in the same sports division share similar strengths) or sparsity (e.g., only a few subjects have distinct merits in large-scale crowdsourcing). Extending the moment estimation framework to incorporate such structures (e.g., group lasso-penalized moment estimation, sparse merit parameter inference) would improve estimation efficiency. Key research questions include designing computationally feasible penalized moment equations and establishing oracle properties for the structured estimators. These directions align with the core theme of sparse paired comparison inference and address practical limitations of the current work. We believe pursuing these avenues will not only extend the theoretical scope of moment estimation but also broaden its applicability to more complex real-world problems.