Estimating Common Mean in Heteroscedastic Variances Model

Abstract

Bayes estimators for the unknown mean against a reference, non-informative prior distribution for both the mean and independent variances are derived. I entertain the scenario with two groups of observables with the same unknown mean. The unknown variances of the the first group are not supposed to be equal or to be restricted; the second homeogeneous group of observations all have the same unknown variance. Under the normality condition, these procedures turn out to have a very explicit form of the weighted average with data-dependent weights that admit of a very clear interpretation. The approximate formulas for the variance of the considered estimators and their limiting behavior are also examined. The related “self-dual” orthogonal polynomials and their properties are examined. Recursive formulas for estimators on the basis of these polynomials are developed.

1. Introduction: Missing Uncertainties

Assume that the available data consist of a sequence of independent observations, $x_j,j=1,\dots ,n$, each having the same mean. We consider the situation where the unknown variances of the observables cannot be supposed to be equal or to be restricted in any way. This scenario is perhaps unusual in the statistical community. However, according to Morris (1983, p. 49) [1], “… almost all applications involve unequal variances”. The problem is to estimate the common mean modeled as a location parameter without traditional conditions on the standard deviations (scale parameters).

This setting appears in instances of heterogeneous research synthesis where $x_j$ represents the summary estimate of the common mean (e.g., the treatment effect) obtained by the j-th study. Commonly, the protocol of such studies demands that $x_j$ be accompanied by its uncertainty estimate, but sometimes these estimates are either unavailable or cannot be trusted. In many applications, the variances of systematic, laboratory-specific errors cannot be reliably estimated; a scientist cannot place confidence in inferences made under unrealistically low noise. The issue of underreported uncertainties, particularly those that stem from asymptotic normal theory, which presupposes large datasets, is prevalent in many applications. The existing imputation techniques (e.g., Rukhin [2], Templ [3]) may not provide justifiable uncertainty values.

The latest point of view (Spiegelhalter [4]) is that uncertainty is a subjective relationship between an observer and what is observed. The issue of underreported uncertainties, particularly those that stem from asymptotic normal theory, which presupposes large datasets, is prevalent in metrology. The challenge of reproducibility within individual centers may be exacerbated by the nature of the measuring instruments employed, leading to heterogeneous unknown uncertainties (see Possolo [5]). The most striking example is provided by “one-shot” devices in the atomic industry, which are limited to single use.

An additional contemporary illustration is found in citizen science or crowd-sourcing projects, where participants contribute measurement results of various random phenomena, with some of them using relatively imprecise instruments, such as smartphones. These measurements can range from precipitation levels to air quality and biological observations. See Hand [6] for an introduction. Despite anticipated heterogeneity, the project organizers are faced with the task of synthesizing data in the absence of reliable uncertainty (${\sigma }_i^2$) estimates.

Our investigation focuses on Bayes estimators obtained from the posterior distribution for an unknown mean, set against a non-informative, objective, or “uniform” prior distribution for both the mean and independent variances. This line of inquiry, initiated by Rukhin [7] under the assumption of normality, grapples with the complete lack of variance information. Needless to say, this framework introduces several statistical complications. For instance, the classical maximum likelihood estimator becomes undefined, as the likelihood function reaches infinity at each data point. Nevertheless, the problem is well defined, as estimating the common mean requires determining at most n parameters, the mean itself, and the variance ratios, ${\omega }_i={\sigma }_i^{-2}/{\sum }_j{\sigma }_j^{-2}$, which belong to the unit simplex of dimension $n-1$, ${\sum }_i{\omega }_i=1$.

In Section 2.1, we investigate the Bayes estimators in the setting allowing for a group of homeogeneous observations, which have the same unknown variance. Under the normality condition, these procedures turn out to have a surprisingly explicit form. In fact, each of the derived rules is a weighted average with data-dependent weights that are invariant under the location–scale transformations, admitting a very clear interpretation. The approximate formulas for the variance of the considered estimators and their limiting behavior are also examined. Section 3 contains several approaches to the distribution of the Bayes estimator. The orthogonal polynomials are discussed in Section 3.3 with recursive formulas derived in Section 3.4.

2. Non-Informative Priors and Bayes Estimators

Consider the situation where distinct independent observables $x_j$ are drawn from a location–scale parameter family with underlying symmetric density p, $x_j\sim {\sigma }_j^{-1}p((·-\mu )/{\sigma }_j),j=1,\dots ,n,$ which has all necessary moments.

The principal interest is in the mean $\mu$, while ${\sigma }_j,j=1,\dots ,n$ are positive nuisance parameters. For this purpose, one needs to estimate the $(n-1)$-dimensional vector $({\omega }_1,\dots ,{\omega }_n)$, with ${\omega }_j={\sigma }_j^{-2}/\left({\sum }_j{\sigma }_j^{-2}\right)$ and $\sum {\omega }_j=1$. If $(w_1,\dots ,w_n)$ is such an estimator, then one can use ${\sum }_j{w}_j{x}_j$ as a $\mu$-statistic. Indeed, if all scale parameters ${\sigma }_j$ are known, the best unbiased estimator of $\mu$ is the weighted means rule, ${\sum }_j{\omega }_j{x}_j$.

Commonly, the estimated weights are taken to be location-invariant—i.e., for any real c,

$$w_j(x_1+c,\dots ,x_n+c)=w_j(x_1,\dots ,x_n).$$

Then, the corresponding estimator $\tilde{\mu }$ is (location) equivariant,

$$\tilde{\mu }(x_1+c,\dots ,x_n+c)=\tilde{\mu }(x_1,\dots ,x_n)+c.$$

Most estimators used in practice are also scale-equivariant,

$$\tilde{\mu }(a{x}_1,\dots ,a{x}_n)=a\tilde{\mu }(x_1,\dots ,x_n),a>0,$$

and this property calls for scale-invariant weights.

In the normal case, the reduction to the invariant procedures leads to an explicit form of the maximum likelihood estimators and of some Bayes procedures.

To eliminate nuisance parameters ${\sigma }_j$ (or ${\omega }_j),j=1,\dots ,n$, one can use a non-informative prior, which is a classical technique. Under mild regularity conditions on the underlying density p, Rukhin [8] derived the Bayes estimator under the quadratic loss (the posterior mean) against the uniform (reference) prior $d\mu {\prod }_{j}d{\sigma }_i/{\sigma }_j$. This statistic coincides with the Bayes rule within the class of invariant procedures.

The discrete posterior distribution is supported by all data points with probabilities

$$w_i^0={\displaystyle \frac{1}{{\prod }_{j\ne i}|x_i-x_j|}}{\left[\sum _k\prod _{j\ne k}{\displaystyle \frac{1}{|x_k-x_j|}}\right]}^{-1}$$

(1)

$$={\displaystyle \frac{s_i}{{\prod }_{j\ne i}(x_i-x_j)}}{\left[\sum _k\prod _{j\ne k}{\displaystyle \frac{s_k}{(x_k-x_{)}|}}\right]}^{-1}.$$

Here, further,

$$s_i=\mathrm{sign}(\prod _{j\ne i}(x_i-x_j)),i=1,\dots ,n,$$

denotes the parity of observations.

Thus, the Bayes estimator has a very explicit form:

$${\delta }^0=\sum _j{w}_j^{\left(0\right)}{x}_j={\displaystyle \frac{{\sum }_j{x}_j{\prod }_{i\ne k}{|x_i-x_j|}^{-1}}{{\sum }_j{\prod }_{i\ne k}{|x_i-x_j|}^{-1}}}.$$

(2)

The magnitude of probabilities (1) describes the intrinsic similarity of observations: the weight of a data-point $x_j$ is large if it is close to the bulk of data, meaning that ${\prod }_{i\ne j}|x_j-x_i|$ is relatively small.

Statistic ${\delta }^0$ also appears in the approximation theory. According to the Tchebycheff interpolation formula, one has

$$|{\delta }^0|=\underset{R}{min}\underset{i}{max}|s_i{x}_i-R\left(x_i\right)|,$$

where $R\left(x\right)$ runs through polynomials of degree not exceeding $n-2$. See Chapter 5 in Trefethen (2013) [9].

The probabilities (1) have their origin in optimization problems involving the discriminant function. Borodin [10] discusses their use in statistical physics. Genest et al. [11] study the remarkable mirror-symmetry (persymmetry) of the underlying Jacobi matrix.

2.1. Heterogeneity and Homogeneity

Here, normality of observations, $x_j\sim {\sigma }_j^{-1}\phi ((·-\mu )/{\sigma }_j)$, $\phi \left(x\right)$ $=exp(-(x^2/2)/\sqrt{2\pi }$, is assumed. We consider the setting where, in addition to x values, there is a possible group of distinct homogeneous data that have the same unknown standard deviation $\sigma$, say, $x_i\sim {\sigma }^{-1}\phi ((y_i-\mu )/\sigma ),i=n+1,\dots ,n+m$. In the context of citizen science projects mentioned in Section 1, $x_i,i=n+1,\dots ,n+m$ may represent data supplied by smartphone users, while x values correspond to measurements derived by other means. In metrology applications, a known homogeneous group of laboratories employing the same techniques may participate in interlaboratory studies.

Then, one has $m+n$ independent observations and altogether $n+2$ unknown parameters $\mu ,{\sigma }_1,\dots ,{\sigma }_n,\sigma$ with the main interest in $\mu$.

We start with the traditional reference prior density of the form

$$\pi (\mu ,{\sigma }_1,\dots ,{\sigma }_n,\sigma )=[\prod _j{\sigma }_j^{-a}]{\sigma }^{-b},$$

(3)

relative to $d\mu [{\prod }_{j}d{\sigma }_j/{\sigma }_j]d\sigma /\sigma$. Here, $a>-1,b+m>-1.$

For any continuous bounded function $h\left(\mu \right)$,

$${\int }_{-\infty }^{\infty }h\left(\mu \right)d\mu \prod _j{\int }_0^{\infty }\phi \left({\displaystyle \frac{x_j-\mu }{{\sigma }_j}}\right){\displaystyle \frac{d{\sigma }_j}{{\sigma }_j^{a+2}}}{\int }_0^{\infty }\prod _i\phi \left({\displaystyle \frac{x_i-\mu }{\sigma }}\right){\displaystyle \frac{d\sigma }{{\sigma }^{b+m+2}}}$$

$$={\displaystyle \frac{{\left[{\int }_0^{\infty }\phi \left(u\right)u^{a}du\right]}^n{\int }_0^{\infty }\phi \left(x\right)u^{b+m}du}{{\left(2\pi \right)}^{(m-1)/2}}}{\int }_{-\infty }^{\infty }{\displaystyle \frac{h\left(\mu \right){[\sum {(x_i-\mu )}^2]}^{(b+m+1)/2}d\mu }{{\prod }_j{|x_j-\mu |}^{a+1}}}.$$

Indeed, for any $j=1,\dots ,n$,

$${\int }_0^{\infty }\phi \left({\displaystyle \frac{x_j-\mu }{{\sigma }_j}}\right){\displaystyle \frac{d{\sigma }_j}{{\sigma }_j^{a+2}}}={\displaystyle \frac{{\int }_0^{\infty }\phi \left(u\right)u^{a}du}{|x_j{-\mu |}^{a+1}}},$$

and

$${\int }_0^{\infty }\prod _i\phi \left({\displaystyle \frac{x_i-\mu }{\sigma }}\right){\displaystyle \frac{d\sigma }{{\sigma }_j^{b+m+2}}}={\displaystyle \frac{{\int }_0^{\infty }\phi \left(u\right){|u|}^{b+m}du}{{\left(2\pi \right)}^{(m-1)/2}{[\sum {(x_i-\mu )}^2]}^{(b+m+1)/2}}}.$$

For any fixed small $ϵ>0$ and fixed $j=1,\dots ,n$, provided that all data points are different,

$$\underset{a\to 0}{lim}a{\int }_{x_j-ϵ}^{x_j+ϵ}{\displaystyle \frac{h\left(\mu \right)d\mu }{{\prod }_k{|x_k-\mu |}^{1+a}{\left[{\sum }_i{(x_i-\mu )}^2\right]}^{(b+m+1)/2}}}$$

$$={\displaystyle \frac{2h\left(x_j\right)}{{\prod }_{1\le k\ne j\le n}|x_j-x_k|{\left[{\sum }_{i=n+1}^{n+m}{(x_i-x_j)}^2\right]}^{(b+m+1)/2}}}.$$

Therefore, for $a\to 0$,

$${\displaystyle \frac{{\int }_{-\infty }^{\infty }h\left(\mu \right)d\mu {\prod }_j{\int }_0^{\infty }\phi \left(\frac{x_j-\mu }{{\sigma }_j}\right)\frac{d{\sigma }_j}{{\sigma }_j^{a+2}}{\int }_0^{\infty }{\prod }_i\phi \left(\frac{x_i-\mu }{\sigma }\right)\frac{d\sigma }{{\sigma }^{b+m+2}}}{{\int }_{-\infty }^{\infty }d\mu {\prod }_j{\int }_0^{\infty }\phi \left(\frac{x_j-\mu }{{\sigma }_j}\right)\frac{d{\sigma }_j}{{\sigma }_j^{a+2}}{\int }_0^{\infty }{\prod }_i\phi \left(\frac{x_i-\mu }{\sigma }\right)\frac{d\sigma }{{\sigma }^{b+m+2}}}}$$

$$\to {\displaystyle \frac{{\sum }_j\frac{h\left(x_j\right)}{{\prod }_{1\le k\ne j\le n}|x_j-x_k|{[{\sum }_i{(x_i-{\overline{x}}_m)}^2+m{(x_j-{\overline{x}}_m)}^2]}^{(b+m+1)/2}}}{\frac{1}{{\prod }_{1\le k\ne j\le n}|x_j-x_k|{[{\sum }_i{(x_i-{\overline{x}}_m)}^2+m{(x_j-{\overline{x}}_m)}^2]}^{(b+m+1)/2}}}},$$

where ${\overline{x}}_m={\sum }_i{x}_i/m$.

Thus, we can formulate the first result.

Theorem 1.

Under the prior (3), when $a\to 0$ the posterior distribution of μ is discrete with finite support $\{x_1,\dots ,x_n\}$ and the probabilities

$$w_j={\displaystyle \frac{1}{[{\prod }_{1\le k\ne j\le n}|x_j-x_k|][1+{(x_j-{\overline{x}}_m)}^2/v^2]{]}^{(b+m+1)/2}}}$$

(4)

$$\times {\left[\sum _j\prod _{k\ne j}{\displaystyle \frac{1}{|x_j-x_k{|[1+{(x_j-{\overline{x}}_m)}^2/v^2]]}^{(b+m+1)/2}}}\right]}^{-1}.$$

Here, ${\overline{x}}_m={\sum }_i{x}_i/m,v^2={\sum }_i{(x_i-{\overline{x}}_m)}^2/m$ are estimators of the common mean and variance based on the homogeneous sub-sample.

The Bayes estimator of μ, i.e., the posterior mean, is

$$\delta =\sum _{j=1}^n{w}_j{x}_j,$$

(5)

with $w_j$ defined by (4).

The probabilities (4) still describe the intrinsic similarity of observations: the weight of a data-point $x_j$ is large if it is close to the greater part of the data. The attenuating factor, ${[1+{(x_j-{\overline{x}}_m)}^2/v^2]}^{-(b+m+1)/2},$ when $b+m+1>0$, encourages $x_j$, which is close to ${\overline{x}}_m$. When the homogeneous data are absent, this factor is 1 and (5) coincides with (2).

In this situation, the posterior mode $\widehat{\delta }=x_{ı}$, if

$$\prod _{k:k\ne ı}|x_{ı}-x_k|{\left[1+{\displaystyle \frac{{(x_{ı}-{\overline{x}}_m)}^2}{v^2}}\right]}^{(b+m+1)/2}$$

$$=\underset{j}{min}\prod _{k:k\ne j}|x_j-x_k|{\left[1+{\displaystyle \frac{{(x_{ı}-{\overline{x}}_m)}^2}{v^2}}\right]}^{(b+m+1)/2},$$

presents the maximum likelihood estimator within the class of invariant procedures.

The prior density (3) for the mean and the variances represents the right Haar measure on the group of linear transforms. In the context of the multivariate normal model, it is known as the the Geisser–Cornfield prior. See Geisser [12], Ch 9.1. This prior is known to be an exact frequentist matching prior yielding as the posterior Fisher’s fiducial distribution (Fernandez and Steel [13], Severini et al. [14]).

Despite this fact, “the prior seems to be quite bad for correlations, predictions and other inferences involving a multivariate normal distribution” (Sun and Berger [15]). Its mentioned drawbacks stem from the fact that if $n\ge 2$, the marginal (or prior predictive) density does not exist. A related weakness of (5) is its sensitivity to observations which are close one to another.

To mitigate these drawbacks, we look now for other prior distributions.

2.2. Conjugate Priors and Variance Formulas

A wide class of Bayes estimators of $\mu$ arises from conjugate prior densities,

$$\pi (\mu ,{\sigma }_1^{-2},\dots ,{\sigma }_n^{-2},{\sigma }^{-2})=exp\left\{-\sum _j{\displaystyle \frac{s^2}{2{\sigma }_j^2}}-{\displaystyle \frac{t_0^2}{2{\sigma }^2}}\right\}[\prod _i{\sigma }_i^{-a}]{\sigma }^{-b},$$

(6)

relative to $d\mu {\prod }_{i}d{\sigma }_i^{-2}d{\sigma }^{-2}$. Here, $a,b,t_0^2$ and $s^2\ge 0$ are hyperparameters to be specified in (6), $a>-1,b+m>-1,s^2\ge 0,t_0^2\ge 0$.

A slightly modified proof of Theorem 1 shows that the posterior distribution of $\mu$ under the prior (6) is proportional to

$$[\prod _j{(s^2+{(x_j-\mu )}^2]}^{-(a+1)/2}{[t^2+{({\overline{x}}_m-\mu )}^2]}^{-(b+m+1)/2},$$

where $t^2=t_0^2/m+v^2$, which is treated as a constant in the following discussion. The posterior distribution in this situation is the product of t-densities (with a degrees of freedom) and t-density (with $b+m$ degrees of freedom). Thus, it is a particular case of the poly-t distribution, which is ubiquitous in multivariate analysis. It appears in the posterior analysis of linear models (Box and Tiao [16]) and is popular in econometrics (Bauwens [17]).

The Bayes estimator has the form

$${\delta }^B={\displaystyle \frac{{\int }_{-\infty }^{\infty }\frac{\mu d\mu }{{\prod }_j{[s^2+{(x_j-\mu )}^2]}^{(a+1)/2}{[t^2+{({\overline{x}}_m-\mu )}^2]}^{(b+m+1)/2}}}{{\int }_{-\infty }^{\infty }\frac{d\mu }{{\prod }_j{[s^2+{(x_j-\mu )}^2]}^{(a+1)/2}{[t^2+{({\overline{x}}_m-\mu )}^2]}^{(b+m+1)/2}}}}.$$

(7)

If $b+m\to -1$, (7) is the classical Pitman estimator of the location parameter involving t-distributions with a degrees of freedom. It is especially well studied for the Cauchy location/scale parameter family ($a=1$).

If, in addition, $s^2=0$,

$${\delta }_B={\int }_{-\infty }^{\infty }{\displaystyle \frac{\mu d\mu }{{\prod }_j{|x_j-\mu |}^{a+1}}}{\left[{\int }_{-\infty }^{\infty }{\displaystyle \frac{d\mu }{{\prod }_j{|x_j-\mu |}^{a+1}}}\right]}^{-1},$$

which corresponds to the formal Pitman estimator of the location parameter derived from the working family ${|x-\mu |}^{a+1}$ employed to estimate the location parameter $\mu$ when the observations are normal.

Needless to say that the functions in this family are not probability densities. Moreover, they have a singularity of the third kind.

When $s^2$ and $t^2$ are fixed positive numbers, the approximate variance of (7) can be found via the usual argument employed for M-estimators, i.e., solutions of the moment-type equation ${\sum }_j{\psi }_j(x_j-\mu )=0$ (or minimizers of ${\sum }_j{\rho }_j(x_j-\mu )$). In our case, the contrast functions are,

$${\psi }_j(x_j-\mu )={\displaystyle \frac{(a+1)(x_j-\mu )}{s^2+{(x_j-\mu )}^2}},j=1,\dots ,n,$$

$$\psi ({\overline{x}}_m-\mu )={\displaystyle \frac{(b+m+1)({\overline{x}}_m-\mu )}{t^2+{({\overline{x}}_m-\mu )}^2}}.$$

The M-estimator $\tilde{\mu }$ satisfies the equation

$$\sum _j{\displaystyle \frac{(a+1)(x_j-\mu )}{s^2+{(x_j-\mu )}^2}}++{\displaystyle \frac{(b+m+1)({\overline{x}}_m-\tilde{\mu })}{t^2+{({\overline{x}}_m-\tilde{\mu })}^2}}=0.$$

According to well-known results (e.g., Huber and Ronchetti [18]),

$$\mathrm{Var}\left(\tilde{\mu }\right)\approx \left[\sum _j{E}_j{\psi }_j^2\left(x_j\right)+E{\psi }^2\left({\overline{x}}_m\right)\right]{\left[\sum _j{E}_j{\psi }_j^{\prime }\left(x_j\right)+E{\psi }^{\prime }\left({\overline{x}}_m\right)\right]}^{-2}$$

(8)

$$=\left[{(a+1)}^2\sum _j{E}_j{\displaystyle \frac{x_j^2}{{(s^2+x_j^2)}^2}}+{(b+m+1)}^{2}E{\displaystyle \frac{{\overline{x}}_m^2}{{(t^2+{\overline{x}}_m^2)}^2}}\right]$$

$$\times {\left[(a+1)\sum _j{E}_j{\displaystyle \frac{s^2-x_j^2}{{(s^2+x_j^2)}^2}}+(b+m+1)E{\displaystyle \frac{t^2-{\overline{x}}_m^2}{t^2+{\overline{x}}_m^2}}\right]}^{-2}.$$

Here, $E_j$ refers to the expectation evaluated under the normal distribution with zero mean and variance ${\sigma }_j^2,j=1,\dots ,n$; the distribution of ${\overline{x}}_m$ is also normal with variance ${\sigma }^2/m$. The main restriction on ${\sigma }^2$ is that the Central Limit Theorem for ${\sum }_j{\psi }_j\left(x_j\right)$ holds. For instance, one can employ (8) if the Liapounov condition for independent non-identically distributed summands is satisfied—i.e., ${\left({\sum }_j{\sigma }_j^{-6}\right)}^2$ $=o\left({\left({\sum }_j{\sigma }_j^{-4}\right)}^3\right)$ (e.g., Lehmann [19], Theorem 2.7.3).

To simplify (8), we need the known formula for the standard normal Z and positive $\beta$,

$$E{\displaystyle \frac{\beta }{Z^2+{\beta }^2}}=M\left(\beta \right),$$

where $M\left(\beta \right)=[1-\Phi (\beta \left)\right]/\phi \left(\beta \right)$ is the familiar Mills ratio (Stuart and Ord, 1994) [20]. With ${\beta }_j=s/{\sigma }_j,{\beta }_{n+1}=m^{1/2}t/\sigma ,x_{n+1}={\overline{x}}_m,$ the differentiation of this identity shows that for $i=1,\dots ,n+1$,

$$E_i{\displaystyle \frac{x_i^2}{{(x_i^2+s^2)}^2}}={\displaystyle \frac{{\beta }_i}{2s^2}}\left[(1+{\beta }_i^2)M\left({\beta }_i\right)-{\beta }_i\right],$$

and

$$E_i{\displaystyle \frac{s^2-x_i^2}{{(x_i^2+s^2)}^2}}={\displaystyle \frac{{\beta }_i^2}{s^2}}\left[(1-{\beta }_{i}M\left({\beta }_i\right)\right],$$

where for $i=n+1$ one has to replace $s^2$ by $t^2$.

These identities allow for expressing the approximate variance (8) in terms of the Mills ratio:

$$\mathrm{Var}\left(\tilde{\mu }\right)=\{{\displaystyle \frac{{(a+1)}^2}{2s^2}}\sum _i[{\beta }_i(1+{\beta }_i^2)M\left({\beta }_i\right)-{\beta }_i^2]$$

(9)

$$+{\displaystyle \frac{{(b+m+1)}^2}{2t^2}}[{\beta }_{n+1}(1+{\beta }_{n+1}^2)M\left({\beta }_{n+1}\right)-{\beta }_{n+1}^2]\}$$

$$\times {\left[{\displaystyle \frac{a+1}{s^2}}\sum _i{\beta }_i^2[(1-{\beta }_{i}M\left({\beta }_i\right)]+{\displaystyle \frac{b+m+1}{t^2}}{\beta }_{n+1}^2[(1-{\beta }_{n+1}M\left({\beta }_{n+1}\right)]\right]}^{-2}.$$

When ${\sigma }_i^2\equiv {\sigma }^2$,

$$\mathrm{Var}\left(\tilde{\mu }\right)={\displaystyle \frac{{\sigma }^2\lambda [(1+{\beta }^2)M\left(\beta \right)-\beta ]}{\beta {[1-\beta M\left(\beta \right)]}^2}},$$

where

$$\lambda ={\displaystyle \frac{n{(a+1)}^2{s}^{-2}+{(b+m+1)}^2{t}^{-2}}{2{[n(a+1)s^{-2}+(b+m+1)t^{-2}]}^2}}.$$

Since $\lambda \ge {(n{s}^{-1}+t^{-1})}^{-1}$, one has

$$\mathrm{Var}\left(\tilde{\mu }\right)\ge {\displaystyle \frac{{\sigma }^2}{n{s}^{-2}+t^{-2}}}=\mathrm{Var}\left({\tilde{\mu }}^0\right),$$

where

$${\tilde{\mu }}^0={\displaystyle \frac{\sum x_j{s}^{-2}+\overline{x}{t}^{-2}}{n{s}^{-2}+t^{-2}}},$$

is the best unbiased estimator of $\mu$ when all variances are equal.

If $\beta =1$, i.e., $s^2$ and $t^2$ are adequate approximations of the common variance, then

$${\displaystyle \frac{\mathrm{Var}\left(\tilde{\mu }\right)}{\mathrm{Var}\left({\tilde{\mu }}^0\right)}}\approx {\displaystyle \frac{\left[2M\right(1)-1]}{2{[1-M\left(1\right)]}^2}}=1.31...$$

Thus, when all ${\sigma }_i^2$ are equal, and hyperparameters in (6) are chosen so that $s^2\approx \sum {(x_j-\overline{x})}^2/(n-1)$ and $t^2\approx v^2$, the variance of $\tilde{\mu }$ is about $1.31$ larger than that of ${\tilde{\mu }}^0.$ Smaller values of $s^2$ lead to the larger variance $\tilde{\mu }$.

If $s^2=s_0^2{\gamma }_n\to 0$ for some sequence ${\gamma }_n\to 0$, $n{\gamma }_n^2\to \infty$, the corresponding estimator $\tilde{\mu }={\tilde{\mu }}_n$ is asymptotically normal, $\sqrt{n}{\gamma }_n({\tilde{\mu }}_n-\mu )\to N(0,{\sigma }^{4}M\left(0\right)/\left(2s_0^2\right))$, albeit at a slower rate than $\sqrt{n}$. Therefore, there is no surprise that in the case of ${\delta }^0$ (for which $s^2=0,m=0)$, when $n\beta \mathrm{Var}\left(\tilde{\mu }\right)\to {\sigma }^{2}M\left(0\right)/2$ as $\beta \to 0$, one has n Var(${\delta }^0)\to \infty$. Indeed, it seems that ${\delta }^0$ bears more resemblance to the nonparametric estimates of the location parameter for which the convergence rate is slower than $\sqrt{n}$. Numerical experiments suggest that in the normal case, $n^{1/2}$Var$\left({\delta }^0\right)/log\left(n\right)\to {\pi }^2/8$.

We summarize now the main results of this section.

Theorem 2.

Under the prior distribution (6), the posterior distribution is the product of t-distributions with a degrees of freedom and a t-distribution with $b+m$ degrees of freedom. The approximate variance of the Bayes estimator (7) satisfies (8) with Expression (9) via the Mills ratio.

For the remainder of this paper, we will concentrate on the estimator ${\delta }^0$.

3. Distribution of ${\mathit{\delta }}^{\mathbf{0}}$

3.1. Jacobian and Moments

Let the vector $\mathbf{e}$ have unit coordinates $\mathbf{w}={(w_1^0,\dots ,w_n^0)}^T$ and $\mathbf{x}={(x_1,\dots ,x_n)}^T$ representing a random sample. By location equivariance,

$$\sum _k{\displaystyle \frac{\partial {\delta }^0}{\partial x_k}}={\mathbf{e}}^T\nabla {\delta }^0=1,$$

(10)

and by scale equivariance,

$$\sum _k{x}_k{\displaystyle \frac{\partial {\delta }^0}{\partial x_k}}={\mathbf{x}}^T\nabla {\delta }^0={\delta }^0.$$

(11)

Define the matrix Q by its elements $q_{ij}={\sum }_{k:k\ne i,j}{(x_i-x_k)}^{-1}$, $i\ne j$, $q_{ii}=0,$ so that

$$Q=\mathbf{r}{\mathbf{e}}^T-\mathrm{diag}\left(\mathbf{r}\right)-Z,$$

(12)

where the i-th coordinate of the vector $\mathbf{r}$ is $r_i={\sum }_{j:j\ne i}{(x_i-x_j)}^{-1}$, and the skew-symmetric matrix Z for $i\ne j$ has elements ${(x_i-x_j)}^{-1}$, and a zero diagonal.

Then, the form of the Jacobian $J\left(\mathbf{w}\right)$ with elements $\partial w_i^0/\partial x_j,i,j=1,\dots ,n$, is

$$J\left(\mathbf{w}\right)=[\mathrm{diag}\left(\mathbf{w}\right)-\mathbf{w}{\mathbf{w}}^T]Q^T,$$

(13)

so that $\mathrm{tr}J\left(\mathbf{w}\right)=-{\mathbf{w}}^{T}Q\mathbf{w},$ and

$$\nabla {\delta }^0=[I+Q(\mathrm{diag}\left(\mathbf{x}\right)-{\delta }^{0}I)]\mathbf{w}=\mathbf{w}-[\mathrm{diag}\left(\mathbf{r}\right)+Z][\mathrm{diag}\left(\mathbf{x}\right)-{\delta }^{0}I]\mathbf{w}.$$

(14)

Because of (14), one obtains

$${\mathbf{w}}^{T}Q[\mathrm{diag}\left(\mathbf{x}\right)-{\delta }^{0}I]\mathbf{w}={\displaystyle \frac{1}{2}}-{\mathbf{w}}^T\mathbf{w},$$

so that

$${\mathbf{w}}^T\nabla {\delta }^0={\displaystyle \frac{1}{2}}.$$

(15)

We will get an extension of (15) to higher moments: ${\delta }_m=\sum x_i^m{w}_i,m=1,2,\dots ,{\delta }_0=1,{\delta }_1={\delta }^0$. It is shown in Rukhin (2023) [8] that for any integer $m\ge 1$,

$$\nabla {\delta }_m=[m\mathrm{diag}{\left(\mathbf{x}\right)}^{m-1}+Q(\mathrm{diag}{\left(\mathbf{x}\right)}^m-{\delta }_{m}I)]\mathbf{w}$$

(16)

$$=[m\mathrm{diag}{\left(\mathbf{x}\right)}^{m-1}-\left(\mathrm{diag}\left(\mathbf{r}\right)+Z)(\mathrm{diag}{\left(\mathbf{x}\right)}^m-{\delta }_{m}I)\right]\mathbf{w},$$

so that

$${\mathbf{w}}^T\nabla {\delta }_m={\displaystyle \frac{m}{2}}\sum _i{\left(w_i^0\right)}^2{x}_i^{m-1}+{\displaystyle \frac{1}{2}}\sum _{p=0}^{m-1}{\delta }_p{\delta }_{m-1-p}-\sum r_i{\left(w_i^0\right)}^2{x}_i^m.$$

The coefficients, ${\tau }_m=2{\mathbf{w}}^T\nabla {\delta }_m-{\sum }_{p=0}^{m-1}{\delta }_p{\delta }_{m-1-p},$ vanish for $0\le m\le 2n-2$.

Indeed,

$${\mathbf{w}}^T(\mathrm{diag}\left(\mathbf{r}\right)[\mathrm{diag}\left({\mathbf{x}}^m\right)-{\delta }_{m}I]\mathbf{w}={\displaystyle \frac{m}{2}}\sum _i{\left(w_i^0\right)}^2{x}_i^{m-1}.$$

Because of Hermite’s (osculatory) interpolation formula, for $0\le m\le 2n-2$,

$$2\sum _i{r}_i{x}_i^m{w}_i^2=m\sum _i{x}_i^{m-1}{w}_i^2.$$

Therefore, for these values of m,

$$2{\mathbf{w}}^T\nabla {\delta }_m=\sum _{p=0}^{m-1}{\delta }_p{\delta }_{m-1-p}.$$

In particular,

$${\mathbf{w}}^T\nabla ({\delta }_2-{\delta }_1^2)={\delta }_1-{\delta }_1=0.$$

Let $W={\left({\sum }_i{\prod }_{j\ne i}{|x_i-x_j|}^{-1}\right)}^{-1},$ so that the probabilities (4) can be written as $w_i={\prod }_{j\ne i}{|x_i-x_j|}^{-1}W$. When $m=2n-1$ and ${\tau }_m=W^2$,

$$2{\mathbf{w}}^T\nabla {\delta }_{2n-1}=\sum _{p=0}^{2n-2}{\delta }_p{\delta }_{2n-2-p}+W^2.$$

For any positive integer m, the coefficients ${\tau }_m$ determine the asymptotic expansion in $x^{-1}$ of $W^2{\prod }_i{(x-x_i)}^{-2}$, which is

$${\displaystyle \frac{W^2}{{\prod }_i{(x-x_i)}^2}}=\sum {\displaystyle \frac{w_i^2}{{(x-x_i)}^2}}-2\sum {\displaystyle \frac{r_i{w}_i^2}{x-x_i}}$$

(17)

$$=\sum _{m=1}^{\infty }{\displaystyle \frac{m{\sum }_i{w}_i^2{x}_i^{m-1}}{x^{m+1}}}-2\sum _{m=1}^{\infty }{\displaystyle \frac{{\sum }_i{r}_i{w}_i^2{x}_i^m}{x^{m+1}}}=\sum _{m=2n-1}^{\infty }{\displaystyle \frac{{\tau }_m}{x^{m+1}}}.$$

For $m\ge 2n-1$, the values ${\tau }_m$ can be found from the formula $W^{-2}{\tau }_{2n-1+m}={\sum }_{p_1+\cdots +p_n=m}{\prod }_1^n(p_i+1)x_i^{p_i}.$

If $s_i=\mathrm{sign}\left({\prod }_{j\ne i}(x_i-x_j)\right),i=1,\dots ,n$, is the parity of $x_i$, then

$$\prod _1^n(x-x_k)=P_e\left(x\right)P_o\left(x\right)=\prod _{i:s_i=1}(x-x_i)\prod _{j:s_j=-1}(x-x_j)$$

is a product of two polynomials of degrees $N_e=N_o=n/2,n$ (even), or $N_e=(n+1)/2=N_o+1,n$ (odd); $w_i=W/\left[P_o\left(x_i\right)P_e^{\prime }\left(x_i\right)\right]$ if $s_i=1$; if $s_j=-1,w_j=-W/\left[P_e\left(x_j\right)P_o^{\prime }\left(x_j\right)\right]$.

According to the classical Lagrange interpolation formula, one has

$${\displaystyle \frac{1}{{\prod }_1^n(x-x_k)}}=\sum _k{\displaystyle \frac{1}{(x-x_k){\prod }_{j\ne k}(x_k-x_j)}}.$$

(18)

For non-negative integer m, define ${\kappa }_m={\sum }_i{s}_i{w}_i{x}_i^m$. Then, the same formula implies that for any $m,m=0,1,\dots ,n-2$,

$${\kappa }_m=W\sum _1^n{\displaystyle \frac{x_i^m}{{\prod }_{j\ne i}(x_i-x_j)}}=0.$$

The coefficients ${\kappa }_m$ determine the asymptotic expansion in $x^{-1}$ of $W{\left[{\prod }_1^n(x-x_k)\right]}^{-1}=\sum {\kappa }_m{x}^{-m-1}.$

For $m\ge n-1$, the values ${\kappa }_m$ can be found from the formula $W^{-1}{\kappa }_{n-1+m}={\sum }_{p_1+\cdots +p_n=m}{\prod }_i{x}_i^{p_i}$, e.g., ${\kappa }_{n-1}=W,{\kappa }_n=W\sum x_k$. With ${\mathcal{E}}_r,r=0,1,\dots ,n$, denoting elementary symmetric functions, one obtains for a positive integer m

$${\kappa }_{n+m-1}={\mathcal{E}}_1{\kappa }_{n+m-2}-{\mathcal{E}}_2{\kappa }_{n+m-3}+\cdots +-{(-1)}^n{\mathcal{E}}_n{\kappa }_{m-1}.$$

Comparison to (17) shows that

$$\sum _{m=2n-1}^{\infty }{\displaystyle \frac{{\tau }_m}{x^{m+1}}}={\left[\sum _{m=n-1}^{\infty }{\displaystyle \frac{{\kappa }_m}{x^{m+1}}}\right]}^2,$$

which means that for any m,

$$2{\mathbf{w}}^T\nabla {\delta }_m=\sum _{p=0}^{m-1}{\delta }_p{\delta }_{m-1-p}+\sum _{p=0}^{m-1}{\kappa }_p{\kappa }_{m-1-p}.$$

(19)

Furthermore, one has

$${\displaystyle \frac{{\delta }_m+{\kappa }_m}{W}}=2\sum _{i:s_i=1}{\displaystyle \frac{x_i^m}{{\prod }_{j\ne i}(x_i-x_j)}},$$

(20)

and

$${\displaystyle \frac{{\delta }_m-{\kappa }_m}{W}}=2\sum _{i:s_i=-1}{\displaystyle \frac{x_i^m}{{\prod }_{j\ne i}(x_i-x_j)}}.$$

(21)

I will now formulate the results.

Theorem 3.

Formulas (17) and (19)–(21) hold for ${\delta }_m={\sum }_i{w}_i{x}_i^m$ and ${\kappa }_m={\sum }_i{s}_i{w}_i{x}_i^m$, $m=0,1,\dots$

The matrix $diag\left(\mathbf{x}\right)\left[diag\right(\mathbf{r})+Z]$ is diagonalizable with the eigenvalues $n-j,j=1,\dots ,n$.

Proof. 

Let

$$V=\left[\begin{array}{ccccc}1& 1& \cdots & 1& 1\\ x_1& x_2& \cdots & x_{n-1}& x_n\\ ⋮& ⋮& \ddots & ⋮\\ x_1^{n-1}& x_2^{n-1}& \cdots & x_{n-1}^{n-2}& x_n^{n-1}\end{array}\right]$$

represent the Vandermonde matrix.

We prove that

$$V\mathrm{diag}\left(\mathbf{x}\right)\left[\mathrm{diag}\right(\mathbf{r})+Z]=\Lambda V.$$

Here, $\Lambda$ is a lower triangular matrix with the elements ${\lambda }_{jj}=n-j,{\lambda }_{ij}={\sum }_k{x}_k^{i-j},i>j,{\lambda }_{ij}=0,i<j,i,j=1,\dots ,n$.

Indeed, the elements of the matrix on the left-hand side are

$$(V\mathrm{diag}\left(\mathbf{x}\right){\left[\mathrm{diag}(+Z]\right)}_{pk}=\sum _{t:t\ne k}{\displaystyle \frac{x_k^p-x_t^p}{x_k-x_t}}=\sum _{t:t\ne k}\sum _{i=0}^{p-1}{x}_k^i{x}_t^{p-1-i}$$

$$=\sum _{i=0}^{p-2}{x}_k^i\sum _t{x}_t^{p-1-i}+(n-p)x_k^{p-1}=\sum _t{\lambda }_{pt}{x}_k^{t-1},1\le p,k\le n.$$

Since all eigenvalues of $\mathrm{diag}\left(\mathbf{x}\right)\left[\mathrm{diag}\right(\mathbf{r})+Z]$ are distinct, it is diagonalizable. □

3.2. Differential Equations and Integration by Parts

Let $L=L(x_1,\dots ,x_n)=log{\prod }_{i<j}|x_j-x_i|,$ and

$$F=F(x_1,\dots ,x_n)=log\left(n^{-1}\sum _k\prod _{i<j,i,j\ne k}|x_i-x_j|\right),$$

where $x_1,\dots ,x_n$ form a standard normal random sample.

Then, $exp\left\{2L\right\}={\prod }_i\left({\prod }_{j:j\ne i}|x_i-x_i|\right),F\ge (n-2)L/n$. In this notation, ${\mathbf{e}}^T\nabla L={\mathbf{e}}^T\nabla F=0$, $W=n^{-1}exp\{L-F\}$, ${\mathbf{x}}^T\nabla L=n(n-1)/2,{\mathbf{x}}^T\nabla F=(n-1)(n-2)/2,$ and

$${\mathbf{w}}^T\nabla F={\mathbf{w}}^{T}Q\mathbf{w}={\mathbf{w}}^T\}={\mathbf{w}}^T\nabla L.$$

According to the celebrated Selberg formula for $z>-2/n$,

$$Eexp\left\{zL(x_1,\dots ,x_n)\right\}=\prod _{k=1}^n{\displaystyle \frac{\Gamma (zk/2+1)}{\Gamma (z/2+1)}},$$

which implies that

$$Eexp\left\{F(x_1,\dots ,x_n)\right\}=\prod _{k=1}^{n-1}{\displaystyle \frac{\Gamma (k/2+1)}{\Gamma (3/2)}}.$$

See Lu and Richards [21] for several results related to uses of the Selberg formula in statistics. In particular, these authors elaborate the Central Limit Theorem for L expressed as a U-statistic,

$${\displaystyle \frac{1}{\sqrt{n}}}\left[{\displaystyle \frac{2L}{n}}+{\displaystyle \frac{Cn}{2}}\right]\to N\left(0,{\displaystyle \frac{{\pi }^2}{18}}\right),$$

where $C=0.577\dots$ is Euler’s constant.

To simplify the formula for the quadratic risk of ${\delta }^0$, we use integration by parts:

$$E{\left({\delta }^0\right)}^2=\sum _{j}E{\delta }^0{w}_j{x}_j=\sum _{j}E{\displaystyle \frac{\partial \left({\delta }^0{w}_j\right)}{\partial x_j}}=\sum _{j}E\left({\displaystyle \frac{\partial {\delta }^0}{\partial x_j}}{w}_j+{\displaystyle \frac{\partial w_j}{\partial x_j}}{\delta }^0\right),$$

so that

$$E{\left({\delta }^0\right)}^2={\displaystyle \frac{1}{2}}-E({\mathbf{w}}^T\mathbf{r}){\delta }^0.$$

Let $q\left(t\right)$ denote the density of the estimator ${\delta }^0$. This density exists and is differentiable as $\delta$ is the sum of two independent random variables: ${\delta }^0-\overline{x}$ and normally distributed $\overline{x}$. Clearly, $q\left(t\right)=q(-t)$, and

$${\displaystyle \frac{q^{\prime }\left(t\right)}{q\left(t\right)}}=-nE(\overline{x}|{\delta }^0=t).$$

This identity holds since the score function of ${\delta }^0$ is the conditional expected value of the score function of $\overline{x}$ for given ${\delta }^0$. It also follows from (10) in the same way as the next formula follows from (11).

For any z, $x^T\nabla e^{z{\delta }^0}=z{e}^{z{\delta }^0}{\delta }^0,$ so that

$$E{e}^{z{\delta }^0}(\sum _i{x}_i^2-n)=zE{e}^{z{\delta }^0}{\delta }^0,$$

or

$$E(\sum _i{x}_i^2|{\delta }^0=t)=n-1-t{\displaystyle \frac{q^{\prime }(t)}{q(t)}}=n-1+tE(\sum _i{x}_i|{\delta }^0=t).$$

More generally,

$${\displaystyle \frac{e^{n{z}^2/2}q(t-z)}{q\left(t\right)}}=E\left[e^{z{\sum }_i{x}_i}|{\delta }^0=t\right],$$

so that $q^{″}\left(t\right)/q\left(t\right)=E[{\left({\sum }_i{x}_i\right)}^2|{\delta }^0=t]-n.$

Similarly, it follows from (15) that

$$E{\mathbf{w}}^{\mathbf{T}}\nabla e^{z{\delta }^0}=E{e}^{z{\delta }^0}[{\delta }^0-\mathrm{tr}J\left(\mathbf{w}\right)]=E{e}^{z{\delta }^0}({\delta }^0+{\mathbf{w}}^{\mathbf{T}}\mathbf{r})={\displaystyle \frac{z{E}^{z{\delta }^0}}{2}},$$

which means that

$${\displaystyle \frac{q^{\prime }\left(t\right)}{q\left(t\right)}}=-2[t+E({\mathbf{w}}^{\mathbf{T}}\mathbf{r}|{\delta }^0=t)].$$

(22)

Since ${\mathbf{w}}^{\mathbf{T}}\mathbf{r}={\mathbf{w}}^{\mathbf{T}}Q\mathbf{w}={\mathbf{w}}^T\nabla L,$ for any differentiable bounded function $g\left(t\right)$ and any z,

$$g\left(t\right)({\mathbf{w}}^T\nabla e^{zL})=zg\left(t\right)\left({\mathbf{w}}^{T}r\right)e^{zL}.$$

Integrating by parts, one obtains for $z>-2/n$

$$zEg\left({\delta }^0\right)({\mathbf{w}}^T\mathbf{r})e^{zL}=Eg\left({\delta }^0\right)({\mathbf{w}}^T\nabla e^{zL})=E[-g\left({\delta }^0\right)\mathrm{tr}J\left(\mathbf{w}\right)-{\mathbf{w}}^T\nabla g+{\delta }^{0}g\left({\delta }^0\right)]e^{zL},$$

or

$$E{e}^{zL}[{\delta }^{0}g\left({\delta }^0\right)-g^{\prime }\left({\delta }^0\right)/2]=(z-1)E{e}^{zL}g\left({\delta }^0\right)({\mathbf{w}}^T\mathbf{r}).$$

It follows that

$$\left[{\displaystyle \frac{q^{\prime }\left(t\right)}{2q\left(t\right)}}+\delta \right]E(e^{zL}|{\delta }^0=t)+{\displaystyle \frac{d}{2dt}}E(e^{zL}|{\delta }^0=t)=(z-1)E(e^{zL}{\mathbf{w}}^T\mathbf{r}|{\delta }^0=t).$$

By putting $z=1$, one obtains

$$q\left(t\right)={\displaystyle \frac{E{e}^L}{\sqrt{\pi }}}{\displaystyle \frac{e^{-t^2}}{E(e^L|{\delta }^0=t)}}={\displaystyle \frac{1}{\sqrt{\pi }}}\prod _{k=1}^n{\displaystyle \frac{\Gamma (k/2+1)}{\Gamma (3/2)}}{\displaystyle \frac{e^{-t^2}}{E(e^L|{\delta }^0=t)}}.$$

(23)

One can show that $E(e^L|{\delta }^0)/E{e}^L=E(e^F|{\delta }^0)/E{e}^F$ by applying the above argument to ${\mathbf{w}}^T\mathbf{r}={\mathbf{w}}^T\nabla F$. Indeed,

$${\displaystyle \frac{d}{2dt}}E(e^{zF}|{\delta }^0=t)-E(w^{T}r|{\delta }^0=t)E(e^{zF}|{\delta }^0=t)=(z-1)E(e^{zF}{\mathbf{w}}^T\mathbf{r}|{\delta }^0=t),$$

so that

$$2E({\mathbf{w}}^T\mathbf{r}|{\delta }^0=t)={\displaystyle \frac{d}{dt}}log\left(E(e^F|{\delta }^0=t)\right)=-{\displaystyle \frac{q^{\prime }\left(t\right)}{q\left(t\right)}}-2t.$$

Theorem 4.

Formulas (22) and (23) hold for the density q of the estimator ${\delta }^0$.

For $-2/n<z<1$,

$$E({\mathbf{w}}^T\mathbf{r}|{\delta }^0=t)E(e^{(1-z)F+zL}|{\delta }^0=t)={\displaystyle \frac{d}{2dt}}E(e^{(1-z)F+zL}|{\delta }^0=t),$$

which means that

$$E(e^F|{\delta }^0,W)E{e}^F=E(e^F|{\delta }^0)E(e^F|W).$$

To proceed, we need some facts about orthogonal polynomials with regard to (random) weights $\mathbf{w}$.

3.3. Random Orthogonal Polynomials

Using notation from the previous section for $m=1,\dots ,n,$ we define Hankel moment matrices as follows:

$$\left[\begin{array}{cccc}1& {\delta }_1& \cdots & {\delta }_m\\ ⋮& ⋮& \ddots & ⋮\\ {\delta }_m& {\delta }_{m+1}& \cdots & {\delta }_{2m}\end{array}\right].$$

Their determinants $G_m$ satisfy the condition $G_m=W^{2m-n+2}{G}_{n-m-2}$. This fact is due to self-duality of the weights $\mathbf{w}$ (Rukhin, 2023 [8]). Then, the sequence $h_m={\sum }_j{\left[T_m\left(x_j\right)\right]}^2{w}_j{,}^0{h}_0=1$ is such that $h_m=G_m/G_{m-1}$, $h_m=W^2/h_{n-m-1}$.

We consider the sequence of monic polynomials $T_m,m=0,1,\dots ,n$, $T_{-1}\left(x\right)=0$, $T_0\left(x\right)=1,T_1\left(x\right)=x-{\delta }_1,\dots ,T_n\left(x\right)=\prod (x-x_k)=P_e\left(x\right)P_o\left(x\right)$, which are orthogonal in the space $L_2$ of all functions over $\{x_1,\dots ,x_n\}$. They are known to satisfy the three-term recurrence:

$$T_{m+1}\left(z\right)=(x-{\alpha }_m)T_m\left(z\right)-{\beta }_m{T}_{m-1}\left(z\right),$$

(24)

where ${\beta }_m=h_m/h_{m-1},{\beta }_m={\beta }_{n-m}$, and ${\alpha }_m={\sum }_t{x}_t{\left[T_m\left(x_t\right)\right]}^2{w}_t/h_m,{\alpha }_m={\alpha }_{n-1-m}$. Clearly, ${\alpha }_m$ depends on ${\delta }_1,\dots ,{\delta }_{2m+1}$ only, while ${\beta }_m$ is determined by the first $2m$ moments, ${\delta }_1,\dots ,{\delta }_{2m}$. For example, ${\alpha }_0={\delta }_1,{\alpha }_1=({\delta }_3-{\delta }_1{\delta }_2+{\delta }_1^3)/({\delta }_2-{\delta }_1^2)$, ${\beta }_0=0,{\beta }_1={\delta }_2-{\delta }_1^2$.

Formula (24) shows that

$$T_m\left(x_i\right)=s_i{h}_m{T}_{n-m-1}\left(x_i\right)/W.$$

(25)

Thus, if n is odd, then $T_{(n-1)/2}=P_o,$ and $h_{(n-1)/2}=W$. Polynomials $P_e$ and $P_o$ are orthogonal for any n.

The polynomial $T_{n-1}\left(x\right)={\sum }_i{w}_i^0{\prod }_{j\ne i}(x-x_j)$ is less deviant from zero in $L_{\infty }$: $T_{n-1}\left(x_i\right)=s_{i}W$. Therefore, any monic polynomial $R\left(x\right)$ of degree not exceeding $n-1$

$$W\le \underset{i}{max}|R\left(x_i\right)|.$$

(26)

The ratio $T_{n-1}\left(z\right)/T_n\left(z\right)$ gives the Stiltjes transform of M,

$$T_{n-1}\left(z\right)=-\sum _i{w}_i^0{\displaystyle \frac{\partial T_n\left(z\right)}{\partial x_i}}=-{\mathbf{w}}^T\nabla T_n\left(z\right).$$

In addition to $x_1,\dots ,x_n$, the polynomial $T_{n-1}^2\left(z\right)-W^2$ has $n-2$ real roots (which interlace those of $T_{n-1}$), with $R_{n-1}$ denoting the monic polynomial of degree $n-2$, which has the following roots:

$$T_{n-1}^2\left(z\right)-W^2=T_n\left(z\right)R_{n-1}\left(z\right).$$

It follows that $R_{n-1}$ coincides with the associated polynomial $T_{n-1}$:

$$R_{n-1}\left(z\right)=\sum _i{\displaystyle \frac{[T_{n-1}\left(z\right)-T_{n-1}\left(x_i\right)]w_i}{z-x_i}}.$$

Associated with $T_m$, the orthogonal monic polynomial $R_m,m=0,1,\dots ,n-1$ has degree $m-1$. These polynomials satisfy the same recurrence (refrek) but with different initial conditions—$R_0=0$ and $R_1=1$—so that $R_2\left(z\right)=z-{\alpha }_1$, $T_{n-1}\left(z\right)=(z-{\delta }_1)R_{n-1}\left(z\right)-{\beta }_1{R}_{n-2}\left(z\right)$.

According to (17),

$$R_{n-1}\left(z\right)=-2\sum _i{w}_i^0{\displaystyle \frac{\partial T_{n-1}\left(z\right)}{\partial x_i}}=-2{\mathbf{w}}^T\nabla T_{n-1}\left(z\right).$$

(27)

Since

$$T_{n-1}^{\prime }\left(x_i\right)={\displaystyle \frac{s_{i}W}{w_i^0}}\sum _{j\ne i}{\displaystyle \frac{w_i^0+w_j^0}{x_i-x_j}},$$

one has

$$R_{n-1}\left(x_i\right)={\displaystyle \frac{2T_{n-1}^{\prime }\left(x_i\right)}{w_i^0}}=2s_{i}W\sum _{j:s_j=-s_i}{\displaystyle \frac{w_j^0}{x_i-x_j}}.$$

(28)

One can represent $R_{n-1}\left(z\right)=R_e^{\left(n\right)}\left(z\right)R_o^{\left(n\right)}\left(z\right)$ as a product of two monic polynomials (with real roots) of degrees $N_e-1$ and $N_o-1$ respectively. Then, $T_{n-1}\left(z\right)-W=P_e\left(z\right)R_o^{\left(n\right)}\left(z\right)$ and $T_{n-1}\left(z\right)+W=P_o\left(z\right)R_e^{\left(n\right)}\left(z\right)$, so that if $s_i=1$, $R_e^{\left(n\right)}\left(x_i\right)=2W/P_o\left(x_i\right),$ and if $s_j=-1$, $R_o^{\left(n\right)}\left(x_j\right)=-2W/P_e\left(x_j\right)$,

$$R_e^{\left(n\right)}\left(z\right)=2W\sum _{s_i=1}{\displaystyle \frac{{\prod }_{s_k=1,k\ne i}(z-x_k)}{P_o\left(x_i\right)P_e^{\prime }\left(x_i\right)}},$$

(29)

and

$$R_o^{\left(n\right)}\left(z\right)=-2W\sum _{s_j=-1}{\displaystyle \frac{{\prod }_{s_{\ell }=-1,\ell \ne j}(z-x_{\ell })}{P_e\left(x_j\right)P_o^{\prime }\left(x_j\right)}}.$$

(30)

Clearly,

$${\displaystyle \frac{2W}{T_n\left(z\right)}}={\displaystyle \frac{R_e^{\left(n\right)}\left(z\right)}{P_e\left(z\right)}}-{\displaystyle \frac{R_o^{\left(n\right)}\left(z\right)}{P_o\left(z\right)}},$$

(31)

and

$${\displaystyle \frac{2T_{n-1}\left(z\right)}{T_n\left(z\right)}}={\displaystyle \frac{R_e^{\left(n\right)}\left(z\right)}{P_e\left(z\right)}}+{\displaystyle \frac{R_o^{\left(n\right)}\left(z\right)}{P_o\left(z\right)}}.$$

(32)

To specify coefficients of $R_e^{\left(n\right)}\left(z\right)$, we use the identity

$$\prod _{s_k=1,k\ne i}(z-x_k)=z^{N_e-1}-(E_1-x_i)z^{N_e-2}+\cdots$$

$$+{(-1)}^{N_e-1}{E}_{N_e-1}-+{(-1)}^{N_e-2}{E}_{N_e-2}{x}_i+\cdots +x_i^{N_e-1},$$

where $E_r=E_r^{\left(n\right)},r=0,1,\dots ,N_e-1$ denotes the elementary symmetric function based on $x_i,s_i=1$. Because of (18) for $m\le N_o-1$,

$$R_e^{\left(n\right)}\left(z\right)=z^{N_e-1}-(E_1-{\delta }_1)z^{N_e-2}+\cdots +\sum _{r=0}^{N_e-1}{(-1)}^r{E}_r{\delta }_{N_e-1-r},$$

(33)

so that

$$z{R}_e^{\left(n\right)}\left(z\right)-P_e\left(z\right)={\delta }_1{z}^{N_e-1}+({\delta }_2-E_1{\delta }_1)z^{N_e-2}+\cdots +$$

$$\left(\sum _{r=0}^{N_e-2}{(-1)}^r{E}_r{\delta }_{N_e-1-r}\right)z+{(-1)}^{N_e-1}{E}_{N_e}.$$

Similarly, one obtains formulas for $R_o^{\left(n\right)}$ and $P_o$ via the elementary symmetric functions $D_r=D_r^{\left(n\right)},r=0,\dots ,N_o-1$ based on $x_j,s_j=-1$.

If $x_i,x_i<x_{i+1},i=1,\dots ,n-1$, represent order statistics, then $R_o^{\left(3\right)}\left(z\right)=1,$

$$R_e^{\left(4\right)}\left(z\right)=z-E_1+{\displaystyle \frac{E_2-D_2}{E_1-D_1}},$$

$$R_o^{\left(4\right)}\left(z\right)=z-D_1+{\displaystyle \frac{E_2-D_2}{E_1-D_1}}.$$

The ratio $R_e^{\left(n\right)}\left(z\right)/P_e\left(z\right)$ coincides with the Stiltjes transform of the discrete measure defined by the weights $\{2w_i^0,s_i=1\}$; $R_e^{\left(n\right)}\left(z\right)$ is associated with $P_e=P_e^{\left(n\right)}$. Thus, $R_e^{\left(n\right)}\left(z\right)/P_e\left(z\right)$ can be written as a finite continued fraction whose coefficients can be found from the three-term recurrence (24) for orthogonal polynomials on $\{x_i,s_i=1\}$ for this measure. Similar facts hold for $P_o=P_o^{\left(n\right)}$ and the probability distribution given by $\{2w_j^0,s_j=1\}$.

I now summarize the obtained results.

Theorem 5.

Let $T_m=T_m^{\left(n\right)},m=0,1,\dots ,n,$ be monic polynomials which are orthogonal under the norm defined by $\left\{w_j^0\right\}$. Then, (25) holds. For the polynomial associated with $T_{n-1}$, $R_{n-1}\left(z\right)=R_e^{\left(n\right)}\left(z\right)R_o^{\left(n\right)}\left(z\right)$, (27) and (28) are valid. The polynomials $R_e^{\left(n\right)}$ and $R_o^{\left(n\right)}$ defined by (29) and (30) satisfy (31)–(33).

3.4. Main Representation

The polynomials $R_e^{(n-1)}$ allow for expressing ${\delta }_m={\delta }_m^{\left(n\right)},m<N_e^{\left(n\right)}$, as a rational function of $x_n=x_{\left(n\right)}={max}_t{x}_t,$ namely,

$${\displaystyle \frac{{\delta }_m^{\left(n\right)}}{W^{\left(n\right)}}}={\displaystyle \frac{1}{W^{(n-1)}}}\left[{\displaystyle \frac{x_n^m{R}_e^{(n-1)}\left(x_n\right)}{P_e^{(n-1)}\left(x_n\right)}}-\sum _{r=0}^{m-1}{\delta }_r^{(n-1)}{x}_n^{m-1-r}\right].$$

(34)

To stress dependence on n, we write $N_e=N_e^{\left(n\right)},N_o=N_o^{\left(n\right)}$, $W=W^{\left(n\right)}$, and $s_j=s_j^{\left(n\right)}$. The functions on the right-hand side of (34) correspond to the sample of size $n-1$ obtained by deleting $x_n$ from the original dataset.

For the reduced sample, $N_e^{(n-1)}=N_o^{\left(n\right)},P_e^{(n-1)}\left(x\right)=P_o^{\left(n\right)}\left(x\right)$; if originally $s_j=s_j^{\left(n\right)}=-1,$ then $P_o^{(n-1)}\left(x_j\right)=-P_e^{\left(n\right)}\left(x_j\right)/(x_n-x_j),s_j^{(n-1)}=1$. Therefore, because of (18),

$${\displaystyle \frac{1}{W}}={\displaystyle \frac{R_e^{(n-1)}\left(x_n\right)}{W^{(n-1)}{P}_e^{(n-1)}\left(x_n\right)}}.$$

(35)

For fixed n and $1\le m\le N_e-1$,

$$z^m{\displaystyle \frac{R_e^{\left(n\right)}\left(z\right)}{P_e^{\left(n\right)}\left(z\right)}}=\sum _{r=0}^{m-1}{\delta }_r{z}^{m-1-r}+2W\sum _{s_i=1}{\displaystyle \frac{x_i^m}{(z-x_i)P_o\left(x_i\right)P_e^{\prime }\left(x_i\right)}},$$

implying (34).

Since $P_e^{\left(1\right)}\left(z\right)=z-x_1,P_e^{\left(2\right)}\left(z\right)=z-x_2,R_e^{\left(1\right)}\left(z\right)=R_e^{\left(2\right)}\left(z\right)=1$, the repeated use of (35) gives

$${\displaystyle \frac{1}{2W}}=\prod _{p=0}^{n-2}{\displaystyle \frac{R_e^{(n-p-1)}\left(x_{n-p}\right)}{P_e^{(n-p-1)}\left(x_{n-p}\right)}}={\displaystyle \frac{{\prod }_{p=0}^{n-2}{R}_e^{(n-p-1)}\left(x_{n-p}\right)}{|S|}}.$$

Here,

$$S=S^{\left(n\right)}=\mathrm{Res}(P_e,P_o)=\prod _{s_i=1,s_j=-1}(x_i-x_j)=\prod _{s_i=1}{P}_o\left(x_i\right)=\prod _{s_j=-1}{P}_e\left(x_j\right),$$

(36)

is the resultant of polynomials $P_e$ and $P_o$ (which are supposed not to have common roots). One can check by using (31) that

$$S=A\left(z\right)P_e\left(z\right)+B\left(z\right)P_o\left(z\right),$$

where the degree of polynomial $A\left(z\right)=-[S/\left(2W\right)]R_o^{\left(n\right)}\left(z\right)$ is $N_o-1$ and the degree of $B\left(z\right)=[S/\left(2W\right)]R_e^{\left(n\right)}\left(z\right)$ is $N_e-1$.

Actually, (34) and (35) hold if $x_n$ is replaced by any $x_k,s_k=1$, in which case $R_e^{(n-1)}$ refers to the monic polynomial of degree $N_o-1=N_e^{(n-1)}-1$ proportional to

$$\sum _{s_j=-1}\prod _{\ell \ne j,s_{\ell }=-1}\left({\displaystyle \frac{z-x_{\ell }}{x_j-{\mathbf{x}}_{\ell }}}\right){\displaystyle \frac{1}{{\prod }_{i\ne k,s_i=1}(x_j-x_i)}}.$$

It corresponds to the sample of size $n-1$ obtained by removing $x_k$, when $P_e^{(n-1)}\left(x_k\right)$ becomes $P_o^{\left(n\right)}\left(x_k\right)={\prod }_{s_j=-1}(x_k-x_j).$

In an alternative form of (34) for $x_{\ell },s_{\ell }=-1$, the monic polynomial is proportional to

$$\sum _{s_i=1}\prod _{k\ne i,s_k=1}\left({\displaystyle \frac{z-x_k}{x_i-{\mathbf{x}}_k}}\right){\displaystyle \frac{1}{{\prod }_{j\ne \ell ,s_j=-1}(x_i-x_j)}}$$

with ${\prod }_{s_i=1}(x_{\ell }-x_i)=P_e^{\left(n\right)}\left(x_k\right)$ instead of $P_o^{(n-1)}\left(x_{\ell }\right)$.

Our goal is to prove the following representation of ${\delta }_m$.

Theorem 6.

For any integer $m,0\le m<N_e^{\left(n\right)}$, $n\ge 3,$ with S defined in (36),

$${\delta }_m=2W{|S|}^{-1}{K}_m(x_1,\dots ,x_n).$$

Here, $K_m=K_m^{\left(n\right)}$ of $x_1,\dots ,x_n$ represents a homogeneous symmetric function of degree $m+p_n$—where $p_n={(n-2)}^2/4,n$ even and $p_n=(n-1)(n-3)/4,n$ odd—which is a linear combination of products of homogeneous symmetric polynomials in $\{x_i,s_i=1,i=1,\dots ,N_e\}$ and $\{x_j,s_j=-1,j=1,\dots ,N_o\}$ with integer coefficients. $K_m(-x_1,\dots ,-x_n)={(-1)}^m{K}_m(x_1,\dots ,x_n)$ if $n\ge 4$, with $K_m(x,\dots ,x)=0.$ The recursive Formula (40) relates $K_m^{\left(n\right)}(x_1,\dots ,x_n)$ to $K_r^{(n-1)}(x_1,\dots ,x_{n-1}),r=0,\dots ,m-1$, based on the reduced sample $x_t,1\le t\le n-1$. One has

$$\sum _i{\displaystyle \frac{\partial K_m(x_1,\dots ,x_n)}{\partial x_i}}=K_{m-1}(x_1,\dots ,x_n),$$

(37)

and

$$\sum _i{x}_i{\displaystyle \frac{\partial K_m(x_1,\dots ,x_n)}{\partial x_i}}=(m+p_n)K_m(x_1,\dots ,x_n).$$

(38)

Proof. 

As was already noticed, for the subsample $x_t,t=1,\dots ,n-1,$ the polynomial $P_e^{(n-1)}\left(z\right)$ coincides with $P_o\left(z\right)=P_o^{\left(n\right)}\left(z\right)$. The corresponding result is $S^{(n-1)}=S^{\left(n\right)}/{\prod }_{\ell \ne j,s_{\ell }=-1}(x_{\ell }-x_n)={(-1)}^{N_o-1}{S}^{\left(n\right)}/P_e^{(n-1)}\left(x_n\right)$.

Therefore, (35) implies that

$$K_0^{\left(n\right)}={\left({\displaystyle \frac{|S|}{2W}}\right)}^{\left(n\right)}=R_e^{(n-1)}\left(x_n\right)K_0^{(n-1)},$$

(39)

where the coefficients of the polynomial $R_e^{(n-1)}$ depend only on $x_1,\dots ,x_{n-1}$, and $K_p^{(n-1)}$ are evaluated on the sample of size $n-1$ obtained by removing $x_k$ (with $s_t^{(n-1)}=s_t^{\left(n\right)}$ for $t\ne k$). The homogeneity degree of $K_0$ is $N_e^{\left(n\right)}-1+N_e^{(n-1)}-1+\cdots +N_e^{\left(2\right)}-1=p_n=(N_e^{\left(n\right)}-1)(N_o^{\left(n\right)}-1)$. If $s_k=1$, then

$$K_0^{\left(n\right)}=\sum _{q+r\le N_o-1}{(-1)}^{N_o-1-q-r}{D}_{N_o-1-q-r}{x}^r,$$

Because of (34) for $1\le m\le N_e^{\left(n\right)}-1$,

$$K_m^{\left(n\right)}=K_0^{(n-1)}[x_n^m{R}_e^{(n-1)}\left(x_n\right)-x_n^{m-1}{P}_e^{(n-1)}\left(x_n\right)]$$

(40)

$$-P_e^{(n-1)}\left(x_n\right)\sum _{r=1}^{m-1}{K}_r^{(n-1)}{x}_n^{m-1-r}.$$

If $K^{\left(n\right)}={(K_0^{\left(n\right)},\dots ,K_{N_e-1}^{(n})}^T$ is an $N_e$-dimensional vector, then (39) and (40) mean that $K^{\left(n\right)}=A^{\left(n\right)}{K}^{(n-1)}$ where $A^{\left(n\right)}$ is a matrix of size $N_e\times N_o$ with elements

$$A_{mp}^{\left(n\right)}=\begin{array}{c}\sum _{r=m}^{N_o-1+m-p}{D}_{N_o-1+m-p}{x}^r,p\le m\le N_o-1\\ -\sum _{r=m-p-1}^{m-1}{D}_{N_o-1+m-p}{x}^r,0\le m\le p-1\end{array}.$$

The induction assumption that for $p<n$ all $K_m^{\left(p\right)},m\le N_e^{\left(p\right)}-1,$ can be represented as linear combinations of products of homogeneous symmetric polynomials in $\{x_i,s_i^{\left(p\right)}=1\}$ and $\{x_j,s_j^{\left(p\right)}=-1\}$ with integer coefficients implies that $K_m^{\left(n\right)},m\le N_e-1$ has the claimed properties. Namely, it is homogeneous of the stated degree, it is symmetric in $\{x_i,s_i=1,i=1,\dots ,N_e\}$, and $\{x_j,s_j=-1,j=1,\dots ,N_o\}$, and it can be written as a linear combination of products of homogeneous symmetric polynomials in these variables with integer coefficients.

To complete the proof, note that $K_m(x_1+c,\dots ,x_n+c)=K_m(x_1,\dots ,x_n)+\dots +c^m{K}_0(x_1,\dots ,x_n),$ so that (37) follows while (38) holds because of homogeneity. □

In particular, with non-negative and shift-invariant $K_0$ specified in (39) as a product of polynomials (29) evaluated at successive order statistics, one obtains the representation of (2) as follows:

$${\delta }^0={\delta }_1^{\left(n\right)}=x_n-{\displaystyle \frac{P_e^{(n-1)}\left(x_n\right)}{R_e^{(n-1)}\left(x_n\right)}}$$

(41)

with polynomials $R_e^{(n-1)}$ and $P_e^{(n-1)}$ based on $x_i,i=1,\dots ,n-1$, for which $s_i^{(n-1)}=1$.

Theorem 6 shows that ${\delta }_1$ has some resistance to extreme observations,

$$\underset{x_n\to \infty }{lim}{\delta }_1^{\left(n\right)}(x_1,\dots ,x_n)={\delta }_1^{(n-1)}(x_1,\dots ,x_{n-1}),$$

and

$$\underset{x_1\to -\infty }{lim}{\delta }_1^{\left(n\right)}(x_1,\dots ,x_n)={\delta }_1^{(n-1)}(x_2,\dots ,x_n).$$

Here are examples of Ks for smaller values of n given in terms of elementary symmetric functions:

$$n=3,p_3=0,K_0(x_1,x_2,x_3)\equiv 1,K_1(x_1,x_2,x_3)=x_2;$$

$$n=4,p_4=1,K_0(x_1,\dots ,x_4)=E_1^{\left(4\right)}-D_1^{\left(4\right)},$$

$$K_1(x_1,\dots ,x_4)=E_2^{\left(4\right)}-D_2^{\left(4\right)},$$

$$K_2(x_1,\dots ,x_4)=E_2^{\left(4\right)}{D}_1^{\left(4\right)}-D_2^{\left(4\right)}{E}_1^{\left(4\right)};$$

$$n=5,p_5=2,K_0(x_1,\dots ,x_5)=D_2^{\left(5\right)}-{\left[D_1^{\left(5\right)}\right]}^2+E_1^{\left(5\right)}{D}_1^{\left(5\right)}-E_2^{\left(5\right)},$$

$$K_1(x_1,\dots ,x_5)=D_2^{\left(5\right)}{D}_1^{\left(5\right)}-D_2^{\left(5\right)}{E}_1^{\left(5\right)}+E_3^{\left(5\right)}.$$

$$K_2(x_1,\dots ,x_5)=D_2^{\left(5\right)}{E}_2^{\left(5\right)}-D_1^{\left(5\right)}{E}_3^{\left(5\right)}-D_2^{\left(5\right)};$$

$$\begin{gathered} n=6,p_5=4,K_0(x_1,\dots ,x_6)=E_3(E_1+D_1)+E_2(E_1{D}_1+2D_2-D_1^2) \\ +E_1{D}_1{D}_2-{(E_2-D_2)}^2, \end{gathered}$$

$$\begin{gathered} K_1(x_1,\dots ,x_6)=E_3(-E_2+E_1{D}_1+D_2-D_1^2)-E_2{D}_2-(E_1^2-2D_2)D_3 \\ +E_1{D}_1{D}_3-D_2{D}_3, \end{gathered}$$

$$K_2(x_1,\dots ,x_6)=E_3{E}_2{D}_2-E_3^2{D}_1+E_3{D}_3(E_1+D_1)-E_2{D}_3(E_2-D_2)-E_1{D}_3^2.$$

4. Conclusions

The present work allows for obtaining meaningful consensus values under unreliable or missing uncertainties by using Bayes estimators (5) or (7). This method leads to mathematically challenging properties of self-dual weights (1) and their extension (4). The orthogonal polynomials that involve rank parity exhibit fascinating symmetry.

The approach explores non-traditional statistical estimation in the absence of variance information. The recursive algorithm detailed in Theorem 6 may be useful for practical calculations. The paper hints at new mathematical findings and sets the stage for a detailed exploration of a novel statistical methodology for estimating common parameters in the face of variance heterogeneity.