Defense of the Least Squares Solution to Peelle’s Pertinent Puzzle

Abstract

Generalized least squares (GLS) for model parameter estimation has a long and successful history dating to its development by Gauss in 1795. Alternatives can outperform GLS in some settings, and alternatives to GLS are sometimes sought when GLS exhibits curious behavior, such as in Peelle's Pertinent Puzzle (PPP). PPP was described in 1987 in the context of estimating fundamental parameters that arise in nuclear interaction experiments. In PPP, GLS estimates fell outside the range of the data, eliciting concerns that GLS was somehow flawed. These concerns have led to suggested alternatives to GLS estimators. This paper defends GLS in the PPP context, investigates when PPP can occur, illustrates when PPP can be beneficial for parameter estimation, reviews optimality properties of GLS estimators, and gives an example in which PPP does occur.

1. Introduction

Generalized least squares (GLS) for parameter estimation has a long and successful history dating to its development by Gauss in 1795. In some settings, alternatives to GLS can be effective, and are sometimes sought when GLS exhibits curious behavior, such as in Peelle's Pertinent Puzzle (PPP).

PPP was introduced in 1987 in the context of estimating fundamental parameters that arise in nuclear interaction experiments [1]. PPP is described below and when it occurs, the GLS estimate of the parameter is guaranteed to be outside the range of the data, which has elicited concerns that GLS is flawed and has led to suggested alternatives to GLS estimators.

A GLS estimate lying outside the range of the data causes heartache among nuclear scientists. Therefore, PPP continues to be of theoretical and practical interest. For example, a summary report of International Evaluation of Neutron Cross Section Standards [2] discusses PPP in terms of a standard least-squares procedure. Neutron cross sections are fundamental parameters that describe the probabilities of various neutron interactions. These cross sections are typically estimated from multiple experiments so some type of weighted average estimation scheme is used. The cross section estimates from any two experiments can have shared errors arising for example from using the same measured background, which can lead to covariance structures such as those described below.

We quote the original PPP problem proposed by [1] from a report of Chiba and Smith [3], “Suppose we are required to obtain the weighted average of two experimental results for the same physical quantity. The first result is 1.5, and the second result 1.0. The full covariance matrix of those data is believed to be the sum of three components. The first component is fully correlated with standard error 20% of each respective value. The second and third components are independent of the first and of each other, and correspond to 10% random uncertainties in each experimental result.

Although this PPP statement is vague, by converting it to something more interpretable, GLS can be applied and the resulting estimate is 0.88 (with an associated standard deviation of 0.22), which is outside the range of the measurements. Zhao and Perey [4] re-interpreted PPP by introducing a third datum c, through which the common error can be explicitly specified as follows, “Suppose we have two independent measurements. One is m₁ = 1.5 ± 10%. Another is m₂ = 1.0 ± 10%. To convert this quantity into another physical quantity, we need a conversion factor c, which after intermediate steps omitted here is 1.0 with uncertainty of 20%. Now the experimental results are, y₁ = cm₁ = 1.5 and y₂ = cm₂ = 1.0. We are required to obtain the weighted average of those experimental data.

In this interpretation, the common error (the “fully correlated” component) is understood to be multiplicative, and m₁ = 1.5 ± 10% is assumed to mean that the true standard deviation is 0.15 for m₁ (and 0.10 for m₂). Even after these interpretations, some vagueness remains. There is no convention regarding what confidence is associated with ±10%. Nor is there a convention for whether the standard deviation includes all error sources, or only includes random error effects, ignoring accuracy. In addition, we show below that it can matter whether the standard deviation is expressed as a fraction of the true quantity or of the measured quantity.

One of our contributions is to make explicit assumptions and examine their implications in order to convert vague statements to statements about which it is possible to find agreement among physical scientists and statisticians regarding suitable approaches. We also defend GLS in the PPP context, illustrate when PPP can be beneficial, briefly describe properties of GLS estimators, show that PPP cannot occur for certain measurement error models, and calculate a covariance matrix Σ for y₁ and y₂ for which PPP occurs that follows from a physical description of a realistic measurement scenario.

2. PPP

The vagueness of the original PPP statement is one reason there are so many interpretations of PPP [5]. PPP can occur if there is a large positive covariance between y₁ and y₂ and the variances of y₁ and y₂ are very different.

Let Σ be the 2-by-2 symmetric covariance matrix for y₁ and y₂ with diagonal entries ${\sigma }_1^2$, ${\sigma }_2^2$, and off-diagonal entry σ₁₂, which denote the variance of y₁, the variance of y₂, and the covariance of y₁ and y₂, respectively. Zhao and Perry [4] approximated Σ for their definition of PPP (using y₁ = cm₁ and y₂ = cm₂ as described in the Introduction) as

$$\begin{array}{ll}\sum & =\text{Cov}(y_1,y_2)\\ & =\left(\begin{array}{cc}{\sigma }_1^2& {\sigma }_{12}\\ {\sigma }_{12}& {\sigma }_2^2\end{array}\right)\\ & \approx {\mu }_c^2\left(\begin{array}{cc}{\sigma }_{m1}^2& \\ & {\sigma }_{m2}^2\end{array}\right)+{\sigma }_c^2\left(\begin{array}{cc}{\mu }_m^2& {\mu }_m^2\\ {\mu }_m^2& {\mu }_m^2\end{array}\right)\\ & \approx \left(\begin{array}{cc}0.1125& 0.06\\ 0.06& 0.05\end{array}\right)\end{array}$$

(1)

where σ_m1, σ_m2, and σ_c are the standard deviations in the quantities m₁, m₂, and c, respectively. The first approximation arises because the relatively small term ${\sigma }_c^2{\sigma }_m^2$ was omitted (see below). The second approximation follows by approximating ${\mu }_m^2$ by m₁m₂. Equation 1 assumes that m and c are independent. And, the relative standard deviations of 10%, 10%, and 20% are assumed to be the fractions relative to the measured values of 1.5, 1.0, and 1.0, respectively by definition, so that σ_m1 = 0.15, σ_m2 = 0.10, and σ_c = 0.2. See Section 4 for further discussion.

Readers might find it informative that those with traditional statistical education among the authors were the most willing to accept GLS estimates despite the apparent flaw of lying outside the range of the data. Statisticians will often consider alternatives to GLS, but recognize that GLS estimation is difficult to beat, at least in terms of typical performance measures such as being close on average to the true parameter value over hypothetical repeats of the pair of experiments [6]. Also, note that because $\frac{y_1-y_2}{{\sigma }_{y_1-y_2}}=0.5/0.21=2.39$, one might consider the original PPP to be an unusual data realization. However, we show using elementary algebra and error modeling in Theorem 2 that when PPP occurs, it occurs for all data realizations.

Although Peelle [1] originally constructed the covariance matrix Σ as in Equation 1 and other authors followed, this result is not exact if the common error is multiplicative because of the omitted term ${\sigma }_c^2{\sigma }_m^2$ and because of the need to estimate μ_m. If we include the ${\sigma }_c^2{\sigma }_m^2$ term, then

$$\text{Var}(cm)={\mu }_c^2{\sigma }_m^2+{\mu }_m^2{\sigma }_c^2+{\sigma }_c^2{\sigma }_m^2$$

(2)

where σ_m is σ_m1 or σ_m2 in Equation 1. The covariance matrix is therefore estimated as

$$\begin{array}{ll}\sum & ={\mu }_c^2\left(\begin{array}{cc}{\sigma }_{m1}^2& \\ & {\sigma }_{m2}^2\end{array}\right)+{\sigma }_c^2\left(\begin{array}{cc}{\mu }_m^2+{\sigma }_{m1}^2& {\mu }_m^2\\ {\mu }_m^2& {\mu }_m^2+{\sigma }_{m2}^2\end{array}\right)\\ & \approx \left(\begin{array}{cc}0.1134& 0.06\\ 0.06& 0.0504\end{array}\right)\end{array}$$

(3)

where again as in Equation 1, μ_m is estimated by m₁ or m₂.

As shown below, prior to substituting the approximation for ${\mu }_m^2$, Equations 1 and 3 satisfy conditions under which PPP cannot occur (Theorem 2 in Section 3). However, because in all published investigations we are aware of, ${\mu }_m^2$ is approximated by m₁m₂ in the off-diagonal and ${\mu }_m^2$ is approximated by $m_1^2$ in the upper left entry in the second term and by $m_2^2$ in the lower right entry in the second term, Σ as estimated does not have the condition referred to in Theorem 2 below Therefore, due to estimation error in Σ, PPP does appear to occur, which means that the GLS estimate of μ is outside the range of the data for Σ as given by Equation 1 or Equation 3.

The fact that μ_m is approximated in the context of estimating variance and covariance in a multiplicative error model raises at least three issues: (1) the issue of how uncertainty in measurement is expressed; (2) an issue related to simulating observations from the assumed measurement error model as a way to consider likelihoods other than the Gaussian, and (3) accounting for uncertainty in Σ. For issue (1), we will make our measurement error model assumptions explicit throughout in order to eliminate needless ambiguities. Issues (2) and (3) are investigated in [6].

To focus on GLS behavior when PPP occurs, this paper assumes Σ is known exactly without error. However, for historical and presentation purposes, Equations 1 and 3 are presented, and both clearly involve approximations. In contrast, Theorem 3 illustrates a measurement error model for which the exact covariance can satisfy the PPP condition.

3. GLS

It is well known that the GLS method can be applied to y₁ and y₂ to obtain the best linear unbiased estimate (BLUE) μ̂ of μ [7]. Here, “best” means minimum variance and unbiased means that on average (across hypothetical or real realizations of the same experiment), the estimate μ̂ will equal its true value μ. The GLS estimate for μ arising from the model

$$\left(\begin{array}{c}{y}_1\\ y_2\end{array}\right)=\left(\begin{array}{c}\mu +e_1\\ \mu +e_2\end{array}\right)$$

(4)

with Σ = Cov(e₁, e₂) = Cov(y₁, y₂) given by

$$\widehat{\mu }=c{G}^t{\sum }^{-1}\left(\begin{array}{c}{y}_1\\ y_2\end{array}\right)$$

(5)

where the scalar c = (G^tΣ⁻¹G)⁻¹. And the variance σ² of the GLS estimate μ̂ is given by

$${\sigma }^2={\left(G^t{\sum }^{-1}G\right)}^{-1}$$

(6)

where the matrix G is given by G^t = (1, 1). Putting the covariance of Equation 1 into Equations 5 and 6 gives μ̂ = c₁y₁ + c₂y₂ = 0.88 and σ = 0.22 where c₁ = −0.24 and c₂ = 1.24. Notice that c₁ + c₂ = 1 (so that μ̂ is unbiased) but also that c₁ < 0 and that μ̂ is smaller than each of the two measured values, y₁ = 1.5 and y₂ = 1.0.

GLS is usually introduced in the context of estimating β and future y values in a linear regression relating the response y to predictors X via y = Xβ + ∊ ([7]). Therefore, in Equation 4, the mean μ plays the role of the unknown β. The GLS solution in Equations 5 and 6 then follows from standard calculus or projection matrix results. For example, one can write μ̂ = a₁y₁ + (1 − a₁)y₂, note that $\text{var}\left(\widehat{\mu }\right)=a_1^2{\sigma }_{y1}^2+{(1-a_1)}^2{\sigma }_{y2}^2+2a_1(1-a_1){\sigma }_{y1,y2}^2$ and solve for a₁ to minimize var(μ̂) by setting the derivative of var(μ̂) with respect to a₁ to 0.

The GLS solution of Equations 5 and 6 with covariance Σ of Equation 3 is 0.89 ± 0.22. Because the last term in Equation 2 is smaller than the others, the Σ in Equation 3 that is slightly modified compared to the Σ in Equation 1 still leads to PPP. But in Section 4 we describe other modifications to Σ that do not lead to PPP.

GLS estimation has a long and successful history, but met with serious objection within the nuclear physics community in the context of combining estimates from multiple experiments upon observing a tendency to produce estimates that are outside the range of the data. More specifically, to date the tendency has been to produce estimates that are less than the minimum data value, so have been criticized as being “too small” [2].

GLS estimation is guaranteed to produce the BLUE even if the underlying data are not Gaussian. However, if the data is not Gaussian, then the minimum variance unbiased estimator (MVUE) is not necessarily linear in the data. Also, though unbiased estimation might sound politically correct, it is not necessarily superior to biased estimation [8]. Therefore, PPP has motivated the nuclear physics community to consider estimators other than GLS.

If the data has a Gaussian distribution, then it is well known that the GLS estimate is the same as the maximum likelihood (ML) estimate. This is because the log of the Gaussian likelihood involves a sum of squares, so choosing an estimate (the GLS estimate) that minimizes a sum of squares corresponds to choosing an estimate (the ML estimate) that maximizes the likelihood. Ordinary LS (OLS), weighted LS (WLS), and GLS are all essentially the same technique, but OLS is used if Σ is proportional to a unit-diagonal matrix, WLS is used if Σ is proportional to a diagonal matrix, and GLS is used if Σ is an arbitrary positive definite covariance matrix. The Gauss-Markov theorem [7] proves that the OLS estimator is the BLUE, and very similar theorems prove the same result for WLS or GLS.

The ML estimate depends on the assumed distributions for the errors. For example, if we replace the Gaussian (Normal) distributions with logNormal distributions, the ML estimate will change. In the cases considered here, ML gives the same estimate as GLS, because the data distribution is Gaussian. Because the ML approach makes strong use of the assumed error distributions, the ML estimate is sensitive to the assumed error distribution. The ML method has desirable properties, including asymptotically minimum variance as the sample size increases. However, in our example, the sample size is tiny (two), so asymptotic results for ML estimates are not relevant. It still is possible that an ML estimator will be better for nonGaussian data than GLS [6]. Typically, “better” is defined as the mean squared error (MSE) of the estimator, which is well known to satisfy MSE = variance + bias². In some cases, biased estimators have lower MSE than unbiased estimators because the bias introduced is more than offset by a reduction in variance [8].

4. Closer Look into PPP

The original PPP does not clearly state whether the common error is additive or multiplicative. This ambiguity was examined in [5]. In the “additive” scenario case, y₁ = m₁ + b and y₂ = m₂ + b, and the source of correlation may be a common background measurement b. And, the situation is somewhat different from PPP, because if σ_b is 20% of y₁ = 1.5, then it is 30% of y₂ = 1.0. The covariance matrix for this case is

$$\begin{array}{ll}\sum & =\left(\begin{array}{cc}{\sigma }_{m1}^2& \\ & {\sigma }_{m2}^2\end{array}\right)+{\sigma }_b^2\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)\\ & =\left(\begin{array}{cc}0.1125& 0.09\\ 0.09& 0.1\end{array}\right)\end{array}$$

(7)

and the GLS solution is 1.15 ± 0.31. With the covariance matrix of Equation 7, PPP does not occur, and the GLS solution of 1.15 is a weighted average of 1.0 ± 0.1 and 1.5 ± 0.15. Although as σ_b changes, the standard deviation σ_μ̂ is scaled accordingly, the GLS solution does not change. It is reasonable that the GLS solution does not change as σ_b changes, because whatever the background fluctuation is, y₁ and y₂ are impacted in the same manner by b.

If instead σ_b is 20% of μ_m, then its value is unknown because μ_m is unknown and must be estimated. However, regardless of the value of σ_b > 0, PPP cannot occur in the additive case as parameterized in Equation 7 (Theorem 1 below).

In Sivia's [9] notation, Equation 5 can be written as

$$\widehat{\mu }=\frac{a_1{y}_1+a_2{y}_2}{a_1+a_2}$$

(8)

with

$$\begin{array}{cc}{a}_1={\sigma }_2({\sigma }_2-\rho {\sigma }_1),& a_2={\sigma }_1({\sigma }_1-\rho {\sigma }_2)\end{array}$$

(9)

where ρ is the correlation coefficient. Sivia demonstrated that PPP does not occur if the condition

$$\rho \le min\left(\begin{array}{cc}\frac{{\sigma }_1}{{\sigma }_2},& \frac{{\sigma }_2}{{\sigma }_1}\end{array}\right)$$

(10)

is fulfilled (and does occur if the condition does not hold). It is not difficult to show that the additive case of Equation 7 satisfies this condition, which we state as Theorem 1.

Theorem 1

If Σ is given by

$$\sum =\left(\begin{array}{cc}{\sigma }_{m1}^2& \\ & {\sigma }_{m2}^2\end{array}\right)+{\sigma }_b^2\left(\begin{array}{cc}1& 1\\ 1& 1\end{array}\right)$$

(11)

then PPP does not occur.

Proof

The covariance matrix Σ can be expressed as

$$\sum =\left(\begin{array}{cc}{\sigma }_1^2& \rho {\sigma }_1{\sigma }_2\\ \rho {\sigma }_1{\sigma }_2& {\sigma }_2^2\end{array}\right)$$

(12)

with $\rho =\frac{{\sigma }_b^2}{{({\sigma }_1^2+{\sigma }_b^2)}^{1/2}{({\sigma }_2^2+{\sigma }_b^2)}^{1/2}}$. If ${\sigma }_1^2<{\sigma }_2^2$ then $\rho \le min\left(\begin{array}{cc}\frac{{\sigma }_1}{{\sigma }_2},& \frac{{\sigma }_2}{{\sigma }_1}\end{array}\right)$ is equivalent to ${\sigma }_b^4\le ({\sigma }_1^2+{\sigma }_b^2)\times ({\sigma }_1^2+{\sigma }_b^2)$ which obviously holds. The proof is identical if ${\sigma }_1^2>{\sigma }_2^2$. And if ${\sigma }_1^2={\sigma }_2^2$, then PPP can never occur.

Notice that Equations 1 and 3 have the same form as assumed in Theorem 1. Therefore, PPP also cannot occur for the multiplicative error model assumed. However, as discussed, PPP can occur with the multiplicative error model case if Σ is estimated as in Equations 1 and 3.

Several authors explored different values for the covariance matrix Σ to understand the relationship between the covariance matrix and the estimates. Some numerical examples are in [2]. Jones et al. [10] and Finn et al. [11] also reported linear regression of strongly correlated data and emphasized that if ${\sigma }_1^2\ne {\sigma }_2^2$ then contrary to common belief, positive or negative correlation can be exploited to produce an unbiased GLS estimate that has lower variance than in the zero correlation case. The historical definition of PPP requires the correlation to be positive because only in that case will the GLS estimate lie outside the range of the data for certain Σ.

Next rewrite Equation 8 as $\widehat{\mu }=a_1^{\prime }{y}_1+a_2^{\prime }{y}_2$ with $a_1^{\prime }=\frac{a_1}{a_1+a_2}$ and $a_2^{\prime }=\frac{a_2}{a_1+a_2}$. A condition equivalent to Equation 10 is obtained as follows. By setting the first derivative with respect to $a_1^{\prime }$ to zero, one can solve for the value of $a_1^{\prime }$ such that the variance of $a_1^{\prime }{y}_1+a_2^{\prime }{y}_2$ is minimum subject to $a_1^{\prime }+a_2^{\prime }=1$. The result is $a_1^{\prime }=\frac{{\sigma }_2^2-{\sigma }_{12}^2}{{\sigma }_1^2+{\sigma }_2^2-2{\sigma }_{12}^2}$. Therefore, a condition equivalent to Equation 10 is: PPP occurs if $a_1^{\prime }<0$ or $a_1^{\prime }>1$. Another interesting fact is that if PPP occurs (so Equation 10 does not hold), then either $a_1^{\prime }<0$ and $a_2^{\prime }>0$ or $a_1^{\prime }>0$ and $a_2^{\prime }<0$, with $a_1^{\prime }+a_2^{\prime }=1$ so that μ̂ is unbiased. We can now state and prove Theorem 2.

Theorem 2

Suppose $a_1^{\prime }$ and $a_2^{\prime }$ have opposite signs. Then either μ̂ < min(y₁, y₂) or μ̂ > max(y₁, y₂). That is, μ̂ will always fall outside the range of the (y₁, y₂).

Proof

First assume $a_1^{\prime }>0$ and $a_2^{\prime }<0$. If y₁ < y₂ then $\widehat{\mu }=a_1^{\prime }{y}_1+a_2^{\prime }{y}_2<a_1^{\prime }{y}_1+a_2^{\prime }{y}_1=y_1$ because $a_2^{\prime }<0$. If y₁ > y₂ then $\widehat{\mu }=a_1^{\prime }{y}_1+a_2^{\prime }{y}_2>a_1^{\prime }{y}_1+a_2^{\prime }{y}_1=y_1$ because $a_2^{\prime }<0$. The proof is completed by next assuming $a_1^{\prime }<0$ and $a_2^{\prime }>0$, and following similar steps.

4.1. Additional Support for GLS by Numerical Example

The fact that the GLS estimate is the BLUE estimate (and also the MVUE estimate if the data is Gaussian) and that it lies below the range of (y₁, y₂) suggests two features. First, μ must be likely to fall outside the range of (y₁, y₂). Second, there must better than random chance capability to guess on which side of the range of (y₁, y₂) that μ lies.

In addition to GLS's BLUE property, we can add support for GLS by numerical example to illustrate features one and two. Figure 1 plots the contours of the bivariate normal density having Σ given by Equation 1 and μ = 0.88 which as shown previously is the OLS estimate of the mean in the case y₁ = 1.5 and y₂ = 1.0. Informally, we can integrate this density over regions 1 and 3 to see that there is a large probability that both y₁ and y₂ lie either above the mean or below the mean, so indeed μ is likely to fall outside the range of (y₁, y₂). Integration of the bivariate normal for Σ given by Equation 1 indicates that with probability approximately 0.40, μ lies below the minimum of (y₁, y₂) and with the same probability μ lies above the maximum of (y₁, y₂). This is a total of approximately 0.80 probability that μ falls outside the range of (y₁, y₂), which is an example of feature one. Having an estimate lie outside the range of the data is therefore defensible, provided (feature two) that one can guess with better than random chance performance whether μ lies below the minimum or whether μ lies above the maximum of (y₁, y₂). To see that one can beat random guessing performance, suppose y₁ > y₂ as in our case (y₁ = 1.5 and y₂ = 1.0). Then, because ${\sigma }_{y1}^2=0.1125$ is larger than ${\sigma }_{y2}^2=0.05$ in Equation 1, μ is more likely to fall below y₂ because if instead μ > y₁ then the distance from y₁ to μ would be smaller than the distance from y₂ to μ, contradicting the fact that ${\sigma }_{y1}^2>{\sigma }_{y2}^2$. To confirm this line of reasoning, in 10,000 simulations (in the statistical computing language R) of (y₁, y₂) pairs having μ = 0.88, 57% of the simulation runs for which y₁ > y₂ did in fact also have μ < y₂. On the basis of 10,000 simulations, 57% is repeatable to within ± 1% or less, so this is better than random chance (50%) guessing. This is not a formal proof but does suggest a direction to understand when a GLS estimate falling outside the range of the data is effective. Note that y₁ = 1.5 and y₂ = 1.0 in the PPP statement, and the GLS estimate is μ̂ = 0.88 < y₂.

5. Example Where PPP Occurs without Approximation

Thus far we have not demonstrated any error model which exactly (without approximation) produces a Σ that leads to PPP. A situation that leads to PPP without the approximations in Equation 1 is expressed in Theorem 3.

Theorem 3

Suppose m₁ = μ + ∊_R1 and m₂ = μ + ∊_R2 where ∊_R1 is random error in m₁ with variance ${\sigma }_{R1}^2$ and similarly for ∊_R2. Then if y₁ = m₁ + ∊_S and y₂ = m₂ + α∊_S, where ∊_S ∼ N(0, σ_S) and α is any positive scale factor other than 1, the covariance matrix Σ of (y₁, y₂) can lie in the PPP region.

Proof

The proof is by demonstration. Specify any values σ₁, σ₂, and σ₁₂ that satisfy the PPP condition $(\rho >min\left(\begin{array}{cc}\frac{{\sigma }_1}{{\sigma }_2},& \frac{{\sigma }_2}{{\sigma }_1}\end{array}\right))$. Choose any α > 0, then ${\sigma }_S^2={\sigma }_{12}^2/\alpha$, ${\sigma }_{R1}^2={\sigma }_1^2-{\sigma }_S^2$, and ${\sigma }_{R1}^2={\sigma }_2^2-{\alpha }^2{\sigma }_S^2$. As an example, let ${\sigma }_1^2=0.1134$, ${\sigma }_2^2=0.0504$, and ${\sigma }_{12}^2=0.06$ as in Equation 3. Then if α = 0.7, we have ${\sigma }_S^2={\sigma }_{12}^2/\alpha =0.06/0.7=0.0857$, ${\sigma }_{R1}^2={\sigma }_1^2-{\sigma }_S^2=0.1134-0.0857=0.0277$, and ${\sigma }_{R2}^2={\sigma }_2^2-{\alpha }^2{\sigma }_S^2=0.06-{0.7}^2\times 0.06/0.7=0.018$.

Note:

• If α = 1, then Σ has the form of Equation 11, so PPP cannot occur ([2]).

• If α < 0 then PPP cannot occur. However, our applications have α > 0. Jones et al. [10] showed that if ρ → ±1 and σ₁ ≠ σ₂, then μ can be estimated with surprisingly small variance. That fact plus the known BLUE property of GLS could convince us to just “live with” the PPP because it can make sense for the GLS estimate μ̂ to lie outside the range of (y₁, y₂) in the case α > 0, or σ₁₂ > 0.

One example in which the assumptions of Theorem 3 hold involves subtracting a background measurement from a region of interest (ROI) measurement to get a net result. Because the background measurement often involves a different number of channels than the ROI measurement, a scale factor k is introduced to estimate the net counts as net = peak − k × background. Figure 2 illustrates a hypothetical example where each of three peak ROIs have a corresponding background measurement in the plot of the square root of detected neutron counts versus neutron energy in arbitrary units (au).

Suppose each ROI and corresponding background are analyzed separately, and consider the first ROI in Figure 2. The count times could vary between the two experiments, so ${\sigma }_{R1}^2\ne {\sigma }_{R2}^2$. Both experiment 1 and experiment 2 measure the ROI counts, but in many situations, the background measurement is made only by experiment 1. In that case, experiments 1 and 2 must use the background measurement made by experiment 1. Also, if the ROI is found by analyzing the shape of the curve (the “spectrum”) that describes detected count rates versus particle energy, then the ROI for experiments 1 and 2 could differ. We then have N₁ = G₁ − a₁ × B and N₂ = G₂ − a₂ × B, where N is net counts, G is gross counts, B is background, a₁ is the scale factor for experiment 1, and a₂ is the scale factor for experiment 2. The scale factor a₁ for experiment 1 is the ratio of the number of ROI channels to the number of background channels, and similarly for the scale factor a₂ for experiment 2. The channel counts have variation from repeat to repeat so the detected counts will vary around the true counts with some error. As an aside, often the channel counts have approximately a Poisson distribution which for large count rates is well approximated by a Gaussian distribution. Regardless of which probability distribution best describes the channel counts, there are measurement errors in N₁, N₂, and one can divide N₁ = G₁ − a₁ × B by a₁ to convert this pair of equations to those assumed in Theorem 3.

6. Conclusions

Because Peelle's original statement is vague, there have been several interpretations and solutions. In our experience, there is considerable variation among experimentalists in the expression of measurement uncertainty, and a wide the range of analyses can result from vague uncertainty statements.

The three main contributions of this paper are: (1) illustrating examples when PPP cannot occur (Theorem 1); (2) providing insight when PPP is effective and appropriate (related to Theorem 2), and (3) deriving a realistic covariance matrix Σ for which PPP occurs according to physical descriptions of realistic measurement scenarios (Theorem 3). We also showed via numerical integration that an estimate lying outside the range of the data is sensible. This is because the unequal variances of x₁ and x₂ provide information regarding whether μ is more likely to be less than the minimum or greater than the maximum of x₁ and x₂.

Of course GLS provides a good estimate μ̂ in general because of its well-known BLUE property (and MVUE if the data is Gaussian) and in particular for the PPP problem if the covariance Σ is well known.

There will almost always be estimation error in Σ̂, and often the measurement errors are nonGaussian. Therefore, we consider the following two topics in [6]: (1) alternatives to GLS when there is estimation error in Σ̂, and to provide estimators other than GLS that use the estimated likelihood in the case of non-Gaussian error models that make ML estimation difficult. Regarding (1), it is known that weighted estimates do not always outperform equally-weighted estimates when there is estimation error in the weights. We have already noted that estimation of Σ in Equation 1 introduces apparent PPP when PPP does not actually occur.

Finally, [10] showed that when the diagonal entries in Σ are different and the correlation is large and positive, the GLS estimate can have lower variance than if the correlation is zero. This fact does not seem to be widely known, and experimental opportunities to exploit this fact are under investigation.