# Recent Extensions to the Cochran–Mantel–Haenszel Tests

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. The CMH Tests

_{ihj}} of count data of a particular structure. Specifically, N

_{ihj}counts the number of times treatment i is classified into outcome category h in the jth stratum, i = 1, …, t, h = 1, …, c and j = 1, …, b. Strata are independent and the treatments present in each stratum are fixed by design. As is usual, note that N

_{ihj}is a random variable and n

_{ihj}is a particular value of that random variable.

- the strata totals ${\sum}_{i,h}{N}_{ihj}}={n}_{\u2022\u2022j$,
- the treatment totals within strata, ${\sum}_{h}{N}_{ihj}}={n}_{i\u2022j$, and
- the outcome totals within strata, ${\sum}_{i}{N}_{ihj}}={n}_{\u2022hj$

- overall partial association (OPA) (S
_{OPA}is asymptotically distributed as ${\chi}_{b(c-1)(t-1)}^{2}$) - general association (GA) (S
_{GA}is asymptotically distributed as ${\chi}_{(c-1)(t-1)}^{2}$) - mean scores (MS) (S
_{MS}is asymptotically distributed as ${\chi}_{t-1}^{2}$) and - correlation (C) (S
_{C}is asymptotically distributed as ${\chi}_{1}^{2}$).

_{GA}takes the value 19.76 with ${\chi}_{4}^{2}$ p-value 0.0006, S

_{MS}takes the value 17.94 with ${\chi}_{2}^{2}$ p-value 0.0001, and S

_{C}takes the value 16.83 with ${\chi}_{1}^{2}$ p-value less than 0.0001. From these tests, we conclude there is strong evidence of an association between the proposition responses and religion. In particular, there is evidence of mean differences in the responses and of a (linear-linear) correlation between responses and religion.

_{MS}= 9.6177 with ${\chi}_{2}^{2}$ p-value 0.0082 and S

_{C}= 1.1029 with ${\chi}_{1}^{2}$ p-value 0.2936. There is evidence of a mean effect; on average the jams are different. However, there is no evidence of a correlation effect: As we pass from jam A to B and then to C there is no evidence of an increasing (or decreasing) response.

## 3. The Nominal CMH Tests

_{ihj}follow an extended hypergeometric distribution. Moreover, since the strata are mutually independent, a product distribution is appropriate for the collection of these strata counts.

_{GA}, first define the vector of counts on the jth stratum U

_{j}= ${({N}_{11j},\dots ,{N}_{1cj},\dots ,{N}_{t1j},\dots ,{N}_{tcj})}^{\mathrm{T}}$. Summing over strata gives

_{GA}has the ${\chi}_{(c-1)(t-1)}^{2}$ distribution. The test statistic is symmetric in the treatments and outcome categories and independent of the choice of the dependent variables.

_{GA}is too complicated for routine hand calculation; it is almost always applied using software in packages such as R.

_{OPA}, the vector of the quadratic form involves the aggregation of the U

_{j}via U = ${({U}_{1}^{\mathrm{T}},\dots ,{U}_{b}^{\mathrm{T}})}^{\mathrm{T}}$. Again, the covariance matrix is calculated using the product extended hypergeometric distribution. The overall partial association statistic so derived is given by

_{OPA}and T

_{OPA}is merely the factor $({n}_{\u2022\u2022j}-1)/{n}_{\u2022\u2022j}$ applied to each stratum. For large stratum counts, this will make little difference in the values of S

_{OPA}and T

_{OPA}.

_{ih}

_{•}}, T

_{GA}say. The Pearson test statistics T

_{OPA}and T

_{GA}will have the same asymptotic distributions as the corresponding conditional tests. Most users will have more familiarity with the unconditional tests and most packages will have routines for their calculation even if they don’t have routines for the CMH tests.

## 4. The Ordinal CMH Tests

#### 4.1. CMH Mean Scores Test

_{ihj}counts the number of times treatment i is classified into outcome category h in the jth stratum, i = 1, …, t, h = 1, …, c and j = 1, …, b. Assume that outcomes are ordinal and assign the score b

_{hj}to the hth response on the jth stratum. All marginal totals are assumed to be known, so the product extended hypergeometric model is assumed. The score sum for treatment i in stratum j is ${M}_{ij}={\displaystyle {\sum}_{h=1}^{c}{b}_{hj}{N}_{ihj}}$. Taking expectations $\mathrm{E}\left[{M}_{ij}\right]={n}_{i\u2022j}{\displaystyle {\sum}_{h=1}^{c}{b}_{hj}{n}_{\u2022hj}/{n}_{\u2022\u2022j}}$ since $\mathrm{E}\left[{N}_{ihj}\right]={n}_{i\u2022j}{n}_{\u2022hj}/{n}_{\u2022\u2022j}$. If M

_{j}= (M

_{ij}) then the inference is based on $M={\displaystyle {\sum}_{j=1}^{b}{M}_{j}}$ through the quadratic form

_{j}), first define

_{j}) is not of full rank, the usual approaches, such as dropping appropriate treatment and/or outcome categories, or using a generalised inverse, can be used.

_{MS}({b

_{hj}}) to emphasise this dependence.

_{j}) requires routine but tedious algebra. If δ

_{uv}is the Kronecker delta, = 1 if u = v and zero otherwise, using standard distribution theory for the product extended hypergeometric distribution, $\mathrm{E}\left[{N}_{ihj}\right]={n}_{i\u2022j}{n}_{\u2022hj}/{n}_{\u2022\u2022j}$ and the covariance between N

_{ihj}and ${N}_{{i}^{\prime}{h}^{\prime}j}$ is

_{MS}can be shown to be asymptotically ${\chi}_{t-1}^{2}$; see Reference [4].

#### 4.2. The CMH Correlation Test

_{hi}, i = 1, …, t, and on the jth stratum the response scores are b

_{hj}, j = 1, …, b.

- ${C}_{j}={\displaystyle {\sum}_{i}{\displaystyle {\sum}_{h}{a}_{hi}{b}_{hj}\{{N}_{ihj}}}-\mathrm{E}[{N}_{ihj}]\}\text{}\mathrm{and}$
- $C={\displaystyle {\sum}_{j}{C}_{j}}$.

^{2}/var(C) = S

_{C}say. The derivation of var(C) is relatively complex if scalars are used but is routine using Kronecker products.

_{j}= (a

_{1j}, …, a

_{tj})

^{T}, b

_{j}= (b

_{1j}, …, b

_{cj})

^{T}and N

_{j}= (N

_{11j}, …, N

_{1cj}, …, N

_{t}

_{1j}, …, N

_{tcj})

^{T}. Then ${C}_{j}={({a}_{j}\otimes {b}_{j})}^{\mathrm{T}}({N}_{j}-\mathrm{E}[{N}_{j}])$ and

_{C}is now fully specified.

_{C}has asymptotic distribution ${\chi}_{1}^{2}$. Again, see Reference [4].

## 5. Alternative Presentations of the Ordinal CMH Test Statistics

#### 5.1. The CMH Correlation Statistic

_{Tj}and V

_{Cj}we have

_{XXj}, S

_{XYj}and S

_{YYj}give unbiased estimators of the stratum variances and covariances. With these definitions,

_{XXj}, S

_{XYj}and S

_{YYj}familiar in formulae for regression coefficients.

- (1)
- The data consists of one stratum only;
- (2)
- the treatment scores are independent of the strata: a
_{ij}= a_{i}for all i and j; - (3)
- the randomised block design (in Section 5.2).

_{XX}is constant over strata. This gives a slight simplification of the S

_{C}formula. See the Jams Example below. If the data come from a randomised block design, a considerable simplification is possible if the same treatment and response scores are used on each stratum or block. See Section 5.2.

_{P}is the Pearson correlation coefficient. This is well known. See, for example, Reference [6], p. 253.

_{Pj}for the Pearson correlation in the jth stratum, it follows that since ${S}_{XYj}={r}_{\mathrm{P}j}{S}_{XXj}{S}_{YYj}$

_{C}is proportional to the square of a linear combination of the Pearson correlations in each stratum. The proportionality factor ensures S

_{C}has the ${\chi}_{1}^{2}$ distribution. This formula demonstrates how the Pearson correlations in each stratum contribute to the overall correlation measure.

_{XX}

_{1}= 50, S

_{XY}

_{1}= −9 and S

_{YY}

_{1}= 39.7333, r

_{P1}= −0.2019 and the CMH C statistic for school takes the value 2.4055 with ${\chi}_{1}^{2}$ p-value 0.1209. Similarly, S

_{XX}

_{2}= 51.6712, S

_{XY}

_{2}= −23.8904, S

_{YY}

_{2}= 42.6301, r

_{P2}= −0.5090 and the CMH C statistic for college takes the value 18.6558 with ${\chi}_{1}^{2}$ p-value 0.0000. From these, the value of the CMH C statistic and its χ

^{2}p-value are confirmed: It was previously noted that S

_{C}takes the value 16.8328 with ${\chi}_{1}^{2}$ p-value 0.0000. Clearly, there is an insignificant Pearson correlation for schools and a highly significant Pearson correlation for college. The latter dominates the former so that overall there is strong evidence of a correlation effect: As religion becomes increasingly liberal there is greater agreement with the proposition that homosexuals should be able to marry. This is due mainly to the stratum college.

_{XX}= 6, S

_{XY}= −12 and S

_{YY}= 43.5. It follows that the Pearson correlation is −0.7428 and the CMH C statistic takes the value 3.8621 with ${\chi}_{1}^{2}$ p-value 0.0494. There is some evidence that as maturity increases so does the grade of the whiskey.

_{XX}is constant over strata, it is not too much extra work to calculate S

_{XY}, S

_{YY}, the Pearson correlation, the CMH correlation statistic and its p-value on each stratum (See Table 4).

_{XX}= 2 on all strata and S

_{C}= 1.1029 with ${\chi}_{1}^{2}$ p-value 0.2936. There is no significant correlation effect, which would, if present, indicate that as we pass from jam A to B to C there is increasing (or decreasing) sweetness. It could be that overall there is no correlation effect with the contrary being the case in a minority of strata. That is not the case here; no stratum shows any evidence of a slight correlation effect. Here this is hardly surprising; with only three observations in each stratum, there can be little power in testing for a correlation.

#### 5.2. The Randomised Blocks Design

_{(t–1),(b–1)(t–1)}distribution or S

_{MS}its ${\chi}_{t-1}^{2}$ distribution, or otherwise. An empirical study would be required to assess which is the more reliable in the sense of closeness to, say, permutation test p-values. In the examples we have analysed, there has been little difference in these methods.

_{i}= 1, 2 and 3, so ${S}_{XX}^{2}=2$. From a one-way ANOVA with judges as treatments, ${\sum}_{j}{S}_{YYj}^{2}}=22\frac{2}{3$. This can be confirmed by summing across the S

_{YY}row in Table 4. By summing the columns in Table 2, the treatment score sums are found to be 18, 30 and 23, and since the centred treatment scores are −1, 0 and 1, C = −18 + 0 + 23 = 5. Substituting gives S

_{C}= 2 × 25/(2 × 68/3) = 75/68 = 1.1029 with ${\chi}_{1}^{2}$ p-value 0.2936, as previously.

## 6. Nonparametric ANOVA

## 7. Extensions of the CMH Mean Scores and Correlation Tests

_{MS}, consider the scores b

_{hj}on the jth stratum. A common choice of scores to give a ‘mean’ assessment would be the ‘natural’ scores 1, 2, …, c on all strata. A ‘dispersion’ assessment could be given by choosing scores 1

^{2}, 2

^{2}, …, c

^{2}and similarly higher order powers might be of interest: b

_{hj}= h

^{r}. One problem with using the scores b

_{hj}= h

^{r}is that the test statistics are correlated. Thus, the significance or not of the test for any order may affect the significance or not of tests at other orders. We now look at using more general scores with the objective of having uncorrelated test statistics. To this end, now denote order r scores that are not stratum-specific by ${b}_{h}^{r}$, h = 1, …, c. Define the order r score sum for treatment i by ${\sum}_{j=1}^{b}{\displaystyle {\sum}_{h=1}^{c}{b}_{h}^{r}{N}_{ihj}}}={\displaystyle {\sum}_{h=1}^{c}{b}_{h}^{r}{N}_{ih\u2022}}={M}_{i}^{r$. Now suppose that $\left\{{b}_{h}^{r}\right\}$ are orthonormal using the weight function $\{{n}_{\u2022h\u2022}/{n}_{\u2022\u2022\u2022}\}$. Then for r ≠ s = 1, 2, …, t it can be shown that $cov({M}_{i}^{r},{M}_{i}^{s})=0$: The ith score sums of different orders, ${M}_{i}^{r}$ and ${M}_{i}^{s}$, are uncorrelated. In this sense, the information provided by the scores sums of different orders for the same treatment is not related.

_{C}, suppose that instead of a single set of outcome scores {b

_{hj}}, we consider c sets of scores, {${b}_{hj}^{(s)}$} for s = 0, 1, …, c − 1. Moreover, suppose the scores are orthonormal in the sense that ${\sum}_{h}{b}_{hj}^{(r)}{b}_{hj}^{(s)}{n}_{\u2022hj}/{n}_{\u2022\u2022j}}={\delta}_{rs$ with r, s = 0, 1, …, c − 1 and with ${b}_{hj}^{(0)}=1$ for h = 1, …, c. Similarly, instead of a single set of treatment scores {a

_{ij}}, consider t sets of scores, {${a}_{ij}^{(r)}$} for r = 0, 1, …, t − 1 and suppose these scores are orthonormal in the sense that ${\sum}_{i}{a}_{ij}^{(r)}{a}_{ij}^{(s)}{n}_{i\u2022j}/{n}_{\u2022\u2022j}}={\delta}_{rs$ with r, s = 0, 1, …, t − 1 and with ${a}_{ij}^{(0)}=1$ for i = 1, …, t. Both sets of scores may be stratum-specific. Define S

_{Crs}as before but using ${a}_{ij}^{(r)}$ and ${b}_{hj}^{(s)}$. It can be shown that the S

_{Crs}are uncorrelated. See Reference [11] for details.

## 8. Development of Unconditional CMH Tests

_{OPA}and S

_{GA}had analogues denoted by T

_{OPA}and T

_{GA}. The latter is based on Pearson tests. The nominal CMH tests were identified as conditional tests while the analogues since they are not based on the product extended hypergeometric distribution, are unconditional tests. Since the unconditional Pearson tests are very familiar, they would be the tests of preference for most users. It is of interest to develop unconditional analogues of the ordinal CMH tests. This was done in Reference [14].

_{uj}(h)} on (${\widehat{p}}_{\u20221j}$, …, ${\widehat{p}}_{\u2022cj}$):

_{uij}reflects the contribution of treatment i to the order u effect on stratum j. Rayner and Best [10], Chapter 4 show that

^{−}is a generalised inverse of Σ. Of course, T

_{M1}gives an unconditional assessment of mean score differences in treatments in contrast with S

_{MS}that gives a conditional assessment of mean score differences in treatments. The T

_{Mu}are all asymptotically ${\chi}_{t-1}^{2}$ distributed.

_{ij}} that follows a multinomial distribution with total count n = ${N}_{\u2022\u2022}$ and cell probabilities {p

_{ij}} with ${p}_{\u2022\u2022}=1$ is assumed. Instead, we need to assume {N

_{ij}} follows a multinomial distribution with total count ${n}_{i\u2022}$ and cell probabilities {p

_{ij}} with ${p}_{i\u2022}=1$. Nevertheless, it may be shown that similar results apply.

_{r}(i)} are orthonormal on the {${p}_{i\u2022}$} with π

_{0}(i) = 1 for all i, if {ω

_{s}(j)} are orthonormal on the {${p}_{\u2022j}$} with ω

_{0}(j) = 1 for all j, and if ${V}_{rs}={\displaystyle {\sum}_{i}{\displaystyle {\sum}_{j}{N}_{ij}{\pi}_{r}(i){\omega}_{s}(j)/\sqrt{n}}}$, then the {V

_{rs}} numerically partition the Pearson statistic and are asymptotically standard normal score test statistics. In particular, V

_{11}/√n is the Pearson product moment correlation for grouped data and, if the scores are ranks, V

_{11}/√n is the Spearman correlation.

_{rj}(i)} on $({\widehat{p}}_{1\u2022j},\dots ,{\widehat{p}}_{c\u2022j})$ with π

_{0}(i) = 1 for all i, and {ω

_{sj}(h)} on $({\widehat{p}}_{\u20221j},\dots ,{\widehat{p}}_{\u2022cj})$ with ω

_{0}(j) = 1 for all j. Next define, for r = 1, …, t − 1 and s = 1, …, c − 1

_{OPA}and T

_{OPA}. It is some consolation that T

_{GA}is defined.

_{OPA}is that in order to sum the Pearson statistics from each stratum the same treatment and responses need to be included in all statistics and with sparse data that need not be the case. The same applies to S

_{OPA}, which is using a weighted sum of Pearson statistics.

## Supplementary Materials

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Rayner, J.C.W.; Best, D.J. Extended ANOVA and rank transform procedures. Aust. N. Z. J. Stat.
**2013**, 55, 305–319. [Google Scholar] [CrossRef] - Rayner, J.C.W.; Best, D.J.; Thas, O. Extended analysis of at least partially ordered multi-factor ANOVA. Aust. N. Z. J. Stat.
**2015**, 57, 211–224. [Google Scholar] [CrossRef] - Rayner, J.C.W. Extended ANOVA. J. Stat. Theory Pract.
**2017**, 11, 208–219. [Google Scholar] [CrossRef] - Landis, J.R.; Heyman, E.R.; Koch, G.G. Average partial association in three-way contingency tables: A review and discussion of alternative tests. Int. Statist. Rev.
**1978**, 46, 237–254. [Google Scholar] [CrossRef] - Landis, J.R.; Cooper, M.M.; Kennedy, T.; Koch, G.G. A computer program for testing average partial association in three-way contingency tables (PARCAT). Comput. Programs Biomed.
**1979**, 9, 223–246. [Google Scholar] [CrossRef] - Davis, C.S. Statistical Methods for the Analysis of Repeated Measurements; Springer: New York, NY, USA, 2002. [Google Scholar]
- Kuritz, S.J.; Landis, J.R.; Koch, G.G. A general overview of Mantel-Maenszel methods: Applications and recent developments. Ann. Rev. Public Health
**1988**, 9, 123–160. [Google Scholar] [CrossRef] [PubMed] - Agresti, A. Categorical Data Analysis, 2nd ed.; Wiley: New York, NY, USA, 2002. [Google Scholar]
- O’Mahony, M. Sensory Evaluation of Food—Statistical Methods and Procedures; Marcel Dekker: New York, NY, USA, 1986. [Google Scholar]
- Rayner, J.C.W.; Best, D.J. A Contingency Table Approach to Nonparametric Testing; Chapman & Hall/CRC: Boca Raton, FL, USA, 2001. [Google Scholar]
- Rayner, J.C.W.; Best, D.J. Extensions to the Cochran–Mantel–Haenszel mean scores and correlation tests. J. Stat. Theory Pract.
**2018**, 12, 561–574. [Google Scholar] [CrossRef] - Rayner, J.C.W. Introductory Nonparametrics; Bookboon: Copenhagen, Denmark, 2016. [Google Scholar]
- Rayner, J.C.W.; Beh, E.J. Towards a Better Understanding of Correlation. Stat. Neerl.
**2009**, 63, 324–333. [Google Scholar] [CrossRef] - Rayner, J.C.W.; Best, D.J. Unconditional analogues of Cochran–Mantel–Haenszel tests. Aust. N. Z. J. Stat.
**2017**, 59, 485–494. [Google Scholar] [CrossRef]

**Table 1.**Opinions on homosexual marriage by religious beliefs and education levels for ages 18 to 25.

Education | Religion | Homosexuals Should Be Able to Marry | ||
---|---|---|---|---|

Agree | Neutral | Disagree | ||

School | Fundamentalist | 6 | 2 | 10 |

Moderate | 8 | 3 | 9 | |

Liberal | 11 | 5 | 6 | |

College | Fundamentalist | 4 | 2 | 11 |

Moderate | 21 | 3 | 5 | |

Liberal | 22 | 4 | 1 |

Judge | Jam | ||
---|---|---|---|

A | B | C | |

1 | 3 | 2 | 3 |

2 | 4 | 5 | 4 |

3 | 3 | 2 | 3 |

4 | 1 | 4 | 2 |

5 | 2 | 4 | 2 |

6 | 1 | 3 | 3 |

7 | 2 | 5 | 4 |

8 | 2 | 5 | 2 |

Years of Maturing | Grade | Total | ||
---|---|---|---|---|

First | Second | Third | ||

One | 0 | 0 | 2 | 2 |

Five | 1 | 1 | 1 | 3 |

Seven | 2 | 1 | 0 | 3 |

Total | 3 | 2 | 3 | 8 |

Stratum | ||||||||
---|---|---|---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |

S_{XY} | 0 | 0 | 0 | 1 | 0 | 2 | 2 | 0 |

S_{YY} | 0.6667 | 0.6667 | 0.6667 | 4.6667 | 2.6667 | 2.6667 | 4.6667 | 6 |

r_{P} | 0 | 0 | 0 | 0.3273 | 0 | 0.8660 | 0.6547 | 0 |

S_{C} | 0 | 0 | 0 | 0.2143 | 0 | 1.5 | 0.8571 | 0 |

p-value | 1 | 1 | 1 | 0.6434 | 1 | 0.2207 | 0.3545 | 1 |

**Table 5.**CMH and nonparametric ANOVA GC p-values when testing for zero generalised correlations for the jam data.

CMH GC Tests | Nonparametric ANOVA | |||
---|---|---|---|---|

Treatment Order | Treatment Order | |||

Category Order | 1 | 2 | 1 | 2 |

1 | 0.2936 | 0.0212 | 0.2249 0.2403 | 0.0333 0.0853 |

2 | 0.1862 | 0.2780 | 0.1410 0.2058 | 0.2640 0.1558 |

3 | 0.6098 | 0.3104 | 0.6041 0.7229 | 0.3069 0.2601 |

**Table 6.**CMH p-values for each stratum and overall when testing for zero generalised correlations for the jam data.

CMH C Extensions | ||||
---|---|---|---|---|

(1, 1) | (1, 2) | (2, 1) | (2, 2) | |

School | 0.1209 | 0.8352 | 0.3246 | 0.8523 |

College | 0.0866 | 0.8183 | 0.2766 | 0.8371 |

Overall | 0.0000 | 0.2328 | 0.1272 | 0.8777 |

Conditional | Unconditional | ||||
---|---|---|---|---|---|

Statistic | Value | p-Value | Statistic | Value | p-Value |

S_{GA} | 19.76 | 0.0006 | T_{GA} | 20.68 | 0.0004 |

S_{OPA} | 26.71 | 0.0008 | T_{OPA} | 27.09 | 0.0007 |

S_{MS} | 17.94 | 0.0001 | |||

S_{M1} | 22.32 | 0.0000 | T_{M1} | 23.71 | 0.0000 |

S_{M2} | 2.59 | 0.2732 | T_{M2} | 2.44 | 0.2954 |

S_{C11} | 17.98 | 0.0000 | ${T}_{\mathrm{C}}={V}_{11\u2022}^{2}/2$ | 17.48 | 0.0000 |

S_{C12} | 1.42 | 0.2328 | ${V}_{12\u2022}^{2}/2$ | 2.35 | 0.1254 |

S_{C21} | 2.33 | 0.1272 | ${V}_{21\u2022}^{2}/2$ | 1.28 | 0.2570 |

S_{C22} | 0.02 | 0.8777 | ${V}_{22\u2022}^{2}/2$ | 0.03 | 0.8726 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Rayner, J.C.W.; Rippon, P.
Recent Extensions to the Cochran–Mantel–Haenszel Tests. *Stats* **2018**, *1*, 98-111.
https://doi.org/10.3390/stats1010008

**AMA Style**

Rayner JCW, Rippon P.
Recent Extensions to the Cochran–Mantel–Haenszel Tests. *Stats*. 2018; 1(1):98-111.
https://doi.org/10.3390/stats1010008

**Chicago/Turabian Style**

Rayner, J. C. W., and Paul Rippon.
2018. "Recent Extensions to the Cochran–Mantel–Haenszel Tests" *Stats* 1, no. 1: 98-111.
https://doi.org/10.3390/stats1010008