# Persistent Confusions about Hypothesis Testing in the Social Sciences

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Hypothesis Testing Concepts and Terminology

H0 is Actually True | H0 is Actually False | |
---|---|---|

Experimental conclusion: reject H0 | Type I error is committed | Experimental conclusion is correct |

Experimental conclusion: do not reject H0 | Experimental conclusion is correct | Type II error is committed |

## 3. Contemporary Textbooks’ Misinterpretations of Hypothesis Testing Concepts

#### 3.1. Erroneous Interpretations of α

In the example at hand, the null hypothesis was rejected and the probability that this decision was incorrect is 0.05.

As z_{obt}> 2.33, you would reject the null hypothesis, knowing there is a 1 in 100 chance that you are making the wrong the decision (Type I error).

_{obt}> 2.33, you would reject H0, knowing that in an experiment where H0 is true there is a 1 in 100 chance that the experimental result will lead to rejection of H0 under the current decision criterion (Type I error)”. This is an entirely different assertion from the one they actually made.

#### 3.2. Erroneous Interpretations of p-Value

Sir Ronald A. Fisher, established the basic guidelines for significance testing. He said that a statistical result may be considered significant if it can be shown that the probability of it being rejected due to chance is 5% or less. In inferential statistics, this probability is called the p-value, 5% is called the significance level (α), and the desired relationship between the p-value and α is denoted as: p ≤ 0.05. The significance level is the maximum level of risk that we are willing to accept as the price of our inference from the sample to the population. If the p-value is less than 0.05 or 5%, it means that we have a 5% chance of being incorrect in rejecting the null hypothesis or having a Type I error.

## 4. Analysis of Causes of Erroneous Interpretations

If we set our alpha level to the conventional .05, then the probability that we will reject the null hypothesis wrongly, that is, make a Type I error, is also .05. After all, by setting the alpha level to .05 for a statistical test we commit ourselves to rejecting the null hypothesis if the result we obtain would occur 5% of the time or less given that the null hypothesis is true. If we did the same experiment again and again, and in fact there is no effect in the population, over the long run 95% of the time we would correctly claim no effect. But 5% of the time, just by the luck of the draw, we would wrongly claim an effect.

## 5. Conclusions

## Acknowledgments

## Author Contributions

## Abbreviations

α | Significance level; |

H0 | Null hypothesis; |

H1 | Alternative hypothesis. |

## Appendix A: Conditional Probability Calculations

= α·Pr[H0 is true in the experiment]

## Appendix B: Textbooks Examined in the Study

## Conflicts of Interest

## References

- Michael W. Oakes. Statistical Inference. Chestnut Hill: Epidemiology Resources, Inc., 1990. [Google Scholar]
- McClave James, and Terry Sincich. Statistics, 12th ed. Boston: Pearson, 2013. [Google Scholar]
- Roger Bakeman. Understanding Social Science Statistics: A Spreadsheet Approach. Hillsdale: Lawrence Erlbaum Associates, 1992. [Google Scholar]
- Leslie Daly, and Geoffrey J. Bourke. Interpretation and Uses of Medical Statistics, 5th ed. Oxford: Blackwell Science, 2000. [Google Scholar]
- Kenneth D. Hopkins, and Gene V. Glass. Basic Statistics for the Behavioral Sciences. Englewood Cliffs: Prentice-Hall, 1978. [Google Scholar]
- Joan Welkowitz, Barry H. Cohen, and R. Brooke. Lea. Introductory Statistics for the Behavioral Sciences. New York: Wiley, 2011. [Google Scholar]
- “Type I and Type II Errors.” Wikipedia, the Free Encyclopedia. Available online: http://en.wikipedia.org/wiki/Type_I_and_type_II_error.s (accessed on 22 April 2015).
- Joseph F. Healey. Statistics: A Tool for Social Research, 9th ed. Stamford: Cengage Learning, 2011. [Google Scholar]
- Ronet Bachman, and Raymond Paternoster. Statistical Methods for Criminology and Criminal Justice. New York: Mcgraw-Hill, 1997. [Google Scholar]
- Anol Bhattacherjee. Social Science Research: Principles, Methods, and Practices. Tampa: Global Text Project, 2012, Available online: http://scholarcommons.usf.edu/oa_textbooks/3 (accessed on 7 May 2015).
- David Weisburd, and Chester Britt. Statistics in Criminal Justice, 3rd ed. New York: Springer Science & Business Media, LLC, 2007. [Google Scholar]
- Denis Anthony. Understanding Advanced Statistics: A Guide for Nurses and Health Care Researchers. Edinburgh: Churchill Livingstone, 1999. [Google Scholar]
- Mendel Suchmacher, and Mauro Geller. Practical Biostatistics: A User-Friendly Approach for Evidence-Based Medicine. Amsterdam: Academic, 2012. [Google Scholar]

© 2015 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Thron, C.; Miller, V.
Persistent Confusions about Hypothesis Testing in the Social Sciences. *Soc. Sci.* **2015**, *4*, 361-372.
https://doi.org/10.3390/socsci4020361

**AMA Style**

Thron C, Miller V.
Persistent Confusions about Hypothesis Testing in the Social Sciences. *Social Sciences*. 2015; 4(2):361-372.
https://doi.org/10.3390/socsci4020361

**Chicago/Turabian Style**

Thron, Christopher, and Vincent Miller.
2015. "Persistent Confusions about Hypothesis Testing in the Social Sciences" *Social Sciences* 4, no. 2: 361-372.
https://doi.org/10.3390/socsci4020361