Next Article in Journal
The Influence of Situational Cues on Children’s Creativity in an Alternative Uses Task and the Moderating Effect of Selective Attention
Previous Article in Journal
How to Compare Psychometric Factor and Network Models
Open AccessArticle

Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis

1
Department of Methodology & Statistics, Tilburg School of Social and Behavioral Sciences, Tilburg University, Warandelaan 2, 5037 AB Tilburg, The Netherlands
2
Section Sociology, Faculty of Social and Behavioral Sciences, Utrecht University, Heidelberglaan 1, 3584 CS Utrecht, The Netherlands
*
Author to whom correspondence should be addressed.
Received: 7 May 2020 / Revised: 14 September 2020 / Accepted: 24 September 2020 / Published: 2 October 2020
In this meta-study, we analyzed 2442 effect sizes from 131 meta-analyses in intelligence research, published from 1984 to 2014, to estimate the average effect size, median power, and evidence for bias. We found that the average effect size in intelligence research was a Pearson’s correlation of 0.26, and the median sample size was 60. Furthermore, across primary studies, we found a median power of 11.9% to detect a small effect, 54.5% to detect a medium effect, and 93.9% to detect a large effect. We documented differences in average effect size and median estimated power between different types of intelligence studies (correlational studies, studies of group differences, experiments, toxicology, and behavior genetics). On average, across all meta-analyses (but not in every meta-analysis), we found evidence for small-study effects, potentially indicating publication bias and overestimated effects. We found no differences in small-study effects between different study types. We also found no convincing evidence for the decline effect, US effect, or citation bias across meta-analyses. We concluded that intelligence research does show signs of low power and publication bias, but that these problems seem less severe than in many other scientific fields. View Full-Text
Keywords: meta-meta-analysis; meta-science; bias; intelligence; power; effect size meta-meta-analysis; meta-science; bias; intelligence; power; effect size
Show Figures

Figure 1

  • Externally hosted supplementary file 1
    Doi: 10.17605/OSF.IO/Z8EMY
    Link: https://osf.io/z8emy/
    Description: The supplementary files include the full data, analysis scripts, information about the included meta-analyses, and supplemental analyses.
MDPI and ACS Style

Nuijten, M.B.; van Assen, M.A.L.M.; Augusteijn, H.E.M.; Crompvoets, E.A.V.; Wicherts, J.M. Effect Sizes, Power, and Biases in Intelligence Research: A Meta-Meta-Analysis. J. Intell. 2020, 8, 36.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop