Current Research and Statistical Practices in Sport Science and a Need for Change
Abstract
:1. The Problem
2. The Solution
- (1)
- Defining the problem
- (2)
- Descriptive research (hypothesis generating)
- (3)
- Predictors of performance
- (4)
- Experimental testing of predictors
- (5)
- Determinants of key performance predictors
- (6)
- Efficacy studies (controlled laboratory or field)
- (7)
- Barriers to uptake
- (8)
- Implementation studies (real sporting setting)
3. Current Alternative Statistical Methods
3.1. Smallest Worthwhile Change
3.2. Comparing Correlations
3.3. Effect Size
3.4. Confidence Intervals
3.5. Magnitude-Based Inferences
3.6. Counter-Argument against Magnitude-Based Inferences
4. Bayesian Estimation
5. Conclusions
Supplementary Materials
Author Contributions
Conflicts of Interest
References
- Greenland, S.; Senn, S.J.; Rothman, K.J.; Carlin, J.B.; Poole, C.; Goodman, S.N.; Altman, D.G. Statistical tests, P values, confidence intervals, and power: A guide to misinterpretations. Eur. J. Epidemiol. 2016, 31, 337–350. [Google Scholar] [CrossRef] [PubMed]
- Ioannidis, J.P. Why most published research findings are false. PLoS Med. 2005, 2, e124. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Ioannidis, J.P. Discussion: Why “An estimate of the science-wise false discovery rate and application to the top medical literature” is false. Biostatistics 2013, 15, 28–36. [Google Scholar] [CrossRef] [PubMed]
- Chambers, C.D.; Feredoes, E.; Muthukumaraswamy, S.D.; Etchells, P. Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. AIMS Neurosci. 2014, 1, 4–17. [Google Scholar] [CrossRef]
- Ioannidis, J.P. Why science is not necessarily self-correcting. Perspect. Psychol. Sci. 2012, 7, 645–654. [Google Scholar] [CrossRef] [PubMed]
- Nakagawa, S.; Cuthill, I.C. Effect size, confidence interval and statistical significance: A practical guide for biologists. Biol. Rev. 2007, 82, 591–605. [Google Scholar] [CrossRef] [PubMed]
- Kerr, N.L. HARKing: Hypothesizing after the results are known. Personal. Soc. Psychol. Rev. 1998, 2, 196–217. [Google Scholar] [CrossRef] [PubMed]
- Simonsohn, U.; Nelson, L.D.; Simmons, J.P. P-curve: A key to the file-drawer. J. Exp. Psychol. Gen. 2014, 143, 534. [Google Scholar] [CrossRef] [PubMed]
- Kuhn, T.S.; Hawkins, D. The structure of scientific revolutions. Am. J. Phys. 1963, 31, 554–555. [Google Scholar] [CrossRef]
- Scargle, J.D. Publication bias (the “file-drawer problem”) in scientific inference. arXiv 1999, arXiv:physics/9909033. [Google Scholar]
- Cramer, A.O.; van Ravenzwaaij, D.; Matzke, D.; Steingroever, H.; Wetzels, R.; Grasman, R.P.; Waldorp, L.J.; Wagenmakers, E.-J. Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies. Psychon. Bull. Rev. 2016, 23, 640–647. [Google Scholar] [CrossRef] [PubMed]
- Cumming, G. The new statistics: Why and how. Psychol. Sci. 2014, 25, 7–29. [Google Scholar] [CrossRef] [PubMed]
- Hopkins, W.G. P values down but not yet out. Sport Sci. 2016, 20. Available online: http://www.sportsci.org/2016/inbrief.htm (assessed on 14 November 2017).
- Buchheit, M. The numbers will love you back in return—I promise. Int. J. Sports Physiol. Perform. 2016, 11, 551–554. [Google Scholar] [CrossRef] [PubMed]
- Wasserstein, R.L.; Lazar, N.A. The ASA’s statement on p-values: Context, process, and purpose. Am. Stat. 2016, 70, 129–133. [Google Scholar] [CrossRef]
- Egbewale, B.E.; Lewis, M.; Sim, J. Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: A simulation study. BMC Med. Res. Methodol. 2014, 14, 49. [Google Scholar] [CrossRef] [PubMed]
- Bishop, D. An applied research model for the sport sciences. Sports Med. 2008, 38, 253–263. [Google Scholar] [CrossRef] [PubMed]
- Tukey, J.W. Some thoughts on clinical trials, especially problems of multiplicity. Science 1977, 198, 679–684. [Google Scholar] [CrossRef] [PubMed]
- Duthie, G.M.; Pyne, D.B.; Ross, A.A.; Livingstone, S.G.; Hooper, S.L. The reliability of ten-meter sprint time using different starting techniques. J. Strength Cond. Res. 2006, 20, 246. [Google Scholar] [CrossRef] [PubMed]
- Pyne, D.B. Interpreting the results of fitness testing. In International Science and Football Symposium; Victorian Institute of Sport Melbourne: Melbourne, Australia, 2003. [Google Scholar]
- Hopkins, W.G.; Hawley, J.A.; Burke, L.M. Design and analysis of research on sport performance enhancement. Med. Sci. Sports Exerc. 1999, 31, 472–485. [Google Scholar] [CrossRef] [PubMed]
- Batterham, A.M.; Hopkins, W.G. Making meaningful inferences about magnitudes. Int. J. Sports Physiol. Perform. 2006, 1, 50–57. [Google Scholar] [CrossRef] [PubMed]
- Hopkins, W.G. How to interpret changes in an athletic performance test. Sport Sci. 2004, 8, 1–7. [Google Scholar]
- Guyatt, G.H.; Kirshner, B.; Jaeschke, R. Measuring health status: What are the necessary measurement properties? J. Clin. Epidemiol. 1992, 45, 1341–1345. [Google Scholar] [CrossRef]
- Beckerman, H.; Roebroeck, M.; Lankhorst, G.; Becher, J.; Bezemer, P.D.; Verbeek, A. Smallest real difference, a link between reproducibility and responsiveness. Qual. Life Res. 2001, 10, 571–578. [Google Scholar] [CrossRef] [PubMed]
- Meng, X.-L.; Rosenthal, R.; Rubin, D.B. Comparing correlated correlation coefficients. Psychol. Bull. 1992, 111, 172. [Google Scholar] [CrossRef]
- Field, A.; Miles, J.; Field, Z. Discovering Statistics Using R; SAGE Publications: Thousand Oaks, CA, USA, 2012; ISBN 978-1-4462-5846-0. [Google Scholar]
- Lakens, D. Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Front. Psychol. 2013, 4, 863. [Google Scholar] [CrossRef] [PubMed]
- Hedges, L.; Olkin, I. Statistical Methods for Meta-Analysis; Academic Press: Orlando, FL, USA, 1985. [Google Scholar]
- Hopkins, W.G. A scale of magnitude for effect statistics. In A New View of Statistics; Will G. Hopkins: Melbourne, Australia, 2002; p. 502. [Google Scholar]
- Rhea, M.R. Determining the magnitude of treatment effects in strength training research through the use of the effect size. J. Strength Cond. Res. 2004, 18, 918–920. [Google Scholar] [PubMed]
- Cohen, J. Statistical Power Analyses for the Social Sciences; Lawrence Erlbauni Associates: Hillsdale, NJ, USA, 1988. [Google Scholar]
- Hopkins, W.; Marshall, S.; Batterham, A.; Hanin, J. Progressive statistics for studies in sports medicine and exercise science. Med. Sci. Sports Exerc. 2009, 41, 3. [Google Scholar] [CrossRef] [PubMed]
- Hopkins, W.G. A spreadsheet for deriving a confidence interval, mechanistic inference and clinical inference from a P value. Sport Sci. 2007, 11, 16–21. [Google Scholar]
- Hopkins, W.G.; Batterham, A.M. Error rates, decisive outcomes and publication bias with several inferential methods. Sports Med. 2016, 46, 1563–1573. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Van Schaik, P.; Weston, M. Magnitude-based inference and its application in user research. Int. J. Hum. Comput. Stud. 2016, 88, 38–50. [Google Scholar] [CrossRef]
- Hankins, M. Still not Significant. Probable Error 2013. Available online: https://mchankins.wordpress.com/2013/04/21/still-not-significant-2/ (assessed on 14 November 2017).
- Mengersen, K.L.; Drovandi, C.C.; Robert, C.P.; Pyne, D.B.; Gore, C.J. Bayesian estimation of small effects in exercise and sports science. PLoS ONE 2016, 11, e0147311. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Welsh, A.H.; Knight, E.J. “Magnitude-based Inference”: A statistical review. Med. Sci. Sports Exerc. 2015. [Google Scholar] [CrossRef] [PubMed]
- Humberstone-Gough, C.E.; Saunders, P.U.; Bonetti, D.L.; Stephens, S.; Bullock, N.; Anson, J.M.; Gore, C.J. Comparison of live high: Train low altitude and intermittent hypoxic exposure. J. Sports Sci. Med. 2013, 12, 394. [Google Scholar] [PubMed]
- McElreath, R. Statistical Rethinking: A Bayesian Course with Examples in R and Stan; Texts in Statistical Science; CRC Press: Boca Raton, FL, USA, 2015. [Google Scholar]
- Hahn, U. The Bayesian boom: Good thing or bad? Front. Psychol. 2014, 5, 765. [Google Scholar] [CrossRef] [PubMed]
- Burton, P.R.; Gurrin, L.C.; Campbell, M.J. Clinical significance not statistical significance: A simple Bayesian alternative to p values. J. Epidemiol. Commun. Health 1998, 52, 318–323. [Google Scholar] [CrossRef]
- Kirk, R.E. The importance of effect magnitude. In Handbook of Research Methods in Experimental Psychology; John Wiley & Sons: Hoboken, NJ, USA, 2003; pp. 83–105. [Google Scholar]
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bernards, J.R.; Sato, K.; Haff, G.G.; Bazyler, C.D. Current Research and Statistical Practices in Sport Science and a Need for Change. Sports 2017, 5, 87. https://doi.org/10.3390/sports5040087
Bernards JR, Sato K, Haff GG, Bazyler CD. Current Research and Statistical Practices in Sport Science and a Need for Change. Sports. 2017; 5(4):87. https://doi.org/10.3390/sports5040087
Chicago/Turabian StyleBernards, Jake R., Kimitake Sato, G. Gregory Haff, and Caleb D. Bazyler. 2017. "Current Research and Statistical Practices in Sport Science and a Need for Change" Sports 5, no. 4: 87. https://doi.org/10.3390/sports5040087
APA StyleBernards, J. R., Sato, K., Haff, G. G., & Bazyler, C. D. (2017). Current Research and Statistical Practices in Sport Science and a Need for Change. Sports, 5(4), 87. https://doi.org/10.3390/sports5040087