# Four Bad Habits of Modern Psychologists

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### 1.1. Bad Habit #1: Inference Conflation

Increases in the Stroop effect have been observed for different frequenciesof congruent word-color pairsIf persons were to change thresholds when exposed to greater frequenciesof word-color pairs, they would show increases in the Stroop effectTherefore, persons changed thresholds

_{o}: μ

_{low}= μ

_{medium}= μ

_{high}

#### 1.2. Bad Habit #2: Anemic Modeling

^{2}= 0.20, with a higher proportion of survival-rated words (~53%) being recalled than vacation-rated words (~40%). These results were statistically replicated by Müller and Renkewitz: F(1, 31) = 12.13, p = 0.002, η

^{2}= 0.28, survival proportion = 0.50, vacation proportion = 0.40.

#### 1.3. Bad Habit #3: Much Ado about Nothing

_{direct}= a + bx

_{indirect}+ ε.

_{direct}= a + b

_{1}x

_{indirect}+ b

_{2}x

_{group}+ ε,

_{direct}= a + b

_{1}x

_{indirect}+ b

_{2}x

_{group}+ b

_{3}x

_{indirect*group}+ ε.

^{2}(ΔR

^{2}= 0.09, p < 0.05), and Vianello replicated this significant finding, showing that the interaction term increased multiple R

^{2}from 0.297 to 0.313 compared to the model with only main effects (ΔR

^{2}= 0.016, p < 0.05).

_{direct}values compared to the more parsimonious main effects model (Equation (1))? This question places emphasis on the individuals in the study as well as the original units of observation for bias (viz., the −2 to +2 rating scale), which is appropriate because explicit or implicit racial bias is a deeply personal experience, and the proxy to that experience in this study is the rating scale. Analyses should consequently eschew aggregate ΔR

^{2}and p-values from NHST and instead provide an individual-level assessment of the predictive accuracy of the regression models in the original units of observation. Only in this way can the theoretical or practical meaningfulness of the results be assessed adequately.

_{direct}are summarized as follows: n = 180, mdn = 0.023, min = 0.002, max = 0.200, 10th percentile = 0.005, 90th percentile = 0.102. Given minimum and maximum possible discrepancies of 0 and 4 on the rating scale, respectively, the median discrepancy of 0.023 shows a 0.58% lift in predictive accuracy when the interaction term is included in the regression model. The histogram in Figure 7 moreover shows that the absolute discrepancies for most of the participants (63%) are 0.04 or less, and 71% of discrepancies are less than 0.05, thus demonstrating the inconsequential impact of adding the interaction term to the main effect regression model. It is difficult to imagine how these results are theoretically or practically meaningful when considered at the level of the person. Picture, for instance, a male student named Nathan with a y

_{direct}score of 1.5 (range: −2 to +2) and a predicted score of 1.2 from the main effects regression model. Suppose further that by adding the interaction term, Nathan’s predicted score increases by 0.023 (the median discrepancy) to a value of 1.223. How can this difference reflect anything remotely meaningful in Nathan’s subjective experience as it pertains to the modifiability of the connection between his implicit and explicit racial biases? This is the central question that is left entirely unanswered by the modern model comparisons habitus which focuses on aggregate ΔR

^{2}values and NHST, and which leads to the unwarranted acceptance of overly complex (i.e., less parsimonious) models and trivial statistical effects in psychological research.

#### 1.4. Bad Habit #4: Measurement and Washing Brains

Strongly | Strongly | |||

Disagree | Disagree | Neutral | Agree | Agree |

1 | 2 | 3 | ④ | 5 |

I have a lot of height | ||||

Strongly | Strongly | |||

Disagree | Disagree | Neutral | Agree | Agree |

1 | 2 | 3 | ④ | 5 |

Strongly | Strongly | |||

Disagree | Disagree | Neutral | Agree | Agree |

1 | 2 | 3 | 4 | ⑤ |

_{(3:5)}˅ Item 16

_{(3:5)}] ˄ Item 18

_{(1:4)}

- Item 4: I wish that something bad would happened to him/her
- Item 16: I have released my anger so I can work on restoring our relationship to health
- Item 18: I withdraw from him/her

## 2. Discussion

## 3. Conclusions

## Author Contributions

## Conflicts of Interest

## References

- Aquinas, T. The Division and Methods of the Sciences; The Pontifical Institute of Mediaeval Studies: Toronto, ON, Canada, 1963. [Google Scholar]
- Hubbard, R. Corrupt Research: The Case for Reconceptualizing Empirical Management and Social Science; Sage: Los Angeles, CA, USA, 2015. [Google Scholar]
- Open Science Collaboration. Estimating the reproducibility of psychological science. Science
**2015**, 349, aac4716. [Google Scholar] [CrossRef] - Alcock, J. Back from the future: Parapsychology and the Bem affair. Skept. Inq.
**2011**, 35, 31–39. Available online: http://www.csicop.org/specialarticles/show/back_from_the_future (accessed on 19 March 2017). - Bem, D.J. Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. J. Personal. Soc. Psychol.
**2011**, 100, 407–425. [Google Scholar] [CrossRef] [PubMed] - Wagenmakers, E.J.; Wetzels, R.; Borsboom, D.; van der Maas, H. Why psychologists must change the way they analyze their data: The case of psi. J. Personal. Soc. Psychol.
**2011**, 100, 426–432. [Google Scholar] [CrossRef] [PubMed] - Haig, B.D. An abductive theory of scientific method. Psychol. Methods
**2005**, 10, 371–388. [Google Scholar] [CrossRef] [PubMed] - Haig, B. Investigating the Psychological World; MIT Press: Cambridge, MA, USA, 2014. [Google Scholar]
- Schmidt, J.R.; Besner, D. The Stroop effect: Why proportion congruent has nothing to do with congruency and everything to do with contingency. J. Exp. Psychol. Learn. Mem. Cogn.
**2008**, 34, 514–523. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Cloud, M.D.; Kyc, M.M. Replication of “The Stroop Effect: Why Proportion Congruent Has Nothing to do with Congruency and Everything to do with Contingency” by JR Schmidt and D Besner (2008, JEPLMC). 2015. Available online: https://osf.io/bscfe/ (accessed on 19 March 2017).
- Grice, J.W. Observation Oriented Modeling: Analysis of Cause in the Behavioral Sciences; Academic Press: New York, NY, USA, 2011. [Google Scholar]
- Grice, J.W. Observation oriented modeling: Preparing students for the research in the 21st Century. Innov. Teach.
**2014**, 3. [Google Scholar] [CrossRef] - Breiman, L. Statistical modeling: The two cultures. Stat. Sci.
**2001**, 16, 199–231. [Google Scholar] [CrossRef] - Rychlak, J.F. The Psychology of Rigorous Humanism, 2nd ed.; New York University Press: New York, NY, USA, 1988. [Google Scholar]
- Nairne, J.S.; Pandeirada, J.N.S.; Thompson, S.R. Adaptive memory: The comparative value of survival processing. Psychol. Sci.
**2008**, 19, 176–180. [Google Scholar] [CrossRef] [PubMed] - Müller, S.; Renkewitz, F. Replication of Nairne, Pandeirada, & Thompson (2008, PS, Study 2). 2015. Available online: https://osf.io/jhkpe/ (accessed on 19 March 2017).
- Edgington, E.; Onghena, P. Randomization Tests, 4th ed.; Chapman & Hall: London, UK, 2007. [Google Scholar]
- Nairne, J.S.; Pandeirada, J.N.S. Adaptive memory: The evolutionary significance of survival processing. Perspect. Psychol. Sci.
**2016**, 11, 496–511. [Google Scholar] [CrossRef] [PubMed] - Maxwell, S.; Delaney, H. Designing Experiments and Analyzing Data: A Model Comparisons Approach, 2nd ed.; Erlbaum: Mahwah, NJ, USA, 2004. [Google Scholar]
- Barrett, P.T. Hierarchical Multiple Linear Regression and the Correct Interpretation of the Magnitude of a Deviation R-square (ΔR
^{2}); Cognadev Technical Report Series; Cognadev UK Ltd.: Middlesex, UK, 2016; Available online: http://www.pbarrett.net/techpapers.html (accessed on 19 March 2017). - Payne, B.K.; Burkley, M.A.; Stokes, M.B. Why do implicit and explicit attitude tests diverge? The role of structural fit. J. Personal. Soc. Psychol.
**2008**, 94, 16–31. [Google Scholar] [CrossRef] [PubMed] - Vianello, M. Replication of Payne, Burkley and Stokes (2008, JPSP, Study 4). 2015. Available online: https://osf.io/rc6mv/ (accessed on 19 March 2017).
- Crowley, C.O.P. Aristotelian-Thomistic Philosophy of Measure and the International System of Units (SI); University Press of America: Lanham, MD, USA, 1996. [Google Scholar]
- Michell, J. Measurement in Psychology: Critical History of a Methodological Concept; Cambridge University Press: Cambridge, UK, 1999. [Google Scholar]
- Stevens, S.S. On the theory of scales of measurement. Science
**1946**, 103, 677–680. [Google Scholar] [CrossRef] [PubMed] - Jung, C.G. Psychological Types; Princeton University Press: Princeton, NJ, USA, 1971. [Google Scholar]
- Barrett, P.T. The consequence of sustaining a pathology: Scientific stagnation—A commentary on the target article “Is psychometrics a pathological science?” by Joel Michell. Measurement
**2008**, 6, 78–123. [Google Scholar] [CrossRef] - Lovie, A.D. Commentary on Michell, Quantitative science and the definition of measurement in psychology. Br. J. Psychol.
**1997**, 88, 393–394. [Google Scholar] [CrossRef] - Michell, J. Is psychometrics pathological science? Measurement
**2008**, 6, 7–24. [Google Scholar] [CrossRef] - Exline, J.J.; Baumeister, R.F.; Zell, A.L.; Kraft, A.J.; Witvliet, C.V.O. Not so innocent: Does seeing one’s own capability for wrongdoing predict forgiveness? J. Personal. Soc. Psychol.
**2008**, 94, 495–515. [Google Scholar] [CrossRef] [PubMed] - Lin, S.; Frank, M.C. Replication of Exline et al. (2008, JPSP, Study 7). 2015. Available online: https://osf.io/imrx2/ (accessed on 19 March 2017).
- Nunnally, J. Psychometric Theory, 2nd ed.; McGraw-Hill: New York, NY, USA, 1978. [Google Scholar]
- Dorough-Morris, S.I.; Cox, R.A.; Grice, J.W. Scale imposition as quantitative alchemy: Studies on the transitivity of self- and other-rankings with respect to neuroticism and known quantitative attributes. Basic Appl. Soc. Psychol.
**2017**, 39, 1–18. [Google Scholar] [CrossRef] - Lord, F.M.; Novick, M. Statistical Theories of Mental Test Scores; Addison-Wesley: Reading, MA, USA, 1968. [Google Scholar]
- Lazarsfeld, P.F. Latent structure analysis. In Psychology: A Study of a Science; Koch, S., Ed.; McGraw-Hill: New York, NY, USA, 1959. [Google Scholar]
- Grice, J.W.; Cota, L.D.; Barrett, P.T.; Wuensch, K.L.; Poteat, G.M. A simple and transparent alternative to logistic regression. Adv. Soc. Sci. Res. J.
**2016**, 3, 147–165. [Google Scholar] [CrossRef] - Ferguson, C.J. “Everybody knows psychology is not a real science”: Public perceptions of psychology and how we can improve our relationship with policymakers, the scientific community, and the general public. Am. Psychol.
**2015**, 70, 527–542. [Google Scholar] [CrossRef] [PubMed] - Lykken, D.T. What’s wrong with psychology anyway? In Thinking Clearly about Psychology. Volume 1: Matters of Public Interest; Cicchetti, D., Grove, W.M., Eds.; University of Minnesota Press: Minneapolis, MN, USA, 1991; Chapter 1; pp. 3–39. [Google Scholar]
- Wood, B. What’s Still Wrong with Psychology, Anyway? Twenty Slow Years, Three Old Issues, and One New Methodology for Improving Psychological Research. Master’s Thesis, University of Canterbury, Christchurch, New Zealand, 2011. Available online: https://ir.canterbury.ac.nz/bitstream/handle/10092/5208/thesis_fulltext.pdf?sequence=1&isAllowed=y (accessed on 19 March 2017).
- Gigerenzer, G.; Marewski, J.N. Surrogate science: The idol of a universal method for scientific inference. J. Manag.
**2015**, 41, 421–440. [Google Scholar] [CrossRef] - Tryon, W.W.; Patelis, T.; Chajewski, M.; Lewis, C. Theory construction and data analysis. Theory Psychol.
**2017**, 27, 126–134. [Google Scholar] [CrossRef] - Trafimow, D.; Earp, B.D. Null hypothesis significance testing and Type I error: The domain problem. New Ideas Psychol.
**2017**, 45, 19–27. [Google Scholar] [CrossRef] - Freedman, D.A.; Berk, R.A. Statistical assumptions as empirical commitments. In Law, Punishment, and Social Control: Essays in Honor of Sheldon Messinger, 2nd ed.; Blomberg, T.G., Cohen, S., Eds.; Aldine de Gruyter: Hawthorne, NY, USA, 2003; pp. 235–254. [Google Scholar]
- Henrich, J.; Heine, S.J.; Norenzayan, A. The weirdest people in the world? Behav. Brain Sci.
**2010**, 33, 61–135. [Google Scholar] [CrossRef] [PubMed] - Tryon, W.W. Underreliance on mechanistic models: Comment on Ferguson (2015). Am. Psychol.
**2016**, 71, 505–506. [Google Scholar] [CrossRef] [PubMed] - Grice, J.W.; Barrett, P.T.; Schlimgen, L.A.; Abramson, C.I. Toward a brighter future for psychology as an observation oriented science. Behav. Sci.
**2012**, 2, 1–22. [Google Scholar] [CrossRef] [PubMed] - Grice, J.W. From means and variances to patterns and persons. Front. Psychol.
**2015**, 6. [Google Scholar] [CrossRef] [PubMed] - Grice, J.W.; Yepez, M.; Wilson, N.L.; Shoda, Y. Observation oriented modeling: Going beyond ‘is it all a matter of chance’? Educ. Psychol. Meas.
**2016**. [Google Scholar] [CrossRef] - Powers, W.T. Behavior: The Control of Perception, 1st ed.; Aldine Publishing Company: Chicago, IL, USA, 1973. [Google Scholar]
- Powers, W.T. Living Control Systems III: Modeling Behavior; Benchmark Publications: Montclair, NJ, USA, 2008. [Google Scholar]
- Cevasco, J.; Marmolejo-Ramos, F. The importance of studying prosody in the comprehension of spontaneous spoken discourse. Rev. Latinoam. Psicol.
**2013**, 45, 21–33. [Google Scholar] - Lamiell, J.T. Statisticism in personality psychologists’ use of trait constructs: What is it? How was it contracted? Is there a cure? New Ideas Psychol.
**2013**, 31, 65–71. [Google Scholar] [CrossRef] - Trafimow, D. The mean as a multilevel issue. Front. Psychol.
**2014**, 5, 180. [Google Scholar] [CrossRef] [PubMed] - Button, K.S.; Ioannidis, J.P.A.; Mokrysz, C.; Nosek, B.A.; Flint, J.; Robinson, E.S.J.; Munafò, M.R. Power failure: Why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci.
**2013**, 14, 365–376. [Google Scholar] [CrossRef] [PubMed] - Ashton, J.C. Experimental power comes from powerful theories—The real problem in null hypothesis testing. Nat. Rev. Neurosci.
**2013**, 14, 585. [Google Scholar] [CrossRef] [PubMed] - Thagard, P. Being interdisciplinary: Trading zones in cognitive science. In Interdisciplinary Collaboration: An Emerging Cognitive Science; Derry, S.J., Schunn, C.D., Gernsbacher, M.A., Eds.; Erlbaum: Mahwah, NJ, USA, 2005; pp. 317–339. [Google Scholar]
- Kirsner, K. Target definition for shipwreck hunting. Front. Psychol. Quant. Psychol. Meas.
**2015**, 6, 1–19. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Means and standard errors for proportions of errors committed on the Stroop task for the low, medium, and high contingency conditions.

**Figure 2.**Proportions of errors committed on the Stroop task for the low, medium, and high contingency conditions. The results have been separated into three categories based on their ordinal patterns.

**Figure 4.**Model representing fitness-related and generic mnemonic processing of the words “truck” and “temple.” Visual images are represented as elongated hexagons; simple predication is represented as a circle; complex judgments are represented as pentagons; memory storage is represented as hexagrams; “Ef” represents efficient cause, and “Fi” represents final cause.

**Figure 5.**Model representing integration of “truck” into imagined scene of survival. Visual images are represented as elongated hexagons; simple predication is represented as a circle; complex judgments are represented as pentagons; memory storage is represented as hexagrams; “if” and “else” represent standard logical operators; “Ef” represents an efficient cause; “Fo” represents a formal cause, and “Fi” represents a final cause.

**Figure 6.**The observed vs. predicted values for each regression model, the two regression lines, and the 95% prediction intervals around each regression line (the dotted lines).

**Figure 7.**The histogram of absolute value discrepancies between Model 1 and Model 2 predicted observations.

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Grice, J.; Barrett, P.; Cota, L.; Felix, C.; Taylor, Z.; Garner, S.; Medellin, E.; Vest, A.
Four Bad Habits of Modern Psychologists. *Behav. Sci.* **2017**, *7*, 53.
https://doi.org/10.3390/bs7030053

**AMA Style**

Grice J, Barrett P, Cota L, Felix C, Taylor Z, Garner S, Medellin E, Vest A.
Four Bad Habits of Modern Psychologists. *Behavioral Sciences*. 2017; 7(3):53.
https://doi.org/10.3390/bs7030053

**Chicago/Turabian Style**

Grice, James, Paul Barrett, Lisa Cota, Crystal Felix, Zachery Taylor, Samantha Garner, Eliwid Medellin, and Adam Vest.
2017. "Four Bad Habits of Modern Psychologists" *Behavioral Sciences* 7, no. 3: 53.
https://doi.org/10.3390/bs7030053