Evidence for the Beneficial Effect of Reward on Working Memory: A Meta-Analytic Study
Abstract
:1. Introduction
2. Possible Moderators of Reward Effects on WM
2.1. The Type of Reward
2.2. Reward Methods
2.3. The Type of WM
2.4. Paradigm of WM
2.5. Outcome Measures
2.6. Participant Group
3. Methods
3.1. Literature Search
3.2. Criteria for Selection of Studies
3.3. Coding Procedures
3.4. Moderator Variables
3.5. Data Analysis
4. Results
4.1. The Overall Effect
4.2. Publication Bias
4.3. Moderator Analyses
4.3.1. The Type of Reward
4.3.2. Reward Methods
4.3.3. Type of WM
4.3.4. Paradigm of WM
4.3.5. Outcome Measures
4.3.6. Participant Group
4.3.7. Secondary Subgroup Analysis
5. Discussion
6. Limitations
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Adcock, Alison, Arul Thangavel, Susan Whitfield-Gabrieli, Brian Knutson, and John D. E. Gabrieli. 2006. Reward-motivated learning: Mesolimbic activation precedes memory formation. Neuron 50: 507–17. [Google Scholar] [CrossRef] [PubMed]
- Allen, Richard, and Taiji Ueno. 2018. Multiple high-reward items can be prioritized in working memory but with greater vulnerability to interference. Attention, Perception, & Psychophysics 80: 1731–43. [Google Scholar] [CrossRef]
- Baddeley, Alan, and Graham Hitch. 1974. Working memory. In Recent Advances in Learning and Motivation; New York: GA Bower, vol. 8. [Google Scholar] [CrossRef]
- Bae, Gi-Yeul, Maria Olkkonen, Sarah R. Allred, and Jonathan I. Flombaum. 2015. Why some colors appear more memorable than others: A model combining categories and particulars in color working memory. Journal of Experimental Psychology: General 144: 744–63. [Google Scholar] [CrossRef]
- Barch, Deanna. 2005. The relationships among cognition, motivation, and emotion in schizophrenia: How much and how little we know. Schizophrenia Bulletin 31: 875–81. [Google Scholar] [CrossRef]
- Bays, Paul. 2015. Spikes not slots: Noise in neural populations limits working memory. Trends in Cognitive Sciences 19: 431–38. [Google Scholar] [CrossRef]
- Bays, Paul, and Masud Husain. 2008. Supplementary: Dynamic shifts of limited working memory resources in human vision. Science 321: 851–54. [Google Scholar] [CrossRef] [PubMed]
- Bays, Paul, Raquel Catalao, and Masud Husain. 2009. The precision of visual working memory is set by allocation of a shared resource. Journal of Vision 9: 7. [Google Scholar] [CrossRef]
- Beck, Stefanie, Hannah S. Locke, Adam C. Savine, Koji Jimura, and Todd S. Braver. 2010. Primary and secondary rewards differentially modulate neural activity dynamics during working memory. PLoS ONE 5: e9251. [Google Scholar] [CrossRef]
- Belayachi, Sanaâ, Steve Majerus, Guido Gendolla, Eric Salmon, Frédéric Peters, and Martial Van der Linden. 2015. Are the carrot and the stick the two sides of same coin? A neural examination of approach/avoidance motivation during cognitive performance. Behavioural Brain Research 293: 217–26. [Google Scholar] [CrossRef]
- Bennett, Daniel, Karen Sasmita, Ryan T. Maloney, Carsten Murawski, and Stefan Bode. 2019. Monetary feedback modulates performance and electrophysiological indices of belief updating in reward learning. Psychophysiology 56: e13431. [Google Scholar] [CrossRef]
- Berridge, Kent, and Morten L. Kringelbach. 2015. Pleasure systems in the brain. Neuron 86: 646–64. [Google Scholar] [CrossRef] [PubMed]
- Bijleveld, Erik, Ruud Custers, and Henk Aarts. 2012. Human Reward Pursuit: From Rudimentary to Higher-Level Functions. Current Directions in Psychological Science 21: 194–99. [Google Scholar] [CrossRef]
- Botvinick, Matthew, and Todd Braver. 2015. Motivation and cognitive control: From behavior to neural mechanism. Annual Review of Psychology 66: 83–113. [Google Scholar] [CrossRef] [PubMed]
- Brady, Timothy F., Viola S. Störmer, and George A. Alvarez. 2016. Working memory is not fixed-capacity: More active storage capacity for real-world objects than for simple stimuli. Proceedings of the National Academy of Sciences of the United States of America 113: 7459–64. [Google Scholar] [CrossRef]
- Capa, Rémi L., and Cédric A. Bouquet. 2018. Individual differences in reward sensitivity modulate the distinctive effects of conscious and unconscious rewards on executive performance. Frontiers in Psychology 9: 148. [Google Scholar] [CrossRef]
- Castro, Juan F., Gustavo Yamada, Hans Contreras, Freddy Linares, and Herwig Watson. 2021. Tracking the Pressure: The Effect of Monetary Incentives on Effort, Emotions, and Performance. Journal of Neuroscience, Psychology, and Economics 14: 234–58. [Google Scholar] [CrossRef]
- Cheng, Si, Ting Jiang, Jingming Xue, Songxue Wang, Chuansheng Chen, and Mingxia Zhang. 2020. The influence of rewards on incidental memory: More does not mean better. Learning & Memory 27: 462–66. [Google Scholar] [CrossRef]
- Chen, Yung-Nien, Suvobrata Mitra, and Friederike Schlaghecken. 2008. Sub-processes of working memory in the N-back task: An investigation using ERPs. Clinical Neurophysiology 119: 1546–59. [Google Scholar] [CrossRef]
- Childress, Anna Rose, Ronald N. Ehrman, Ze Wang, Yin Li, Nathan Sciortino, Jonathan Hakun, William Jens, Jesse Suh, John Listerud, Kathleen Marquez, and et al. 2008. Prelude to Passion: Limbic Activation by “Unseen” Drug and Sexual Cues. PLoS ONE 3: e1506. [Google Scholar] [CrossRef]
- Choi, Jong Moon, Srikanth Padmala, and Luiz Pessoa. 2015. Counteracting effect of threat on reward enhancements during working memory. Cognition and Emotion 29: 1517–26. [Google Scholar] [CrossRef] [PubMed]
- Cho, Youngsun T., Flora Moujaes, Charles H. Schleifer, Martina Starc, Jie Lisa Ji, Nicole Santamauro, Brendan Adkinson, Antonija Kolobaric, Morgan Flynn, John H. Krystal, and et al. 2022. Reward and loss incentives improve spatial working memory by shaping trial-by-trial posterior frontoparietal signals. NeuroImage 254: 119139. [Google Scholar] [CrossRef]
- Cho, Youngsun T., Norman H. Lam, Martina Starc, Nicole Santamauro, Aleksandar Savic, Caroline K. Diehl, Charles H. Schleifer, Flora Moujaes, Vinod H. Srihari, Grega Repovs, and et al. 2018. Effects of reward on spatial working memory in schizophrenia. Journal of Abnormal Psychology 127: 695. [Google Scholar] [CrossRef] [PubMed]
- Christie, Thomas, and Paul Schrater. 2015. Cognitive cost as dynamic allocation of energetic resources. Frontiers in Neuroscience 9: 289. [Google Scholar] [CrossRef] [PubMed]
- Codol, Olivier, Peter J. Holland, Sanjay G. Manohar, and Joseph M. Galea. 2020. Reward-based improvements in motor control are driven by multiple error-reducing mechanisms. Journal of Neuroscience 40: 3604–20. [Google Scholar] [CrossRef]
- Cohen, Jacob. 1988. Statistical Power Analysis for the Behavioral Sciences, rev. ed. Cambridge: Academic Press. [Google Scholar] [CrossRef]
- Cohen, Michael S., Larry Y. Cheng, Ken A. Paller, and Paul J. Reber. 2019. Separate memory-enhancing effects of reward and strategic encoding. Journal of Cognitive Neuroscience 31: 1658–73. [Google Scholar] [CrossRef]
- Cowan, Nelson. 2001. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences 24: 87–114. [Google Scholar] [CrossRef]
- Deci, Edward. 1971. Effects of externally mediated rewards on intrinsic motivation. Journal of Personality and Social Psychology 18: 105–15. [Google Scholar] [CrossRef]
- Dehaene, Stanislas, and Laurent Cohen. 2007. Cultural recycling of cortical maps. Neuron 56: 384–98. [Google Scholar] [CrossRef]
- Dehaene, Stanislas, Michel Kerszberg, and Jean-Pierre Changeux. 1998. A Neuronal Model of a Global Workspace in Effortful Cognitive Tasks. Proceedings of the National Academy of Sciences of the United States of America 95: 14529–34. [Google Scholar] [CrossRef]
- Di Rosa, Elisa, Sabrina Brigadoi, Simone Cutini, Vincenza Tarantino, Roberto Dell’Acqua, Daniela Mapelli, Todd S. Braver, and Antonino Vallesi. 2019. Reward motivation and neurostimulation interact to improve working memory performance in healthy older adults: A simultaneous tDCS-fNIRS study. Neuroimage 202: 116062. [Google Scholar] [CrossRef]
- Engelmann, Jan B., Eswar Damaraju, Srikanth Padmala, and Luiz Pessoa. 2009. Combined effects of attention and motivation on visual task performance: Transient and sustained motivational effects. Frontiers in Human Neuroscience 3: 4. [Google Scholar] [CrossRef] [PubMed]
- Engle, Randall W., and Michael J. Kane. 2004. Executive attention, working memory capacity, and a two-factor theory of cognitive control. Psychology of Learning and Motivation 44: 145–200. [Google Scholar] [CrossRef]
- Fairclough, Stephen H., Christopher Burns, and Ute Kreplin. 2018. FNIRS activity in the prefrontal cortex and motivational intensity: Impact of working memory load, financial reward, and correlation-based signal improvement. Neurophotonics 5: 035001. [Google Scholar] [CrossRef] [PubMed]
- Fang, Wei, Kaiyue Wang, Ke Zhang, and Jiehui Qian. 2023. Spatial attention based on 2-D location and relative depth order modulates visual working memory in a 3-D environment. British Journal of Psychology 114: 112–31. [Google Scholar] [CrossRef] [PubMed]
- Fitts, Paul. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology 47: 381. [Google Scholar] [CrossRef]
- Gaillard, Claudie, Matthias Guillod, Monique Ernst, Andrea Federspiel, Dominik Schoebi, Romina Evelyn Recabarren, Xinyi Ouyang, Christoph Mueller-Pfeiffer, Antje Horsch, Philipp Homan, and et al. 2020. Striatal reactivity to reward under threat-of-shock and working memory load in adults at increased familial risk for major depression: A preliminary study. NeuroImage: Clinical 26: 102193. [Google Scholar] [CrossRef]
- Gaillard, Claudie, Matthias Guillod, Monique Ernst, Salvatore Torrisi, Andrea Federspiel, Dominik Schoebi, Romina E. Recabarren, Xinyi Ouyang, Christoph Mueller-Pfeiffer, Antje Horsch, and et al. 2019. Striatal responsiveness to reward under threat-of-shock and working memory load: A preliminary study. Brain and Behavior 9: e01397. [Google Scholar] [CrossRef]
- Gilbert, Alison M., and Julie A. Fiez. 2004. Integrating rewards and cognition in the frontal cortex. Cognitive, Affective, & Behavioral Neuroscience 4: 540–52. [Google Scholar] [CrossRef]
- Gong, Mengyuan, and Sheng Li. 2014. Learned reward association improves visual working memory. Journal of Experimental Psychology: Human Perception and Performance 40: 841–56. [Google Scholar] [CrossRef]
- Grogan, John P., Govind Randhawa, Minho Kim, and Sanjay G. Manohar. 2022. Motivation improves working memory by two processes: Prioritisation and retrieval thresholds. Cognitive Psychology 135: 101472. [Google Scholar] [CrossRef]
- Hager, Oliver M., Matthias Kirschner, Martin Bischof, Matthias N. Hartmann-Riemer, Agne Kluge, Erich Seifritz, Philippe N. Tobler, and Stefan Kaiser. 2015. Reward-dependent modulation of working memory is associated with negative symptoms in schizophrenia. Schizophrenia Research 168: 238–44. [Google Scholar] [CrossRef] [PubMed]
- Hammer, Rubi, Michael Tennekoon, Gillian E. Cooke, Jessica Gayda, Mark A. Stein, and James R. Booth. 2015. Feedback associated with expectation for larger-reward improves visuospatial working memory performances in children with ADHD. Developmental Cognitive Neuroscience 14: 38–49. [Google Scholar] [CrossRef]
- Heitz, Richard P., Josef C. Schrock, Tabitha W. Payne, and Randall W. Engle. 2008. Effects of incentive on working memory capacity: Behavioral and pupillometric data. Psychophysiology 45: 119–29. [Google Scholar] [CrossRef]
- Heuer, Anna, and Anna Schubö. 2018. Separate and combined effects of action relevance and motivational value on visual working memory. Journal of Vision 18: 14. [Google Scholar] [CrossRef] [PubMed]
- Hinault, Thomas, Kara J. Blacker, Matt Gormley, Brian A. Anderson, and Susan M. Courtney. 2019. Value-driven attentional capture is modulated by the contents of working memory: An EEG study. Cognitive, Affective, & Behavioral Neuroscience 19: 253–67. [Google Scholar] [CrossRef]
- Hitch, Graham J., Yanmei Hu, Richard J. Allen, and Alan D. Baddeley. 2018. Competition for the focus of attention in visual working memory: Perceptual recency versus executive control. Annals of the New York Academy of Sciences 1424: 64–75. [Google Scholar] [CrossRef]
- Ilkowska, Małgorzata, and Randall W. Engle. 2010. Trait and state differences in working memory capacity. In Handbook of Individual Differences in Cognition: Attention, Memory, and Executive Control. New York: Springer, pp. 295–320. [Google Scholar]
- Infanti, Elisa, Clayton Hickey, and Massimo Turatto. 2015. Reward associations impact both iconic and visual working memory. Vision Research 107: 22–29. [Google Scholar] [CrossRef]
- Jiang, Zanzan. 2013. Reward Effect in Spatial Working Memory. Master’s thesis, Capital Normal University, Beijing, China. Available online: https://kns-cnki-net-443.webvpn.sysu.edu.cn/KCMS/detail/detail.aspx?dbname=CMFD201402&filename=1013289518.nh (accessed on 15 December 2022). (In Chinese).
- Jimura, Koji, Hannah S. Locke, and Todd S. Braver. 2010. Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts. Proceedings of the National Academy of Sciences of the United States of America 107: 8871–76. [Google Scholar] [CrossRef]
- Juckel, Georg, Florian Schlagenhauf, Michael Koslowski, Dimitri Filonov, Torsten Wüstenberg, Arno Villringer, Brian Knutson, Thorsten Kienast, Jürgen Gallinat, Jana Wrase, and et al. 2006. Dysfunction of ventral striatal reward prediction in schizophrenic patients treated with typical, not atypical, neuroleptics. Psychopharmacology 187: 222–28. [Google Scholar] [CrossRef]
- Kalma, Akko. 1991. Hierarchisation and dominance assessment at first glance. European Journal of Social Psychology 21: 165–81. [Google Scholar] [CrossRef]
- Kawasaki, Masahiro, and Yoko Yamaguchi. 2013. Frontal theta and beta synchronizations for monetary reward increase visual working memory capacity. Social Cognitive and Affective Neuroscience 8: 523–30. [Google Scholar] [CrossRef] [PubMed]
- Klink, Christiaan, Danique Jeurissen, Jan Theeuwes, Damiaan Denys, and Pieter R. Roelfsema. 2017. Working memory accuracy for multiple targets is driven by reward expectation and stimulus contrast with different time-courses. Scientific Reports 7: 9082. [Google Scholar] [CrossRef] [PubMed]
- Klyszejko, Zuzanna, Masih Rahmati, and Clayton E. Curtis. 2014. Attentional priority determines working memory precision. Vision Research 105: 70–76. [Google Scholar] [CrossRef] [PubMed]
- Kotyusov, Alexander I., Dauren Kasanov, Alexandra I. Kosachenko, Anastasia S. Gashkova, Yuri G. Pavlov, and Sergey Malykh. 2023. Working Memory Capacity Depends on Attention Control, but Not Selective Attention. Behavioral Sciences 13: 92. [Google Scholar] [CrossRef]
- Krawczyk, Daniel C., Adam Gazzaley, and Mark D’Esposito. 2007. Reward modulation of prefrontal and visual association cortex during an incentive working memory task. Brain Research 1141: 168–77. [Google Scholar] [CrossRef]
- Kuhbandner, Christof, Alp Aslan, Kathrin Emmerdinger, and Kou Murayama. 2016. Providing extrinsic reward for test performance undermines long-term memory acquisition. Frontiers in Psychology 7: 79. [Google Scholar] [CrossRef]
- Kuvaas, Bård, Robert Buch, Antoinette Weibel, Anders Dysvik, and Christina G.L. Nerstad. 2017. Do intrinsic and extrinsic motivation relate differently to employee outcomes? Journal of Economic Psychology 61: 244–58. [Google Scholar] [CrossRef]
- Lee, Chong Ho, Myung Sook An, Jae Wook Um, and Dong Myong Lee. 2011. Effects of Organizational and Social Rewards on Positive Emotional Displays among Flight Attendants. Journal of the Aviation Management Society of Korea 9: 105. [Google Scholar]
- Lennie, Peter. 2003. The cost of cortical computation. Current Biology 13: 493–97. [Google Scholar] [CrossRef]
- Lepper, Mark, and David Greene. 1976. On Understanding “Overjustification”: A Reply to Reiss and Sushinsky. Journal of Personality and Social Psychology 33: 25–35. [Google Scholar] [CrossRef]
- Li, Jiaofeng, Jiehui Qian, and Fang Liang. 2018. Evidence for the beneficial effect of perceptual grouping on visual working memory: An empirical study on illusory contour and a meta-analytic study. Scientific Reports 8: 13864. [Google Scholar] [CrossRef] [PubMed]
- Lin, Yin-Ting, and Daryl Fougnie. 2022. No evidence that the retro-cue benefit requires reallocation of memory resources. Cognition 229: 105230. [Google Scholar] [CrossRef] [PubMed]
- Lipsey, Mark W., and David B. Wilson. 2001. Practical Meta-Analysis. Thousand Oaks: Sage Publications. [Google Scholar]
- Li, Shu-Chen, and Anna Rieckmann. 2014. Neuromodulation and aging: Implications of aging neuronal gain control on cognition. Current Opinion in Neurobiology 29: 148–58. [Google Scholar] [CrossRef]
- Lisman, John, and Marco Idiart. 1995. Storage of 7 + 2 short-term memories in oscillatory subcycles. Science 267: 1512–15. [Google Scholar] [CrossRef]
- Li, Zhuolun, Meixuan Tong, Shiting Chen, and Jiehui Qian. 2021. Effect of attentional selection on working memory for depth in a retro-cueing paradigm. Memory & Cognition 49: 747–57. [Google Scholar] [CrossRef]
- Luck, Steven J., and Edward K. Vogel. 1997. The capacity of visual working memory for features and conjunctions. Nature 390: 279–81. [Google Scholar] [CrossRef]
- Magis-Weinberg, Lucia, Ruud Custers, and Iroise Dumontheil. 2019. Rewards enhance proactive and reactive control in adolescence and adulthood. Social Cognitive and Affective Neuroscience 14: 1219–32. [Google Scholar] [CrossRef]
- Manga, Annamária, Pál Vakli, and Zoltán Vidnyánszky. 2020. The influence of anticipated monetary incentives on visual working memory performance in healthy younger and older adults. Scientific Reports 10: 8817. [Google Scholar] [CrossRef]
- Manohar, Sanjay G., Kinan Muhammed, Sean J. Fallon, and Masud Husain. 2019. Motivation dynamically increases noise resistance by internal feedback during movement. Neuropsychologia 123: 19–29. [Google Scholar] [CrossRef]
- Marquand, Andre F., Sara De Simoni, Owen G. O’Daly, Steven C. R. Williams, Janaina Mouro-Miranda, and Mitul A. Mehta. 2011. Pattern classification of working memory networks reveals differential effects of methylphenidate, atomoxetine, and placebo in healthy volunteers. Neuropsychopharmacology 36: 1237–47. [Google Scholar] [CrossRef]
- Martin, Joanne, and Joseph W. Harder. 1994. Bread and roses: Justice and the distribution of financial and socioemotional rewards in organizations. Social Justice Research 7: 241–64. [Google Scholar] [CrossRef]
- May, Christopher, Mauricio R. Delgado, Ronald E. Dahl, V. Andrew Stenger, Neal D. Ryan, Julie A. Fiez, and Cameron S. Carter. 2004. Event-related functional magnetic resonance imaging of reward-related brain circuitry in children and adolescents. Biological Psychiatry 55: 359–66. [Google Scholar] [CrossRef] [PubMed]
- Melin, Jeanette, Stefan Cano, Agnes Flöel, Laura Göschel, and Leslie Pendrill. 2022. The role of entropy in construct specification equations (CSE) to improve the validity of memory tests: Extension to word lists. Entropy 24: 934. [Google Scholar] [CrossRef]
- Miller, George. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 101: 343–52. [Google Scholar] [CrossRef]
- Moos, Daniel C. 2010. Nonlinear technology: Changing the conception of extrinsic motivation? Computers & Education 55: 1640–50. [Google Scholar] [CrossRef]
- Morey, Candice Coker, Nelson Cowan, Richard D. Morey, and Jeffery N. Rouder. 2011. Flexible attention allocation to visual and auditory working memory tasks: Manipulating reward induces a trade-off. Attention, Perception, & Psychophysics 73: 458–72. [Google Scholar] [CrossRef]
- Mosteller, Frederick, and Robert R. Bush. 1954. Selected Quantitative Techniques. Boston: Addison-Wesley. [Google Scholar]
- Ng, Tommy, Cathy Kao, Y. C. Chan, Effie Chew, Kai Hsiang Chuang, and Shen-Hsing A. Chen. 2016. Modality specificity in the cerebro-cerebellar neurocircuitry during working memory. Behavioural Brain Research 305: 164–73. [Google Scholar] [CrossRef]
- Nieoullon, André, and Antoine Coquerel. 2003. Dopamine: A key regulator to adapt action, emotion, motivation and cognition. Current Opinion in Neurology 16: S3–S9. [Google Scholar] [CrossRef]
- Pashler, Harold. 1988. Familiarity and visual change detection. Perception & Psychophysics 44: 369–78. [Google Scholar] [CrossRef]
- Pessoa, Luiz. 2009. How do emotion and motivation direct executive control? Trends in Cognitive Sciences 13: 160–66. [Google Scholar] [CrossRef]
- Pessoa, Luiz, and Jan B. Engelmann. 2010. Embedding reward signals into perception and cognition. Frontiers in Neuroscience 4: 17. [Google Scholar] [CrossRef] [PubMed]
- Peternel, Luka, Oliver Sigaud, and Jan Babič. 2017. Unifying speed-accuracy trade-off and cost-benefit trade-off in human reaching movements. Frontiers in Human Neuroscience 11: 615. [Google Scholar] [CrossRef] [PubMed]
- Qian, Jiehui, Ke Zhang, Quan Lei, Yifei Han, and Wenwen Li. 2020. Task-dependent effects of voluntary space-based and involuntary feature-based attention on visual working memory. Psychological Research 84: 1304–19. [Google Scholar] [CrossRef] [PubMed]
- Qian, Jiehui, Ke Zhang, Shengxi Liu, and Quan Lei. 2019. The transition from feature to object: Storage unit in visual working memory depends on task difficulty. Memory & Cognition 47: 1498–514. [Google Scholar] [CrossRef]
- Rademacher, Lena, Sören Krach, Gregor Kohls, Arda Irmak, Gerhard Gründer, and Katja N. Spreckelmeyer. 2010. Dissociation of neural networks for anticipation and consumption of monetary and social rewards. NeuroImage 49: 3276–85. [Google Scholar] [CrossRef]
- Redick, Thomas S., and Dakota R. Lindsey. 2013. Complex span and n-back measures of working memory: A meta-analysis. Psychonomic Bulletin & Review 20: 1102–13. [Google Scholar] [CrossRef]
- Rhodes, Rebecca E., and Benjamin Katz. 2017. Working memory plasticity and aging. Psychology and Aging 32: 51–59. [Google Scholar] [CrossRef]
- Ricker, Timothy J., Alessandra S. Souza, and Evie Vergauwe. 2023. Feature identity determines representation structure in working memory. Journal of Experimental Psychology: General 152: 2925–40. [Google Scholar] [CrossRef]
- Rosenthal, Robert. 1979. The file drawer problem and tolerance for null results. Psychological Bulletin 86: 638. [Google Scholar] [CrossRef]
- Rowe, James B., Doris Eckstein, Todd Braver, and Adrian M. Owen. 2008. How Does Reward Expectation Influence Cognition in the Human Brain? Journal of Cognitive Neuroscience 20: 1980–92. [Google Scholar] [CrossRef]
- Sanada, Motoyuki, Koki Ikeda, Kenta Kimura, and Toshikazu Hasegawa. 2013. Motivation enhances visual working memory capacity through the modulation of central cognitive processes. Psychophysiology 50: 864–71. [Google Scholar] [CrossRef]
- Sandry, Joshua, and Timothy J. Ricker. 2020. Prioritization within visual working memory reflects a flexible focus of attention. Attention, Perception, & Psychophysics 82: 2985–3004. [Google Scholar] [CrossRef]
- Savine, Adam C., Stefanie M. Beck, Bethany G. Edwards, Kimberly S. Chiew, and Todd S. Braver. 2010. Enhancement of cognitive control by approach and avoidance motivational states. Cognition and Emotion 24: 338–56. [Google Scholar] [CrossRef] [PubMed]
- Schmeichel, Brandon J. 2007. Attention control, memory updating, and emotion regulation temporarily reduce the capacity for executive control. Journal of Experimental Psychology: General 136: 241. [Google Scholar] [CrossRef] [PubMed]
- Schneegans, Sebastian, and Paul M. Bays. 2016. No fixed item limit in visuospatial working memory. Cortex 83: 181–93. [Google Scholar] [CrossRef] [PubMed]
- Shadmehr, Reza, de Jean Jacques Orban Xivry, Minnan Xu-Wilson, and Ting-Yu Shih. 2010. Temporal discounting of reward and the cost of time in motor control. The Journal of Neuroscience 30: 10507–16. [Google Scholar] [CrossRef]
- Shaw, Rhonda M., Edward Helmes, and David Mitchell. 2006. Age-related change in visual, spatial and verbal memory. Australasian Journal on Ageing 25: 14–19. [Google Scholar] [CrossRef]
- Smith, Edward E., John Jonides, and Robert A. Koeppe. 1996. Dissociating verbal and spatial working memory using PET. Cerebral Cortex 6: 11–20. [Google Scholar] [CrossRef]
- Stebbins, Glenn T., John DE Gabrieli, Francesca Masciari, Laura Monti, and Christopher G. Goetz. 1999. Delayed recognition memory in Parkinsons disease: A role for working memory? Neuropsychologia 37: 503–10. [Google Scholar] [CrossRef]
- Sterling, Peter. 2017. Principles of Neural Design. Cambridge: The MIT Press. [Google Scholar]
- Szatkowska, Iwona, Piotr Bogorodzki, Tomasz Wolak, Artur Marchewka, and Wojciech Szeszkowski. 2008. The effect of motivation on working memory: An fMRI and SEM study. Neurobiology of Learning and Memory 90: 475–78. [Google Scholar] [CrossRef]
- Taylor, Stephan F., Robert C. Welsh, Tor D. Wager, K. Luan Phan, Kate D. Fitzgerald, and William J. Gehring. 2004. A functional neuroimaging study of motivation and executive function. Neuroimage 21: 1045–54. [Google Scholar] [CrossRef]
- Thornton, Allen E., Vanessa G. Boudreau, Stephanie Y. Griffiths, Todd S. Woodward, Tanya Fawkes-Kirby, and William G. Honer. 2007. The impact of monetary reward on memory in schizophrenia spectrum disorder. Neuropsychology 21: 631. [Google Scholar] [CrossRef] [PubMed]
- Thurm, Franka, Nicolas Zink, and Shu-Chen Li. 2018. Comparing effects of reward anticipation on working memory in younger and older adults. Frontiers in Psychology 9: 2318. [Google Scholar] [CrossRef] [PubMed]
- Tian, Shuang. 2020. Effects of Reward Directing Attention and Space-Based Attention on VWM Representation. Master’s thesis, Shangdong Normal University, Jinan, China. Available online: https://kns-cnki-net-443.webvpn.sysu.edu.cn/KCMS/detail/detail.aspx?dbname=CMFD202002&filename=1020028310.nh (accessed on 15 December 2022). (In Chinese).
- Turner, Marilyn L., and Randall W. Engle. 1989. Is working memory capacity task dependent? Journal of Memory and Language 28: 127–54. [Google Scholar] [CrossRef]
- Unsworth, Nash, and Randall W. Engle. 2007. The nature of individual differences in working memory capacity: Active maintenance in primary memory and controlled search from secondary memory. Psychological Review 114: 104. [Google Scholar] [CrossRef]
- Unsworth, Nash, Keisuke Fukuda, Edward Awh, and Edward K. Vogel. 2014. Working memory and fluid intelligence: Capacity, attention control, and secondary memory retrieval. Cognitive Psychology 71: 1–26. [Google Scholar] [CrossRef]
- van den Berg, Ronald, and Wei Ji Ma. 2018. A resource-rational theory of set size effects in human visual working memory. eLife 7: e34963. [Google Scholar] [CrossRef] [PubMed]
- van den Berg, Ronald, Hongsup Shin, Wen-Chuang Chou, Ryan George, and Wei Ji Ma. 2012. Variability in encoding precision accounts for visual short-term memory limitations. Proceedings of the National Academy of Sciences of the United States of America 109: 8780–85. [Google Scholar] [CrossRef]
- van den Berg, Ronald, Qijia Zou, Yuhang Li, and Wei Ji Ma. 2023. No effect of monetary reward in a visual working memory task. PLoS ONE 18: e0280257. [Google Scholar] [CrossRef] [PubMed]
- Wallis, George, Mark G. Stokes, Craig Arnold, and Anna C. Nobre. 2015. Reward boosts working memory encoding over a brief temporal window. Visual Cognition 23: 291–312. [Google Scholar] [CrossRef]
- Watanabe, Masataka. 2007. Role of anticipated reward in cognitive behavioral control. Current Opinion in Neurobiology 17: 213–19. [Google Scholar] [CrossRef]
- Wei, Ziqiang, Xiao-Jing Wang, and Da-Hui Wang. 2012. From distributed resources to limited slots in multiple-item working memory: A spiking network model with normalization. The Journal of Neuroscience 32: 11228–40. [Google Scholar] [CrossRef] [PubMed]
- Wilken, Patrick, and Wei Ji Ma. 2004. A detection theory account of change detection. Journal of Vision 4: 1120–35. [Google Scholar] [CrossRef]
- Xu, Shiyang, Senqing Qi, Haijun Duan, Juan Zhang, Miriam Akioma, Fei Gao, Anise M. S. Wu, and Zhen Yuan. 2022. Task Difficulty Regulates How Conscious and Unconscious Monetary Rewards Boost the Performance of Working Memory: An Event-Related Potential Study. Frontiers in Systems Neuroscience 139: 716961. [Google Scholar] [CrossRef]
- Young, Mark R. 2005. The Motivational Effects of the Classroom Environment in Facilitating Self-Regulated Learning. Journal of Marketing Education 27: 25–40. [Google Scholar] [CrossRef]
- Zedelius, Claire M., Harm Veling, and Henk Aarts. 2012. When unconscious rewards boost cognitive task performance inefficiently: The role of consciousness in integrating value and attainability information. Frontiers in Human Neuroscience 6: 219. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Ke, and Jiehui Qian. 2022. The role of ensemble average differs in working memory for depth and planar information. Journal of Vision 22: 4. [Google Scholar] [CrossRef]
- Zhang, Ke, Dingguo Gao, and Jiehui Qian. 2021. Overestimation and contraction biases of depth information stored in working memory depend on spatial configuration. British Journal of Psychology 112: 230–46. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Sheng. 2021. Adaptive Invocation of Visual Working Memory Resources. Master’s thesis, Southwest University, Chongqing, China. Available online: https://kns-cnki-net-443.webvpn.sysu.edu.cn/KCMS/detail/detail.aspx?dbname=CMFD202201&filename=1021768929.nh (accessed on 15 December 2022). (In Chinese).
- Zhang, Weiwei, and Steven J. Luck. 2008. Discrete fixed-resolution representations in visual working memory. Nature 453: 233–35. [Google Scholar] [CrossRef]
- Zheng, Weixi, Jiayang Geng, Dexiang Zhang, Jie Zhang, and Jingpeng Qiao. 2022. Task difficulty rather than reward method modulates the reward boosts in visual working memory. Journal of Vision 22: 1. [Google Scholar] [CrossRef]
- Zhou, Qingjie, Zanzan Jiang, and Jinhong Ding. 2021. Reward Expectation Differentially Modulates Global and Local Spatial Working Memory Accuracy. Frontiers in Psychology 12: 744400. [Google Scholar] [CrossRef]
Study (Author/Year) | Sample Size | Type of Reward | Reward Methods | Type of WM | Paradigm of WM | Outcome Measures | Participant Group | Effect Size (d) |
---|---|---|---|---|---|---|---|---|
Allen and Ueno (2018) Experiment 1 | 20 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 1.12 |
Allen and Ueno (2018) Experiment 2 | 20 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 1.34 |
Allen and Ueno (2018) Experiment 3 | 20 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 2.85 |
Allen and Ueno (2018) Experiment 4 | 20 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 2.56 |
Beck et al. (2010) | 31 | Monetary | Reward expectation | Verbal | Recognition | RT | Healthy adults | 3.89 |
Belayachi et al. (2015) | 18 | Monetary | Reward expectation | Verbal | N-back | ACC, RT | Healthy adults | 0.18 |
Capa and Bouquet (2018) | 60 | Monetary | Reward expectation, Subliminal reward | Verbal | Others | ACC | Healthy adults | 1.46 |
Cho et al. (2018) | 65 | Monetary | Reward binding | Visual | Delay estimation | Error | Healthy adults, Others | 0.79 |
Cho et al. (2022) | 33 | Monetary | Reward binding | Visual | Delay estimation | Error | Healthy adults | 0.68 |
Choi et al. (2015) | 26 | Monetary | Reward expectation | Visual | Recognition | ACC | Healthy adults | 1.38 |
Cohen et al. (2019) | 20 | Monetary | Reward expectation | Verbal | Recognition | ACC | Healthy adults | 0.99 |
Di Rosa et al. (2019) | 21 | Monetary | Reward expectation | Visual | Recognition | ACC, RT | Others | 0.24 |
Gaillard et al. (2019) | 23 | Monetary | Reward expectation | Visual | Change detection | ACC, RT | Healthy adults | 0.45 |
Gaillard et al. (2020) | 32 | Monetary | Reward expectation | Visual | Change detection | ACC, RT | Healthy adults | 0.30 |
Grogan et al. (2022) Experiment 1 | 30 | Monetary | Reward expectation | Visual | Delay estimation | ACC, RT | Healthy adults | 0.86 |
Grogan et al. (2022) Experiment 4 | 30 | Monetary | Reward binding | Visual | Delay estimation | ACC, RT | Healthy adults | 0.98 |
Hager et al. (2015) | 56 | Monetary | Reward expectation | Verbal | N-back | ACC, RT | Healthy adults, Others | −0.01 |
Hammer et al. (2015) | 34 | Monetary | Reward expectation | Verbal | N-back | ACC | Healthy adults | 0.84 |
Heitz et al. (2008) | 30 | Monetary | Reward expectation | Verbal | Others | ACC, RT | Healthy adults | 0.52 |
Heuer and Schubö (2018) Experiment 1B | 14 | Monetary | Reward binding | Visual | Recognition | ACC, RT | Healthy adults | 1.17 |
Heuer and Schubö (2018) Experiment 2 | 24 | Monetary | Reward binding | Visual | Change detection | ACC | Healthy adults | 1.15 |
Hinault et al. (2019) | 19 | Monetary | Reward binding | Visual | Others | ACC, RT | Healthy adults | 1.88 |
Infanti et al. (2015) | 30 | Monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 2.85 |
Jiang (2013) Experiment 1 | 20 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 1.31 |
Jiang (2013) Experiment 2 | 20 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 1.13 |
Jimura et al. (2010) | 31 | Monetary | Reward expectation | Verbal | Recognition | RT | Healthy adults | 1.58 |
Kawasaki and Yamaguchi (2013) | 14 | Monetary | Reward expectation | Visual | Change detection | ACC | Healthy adults | 0.39 |
Klink et al. (2017) Experiment 1 | 10 | Monetary | Reward binding | Visual | Delay estimation | Error | Healthy adults | 3.23 |
Klink et al. (2017) Experiment 2 | 8 | Monetary | Reward expectation | Visual | Delay estimation | Error | Healthy adults | −0.02 |
Lin and Fougnie (2022) Experiment 1 | 56 | Monetary | Reward binding | Visual | Delay estimation | Error | Healthy adults | 1.04 |
Lin and Fougnie (2022) Experiment 2 | 56 | Monetary | Reward binding | Visual | Delay estimation | Error | Healthy adults | 1.52 |
Magis-Weinberg et al. (2019) | 50 | Monetary | Reward expectation | Verbal | Recognition | ACC, RT | Healthy adults | 0.50 |
Manga et al. (2020) | 146 | Monetary | Reward expectation | Visual | Delay estimation | Error, RT | Healthy adults, Others | 0.02 |
Morey et al. (2011) Experiment 1 | 32 | Monetary | Reward binding | Visual | Change detection | ACC | Healthy adults | 0.82 |
Morey et al. (2011) Experiment 2 | 30 | Monetary | Reward expectation | Visual | Change detection | ACC | Healthy adults | 0.67 |
Morey et al. (2011) Experiment 3 | 24 | Monetary | Reward binding | Visual | Change detection | ACC | Healthy adults | 1.08 |
Sanada et al. (2013) | 12 | Monetary | Reward expectation | Visual | Change detection | ACC | Healthy adults | 2.16 |
Sandry and Ricker (2020) Experiment 1 | 29 | Non-monetary | Reward binding | Visual | Recognition | ACC, RT | Healthy adults | 1.06 |
Sandry and Ricker (2020) Experiment 2 | 31 | Non-monetary | Reward binding | Visual | Recognition | ACC, RT | Healthy adults | 0.95 |
Sandry and Ricker (2020) Experiment 3 | 30 | Non-monetary | Reward binding | Visual | Recognition | ACC, RT | Healthy adults | 1.43 |
Savine et al. (2010) Experiment 2 | 30 | Monetary | Reward expectation | Verbal | Recognition | ACC, RT | Healthy adults | 0.24 |
Szatkowska et al. (2008) | 16 | Monetary | Reward expectation | Verbal | N-back | ACC, RT | Healthy adults | 0.00 |
Taylor et al. (2004) | 12 | Monetary | Reward binding | Visual | Recognition | ACC, RT | Healthy adults | 0.47 |
Thornton et al. (2007) | 102 | Monetary | Reward expectation | Visual | N-back | ACC | Healthy adults, Others | 0.00 |
Thurm et al. (2018) | 44 | Monetary | Reward binding | Verbal | N-back | ACC, RT | Healthy adults, Others | 0.97 |
Tian (2020) Experiment 1 | 24 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 2.12 |
Tian (2020) Experiment 2 | 24 | Non-monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 2.35 |
Xu et al. (2022) | 34 | Monetary | Subliminal reward | Verbal | N-back | ACC, RT | Healthy adults | 1.04 |
Zedelius et al. (2012) Experiment 1 | 41 | Monetary | Subliminal reward | Verbal | Recall | ACC | Healthy adults | 0.50 |
Zedelius et al. (2012) Experiment 2 | 33 | Monetary | Subliminal reward | Verbal | Recall | ACC | Healthy adults | 0.94 |
Zhang (2021) Experiment 1 | 26 | Non-monetary | Reward expectation | Visual | Recall | ACC | Healthy adults | 1.74 |
Zhang (2021) Experiment 2 | 30 | Non-monetary | Reward expectation | Visual | Recall | ACC | Healthy adults | 2.26 |
Zheng et al. (2022) Experiment 1 | 56 | Monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 0.55 |
Zheng et al. (2022) Experiment 2 | 56 | Monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 0.74 |
Zhou et al. (2021) | 20 | Monetary | Reward binding | Visual | Recall | ACC | Healthy adults | 1.22 |
No. of Studies | Sample Size | Cohen’s d | Standard Error | Z | p | 95% CI | Heterogeneity | Fail-Safe N | |||
---|---|---|---|---|---|---|---|---|---|---|---|
Q | p | I2 | |||||||||
Overall | 51 | 1711 | 0.93 | 0.08 | 11.71 | <0.001 | [0.77, 1.08] | 314.88 | <0.001 | 84.12 | 9362 |
Moderator analysis | |||||||||||
Type of reward | |||||||||||
Monetary | 39 | 1361 | 0.78 | 0.08 | 9.61 | <0.001 | [0.62, 0.94] | 210.23 | <0.001 | 81.92 | 4440 |
Non-monetary | 10 | 254 | 1.62 | 0.18 | 9.20 | <0.001 | [1.28, 1.97] | 30.66 | <0.001 | 70.65 | 713 |
Reward methods | |||||||||||
Reward binding | 23 | 703 | 1.10 | 0.10 | 11.30 | <0.001 | [0.91, 1.29] | 79.75 | <0.001 | 72.41 | 2581 |
Reward expectation | 28 | 1008 | 0.82 | 0.12 | 7.00 | <0.001 | [0.59, 1.05] | 226.72 | <0.001 | 88.09 | 2307 |
Subliminal reward | 4 | 168 | 0.72 | 0.23 | 3.16 | 0.002 | [0.27, 1.17] | 18.15 | 0.006 | 83.47 | 59 |
Type of WM | |||||||||||
Visual | 37 | 1150 | 1.05 | 0.10 | 11.15 | <0.001 | [0.87, 1.24] | 209.82 | <0.001 | 82.84 | 5712 |
Verbal | 14 | 561 | 0.62 | 0.14 | 4.58 | <0.001 | [0.36, 0.89] | 84.28 | <0.001 | 84.58 | 433 |
Paradigm of WM | |||||||||||
N-back | 7 | 300 | 0.43 | 0.19 | 2.25 | 0.025 | [0.05, 0.80] | 37.20 | <0.001 | 83.87 | 46 |
Change detection | 6 | 136 | 0.94 | 0.18 | 5.40 | <0.001 | [0.60, 1.29] | 13.34 | 0.002 | 62.51 | 118 |
Recognition | 13 | 349 | 0.79 | 0.13 | 5.97 | <0.001 | [0.53, 1.05] | 56.10 | <0.001 | 78.61 | 477 |
Recall | 10 | 306 | 1.23 | 0.19 | 6.40 | <0.001 | [0.85, 1.60] | 50.51 | <0.001 | 82.18 | 508 |
Delay estimation | 8 | 352 | 0.78 | 0.18 | 4.41 | <0.001 | [0.43, 1.13] | 41.82 | <0.001 | 83.26 | 240 |
Others | 3 | 109 | 1.24 | 0.40 | 3.07 | 0.002 | [0.45, 2.03] | 15.82 | <0.001 | 87.36 | 52 |
Outcome measures | |||||||||||
Error | 8 | 423 | 0.75 | 0.16 | 4.60 | <0.001 | [0.43, 1.08] | 43.06 | <0.001 | 83.75 | 250 |
ACC | 42 | 1257 | 0.95 | 0.09 | 10.88 | <0.001 | [0.78, 1.13] | 241.85 | <0.001 | 83.05 | 6265 |
RT | 20 | 690 | 0.69 | 0.14 | 5.08 | <0.001 | [0.42, 0.96] | 161.75 | <0.001 | 88.25 | 806 |
Participant group | |||||||||||
Healthy adults | 49 | 1452 | 0.95 | 0.08 | 11.53 | <0.001 | [0.79, 1.11] | 290.38 | <0.001 | 83.47 | 8503 |
Others | 6 | 225 | 0.33 | 0.16 | 1.99 | 0.047 | [0.01, 0.65] | 30.26 | <0.001 | 83.48 | 23 |
No. of Studies | Sample Size | Cohen’s d | Standard Error | Z | p | 95% CI | Heterogeneity | Fail-Safe N | |||
---|---|---|---|---|---|---|---|---|---|---|---|
Q | p | I2 | |||||||||
Monetary reward | |||||||||||
Outcome measures | |||||||||||
Error | 8 | 418 | 0.86 | 0.20 | 4.25 | <0.001 | [0.46, 1.26] | 54.02 | <0.001 | 87.04 | 247 |
ACC | 19 | 628 | 0.88 | 0.12 | 7.51 | <0.001 | [0.66, 1.12] | 90.71 | <0.001 | 80.16 | 1230 |
RT | 15 | 541 | 0.48 | 0.14 | 3.54 | <0.001 | [0.21, 0.74] | 95.86 | <0.001 | 85.40 | 227 |
Visual WM | |||||||||||
Outcome measures | |||||||||||
Error | 7 | 415 | 0.79 | 0.17 | 4.71 | <0.001 | [0.46, 1.12] | 41.77 | <0.001 | 85.64 | 252 |
ACC | 21 | 524 | 1.24 | 0.12 | 9.96 | <0.001 | [1.00, 1.49] | 81.13 | <0.001 | 75.35 | 2006 |
RT | 10 | 355 | 0.75 | 0.22 | 3.41 | 0.001 | [0.32, 1.18] | 97.27 | <0.001 | 90.76 | 170 |
Reward binding | |||||||||||
Type of reward | |||||||||||
Monetary | 15 | 469 | 1.03 | 0.08 | 13.40 | <0.001 | [0.88, 1.19] | 22.85 | 0.063 | 38.73 | 1155 |
Non-monetary | 8 | 198 | 1.53 | 0.19 | 7.88 | <0.001 | [1.15, 1.91] | 22.94 | 0.002 | 69.48 | 400 |
Reward expectation | |||||||||||
Type of reward | |||||||||||
Monetary | 20 | 724 | 0.52 | 0.10 | 5.08 | <0.001 | [0.32, 0.72] | 91.78 | <0.001 | 79.30 | 512 |
Non-monetary | 2 | 56 | 1.98 | 0.26 | 7.71 | <0.001 | [1.48, 2.49] | 1.43 | 0.232 | 29.98 | -- |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, W.; Yan, X.; He, X.; Qian, J. Evidence for the Beneficial Effect of Reward on Working Memory: A Meta-Analytic Study. J. Intell. 2024, 12, 88. https://doi.org/10.3390/jintelligence12090088
Wang W, Yan X, He X, Qian J. Evidence for the Beneficial Effect of Reward on Working Memory: A Meta-Analytic Study. Journal of Intelligence. 2024; 12(9):88. https://doi.org/10.3390/jintelligence12090088
Chicago/Turabian StyleWang, Weiyu, Xin Yan, Xinyu He, and Jiehui Qian. 2024. "Evidence for the Beneficial Effect of Reward on Working Memory: A Meta-Analytic Study" Journal of Intelligence 12, no. 9: 88. https://doi.org/10.3390/jintelligence12090088
APA StyleWang, W., Yan, X., He, X., & Qian, J. (2024). Evidence for the Beneficial Effect of Reward on Working Memory: A Meta-Analytic Study. Journal of Intelligence, 12(9), 88. https://doi.org/10.3390/jintelligence12090088