Journal of Intelligence

Research

Jump to: Other

21 pages, 706 KiB

Open AccessArticle

Progressing the Development of a Collaborative Metareasoning Framework: Prospects and Challenges

by Beth H. Richardson and Linden J. Ball

J. Intell. 2024, 12(3), 28; https://doi.org/10.3390/jintelligence12030028 - 1 Mar 2024

Cited by 2 | Viewed by 2007

Abstract

Metareasoning refers to processes that monitor and control ongoing thinking and reasoning. The “metareasoning framework” that was established in the literature in 2017 has been useful in explaining how monitoring processes during reasoning are sensitive to an individual’s fluctuating feelings of certainty and [...] Read more.

Metareasoning refers to processes that monitor and control ongoing thinking and reasoning. The “metareasoning framework” that was established in the literature in 2017 has been useful in explaining how monitoring processes during reasoning are sensitive to an individual’s fluctuating feelings of certainty and uncertainty. The framework was developed to capture metareasoning at an individual level. It does not capture metareasoning during collaborative activities. We argue this is significant, given the many domains in which team-based reasoning is critical, including design, innovation, process control, defence and security. Currently, there is no conceptual framework that addresses the nature of collaborative metareasoning in these kinds of domains. We advance a framework of collaborative metareasoning that develops an understanding of how teams respond to the demands and opportunities of the task at hand, as well as to the demands and opportunities afforded by interlocuters who have different perspectives, knowledge, skills and experiences. We point to the importance of a tripartite distinction between “self-monitoring”, “other monitoring” and “joint monitoring”. We also highlight a parallel distinction between “self-focused control”, “other-focused control” and “joint control”. In elaborating upon these distinctions, we discuss the prospects for developing a comprehensive collaborative metareasoning framework with a unique focus on language as a measure of both uncertainty and misalignment. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

15 pages, 640 KiB

Open AccessArticle

The Development of Intuitive and Analytic Thinking in Autism: The Case of Cognitive Reflection

by Kinga Morsanyi and Jayne Hamilton

J. Intell. 2023, 11(6), 124; https://doi.org/10.3390/jintelligence11060124 - 20 Jun 2023

Cited by 2 | Viewed by 20054

Abstract

The cognitive reflection test (CRT) is a short measure of a person’s ability to resist intuitive response tendencies, and to produce normatively correct responses that are assumed to be based on effortful, analytic thinking. A remarkable characteristic of the CRT is that although [...] Read more.

The cognitive reflection test (CRT) is a short measure of a person’s ability to resist intuitive response tendencies, and to produce normatively correct responses that are assumed to be based on effortful, analytic thinking. A remarkable characteristic of the CRT is that although the questions are open-ended, for each item, the vast majority of people either produce a correct, analytic response or a typical incorrect (i.e., intuitive) response. This unique feature of the CRT makes it possible to investigate the question of whether autistic and neurotypical people share the same intuitions. We report a study that included adolescents and young adults. In both age groups, autistic and neurotypical participants were matched on age, gender, cognitive ability, and educational background. In line with previous findings, the results showed an age-related increase in analytic responding on the CRT, and a decrease in intuitive responding. Crucially, the proportion of both intuitive and analytic responses across autistic and neurotypical participants was identical in both age groups. The current results are in contrast with claims that autistic individuals have an increased tendency toward an analytic/rational type of processing, which is commonly attributed to an impairment within their intuitive reasoning mechanisms. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

17 pages, 5104 KiB

Open AccessArticle

Tracing Cognitive Processes in Insight Problem Solving: Using GAMs and Change Point Analysis to Uncover Restructuring

by Mario Graf, Amory H. Danek, Nemanja Vaci and Merim Bilalić

J. Intell. 2023, 11(5), 86; https://doi.org/10.3390/jintelligence11050086 - 3 May 2023

Viewed by 3224

Abstract

Insight problems are likely to trigger an initial, incorrect mental representation, which needs to be restructured in order to find the solution. Despite the widespread theoretical assumption that this restructuring process happens suddenly, leading to the typical “Aha!” experience, the evidence is inconclusive. [...] Read more.

Insight problems are likely to trigger an initial, incorrect mental representation, which needs to be restructured in order to find the solution. Despite the widespread theoretical assumption that this restructuring process happens suddenly, leading to the typical “Aha!” experience, the evidence is inconclusive. Among the reasons for this lack of clarity is that many measures of insight rely solely on the solvers’ subjective experience of the solution process. In our previous paper, we used matchstick arithmetic problems to demonstrate that it is possible to objectively trace problem-solving processes by combining eye movements with new analytical and statistical approaches. Specifically, we divided the problem-solving process into ten (relative) temporal phases to better capture possible small changes in problem representation. Here, we go a step further to demonstrate that classical statistical procedures, such as ANOVA, cannot capture sudden representational change processes, which are typical for insight problems. Only nonlinear statistical models, such as generalized additive (mixed) models (GAMs) and change points analysis, correctly identified the abrupt representational change. Additionally, we demonstrate that explicit hints reorient participants’ focus in a qualitatively different manner, changing the dynamics of restructuring in insight problem solving. While insight problems may indeed require a sudden restructuring of the initial mental representation, more sophisticated analytical and statistical approaches are necessary to uncover their true nature. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

18 pages, 390 KiB

Open AccessArticle

Overconfidence in the Cognitive Reflection Test: Comparing Confidence Resolution for Reasoning vs. General Knowledge

by André Mata

J. Intell. 2023, 11(5), 81; https://doi.org/10.3390/jintelligence11050081 - 27 Apr 2023

Cited by 1 | Viewed by 2349

Abstract

This research examines the metacognitive awareness that people have about their reasoning performance in the Cognitive Reflection Test (CRT). The first two studies compare confidence judgments about the CRT vs. general knowledge (GK) questions. Results show that (1) people are generally able to [...] Read more.

This research examines the metacognitive awareness that people have about their reasoning performance in the Cognitive Reflection Test (CRT). The first two studies compare confidence judgments about the CRT vs. general knowledge (GK) questions. Results show that (1) people are generally able to discriminate between correct and incorrect answers, but this ability is far from perfect, and it is greater for GK questions than for CRT problems. Indeed, and strikingly, (2) incorrect responses to CRT problems are produced with approximately the same level of confidence as correct responses to GK questions. However, (3) even though confidence is high for incorrect responses to CRT problems, it is even higher for correct responses. The results of two additional studies show that these differences in confidence are ultimately related to the conflict that CRT problems pose between intuition and deliberation. These findings have implications for the possibility of implicit error monitoring and dual-process models of overconfidence. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

17 pages, 2816 KiB

Open AccessArticle

Tell Us What You Really Think: A Think Aloud Protocol Analysis of the Verbal Cognitive Reflection Test

by Nick Byrd, Brianna Joseph, Gabriela Gongora and Miroslav Sirota

J. Intell. 2023, 11(4), 76; https://doi.org/10.3390/jintelligence11040076 - 21 Apr 2023

Cited by 4 | Viewed by 7480

Abstract

The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of mathematical reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol [...] Read more.

The standard interpretation of cognitive reflection tests assumes that correct responses are reflective and lured responses are unreflective. However, prior process-tracing of mathematical reflection tests has cast doubt on this interpretation. In two studies (N = 201), we deployed a validated think-aloud protocol in-person and online to test how this assumption is satisfied by the new, validated, less familiar, and non-mathematical verbal Cognitive Reflection Test (vCRT). Verbalized thoughts in both studies revealed that most (but not all) correct responses involved reflection and that most (but not all) lured responses lacked reflection. The think-aloud protocols seemed to reflect business-as-usual performance: thinking aloud did not disrupt test performance compared to a control group. These data suggest that the vCRT usually satisfies the standard interpretation of the reflection tests (albeit not without exceptions) and that the vCRT can be a good measure of the construct theorized by the two-factor explication of ‘reflection’ (as deliberate and conscious). Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

20 pages, 1309 KiB

Open AccessEditor’s ChoiceArticle

The Role of Semantic Associations as a Metacognitive Cue in Creative Idea Generation

by Yoed N. Kenett, Noam Gooz and Rakefet Ackerman

J. Intell. 2023, 11(4), 59; https://doi.org/10.3390/jintelligence11040059 - 27 Mar 2023

Cited by 11 | Viewed by 2865

Abstract

Is my idea creative? This question directs investing in companies and choosing a research agenda. Following previous research, we focus on the originality of ideas and consider their association with self-assessments of idea generators regarding their own originality. We operationalize the originality score [...] Read more.

Is my idea creative? This question directs investing in companies and choosing a research agenda. Following previous research, we focus on the originality of ideas and consider their association with self-assessments of idea generators regarding their own originality. We operationalize the originality score as the frequency (%) of each idea within a sample of participants and originality judgment as the self-assessment of this frequency. Initial evidence suggests that originality scores and originality judgments are produced by separate processes. As a result, originality judgments are prone to biases. So far, heuristic cues that lead to such biases are hardly known. We used methods from computational linguistics to examine the semantic distance as a potential heuristic cue underlying originality judgments. We examined the extent to which the semantic distance would contribute additional explanatory value in predicting originality scores and originality judgments, above and beyond cues known from previous research. In Experiment 1, we re-analyzed previous data that compared originality scores and originality judgments after adding the semantic distance of the generated ideas from the stimuli. We found that the semantic distance contributed to the gap between originality scores and originality judgments. In Experiment 2, we manipulated the examples given in task instructions to prime participants with two levels of idea originality and two levels of semantic distance. We replicated Experiment 1 in finding the semantic distance as a biasing factor for originality judgments. In addition, we found differences among the conditions in the extent of the bias. This study highlights the semantic distance as an unacknowledged metacognitive cue and demonstrates its biasing power for originality judgments. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

16 pages, 2448 KiB

Open AccessArticle

Single-Heuristic Reasoning: Is It Still Dual-Process?

by Pavle Valerjev and Marin Dujmović

J. Intell. 2023, 11(2), 33; https://doi.org/10.3390/jintelligence11020033 - 8 Feb 2023

Viewed by 2605

Abstract

Traditionally, paradigms used to study conflict in reasoning (and metacognition during reasoning) pit heuristic processes against analytical processes. Findings indicate that the presence of conflict between processes prolongs reasoning and decreases accuracy and confidence. In this study, we aimed to explore reasoning and [...] Read more.

Traditionally, paradigms used to study conflict in reasoning (and metacognition during reasoning) pit heuristic processes against analytical processes. Findings indicate that the presence of conflict between processes prolongs reasoning and decreases accuracy and confidence. In this study, we aimed to explore reasoning and metacognition when only one type of heuristic process is exploited to cue multiple responses. In two experiments, a novel modification of the Base Rate neglect task was used to create versions in which one belief-based heuristic competes, or works in concert, with another of the same type to provide a response. Experiment 1 results reveal that the presence of conflict between cued responses does not affect meta-reasoning, which indicates that reasoning defaulted to a single process. An alternative explanation was that the effect of conflict was masked due to an imbalance in the strength of the dominant response between conflicting and congruent versions. Experiment 2 was designed to test hypotheses based on these competing explanations. Findings show that when the strength of a response was no longer masking the effect, the conflict did result in longer reasoning times and lower confidence. The study provides more robust evidence in favor of the dual-process account of reasoning, introduces a new methodological approach, and discusses how conflict may be modulated during reasoning. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

22 pages, 482 KiB

Open AccessEditor’s ChoiceArticle

Actively Open-Minded Thinking and Its Measurement

by Keith E. Stanovich and Maggie E. Toplak

J. Intell. 2023, 11(2), 27; https://doi.org/10.3390/jintelligence11020027 - 28 Jan 2023

Cited by 19 | Viewed by 12244

Abstract

Actively open-minded thinking (AOT) is measured by items that tap the willingness to consider alternative opinions, sensitivity to evidence contradictory to current beliefs, the willingness to postpone closure, and reflective thought. AOT scales are strong predictors of performance on heuristics and biases tasks [...] Read more.

Actively open-minded thinking (AOT) is measured by items that tap the willingness to consider alternative opinions, sensitivity to evidence contradictory to current beliefs, the willingness to postpone closure, and reflective thought. AOT scales are strong predictors of performance on heuristics and biases tasks and of the avoidance of reasoning traps such as superstitious thinking and belief in conspiracy theories. Nevertheless, AOT is most commonly measured with questionnaires rather than performance indicators. Questionnaire contamination becomes even more of a danger as the AOT concept is expanded into new areas such as the study of fake news, misinformation, ideology, and civic attitudes. We review our 25-year history of studying the AOT concept and developing our own AOT scale. We present a 13-item scale that both is brief and accommodates many previous criticisms and refinements. We include a discussion of why AOT scales are such good predictors of performance on heuristics and biases tasks. We conclude that it is because such scales tap important processes of cognitive decoupling and decontextualization that modernity increasingly requires. We conclude by discussing the paradox that although AOT scales are potent predictors of performance on most rational thinking tasks, they do not predict the avoidance of myside thinking, even though it is virtually the quintessence of the AOT concept. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

19 pages, 2545 KiB

Open AccessEditor’s ChoiceArticle

Alpha Suppression Is Associated with the Tip-of-the-Tongue (TOT) State Whereas Alpha Expression Is Associated with Knowing That One Does Not Know

by Edmund Qian-Long Shen, David Friedman, Paul Alexander Bloom and Janet Metcalfe

J. Intell. 2022, 10(4), 121; https://doi.org/10.3390/jintelligence10040121 - 8 Dec 2022

Cited by 1 | Viewed by 2851

Abstract

The tip-of-the-tongue (TOT) state is a spontaneously occurring metacognitive state that indicates that the answer to a query is almost, but not quite, at hand, i.e., that resolution is imminent. Since the time of William James, a distinctive feeling of nagging frustration has [...] Read more.

The tip-of-the-tongue (TOT) state is a spontaneously occurring metacognitive state that indicates that the answer to a query is almost, but not quite, at hand, i.e., that resolution is imminent. Since the time of William James, a distinctive feeling of nagging frustration has been observed to be associated with TOT states. On a more positive note, TOT states are also associated with intense goal-directed curiosity and with a strong desire to know that translates into successful mental action. The present study showed that prior to the presentation of resolving feedback to verbal queries—if the individual was in a TOT state—alpha suppression was in evidence in the EEG. This alpha suppression appears to be a marker of a spontaneously occurring, conscious, and highly motivating goal-directed internal metacognitive state. At the same time, alpha expression in the same time period was associated with the feeling of not knowing, indicating a more discursive state. Both alpha and alpha suppression were observed broadly across centro-parietal scalp electrodes and disappeared immediately upon presentation of the resolving feedback. Analyses indicated that the occurrence of alpha suppression was associated with participants’ verbal affirmations of being in a TOT state, which is also related to subsequent expression of a late positivity when feedback is provided, and to enhanced memory. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

29 pages, 8566 KiB

Open AccessArticle

A Validated Ontology for Metareasoning in Intelligent Systems

by Manuel F. Caro, Michael T. Cox and Raúl E. Toscano-Miranda

J. Intell. 2022, 10(4), 113; https://doi.org/10.3390/jintelligence10040113 - 24 Nov 2022

Cited by 4 | Viewed by 2195

Abstract

Metareasoning suffers from the heterogeneity problem, in which different researchers build diverse metareasoning models for intelligent systems with comparable functionality but differing contexts, ambiguous terminology, and occasionally contradicting features and descriptions. This article presents an ontology-driven knowledge representation for metareasoning in intelligent systems. [...] Read more.

Metareasoning suffers from the heterogeneity problem, in which different researchers build diverse metareasoning models for intelligent systems with comparable functionality but differing contexts, ambiguous terminology, and occasionally contradicting features and descriptions. This article presents an ontology-driven knowledge representation for metareasoning in intelligent systems. The proposed ontology, called IM-Onto, provides a visual means of sharing a common understanding of the structure and relationships between terms and concepts. A rigorous research method was followed to ensure that the two main requirements of the ontology (integrity based on relevant knowledge and acceptance by researchers and practitioners) were met. The high accuracy rate indicates that most of the knowledge elements in the ontology are useful information for the integration of multiple types of metareasoning problems in intelligent systems. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

29 pages, 1870 KiB

Open AccessArticle

When Type 2 Processing Misfires: The Indiscriminate Use of Statistical Thinking about Reasoning Problems

by Mário B. Ferreira, Jerônimo C. Soro, Joana Reis, André Mata and Valerie A. Thompson

J. Intell. 2022, 10(4), 109; https://doi.org/10.3390/jintelligence10040109 - 17 Nov 2022

Cited by 3 | Viewed by 2438

Abstract

Research on dual-process theories of judgment makes abundant use of reasoning problems that present a conflict between Type 1 intuitive responses and Type 2 rule-based responses. However, in many of these reasoning tasks, there is no way to discriminate between the adequate and [...] Read more.

Research on dual-process theories of judgment makes abundant use of reasoning problems that present a conflict between Type 1 intuitive responses and Type 2 rule-based responses. However, in many of these reasoning tasks, there is no way to discriminate between the adequate and inadequate use of rules based on logical or probabilistic principles. To experimentally discriminate between the two, we developed a new set of problems: rule-inadequate versions of standard base-rate problems (where base rates are made irrelevant). Across four experiments, we observed conflict sensitivity (measured in terms of response latencies and response confidence) in responses to standard base-rate problems but also in responses to rule-inadequate versions of these problems. This failure to discriminate between real and merely apparent (or spurious) conflict suggests that participants often misuse statistical information and draw conclusions based on irrelevant base rates. We conclude that inferring the sound use of statistical rules from normatively correct responses to standard conflict problems may be unwarranted when this kind of reasoning bias is not controlled for. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

24 pages, 1796 KiB

Open AccessArticle

I Choose to Opt-Out of Answering: Individual Differences in Giving Up Behaviour on Cognitive Tests

by Marvin K. H. Law, Lazar Stankov and Sabina Kleitman

J. Intell. 2022, 10(4), 86; https://doi.org/10.3390/jintelligence10040086 - 13 Oct 2022

Cited by 6 | Viewed by 3349

Abstract

Under the Meta-reasoning model, the process of giving up when a solution may not be feasible reflects an adaptive metacognitive strategy, where individuals opt-out of responding to mitigate error and resource costs. However, research is still needed to determine whether individuals systematically vary [...] Read more.

Under the Meta-reasoning model, the process of giving up when a solution may not be feasible reflects an adaptive metacognitive strategy, where individuals opt-out of responding to mitigate error and resource costs. However, research is still needed to determine whether individuals systematically vary in this behaviour and if so, which variables it meaningfully relates with. The current study (N = 176) is the first to examine factorial stability in giving up tendencies and its relationships with on-task confidence, cognitive ability, decision-making predispositions, and academic performance. To measure giving up tendencies, participants completed three cognitive tests which allowed for opting out, thereby capturing giving up frequency within each test and its consistency across tests. Participants also completed five other cognitive tasks embedded with confidence ratings, and a decision-making styles questionnaire. Confirmatory factor analyses were conducted on all giving up, confidence, and accuracy variables, with a three-factor solution having the best fit (containing a giving up factor, confidence factor, and cognitive ability factor). Supporting the proposed adaptive nature of giving up tendencies, the giving up factor correlated positively with cognitive ability, rational decision making, and academic performance. This research establishes factorial stability in giving up tendencies and provides a foundation for further investigation into its role within Meta-reasoning theory. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

► Show Figures

Figure 1

Other

Jump to: Research

14 pages, 366 KiB

Open AccessEssay

Why Should We Study the Foreign Language Effect: Debiasing through Affecting Metacognition?

by Michał Białek

J. Intell. 2023, 11(6), 103; https://doi.org/10.3390/jintelligence11060103 - 30 May 2023

Cited by 2 | Viewed by 3992

Abstract

Debiasing is a method of improving people’s decisions by reducing their reliance on salient intuitions causing them to behave suboptimally or biasedly. However, many of the known debiasing techniques have limited effectiveness or can only remedy a one-shot decision, rather than having a [...] Read more.

Debiasing is a method of improving people’s decisions by reducing their reliance on salient intuitions causing them to behave suboptimally or biasedly. However, many of the known debiasing techniques have limited effectiveness or can only remedy a one-shot decision, rather than having a lasting impact. In this work, I focus on the role of metacognition in debiasing decision-making and discuss how it can be better understood through the lens of the foreign language effect. The foreign language effect suggests that using a foreign language can sometimes benefit people’s decision-making without providing them with additional information or instructions regarding the task. However, we do not fully understand how the foreign language effect works and its limitations. I conclude by urging scientists to research this effect with the hope of having a lasting positive impact on society. Full article

(This article belongs to the Special Issue Metareasoning: Theoretical and Methodological Developments)

Journal Menu

Journal Browser

Metareasoning: Theoretical and Methodological Developments

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Published Papers (13 papers)

Research

Other

Further Information

Guidelines

MDPI Initiatives

Follow MDPI