GenAI Knowledge, Epistemic Orientation, and Intellectual Values Predict Undergraduate Students’ Critical GenAI Use

Abstract

Critically using generative AI (genAI) tools, such as comparing their outputs with other sources or the literature, is vital for students to avoid risks of reduced cognitive processing and inaccurate results. We propose two potential protective factors against these risks: knowledge about genAI and the disposition to engage in critical thinking encompassing epistemic orientation (i.e., a tendency to move away from more absolutist beliefs toward more evaluativist beliefs) and intellectual values (i.e., viewing intellectual engagement as worthwhile). Our correlational study with 67 undergraduate psychology students (59 female; mean age = 22.58) aimed to investigate how these factors predict students’ critical use of genAI. Regression and correlation analyses revealed that both knowledge about genAI and the disposition to engage in critical thinking predicted the critical use of genAI. Exploratory analyses revealed that students’ knowledge about genAI partly comprises misconceptions (i.e., incorrect knowledge with high confidence) and their critical genAI use correlates with need for cognition (NFC). These findings suggest that higher education institutions should implement interventions targeting conceptual misconceptions as well as epistemic orientation and intellectual values. Consequently, future research should experimentally evaluate such interventions to foster responsible AI engagement in higher education.

1. Introduction

Using generative AI (genAI), particularly chatbots such as ChatGPT or Google Gemini, has become an omnipresent element of university students’ everyday lives. Across the world, prevalence numbers keep rising and become outdated even before they are published. Already in 2023, one year after the public release of ChatGPT, about 70% of 500 students at an Italian university reported using chatbots for personal activities, and about 41% for university-related activities (Farinosi & Melchior, 2025). In a 2023 survey of 360 Harvard students, about 65% of respondents reported using or planning to use chatbots for academic work (Deschenes & McMahon, 2024). Furthermore, the UK’s Higher Education Policy Institute reported in 2025 that 92% of the 1000 surveyed students (66% in 2024) used some form of AI, with 88% (53% in 2024) using it for assessments (Freeman, 2025). Explaining concepts and summarizing materials were disclosed as the main purposes. Of about 500 students and scholars at three Indian universities, about 90% use AI for their academic activities (Archana et al., 2025). Finally, the Digital Education Council (2026) reported 92% prevalence of AI use among over 22,000 Latin American students. Two years earlier, they reported 86% prevalence among 3800 students from 16 countries (Digital Education Council, 2024).

Given genAI’s obvious ubiquity among university students, it makes sense to note that a mindless, uncritical use of genAI tools, such as chatbots, generally carries two risks: the first concerns reduced cognitive processing and weaker learning outcomes. These detrimental consequences, referred to as inversion effects, occur when students outsource the majority of their cognitive effort to genAI (Bauer et al., 2025). Such reduced performance and decline in knowledge is described as deskilling (Ferdman, 2026). For instance, in their field experiment, Bastani et al. (2025) found reduced performance in math education among students who had used generative AI, after access was subsequently removed. Furthermore, a recent systematic review revealed that students’ excessive reliance on genAI undermines the development of critical thinking skills (Zhai et al., 2024). The second risk is that the potential inaccuracy of genAI outputs may go unnoticed. After all, genAI tools, particularly chatbots, tend to generate results that are misleading at best and plain inaccurate at worst (Bauer et al., 2025). Such misleading or false outputs cannot be considered an aberration but a fundamental property of genAI with no technological fix (Banerjee et al., 2025; Ilievski et al., 2025).

Considering these risks, the importance of critical genAI use cannot therefore be overstated. Such a critical use of genAI comprises, for instance, comparing AI-generated answers with other sources or literature. The present paper proposes and empirically tests two potential protective factors against these risks and thus potential predictors of critical use of genAI: knowledge about genAI and a disposition to engage in critical thinking. Both have frequently been named as components of AI literacy (e.g., Allen & Kendeou, 2024; OECD/European Commission, 2026).

1.1. Knowledge About GenAI

Knowledge about genAI refers to at least a basic understanding of large language models, the technology behind genAI tools. Such knowledge should include, for instance, that a genAI system is a statistical text-completion machine rather than “intelligent” in a human sense (Bender et al., 2021; Ilievski et al., 2025). Seeking appropriate instruments to assess students’ knowledge about genAI, Lintner’s (2024) systematic review of 22 studies and 16 AI literacy scales is a sensible starting point. It should be noted that 13 of these 16 reviewed instruments rely on self-assessment items to be answered on Likert-type scales. Only three of these 16 instruments are performance-based. Obviously, self-assessment has its limitations, as it can be influenced by bias, overconfidence, and other factors (Praslova, 2018). Therefore, in the present paper, we decided to assess both self-reported and objective knowledge (i.e., knowledge assessed via a performance test).

Finally, we also aimed to examine potential misconceptions students might have about genAI. We follow the approach of Eitel et al. (2021) and Hefter et al. (2022) and differentiate between misconceptions and a lack of knowledge (i.e., missing conceptions) by considering respondents’ certainty. Briefly put, whereas a misconception manifests in an incorrect answer given with high confidence, a simple lack of knowledge is rather indicated by low confidence regardless of correctness.

1.2. Epistemic Orientation and Intellectual Values—The Disposition Toward Critical Thinking

Critical thinking has been receiving unabated enthusiasm and appeal as a key goal of education for many decades. Kuhn (1999) described critical thinking as “reflecting on what is known and how that knowledge is justified.” (p. 23). Almost 30 years later, the relevance of critical thinking has been further increased by the Internet (Tarchi et al., 2025), and it is still considered one of the most important future skills of the 21st century (BMBF, 2024). It makes sense to assume that an individual with a disposition towards critical thinking would also use genAI in a rather critical way, such as comparing AI-generated answers with other sources or literature.

In the current paper, we therefore focus on the disposition toward critical thinking, in particular on two prerequisites for critical (and argumentative) thinking proposed by Kuhn (2001): evaluativist beliefs and intellectual values. Evaluativist beliefs reflect the highest level of epistemological understanding (Kuhn, 2001). Briefly put, epistemological understanding basically describes how an individual thinks about the nature of knowledge and knowing (Hofer & Pintrich, 1997). The evaluativist level of epistemological understanding provides a rational basis for considering critical thinking reasonable. It is the final and most sophisticated level of epistemological understanding (following the absolutist and multiplist levels). Because such an evaluativist level of epistemological understanding develops only over many years (if at all), it seems reasonable not to focus directly on this level but rather on a tendency to move away from more absolutist beliefs toward more evaluativist beliefs. This tendency is referred to as the so-called epistemic orientation (Hefter et al., 2015).

The second prerequisite, intellectual values, refers to viewing intellectual engagement, such as critical thinking, as worthwhile (Kuhn & Park, 2005). Hefter (2024) identified intellectual values as a protective factor for distance learning from a how-to video. Whereas smartphone usage time negatively predicted learning outcomes, intellectual values positively predicted them. Furthermore, Hefter et al. (2015, 2018) successfully promoted both epistemic orientation and intellectual values through short-term video-based training interventions.

At first glance, intellectual values might seem similar to the need for cognition (NFC). However, in contrast to NFC, which refers to an individual’s tendency to “engage in and enjoy cognitively challenging activities” (Petty et al., 2009, p. 318), intellectual values represent the “perceived value of intellectual activity to a cultural group” (Kuhn & Park, 2005, p. 115). Furthermore, NFC is rather considered a personality trait, whereas intellectual values appear malleable via interventions (Hefter et al., 2015, 2018). Nevertheless, in addition to assessing epistemic orientation and intellectual values as central variables, we will also assess NFC to address exploratory research questions.

1.3. The Present Study

Despite the growing literature on educational generative AI, a notable knowledge gap and practice deficiency remain regarding the specific individual factors that prevent uncritical overreliance. While prior research often examines general AI literacy, the novelty of this study lies in the simultaneous investigation of both subjective and objective genAI knowledge (Section 1.1) alongside deeper psychological dispositions, namely epistemic orientation and intellectual values (Section 1.2), as predictors of critical genAI use. Addressing this gap holds substantial research significance and practical value for higher education, as it shifts the focus from purely technical tool training toward cultivating potential protective factors required for responsible genAI engagement.

Therefore, the present study’s main goal is to investigate possible connections between students’ critical use of genAI, their knowledge about genAI, and their disposition to engage in critical thinking (i.e., epistemic orientation and intellectual values).

To explore these connections, we chose a quantitative, correlational approach in an ecologically valid setting. In this study, we focused specifically on undergraduate university students, as this population represents a primary demographic navigating the rapid integration of genAI tools into higher education coursework.

Referring to our specific hypotheses in light of the aforementioned background, we assume that students’ knowledge about genAI would predict their critical use of genAI (Hypothesis 1). Furthermore, students’ disposition to engage in critical thinking (i.e., a high degree of epistemic orientation and intellectual values) would predict their critical use of genAI (Hypothesis 2).

The present study pursues two additional exploratory goals. First, we aim to take a closer look at students’ knowledge about genAI and identify potential misconceptions. In this regard, we will analyze the correctness of the students’ answers along with their reported certainty. Second, we aim to examine potential relationships between students’ actual use of genAI, their perceived usefulness of genAI, their interest in genAI, and their need for cognition (NFC).

2. Method

2.1. Sample and Procedure

To determine the required sample size, we conducted an a priori power analysis using GPower (Faul et al., 2007). For a multiple linear regression with two predictors (fixed model, R² deviation from zero), assuming a medium effect size (f² = 0.15, an alpha level of α = 0.05, and power of 1 − β = 0.80, the analysis indicated a required sample size of N = 68.

We recruited 70 undergraduate psychology students via our university’s online platform for research participants. The final sample that provided informed consent and completed the study comprised N = 67 (59 female, 7 male, 1 n/a; M_age = 22.58 years, SD_age = 5.47). Almost all students (95.5%) reported using ChatGPT for their studies, followed by Google Gemini (14.5%) and Microsoft Copilot (4.5%).

To address the hypotheses outlined in Section 1.3, this study employed a quantitative, non-experimental correlational research design. We selected this design because it is uniquely suited to examining the predictive relationships between psychological dispositions (i.e., epistemic orientation, intellectual values) and students’ (self-reported) behavioral outcomes in an ecologically valid educational setting. Establishing this correlational baseline is a necessary first step to identify the potential cognitive and dispositional predictors of critical genAI use before targeted instructional interventions or experimental manipulations can be designed and evaluated.

Our study took place completely online, and the participants completed the following questionnaires in that order: demographics (age, sex, and genAI tools used for their studies), interest in genAI, self-reported knowledge about genAI, knowledge test about genAI, usage of genAI, critical use of genAI, epistemic orientation, need for cognition, and intellectual values. We chose this fixed sequence intentionally to minimize potential priming effects and social desirability bias. Specifically, behavioral measures (usage and critical use) were assessed prior to eliciting reflections on deeper psychological dispositions and normative constructs (epistemic orientation, need for cognition, and intellectual values). This order ensured that participants were not unconsciously primed to alter their reported critical genAI behaviors to align with their subsequent reflections on intellectual values or cognitive dispositions. Furthermore, self-reported knowledge assessments were administered before the performance knowledge tests to reduce self-report biases driven by test performance feedback.

2.2. Measures and Instruments

2.2.1. Interest in GenAI

To assess students’ interest in genAI, we used four items based on Schiefele’s (1991) studies on topic interest and its emotional and value components. The stem “I consider the topic ‘learning with genAI’ as…” was followed by the attributes “boring” and “interesting” for the emotional component and “unimportant” and “useful” for the value component. Students had to answer all four items on a Likert-type scale ranging from 1 (fully disagree) to 4 (fully agree). After reverse-scoring the negatively keyed items, we used the mean of all four items as a measure of interest in genAI, with higher means indicating greater topic interest. The internal consistency was acceptable (Cronbach’s α = 0.76).

2.2.2. Self-Reported Knowledge About GenAI

We selected 18 items from Laupichler et al.’s (2023) 31-item “Scale for the assessment of non-experts’ AI literacy (SNAIL)” to assess self-reported knowledge about genAI. Please see Appendix A for the full 18-item list. Each item was an “I can…”-statement about a genAI-related skill and was answered on a Likert-type scale from 1 (fully disagree) to 6 (fully agree). We selected the 18 items from the 31 original items based on content relevance and ecological validity for the higher education context. We focused on knowledge we assume is relevant for students who use genAI for university or leisure, such as “I can tell if the technologies I use are supported by artificial intelligence.” We deliberately did not select items that we considered too far away from a typical university learning scenario, such as “I can explain why data security must be considered when developing and using artificial intelligence applications.” We also decided not to select items that focus too much on the technical background of AI, such as “I can explain how ‘reinforcement learning’ works on a basic level (in the context of machine learning).” We used the mean of all 18 items (Please see Appendix A) as a measure of self-reported knowledge about genAI, with higher values indicating greater knowledge. The internal consistency was high (Cronbach’s α = 0.90).

2.2.3. Objective Knowledge About GenAI

For our questionnaire on objective knowledge about genAI, we used 12 single-choice items. The first six items were adapted from Köhler and Hartig’s (2024) ChatGPT knowledge scale and featured two alternative answers: “correct” or “incorrect.” An example item is “GenAI provides the same answer to the same question. (incorrect)”

The other six items featured four alternative answers, with one being correct. We selected these six items from Hornberger et al.’s (2023) AI literacy test, which comprised 30 items. Our decision on the six items was based on face validity related to knowledge we assume is relevant to students who use genAI for university and leisure purposes. For instance, we selected the item “Imagine you are chatting with an assistant on the Internet. How could you proceed to find out whether you are interacting with a human or with an AI?” We decided not to include items we considered irrelevant to students’ daily life or too technical, such as “What is a key criterion for the quality of a model in machine learning?” We simply coded correct answers as +1, incorrect answers as 0, and calculated the mean across all 12 items, with higher values indicating greater objective knowledge about genAI. Please see Appendix B for the full 12-item list.

2.2.4. Misconception Score

Each of the 12 items we administered to assess objective knowledge about genAI was accompanied by a subjective certainty rating. Students had to answer “How certain are you?” on a Likert-type scale from 0 (very uncertain) to 4 (very certain). This approach enabled us to calculate a so-called misconception score according to the procedures of Eitel et al. (2021) and Hefter et al. (2022). First, we coded correct answers with +1 and incorrect answers with −1. Then, we multiplied each correctness value (+1 or −1) by the subjective certainty rating for each item, resulting in a possible range from −4 to +4. Higher positive values represent greater confidence in a correct answer, indicating “stronger certainty about correct knowledge.” Higher negative values, however, represent greater confidence in an incorrect answer, indicating stronger endorsement of a misconception. Finally, values near zero indicate low confidence regardless of correctness, suggesting missing conceptions rather than misconceptions or correct knowledge. The mean of all twelve multiplications yielded the misconception score, with higher values indicating fewer misconceptions.

2.2.5. Actual Usage of GenAI

To assess the student’s genAI usage, we adapted the ten items by Köhler and Hartig (2024) concerning typical AI usage scenarios, such as “I have used genAI to get feedback on the texts I created.” Furthermore, we developed nine additional items for even more scenarios we considered typical for students, such as “I have used gen AI to generate study materials or summaries to prepare for exams.” Appendix C provides the complete list of all 19 items. We used Köhler and Hartig’s (2024) four-option response format:

(a)

No, I don’t intend to either.

(b)

No, but I can imagine it.

(c)

Yes, but was not helpful.

(d)

Yes, was helpful.

This response format enabled us to address two dimensions, namely actual usage and perceived usefulness. To assess students’ actual use of genAI, we followed Köhler and Hartig’s (2024) procedure and coded “yes” as 1 and “no” as 0. We used the mean score across all 19 items as a measure of actual usage, with higher values indicating a greater extent of actual usage. The internal consistency was good (Cronbach’s α = 0.83).

2.2.6. Use Value of GenAI

We relied on the aforementioned 19 items to measure students’ actual use of genAI. Again, we followed Köhler and Hartig’s (2024) procedure, but this time we coded “can image it” and “was helpful” as 1, and “don’t intend to” and “was not helpful” as 0. We used the mean score across all 19 items as a measure of use value, with higher values indicating greater perceived usefulness. The internal consistency was good (Cronbach’s α = 0.75).

2.2.7. Critical Use of GenAI

To assess students’ critical use of genAI, we developed 10 theory-guided items mapping critical validation behaviors (such as cross-checking outputs and verifying factual accuracy), with students rating their agreement on a 5-point Likert-type scale from 1 (fully disagree) to 5 (fully agree). One example item is “I verify whether AI-generated calculations or statistical results are correct.” After reverse-scoring the negatively keyed item, we used the mean score across all 10 items as a measure of critical use of genAI, with higher values indicating a more critical use. The questionnaire showed an acceptable internal consistency (Cronbach’s α = 0.78). Please see Appendix D for the complete list of all 10 items.

2.2.8. Epistemic Orientation

We used Hefter et al.’s (2015) short scale to assess epistemic orientation. It comprised five statements, with which an individual who has moved away from the level of absolutist epistemological understanding would either agree (positively keyed items, e.g., “Even uncertain knowledge, such as is the case when scientific positions contradict each other, can be evaluated”) or disagree (negatively keyed items, e.g., “Scientific statements are either clearly true or false”). We let students mark their agreement on a 5-point Likert-type scale from 1 (fully disagree) to 5 (fully agree). After reverse-scoring the negatively keyed items, we used the mean of all five items as a measure of epistemic orientation, with higher values indicating a greater tendency toward non-absolutist beliefs. The internal consistency was acceptable (Cronbach’s α = 0.70).

2.2.9. Need for Cognition (NFC)

We relied on the 4-item NFC short scale by Beißert et al. (2014). It comprised four statements, such as “I prefer my life to be filled with puzzles that I solve.” Students had to mark their agreement to these statements on a 7-point Likert-type scale from 1 (fully disagree) to 7 (fully agree). After reverse-scoring the negatively keyed items, we used the mean of all four items as a measure of NFC, with higher values indicating a greater extent of NFC. The internal consistency was low (Cronbach’s α = 0.54), which is in line with the developers’ validation studies’ values of α = 0.51 to 0.54 (Beißert et al., 2014). They argue that their 4-item short-scale was never expected to reach high internal consistency because its four items address different facets of NFC. Consequently, a lower internal consistency is an acceptable trade-off for such an ultra-short scale. It prioritizes broad construct coverage and reduced participant burden over item homogeneity, provided that the instrument maintains its conceptual validity.

2.2.10. Intellectual Values

We assessed students’ intellectual values with Hefter et al.’s (2015) 14-item scale. Each item described a fictive person who considers an aspect of intellectual engagement worthwhile, such as “Person B enjoys discussing complex scientific subjects critically with other people.” or “Person K thinks it is worthwhile, to connect knowledge from one domain with knowledge from another.” Participants had to rate their similarity (“How similar is this person to you?”) to these 14 fictive persons subjectively on a Likert-type scale from 1 (not like me at all) to 6 (very much like me). We used the mean of all 14 items as a measure of intellectual values, with higher means referring to higher intellectual values. The internal consistency was high (Cronbach’s α = 0.90).

3. Results

We set the alpha level at 0.05 for all statistical analyses. Prior to testing our hypotheses, we conducted a descriptive overview of the sample’s psychometric profiles (see Table 1 for means and standard deviations). Overall, our undergraduate psychology students exhibited a rather high baseline interest in generative AI and a substantial level of critical genAI use. Regarding their disposition to engage in critical thinking, participants demonstrated rather strong epistemic orientations and highly valued intellectual engagement. Similarly, their need for cognition was robust. Concurrently, while students evaluated their subjective genAI knowledge moderately, the objective performance test indicated a more nuanced baseline understanding, setting a solid foundation for subsequent inferential analyses. Table 1 provides descriptive statistics and intercorrelations for all measures.

Table 1. Descriptive Statistics and Correlations for all Measures.

	Descriptives		Manifest Correlations
	M	SD	1	2	3	4	5	6	7	8	9	10
(1) Interest 1	3.32	0.60	—
(2) Self-reported knowledge 2	3.95	0.70	−0.11	—
(3) Objective knowledge 3	0.60	0.12	0.06	0.28	—
(4) Misconception score 4	0.70	0.64	−0.01	0.28	0.87	—
(5) Actual usage 3	0.55	0.23	0.53	−0.14	−0.13	−0.13	—
(6) Use value 3	0.63	0.19	0.54	−0.21	−0.11	−0.18	0.58	—
(7) Critical use 5	4.05	0.61	0.15	0.55	0.31	0.24	0.05	−0.18	—
(8) Epistemic orientation 5	4.07	0.61	−0.03	0.26	0.21	0.23	0.03	−0.19	0.45	—
(9) NFC 6	4.53	0.92	−0.18	0.37	0.14	0.11	−0.13	−0.27	0.25	0.26	—
(10) Intellectual values 2	4.26	0.78	0.04	0.66	0.29	0.31	−0.13	−0.13	0.48	0.22	0.46	—

Note. ¹ Mean score from 1 (lowest) to 4 (highest), ² Mean score from 1 (lowest) to 6 (highest). ³ Mean score from 0 (lowest) to 1 (highest), ⁴ Mean score from −4 (lowest) to 4 (highest), ⁵ Mean score from 1 (lowest) to 5 (highest), ⁶ Mean score from 1 (lowest) to 7 (highest), bold correlations: p < 0.05 (two-sided).

3.1. Knowledge About GenAI Predicts Critical GenAI Use

In our first hypothesis, we assumed that students’ knowledge about genAI would predict their critical genAI use. Correlation analyses revealed a significant positive correlation between subjective knowledge about genAI and critical use of genAI, r = 0.55, p < 0.001. Furthermore, we found a significant positive correlation between objective knowledge about genAI and critical use of genAI, r = 0.31, p = 0.011. Figure 1 illustrates both correlations in scatterplots.

Figure 1. Critical use of genAI predicted by (a) self-reported knowledge and (b) objective knowledge about genAI.

3.2. Epistemic Orientation and Intellectual Values Predict Critical GenAI Use

In our second hypothesis, we assumed that the students’ disposition to engage in critical thinking (i.e., a high degree of epistemic orientation and intellectual values) would predict their critical use of genAI. We therefore conducted a multiple linear regression analysis to examine whether and to what extent students’ epistemic orientation and intellectual values predict their critical use of genAI. The variance inflation factors (VIFs) were low (VIF = 1.05) for both predictors, indicating that multicollinearity was not a concern. Furthermore, the data met the assumption of independent errors (Durbin-Watson value = 2.23). Additionally, inspection of residual plots indicated that assumptions of normality, linearity, and homoscedasticity were met. The overall result of the regression was statistically significant, F(2, 64) = 17.19, p < 0.001, R² = 0.33.

More specifically, epistemic orientation was a significant positive predictor of critical use of genAI, β = 0.36, p < 0.001, as were intellectual values, β = 0.40, p < 0.001. The overall model fit (R² = 0.33) describes how 33% of the variance in students’ critical genAI use is predicted by their epistemic orientation and intellectual values. Both standardized coefficients reflect substantial medium predictive contributions. Figure 2 illustrates these results in a 3D scatter plot with a regression plane.

Figure 2. Critical use of genAI predicted by epistemic orientation and intellectual values. Black dots represent individual participants. To facilitate 3D spatial orientation, colored dots and thin lines indicate the orthogonal projections (shadows) of each data point onto the respective coordinate planes. The multicolored plane represents the estimated linear regression surface.

3.3. Explorative Look at Potential Misconceptions

Our first exploratory aim was to take a closer look at students’ knowledge about genAI and explore potential misconceptions. When examining the objective knowledge test about genAI, the correctness rates for our 12 items ranged from 3.0% (Item 4) to 91.0% (Item 2). In other words, 97.0% of the participants believed that “AI chatbots perform web searches.” (Item 4). However, when looking at the misconception score, which combines correctness and certainty ratings, only 9.0% of participants had a misconception score below zero. This result indicates a mixture of a few actual misconceptions and a larger number of missing concepts. Looking at individual item scores, 95.5% of participants had values below zero for the most difficult item, Item 4. The scores were below zero for Items 4, 10, and 11, indicating the presence of actual misconceptions rather than missing concepts. Table 2 shows each item’s scores. For the answer options and the correct answers, please see Appendix B.

Table 2. Percentages of incorrect answers and mean misconception score with standard deviations in parentheses for each item.

Item	Incorrect Answers	Misconception Score
GenAI can reliably solve mathematical problems.	47.8	0.22 (2.94)
GenAI provides the same answer to the same question.	9.0	2.70 (1.70)
GenAI is designed to imitate human speech behavior as closely as possible.	23.9	1.52 (2.45)
GenAI performs web searches.	97.0	−3.00 (1.26)
GenAI may provide content that is not based on facts.	10.4	2.91 (1.95)
GenAI retrieves daily updated information.	47.8	0.09 (2.72)
Imagine you are chatting with an assistant on the Internet. How could you proceed to find out whether you are interacting with a human or with an AI?	41.8	0.31 (2.41)
What makes AI intelligent?	31.3	0.91 (2.30)
Why do AI systems behave intelligently?	11.9	2.05 (1.88)
What are knowledge representations in the field of AI?	79.1	−0.79 (1.75)
How do AI systems make decisions?	68.7	−0.61 (2.27)
What determines the behavior of AI systems?	10.4	2.12 (1.77)

3.4. Exploratory Correlation Analyses

To address our second exploratory goal outlined in Section 1.3, we examined the relationships between students’ actual genAI use, perceived usefulness, interest, and their need for cognition (NFC). In our more exploratory correlation analyses, we found that students’ actual use of genAI correlated significantly only with their interest in genAI, r = 0.54, p < 0.001, and their perceived usefulness of genAI, r = 0.58, p < 0.001. However, we could not detect correlations with other factors such as students’ knowledge about genAI, their disposition to engage in critical thinking (i.e., epistemic orientation and intellectual values), or their need for cognition.

Moreover, regarding the specific role of cognitive dispositions, students’ perceived usefulness correlated negatively with their need for cognition, r = −0.27, p = 0.027, suggesting that individuals with a higher need for cognition perceive the genAI tools’ utility more skeptically. Finally, the students’ need for cognition correlated positively with their critical use of genAI, r = 0.25, p = 0.044, empirically suggesting that a stable disposition toward cognitive effort is directly linked to more critical genAI use.

4. Discussion

4.1. Knowledge About genAI and the Disposition Toward Critical Thinking

In the present study, our main goal was to examine possible connections between students’ critical use of genAI and two distinct predictors thereof: knowledge about genAI and a disposition toward critical thinking (i.e., a high degree of epistemic orientation and intellectual values). We found empirical evidence that critical use of genAI is indeed positively associated with these two predictors. First, knowledge about genAI (both subjective and objective knowledge) predicted students’ critical use of genAI (Hypothesis 1). It is noteworthy that we relied on instruments to assess both subjective knowledge (via self-report scale) and objective knowledge (via performance–based test). This approach enabled us to cross-validate both instruments, as recommended by Lintner (2024).

Descriptively, students’ critical genAI use exhibited a stronger correlation with subjective knowledge (r = 0.55) compared to objective knowledge (r = 0.31). However, this descriptive, not statistically significant difference should not be over-interpreted on a theoretical level. It likely reflects methodological rather than substantial cognitive differences. Specifically, both the subjective knowledge scale and the critical use scale rely on self-reported, Likert-type questionnaires. This structural and partly content-related similarity (see Appendix A and Appendix B) may introduce a common-method bias, artificially elevating their association. In contrast, the objective knowledge assessment utilized a performance-based test format, which naturally minimizes shared method variance.

Second, both facets of the disposition to engage in critical thinking (i.e., epistemic orientation and intellectual values) accounted for about one-third of the variance in students’ critical use of genAI (Hypothesis 2). This result underscores the importance of intellectual values as a protective factor against cognitive miserliness (Barr et al., 2015). Cognitive miserliness refers to the unwillingness to incur cognitive costs and the preference to outsource information and cognitive processes to technology, such as genAI tools, but also smartphones. For instance, Hefter (2024) found that intellectual values were beneficial to distance learning outcomes, whereas smartphone usage time was detrimental.

Overall, the present findings allow us to conclude that both knowledge about genAI and the disposition to engage in critical thinking (i.e., a high degree of epistemic orientation and intellectual values) can serve as protective factors against uncritical overreliance on genAI. From an educational and practical perspective, these findings can lay the groundwork for promising instructional interventions. After all, both protective factors against uncritical overreliance on genAI (i.e., knowledge about genAI and the disposition to engage in critical thinking) can be effectively trained and fostered. Obviously, knowledge about genAI can be acquired through various (online) courses, both within and outside academia, such as those available on the KI-Campus online platform (Stifterverband, 2026). Furthermore, even the disposition to engage in critical thinking can be fostered. For instance, Hefter et al. (2015) successfully promoted epistemic orientation and Hefter et al. (2015, 2018) fostered intellectual values through short-term video-based training interventions.

4.2. Potential Misconceptions About GenAI

Through an exploratory lens, we were interested in potential misconceptions about gen AI. Calculating misconceptions scores by combining students’ answer correctness and confidence enabled us to identify potential misconceptions. As mentioned above, misconceptions manifest in incorrect answers given with high confidence. Our findings showed that only 9% of participants had misconceptions. Overall, only three of the 12 items had misconception scores below zero, indicating the presence of actual misconceptions. For instance, 96% of the participants believed with confidence that “AI chatbots perform web searches,” a typical misconception. However, this item, adapted from Köhler and Hartig’s (2024) ChatGPT knowledge scale, might be misleading because it can be considered surpassed by the technical capabilities of current chatbots. This aspect will be discussed further in the Limitations section.

Most items had positive scores, indicating that participants did not give incorrect answers with high confidence, suggesting they did not have misconceptions. For instance, about 90% of the participants correctly answered that “GenAI may provide content that is not based on facts.” This item’s misconceptions score was 2.91, the highest of all items, indicating correct answers with rather high confidence.

Overall, with our specific sample and questionnaire, we could not detect a high prevalence of misconceptions about genAI. Rather, only certain aspects of genAI, such as the aforementioned web search, were associated with misconceptions. From an educational and practical perspective, these findings give cause for cautious optimism. A general lack of knowledge about genAI could be tackled with various (online) courses. For the specific misconceptions, interventions based on extended refutation texts are recommended for an effective and convenient remedy (e.g., Hefter, 2026).

4.3. Exploratory Correlation Analyses and Role of NFC

Finally, the results of our additional exploratory analyses also appear noteworthy. In particular, the negative correlation between students’ perceived usefulness of genAI and their NFC is understandable as NFC reflects a positive attitude toward engaging in cognitive tasks (Cacioppo et al., 1984). Delegating cognitive tasks to genAI tools would contradict this need. Furthermore, the positive correlation between students’ NFC and their critical use of genAI further validates our newly developed questionnaire for critical use of genAI, as uncritical use of genAI outputs would also run counter to NFC.

Synthesizing these findings, NFC should not be viewed as an isolated exploratory variable, but rather as a foundational individual disposition that contextualizes our main hypotheses. While genAI knowledge and critical thinking dispositions provide the cognitive frameworks necessary for critical genAI use (Hypotheses 1 and 2), NFC points to the intrinsic motivational drive required to execute these cognitively demanding validation behaviors. Individuals with higher NFC are inherently more willing to invest the mental effort needed to cross-check outputs, thereby bridging the gap between theoretical genAI literacy and actual critical behavior. Integrating NFC into the broader argument of the paper suggests that considering learners’ NFC might be advisable when developing the instructional interventions to enhance critical use of genAI. For instance, during an intervention to foster the disposition to engage in critical thinking, learners with low NFC might need extra incentives to engage mentally, such as feedback or interactivity.

4.4. Limitations and Outlook for Future Research

For a start, our study’s relatively small sample consisted of undergraduate psychology students. Hence, our sample is not representative of students in other subjects, let alone of other populations, limiting our study’s generalizability. Furthermore, voluntary participation in the study might have introduced selection bias, as our participants may have had a greater interest in or knowledge of genAI than average students. Additionally, the sample exhibits a substantial gender imbalance (88% female), which reflects typical demographics in undergraduate psychology programs. These sample characteristics may further constrain generalizability in specific ways. For instance, psychology students receive rigorous training in evaluating scientific sources, potentially resulting in higher baseline levels of epistemic orientation and intellectual values than in other disciplines. In contrast, students in STEM fields might possess higher objective technical genAI knowledge but different critical thinking dispositions. While the foundational mechanisms (i.e., knowledge and critical thinking dispositions predict critical genAI use) are expected to operate universally, the relative weight and baseline levels of these predictors likely vary across disciplines, educational levels, and demographic backgrounds. Finally, while our sample size was explicitly powered to detect medium-large predictive relationships, it limits the statistical power to identify very small or nuanced exploratory correlation effects, which warrants replication in larger samples.

Moreover, our new instrument of critical genAI use relies solely on the learners’ self-reports. Hence, it shares the typical disadvantages of subjective scales, such as a possible bias (Praslova, 2018). Future research could thus also develop and implement performance-based assessments of critical genAI use. More specifically, researchers could adopt standardized performance-based measures such as Jin et al.’s (2025) Generative AI Literacy Assessment Test (GLAT), which was designed to objectively assess higher education students’ genAI literacy. Implementing such performance-based instruments as well as objective tracking of real-time student behavior (e.g., via logs) would reduce self-report biases and provide a more objective assessment of students’ actual critical evaluation skills.

Beyond behavioral testing, future research could also implement multi-method validation approaches to strengthen confidence in the observed predictive paths. To mitigate common-method variance, studies could integrate educator or peer evaluations of students’ genAI engagement. Furthermore, establishing criterion validity through objective behavioral artifacts—such as screen-capturing or browser-tracking—would cross-validate subjective metrics against real-world actions. Finally, transitioning from correlational to experimental or longitudinal research designs is required to establish causality and demonstrate whether fostering genAI knowledge, epistemic orientation, and intellectual values directly enhances critical genAI behavior.

Another outlook for future research concerns the knowledge and potential misconceptions about genAI and its impact on critical genAI use. We suppose that it is not only knowledge about genAI but also knowledge about human learning that plays a major role when it comes to critical genAI use. For instance, an individual who compares genAI responses with other sources or literature might do so due to knowledge that genAI outputs could be incorrect. However, it seems plausible that this individual also does it because of knowing that outsourcing cognitive processes harms one’s own learning. Future research might thus explore the role of knowledge about human learning as a sort of meta-knowledge for informed decisions of critical genAI use. This metacognitive understanding of human cognition and learning processes could serve as an additional, powerful predictor of critical genAI use. According to theories of self-regulated learning (SRL) and metacognition, learners with metacognitive knowledge about how memory, cognitive effort, and deep processing operate are better equipped to monitor their learning and adapt their tactics and strategies accordingly (e.g., Winne & Hadwin, 1998). They potentially understand that premature cognitive offloading to genAI curtails their own competence development (cf. Ngai & Gilbert, 2026; Xu et al., 2025). Future research should therefore explicitly assess knowledge about human learning alongside knowledge about genAI to map the nomological network of critical genAI use fully.

Another key focus for future research concerns the development of interventions on critical genAI use. Recent articles (Abedin et al., 2026; Lund et al., 2025) underscore the importance of teaching students about AI literacy, AI ethics, and the risk of AI overreliance. Based on our findings, interventions to foster critical use of genAI could focus on addressing its predictors: knowledge about genAI and a disposition toward critical thinking. As critical thinking remains highly relevant (Tarchi et al., 2025), digital training interventions that foster the disposition toward it (e.g., Hefter et al., 2015) provide a valuable framework to build upon.

Moreover, future research might also focus on directly promoting the critical use of genAI without taking the detour through its predictors. A promising starting point for such direct interventions is modeling examples (e.g., Hefter et al., 2022). Such modeling examples could demonstrate concrete strategies of critical genAI use, such as comparing AI-generated answers with other sources or with the literature.

Finally, genAI tools are evolving constantly. As a consequence, an item such as “AI chatbots perform web searches”, which we adapted from Köhler and Hartig (2024) and used for assessing objective knowledge and potential misconceptions about genAI, can already be considered surpassed by the technical capabilities of current chatbots. Strictly speaking, the large language model (LLM) underlying a chatbot such as ChatGPT cannot perform a web search. However, the students’ subjective experience is, of course, that it does, because ChatGPT triggers a web search and uses the results in the final generated output. Furthermore, an item such as “What makes AI intelligent?” by Hornberger et al. (2023) may be more applicable to symbolic AI systems and reinforcement learning agents, less so to modern chatbots (which most students associate with AI). Such an item might even be misleading if students equate LLMs and AI. Students will probably not consider the goal of an LLM (i.e., maintaining a high likelihood of output token sequences) as a rational act.

Importantly, the ongoing technical shift raises broader questions about the instrument’s validity for assessing genAI knowledge. Because the technological landscape changes rapidly, static items can shift from measuring technical understanding to measuring mere awareness of user interface features, such as the aforementioned web search item. While this does not compromise the internal validity of our current findings—as it accurately captured the students’ conceptual understanding at the time of data collection—it underscores a critical limitation and outlook for future research. To maintain psychometric rigor, future measurement frameworks should shift toward dynamic, feature-independent items that evaluate foundational architectural principles of AI rather than evolving and changing software functionalities. Overall, future research should focus on these updated items for the genAI knowledge test that better reflect current genAI tools’ evolving technology and performance.

5. Conclusions

This study demonstrated the predictive power of both knowledge about genAI and the disposition to engage in critical thinking (i.e., epistemic orientation and intellectual values) on students’ critical genAI use. Furthermore, our exploratory analyses highlighted widespread conceptual misconceptions and identified the need for cognition (NFC) as another key variable. Although the small, subject-specific sample limits immediate generalizability and long-term instrument validity, this study contributes foundational theoretical knowledge to the literature by establishing a multifaceted framework of individual protective factors against uncritical AI overreliance. Practically, our findings imply that higher education institutions should move beyond basic AI training. Consequently, we recommend that future research focus on developing and experimentally evaluating instructional interventions that target conceptual misconceptions, epistemic orientation, and intellectual values to foster responsible AI engagement in higher education.