Metacognitive Scaffolding in the Age of GenAI: A Behavioral Analysis of Student–Chatbot Interactions During Course Selection

Abstract

Course selection presents a persistent challenge for students who often have difficulty articulating clear goals, integrating multiple considerations, and aligning academic choices with personal and professional aspirations. This study investigates whether concept mapping, as a metacognitive scaffolding tool, may shape how students interact with Generative AI (GenAI) systems during academic decision-making. In a randomized controlled experiment, 180 undergraduates at a polytechnic university in China were assigned to either a GenAI-only condition or a GenAI + Concept Map condition. After excluding 3 outlier participants, 177 students were included in the final analysis. Controlling for prior academic performance via ANCOVA, students with concept-map support showed different interaction patterns: they had a longer maximum consecutive-question chain within a session (GPA-adjusted means: 11.92 vs. 9.07 questions), formulated longer questions (15.27 vs. 11.93 words), and spent more time per conversation session on average (8.05 vs. 6.77 min). An analysis of conversation content showed that the concept-map group discussed a wider range of course selection factors (covering 4.46 vs. 3.66 main dimensions and 8.70 vs. 6.36 detailed factors). Epistemic Network Analysis further suggested that concept-map users linked different factors more frequently in their conversations, connecting academic requirements with career development, intrinsic interests, and external recognition in their discourse. Notably, these group differences remained after controlling for GPA in the ANCOVA models. These findings suggest that metacognitive scaffolding may reshape the way students engage with GenAI, with concept-map users shifting from brief exchanges to extended conversations covering multiple integrated factors related to their academic choices.

1. Introduction

Course selection represents a complex decision-making process in higher education, as students need to balance academic requirements, interests, and career goals while navigating extensive course catalogs. Course recommendation systems have emerged as a critical component of academic counseling, providing personalized learning suggestions to facilitate this process (Atalla et al., 2023). Despite their value, these systems face persistent challenges, including cold-start and data-sparsity problems, limited use of learner attributes, large and dynamic course-decision spaces, and a tendency in some deep-learning recommenders to prioritize accuracy over explainability (Algarni & Sheldon, 2023; Nafea et al., 2019; Yang et al., 2024). Researchers have proposed improving explainability through deep learning with semantic information and ontology-based semantic association (Li et al., 2022; Yang et al., 2024), as well as introducing feedback mechanisms to adjust recommendation strategies (Nafea et al., 2019). Generative AI (GenAI) offers a promising solution to these limitations. GenAI systems can analyze students’ requirements, interests, and career goals to provide personalized recommendations through natural language interactions. Research has demonstrated that advanced language models can improve decision support in major selection (Lekan & Pardos, 2024). Evidence suggests (Akiba & Fraboni, 2023; Chen et al., 2024) that GenAI integration offers improved adaptability and personalization through processing diverse information and engaging in natural language dialogues. Despite substantial research on recommendation technologies, in-depth exploration of using large language models for course recommendations in higher education remains limited.

Despite these promising developments, three critical gaps persist in the current research landscape. First, existing GenAI-based course recommendation studies predominantly focus on technical implementation, such as improving recommendation algorithms or expanding knowledge bases, rather than addressing students’ real-time cognitive needs during the recommendation process. Second, research examining how metacognitive tools can enhance human–AI collaboration in academic advising contexts remains scarce. Third, there is a lack of validated frameworks for analyzing student–AI interaction patterns beyond simple satisfaction metrics. These gaps highlight the need for research that examines how metacognitive tools affect student–AI interaction patterns and conversation dynamics in educational decision-making contexts. Current GenAI systems still exhibit limitations in supporting students’ decision-making processes. Consistent with earlier findings on information-seeking difficulty (Kuhlthau, 1991), students often struggle to articulate their needs clearly and consider various factors comprehensively during course selection. This challenge often manifests in brief, surface-level interactions where students ask simple questions and accept recommendations without exploring alternatives. Furthermore, existing systems lack tools to help students articulate and connect different selection criteria, potentially leading to limited exploration of options and quick acceptance of initial suggestions.

To address these limitations, this study introduces concept maps as a metacognitive support tool when students interact with GenAI systems. Concept maps (Novak, 1990) are visual metacognitive tools representing relationships between concepts (Fang et al., 2023). By illustrating hierarchical relationships, concept maps facilitate understanding and promote meaningful learning (Novak, 1990). Research suggests that concept maps can improve learning outcomes in higher education (Machado & Carvalho, 2020). By providing students with visual knowledge structures, this approach may change how they communicate with AI systems, potentially leading to more detailed questions, longer conversations, and more comprehensive discussions of selection factors.

While concept maps in education have been extensively researched (Lenski et al., 2022; Schroeder et al., 2018), studies on their application in AI-assisted course recommendations remain scarce. This study explores how integrating concept maps with GenAI chatbots is associated with changes in students’ interaction behaviors and the range of factors they discuss during course selection conversations. By examining these technologies within academic advising contexts, this research aims to understand how concept maps influence student–AI interaction patterns and conversation content. Specifically, this study addresses three interconnected research questions that progress from interaction quality to decision-related discourse and its relational structure:

RQ1: How do concept mapping tools change the way students interact with AI during course selection?

RQ2: Do concept maps help students express a wider variety of course selection factors in their AI conversations?

RQ3: What patterns emerge in how students link different reasons together when discussing course selection with and without concept maps?

This study makes three main contributions. First, this study provides evidence that the concept map, as a metacognitive scaffold, was associated with changes in how students interact with generative AI assistants in authentic course-selection scenarios. Second, this study combines GPA-adjusted analysis of covariance with Epistemic Network Analysis, and thereby characterizes the quality of human–AI interaction across three dimensions: quantity, breadth, and relational structure. Third, this study not only extends the classic Academic Motivation Scale into a five-dimensional, twenty-subcategory analytic framework tailored to course selection but also shows that the group differences in several interaction-quality indicators remained after GPA adjustment, which suggests that AI-assisted course advising may support broader access to deeper engagement across levels of prior academic achievement.

2. Literature Review

2.1. Metacognitive Strategies in Human and AI Interactions

The application of metacognitive strategies in human–AI interactions is gaining increasing attention, particularly in AI recommendation systems (Afzaal et al., 2024) and automated learning environments (Liang et al., 2024; Xu et al., 2025). As AI systems become more prevalent in educational contexts, researchers have begun examining how metacognitive skills enable users to more effectively leverage these tools (Goyal, 2025).

Metacognitive skills, which include the ability to monitor, regulate, and control one’s cognitive processes (Brown et al., 1983), are becoming increasingly important for users of collaborative AI systems (Sidra & Mason, 2024). Metacognitive strategies such as self-monitoring, reflection, and goal setting not only improve learners’ control over their learning processes but also help them better understand and evaluate the information provided by AI. These strategies allow students to more efficiently plan and adjust their learning processes, leading to more effective learning outcomes (Fang et al., 2023). In complex AI interactions, these strategies encourage users to actively organize their questions, ensuring that the feedback provided by AI aligns more closely with their personal learning needs and goals.

In AI recommendation systems, the effective application of metacognitive strategies is reflected at multiple levels. First, self-monitoring strategies help users track AI-generated recommendations, analyzing whether these suggestions align with their personal learning goals. Through this strategy, users can promptly evaluate how well AI recommendations match their learning progress, interests, and needs. Second, reflection strategies encourage users to engage in critical thinking while using AI tools, thoroughly assessing the AI’s suggestions to avoid over-reliance on technology, which might cause them to overlook their own learning needs. In AI-assisted learning environments, AI applications can support students’ self-regulated learning across the forethought, performance, and reflection phases (Lan & Zhou, 2025).

Moreover, goal-setting strategies enable learners to clarify their learning directions and organize their cognitive processes toward clear learning objectives. Systematic reviews have demonstrated that effective self-regulated learning (SRL) strategies, including goal setting, planning, monitoring, and reflection, form the foundation for academic success, with these strategies significantly enhancing learning performance, motivation, and engagement (Waheed et al., 2025). Research further indicates that technologies such as AI, learning management systems, and collaborative platforms can support SRL by providing personalized feedback and facilitating autonomous learning (Faza & Lestari, 2025). Through goal setting and self-monitoring, learners can maintain a high sense of control and engagement, prompting timely adjustments to their learning paths and improving learning outcomes. Together with autonomy, these SRL strategies enable individuals to more effectively regulate their behaviors when facing learning challenges, ultimately achieving their learning goals.

While the importance of metacognitive strategies in education and AI-assisted learning has been acknowledged to some extent, research on how metacognitive tools affect the effectiveness of AI-assisted course recommendation services is still limited. In particular, there is a lack of investigation into the concrete applications and potential impacts of these strategies in GenAI chatbot environments. The role of metacognitive strategies in improving the effectiveness of AI-assisted course recommendation and their broader implications warrant further study.

2.2. Concept Maps in Education

Concept maps are visual tools that graphically represent knowledge by organizing concepts and illustrating the relationships between them through connecting lines. These maps help users structure information hierarchically, with the most general concepts at the top and more specific ones below, connected by linking words that explain their relationships (Koufou, 2016; Novak, 1990).

Whether in primary or higher education, concept maps have demonstrated broad applicability and value as a versatile learning tool, contributing to the overall improvement of educational quality (Schwendimann, 2015). Based on our synthesis of the literature on concept maps in educational contexts, we identified several key advantages that are particularly relevant to our study.

Concept maps effectively facilitate knowledge comprehension and construction by visualizing complex systems and their interrelationships (Khajeloo & Siegel, 2022). They help students organize knowledge structures and encourage deeper understanding through reflection (Chang et al., 2022). As metacognitive tools, concept maps help students regulate their learning processes by assisting with goal setting and study plan adjustments (Fang et al., 2023). They support self-monitoring activities, enabling students to identify knowledge gaps and modify learning strategies accordingly (Eshuis et al., 2022; Ouyang & Xu, 2022). The structured design of concept maps reduces cognitive load when dealing with complex concepts, improving overall learning efficiency (Schneider et al., 2021). Additionally, concept maps help students visualize and structure the information problem-solving process, promoting self-directed learning management (Whitelock-Wainwright et al., 2020).

Particularly relevant to AI-assisted learning environments, concept maps have the potential to function as interfaces between human cognition and AI systems. Research has demonstrated that concept maps support reflection and help students externalize their thought processes (Eshuis et al., 2022). Building on these findings, we propose that when students engage in concept mapping before interacting with AI tools, they may establish clearer learning intentions and develop more precise queries. This potential application of concept maps in AI-assisted contexts warrants empirical investigation. Such preparation may improve the quality of human–AI interaction by helping students formulate more targeted questions and better evaluate AI-generated recommendations against their own knowledge frameworks. However, empirical research examining this proposed benefit remains limited. In the specific context of course selection, we propose that concept maps may offer unique advantages by helping students visualize relationships between academic interests, career goals, and other relevant factors. By mapping these interconnections, students could better articulate their educational priorities and constraints when seeking AI recommendations. This theoretical proposition, which forms the basis of the present study, suggests that such visualization may lead to more personalized and relevant suggestions from GenAI systems.

2.3. Research Gaps

Despite growing interest in applying GenAI to educational contexts, several critical research gaps remain unaddressed. Current GenAI-based course recommendation systems primarily focus on technical implementation rather than addressing students’ real-time needs and metacognitive abilities. Even studies employing LLMs for course recommendations (Khan et al., 2024) primarily evaluate recommendation generation and performance, leaving the content of students’ interactive consultation processes outside their main analytic focus.

Furthermore, while metacognitive strategies have proven effective in education, research specifically examining how tools like concept maps change student–AI interaction patterns in academic counseling is limited. Previous studies have confirmed the effectiveness of metacognitive strategies in reflection and self-regulation (Chang et al., 2022; Fang et al., 2023), with some highlighting the benefits of knowledge visualization (Khajeloo & Siegel, 2022). However, these studies typically examine these approaches in isolation rather than exploring their combined potential with GenAI systems.

A fundamental issue with most AI course recommendation systems is their reliance on historical data, which often leads to brief, limited conversations where students ask simple questions and quickly accept suggestions. This creates a problematic pattern: students engage in minimal interaction with AI, missing opportunities to explore diverse factors or ask follow-up questions. Research has not adequately addressed this issue because studies tend to treat technology (AI), human cognition, and metacognitive tools as separate domains rather than components of an integrated system.

The gaps in current research highlight the need for investigating how concept maps influence student behaviors when using GenAI systems for course recommendations. This study aims to explore how concept mapping tools change the way students interact with GenAI chatbots, examining differences in question patterns, conversation length, and the range of factors discussed during course selection.

3. Materials and Methods

To answer the research questions, we conducted a randomized between-subjects experiment involving 180 undergraduate students at a polytechnic university (University A) in mainland China. Half of the participants used a GenAI chatbot system alone, while the other half used the same GenAI system together with a concept map as a visual tool.

3.1. Research Context

Course selection represents a critical academic decision that students consistently face in higher education. At our partner University A, as at many institutions, students must fulfill prescribed credit requirements to graduate, course grades constitute a fundamental component of their Grade Point Average (GPA), and course choices impact their subsequent employment and graduate school decisions. Students from different academic majors must navigate complex decisions involving personal interests, course difficulty, instructor evaluations, scheduling considerations, alignment with academic and career objectives, and prerequisite requirements at the beginning of each semester. This context provided an ideal setting for examining how concept mapping tools could enhance GenAI-assisted course recommendation processes.

3.2. The Metacognitive Tool: The Concept Map in Course Selection

Course selection decisions are influenced by a combination of factors, including personal factors (such as advice from friends and family, rumors about courses, and non-academic obligations) and institutional factors (such as placement test results, discussions with advisors, and academic programs). These factors not only significantly impact students’ course choices in specific contexts but also have profound effects on their long-term development trajectories (Hinds & Shultz, 2018). Course-enrollment research grounded in the Theory of Reasoned Action (TRA) indicates that students’ attitudes toward a course, subjective norms, and available course information shape enrollment intentions (Amin et al., 2009). Research on student course selection has identified multiple motivational factors that influence academic choices, including career relevance and future benefits (Ma et al., 2021), perceived difficulty and academic performance (Babad & Tayeb, 2003), and institutional and structural factors (Hinds & Shultz, 2018).

The Academic Motivation Scale (AMS) framework (Vallerand et al., 1992) categorizes academic motivation into three broad categories: Intrinsic Motivation (with subtypes to know, toward accomplishment, and to experience stimulation); Extrinsic Motivation (with regulatory subtypes external, introjected, and identified regulation); and Amotivation. Building on this framework, we systematically extended it as follows:

First, we identified that the original AMS framework, while comprehensive for general academic motivation, needed adaptation for the specific context of course selection decisions. The AMS intrinsic motivation category was refined into two distinct dimensions: Academic Development (focusing on knowledge and skill acquisition) and Interest-Driven factors (emphasizing personal curiosity and engagement). This separation recognized that students might pursue courses for intellectual growth independent of personal interest, or vice versa. This separation is theoretically grounded in Self-Determination Theory (Ryan & Deci, 2017), which distinguishes between different qualities of intrinsic motivation, and operationalized following the AMS tripartite distinction among intrinsic motivation to know, toward accomplishment, and to experience stimulation (Vallerand et al., 1992), consistent with course-choice studies showing that students weigh knowledge acquisition, career usefulness, interest, challenge, and social/contextual factors as separable reasons (Lee et al., 2021; Ma et al., 2021). While both dimensions represent internally driven motivations, Academic Development emphasizes competence-oriented goals (mastering knowledge, improving skills), whereas Interest-Driven factors emphasize enjoyment and curiosity-oriented engagement. Second, the AMS extrinsic motivation category was similarly differentiated into Career Development (future professional benefits) and Emotion-Driven factors (immediate psychological rewards such as accomplishment or stress relief). This distinction reflects course-selection evidence that students weigh instrumental benefits such as career preparation alongside evaluative and affective considerations such as attitudes, confidence, perceived difficulty, and anticipated enjoyment (Amin et al., 2009; Babad & Tayeb, 2003; Hinds & Shultz, 2018; Ma et al., 2021). Third, we introduced a new dimension, Habit-Driven factors, which encompasses routine behaviors and unconscious decision patterns not captured in the original AMS framework. This addition was motivated by empirical findings (Ma et al., 2021; Rubineau et al., 2024) showing that academic choices (such as major and course selection) often involve socially influenced behaviors, such as following peer choices or maintaining familiar schedules. Finally, rather than treating amotivation as a separate category, we integrated negative motivational factors (such as “Absence of enthusiasm” or “Absence of requirements”) within each dimension as subcategories, recognizing that students may simultaneously experience positive motivation in some areas while lacking motivation in others.

Our framework does not directly adopt the intrinsic/extrinsic dichotomy from the original AMS. Instead, we reorganized motivational factors into five functionally distinct dimensions based on their role in course selection decisions: Academic Development (goals related to knowledge and skills within the academic system), Career Development (goals related to future professional outcomes), Interest-Driven (factors stemming from personal curiosity and engagement), Emotion-Driven (factors related to immediate psychological states and well-being), and Habit-Driven (factors reflecting routine behaviors and established patterns). This reorganization was motivated by course selection literature (Amin et al., 2009; Hinds & Shultz, 2018; Lee et al., 2021), suggesting that students consider these as distinct decision-making criteria rather than as variations of intrinsic or extrinsic motivation.

We further subdivided each dimension into specific subcategories based on the literature. Each dimension was divided into 3–5 subcategories to capture the nuanced factors within each motivational area, resulting in a comprehensive framework for understanding student course selection motivations. As shown in Figure 1, we represented each dimension through conceptual elements and descriptive keywords, avoiding directive questions to maintain students’ exploratory autonomy. We note that this five-dimension framework is a theoretically grounded adaptation rather than a psychometrically validated instrument. The extension process was guided by three principles: (1) theoretical coherence with the original AMS dimensions, (2) alignment with empirical findings from course selection literature, and (3) practical applicability to the course selection context. The inter-rater reliability achieved during coding ($\kappa =0.73$) provides evidence that the categories are distinguishable and applicable, but it does not establish construct validity. Accordingly, in the present study, the framework functions as an analytic coding scheme for comparing discourse breadth between conditions and not as a measurement instrument for motivational structure. Future research could further validate this framework through factor analysis or expert review processes.

Figure 1. Concept map used to scaffold course-selection considerations.

3.3. Chatbot Design

We developed a GenAI-powered chatbot system (see Figure 2) with backend services implemented in Go 1.21.5 and a frontend built in React 18.2.0. The chatbot’s core conversational capabilities were powered by the DeepSeek-V3 large language model (64k-token context window), released on 26 December 2024, accessed via its API. DeepSeek-V3 was chosen for three practical reasons: the partner institution required student-identifiable interaction data to be processed within mainland China; all student–chatbot exchanges in this study were conducted in Chinese, for which the model offers fluent native-language generation; and the partner institution had existing API access to the model. The system was further coupled with a course knowledge base provided by University A, covering 80 elective courses with content descriptions, instructor information, historical enrollment data, average grades, and other relevant attributes.

Figure 2. Knowledge-enhanced GenAI system used for course advising.

The chatbot was configured with the following technical specifications: the DeepSeek-V3 model with a 64k-token context window, a temperature parameter set to 0.85 to balance response accuracy with conversational naturalness and diversity, a maximum response length of 2048 tokens, and conversation history maintained for up to 30 dialogue turns.

The chatbot was also provided with specific instructions to guide its behavior: Constraints: Must use the provided knowledge base for course-related queries; for non-course questions, use inherent knowledge to respond. Skills: Interpret course information and maintain effective communication abilities. Output format: Provide concise and clear course information for course queries; offer helpful responses for other questions. Workflow: (1) Receive student questions, (2) Determine if questions are course-related, (3) If course-related, query the knowledge base and provide answers, (4) If not course-related, respond using inherent knowledge, (5) Ensure accurate and clear responses, (6) Answer directly without restating questions, (7) Do not reveal model or company information.

3.4. Participants

This study recruited 180 undergraduate students from University A in mainland China using convenience sampling. Inclusion criteria were: enrollment at University A, diverse academic backgrounds, not having yet completed course selection for the upcoming semester, availability to participate, and willingness to use chatbots.

The sample comprised 51 males (28.3%) and 129 females (71.7%), ranging in age from 18 to 22 years (M = 19.87, SD = 1.07). Grade distribution included first-year (n = 57, 31.7%), second-year (n = 87, 48.3%), and third-year students (n = 36, 20.0%), representing 11 academic majors.

Participants were randomly assigned to two groups using computer-generated random number sequences in a 1:1 ratio, with the control group receiving only GenAI ($n=90$) and the experimental group receiving GenAI + Concept Map ($n=90$). The research assistant responsible for randomization did not participate in subsequent data collection and analysis to ensure the concealment of the allocation process. To verify the effectiveness of random assignment, baseline equivalence testing was conducted. Chi-square tests and independent samples t-tests revealed no significant differences between the two groups in gender distribution (${\chi }^2\left(1\right)=1.7510$, $p=0.1857$), grade distribution (${\chi }^2\left(2\right)=5.5245$, $p=0.0631$), or age ($t\left(178\right)=1.2502$, $p=0.2129$), indicating successful random assignment and baseline comparability between the two groups.

3.5. Experiment Procedure

After system testing, 180 students participated in a 7-day home-based experiment. Students were randomly assigned to two groups with different initial instructions:

Only GenAI Group: Students received direct access to the chatbot system and could begin interactions immediately.

GenAI + Concept Map Group: Students first completed a concept-map familiarization process before accessing the chatbot. This process involved reviewing the concept map (Figure 1) for approximately 10 min to understand the key factors that might influence their course selection decisions. Only after completing this familiarization process could students access the chatbot system. The concept map remained available as an optional reference throughout the experiment.

Both groups were instructed to engage with the chatbot at least once daily, with no restrictions on conversation duration, and could access the system from their personal devices at any time and location.

All interactions were conducted in Chinese, and all conversational data were collected and coded using English labels (e.g., A–E for the five main dimensions and A1–E3 for the twenty subcategories) for consistency and ease of statistical analysis during the annotation and coding process.

3.6. Data Collection and Annotation

We collected comprehensive system logs that included all messages exchanged between students and the chatbot, along with timestamps, user identifiers, and conversation metadata. Additionally, we obtained students’ GPA data from University A’s academic management system to control for academic performance differences between groups in subsequent analyses. This data capture approach enabled us to analyze both the content and temporal patterns of student–AI interactions while accounting for baseline academic capabilities.

To understand how students interact with AI during course selection and what factors they prioritize in their decision-making, each student’s message was annotated according to our five-dimensional concept-map framework (Academic Development, Career Development, Interest-Driven, Emotion-Driven, and Habit-Driven). Each student’s message was coded according to the specific subcategories within our concept map (A1–A5, B1–B4, C1–C4, D1–D4, E1–E3). The coding categories were derived from our theoretical framework combining the Academic Motivation Scale (AMS) and empirical course selection literature, as described in Section 3.2.

Table 1 presents representative examples of student messages and their corresponding codes to illustrate our annotation process.

Table 1. Representative Examples of Student Messages and Coding.

Student Message 1	Code	Coding Rationale
“What is the difficulty level of this elective course? Is it easy to pass?”	A3	The student explicitly focuses on course difficulty and passing likelihood, reflecting a grade-oriented motivation within academic development.
“Will this Animation Audio-Visual Language course be very helpful for finding a job in the future? It seems like it would be really useful!”	B2	The student explicitly inquires about the connection between the course and future employment, reflecting a career-preparation orientation within the Career Development dimension.
“Does this course have any practical activities? I really love hands-on practice!”	D2	The student expresses enthusiasm for experiential learning activities, reflecting love of learning within the Emotion-Driven dimension.

¹ Original messages were in Chinese. English translations are provided here for readability.

To ensure coding objectivity, two independent coders analyzed all student messages without knowledge of group assignment (GenAI-only vs. GenAI + Concept Map). Prior to formal coding, both coders underwent training using a subset of messages to establish coding consistency. Inter-rater reliability was assessed using Cohen’s kappa coefficient ($\kappa =0.73$), indicating good agreement between coders. Discrepancies were resolved through discussion and consensus. This coding framework enables us to track which motivational dimensions students emphasize when seeking AI assistance, providing insights into their self-regulatory strategies and interaction patterns.

3.7. Data Analysis

We employed a mixed-methods approach to comprehensively examine the interaction data and compare the two groups. Prior to analysis, we conducted thorough data cleaning and outlier removal procedures to ensure data quality.

3.7.1. Outlier Detection and Removal

Outliers were identified using the interquartile range (IQR) method with a threshold of 1.5. For each behavioral indicator, we calculated the first quartile (Q1) and third quartile (Q3), then defined outliers as values falling below $Q1-1.5\times IQR$ or above $Q3+1.5\times IQR$. This procedure was applied separately to each group to account for potential distributional differences between experimental conditions.

To maintain data integrity while removing extreme values, we implemented a unified outlier removal strategy with the following criteria: (1) Each behavioral indicator was examined independently for outliers. (2) Participants with outliers in more than three indicators were excluded from analysis. (3) Participants with three or fewer outlying indicators had only those specific values treated as missing data. This approach balanced the need to remove genuinely anomalous data points while preserving the maximum number of valid participants. The outlier removal process resulted in the exclusion of 3 participants (all from the experimental group), yielding a final analytical sample of 177 participants (experimental group: $n=87$; control group: $n=90$).

3.7.2. Session Definition

For the purpose of this study, we defined individual chat sessions using the chatbot application’s server-side conversation identifier, which is assigned when the user either (i) initiates a new conversation or (ii) has been idle for more than 30 min within an ongoing conversation. The 30-min idle threshold is consistent with the industry-standard convention used in prior session-segmentation work on user-activity logs (Jansen et al., 2007). To assess robustness, we re-segmented sessions using a pure inter-activity gap criterion at 15, 30, and 60 min and re-ran the ANCOVA on the four session-dependent indicators (Table 2). Three indicators (chat count, maximum consecutive questions, and total chat duration) are qualitatively unchanged across thresholds, as they are either count-based or sum-based and therefore stable under different segmentation rules. Average chat duration, by contrast, is a per-session ratio (total duration divided by session count) and is inherently more sensitive to how activity is split; group means remain direction-consistent (GenAI + Concept Map > Only GenAI) at all three thresholds, with the difference largest at finer segmentation. The eight non-session-dependent indicators do not depend on the segmentation rule and are reported later in the ANCOVA results table.

Table 2. Sensitivity analysis for the four session-dependent indicators under a pure inter-activity gap criterion at 15, 30, and 60 min. Means are reported as GenAI + Concept Map/Only GenAI. ANCOVA controls for GPA, $df=(1,174)$.

	15 min		30 min		60 min
Indicator	Means	$\mathit{F}$ ( $\mathit{p}$ )	Means	$\mathit{F}$ ( $\mathit{p}$ )	Means	$\mathit{F}$ ( $\mathit{p}$ )
Chat count	5.93/6.23	0.55 (0.459)	5.72/5.93	0.28 (0.595)	5.66/5.73	0.04 (0.837)
Max consecutive questions	13.95/10.75	17.06 ***	14.03/11.17	13.66 ***	14.07/11.30	12.87 ***
Avg chat duration (min)	8.82/6.56	12.89 ***	9.97/8.53	2.26 (0.135)	10.53/9.84	0.49 (0.483)
Total chat duration (min)	43.62/38.91	2.21 (0.139)	48.11/44.63	0.86 (0.354)	50.81/53.19	0.28 (0.599)

Note. *** $p<$ 0.001.

3.7.3. Behavioral Indicators

We evaluated students’ interactions using key behavioral indicators summarized in Table 3. These metrics captured multiple dimensions of student interaction behaviors, including frequency indicators (chat count, question count), elaboration-volume indicators (total user words, total assistant words), per-unit elaboration indicators (max consecutive questions, avg words per question), temporal patterns (questions per minute, avg chat duration, total chat duration), system responsiveness (avg response time), and coverage metrics assessing the breadth of factors considered. Upper coverage measured how many main dimensions (A–E) students addressed, while detailed coverage captured the number of specific subcategories (A1–E3) discussed.

Table 3. Behavioral Indicators and Their Descriptions.

Indicator	Description	Measurement Reference
Chat count	Number of separate chat sessions initiated by students	number of sessions Kassab et al. (2023)
Question count	Total quantity of questions asked across all interactions	number of questions Kassab et al. (2023)
Max consecutive questions	Longest chain of sequential follow-up questions within a single chat session	follow-up questions Tix (2025)
Questions per minute	Questioning frequency during interactions	interaction frequency Srivastava (2025)
Total user words	Cumulative word count of all student messages	number of words Chin et al. (2023)
Avg words per question	Average level of detail and specificity per question	number of words Chin et al. (2023)
Total assistant words	Cumulative word count of all AI responses	response length Abd-Alrazaq et al. (2020)
Avg chat duration (min)	Average time invested in each chat session	interaction time Martins et al. (2024)
Total chat duration (min)	Cumulative time spent across all chat sessions	interaction time Martins et al. (2024)
Avg response time	Average time for AI to generate responses	system responsiveness Wang and Lo (2025)
Upper coverage	Number of main motivational dimensions (A–E) addressed	AMS motivation dimensions Vallerand et al. (1992)
Detailed coverage	Number of subcategories (A1–E3) covered in conversations	AMS motivation subcategories Vallerand et al. (1992)

3.7.4. Statistical Analysis

To compare differences in these indicators between the control group (GenAI only) and the experimental group (GenAI + Concept Map), we employed analysis of covariance (ANCOVA) with students’ GPA as a covariate to control for baseline academic performance differences. GPA was selected because prior academic achievement may influence students’ interaction patterns and questioning behaviors with AI systems, and we examined whether this covariate effect was significant across all dependent variables. Group means in Section 4 are reported as estimated marginal means (EMMs) from the fitted ANCOVA model, evaluated at the grand-mean GPA, with their standard errors.

Effect sizes were reported as partial eta squared (${\eta }_p^2$), interpreted following conventional thresholds of 0.01, 0.06, and 0.14 for small, medium, and large effects, respectively (Cohen, 1988). All statistical analyses were conducted using R version 4.3.1, with significance set at $\alpha$ = 0.05.

3.7.5. Epistemic Network Analysis

To complement the quantitative interaction measures, we employed Epistemic Network Analysis (ENA) to examine the underlying conceptual relationships in students’ decision-making processes. For network modeling, we established analytical units corresponding to individual student chat sessions with the chatbot system. Motivational factors co-occurring within single sessions were treated as conceptually associated, enabling us to map cognitive connection patterns across different conditions. The ENA was conducted using the ENA Web Tool (https://www.epistemicnetwork.org/) to visualize and quantify the connections between different motivational factors in students’ discourse.

4. Results

4.1. Group Differences in Behavioral Indicators

ANCOVA was conducted to examine group differences in behavioral indicators while controlling for GPA. The analysis revealed a clear pattern: basic engagement metrics remained comparable between groups, whereas interaction quality measures showed substantial and consistent differences. Figure 3 displays only the statistically significant results, while complete GPA-adjusted estimated marginal means (EMMs), their standard errors, F statistics, p-values, and partial ${\eta }_p^2$ effect sizes for all 12 indicators are reported in Table 4.

Figure 3. ANCOVA results for the behavioral indicators where the group effect reached statistical significance, controlling for GPA. Group M = GenAI + Concept Map ($n=87$); Group NoM = Only GenAI ($n=90$). Box edges denote the 25th and 75th percentiles, the central line denotes the median, and whiskers extend to 1.5 × IQR; black/gray circles indicate participant-level raw values. The red diamond (⧫) marks the estimated marginal mean (EMM) of each group, evaluated at the grand-mean GPA, and the red vertical bar represents the 95% confidence interval for the EMM. Reported F, p, and ${\eta }_p^2$ are GPA-adjusted ($df=1,174$). Non-significant indicators are omitted from the figure.

Table 4. Complete ANCOVA results for all 12 behavioral indicators, with GPA as a covariate. Reported values are estimated marginal means (EMMs) and their standard errors (SE) from the ANCOVA model, adjusted for GPA and evaluated at the grand-mean GPA, for the GenAI + Concept Map ($n=87$) and Only GenAI ($n=90$) conditions. The Type I F-test for the group effect is GPA-adjusted. All F tests have $d{f}_{\mathrm{effect}}=1$ and $d{f}_{\mathrm{error}}=174$.

Indicator	EMM (SE)		F	p	${\mathit{\eta }}_{\mathit{p}}^2$
	GenAI + Concept Map	Only GenAI
Chat count	6.90 (0.41)	6.97 (0.40)	0.13	0.724	0.001
Question count	69.99 (3.27)	66.81 (3.22)	3.21	0.075	0.018
Max consecutive questions	11.92 (0.59)	9.07 (0.58)	15.42	<0.001 ***	0.081
Questions per minute	2.51 (0.14)	2.75 (0.14)	2.45	0.119	0.014
Total user words	949.96 (40.73)	804.80 (40.04)	17.27	<0.001 ***	0.090
Avg words per question	15.27 (0.62)	11.93 (0.61)	17.25	<0.001 ***	0.090
Total assistant words	10,218.24 (513.90)	8845.67 (505.11)	5.89	0.016 *	0.033
Avg chat duration (min)	8.05 (0.41)	6.77 (0.40)	5.27	0.023 *	0.029
Total chat duration (min)	43.34 (2.28)	42.01 (2.24)	1.97	0.162	0.011
Avg response time (s)	0.72 (0.05)	0.78 (0.05)	1.41	0.236	0.008
Upper coverage	4.46 (0.11)	3.66 (0.11)	35.58	<0.001 ***	0.170
Detailed coverage	8.70 (0.34)	6.36 (0.33)	33.73	<0.001 ***	0.162

Note. Means and standard errors reported in this table are GPA-adjusted estimated marginal means (EMMs) from the ANCOVA model, evaluated at the grand-mean GPA. * $p<$ 0.05; *** $p<$ 0.001.

Both groups displayed comparable overall levels of basic engagement. No significant group differences emerged in chat count, question count, questions per minute, total chat duration, or average response time (all $p>$ 0.05, ${\eta }_p^2\le$ 0.018). The lack of significant differences in basic engagement metrics indicates that any observed group differences reflect qualitative distinctions in interaction patterns rather than mere differences in frequency.

By contrast, substantial differences emerged in indicators of interaction depth and elaboration. The means reported in this section are GPA-adjusted estimated marginal means (EMMs) from the ANCOVA model (Table 4). Students using concept maps exhibited more sustained inquiry, showing a 31% longer maximum consecutive-question chain within a session ($M=11.92$ vs. $9.07$ questions; $F(1,174)=15.42$, $p<$ 0.001, ${\eta }_p^2$ = 0.081) and formulating questions that were on average 28% longer ($M=15.27$ vs. $11.93$ words; $F(1,174)=17.25$, $p<$ 0.001, ${\eta }_p^2$ = 0.090). These students also maintained 19% longer average chat sessions ($M=8.05$ vs. $6.77$ min; $F(1,174)=5.27$, p = 0.023, ${\eta }_p^2$ = 0.029), generated 18% more total user words ($M=949.96$ vs. $804.80$; $F(1,174)=17.27$, $p<$ 0.001, ${\eta }_p^2$ = 0.090) and received 16% more total assistant words ($M=10,218.24$ vs. 8845.67; $F(1,174)=5.89$, p = 0.016, ${\eta }_p^2$ = 0.033). The largest group differences appeared in content coverage indicators. Students in the concept-map group addressed significantly more upper-level motivational dimensions ($M=4.46$ vs. $3.66$; $F(1,174)=35.58$, $p<$ 0.001, ${\eta }_p^2$ = 0.170) and detailed subcategories ($M=8.70$ vs. $6.36$; $F(1,174)=33.73$, $p<$ 0.001, ${\eta }_p^2$ = 0.162).

Inspection of GPA as a covariate also revealed differential predictive patterns across outcome types. GPA significantly predicted several aggregate-volume metrics, including question count (${\eta }_p^2$ = 0.158, $p<$ 0.001), total user words (${\eta }_p^2$ = 0.284, $p<$ 0.001), and total chat duration (${\eta }_p^2$ = 0.132, $p<$ 0.001), but did not predict selected per-unit indicators such as average words per question (${\eta }_p^2$ = 0.012, $p=$ 0.148) or average chat duration (${\eta }_p^2$ = 0.001, p = 0.736).

4.2. ENA Analysis

The study employed Epistemic Network Analysis (ENA) to visually model and compare the decision-making patterns of course consultations between the two groups. The primary distinction emerges through decision boundary expansion rather than simple connectivity changes, with the concept-map group showing wider motivational networks and stronger cross-dimensional integration in their discourse.

The analysis (Figure 4) reveals motivational factors that were rarely activated in the control group, specifically C2 (External validation), C4 (Absence of enthusiasm) and D4 (Absence of requirements)—factors that were barely present in the control group’s decision-making. Figure 4 (right panel) presents the difference network, where blue connections indicate stronger associations in the GenAI + Concept Map group, while red connections represent stronger associations in the control group. The difference network reveals that A1 (Acquire fresh insights) serves as a central hub with multiple strong connections, a pattern suggesting that concept-map scaffolding may facilitate a reorganization of students’ decision-making around knowledge acquisition as a core connecting principle. Red connections are sparse, suggesting that the control group maintains simpler, more direct motivational pathways.

Figure 4. Epistemic Network Analysis (ENA) results. Nodes correspond to the 20 motivational subcategories (A1–E3) defined in Section 3.2; node position is determined by ENA’s singular-value-decomposition projection of co-occurrence space and is held constant across the three panels. Node size is proportional to the frequency with which each code was activated, and edge thickness is proportional to the strength of within-session co-occurrence between the two connected codes. Left: network for the control group (Only GenAI). Center: network for the experimental group (GenAI + Concept Map). Right: difference network, where blue edges denote connections that are stronger in the GenAI + Concept Map group and red edges denote connections that are stronger in the control group; line thickness encodes the magnitude of the between-group difference.

The most prominent differences occur in both intra-dimensional and cross-dimensional connections. The A1∼A3 (knowledge-performance integration) connection, linking two subcategories within the Academic Development dimension, demonstrates significantly stronger co-occurrences in the GenAI + Concept Map group, consistent with students simultaneously pursuing academic knowledge and grade improvement. Meanwhile, the A4∼B2 (academic-career linkage) connection shows enhanced cross-dimensional association between academic requirements and career development, suggesting that concept-map use may have helped students recognize the importance of completing studies for career preparation. These patterns were weaker in the control group, consistent with a more fragmented pattern of decision-related discourse.

The prominence of absence-related factors warrants particular attention. D4 (Absence of requirements) shows extensive connections in the GenAI + Concept Map group while being almost absent in the control group, which may reflect greater attention to autonomous course-selection considerations. E3 (Absence of strategy) plays a more prominent bridging role in the GenAI + Concept Map group, with substantially stronger associations to D1 (Feeling of accomplishment) and B2 (Professional readiness) compared to the control group, where these connections were present but considerably weaker. Rather than indicating deficiencies, these connections are consistent with a compensatory pattern where students acknowledge strategic limitations while seeking emotionally satisfying outcomes, suggesting a more reflective approach to decision-making.

Students in the GenAI + Concept Map group more frequently engaged with factors related to external validation (C2), a pattern consistent with a broader engagement that includes the external and social-recognition aspects of knowledge in addition to its academic value. The predominance of blue over red connections suggests that concept maps may primarily strengthen existing motivational associations rather than creating entirely new pathways, with the intervention plausibly operating by making implicit connections explicit.

5. Discussion

Before discussing the specific findings, we first distinguish between course recommendation and course selection in this study. While GenAI systems provide course recommendations, students ultimately make course selection decisions. Our research focuses on the patterns of human–AI interactions during this recommendation-seeking process, examining how metacognitive scaffolding influences students’ engagement with AI-generated recommendations rather than the recommendation algorithms themselves. This distinction is critical because effective course selection depends not only on the quality of AI recommendations but also on students’ ability to articulate their needs and critically evaluate the suggestions they receive. The following discussion addresses our three research questions by examining how concept maps change interaction patterns (RQ1), expand the range of factors discussed (RQ2), and alter how students connect different reasons in conversation (RQ3).

5.1. Interaction Patterns Under Concept-Map Scaffolding

Regarding RQ1, the results (Figure 3) showed that concept-map use was associated with substantially different interaction-quality indicators, while basic engagement metrics remained comparable. Despite no significant differences in basic engagement metrics such as chat count (p = 0.724), question count (p = 0.075), questions per minute (p = 0.119), total chat duration (p = 0.162), or average response time (p = 0.236), the concept-map group showed a 31% longer maximum consecutive-question chain within a session (GPA-adjusted $M=11.92$ vs. $9.07$ questions; ${\eta }_p^2$ = 0.081), maintained 19% longer average chat durations ($M=8.05$ vs. $6.77$ min; ${\eta }_p^2$ = 0.029), formulated 28% more detailed questions ($M=15.27$ vs. $11.93$ words per question; ${\eta }_p^2$ = 0.090), generated 18% more total user words ($M=949.96$ vs. $804.80$; ${\eta }_p^2$ = 0.090), and received 16% more assistant words ($M=10,218.24$ vs. $8845.67$; ${\eta }_p^2$ = 0.033).

These observed behavioral changes address fundamental limitations of traditional recommendation systems, which rely predominantly on algorithms while neglecting students’ cognitive needs during the consultation process. The shift from breadth-focused to depth-focused engagement suggests that concept maps do not simply increase information processing volume but rather facilitate a qualitatively different mode of interaction characterized by sustained inquiry and elaboration. Specifically, concept maps significantly influenced behavioral indicators associated with deeper cognitive engagement, including consecutive question sequences and question elaboration patterns. These differences are consistent with a shift in engagement style rather than a mere quantitative increase in activity.

These shifts move beyond the simple question-and-answer interaction modes that characterize most current GenAI systems. Rather than each query appearing as an isolated transaction, students in the concept-map group tended to build upon previous exchanges, asking longer follow-up questions and articulating more elaborate considerations. This pattern of sustained, cumulative inquiry aligns with cognitive load theory (Risko & Gilbert, 2016; Sweller et al., 2019), which suggests that external representations free up working memory resources for higher-order thinking processes. By offloading the organizational burden of tracking multiple decision factors onto the visual scaffold, students could dedicate more cognitive capacity to reflecting on the relationships between factors and formulating more elaborate questions.

Concept maps did not just push students to process more information; they appeared to surface decision-making considerations that students already held but rarely applied in a systematic way. This pattern offers a new perspective on the findings of Kuhlthau (1991) regarding students’ inherent difficulties in articulating information needs clearly. Our results suggest that these articulation challenges may stem not from fundamental cognitive deficiencies but rather from the absence of adequate cognitive tools to structure and express complex, multidimensional considerations. When provided with appropriate metacognitive scaffolding, students demonstrated substantial capacity for detailed articulation and sustained inquiry, challenging deficit-oriented interpretations of student consultation behaviors.

5.2. Scope Expansion of Decision Considerations

Regarding RQ2, the large effect sizes observed in motivational factor coverage indicate that concept-map users discussed a substantially wider variety of considerations. The concept-map group covered 22% more upper-level motivational dimensions (GPA-adjusted $M=4.46$ vs. $3.66$; ${\eta }_p^2$ = 0.170) and 37% more detailed factors ($M=8.70$ vs. $6.36$; ${\eta }_p^2$ = 0.162). These represent the largest effects observed in the entire study, indicating that the primary impact of concept-map scaffolding lies in broadening the scope of decision consideration rather than merely deepening engagement with a fixed set of factors.

Theoretically, this finding suggests that when students systematically evaluate multiple motivational dimensions spanning knowledge acquisition, external outcomes, intrinsic interests, emotional responses, and absence considerations, this represents a qualitative shift from fragmented to comprehensive decision-making. The observed pattern is consistent with course-choice and enrollment evidence showing that students’ decisions are shaped by attitudes, social expectations, information availability, career usefulness, interests, and contextual constraints (Amin et al., 2009; Lee et al., 2021; Ma et al., 2021). Our results suggest that when provided with appropriate cognitive tools, students may engage in more systematic multi-aspect evaluation rather than defaulting to narrow, convenience-driven choices.

Furthermore, the expanded coverage supports the practical utility of our extension of the Academic Motivation Scale framework (Vallerand et al., 1992) as an analytic scheme for course selection. The finding that students meaningfully engaged with all five major motivational categories when scaffolded by concept maps suggests that this dimensional structure can capture a broad range of considerations relevant to authentic student decision-making. This stands in contrast to approaches that reduce course selection to utility maximization or simple preference matching, revealing a richer landscape of considerations that students can access when appropriately supported.

The expansion of factor coverage also addresses practical limitations identified in current recommendation systems, including adaptability and data constraints (Algarni & Sheldon, 2023; Nafea et al., 2019) and the need for more explainable recommendation rationales (Yang et al., 2024). By providing a visible cognitive scaffold, concept maps may make AI consultation more transparent and collaborative, with students able to see which factors they have considered and which remain unexplored. This transparency enables more informed decision-making and helps students develop stronger ownership of their choices.

5.3. Cross-Dimensional Integration Patterns

Regarding RQ3, ENA analysis (Figure 4) shows that concept-map users displayed substantially different patterns in how they connected various motivational factors within their discourse. The analysis illuminates not merely which factors students discussed but how they linked these factors together in their thinking, providing insight into the relational structure of their decision-related discourse.

When concept maps were not used, students often treated related factors in isolation from one another. For instance, A1 (Acquire fresh insights) and A3 (Boost academic performance) appeared as separate, unlinked considerations in the control group, as did A4 (Meet academic obligations) and B2 (Professional readiness). This pattern of weak cross-factor co-occurrence may reflect, in part, educational practices in which subject knowledge is transmitted in isolation and explicit instruction in connecting academic learning to broader life goals is uncommon. As a result, students may have fewer ready-made schemas for linking different motivational domains during course selection, leading to discourse that addresses factors sequentially rather than in an integrated way.

The use of concept maps was associated with substantially denser connections in students’ discourse, both within and across motivational dimensions. The difference network shows that A1 (Acquire fresh insights) serves as a central hub with multiple strong connections, a pattern suggesting that concept maps may help students reorganize their decision-making around knowledge acquisition as a core connecting principle. Specifically, the strengthened A1∼A3 connection, an intra-dimensional link within Academic Development, is consistent with students drawing tighter associations between knowledge acquisition and academic performance, treating them as mutually reinforcing rather than competing objectives. Meanwhile, the enhanced A4∼B2 cross-dimensional linkage further suggests that concept maps may have helped students understand academic obligation fulfillment as foundational to professional preparation rather than as merely bureaucratic requirements to satisfy.

These observed integration patterns are consistent with the foundational theory of Novak (1990) that concept maps facilitate deeper understanding through hierarchical relationship visualization. More importantly, our findings extend the Academic Motivation Scale framework (Vallerand et al., 1992) by suggesting how metacognitive tools may activate dormant motivational connections in authentic decision-making contexts. While the AMS identifies distinct motivational dimensions, our ENA results suggest that these dimensions may co-occur and interact in students’ decision-related discourse. Moreover, when appropriately scaffolded, students may articulate connections between motivational factors more readily, treating them as related rather than isolated considerations.

The observed differences in cross-dimensional integration align with established cognitive principles regarding external representations. Research on cognitive load, cognitive offloading, and decision-making with visualizations indicates that external representations can reduce internal processing demands and support more deliberate reasoning (Padilla et al., 2018; Risko & Gilbert, 2016; Sweller et al., 2019). In our context, the concept map could serve as an external memory aid for tracking which factor combinations students had explored while simultaneously considering new connections. Such cognitive offloading is theorized to support a slower, more contemplative decision-making process essential for integrating complex, multidimensional information (Padilla et al., 2018), contrasting sharply with the rapid, heuristic-driven choices often observed in unsupported decision contexts.

Additionally, the concept-map group more frequently activated absence-related factors that were nearly dormant in the control group, particularly C4 (Absence of enthusiasm), D4 (Absence of requirements), and their connections to positive factors. The prominence of D4 (Absence of requirements) in the concept-map group may reflect greater attention to autonomous course-selection considerations, with students weighing intrinsic value alongside external mandates rather than treating the latter as the sole basis of choice. Similarly, E3 (Absence of strategy) played a crucial bridging role in the concept-map group, connecting to D1 (Feeling of accomplishment) and B2 (Professional readiness). Rather than indicating deficiencies, these absence-factor connections are consistent with a more reflective compensatory pattern of thinking in which students acknowledge strategic limitations while seeking emotionally satisfying and developmentally valuable outcomes.

This finding aligns with research recognizing that motivation involves not only positive drivers but also factors related to the absence of motivation (Vallerand et al., 1992) and the navigation of avoidance-related concerns (Elliot & Church, 1997). By making these typically implicit considerations explicit and structurally integrated, concept maps may support the development of more sophisticated self-regulation and internalized motivation. The activation of absence factors can be interpreted as consistent with a more reflective engagement style rather than as negative affect, suggesting attention to nuances that acknowledge both opportunities and limitations.

5.4. GPA-Adjusted Patterns and Equity Implications

The relationship between GPA and the interaction metrics was not uniform across outcome types. GPA significantly predicted several aggregate-volume measures, including question count (${\eta }_p^2$ = 0.158, $p<$ 0.001), total user words (${\eta }_p^2$ = 0.284, $p<$ 0.001), and total chat duration (${\eta }_p^2$ = 0.132, $p<$ 0.001), but did not significantly predict selected per-unit indicators such as average words per question (${\eta }_p^2$ = 0.012, $p=$ 0.148) or average chat duration (${\eta }_p^2$ = 0.001, p = 0.736).

This divergence suggests two distinct patterns. The finding that GPA predicts engagement quantity aligns with extensive meta-analytic evidence showing significant associations between academic performance and self-regulated learning behaviors (Richardson et al., 2012). Higher-achieving students typically demonstrate greater overall engagement across educational contexts, reflecting accumulated advantages in motivation, skill, and academic identity. However, the absence of a significant GPA main effect on the per-unit indicators indicates that, in the present ANCOVA, per-unit interaction quality is not strongly tied to prior academic achievement, even though aggregate volume of engagement is.

Because GPA did not significantly predict per-unit interaction quality in this sample, there is no observable GPA-based gap on these indicators that concept-map use could widen. From an equity perspective, this matters because GenAI tools deployed without scaffolding may primarily benefit students who already possess strong metacognitive and self-regulatory skills, potentially exacerbating existing achievement gaps in domains where such gaps do exist. The present pattern suggests that a shared external scaffold, such as a concept map, may support deeper engagement in AI-supported learning and advising contexts (Padilla et al., 2018; Wan & Gu, 2026).

The theoretical implication is that interaction quality may be shaped more by the availability of appropriate cognitive tools than by inherent academic ability. This challenges deficit-oriented frameworks that attribute shallow engagement patterns to student limitations rather than to inadequate support structures (McKay & Devlin, 2016; Weuffen et al., 2018). Our results suggest that many students can engage in deeper, more systematic interaction when provided with appropriate scaffolding, even if such engagement is less evident in unsupported settings. This perspective aligns with sociocultural theories of learning that emphasize how cognitive tools mediate and potentially transform human capabilities rather than merely revealing fixed abilities.

5.5. Broader Implications for GenAI in Education

By integrating concept maps with GenAI systems, this study directly addresses the identified deficiencies in current recommendation systems, particularly the disconnect between students’ evolving cognitive needs and system responses and the lack of metacognitive support for structured thinking and reflection.

Our findings support previous research demonstrating that concept maps help students identify and organize relationships and interdependencies in complex systems, thereby improving their self-regulation skills in technology-enhanced learning environments (Chang et al., 2022). This pattern also aligns with the view that concept maps facilitate knowledge comprehension by visualizing complex relationships and system structures (Khajeloo & Siegel, 2022). This approach contrasts sharply with studies focusing primarily on algorithmic improvements without considering metacognitive scaffolding (Algarni & Sheldon, 2023; Nafea et al., 2019), and supports arguments that GenAI integration requires comprehensive solutions beyond mere technical refinement to meaningfully address educational challenges (Yan et al., 2024).

The improved interaction quality observed here is consistent with the view of Brown et al. (1983) that metacognitive strategies serve as tools for monitoring and regulating one’s own cognitive processes. By making decision-making structures visible and manipulable, concept maps may help students become more aware of their thinking patterns and more deliberate in their exploration strategies. This aligns with contemporary reconceptualizations of AI literacy (Sidra & Mason, 2024) that emphasize not just understanding AI capabilities but also developing competencies for effective human–AI collaboration. Our results indicate that metacognitive scaffolding represents an important component of this collaborative competency, supporting students in engaging more deliberately with AI while critically evaluating its outputs.

With concept-map scaffolding, GenAI may shift from a passive question-answering tool toward a more active collaborative partner that supports comprehensive thinking. Rather than students simply submitting isolated queries, the pre-interaction concept map can encourage broader and more systematic exploration while leaving students free to decide which factors to pursue more deeply. This design philosophy offers valuable insights for cognitive tool development in intelligent educational environments, suggesting that effective AI systems should prioritize supporting human metacognition rather than attempting to replace human decision-making with algorithmic optimization.

6. Conclusions

This study examined how concept-map scaffolding shapes student engagement with GenAI advising, and three findings stand out. In this sample, concept-map use was associated with changes in the quality of student–AI interaction without changing basic engagement metrics: students engaged in longer follow-up chains and more elaborated questions, while the overall number of sessions and queries, the questioning rate, and the cumulative chat time remained comparable to those of the control group. The same students also brought a substantially wider range of motivational considerations into the conversation, spanning more high-level dimensions as well as finer-grained factors within those dimensions. The most striking pattern, however, came from Epistemic Network Analysis: these considerations were not merely more numerous but more integrated. Mastery and performance goals, academic and career orientations, and absence-related factors—many of which rarely surfaced in the control group—became interconnected within a unified discourse, a pattern consistent with students engaging with motivation in a more integrated way rather than as a checklist of isolated concerns. Crucially, all statistically significant group differences reported above remained significant after adjusting for GPA in the ANCOVA, a pattern suggesting that AI-assisted course advising may support broader access to interaction-quality benefits across levels of prior academic achievement, rather than amplifying existing gaps.

These findings carry concrete implications for advising-system design. Visual decision frameworks deployed as pre-consultation modules, dashboards that surface which factors a student has and has not yet considered, and conversational flows that prompt evaluation rather than passive acceptance could together help move GenAI advising from a Q&A interface toward a more metacognitively oriented collaborator.

This study has several limitations. First, the experimental condition included approximately 10 min of concept-map familiarization without a matched preparatory activity in the control, leaving time-on-task confounds and Hawthorne effects (McCambridge et al., 2014) as plausible alternative explanations. The Hawthorne concern is asymmetric here: only the experimental group received a visible pre-task scaffold, which by itself may have signaled additional researcher attention and motivated more deliberate engagement independent of the map’s metacognitive content. An active-control replication that equates both pre-interaction preparation time and the salience of researcher attention across conditions, such as an unrelated reflective task of equivalent length, is therefore required to isolate the scaffold’s metacognitive content from generic cognitive priming and observation-related effects. Second, we did not log within-session re-consultation of the concept map, and the unsupervised home setting prevented control over external-resource use, peer discussion, or concurrent advising, so the precise mechanism through which the scaffold operated cannot be directly verified. Third, the five-dimension AMS extension achieved acceptable inter-rater reliability ($\kappa =0.73$), but its construct validity has yet to be established through factor analysis or expert review; the instrument is sufficient for the present within-study comparisons but should not be treated as validated. Fourth, participants were recruited from a single polytechnic university in mainland China with a female-skewed distribution (71.7% female, 28.3% male); because the Chinese higher-education context—fixed-major enrollment, GPA-weighted progression, and centralized academic authority—may amplify the perceived value of an explicit decision scaffold, the present effect sizes require replication in Western liberal-arts, community-college, online, and gender-balanced populations before broader generalization. Fifth, our findings rely on a single system (DeepSeek-V3); we expect the human-side mechanism to be model-agnostic in direction, but discourse-level magnitudes are likely model-dependent, and cross-model replication on a Western-trained frontier LLM (GPT-4, Claude, or Gemini) is therefore needed before specific effect sizes are extrapolated. Finally, the 7-day window captures only immediate effects and yields no information on downstream decision quality, such as course satisfaction, goal congruence, or grades earned.

Future work should address these limitations along two complementary tracks. Methodologically, the most pressing priority is an active-control replication that equates preparation time and researcher-attention salience across conditions, given the time-on-task and Hawthorne concerns discussed above; in addition, subsequent studies should incorporate explicit manipulation checks (e.g., panel-access logs, eye-tracking, or post-session self-reports) and formal validation of the AMS extension. Substantively, longitudinal follow-up linking interaction patterns to enrollment, retention, and grades one and two semesters downstream, comparative studies of alternative scaffolding types, and extension to other high-stakes decision contexts such as career planning or research-topic selection would each meaningfully extend the present findings.