Generative AI-Supported Personalized Learning Resources: Efficacy, Reliability, and Mechanisms of Student Preference

Abstract

This study examines a context engineering workflow for the generation, evaluation, and integration of large language model-generated personalized preview learning materials. Under an LLM-based rubric evaluation, the workflow achieved significantly higher scores than single-prompt, dual-agent, and deep-research approaches across the evaluated preview materials. The stability analysis showed relatively stable repeated LLM ratings, while paired-samples t-tests indicated no statistically significant mean differences between LLM and expert ratings for either the basic or advanced version. Mediation analyses further identified distinct associations between motivation, engagement, and students’ preview preferences: extrinsic motivation showed a direct positive association with preference for advanced materials, whereas intrinsic motivation showed significant indirect associations with preferences for both basic and advanced materials through emotional and cognitive engagement. No significant engagement-mediated pathways were observed for extrinsic motivation. These findings suggest that generative AI workflows based on context engineering may provide a promising approach for developing differentiated preview materials and supporting learner diversity in voluntary pre-class learning.

1. Introduction

The deep integration of digital technologies and artificial intelligence is reshaping instructional models and learning practices in higher education [1]. As smart courses and digital learning platforms become increasingly prevalent, instructors are confronted with the tension between learner diversity and fixed curricular structures [2]. This challenge is especially pronounced in first-year undergraduate education, where students show different motivations and learning skills [3]. Research shows that first-year students in these disciplines often struggle to keep pace with instruction, develop abstract conceptual understanding, and maintain sufficient engagement [4].

Prior research indicates these learning difficulties arise from the interaction of cognitive and affective–motivational factors. Cognitively, variations in prior knowledge and reasoning ability hinder integration of new information under uniform pacing, increasing cognitive load [5]. The transition to university also leads to declines in motivation, self-efficacy, and engagement [6,7]. When such difficulties persist without adequate instructional support, students are more likely to disengage or withdraw [8].

In response, personalized learning has emerged as a promising approach to improve engagement and outcomes in higher education [9]. However, fully individualized classroom instruction remains impractical due to fixed curricula. Instead, personalized learning is more feasibly achieved through differentiated learning resources [10,11]. Preview learning activities allow students to engage with core content at their own pace [12], yet traditional resources are often uniform in difficulty, limiting accommodation of diverse academic foundations and motivational profiles [13]. This one-size-fits-all approach can overburden students with learning difficulties while under-challenging advanced learners.

Recent advances in generative artificial intelligence (GAI), particularly large language models (LLMs), enable the production of instructional texts, examples, and cases on a large scale [14]. However, current educational applications of GAI often rely on single-prompt or simple workflow approaches, which may result in unstable structures and inconsistent pedagogical quality [15].

In formal education, learning resources must be accurate, curriculum-aligned, and cognitively appropriate [16]. Without explicit constraints and systematic quality control, LMM-generated content risks increasing cognitive load, introducing misconceptions, or reducing instructional effectiveness [17]. These risks are particularly pronounced in preview scenarios, where students often engage with resources independently and without immediate guidance. Existing studies have primarily demonstrated LLMs’ generative capabilities, with less focus on systematic quality control during generation [18,19].

The integration of GAI into preview materials production expands student choices by offering personalized resources varying in difficulty, structure, and cognitive demand. This shifts learners from passive recipients to active decision-makers, increasing their interest and engagement in preview sections.

From a learner-centered perspective, effectively generating and evaluating LLM-supported preview resources, as well as understanding the key factors influencing students’ preferences for personalized options, is essential for realizing GAI’s broader educational value [20]. Although motivation is a well-established driver of learning behaviors and engagement, its multidimensional construct—including intrinsic and extrinsic pathways, mediation through engagement, and differential influences on preferences—remains largely unknown in LLM-supported preview contexts [21].

Accordingly, this study adopts a mechanism-oriented approach to investigate the qualities of LLM-supported preview resources, as well as the factors influencing students’ preferences for personalized preview options. The study addresses the following research questions:

RQ1: How can high-quality preview learning content be generated reliably using multi-agent LLM systems?

RQ2: What factors are associated with students’ preferences for personalized preview options?

2. Literature Review

2.1. Personalized Learning and Learner Diversity in Higher Education

The U.S. Department of Education [22] defines personalized learning as instruction in which the pace of learning and the instructional approaches are optimized to address individual learner needs. In contrast to uniform models, personalized learning emphasizes responsiveness to differences in prior knowledge, cognitive abilities, and motivational profiles [2]. Although early implementations focused on primary and secondary education, growing diversity in higher education has intensified interest in university-level applications.

Learner diversity in higher education is widely recognized as a multidisciplinary phenomenon. University students differ substantially in prior academic skills, learning strategies, self-regulation skills, and motivational orientations [23]. These differences are especially evident among first-year undergraduates transitioning from varied backgrounds. Prior knowledge strongly predicts learning outcomes by shaping information processing and conceptual understanding [24], while other factors, such as self-efficacy, interest, and engagement, influence learning behaviors [25].

Despite its strong appeal, implementing personalized learning in higher education is constrained by fixed curricula, predefined sequences, and cohort-based teaching, rendering fully individualized classroom instruction impractical [26,27]. Consequently, personalized learning is better viewed as selective and strategic, focusing on targeted adaptations rather than comprehensive individualization of all instructional processes [28].

Within these constraints, research conceptualizes personalized learning in higher education as being primarily realized through differentiated learning resources rather than restructuring of classroom teaching [29]. Resources varying in difficulty, content representation, and enrichment activities can accommodate diversity while maintaining coherence, offering scaffolding for weaker students and challenges for advanced learners [30,31]. This approach aligns with constructivism principles emphasizing active engagement and developmental scaffolding [32].

Differentiated learning resources are particularly salient in preview contexts, where students are expected to independently engage with instructional resources prior to classroom instruction. Empirical studies suggest that well-designed personalized resources can positively influence student engagement and learning outcomes by aligning instructional content with learners’ backgrounds and needs [33]. However, these benefits depend heavily on the quality and coherence of the personalized resources, as poorly designed adaptations may increase cognitive load or create confusion rather than support learning [34].

While differentiated resources offer a viable path for addressing learner diversity, their effectiveness depends on students’ active engagement. In preview settings, students exercise autonomy in deciding whether—and which—resources to use. Thus, personalized learning benefits require understanding learners’ decision-making processes. Examining how motivational orientations and engagement profiles shape responses to differentiated preview resources is crucial for assessing the real-world impact of such strategies in higher education.

2.2. GAI for Educational Content Creation: Opportunities and Limitations

Recent advances in GAI have introduced new opportunities for educational content creation in higher education. Research suggests that GAI can support multiple instructional functions, including content explanation, example generation, feedback provision, and task construction [35,36,37,38]. In practice, LLMs have been shown to help instructors quickly produce curriculum-aligned resources and examples, which are especially valuable in large-enrollment courses where manual differentiation is unsustainable [39].

Beyond instructor support, GAI can directly aid students as a supplementary tool. AI-generated explanations and feedback help clarify misunderstandings, explore problem-solving pathways, and foster reflection for self-regulation [40]. For example, LLM tools address misconceptions and provide tailored feedback during independent study, enhancing conceptual understanding in physics education [41,42]. These features are particularly useful in preview scenarios with autonomous engagement. From a motivational perspective, well-designed AI responses can create perceived dialogue and responsiveness, enhancing interest and persistence in technology-supported educational environments [43,44]. Chatbots such as ChatGPT-4 and Llama 2 that emulate Socratic questioning and offer immediate feedback have been found to promote personalized learning and strengthen critical thinking skills in higher education [45,46].

Despite these benefits, GAI faces significant limitations, including output instability and factual inaccuracies. Due to their probabilistic nature, LLMs often yield inconsistent outputs for the same prompt, varying in structure and depth, which undermines curricular coherence [47,48]. Hallucinations, referring to plausible but incorrect generated content, remain a potential risk in AI-generated educational materials. This risk may be particularly important in independent preview settings, where undetected errors could increase cognitive load, reinforce misconceptions, and undermine learners’ trust in the materials [49,50].

Overall, generative AI offers automatic production of personalized preview resources in higher education, but its benefits depend on robust quality controls beyond simple prompting. LLM-based workflows that decompose tasks and embed checks for accuracy, coherence, and alignment show promise for improving content reliability. However, even high-quality AI-generated materials cannot guarantee effective personalized learning without student engagement. From a learner-centered perspective, fully evaluating GAI-supported personalized learning therefore requires examining how psychological factors, such as motivation and engagement, shape students’ resource selection. This gap highlights the need to integrate generative AI research with established theories of learning motivation and engagement to explain students’ choices in AI-generated environments better.

2.3. Learning Motivation and Engagement in Technology-Supported Contexts

Research consistently shows that students’ learning behaviors in technology-supported environments are shaped by motivational orientations and multidimensional engagement, encompassing cognitive, emotional, and behavioral components [51,52,53]. In preview contexts with multiple resource options, engagement guides resource selection: cognitively engaged learners favor challenging, high-cognitive resources, while emotionally engaged learners prefer interesting and enjoyable ones [54].

Learning motivation provides the foundational psychological basis for such engagement patterns. Intrinsic motivation, characterized by interest, enjoyment, and challenge, has been shown to foster deep cognitive investment and emotional involvement [55]. Intrinsically motivated students are more likely to persist in effortful processing and experience positive affect [56]. Thus, intrinsic motivation is expected to be positively associated with both cognitive and emotional engagement in preview. In contrast, extrinsic motivation is oriented toward external outcomes. Extrinsically motivated learners tend to allocate effort strategically to tasks perceived as instrumental for achievement, emphasizing performance-oriented cognitive engagement while exhibiting weaker emotional involvement [57]. In GAI-supported scenarios, this pattern may manifest as a preference for resources with higher perceived academic or competitive value [58].

Engagement serves as a potential mediating mechanism linking motivation to learning-related behaviors and choices [59]. In preview scenarios, different forms of engagement may be associated with different resource preferences. Emotional engagement may be more closely related to learners’ responses to accessible and interesting materials, whereas cognitive engagement may be more closely related to preferences for resources involving greater conceptual challenge [60]. Accordingly, motivation may be associated with students’ resource preferences through distinct engagement pathways. This theoretical perspective provides a basis for examining whether students’ motivational and engagement characteristics are associated with their preferences for differentiated preview resources.

3. Methodology

3.1. Generation of Personalized Learning Materials

Preview materials were developed as structured HTML resources to support independent pre-class learning in university physics. For each topic, two parallel versions were produced with identical content coverage and structure but differing in cognitive demand and contextual framing. The basic version featured lower cognitive load and included step-by-step explanations, worked examples with everyday real-life contexts, and simpler language to support foundational understanding. The advanced version featured greater conceptual depth, more abstract reasoning, more challenging tasks, and related disciplinary cases. It was designed to accommodate learners who were comfortable with more cognitively demanding content

To ensure consistency in the generation process, we developed a context engineering workflow for generation, evaluation, and integration (hereafter, the context engineering workflow) implemented through a multi-agent LLM system. Context engineering refers to a structured prompting technique that explicitly defines and constrains the generation environment for LLMs to enhance output relevance, consistency, and quality [61]. In this study, the generation context was explicitly specified through instructional objectives, learner profiles, topic boundaries, pedagogical constraints, and output specifications [62,63,64]. These contextual constraints formed the basis for the workflow architecture and were used consistently across the generation, evaluation, revision, and integration stages.

The proposed workflow was specifically designed for the generation of personalized pre-class physics resources. Although individual techniques used in the workflow, such as task decomposition, specialized agents, and iterative evaluation, have been explored in previous LLM-based agent research, the workflow presented in Figure 1 was designed and implemented specifically in this study. Its contribution lies in the way these components were organized into a unified generation process for educational resources and in the explicit use of context engineering to coordinate the generation and revision of different instructional components.

Figure 1. The Context Engineering Workflow for Generation, Evaluation, and Integration.

The workflow comprised four specialized generation agents: knowledge expansion, case construction, example explanation, and practice design. Each agent was assigned a distinct instructional function rather than being prompted to generate the entire webpage as a single output. Kimi-k2 (Moonshot AI, Beijing, China) was used for knowledge expansion, case construction, and example explanation, whereas GPT-5.1 (OpenAI, San Francisco, CA, USA) was used for practice design, including physics questions and solution rationales. The specific model identifiers, providers, generation settings, and prompt templates are reported in the Appendix A.

The workflow consisted of three interconnected stages: generation, quality control and revision, and integration. In the generation stage, the webpage-generation task was decomposed into instructional components and assigned to the corresponding specialized agents. Each agent received structured contextual information relevant to its assigned task and independently generated the corresponding component. In the quality-control and revision stage, each generated component was assessed by an evaluation agent against predefined quality requirements. The evaluation agent produced dimension-level scores and overall feedback. A score and feedback extraction node then transferred only the structured evaluation information required for revision to the corresponding generation agent. The component was subsequently revised and resubmitted for evaluation. This iterative process continued until the component reached the predefined quality threshold of 4.0. In the integration stage, components that satisfied the threshold were transferred to the content integration agent, which assembled them into a coherent HTML webpage while preserving the predefined structure and instructional requirements (Figure 1).

Importantly, the evaluation performed within the workflow served exclusively as a quality-control mechanism for identifying and correcting deficiencies before content integration. Its scores were not used in the subsequent assessment of the finalized webpages or in the comparative analysis of the generation methods. Moreover, the generation and evaluation functions were connected through controlled information transfer: the generation agents did not receive the complete evaluation interaction or context, but only the extracted scores and feedback relevant to revision. This procedural separation ensured that the within-workflow evaluation functioned as an internal quality-control mechanism rather than as the basis for the subsequent evaluation of the finalized materials.

After all components had passed the predefined quality threshold and had been integrated into complete webpages, the finalized materials entered a separate post-generation assessment stage. In this stage, the finalized webpages were independently evaluated using the predefined rubric, without affecting the generation or revision process. Kimi-k2 conducted repeated rubric-based assessments of the finalized webpages, while five university physics teachers independently evaluated the same materials using the identical rubric. The procedures for this post-generation assessment and the corresponding results are described in Section 3.2.

Using this workflow, two parallel preview resources were generated for each of nine university-level physics topics, resulting in 18 HTML files. The nine topics were selected from actual university physics curricula using stratified sampling and covered mechanics (three topics), electromagnetism (three topics), and quantum mechanics (three topics), thereby providing diversity in disciplinary content and conceptual complexity.

3.2. Personalized Learning Material Assessment

Following completion of the workflow, the finalized HTML webpages underwent post-generation assessment. This stage was separate from the within-workflow evaluation used to support iterative revision and was intended to examine the quality and instructional characteristics of the completed materials.

Assessment was based on an eight-dimensional rubric developed with reference to established educational quality frameworks and prior research on AI-generated educational content assessment. The rubric comprised accuracy, vividness, organization, relevance, adaptability, practicality, currency, and challenge. Each dimension was rated on a 5-point scale (1 = Poor, 5 = Excellent). Although the same eight dimensions were applied across all content components, the interpretation of each dimension was adapted to the pedagogical function of the respective section. For instance, accuracy in knowledge explanations emphasized conceptual precision and the avoidance of misconceptions, whereas in practice design it focused on the correctness of question stems, answer options, and solution rationales. Similar contextual adaptations were applied to the other dimensions to ensure construct alignment while maintaining a unified evaluation framework. Table 1 presents the eight evaluation dimensions and their general scoring criteria, which were consistently applied across the different content components.

Table 1. Evaluation Rubric Dimensions of LLM-Generated Educational Resources.

Dimension	Description
Accuracy	Correctness and factual precision of the content [65]
Vividness	Expressiveness and attractiveness of explanations and examples [66]
Organization	Clarity, logical flow, and coherence of the overall structure [67]
Relevance	Degree of alignment with course objectives and learners’ prerequisite knowledge [68]
Adaptability	Appropriateness and accessibility for learners with diverse academic backgrounds [69]
Practicality	Strength of connection to real-life applications and practical use cases [70]
Currency	Extent to which the content reflects contemporary issues and current academic/research frontiers [71]
Challenge	Potential to stimulate inquiry, intellectual curiosity, and critical thinking [72]

To examine the stability of the LLM-based assessment with expert judgment, the finalized preview webpages were independently evaluated by the LLM evaluator and five university physics teachers with more than twenty years of teaching experience using the same predefined rubric. The LLM evaluator conducted ten independent assessments of each webpage to examine the stability of repeated automated scoring [73].

The stability of the LLM evaluator across repeated assessments was assessed using the coefficient of variation (CV), calculated from the mean and standard deviation of the ten repeated scoring runs for each webpage. Lower CV values indicated less variation across repeated assessments. For the comparison with expert judgment, the ratings provided by the five university physics teachers were averaged for each webpage to obtain a mean expert rating. The mean LLM rating for each webpage was then compared with the corresponding mean expert rating separately for the basic and advanced versions using paired-samples t-tests, with the webpage as the unit of analysis (n = 9 for each version).

3.3. Investigation of Personalized Learning Preferences

A quantitative explanatory design was employed to examine how intrinsic and extrinsic motivation were associated with students’ preferences for previewing HTML files, and whether cognitive and emotional engagement mediated these associations. The study was conducted in a compulsory first-year university physics course delivered in blended form. Participants were 160 first-year undergraduate students who volunteered for the study. All students had access to the same set of LLM-generated preview HTML files.

The questionnaire consisted of two parts. The first part presented the basic version of the pre-class preview webpage, and the second part presented the advanced version. All students used and evaluated both versions. After using each webpage, students evaluated the specified content modules and then responded to the item, “I am willing to use this webpage for pre-class preparation.” For each version, the ratings of the specified modules and the willingness-to-use item were averaged for each student to obtain an overall preference score for that version. Thus, each student had two preference scores: one for the basic version and one for the advanced version.

3.3.1. Learning Motivation and Engagement

Motivation was assessed using an adapted Work Preference Inventory [74], modified for the academic preview contexts. Exploratory factor analysis supported a two-factor structure: intrinsic motivation (challenge and enjoyment) and extrinsic motivation (socially oriented). Items were rated on a 5-point Likert scale. Internal consistency was acceptable (Cronbach’s α = 0.827).

Engagement was measured with a modified Student Engagement in Schools Questionnaire [75], adapted to preview learning. Given the study’s focus on motivational pathways to resource preferences in a voluntary, choice-based context, only the cognitive and emotional engagement dimensions were retained in the model. Behavioral engagement was not included in the model because the study focused on cognitive and emotional engagement as potential mediators of students’ preview preferences. Preview preference was operationalized separately using students’ willingness-to-use evaluations of the corresponding preview webpages. The analysis therefore examined how cognitive and emotional engagement mediate the associations of intrinsic and extrinsic motivation with students’ preferences for basic and advanced preview webpages. All items were rated on a 5-point Likert scale, with excellent internal consistency (Cronbach’s α = 0.876).

3.3.2. Data Collection and Analysis

Regression-based mediation analyses were conducted using Hayes’ PROCESS macro (Model 4; Andrew F. Hayes, Columbus, OH, USA) to examine whether emotional engagement and cognitive engagement statistically mediated the associations between learning motivation and students’ preferences for basic and advanced preview webpages. Separate mediation models were estimated for each combination of motivation type (intrinsic vs. extrinsic), engagement type (emotional vs. cognitive), and webpage preference (basic vs. advanced). For the mediation analyses, indirect effects were estimated using 5000 bootstrap samples. An indirect effect was considered statistically significant when its 95% bootstrap confidence interval did not include zero. Given the cross-sectional and non-experimental design, the results are interpreted as statistical associations and indirect pathways rather than causal effects.

These analyses were conducted within an integrated framework of motivation, engagement, and preference. The hypotheses collectively addressed RQ2, which examined the factors associated with students’ preferences for personalized preview options. Specifically, H1a–H1d examined the associations between learning motivation and emotional and cognitive engagement, whereas H2a–H2d and H3a–H3d examined the indirect associations between learning motivation and preview-material preferences through emotional and cognitive engagement, respectively. The following hypotheses were tested:

H1a: 

Intrinsic motivation is positively associated with emotional engagement.

H1b: 

Intrinsic motivation is positively associated with cognitive engagement.

H1c: 

Extrinsic motivation is positively associated with cognitive engagement.

H1d: 

Extrinsic motivation is positively associated with emotional engagement.

H2a: 

Emotional engagement mediates the association between intrinsic motivation and basic preference.

H2b: 

Emotional engagement mediates the association between extrinsic motivation and basic preference.

H2c: 

Emotional engagement mediates the association between intrinsic motivation and advanced preference.

H2d: 

Emotional engagement mediates the association between extrinsic motivation and advanced preference.

H3a: 

Cognitive engagement mediates the association between intrinsic motivation and basic preference.

H3b: 

Cognitive engagement mediates the association between extrinsic motivation and basic preference.

H3c: 

Cognitive engagement mediates the association between intrinsic motivation and advanced preference.

H3d: 

Cognitive engagement mediates the association between extrinsic motivation and advanced preference.

Using an integrated framework, the study examines the relationships among motivation, multidimensional engagement, and learning choices. This approach helps elucidate how the interaction between personalized learning materials and learner characteristics is associated with student engagement and learning decisions.

4. Results

4.1. Quality Assessment of the Generated Preview Resources

4.1.1. Quality of LLM-Based Generation Strategies for Preview Resources

To evaluate the quality of the generated preview resources and examine the effectiveness of the generation strategy adopted in this study, four LLM-based approaches were compared across the nine university physics topics. All four approaches generated preview webpages using the same predefined webpage structure and content requirements, including conceptual explanations, case analyses, worked examples, and practice exercises. The strategies differed in their generation workflows, quality control procedures, and, where applicable, model configurations (see Appendix B for the detailed prompts of the first three approaches):

(1)

Direct generation with single-prompt: A single comprehensive prompt was provided to Kimi-k2 to directly generate the entire webpage without any iterative refinement or external augmentation.

(2)

Dual-agent generate-evaluate approach: Kimi-k2 was used as both the content generation and evaluation agent in an iterative loop. Feedback from the evaluation agent triggered revisions if criteria were not met (see Figure 2 for the workflow mechanism).

Figure 2. Mechanism of the Dual-Agent Generate-Evaluate Approach.

(3)

Deep-research supported generation approach: This method incorporated Meta AI’s Deep Research functionality (Shanghai Metaso Network Technology Co., Ltd., Shanghai, China). Deep Research refers to an agentic, multi-step generation process that augments content generation with web-based information retrieval, external reference synthesis, and iterative reasoning [76]. In this study, it was included as a comparison approach representing a research-oriented generation process augmented by information retrieval.

(4)

The context engineering workflow: This approach is the strategy adopted in this research work. It employed structured context engineering to guide multiple specialized agents in parallel; only content meeting quality thresholds was passed to a final Content Integration Agent for coherent assembly. The resulting basic and advanced webpage examples are illustrated in Figure 3. The final webpages contain LLM-generated textual content together with visual and multimedia resources. Images and other visual materials were incorporated from the instructional resources prepared for the study, whereas the explanatory videos embedded in the webpages were generated using NotebookLM (Google LLC, Mountain View, CA, USA) based on the corresponding instructional content. These videos were subsequently embedded into the HTML webpages as supplementary explanations.

Figure 3. Sample Preview Webpages.

To reduce potential procedural confounding, the post-generation evaluation was conducted only after all webpages had been finalized, using the same predefined rubric and identical evaluation procedure across all four approaches (see Appendix C for the post-generation evaluation prompt). The evaluation was conducted in fresh, context-isolated sessions without access to the preceding generation conversations or generation records. No information regarding the generation model, generation approach, or source of the webpages was provided in the evaluation input. Thus, the Kimi-k2 evaluator assessed the finalized webpages solely on the basis of the submitted content and predefined evaluation criteria, without being informed which generation approach had produced each webpage. The resulting evaluation scores were not fed back into or used to modify any finalized webpage.

For each webpage, the rubric dimensions were averaged to obtain an overall rubric score. Paired-samples t-tests were then conducted across the nine physics topics within each version to compare the context engineering workflow with each alternative generation strategy. The results are presented in Table 2.

Table 2. Paired-Samples t-Test Results Comparing Generation Methods with the Context Engineering Workflow.

Version	Comparison	Mean Difference	95% CI	dz	t	df	p
Basic	Context Engineering Workflow vs. Dual Agent	1.878	[1.620, 2.136]	5.594	16.782	8	<0.001
	Context Engineering Workflow vs. Single Prompt	0.435	[0.193, 0.677]	1.383	4.149	8	0.003
	Context Engineering Workflow vs. Deep Research	0.406	[0.197, 0.615]	1.494	4.481	8	0.002
Advanced	Context Engineering Workflow vs. Dual Agent	2.314	[1.970, 2.659]	5.163	15.488	8	<0.001
	Context Engineering Workflow vs. Single Prompt	0.720	[0.455, 0.985]	2.092	6.276	8	<0.001
	Context Engineering Workflow vs. Deep Research	0.404	[0.227, 0.582]	1.753	5.260	8	0.001

For the basic version, the context engineering workflow received a significantly higher mean rubric score than the dual-agent approach (M = 4.62 vs. 2.74), t(8) = 16.782, p < 0.001, d_z = 5.594, 95% CI [1.620, 2.136]. The context engineering workflow also received a significantly higher score than the single-prompt approach (M = 4.62 vs. 4.18), t(8) = 4.149, p = 0.003, d_z = 1.383, 95% CI [0.193, 0.677], and the deep-research approach (M = 4.62 vs. 4.21), t(8) = 4.481, p = 0.002, d_z = 1.494, 95% CI [0.197, 0.615]. A similar pattern was observed for the advanced version. The context engineering workflow received a significantly higher mean rubric score than the dual-agent approach (M = 4.83 vs. 2.52), t(8) = 15.488, p < 0.001, d_z = 5.163, 95% CI [1.970, 2.659]. It also outperformed the single-prompt approach (M = 4.83 vs. 4.11), t(8) = 6.276, p < 0.001, d_z = 2.092, 95% CI [0.455, 0.985], and the deep-research approach (M = 4.83 vs. 4.43), t(8) = 5.260, p = 0.001, d_z = 1.753, 95% CI [0.227, 0.582].

The assumptions underlying the paired-samples t-tests were assessed using Shapiro–Wilk tests and Q-Q plots of the six paired difference scores. Five of the six difference scores showed no significant departure from normality (all p > 0.05). The difference score for the advanced context engineering workflow versus single-prompt comparison showed a significant departure from normality, W = 0.809, p = 0.026. A Wilcoxon signed-rank test was therefore conducted as a sensitivity analysis for this comparison. The non-parametric test also indicated a significant difference (p = 0.008), consistent with the paired-samples t-test.

Overall, the context engineering workflow received significantly higher rubric scores than all three comparison approaches for both the basic and advanced versions under the automated assessment procedure used in this study. The magnitude of the paired differences varied across comparison strategies, with the largest differences observed between the workflow and dual-agent approaches. The results should be interpreted as comparative differences in rubric-based evaluation scores rather than as evidence of a causal effect of the generation strategy. The preview webpages generated through the context engineering workflow were subsequently used in the investigation of students’ motivational and preference-related responses.

4.1.2. Validation of LLM-Based Assessment

To examine the stability of the automated assessment procedure and the difference between LLM-based and expert ratings, the finalized preview webpages were evaluated using repeated LLM-based assessments and an independent expert assessment. The LLM evaluator conducted ten independent scoring runs for each of the 18 finalized preview webpages, including nine basic webpages and nine advanced webpages, using the same predefined rubric. For each webpage, the mean score, standard deviation, and CV were calculated across the ten repeated assessments.

As shown in Table 3, the mean CV was 3.45% for the advanced version and 9.42% for the basic version. The advanced version also showed a lower standard deviation (SD = 0.17) than the basic version (SD = 0.44), indicating less variation in the repeated LLM ratings for the advanced materials.

Table 3. Stability of LLM-Based Assessment Across Repeated Evaluations.

Version	Mean Score	Standard Deviation	CV
Basic	4.62	0.44	9.42%
Advanced	4.83	0.17	3.45%

The differences between the LLM-based and expert ratings were further examined using paired-samples t-tests. For each webpage, the mean LLM rating obtained from the ten repeated assessments was compared with the mean rating of the five expert teachers. For the advanced version, the mean difference between the LLM and expert ratings was −0.039 points, and the difference was not statistically significant, t(8) = −0.989, p = 0.352. For the basic version, the mean difference was −0.089 points, which was also not statistically significant, t(8) = −1.839, p = 0.103. The results are presented in Table 4.

Table 4. Paired-Samples t-Test Comparing LLM and Expert Ratings.

Version	Mean Difference	SD	SE	95% CI	t	df	p
Basic	−0.089	0.145	0.048	[−0.200, 0.023]	−1.839	8	0.103
Advanced	−0.039	0.118	0.039	[−0.130, 0.052]	−0.989	8	0.352

Taken together, the repeated-assessment results indicate relatively stable LLM scoring across the finalized preview webpages. The paired-samples t-tests further showed that the mean LLM ratings did not differ significantly from the mean ratings of the five expert teachers for either the basic or advanced version. These findings indicate that, under the evaluation conditions used in this study, the automated assessment procedure produced relatively stable repeated ratings and did not show a statistically significant mean difference from the expert ratings within either version.

4.2. Associations Between Learning Motivation and Preview Material Preferences

Based on the validated evaluation data, this section examines the associations between intrinsic and extrinsic motivation, learning engagement, and students’ preferences for basic and advanced preview webpages. Regression-based mediation analyses were conducted using PROCESS Model 4, with emotional and cognitive engagement examined separately as mediators. The results are summarized in Table 5 and Table 6.

Table 5. Results of Hypotheses Concerning Motivation and Engagement.

Hypothesis	b	SE	t	p	95% CI	Result
H1a	0.490	0.090	5.427	<0.001	[0.312, 0.668]	Supported
H1b	0.453	0.070	6.438	<0.001	[0.314, 0.592]	Supported
H1c	0.094	0.064	1.475	0.142	[−0.032, 0.221]	Not supported
H1d	0.104	0.080	1.304	0.194	[−0.054, 0.261]	Not supported

Note. b represents the unstandardized regression coefficient. For indirect effects, statistical significance was determined based on 5000 bootstrap samples and 95% bootstrap confidence intervals. An indirect effect was considered significant when the confidence interval did not include zero.

Table 6. Mediation Analysis and Hypothesis Testing Results for Learning Motivation and Preview Material Preferences.

Hypothesis	X	Mediator	Y	Path a	Path b	Indirect Effect	BootSE	95% CI	Direct Effect	Total Effect	Result
H2a	Intrinsic motivation	Emotional engagement	Basic preference	0.490	0.528	0.259	0.062	[0.150, 0.392]	−0.034	0.225	Supported
H2b	Extrinsic motivation	Emotional engagement	Basic preference	0.104	0.508	0.053	0.042	[−0.031, 0.137]	0.097	0.150	Not supported
H2c	Intrinsic motivation	Emotional engagement	Advanced preference	0.490	0.521	0.255	0.064	[0.138, 0.384]	0.100	0.356	Supported
H2d	Extrinsic motivation	Emotional engagement	Advanced preference	0.104	0.532	0.055	0.045	[−0.032, 0.149]	0.202	0.258	Not supported
H3a	Intrinsic motivation	Cognitive engagement	Basic preference	0.453	0.575	0.260	0.061	[0.155, 0.394]	−0.035	0.225	Supported
H3b	Extrinsic motivation	Cognitive engagement	Basic preference	0.094	0.544	0.051	0.035	[−0.015, 0.127]	0.099	0.150	Not supported
H3c	Intrinsic motivation	Cognitive engagement	Advanced preference	0.453	0.555	0.251	0.069	[0.127, 0.397]	0.105	0.356	Supported
H3d	Extrinsic motivation	Cognitive engagement	Advanced preference	0.094	0.573	0.054	0.038	[−0.014, 0.137]	0.203	0.258	Not supported

Note. Path a represents the association between X and the mediator; path b represents the association between the mediator and Y controlling for X; the indirect effect represents a × b; the direct effect represents c′, and the total effect represents c. Unstandardized coefficients are reported.

4.2.1. Hypothesis Testing

The results for H1a–H1d are as follows. The findings supported H1a and H1b. Intrinsic motivation was positively associated with emotional engagement (b = 0.490, SE = 0.090, t = 5.427, p < 0.001, 95% CI [0.312, 0.668]) and cognitive engagement (b = 0.453, SE = 0.070, t = 6.438, p < 0.001, 95% CI [0.314, 0.592]). In contrast, extrinsic motivation was not significantly associated with cognitive engagement (b = 0.094, p = 0.142) or emotional engagement (b = 0.104, p = 0.194). Thus, H1c and H1d were not supported. Detailed results are presented in Table 5.

The mediation hypotheses examined whether emotional and cognitive engagement mediated the associations between intrinsic or extrinsic motivation and preferences for basic and advanced preview materials. The bootstrap results supported H2a, H2c, H3a, and H3c. Specifically, intrinsic motivation showed significant indirect associations with both basic and advanced preferences through emotional and cognitive engagement, with all corresponding 95% bootstrap confidence intervals excluding zero. In contrast, none of the indirect associations involving extrinsic motivation were statistically significant, as all corresponding bootstrap confidence intervals included zero. Therefore, H2b, H2d, H3b, and H3d were not supported. The detailed path coefficients and mediation estimates are presented in Table 6.

4.2.2. Direct and Total Associations of Intrinsic and Extrinsic Motivation

The direct and total associations provide additional information for interpreting the mediation models. The total association represents the association between motivation and webpage preference before accounting for the mediator, whereas the direct association represents the association between motivation and webpage preference after accounting for the mediator.

For intrinsic motivation, the total association with basic preference was not statistically significant (b = 0.225, p = 0.051), whereas the total association with advanced preference was positive and statistically significant (b = 0.356, p = 0.002). However, the direct associations between intrinsic motivation and basic preference (b = −0.034, p = 0.764) and advanced preference (b = 0.100, p = 0.376) were not statistically significant after accounting for the mediator. For extrinsic motivation, the total association with basic preference was not statistically significant (b = 0.150, p = 0.111). In contrast, extrinsic motivation showed a significant positive total association with advanced preference (b = 0.258, p = 0.007). Its direct association with basic preference was not statistically significant (b = 0.097, p = 0.256), whereas the direct association with advanced preference remained statistically significant after accounting for emotional engagement (b = 0.202, p = 0.017) or cognitive engagement (b = 0.203, p = 0.020).

Overall, intrinsic motivation was significantly associated with advanced preference at the total-association level, but its direct association was no longer statistically significant after accounting for engagement. In contrast, extrinsic motivation showed both a significant total association and a significant direct association with advanced preference, whereas neither motivation type showed a significant total or direct association with basic preference

4.2.3. Mediating Role of Cognitive and Emotional Engagement

The bootstrap mediation analyses revealed distinct patterns for intrinsic and extrinsic motivation. For intrinsic motivation, both emotional and cognitive engagement constituted significant indirect pathways to preferences for basic and advanced preview materials. The indirect effects through emotional engagement were 0.259 for basic preference and 0.255 for advanced preference, while those through cognitive engagement were 0.260 and 0.251, respectively. In all four cases, the 95% bootstrap confidence intervals excluded zero, indicating statistically significant indirect associations. For extrinsic motivation, neither emotional nor cognitive engagement constituted a statistically significant indirect pathway to either type of preview preference. The indirect effects ranged from 0.051 to 0.055, and all corresponding 95% bootstrap confidence intervals included zero.

Taken together, the findings suggest differentiated motivational pathways. Intrinsic motivation was associated with preview-material preferences primarily through emotional and cognitive engagement, whereas extrinsic motivation showed a direct association with advanced preference without evidence of an engagement-mediated pathway. The significant associations identified in these analyses are summarized in Figure 4.

Figure 4. Significant Associations Between Motivation, Engagement, and Preview Material Preferences. Note. * p < 0.05, *** p < 0.001.

5. Discussion

This study investigated three key aspects of using GAI for personalized preview in university physics: namely, how to evaluate the quality of LLM-generated learning materials, how to generate reliable high-quality content through multi-agent workflows, and how motivational and engagement factors shape students’ differentiated options. The findings highlight both technical and learner-centered considerations in AI-supported personalized education.

5.1. Advantages of the Context Engineering Workflow for Content Generation

A central finding in this study is that the context engineering yielded more favorable evaluation scores than the single-prompt, dual-agent, and deep-research approaches. These differences should be interpreted as differences among the complete generation approaches under the configurations adopted in this study, rather than as evidence of an isolated causal effect of context engineering alone. Context engineering goes beyond traditional prompt engineering by systematically designing the entire informational environment provided to the LLM, including explicit instructional objectives, learner profiles, pedagogical constraints, topic boundaries, output formats, and quality criteria. This structured contextual framing enables more consistent generation, particularly when combined with role-specialized agents and iterative evaluation loops.

Literature supports this result. Recent surveys and empirical studies show that providing richer and more structured contextual information can improve the reliability and coherence of LLM-generated outputs by constraining the generation process and reducing ambiguity in task specifications [68]. Integrating context engineering with LLMs enables the creation of personalized preview resources that adapt to individual contexts, thereby supporting differentiated cognitive demands and learner needs [77]. In multi-agent systems, providing constrained context allows task decomposition and parallel processing while maintaining coherence through centralized integration [78]. These features may help explain the more favorable evaluation scores observed for the context engineering workflow in the present study. Deep-research approaches, while benefiting from external retrieval, still exhibit moderate performance because they typically apply augmentation at a surface level without deep pedagogical constraints or specialized role separation, leading to less targeted educational depth [79]. The higher rubric-based evaluation scores obtained by the context engineering workflow across both basic and advanced versions suggest its potential suitability for foundational STEM courses, where content quality, accuracy, and cognitive appropriateness are important considerations. Its ability to support differentiated content generation may also be relevant to courses serving learners with heterogeneous levels of prior knowledge.

5.2. Factors Influencing Students’ Preferences for Personalized Preview Resources

Regarding factors influencing preferences, the results indicate differentiated associations between motivation, engagement, and students’ preview preferences. Intrinsic motivation showed significant indirect associations with preferences for both basic and advanced resources through emotional and cognitive engagement (indirect effects = 0.251–0.260; all 95% bootstrap CIs excluded zero). In contrast, extrinsic motivation showed a significant positive direct association with preference for advanced resources (b = 0.202–0.203, p < 0.05), while no significant indirect associations were observed through either emotional or cognitive engagement.

These patterns are broadly consistent with self-determination theory (SDT) and prior research on motivation in technology-supported and online learning contexts. Previous studies have reported positive associations between intrinsic motivation and both cognitive engagement and emotional engagement, as well as between engagement and various learning-related outcomes [80]. Extrinsic motivation often shows direct links to performance-oriented or instrumental choices, with weaker or non-significant mediation through engagement, especially in structured or high-stakes settings [81].

The current results are broadly consistent with these patterns, particularly the stronger indirect associations of intrinsic motivation via engagement and the direct associations between extrinsic motivation and advanced resource preference. This pattern may reflect the study’s specific pre-class, voluntary choice context in a foundational physics course. For students with strong intrinsic motivation, their preferences may have been more closely associated with emotional and cognitive engagement, which were in turn associated with willingness to use both basic and advanced preview resources. In contrast, students with stronger extrinsic motivation may have been more likely to prefer advanced resources because they perceived these materials as more challenging or potentially useful for achieving academic goals.

5.3. Implications for Education

From an educational perspective, the regression-based path analysis results underscore the differential roles of motivation types in shaping student engagement and resource preferences. Intrinsic motivation showed relatively strong indirect associations with preferences for both basic and advanced resources through emotional and cognitive engagement, highlighting the potential relevance of intrinsic motivation to students’ voluntary engagement with differentiated preview resources. Vividness, interest, and situational appeal in content are key facilitators of intrinsic motivation, as they trigger curiosity, enjoyment, and deep-thinking processes [82]. To support engagement across students with different learning needs, preview materials should combine vivid and interesting elements with appropriately differentiated levels of cognitive challenge. Such differentiation may allow students to select resources that better match their perceived learning needs without assuming that motivation alone determines the appropriate level of difficulty. It helps intrinsically motivated students thrive through enjoyment and challenge, while also drawing in extrinsically oriented students by linking content to performance goals.

6. Conclusions

This study provides answers to the two research questions concerning the evaluation, generation, and personalization of LLM-generated preview resources. Regarding RQ1, a context engineering workflow for generation, evaluation, and integration was developed to support the systematic production of differentiated preview resources. The workflow decomposed content generation into specialised tasks, incorporated iterative evaluation and revision, and integrated components that met predefined quality thresholds into coherent HTML webpages. Under the automated assessment procedure adopted in this study, this workflow achieved higher rubric scores than the single-prompt, dual-agent, and deep-research approaches. Regarding RQ2, the mediation analyses revealed differentiated pathways between learning motivation and students’ preview preferences. Intrinsic motivation was positively associated with both emotional and cognitive engagement, and significant indirect associations with preferences for both basic and advanced resources were observed through these engagement dimensions. Extrinsic motivation, in contrast, showed a significant direct association with preference for advanced resources but no significant indirect pathway through either engagement dimension. Together, these findings suggest that incorporating learners’ motivational and engagement characteristics may provide a useful basis for designing differentiated preview options for voluntary pre-class learning.

Despite these contributions, this study has several limitations. It is confined to a single foundational physics course at one institution, relies primarily on self-reported motivational measures, and employs a cross-sectional design, which constrains application across disciplines, cultural contexts, and longer-term outcomes. In addition, students’ prior physics knowledge or competence was not directly measured in the preference analysis. Although the basic and advanced resources were designed to differ in cognitive demand, the study therefore cannot determine whether students’ preferences reflected their actual levels of subject-matter competence. This limitation may introduce an alternative explanation for the observed associations between motivation and resource preference. The relatively modest sample size may also limit the stability and generalizability of the multiple mediation effects examined in this study. Future research should adopt longitudinal designs with larger and more diverse samples, incorporate objective measures of prior knowledge or physics competence, and examine whether motivation and engagement interact with competence to predict preferences for resources with different levels of cognitive demand. Future research should also extend the approach to diverse subjects and student populations, and explore real-time adaptive AI systems that dynamically tailor resources based on ongoing learner feedback and engagement signals.

Appendix A.1. Overview of Prompt Design

The prompts were developed according to the functional architecture of the multi-agent workflow. Rather than using a single general-purpose prompt for all content generation, the workflow decomposed the instructional task into five components. A task-decomposition agent first transformed the overall instructional task and learner category into structured subtask specifications. These specifications were subsequently provided as contextual inputs to the corresponding generation agents. Prompt design followed four principles: task specificity, learner-level adaptation, content-boundary control, and structured output. Each generation prompt specified the target learner category, learning objectives, subtask requirements, content constraints, and output format. The prompts for the basic and advanced versions used the same instructional content scope but imposed different requirements for cognitive demand, contextualization, mathematical reasoning, and application depth. This enabled the two versions to maintain comparable content coverage while differing systematically in the level of cognitive demand. All prompts were implemented as structured system-level instructions, with task-specific contextual variables dynamically inserted at runtime. The principal variables included ‘student_category’, ‘target’, and the corresponding subtask specification. Where iterative revision was required, structured evaluation feedback was also provided to the relevant generation agent.

Appendix A.2. Task Decomposition Prompt

You are an instructional task-decomposition agent. Based on the instructional task and student category, decompose the task into explicit subtasks and return the result in a fixed JSON structure.

Dynamic inputs: ‘task = {}’, ‘student_category = {}’

Required outputs: ‘student_category’, ’target’, ’knowledge’, ’case’, ’example’, ’exercise’

Curriculum constraint: Automatically identify the teaching week corresponding to the instructional task according to the university Physics B curriculum. All generated task specifications must be restricted to knowledge introduced by that teaching week and must not include later concepts.

Task constraints: Generate task descriptions rather than actual questions, examples, or content. Specify the instructional focus and intended learning process for each component. Adapt the task description to the learner category. For students with relatively weaker foundations, emphasize intuitive understanding, foundational concepts, and connections with prior knowledge. For students with relatively stronger foundations, emphasize reasoning, modeling, conceptual depth, and exploratory thinking. The exercise component must specify the direction of multiple-choice question design.

Output constraint: Return only the specified JSON structure.

Appendix A.3. Generation Prompts

Appendix A.3.1. Knowledge Expansion

You are a university physics instructor. Generate the knowledge-explanation component for first-year engineering undergraduates.

Dynamic inputs: ‘student_category = {}’, ‘target = {}’, ‘task_requirement = {}’, ‘feedback = {}’

General requirements: Present the content with clear headings, paragraphs, and a summary. Avoid conversational or AI-like introductory expressions. Explain concepts systematically and accurately. Adapt the level of explanation to the specified learner category.

Basic version: Emphasize conceptual understanding, connections with secondary-school physics, gradual explanation, and everyday examples.

Advanced version: Provide greater conceptual detail, emphasize physical modeling and derivation, and appropriately incorporate engineering, research, or frontier contexts while maintaining connections with real-world phenomena.

Output constraint: Return only the specified JSON structure.

Appendix A.3.2. Case Construction

You are a university physics instructor. Generate two differentiated case analyses for first-year engineering undergraduates.

Dynamic inputs: ‘student_category = {}’, ‘target = {}’, ‘task_requirement = {}’, ‘feedback = {}’

Required structure for each case: Case title, Case background, Qualitative analysis of the physical process, Estimation or order-of-magnitude judgment of relevant physical quantities, Connection between the case and the target knowledge

Basic version: Use authentic everyday contexts and emphasize intuitive understanding of the relationship between phenomena, causes, and trends.

Advanced version: Use authentic engineering or research contexts and emphasize physical modeling, simplifying assumptions, symmetry, selection of key physical quantities, and engineering reasonableness.

Output constraint: Return only the specified JSON structure.

Appendix A.3.3. Example Explanation

You are a university physics instructor. Generate example-based explanations for first-year engineering undergraduates.

Dynamic inputs: ’student_category = {}’, ‘target = {}’, ‘task_requirement = {}’, ‘feedback = {}’

Required structure for each example: Example title, Problem description, Reasoning analysis, Solution procedure, Conclusion, Common errors

Basic version: Use relatively accessible applications of fundamental principles, provide detailed step-by-step reasoning, and emphasize the interpretation of physical quantities and common errors.

Advanced version: Include moderately more complex applications, conditional changes, or simple mathematical reasoning, with greater emphasis on physical modeling and transfer of knowledge.

Output constraint: Return only the specified JSON structure.

Appendix A.3.4. Practice Design

You are a university physics instructor. Generate an interactive multiple-choice exercise for first-year engineering undergraduates.

Dynamic inputs: ‘student_category = {}’, ‘target = {}’, ‘task_requirement = {}’, ‘feedback = {}’

Requirements: Generate four multiple-choice questions. Each question must have four options and exactly one correct answer. Ensure physical laws, mathematical expressions, units, answers, and explanations are scientifically correct. Provide an explanation indicating why the correct option is correct and why the alternatives are incorrect.

Basic version: Focus on conceptual understanding and simple calculations involving fundamental principles.

Advanced version: Include three conceptual discrimination questions and one simple application question. Questions should emphasize application and variation of conditions without requiring complex derivations.

Output format: Generate valid HTML containing the interactive questions, answer information, explanations, and a submission-and-scoring function.

Restriction: Do not introduce concepts beyond the current instructional scope.

Appendix A.4. Evaluation Prompts

You are a university physics educational-content evaluation agent.

Context: ‘student_category = {}’, ‘target = {}’, ‘task_requirement = {}’, ‘content = {}’.

Criteria: {criteria}.

Scoring: Rate each dimension from 1 to 5 using predefined descriptors.

Output: Return only a valid JSON object containing the dimension scores and a concise overall evaluation.

Appendix A.5. Model and Generation Settings

Table A1. The principal model configurations used in the workflow.

Workflow Component	Model	Temperature	Maximum Tokens
Task decomposition	Kimi-k2 (kimi-k2-0905-preview)	0.7	8000
Knowledge generation	Kimi-k2 (kimi-k2-0905-preview)	0.7	8000
Knowledge evaluation	Kimi-k2 (kimi-k2-0905-preview)	0.5	10,000
Case generation	Kimi-k2 (kimi-k2-0905-preview)	0.7	12,000
Case evaluation	Kimi-k2 (kimi-k2-0905-preview)	0.5	7000
Example generation	Kimi-k2 (kimi-k2-0905-preview)	0.7	8000
Example evaluation	Kimi-k2 (kimi-k2-0905-preview)	0.5	7000
Practice generation	GPT-5.1	0.7	10,000
Practice evaluation	GPT-5.1	0.3	7000

Table A1. The principal model configurations used in the workflow.

Workflow Component	Model	Temperature	Maximum Tokens
Task decomposition	Kimi-k2 (kimi-k2-0905-preview)	0.7	8000
Knowledge generation	Kimi-k2 (kimi-k2-0905-preview)	0.7	8000
Knowledge evaluation	Kimi-k2 (kimi-k2-0905-preview)	0.5	10,000
Case generation	Kimi-k2 (kimi-k2-0905-preview)	0.7	12,000
Case evaluation	Kimi-k2 (kimi-k2-0905-preview)	0.5	7000
Example generation	Kimi-k2 (kimi-k2-0905-preview)	0.7	8000
Example evaluation	Kimi-k2 (kimi-k2-0905-preview)	0.5	7000
Practice generation	GPT-5.1	0.7	10,000
Practice evaluation	GPT-5.1	0.3	7000

Appendix B.1. Dual-Agent Generate–Evaluate Approach

Appendix B.1.1. Generation Prompt

The dual-agent approach used a generation–evaluation workflow in which the generation agent produced the webpage content based on the specified learner category, learning objective, and task requirements. The generation model was Kimi-k2 (kimi-k2-0905-preview) with a temperature of 0.7 and a maximum output length of 14,000 tokens. The system prompt was:

System role: You are a university physics instructor. Based on the following inputs, develop pre-class learning materials for undergraduate engineering students.

Input information: Student category: {{ $json.student_category }}, Learning objective: {{ $json.target }}, Knowledge-explanation requirements: {{ $json.knowledge }}, Case-generation requirements: {{ $json.case }}, Example-explanation requirements: {{ $json.example }}, Practice-design requirements: {{ $json.exercise }}, Revision feedback, if available: {{ $json.feedback }}

Overall requirements: Generate a complete HTML webpage that can be directly used for instruction. Use standard HTML elements such as <h2>, <h3>, <p>, <ul>, and <li>. Do not use conversational introductory expressions such as “I will generate...” or “The following is...”. Ensure that the content is readable and pedagogically coherent.

Four instructional modules:

• Knowledge explanation (500–1000 Chinese characters): For students with relatively weaker foundations, emphasize conceptual explanation, gradual progression, everyday examples, and intuitive understanding. For students with relatively stronger foundations, provide more concise explanations emphasizing reasoning and modeling, with appropriate engineering or research examples.
• Case analysis: Generate two cases of approximately 500 Chinese characters each. For students with relatively weaker foundations, use everyday contexts or familiar physical phenomena and emphasize conceptual understanding rather than complex calculations. For students with relatively stronger foundations, cases may involve engineering, research, or technical contexts and may appropriately introduce abstraction and model simplification.
• Examples: For students with relatively weaker foundations, generate two examples at an accessible level, emphasizing fundamental principles, step-by-step reasoning, and common errors. For students with relatively stronger foundations, generate two more challenging examples involving reasoning, changes in conditions, or simple mathematical derivations, with emphasis on physical reasoning and model application.
• Practice: Generate multiple-choice questions using the specified learner-level requirements. Each question should contain four options and exactly one correct answer. Provide the correct answer and an explanation of the alternatives. Add an interactive submission and scoring function to the HTML page. Relatively weaker foundations: conceptual understanding and simple calculations. Relatively stronger foundations: changes in conditions and reasoning.

Output requirements: Generate a complete HTML webpage containing the four modules: knowledge explanation, case analysis, worked examples, and classroom practice. The final output must follow the specified JSON structure and contain only the required fields.

Appendix B.1.2. Evaluation Prompt

The dual-agent approach used a separate evaluation agent to assess the generated content and provide feedback for revision. The evaluation model was Kimi-k2 (kimi-k2-0905-preview), with a temperature of 0 and a maximum output length of 3000 tokens. The evaluation prompt was:

System role: You are an intelligent agent for evaluating the quality of university physics pre-class learning materials.

Student category: {{ $json.student_category }}

Learning objective: {{ $json.target }}

Teaching-task requirements: {{ $json.task_requirement }}

Evaluation criteria (1–5 points each): {{ criteria}}

Output format: strict JSON

Appendix B.2. Direct Single-Prompt Approach and Deep Research Approach

The direct single-prompt and Deep Research approaches used the same natural-language instructional specification to define the webpage content requirements. Separate prompt instances were used for the basic and advanced learner versions by varying the learner category and topic. In the direct single-prompt approach, Kimi-k2 (kimi-k2-0905-preview) was used as the generation model. The Deep Research approach used the Deep Research functionality to generate the complete webpage directly from the same type of natural-language specification. The common prompt template was:

System role: Assume that you are a university physics instructor teaching students with relatively {weak/strong} foundations. The instructional topic is {topic}. The webpage should contain four sections: knowledge explanation, case analysis, examples, and practice.

Requirements:

Knowledge explanation: The knowledge explanation should be vivid, emphasize the {key learning points/difficult points of the topic}, and contain approximately 500–1000 Chinese characters.

Case analysis: Generate two case analyses closely related to {topic}. The cases {should be connected with everyday life to improve interest and accessibility/should connect with technological or engineering applications}. Each case should contain approximately 500 Chinese characters.

Worked examples: The examples should have {a moderate/an appropriate} level of difficulty and emphasize the {key learning points/difficult points of the topic}.

Classroom practice: Practice should consist of multiple-choice questions that assess whether students have mastered the key concepts of {topic}.

Output requirement: Generate the complete instructional content in HTML format and provide the complete code that can be inserted into the webpage template.

Appendix B.3. Summary of Comparison-Approach Configurations

Table A2. Summary of Comparison-Approach Configurations.

Approach	Model	Generation Mechanism	Evaluation/Revision
Dual-agent	Kimi-k2 (kimi-k2-0905-preview)	Generation agent + evaluation agent	Separate evaluation agent; feedback used for revision
Direct single-prompt	Kimi-k2 (kimi-k2-0905-preview)	Single prompt generating the complete webpage	No iterative evaluation—revision loop
Deep Research	Meta AI Deep Research	Direct generation from natural-language instructional specification	No iterative evaluation—revision loop

Table A2. Summary of Comparison-Approach Configurations.

Approach	Model	Generation Mechanism	Evaluation/Revision
Dual-agent	Kimi-k2 (kimi-k2-0905-preview)	Generation agent + evaluation agent	Separate evaluation agent; feedback used for revision
Direct single-prompt	Kimi-k2 (kimi-k2-0905-preview)	Single prompt generating the complete webpage	No iterative evaluation—revision loop
Deep Research	Meta AI Deep Research	Direct generation from natural-language instructional specification	No iterative evaluation—revision loop

Appendix C.1. Overview

After the webpages generated by the four approaches were finalized, all webpages were evaluated using the same post-generation evaluation prompt and predefined evaluation rubric. The four approaches included the workflow, dual-agent generation–evaluation, direct single-prompt generation, and Deep Research generation.

The evaluation was conducted only after all webpages had been finalized. Each webpage was submitted to the evaluator in a fresh, context-isolated session without access to the preceding generation conversations or generation records. The evaluation input did not identify the generation approach or the model used to generate the webpage. The same evaluation criteria and scoring procedure were applied to all four approaches. Evaluation results were used only for comparative assessment and were not fed back into the generation or revision process.

Appendix C.2. Post-Generation Evaluation Prompt

System role: You are an expert in university physics educational content evaluation at a leading university, with extensive experience in evaluating introductory university physics courses and pre-class learning materials. You are responsible for evaluating university physics pre-class preview webpages designed for {{ student_category }} students.

Learner characteristics: Basic students: First-year engineering undergraduates who are studying introductory university physics systematically for the first time and have average mathematics and physics foundations. Advanced students: First-year engineering undergraduates with relatively strong learning foundations and greater capacity for independent learning, conceptual reasoning, and physical modeling.

Strictly evaluate the submitted preview webpages according to the scoring criteria and output format below. Each dimension must be assigned a score from 1 to 5.

Evaluation Criteria: {criteria}

Reference Evaluation Examples: {examples}

You will receive four HTML preview webpages, evaluating each webpage independently and completely. For every webpage: Compare its content item by item with the criteria and reference evaluation. Deduct points objectively when the submitted webpage falls below the reference standard. Provide a detailed, specific, and constructive rationale for every dimension. Keep each rationale concise, with approximately 50 characters in the original evaluation setting. For the practice module, evaluate the actual number, type, difficulty, scientific accuracy, answers, and explanations of the questions rather than assuming that the required structure has been satisfied. Evaluate all four webpages before producing the final response. Do not provide intermediate acknowledgements or other conversational statements.

Output Format: {content}