Beyond Code Assistants: A Technical–Ethical–Agentic Partnership Framework for LLM Integration in Project-Based CS Education

Abstract

The rapid emergence of large language models (LLMs) is reshaping Computer Science (CS) education. Yet, little is known about how students engage with these tools as both technical and ethical learning partners in authentic project-based environments. This qualitative study investigates how 29 undergraduate CS students used LLMs while developing open-ended AI applications in a project-based course. Analysis of project documentation, reflection logs, and presentation transcripts revealed three interconnected forms of student–LLM interaction. First, we observed a technical partnership, in which students leveraged LLMs for code generation, debugging, architectural planning, and API integration while working on complex development challenges. Second, there was an ethical-reflective partnership, as students negotiated transparency, originality, bias, and the risks of over-reliance, demonstrating elements of critical AI literacy. Third, students reported perceived shifts in their learning practices, including increased confidence, more intentional problem-solving, and changes in how they sought support from peers and instructors, suggesting a reconfiguration of self-regulatory learning practices in interaction with LLMs. Together, these findings suggest that LLMs can be understood not merely as productivity tools but as multifaceted partners involved in the technical, ethical, and self-regulatory dimensions of learning in CS education. The study offers theoretical and practical implications for designing AI-enabled curricula that cultivate responsible, reflective, and critically engaged use of LLMs.

1. Introduction

Large language models (LLMs) are rapidly transforming Computer Science (CS) education by reshaping how students learn, develop, and reason about computational systems (Azoulay et al., 2025; Raihan et al., 2024). In CS courses, students increasingly rely on LLM-powered tools for debugging, code generation, architectural planning, and conceptual clarification (Khendek et al., 2025; Lyu et al., 2024). Early research highlights that using LLMs in CS courses presents both opportunities and risks, with improvements in productivity and understanding, alongside concerns regarding accuracy, bias, originality, and over-reliance (Jost et al., 2024; Zdravkova & Ilijoski, 2025; Pereira & Mello, 2025; Bernabei et al., 2023). Yet, despite growing interest, several gaps remain. While LLMs are well-studied in introductory programming contexts, their role in advanced, AI-focused project-based learning (PBL) courses remains underexplored (Cambaz & Zhang, 2024; Denny et al., 2024). Moreover, the impact of LLMs on metacognitive skills, such as prompt engineering and self-regulation, remains under-analyzed (Kazemitabaar et al., 2023), and ethical self-regulation in authentic, project-based contexts requires further investigation (Nguyen et al., 2023; Slimi & Carballido, 2023). We still know little about how undergraduate students actually engage with LLMs during authentic, open-ended project work, where technical decisions, ethical considerations, and learning processes unfold simultaneously.

Existing studies typically examine either the technical contributions of LLMs (e.g., code assistance, debugging, workflow acceleration) (Pereira & Mello, 2025) or the ethical challenges they raise (e.g., transparency, authorship, academic integrity) (Azoulay et al., 2025; Zhou et al., 2024; Perkins, 2023; Kasneci et al., 2023). Much less is known about how these dimensions intersect in real-world development practices or how students navigate them in situ while building functioning AI applications. Moreover, emerging discussions about AI literacy highlight the need to understand not only what students produce with LLMs but also how they reason, regulate, and learn with them (Knoth et al., 2024; Cain, 2023; Lee & Palmer, 2025). Yet existing studies often examine these dimensions in isolation, focusing on either technical affordances or ethical concerns, with limited attention to how they intersect within authentic learning practices.

This study addresses these gaps by investigating how CS undergraduates engage with LLMs as learning partners in a project-based AI course. Specifically, we examine how students describe the role of LLMs in (1) technical development processes, (2) ethical and critical reasoning, and (3) their self-regulatory learning practices and perceived autonomy. By conceptualizing LLMs as multifaceted partners rather than mere assistive tools, this study offers new insights into the evolving role of generative AI in CS education and provides implications for designing responsible, reflective, and empowering AI-enabled learning environments. Drawing on these three dimensions, the study further aims to develop a conceptual framework that captures the interrelated technical, ethical-reflective, and agentic dimensions of students’ engagement with LLMs in CS education.

2. Theoretical Background

2.1. Large Language Models in Computer Science Education

The adoption of LLMs in CS education has transformed how students learn programming, engage with complex concepts, and solve problems. Denny et al. (2024) argue that generative AI tools such as Codex and GPT-4 provide immediate, context-aware feedback, helping students approach topics such as algorithms and machine learning with greater confidence. Similarly, Finnie-Ansley et al. (2022) found that Codex generates functional code that outperforms most students in introductory programming courses. Cambaz and Zhang (2024) highlight how AI-driven code generators automate repetitive tasks, enhancing productivity and enabling students to focus on design and logic. Kasneci et al. (2023) further emphasize the democratizing potential of LLMs in CS education by offering tailored explanations. However, LLM integration also raises challenges. Kazemitabaar et al. (2023) stress the need for “prompt literacy,” a skill reflected in this study’s theme of progressive prompting. Denny et al. (2024) and Becker et al. (2023) caution against over-reliance, especially among novices. Structured pedagogies such as PBL can mitigate these risks by fostering reflective engagement (Kong et al., 2024; Belkina et al., 2025).

Finally, institutional policy plays a key role. Spivakovsky et al. (2023) call for clear guidelines to ensure that LLM use promotes deep learning. Michel-Villarreal et al. (2023) note that although ChatGPT supports exploration of advanced topics, instructor guidance remains essential. These findings underscore LLMs’ dual role as enablers and disruptors.

2.2. Project-Based Learning as a Context for Self-Regulated Learning with LLMs

PBL is a foundational constructivist pedagogy that promotes learner autonomy through authentic, problem-driven inquiry (Blumenfeld et al., 1991; Thomas, 2000). In the course examined in this study, students worked in project teams to design and implement complex technological systems, requiring them to set goals, identify resources, select strategies, monitor their progress, and evaluate their work. These processes are conceptualized here through the lens of self-regulated learning (SRL), which emphasizes learners’ active regulation of cognition, motivation, and behavior in pursuit of learning goals (Zimmerman, 2000). In this study, autonomy refers more specifically to students’ perceived control over the direction and pace of their learning, whereas agency refers to their intentional engagement with available choices, resources, and forms of support (Bandura, 2001). Thus, SRL serves as the primary process-oriented construct, while autonomy and agency describe related aspects of how students experience and enact control within the learning environment.

In this context, LLMs support the design and implementation of complex systems by guiding planning, enabling technical prototyping, and aiding debugging (Bilgram & Laarmann, 2023). In educational settings, this enables students to progress from conceptualization to working prototypes in open-ended, project-based courses (Shaer et al., 2024). Kim et al. (2022) emphasize that AI tools reinforce autonomy and problem-solving in these environments.

Structured reflection logs, used in this study as a core pedagogical tool, were designed to support metacognitive awareness and critical engagement with AI-generated content. Xu et al. (2025) found that college students using GenAI tools without metacognitive support showed declines across multiple dimensions of SRL, whereas those who received structured metacognitive prompts demonstrated improvements in task strategy and self-evaluation, underscoring the importance of such support in GenAI-supported learning environments. Belkina et al. (2025) further support this perspective, as their systematic review found that GenAI implementations yielding the strongest learning outcomes incorporated critical discussion and reflective activities rather than passive content delivery.

When used passively, LLMs may undermine self-regulated learning by reducing opportunities for independent reasoning and learner initiative (Wood & Moss, 2024). Hamza et al. (2024) similarly caution that educational benefits depend on active and intentional engagement with AI tools. In the course examined here, reflection was embedded as a required component in order to encourage students to evaluate their use of LLMs rather than rely on them uncritically. How students navigate the tension between using AI support and maintaining control over their own learning is therefore central to the present study. Building on Calvo et al. (2020), autonomy is understood here as students’ perceived capacity to act in accordance with their own goals and values while interacting with AI systems. In this sense, LLM use may support or constrain learner agency depending on whether students use AI to extend their own reasoning or allow it to substitute for that reasoning.

2.3. Technical Applications of LLMs in Project Development

LLMs have become essential tools for technical tasks in CS education, including code generation, debugging, API integration, and system design. Finnie-Ansley et al. (2022) demonstrated that Codex produces functional solutions that outperform most students in introductory programming courses. Cambaz and Zhang (2024) similarly noted that LLMs provide starter templates and exemplar solutions. LLMs also improved task performance more broadly: Fan et al. (2025) found that AI-assisted programming significantly improved performance outcomes and reduced programming anxiety compared with individual work, while Kazemitabaar et al. (2023) showed that novices using AI code generators encountered fewer syntax errors and completed tasks more efficiently.

At the system level, students used LLMs to integrate APIs and plan workflows, capabilities consistent with Hamza et al. (2024), who found that professional software engineers used ChatGPT effectively for code generation and for structuring workflows during collaborative development tasks. Cross-validation across models, as practiced by students, supports Vaithilingam et al.’s (2022) claim that evaluating AI output is key to usability.

Effective LLM use depends on prompt engineering. Liang et al. (2024) found that providing clear, explicit explanations was the most commonly cited strategy among developers for obtaining useful AI output, while Denny et al. (2024) emphasize that prompt decomposition and precise task specification are emerging skills that educators should explicitly teach. Taken together, these studies demonstrate substantial technical affordances of LLMs, but much of the existing evidence concerns discrete programming tasks or professional development workflows. Less is known about how students integrate these affordances across the extended, open-ended development process of an authentic project-based course.

2.4. Ethical Considerations and Critical AI Literacy

The use of LLMs in CS education raises significant ethical challenges, including transparency, originality, bias, and over-reliance. Zohny et al. (2023) argue that generative AI introduces ethical dilemmas, such as distinguishing human from AI-generated work. Slimi and Carballido (2023) analyzed seven global AI ethics policies and highlighted transparency as a cornerstone of ethical LLM use in education.

Originality is a pressing concern. Doshi and Hauser (2024) found that while GenAI can enhance individual creativity, it also leads to greater homogeneity across outputs, potentially narrowing the range of novel solutions students produce in innovation tasks.

Bias and misinformation are critical issues. Capraro et al. (2024) note that generative AI can perpetuate socioeconomic inequalities by producing biased outputs that reflect cultural or demographic assumptions, and that students must evaluate LLM responses critically. Kasneci et al. (2023) highlight the risk of misinformation, as LLMs can generate inaccurate or biased content, necessitating cross-validation with other sources to ensure reliability. Nguyen et al. (2023) advocate addressing biases through critical evaluation. Long and Magerko (2020) define critical AI literacy as the ability to evaluate and responsibly use AI outputs.

Sharples (2023) proposes an ethical framework for social generative AI, emphasizing reflective practices to enhance student agency. Taken together, this literature establishes transparency, originality, bias, misinformation, and over-reliance as central concerns in educational LLM use. However, these issues are often examined separately from students’ technical practices. Less is known about how ethical judgment develops within the same project activities in which students use LLMs to generate code, troubleshoot systems, compare tools, and make design decisions.

3. Materials and Methods

3.1. Research Questions

The following three research questions guide this study:

RQ1: How do undergraduate CS students engage with LLMs as technical learning partners while developing AI projects?

RQ2: How do students negotiate the ethical, critical, and responsible use of LLMs within authentic project workflows?

RQ3: How do students describe perceived changes in their self-regulatory learning practices and learning autonomy when working with LLMs?

3.2. Research Setting and Participants

The study took place in an AI Applications course for 3rd- and 4th-year undergraduate students in a CS program. The course met for two lecture hours and one laboratory hour per week (three academic hours). It followed a PBL model that emphasized student autonomy, creativity, and inquiry. Students worked in teams to develop an AI-based application, research prototype, or software tool.

Participants included 29 undergraduate students enrolled in the course. They had 2–3 years of programming experience but limited hands-on experience with AI or ML tools. Of 11 project teams, 4 worked in pairs, and 7 worked in groups of three. The cohort comprised 15 female and 14 male students. Female participants were younger on average (M = 22 years) than male participants (M = 25 years). Two instructors co-taught the course and collaborated on course design and student supervision. Students worked on 11 diverse AI application projects, ranging from image recognition systems (e.g., Flowerfy, TreeTails) and natural language processing tools (e.g., FactCheckAI) to assistive and accessibility applications (e.g., SeniorSafe_Fall_Detection, Interactive Kids App) and recommendation systems (e.g., AIJobs, Nostalgify).

Following a formal request, the institutional ethics committee determined that formal ethical approval was not required for the retrospective analysis of the anonymized course materials. The materials later analyzed for research were produced as part of routine course activities. All identifying information was removed prior to analysis.

The AI Applications course provided a scaffolded yet flexible PBL environment in which students developed AI-based prototypes throughout the semester. The instructors primarily served as mentors, offering feedback at scheduled milestones. Students independently identified problems, proposed solutions, and implemented working systems using AI tools.

During the first four weeks, the course featured instructor-led lectures introducing key AI topics, including computer vision, natural language processing, and speech recognition, along with relevant tools such as pre-trained models and open-source libraries. This initial phase aimed to expose students to the breadth of AI applications and prepare them for independent project work. After this introduction, students submitted project proposals that required instructor approval before development could begin. The remainder of the course provided no formal instruction in ML theory or coding practices. Instead, students were expected to acquire the necessary knowledge through online resources, documentation, and AI tools, including LLMs. Advisory meetings were available upon request, enabling flexible, student-driven mentorship. Peer presentations and class-wide discussions served as collaborative checkpoints, fostering knowledge exchange and accountability. Students presented their progress twice during the semester and delivered a final project presentation at the end of the semester. Projects addressed real-world problems and leveraged AI technologies in creative ways. Examples include Flowerfy, an application that identifies flowers from images or smartphone camera input, and Nostalgify, which detects a user’s age and gender via camera input and plays era-appropriate nostalgic music. These projects exemplify the course’s emphasis on autonomy, creativity, and ethical engagement in AI development.

Although the college lacked a formal policy on generative AI, the instructor explicitly permitted its use. Students were required to document each use by appending complete transcripts of their LLM interactions, including the exact prompts submitted and responses received, along with critical commentary explaining how they evaluated and acted on each response. This requirement ensured that reflective accounts were grounded in verifiable interaction data rather than retrospective reconstruction. Submissions that did not meet minimum documentation standards were returned for revision. This process aimed to promote transparency, critical engagement, and ethical awareness in their learning while providing the research team with a rich, auditable record of student–LLM interactions. This structured documentation requirement may have influenced how students engaged with and reported their use of LLMs.

3.3. Data Sources

To examine how students used LLMs during project work, three complementary data sources were analyzed. All were produced within the same course and under structured pedagogical guidelines.

• Project Reports: Students submitted written reports detailing project goals, development steps, technologies used, challenges faced, and reflections. These reports often described LLM interactions. We analyzed 29 individual project reports, one from each student. Students received written guidelines for report preparation. Those that did not meet minimum documentation standards were returned for revision.
• Final Presentations: Teams presented their completed projects at the end of the course. One final presentation from each of the 11 project teams was transcribed and analyzed. The presentations provide evidence of LLM use, problem-solving strategies, and ethical considerations.
• Reflection Logs: Each project team maintained a shared reflection log throughout the course, documenting LLM interactions and reflections. We analyzed 11 team-level reflection logs. Students received structured guidelines for journal entries, which required documenting LLM prompts and responses, the context of use, and reflective commentary on effectiveness and limitations. Journal entries described how teams used LLMs for tasks such as code generation, debugging, conceptual learning, and navigating ethical issues.

3.4. Data Analysis

A thematic analysis was conducted using Braun and Clarke’s (2006) six-phase method. In Phase 1 (Familiarization), the course instructors read and reread all 29 project reports, 11 team reflection logs, and transcripts of 11 final presentations as part of an ongoing pedagogical review to inform future course design. During this process, the team generated initial notes on recurring patterns in students’ LLM use. In Phase 2 (Generating Initial Codes), the instructional team applied an open, inductive coding approach to generate descriptive labels for meaningful segments related to LLM use. A second researcher, who had no involvement in teaching the course, independently read the full dataset and reviewed the proposed codes for consistency, coherence, and grounding in the data. Following this review, several codes were excluded because they extended beyond the scope of the present study. The codes retained for the analysis reported in this article were not substantively revised. In Phase 3 (Searching for Themes), the initial codes were grouped into candidate themes that reflected patterns across the dataset. In Phase 4 (Reviewing Themes), themes were refined by checking them against the full dataset; overlapping or insufficiently supported themes were merged or discarded. In Phase 5 (Defining and Naming Themes), final themes were clearly defined and named, each anchored in illustrative student quotations. In Phase 6 (Producing the Report), the analysis was written up with representative excerpts. The second researcher supported the credibility of the analysis by reviewing the full dataset and proposed coding framework, resulting in the exclusion of several codes outside the scope of the present study. After completing thematic analysis across all three research questions, the emergent themes were synthesized into a conceptual framework that captures the interrelationships among the identified dimensions of student–LLM interaction. The model thus represents an inductive theoretical contribution grounded in the data, rather than a framework imposed prior to analysis. Although this study is primarily qualitative, descriptive frequency counts were calculated from the 29 individual project reports. For each theme, a report was counted once if the theme appeared at least once, regardless of the number of occurrences within that report. These counts are presented only to indicate how commonly each theme appeared across the reports and are not intended to represent the conceptual importance of a theme.

3.5. Trustworthiness

To enhance the credibility of the analysis, the study drew on multiple course-based data sources. The study compared patterns identified in individual reports with evidence from team reflection logs and final presentations to examine consistency across sources. To minimize misinterpretation of student documentation, the instructional team engaged in ongoing clarification dialogues with project teams throughout the course. During scheduled progress presentations, instructors systematically questioned students about their documented LLM interactions, cross-referencing their oral responses with the submitted transcripts and reflective commentary. These exchanges clarified the meaning and context of the documented interactions rather than serving as a formal member-checking procedure. In most cases, students’ oral accounts were consistent with their written documentation, strengthening confidence in the authenticity of the reflective data. Transferability was supported through detailed, contextual descriptions of the course structure and pedagogy. This “thick description” enables readers to assess how the findings may apply to similar educational contexts (Creswell & Poth, 2018, pp. 260–263).

3.6. Researcher Positionality

One of the authors served as an instructor in the course under study, raising concerns about the researcher’s positionality. To address this, several structural safeguards were in place. First, the course was co-taught by multiple instructors, and course design, supervision, and grading responsibilities were shared, limiting the extent to which any single instructor-researcher shaped students’ documented experiences. Second, a second researcher who had no involvement in teaching the course independently reviewed the full dataset and the proposed coding framework, providing an additional analytical perspective. Third, students were not informed that their course artifacts would later serve as research data, consistent with the institutional ethics committee’s determination that the retrospective analysis of anonymized course documentation did not require active participant disclosure. Importantly, the requirement to document LLM use, including prompts, responses, and reflective commentary, was a pedagogical component of the course established prior to and independently of the research design. Students, therefore, produced these artifacts as part of their academic obligations, not in response to research participation. While the graded nature of these submissions may have introduced some social desirability into students’ self-presentation, this limitation is acknowledged and discussed further in Section 6.

4. Results

This section presents findings for the three research questions. The results are organized thematically and grounded in student documentation, project artifacts, reflection logs, and presentation transcripts. Each subsection addresses one of the guiding research questions.

4.1. LLMs as Technical Learning Partners in AI Project Development

Analysis of reflection logs, project documentation, and presentation transcripts revealed ten key themes regarding students’ technical engagement with LLMs, grouped into two interrelated dimensions: learning-oriented technical use, in which LLMs were used to support students’ inquiry and conceptual development, and task-oriented technical use, in which LLMs served as direct tools for implementation and system development. Together, these dimensions reflect LLMs’ dual role as both cognitive scaffolds and practical development partners.

Learning-Oriented Technical Use as presented in Table 1:

Table 1. Technical engagement with LLMs: learning-oriented.

Theme	Occurrences	Example Quote Excerpt
Just-in-Time Conceptual Clarification	18/29 student reports (62%)	“Gemini explained transfer learning… more than 3 YouTube videos” (TreeTails project)
Exploration of New AI Libraries	14/29 student reports (48%)	“Copilot autocompleted using speech_recognition… didn’t know this library existed” (Nostalgify project)
Progressive Prompting	11/29 student reports (38%)	“We asked ‘How does k-means work?’ then ‘How to choose clusters?’… each answer built on last” (Flowerfy project)
Tool Comparison & Decision-Making	9/29 student reports (31%)	“’Which better for emotion detection: OpenCV or Azure API?’… chose OpenCV for offline” (Interactive Kids App project)
Multiple LLM Cross-Validation	6/29 student reports (21%)	“ChatGPT said use TfidfVectorizer in Keras… Gemini explained why that doesn’t work” (FactCheckAI project)

Theme 1: Just-in-Time Conceptual Clarification

Students commonly use LLMs to clarify concepts encountered in project work, especially those not covered in class. Rather than relying on textbooks or formal instruction, they turn to LLMs for immediate, context-specific explanations tailored to their problems.

This pattern is consistent with students incorporating LLMs into their self-regulatory learning practices. Students described the immediacy of LLM responses as helping them continue working without waiting for external assistance.

For example, a student from the TreeTails project reported:

“I asked Gemini to explain transfer learning with ResNet50. The answer included code snippets and a diagram explaining the model architecture. It helped more than the three YouTube videos I had watched.”

This example illustrates how a student used an LLM response that combined explanation, code, and a visual representation to clarify a concept that had remained difficult to understand through previously consulted resources.

Another student from the FactCheckAI team described:

“ChatGPT explained cosine similarity in simple terms and how it works in vector embeddings. That was the turning point for understanding how to measure textual similarity in our project.”

Here, the LLM served as a personalized tutor, providing a simplified yet accurate explanation that enabled the student to implement a key technical feature of their project.

Theme 2: Exploration of New AI Libraries and Frameworks

LLMs also played a critical role in introducing students to libraries, APIs, and tools they had not previously used. These interactions went beyond explanation; they served as personalized discovery mechanisms that tailored suggestions to students’ needs and project constraints.

For instance, in the Nostalgify project, a student recounted how GitHub Copilot (2024) introduced an unfamiliar Python library:

“I was typing a comment about converting audio to text in Hebrew, and Copilot autocompleted it using the speech_recognition package and even suggested error handling. I didn’t know this library existed.”

This example shows that LLMs can surface useful tools at the point of need, guiding students toward implementation paths aligned with their goals, even when they lack domain-specific vocabulary.

Similarly, a student working on a chatbot for children used BingAI to explore model selection:

“I asked Bing Chat what the best open-source model is for a Hebrew-language chatbot. It listed several options and gave links, so I read about AlephBERT and decided to test it.”

This interaction highlights a key cognitive function of LLMs: helping students identify solution spaces and filter choices based on relevance and essential skills in real-world CS.

Theme 3: Progressive Prompting for Deeper Understanding

A hallmark of more advanced use was the iterative refinement of prompts to achieve deeper or more context-specific understanding. Students reported learning to improve prompt quality to get better answers, which is a form of emergent prompt literacy or prompt engineering.

This recursive engagement marks a shift from passive consumption to active learning dialogue, in which students use LLMs to construct knowledge through elaboration and questioning.

For example, a student working on a mental health assistant app explained:

“I asked ChatGPT what dropout does in neural networks. It answered, but I didn’t understand when it’s useful. So I asked again: ‘When would dropout help prevent overfitting in a small dataset with 500 samples?’ That helped a lot.”

This demonstrates how LLMs support contextualized learning, in which students are dissatisfied with general answers and use clarifying prompts to tailor their understanding to their specific use case.

A team member from Flowerfy described a multi-stage interaction:

“First, we asked, ‘How does k-means clustering work?’ Then, ‘Can you show me how to use k-means with color palettes in Python?’ Then, ‘How do I choose the right number of clusters for flower color classification?’ Each answer built on the last.”

Here, the LLM mimics a scaffolded learning progression, where each prompt deepens the conceptual and practical grasp of a complex idea. This iterative behavior also suggests increasing control over how students used LLMs to support their learning.

Theme 4: Tool Comparison and Technical Decision-Making

Beyond explanation and discovery, LLMs helped students compare technical tools and methods. These comparisons often informed project architecture decisions and reflected students’ consideration of technical trade-offs and alternative solutions.

Students used LLMs not only to obtain answers but also to weigh trade-offs among competing strategies.

For example, in the Interactive Kids App, students asked ChatGPT to evaluate two options for facial expression recognition:

“We asked ChatGPT: ‘Which is better for real-time child emotion detection: OpenCV with Haar cascades or Azure Emotion API?’ It explained accuracy, latency, and privacy implications. We chose OpenCV because we wanted offline functionality.”

This shows how LLMs served as decision-support systems, enabling students to align technical decisions with project values such as privacy or accessibility.

Another group working on AIJobs used Gemini for mathematical comparison:

“We asked Gemini: ‘Compare cosine similarity with Euclidean distance for embedding vectors in job-matching tasks.’ It helped us understand why cosine was better in high-dimensional space.”

This quote shows how LLMs helped students move beyond surface-level implementation toward principled justification, an essential part of advanced SE practice.

Theme 5: Use of Multiple LLMs for Cross-Validation

A more sophisticated subset of students used multiple LLMs to validate answers and reduce reliance on any single model, indicating the emergence of critical AI literacy.

Students became aware of hallucinations, inconsistencies, or overgeneralizations in LLM responses and developed strategies to evaluate outputs for accuracy and reliability.

A student from FactCheckAI noted:

“When ChatGPT told us to use TfidfVectorizer inside Keras, we felt something was wrong. Gemini explained why that doesn’t work inside the model pipeline. It saved us a lot of time.”

This example reveals a vital learning outcome: students began to treat LLMs not as authorities but as tools that require verification, a valuable critical disposition.

Similarly, a student from the SeniorSafe_Fall_Detection group reported:

“We tested the same code explanation in ChatGPT and Copilot. One gave us buggy logic, the other gave correct pseudocode. We learned to compare before trusting.”

This behavior reflects a more critical and flexible approach to handling AI-generated information.

Students leveraged LLMs extensively across conceptual, technical, and strategic dimensions of their project work. The five emergent themes: on-demand explanation, tool discovery, iterative prompting, comparative reasoning, and cross-validation highlight that LLMs function not only as knowledge sources but also as interactive learning partners.

The diversity of tools used (e.g., ChatGPT, Gemini, Copilot, BingAI) and the range of tasks addressed (e.g., theoretical clarification, library selection, decision-making) demonstrate that students used LLMs in nuanced, purposeful ways. This reflects more sophisticated engagement with AI tools, marked by decision-making and critical reflection.

Task-Oriented Technical Use as presented in Table 2:

Table 2. Technical engagement with LLMs: task-oriented.

Code Generation and Starter Templates	17/29 student reports (59%)	“We asked ChatGPT to create a basic Flask API… gave full working example” (AIJobs project)
Debugging and Error Resolution	15/29 student reports (52%)	“Gemini explained input shape and helped reformat time series data” (SeniorSafe_Fall_Detection project)
Code Refactoring and Optimization	10/29 student reports (34%)	“ChatGPT refactored into smaller functions. Easier to debug” (FactCheckAI project)
API Integration and Toolchain Assembly	13/29 student reports (45%)	“ChatGPT helped connect speech recognition to Pygame storytelling” (Interactive Kids App project)
Designing System Architectures	8/29 student reports (28%)	“ChatGPT drew architecture with arrows. Became our blueprint” (AIJobs project)

Theme 6: Code Generation and Starter Templates

Students frequently used LLMs to scaffold initial script versions or generate code templates for new features. This usage pattern was most common in the early stages of project development, when students sought to overcome “blank screen” paralysis.

This behavior demonstrates how LLMs reduce entry barriers by converting abstract intentions into structured, runnable code, allowing students to focus on adapting and extending rather than starting from scratch.

For example, in the AIJobs project, one student wrote:

“We asked ChatGPT to create a basic Flask API to expose our model. It gave us a full working example with endpoints, which we then customized.”

This reflects how LLMs assist with boilerplate scaffolding, which is often time-consuming to write manually, so students can invest their energy in their project’s unique logic.

In the Flowerfy project, the team used Copilot to generate a clustering script:

“We were trying to segment flower images by color. Copilot implemented k-means with elbow method support. We didn’t have to build it from scratch.”

Here, the LLM facilitated the technical implementation of an unsupervised learning technique by embedding it directly into the development workflow, shortening the trial-and-error phase.

Theme 7: Debugging and Error Resolution

LLMs were also instrumental in resolving bugs and syntax issues. Students routinely copied error messages into LLMs and received explanations along with suggested fixes, sometimes more clearly and effectively than through IDE suggestions or online forums.

This shows how LLMs helped students interpret errors and identify context-specific repair strategies.

In the SeniorSafe_Fall_Detection project, a student reported:

“Our model training was crashing due to a shape mismatch in the LSTM layer. Gemini explained the expected input shape and helped us reformat our time series data. It was clearer than Stack Overflow.”

The key insight here is the LLM’s ability to synthesize documentation, code context, and specific troubleshooting tips, providing just-in-time debugging support that would be hard to find in a single source.

Another student in the Interactive Kids App project wrote:

“When we connected OpenCV to our GUI, the video feed kept freezing. ChatGPT helped us restructure the loop to run asynchronously. It saved us a full day.”

This indicates that LLMs served not only as syntax correctors but also as design-level problem solvers, guiding students in realigning their logic flows with best practices.

Theme 8: Code Refactoring and Optimization

Several students reported using LLMs to improve the readability, performance, or modularity of code they had already written, either independently or borrowed from tutorials.

This behavior shows that students used LLMs not only to make their code work but also to improve its readability, efficiency, and maintainability.

In the FactCheckAI project, one student shared:

“We had a long function that mixed text preprocessing, model inference, and output formatting. We asked ChatGPT to refactor it into smaller, cleaner functions. The result was much easier to debug.”

This shows how LLMs helped reinforce principles of clean code and modular design, even for students without formal software architecture training.

Similarly, in the Nostalgify team:

“Copilot suggested replacing our nested loops with a NumPy vectorized operation. We saw a huge speedup.”

This quote illustrates how LLMs supported attention to performance as well as correctness in their development work.

Theme 9: API Integration and Toolchain Assembly

Students used LLMs to assemble workflows that involved chaining together multiple tools, APIs, or data formats. This included integrating AI services (e.g., the OpenAI API and Google Cloud Vision), databases (e.g., Firebase), and front-end interfaces.

This theme shows how LLMs helped students integrate multiple tools, APIs, and system components into their project workflows.

In the Interactive Kids App, one student explained:

“We asked ChatGPT how to connect our speech recognition output to a visual storytelling component using Pygame. It helped us pass the transcribed text into our event manager.”

This example shows how LLMs supported integrating heterogeneous components across different libraries or technologies.

The SeniorSafe_Fall_Detection team reported:

“We needed to store fall detection alerts in Firebase with timestamps. Gemini showed us the exact Python code for structuring the data and writing it to the database.”

This demonstrates how LLMs enable students to prototype complete systems rather than just isolated components.

Theme 10: Designing System Architectures and Workflows

Some students went beyond coding to seek help in architectural planning, asking LLMs to design the structure of their applications, propose component interactions, or suggest development roadmaps.

This advanced use of LLMs suggests their utility as technical planning assistants, helping students scaffold both their logic and their processes.

In the AIJobs project, a student reflected:

“We weren’t sure how to combine the recommendation model, the user interface, and the job scraping script. ChatGPT drew us a simple architecture with arrows and dependencies. It became our project blueprint.”

Here, the LLM supported visual thinking and systems planning, enabling students to reason about component relationships before writing code.

A student from FactCheckAI described:

“We gave Gemini our idea and asked what modules we’d need. It suggested a pipeline with scraping, NLP, credibility scoring, and a Flask interface. That shaped our entire dev plan.”

This points to the role of LLMs not just in execution, but in strategic development, acting as partners in ideation and project structuring.

Across all projects, students used LLMs for a broad range of technical tasks, including generating boilerplate code, debugging, optimizing, integrating APIs, and designing system architectures. These interactions were not limited to surface-level usage; instead, students engaged LLMs in recursive problem-solving, refining prompts, and experimenting with suggestions until they arrived at usable solutions.

LLMs supported different types of tasks depending on project complexity, ranging from code generation and explanation to architectural planning and development strategy. Students’ use of these tools reflected flexible technical reasoning and adaptive collaboration with LLMs across different project tasks.

These findings indicate that LLMs were used not merely as coding assistants but as multipurpose scaffolds across multiple stages of project-based software development.

Across all projects, students engaged with LLMs across a broad, interconnected spectrum of technical activities. Learning-oriented use helped students build conceptual understanding, discover new tools, and develop prompt literacy through iterative inquiry. Task-oriented use translated these capabilities into concrete development outcomes, including functional code, debugged systems, integrated APIs, and planned architectures. These two dimensions were not sequential but recursive: students moved fluidly between conceptual clarification and implementation, using LLMs as both thinking partners and coding collaborators. This pattern suggests that, when embedded in structured PBL environments, LLMs support the full lifecycle of technical development, from initial understanding to system deployment.

4.2. Ethical and Critical LLM Use in Practice

Analysis of student documentation and reflective logs yielded five major themes related to ethical reflection and self-regulation in the use of LLMs, as presented in Table 3:

Table 3. Ethical-reflective partnership.

Theme	Occurrences	Example Quote Excerpt
Transparency in LLM Use	22/29 student reports (76%)	“Documented every ChatGPT use. Important to be honest” (TreeTails project)
Concerns about Originality	18/29 student reports (62%)	“Didn’t want app to feel like anyone could generate with ChatGPT” (Nostalgify project)
Awareness of Bias/Misinformation	15/29 student reports (52%)	“ChatGPT added details that weren’t there. Cross-checked everything” (FactCheckAI project)
Over-Reliance and Dependency	12/29 student reports (41%)	“Realized we weren’t thinking on our own and stopped” (Interactive Kids App project)
Ethical Self-Regulation	10/29 student reports (34%)	“Decided not to use LLMs for model training. Wanted to understand every line” (TreeTails project)

Theme 1: Transparency in Use of LLMs

The majority of students emphasized the importance of transparency about their use of LLMs in both project documentation and presentations. This theme reflects a conscious effort to distinguish between student-authored work and AI-assisted components.

For instance, in the TreeTails project, the team wrote:

“We documented every place we used ChatGPT in our logbook. It felt important to be honest, even if we only used it for suggestions.”

This quote reflects students’ explicit attention to transparency in a course that required documentation of LLM use. The student’s emphasis on honesty suggests that, in this case, the requirement was accompanied by a concern for responsible disclosure.

Similarly, in the FactCheckAI project, a student stated:

“In the final presentation, we explained that part of the pipeline was generated with help from Gemini. We didn’t want to take credit for something we didn’t fully understand.”

This reflects a mature understanding of the distinction between authorship and assistance, which is a critical ethical line in the age of generative AI.

Theme 2: Concerns about Originality and Creativity

Students expressed concern that excessive reliance on LLMs might reduce the originality of their work or lead to overly generic solutions. This theme appeared especially in groups working on user experience design, where creative input was critical.

For example, a member of the Flowerfy team wrote:

“We tried asking Copilot for ideas for color clustering, but the suggestions felt repetitive. We went back to brainstorming ourselves.”

Here, the LLM’s perceived limitations became a prompt for renewed human creativity. This behavior suggests that students were not merely consuming LLM content but evaluating its contribution critically.

The Nostalgify group also reflected on this theme:

“We didn’t want the app to feel like something anyone could generate with ChatGPT. That’s why we wrote the entire user interface logic ourselves.”

This reflects a protective stance on creative ownership, with students drawing ethical boundaries around the role of automation in tasks they considered inherently human-driven.

Theme 3: Awareness of Bias or Misinformation

More than half of the students, particularly those working on socially sensitive topics, demonstrated awareness of the limitations of LLMs, including issues of bias, hallucinated information, and a lack of cultural sensitivity.

In the FactCheckAI project, one student observed:

“We asked ChatGPT to summarize articles, and it sometimes added details that weren’t there. We had to cross-check everything.”

This highlights an awareness of epistemic responsibility. Students recognized that LLMs can be unreliable and adjusted their practices accordingly.

A student from SeniorSafe_Fall_Detection expressed:

“We tried using ChatGPT to suggest alert messages for elderly users, but some responses felt condescending. We rewrote them to be more respectful.”

Here, students identified subtle forms of algorithmic bias and took corrective action. Algorithmic bias refers to unintended patterns or stereotypes that models may generate because of imbalances or distortions in their training data. This example suggests that LLMs may prompt not only ethical awareness but also ethical learning moments embedded in the design process.

Theme 4: Over-Reliance and Dependency

A recurring concern was the fear of becoming overly dependent on LLMs, especially for technical implementation. Students reflected on the tension between using LLMs to accelerate work and the potential erosion of problem-solving skills.

In the Interactive Kids App, one student wrote:

“At one point, we found ourselves checking everything with ChatGPT. We realized we weren’t thinking on our own and stopped.”

This quote illustrates a key self-regulatory behavior: recognizing when tool use becomes a crutch and making a conscious effort to restore balance.

Similarly, a student from the AIJobs team stated:

“We didn’t want to end up as button-pushers. That’s why we did code reviews without LLMs to test what we understood.”

This quote illustrates a conscious effort to ensure genuine understanding rather than passive reliance on AI-generated output. By choosing to conduct code reviews independently, the student demonstrated a desire to retain control over their learning and actively verify their comprehension. This behavior reflects an emerging sense of responsibility and autonomy, as well as a commitment to developing authentic technical mastery rather than merely completing tasks with the help of LLMs.

Theme 5: Ethical Decision-Making and Self-Regulation

Beyond concerns, a third of students made proactive ethical choices about when and how to use LLMs. These decisions reflect not only awareness, but agency and self-regulation.

In TreeTails, the team wrote:

“We decided not to use any LLM for our model training pipeline. We wanted to understand every line we wrote.”

This decision shows how students used personal ethical reasoning to guide technical choices, even when LLM assistance was available.

The Nostalgify team expressed a similar mindset:

“We asked Gemini for help with architecture, but decided to design the flow ourselves. It was important for us to struggle through it.”

Rather than rejecting LLMs outright, students made contextual decisions about when AI use aligned with their learning goals and when it did not.

Students demonstrated ethical reflection and awareness in their use of LLMs throughout the course. Their considerations went beyond institutional expectations to include deeply personal judgments about integrity, responsibility, and the role of AI in their learning. The five emergent themes, transparency, concerns about originality, awareness of bias, over-reliance, and ethical decision-making, highlight the complexity of student engagement with AI tools.

Concerns about originality and creativity led students to deliberately avoid overreliance on LLM-generated content in areas where they felt personal contribution was essential. Notably, some students also identified subtle forms of algorithmic bias, such as patronizing or culturally inappropriate language, and revised or rewrote outputs accordingly. These actions reflect awareness of the ethical challenges posed by generative AI and of students’ capacity for critical AI literacy.

Students also reflected on their usage patterns and took intentional steps to avoid over-reliance. Some reported stepping away from LLMs to validate their understanding through manual review or independent problem-solving. For example, conducting code reviews without AI assistance was seen as a way to test proper comprehension and maintain control over the learning process. These behaviors indicate self-regulatory strategies and a commitment to cultivating mastery rather than superficial completion.

Overall, LLMs served not only as technical aids but also as tools that prompted reflection, negotiation, and value-driven decision-making. In the context of PBL and self-regulated learning, this environment supported nuanced and thoughtful engagement with LLMs, including reflection on technical and ethical decisions.

4.3. Reported Shifts in Learning Autonomy and Self-Regulation in LLM-Assisted Learning

Thematic analysis of student documentation and presentations identified five key themes regarding how students experienced the role of LLMs in their learning. These themes reflect reported shifts in SRL practices, including how students set goals, identify resources, and monitor their progress in LLM-assisted project work (see Table 4). Rather than indicating a direct increase in learner independence, these findings point to changes in how students engaged with and managed their learning in interaction with AI tools.

Table 4. Reported changes in self-regulatory learning practices and perceived autonomy.

Theme	Occurrences	Example Quote Excerpt
Increased Confidence	24/29 student reports (83%)	“Now I feel I can learn anything if I ask right questions” (SeniorSafe_Fall_Detection project)
Enhanced Autonomy & Motivation	20/29 student reports (69%)	“Built learning roadmap using ChatGPT as guide. Chose direction” (TreeTails project)
Supplemented Peer Collaboration	18/29 student reports (62%)	“We each asked different things then shared. Better conversations” (Flowerfy project)
Shift in Instructor Role	15/29 student reports (52%)	“With mentor talked about goals, not definitions. Meetings focused” (FactCheckAI project)
Self-Awareness of Learning Strategies	12/29 student reports (41%)	“Learned to ask better questions. Need details/examples/constraints” (SeniorSafe_Fall_Detection project)

Importantly, these accounts reflect students’ perceptions of their learning experiences and should not be taken as direct evidence of increased autonomy or reduced reliance on LLMs. In several cases, increased confidence, efficiency, and task progression coexisted with continued dependence on AI-generated support. This suggests that what students experience as greater autonomy may, in practice, reflect a reconfiguration of self-regulatory processes rather than a shift toward fully independent learning.

Theme 1: Increased Confidence in Tackling Complex Problems

Students in 24 of the 29 individual reports (83%) reported that access to LLMs gave them the confidence to approach technically demanding tasks they might have otherwise avoided. Students’ accounts suggest that access to LLM support made unfamiliar or difficult topics feel more approachable.

A student from the SeniorSafe_Fall_Detection team wrote:

“Before this project, I never thought I could work with LSTM or time-series data. ChatGPT helped me understand it bit by bit, and now I feel I can learn anything if I ask the right questions.”

This quote illustrates how the student experienced LLM support as making complex technical work more approachable.

In the FactCheckAI group, a student described:

“We got stuck on how to preprocess Hebrew text. Normally, we’d give up or skip it. But with ChatGPT, we tried several versions and found one that worked.”

This suggests that LLMs encouraged resilience in problem-solving, reducing the cost of failure and making iterative experimentation more accessible. However, while increased confidence enabled students to engage more readily with complex tasks, it does not necessarily indicate independent problem-solving, as students often continued to rely on LLM support throughout the process.

Theme 2: Enhanced Learning Autonomy and Motivation

Students consistently described LLMs as enabling self-paced exploration. Instead of waiting for external help or relying solely on classroom materials, students took the initiative to shape their own learning paths.

A participant from TreeTails noted:

“We built our own learning roadmap, first image preprocessing, then model tuning, using ChatGPT as a guide. It was like having a personal tutor, but we chose the direction.”

This illustrates active management of the learning process, in which students not only managed their own learning but also actively shaped its direction based on emerging needs. However, this pattern reflects perceived agency rather than fully independent self-regulated learning, as students’ self-direction remained closely intertwined with ongoing reliance on LLM support.

In the AIJobs project, a student shared:

“Every time we had a question, we asked it immediately. It kept us moving. I was more engaged than in any other course because I could keep learning without getting stuck.”

Here, students described LLMs as helping sustain motivation by maintaining workflow momentum, removing bottlenecks, and enabling continuous progress.

Theme 3: Supplementing Rather Than Replacing Peer Collaboration

While students used LLMs heavily for independent learning, they did not report abandoning collaboration. Instead, LLMs often complemented peer discussions, enriching them rather than replacing them.

In the Flowerfy team, a student wrote:

“We each asked different things from ChatGPT and then shared our results. It led to better group conversations because we came up with ideas.”

This suggests that LLMs served as idea catalysts, preparing students for more productive peer interaction.

A student from Nostalgify reflected:

“We disagreed about how to design the backend, so we asked Gemini and ChatGPT separately, then compared answers and discussed. It didn’t replace teamwork; it made it smarter.”

This example highlights how LLMs can diversify perspectives, enriching collaborative dialogue by providing concrete alternatives for debate and synthesis.

Theme 4: Shift in Perceived Role of the Instructor

The availability of LLMs gradually reconfigured how students viewed the instructor’s role in the learning process. Instead of relying on the instructor for every question, students turned to LLMs for micro-level issues and reserved instructor time for higher-level feedback.

In the Interactive Kids App, one student wrote:

“We asked ChatGPT all the technical stuff. With our mentor, we talked about project goals and user experience. It made our meetings more focused.”

This suggests that LLMs helped students triage their questions, delegating routine or clarifying tasks to AI and preserving instructional interactions for reflective and strategic thinking.

Another student from FactCheckAI added:

“I still needed my mentor, but not for things like ‘what is TF-IDF.’ I came to meetings with a draft or a question about design, not definitions.”

This reflects a more active role in mentor interactions, with students arriving with drafts, questions, and design issues rather than relying on instructors primarily for basic explanations.

Theme 5: Self-Awareness of Changing Learning Strategies

Several students expressed meta-awareness about how LLMs had changed the way they approached learning, becoming more intentional, iterative, and exploratory.

A student from SeniorSafe_Fall_Detection explained:

“I learned to ask better questions. At first, I asked ChatGPT something vague and got nonsense. Later, I realized I needed to give it details, examples, and constraints.”

This reflects prompt literacy; students became aware of how to extract value from the tool and used that insight to refine their learning processes.

Another student from TreeTails wrote:

“Instead of watching 10 videos, I wrote one good prompt and edited it until I understood. I didn’t expect learning like this to be so active.”

This insight highlights active engagement with LLMs and suggests that, when used critically and reflectively, these tools can support students’ perceived agency.

Students reported greater perceived autonomy, motivation, and confidence when using LLMs to tackle complex problems. Rather than replacing collaboration, LLMs complemented peer interactions, changing how students prepared for team meetings and mentor appointments. Students described using LLMs to manage the pace, depth, and focus of their learning, taking charge of both what and how they learned it.

Importantly, students were not only affected by LLMs; they also reflected on how their strategies, habits, and identities as learners evolved through their use. These accounts reflect students’ awareness of changes in their own learning strategies. The findings suggest that when embedded in PBL environments, LLMs may serve as scaffolds that support students’ reported changes in learning practices.

5. Discussion

This study examined how undergraduate CS students engaged with LLMs as learning partners across three intersecting dimensions—technical development, ethical reflection, and reported shifts in learning autonomy—within a project-based AI course. Based on the findings, we developed a theoretical framework, the Technical–Ethical–Agentic Partnership Framework, comprising three interrelated dimensions: (a) a technical dimension, involving code generation, debugging, and system design; (b) an ethical-reflective dimension, involving reflection on transparency, bias, and dependency; and (c) an agentic dimension, involving reported changes in self-regulatory learning practices, perceived autonomy, and students’ management of learning support. Drawing on qualitative analysis of 29 project reports, 11 team reflection logs, and 11 final presentation transcripts, the discussion contextualizes each dimension within the broader academic literature.

The results suggest that LLMs support aspects of SRL by providing on-demand conceptual clarification, facilitating the discovery of new tools, and supporting iterative knowledge construction through progressive prompting. These practices align with the AI literacy framework proposed by Kong et al. (2024), in which LLMs are leveraged not only as informational resources but also as cognitive scaffolds that support learners in directing their inquiry.

The just-in-time learning documented in this study aligns with the affordances described by Denny et al. (2024), who highlighted generative AI’s role in reducing cognitive load and supporting context-sensitive learning. Students’ iterative refinement of prompts mirrors what Kazemitabaar et al. (2023) termed “prompt literacy,” a metacognitive skill set increasingly central to navigating LLM-based environments. Importantly, students’ use of multiple models for cross-validation demonstrates elements of critical AI literacy, as defined by Long and Magerko (2020), in which learners treat AI outputs with skepticism and engage in epistemic evaluation.

These behaviors suggest that LLMs, when embedded within student-centered pedagogical frameworks, may support not only procedural knowledge but also critical thinking and perceived autonomy. This aligns with prior findings from Nguyen et al. (2023) and Chan (2023), who argue that effective use of AI tools requires deliberate instructional design that cultivates agency rather than passive consumption.

The findings confirm that LLMs are deeply integrated into students’ technical workflows across multiple stages of development, including code generation, debugging, optimization, API integration, and architectural planning. These results align with the conclusions of Cambaz and Zhang (2024) and Finnie-Ansley et al. (2022), who documented LLMs’ utility in scaffolding complex programming tasks.

Students did not merely use LLMs to automate tasks; they engaged them as collaborators to refine, optimize, and design software systems. This aligns with the findings of Fan et al. (2025), who found that AI-assisted pair programming enhances performance and motivation through interactive debugging. Similarly, Hamza et al. (2024) demonstrated that LLMs support complex system integration, a pattern replicated in this study’s projects, such as assembling multimodal toolchains involving speech recognition and real-time user interfaces.

However, the effectiveness of these tools appeared to depend on students’ prompt engineering skills and critical engagement. As Vaithilingam et al. (2022) and Liang et al. (2024) note, the usability of LLMs in technical contexts is closely tied to user proficiency. These findings suggest that students reported purposeful technical use of LLMs, extending beyond generic templates to purposeful architectural inquiry. The instructor’s role in facilitating this transition by embedding LLM use within a structured PBL environment supports Michel-Villarreal et al.’s (2023) assertion that generative AI requires guided integration to be pedagogically beneficial.

A key contribution of this study is the depth of ethical reflection students exhibited in their interactions with LLMs. Consistent with the literature on academic integrity and AI ethics (Ward et al., 2024; Zohny et al., 2023), students demonstrated a nuanced awareness of issues such as authorship, originality, bias, and over-reliance. Many participants documented their use of LLMs and described principled decisions about when to accept or reject AI-generated content.

These behaviors resonate with the “reflective AI literacy” described by Sharples (2023) and the call for responsible tool use advocated by Slimi and Carballido (2023). Using reflection logs as a course requirement may have contributed to students’ metacognitive awareness. As Belkina et al. (2025) note, structured reflection can transform LLMs from mere tools into ethical provocateurs, stimulating student agency and judgment in morally complex learning environments.

Notably, students reported self-regulatory behaviors aimed at avoiding dependency. This aligns with Wood and Moss’s (2024) concern that LLMs may inadvertently undermine autonomy if not critically monitored. Yet, in this study, students reported attempts to step back, reassess their habits, and assert control over their learning trajectories, reflecting deliberate ethical engagement. Importantly, these accounts reflect reported practices and perceptions, and do not necessarily indicate fully independent regulation of learning.

Students’ reflections on the impact of LLMs on their learning autonomy, collaboration, and instructional relationships suggest notable changes in how they experience and organize learning processes. LLMs were perceived not as replacements for instructors or peers, but as complementary agents that reshaped how support was accessed and used, as well as classroom dynamics. This echoes Kim et al.’s (2022) findings that AI tools can support open-ended inquiry while preserving the role of human mentorship for higher-order tasks.

The reconfiguration of instructor–student interactions, where students used LLMs for micro-level support and reserved mentors for strategic dialogue, parallels the blended support models proposed by Chan (2023) and Denny et al. (2024). LLMs, in this case, helped students triage their learning needs, fostering a more efficient and empowered relationship with instructional support.

Furthermore, the continued presence of peer collaboration, enhanced rather than displaced by LLMs, supports Kasneci et al.’s (2023) claim that generative AI can act as a catalyst for social learning. The finding that students shared and debated AI-generated suggestions during group decision-making suggests a blending of individual and collaborative learning processes.

Finally, students’ reflections of how their learning strategies changed, particularly regarding prompt refinement and task decomposition, indicate perceived shifts in how they approached learning processes. Rather than merely seeking answers, students began to see themselves as active co-constructors of knowledge in dialogue with AI systems. These findings suggest that what students experience as increased autonomy may reflect a reconfiguration of self-regulated learning processes rather than fully independent learning. This mirrors the “exploratory engagement” advocated by Sperling et al. (2024) and exemplifies the potential of LLMs to reshape learning experiences when embedded in reflective, learner-centered environments.

Several limitations should be considered. First, the study was conducted in a single project-based AI course at one institution with 29 CS students, which limits the transferability of the findings. Second, the analysis relied primarily on course-generated reports and reflections that were structured and graded; this may have amplified ethical reflection and introduced social desirability into students’ accounts. Third, because the study did not include a baseline, control group, longitudinal follow-up, or direct measures of autonomy, self-regulated learning, or learning outcomes, the reported changes should be interpreted as students’ perceptions rather than as causal effects of LLM use.

Future research should examine these findings across different institutions, student populations, and disciplines, and use longitudinal and quantitative designs to investigate whether the reported patterns are associated with measurable changes in learning outcomes, self-regulated learning, and perceived autonomy. Behavioral data, such as API interaction logs, could also complement students’ self-reported accounts.

6. Theoretical and Practical Implications

Taken together, the findings contribute to a growing understanding of LLMs not only as technical tools but as epistemic and ethical actors within educational ecosystems. The study supports the need for AI literacy frameworks (Kong et al., 2024; Long & Magerko, 2020) that integrate technical competence, critical reflection, and ethical reasoning.

Practically, the course’s emphasis on documentation, transparency, and reflection offers a replicable model for integrating LLMs into PBL contexts while explicitly foregrounding academic integrity. Institutional policies that emphasize reflective engagement rather than restriction, as advocated by Spivakovsky et al. (2023), may better align with students’ actual needs and behaviors.

Moreover, the documented benefits of LLMs in scaffolding unfamiliar topics, making complex tasks feel more approachable, and promoting students’ sense of agency suggest that AI-powered support systems may help broaden access to advanced computing education, as proposed by Kasneci et al. (2023). However, this potential appears to depend on thoughtful instructional design, support for prompt literacy, and opportunities for ethical dialogue.

Course designers should integrate explicit “prompt literacy” modules into AI-focused CS curricula, teaching students to refine queries iteratively (e.g., progressing from “Explain CNN” to “How does layer depth affect accuracy in small-image datasets?”). Courses should require students to document and reflect on their interactions with LLMs—as demonstrated in this study—to promote transparency and critical AI literacy. Assessment should consider not only the final product but also students’ documentation of prompts, verification decisions, revisions, and the rationale for accepting or rejecting AI-generated outputs. Embedding reflective journaling as a mandatory artifact, alongside structured presentation of LLM usage, provides educators with documented evidence of students’ reported learning processes and encourages ethical self-regulation. PBL structures that grant students autonomy over tool selection, project scope, and learning pace may provide favorable conditions for reflective and purposeful LLM use and should be considered in AI education course design.

LLM tool developers should consider incorporating educational affordances that support responsible use. These include (a) transparency indicators (e.g., confidence scores for generated code), (b) bias-alert features that flag potentially stereotypical or culturally inappropriate outputs, (c) interaction history exports to facilitate student documentation, and (d) prompt-scaffolding suggestions that guide novice users toward more precise and contextualized queries. Institutions and tool providers should collaborate to develop LMS integrations that automatically log and timestamp LLM interactions, reducing students’ documentation burden while preserving the reflective intent.

The Technical–Ethical–Agentic Partnership Framework proposed in this study offers a conceptual contribution to AI literacy scholarship. While existing AI literacy frameworks identify technical and ethical competencies (Long & Magerko, 2020; Kong et al., 2024), the present framework foregrounds how these dimensions intersect with students’ self-regulatory practices, perceived autonomy, and intentional management of AI support within authentic project work. AI literacy frameworks should therefore explicitly address the intersections between technical skill development, ethical deliberation, and self-regulatory practices. Institutional adoption of reflective AI policies, mandating documentation logs and ethical reflection as core course requirements, can serve as a scalable mechanism for supporting these interconnected technical, ethical-reflective, and agentic dimensions. Policymakers and curriculum developers should consult existing AI ethics guidelines (Slimi & Carballido, 2023) and contextualize them to the specific demands and cultures of CS and engineering education programs.

Importantly, the findings of this study are based on students’ reported experiences within a specific instructional context and should be interpreted accordingly. While students described increased confidence, engagement, and perceived autonomy, these outcomes reflect changes in how learning processes were experienced rather than direct evidence of fully independent self-regulated learning. Future research should further examine how such AI-mediated learning practices translate into measurable learning outcomes across diverse contexts.

7. Conclusions

This study examined how 29 undergraduate CS students engaged with LLMs while developing open-ended AI applications in a project-based course, identifying three interconnected forms of interaction: a technical partnership, an ethical-reflective partnership, and an agentic dimension reflected in reported shifts in self-regulatory learning practices and perceived autonomy. Together, these findings support conceptualizing LLMs as multifaceted learning partners rather than mere productivity tools, and motivate the proposed Technical–Ethical–Agentic Partnership Framework as a contribution to AI literacy scholarship in CS education. Future work should extend this analysis with behavioral log data and examine how these patterns generalize across institutions and disciplines.