Learning to Use Generative AI and Using It to Improve Learning: A Systems Engineering Research Seminar Case Study

Abstract

The rapid advancement of generative artificial intelligence (GenAI) has significantly impacted educational and professional practices, presenting both opportunities and challenges. This study explores the integration of GenAI into a systems engineering seminar, aiming to develop essential GenAI skills and enhance disciplinary knowledge. Two hypotheses guide this research: (H1) engaging with GenAI in research and design activities improves student proficiency in using GenAI, and (H2) engaging with GenAI in design activities related to advanced disciplinary knowledge improves their understanding and use. The study employs a case study approach combined with a survey, involving 26 graduate students in a systems engineering seminar. Students were encouraged to use GenAI tools for all tasks, including literature reviews, presentations, and a drone design challenge. Data was collected through recorded presentations and student interactions with GenAI tools. Data analysis involved systematic coding and thematic analysis of presentations, student–GenAI interactions, and survey responses, with triangulation across multiple data sources to ensure validity. The findings indicate that the students effectively learned about GenAI tools, demonstrated gradual improvements in using tools, criticized and selected among them, and even built a new GenAI tool. They demonstrated improved critical thinking and creativity, as evidenced by their ability to critically assess GenAI outputs and apply them to practical challenges like the drone design task. One student developed a custom GenAI tool by training ChatGPT-4o for specialized modeling tasks. The integration of GenAI in educational settings through self-directed learning, peer presentations, and design challenges appears to enhance learning experiences by fostering critical thinking and creativity. The evidence suggests that GenAI tools, when used with appropriate validation and critical assessment, may serve as valuable aids in developing engineering skills and addressing complex problems. Best practices in teaching about GenAI are provided.

1. Introduction

Generative artificial intelligence (GenAI) has emerged as a transforming technology in our lives, including education [1,2,3,4,5,6]. As this technology changes continually and dramatically, it becomes necessary to keep abreast of its evolving capabilities. Developing advanced skills, as reflected by the higher levels in Bloom’s taxonomy [7,8], requires using GenAI in meaningful tasks such as active learning [9,10,11,12,13]. This approach is embedded in practices such as project-based learning [12,14]. This necessitates careful design of learning contexts that effectively integrate GenAI instruction with discipline-specific content [1,3,5]. Such an approach might be prohibitive for many educational programs that operate with minimally required courses, creating a challenge for these programs.

However, in many educational programs, especially graduate programs, students are taught advanced material through research seminars. In such seminars, students read, analyze, and report on studies they have read, supporting their advancement to levels such as understanding or applying in Bloom’s taxonomy. This may be insufficient, posing another challenge.

These two challenges seem to compete for similar resources of course time; hence, they are contradictory. Design theory can help educators resolve this competition or contradiction. TRIZ [15], the theory of inventive thinking, and ASIT [16], a simpler advanced systematic inventive thinking, offer mechanisms to resolve contradictions. This study applied ASIT’s law of qualitative change as an inspiration and arrived at a solution that is presented in this paper: by combining GenAI learning with a research seminar, the stressful arrival of GenAI is turned into part of the solution. The result is a better learning outcome for the seminar and “free” GenAI skills at the highest levels of Bloom’s taxonomy. With proper design, GenAI can make education antifragile. Rather than deteriorating due to new challenges, it can become better equipped to handle them.

This paper documents the use of GenAI in a graduate-level seminar on systems engineering to address the aforementioned challenges. All 26 students enrolled in the class had an engineering or similar undergraduate degree and practical experience of one year or more. Two specific hypotheses guided this research: (H1) engaging with GenAI in research and design activities improves student proficiency in using GenAI, and (H2) engaging with GenAI in design activities related to advanced disciplinary knowledge improves their understanding and use.

This study employs a case study methodology, providing detailed qualitative documentation of the seminar progression complemented by quantitative survey data. Data collection involved recording all student presentations, collecting slides and GenAI interaction materials, and distributing these resources to all participants. The analytical process included three phases: immediate post-session analysis, end-of-seminar comprehensive review, and final analysis during manuscript preparation. While analysis was conducted by the author, student references to previous materials and their comments about them provided validation of observed learning patterns and supported the study’s conclusions.

The structure of the paper is as follows. Section 2 provides background material on education. Section 3 describes the research methodology of this study. Section 4 presents some learning experiences observed from the students’ presentations at the seminar. Section 5 presents a survey conducted at the eighth meeting. Section 6 discusses the seminar output and concludes the paper.

2. Background

2.1. Teaching Generative AI

The recent literature reveals diverse perspectives on integrating GenAI into educational practices, with studies highlighting both transformative potential and significant implementation challenges [1,3,5]. Empirical studies demonstrate that effective GenAI integration requires substantial pedagogical adaptation, with successful implementations emphasizing personalized learning pathways and critical skill development [2,6]. These findings informed our approach by highlighting the importance of structured skill development and critical assessment capabilities.

Research identifies several critical challenges in educational GenAI use: potential algorithmic bias and factual inaccuracies, academic integrity concerns, and the imperative to develop students’ critical evaluation skills [2,3]. These challenges directly informed our emphasis on validation processes and ethical considerations in the seminar design. Additionally, over-dependence on GenAI may potentially inhibit the development of fundamental cognitive skills, including critical thinking, problem-solving, and creativity [5]. However, studies in design education suggest that when properly integrated, technology-enhanced learning can strengthen these core capabilities [12].

Current pedagogical approaches to GenAI education encompass three primary strategies: (1) integration into existing coursework for authentic task completion [2,4]; (2) development of foundational GenAI literacy through technical understanding [5]; and (3) advanced engagement through GenAI training and customization, which may enhance conceptual understanding [3,17]. Our seminar design synthesizes these approaches by combining authentic task integration with progressive skill development toward advanced GenAI customization.

Due to the fast pace of GenAI development, educators and students must be constantly updated. Rather than focusing on the tools themselves, their details, and their contemporary capabilities, the nature of the tools should be studied, as well as finding relevant tools, evaluating them, and learning to use them effectively [3,6,18]. The rapid evolution of GenAI capabilities is exemplified by the observation that a custom GenAI tool developed by one student during the ninth seminar meeting was later incorporated into Claude’s standard features, demonstrating the dynamic nature of this technological landscape.

2.2. Theoretical Perspective

The theoretical framework for GenAI education benefits from applying Bloom’s revised taxonomy [8], specifically adapted for GenAI literacy development [19]. This adaptation by Hershkovitz et al. provides a structured progression for skill development, as illustrated in Figure 1. When referring to levels and GenAI skills, this framework interprets them as follows:

• Remember: Knowing about GenAI technology and particular tools.
• Understand: Understanding ideas about GenAI tools, including the ability to explain, compare, and contrast them.
• Apply: Using GenAI to obtain answers to challenges. This may involve acquiring skills for using GenAI more effectively, e.g., by prompt optimization.
• Analyze: This is the critical skill of critically assessing the results of GenAI tools and building best practices.
• Evaluate: This level deals with making value judgments, including the ethical implications of using GenAI.
• Create: This level concerns generating novel ideas and designing new solutions, escaping biases and fixations.

Figure 1. Bloom’s and Hershkovitz et al. revised taxonomy.

Figure 1. Bloom’s and Hershkovitz et al. revised taxonomy.

The goal of the instruction was to systematically progress through all taxonomy levels within a single course. This comprehensive approach requires integration of multiple learning theories to achieve effective skill development across all levels. Achievement of higher-order learning objectives necessitates integration of complementary pedagogical frameworks.

Experiential learning emphasizes learning through concrete experiences, reflective observation, abstract conceptualization, and active experimentation [9,11]. Situated learning, which explains skill acquisition through practice, overlaps with these activities [20]. These theoretical foundations align naturally with project-based learning approaches [13,21], particularly when systematically incorporating all four experiential learning phases [12]. The integration of these theories directly informed our seminar design, ensuring students engaged in concrete GenAI experiences, reflected on their effectiveness, conceptualized best practices, and experimented with advanced applications.

Social cognitive theory [22] provided additional theoretical grounding for peer-to-peer learning opportunities. Students could learn from each other’s successes and failures in addition to their individual learning activities. When considering the virtual course format, sharing information about learning activities and outcomes enhanced learning through connectivism principles [23,24]. Finally, the competitive element inherent in progressive presentations provided engagement incentives that research shows can significantly improve learning outcomes [12,25].

The logic of this study is captured in Figure 2. The original research seminar goals were (1) to acquire and improve research skills and to study advanced systems engineering topics. The appearance of GenAI allowed the implementation to achieve these objectives faster, despite the need to study the material alone (2), study it with GenAI, and compare and reflect on the differences and the process. In contrast, a PBL course (3) is usually time-consuming, focusing on acquiring design-related skills, which would be prohibitive to combine with a research seminar. Nevertheless, (4) GenAI allows us to conduct some PBL-related tasks faster and (5) affords their incorporation into the research seminar. The theoretical framework suggests that the resulting research seminar with PBL tasks has a bootstrapping effect: (6) the PBL component improves the skills related to GenAI, making their learning process faster and more effective in the seminar. Using the disciplinary knowledge studied by students in a design task improves their learning according to Bloom/Hershkovitz et al.’s taxonomy.

In summary, the following hypotheses are stated:

H1: 

Engaging with GenAI in performing research and design activities improves student proficiency in using GenAI.

H2: 

Engaging with GenAI in design activities related to advanced disciplinary knowledge improves its understanding and use.

Given the exploratory nature of this research and the educational setting constraints, a case study methodology was selected as most appropriate, with quantitative survey data providing supplementary validation. While the sample size (26 students) limits generalizability, the rich qualitative data and multi-source triangulation provide valuable insights for hypothesis examination.

3. Methodological Approach

This study employs a single-case-study design [26] with embedded units of analysis corresponding to individual student learning trajectories and weekly seminar sessions. The case study approach was selected as most appropriate for this exploratory research examining a novel educational intervention in its natural setting.

3.1. Data Collection

Multiple data sources were systematically collected to enable triangulation:

-

All student presentations were video recorded (18 h total) and shared with students.

-

GenAI interaction materials (prompts, outputs, student reflections) were collected and shared with students.

-

Anonymous survey responses were recorded from 14 of 26 participants at week 8.

-

Student self-reports and peer discussions during seminar sessions were recorded.

-

Weekly observation notes were written documenting student engagement and learning progression.

3.2. Data Analysis

A three-phase analytical approach was implemented:

• Immediate post-session analysis: Pattern identification and preliminary analysis after each seminar session.
• Mid-course thematic analysis: Systematic review of accumulated data at week 8, informing survey design. This involved qualitative coding of student presentations and interactions based on thematic categories related to GenAI usage sophistication (H1 evidence) and the successful application of disciplinary knowledge in the design task (H2 evidence), guided by the levels of Bloom/Hershkovitz et al.’s adapted taxonomy. Examples of thematic categories included the following: tool exploration and familiarity (remember/understand), prompt sophistication and iterative dialogue (apply/create), critical assessment and validation (analyze/evaluate), and workflow development and multi-tool integration (apply/analyze).
• Final comprehensive analysis: Theoretical coding and hypothesis evaluation during manuscript preparation. Triangulation was achieved by cross-referencing findings across data sources, with particular attention to instances where student self-reports aligned with observed behaviors and documented GenAI interactions. The analytical framework was guided by the two research hypotheses, with systematic attention to evidence of skill progression (H1) and knowledge integration (H2).

3.3. Limitations

This study acknowledges several methodological constraints: single-site implementation limits transferability; lack of a control group prevents causal attribution; and analysis conducted by a single researcher may introduce bias, though triangulation across multiple data sources provides some mitigation. While the sample (26 students) limits generalizability, the rich qualitative data and multi-source triangulation provide valuable insights.

3.4. General Seminar Structure

A research seminar is mandatory in the first year of the graduate program in systems engineering at Tel Aviv University. It introduces students to advanced topics in systems engineering, teaching them to read papers critically, reflect on their scientific and practical merit, and present the papers and their analysis to the class. Instruction occurs primarily through targeted mentoring and structured feedback following student presentations. The seminar is conducted virtually. Each week of the seminar, a team of students selects a topic from a given set, finds quality research papers on the subject, and, upon my approval of their suitability, prepares a presentation on the subject that integrates the papers. The students present the papers’ contents and assess them critically. Following the presentation, the students discuss the topic with all the seminar attendees.

3.5. Participants’ Demographics

Twenty-six students participated in the seminar. They were all in their first year of the graduate program. Their undergraduate degrees included mechanical, electrical, software, aeronautic, material, and computer engineering from diverse universities in Israel. They had an average of 4.4 years of professional experience after completing their undergraduate degree, and they were employed by diverse industries, from public to private, large and small.

3.6. Use of GenAI in the Seminar

This year, due to the appearance of usable GenAI tools and the goal to gain proficiency in using them, the seminar incorporated a new primary objective: to explore the capabilities of GenAI tools to support research activities and the systems engineering process and acquire skills in using these tools.

The introductory session included standard seminar administration and topic selection, with the addition of a new project-based learning component: development of a package delivery drone (

Table 1. Drone design task requirements.

Design a Drone That Can Be Used for Package Delivery in Urban Environments. The Drone Must Meet the Following Requirements:

Payload Capacity: Carry a payload of up to 5 kg. Flight Range: Operate within a 10 km radius. Battery Life: Minimum 30 min of flight time per charge. Navigation: GPS and obstacle avoidance capabilities. Communication: Real-time communication with the control center. Speed: Capable of reaching speeds of up to 40 km/h. Weather Resistance: Operate in light rain and wind conditions. Safety: Emergency landing capability in case of failure. Autonomy: Autonomous take-off, flight, and landing. Noise Level: Operate below 70 decibels.

). The decision to use this task was based on its familiarity and the ability to find significant material on the Internet, as well as initial tests conducted to verify that current GenAI tools could provide reasonable responses to related queries, enabling meaningful learning experiences with evaluable outputs. The PBL task requested that the students employ the topic of their papers in the drone design problem. For example, students whose topic was fast systems engineering processes would have to explore developing such processes for the drone design challenge based on the details they learned from the research papers. Those who presented cyber–physical issues were tasked to consider the impact of this topic on their drone design.

Students received structured guidelines for effective GenAI utilization (

Table 2. General guidelines for using GenAI.

Understand the Tool Capabilities: Familiarize yourself with what each AI tool can do. Define Clear Objectives: Know what you want to achieve with the tool. Data Privacy and Security: Ensure data privacy when using AI tools. Iterative Testing and Validation: Continuously test and validate the outputs. Integrate with Existing Workflows: Seamlessly incorporate AI tools into your existing processes.

), with emphasis on iterative questioning and persistent interrogation to achieve desired outcomes rather than accepting initial responses.

The idea behind the seminar tasks was to let the students explore what the tools can do. They were supposed to find tools, learn how to use them, decide when to use them, etc. They were guided to try to use them on any task they had but to make sure they documented everything and could comment on the quality of the operation and outcome.

The research anticipated that students would demonstrate rapid skill acquisition, with initial presenters showing basic tool usage and subsequent students building upon their peers’ experiences. Additionally, the diversity of student backgrounds and professional interests in GenAI was expected to drive high engagement and innovative applications. The unstructured nature of the problem would also encourage creativity.

There were 9 meetings where students presented their work. The following section describes the evolution of using GenAI through the seminar.

4. Seminar Meetings

This section provides detailed documentation of GenAI utilization throughout the seminar, with emphasis on skill development rather than discipline-specific content. In each of the 2nd through 10th meetings, each subsection discussed using GenAI to address the topic presentation and the drone design challenge separately. The students were expected to read papers on the topic they selected, understand and critically present them, reflect on the GenAI tools they used, and use GenAI tools to design a drone with specific requirements that implement the papers’ topics. Section 4.11 provides a summary of the tools used by students, and

Table 3. Tools used for different purposes by the students. The column number indicates the week of the seminar.

	Week	2	3	4	5	6	7	8	9	10
Task
Identifying papers		Perplexity	ChatGPT	Elicit, ChatGPT	Gemini, ChatGPT	HyperWrite	Perplexity, ChatPDF	Perplexity, Gemini	Perplexity, ChatGPT
Summarizing papers		ChatGPT, Copilot	ChatPDF	ChatPDF SCISPACE, NotebookLM	Sharly, perplexity, pdf	Copilot ChatPDF NotebookLM ChatGPT	ChatGPT, ChatPDF	TLDRThis, SCISPACE, ScholarGPT	ChatPDF, AskPDF, PDF, SummaryPLUS, Copilot	ChatGPT
General dialogue			Perplexity, ChatGPT	Perplexity, ChatGPT, and Gemini	Gemini ChatGPT, Cloude		ChatGPT, ChatPDF	Elicit, NotebookLM, SCISPACE	ChatGPT	ChatGPT
Generating pictures		PIXLR	Ideogram	Ideogram, Copilot		Copilot	Copilot, OpenArt	PIXLR	PIXLR
Generating presentations					Presentation	Presentation, DrLambda, ChatGPT, Copilot	SlidePilot, SlideSpeake, SlidesGo	SlidesPilot, Gamma	Gamma	ChatGPT, SlideSpeak, Gamma
Prompt improvement					PromptPerfect	PromptPerfect	PromptPerfect	PromptPerfect		Grammarly
Slides layout					Designer (Microsoft)		Designer (Microsoft)
Drawing diagrams			Mermaidchart	Mermaidchart					ChatGPT + PlantUML, PlantText, DiagrammingAI, ChatUML
Combining documents				NotebookLM
Misc.						Wordtune, Grammarly	PDFAid, QuillBot	NaturalReader		Speechify

has a breakdown of the tools used each week.

4.1. Meeting 1—Seminar Introduction

Following the principle of reflexive practice [27], the seminar design incorporated GenAI tools wherever feasible. Initial attempts to generate introductory slides using GenAI produced reasonable but incomplete results, requiring integration with existing materials. However, the drone design requirements (Table 1) were successfully generated by querying GenAI for the most critical specifications for package delivery drones, with minor refinements for educational appropriateness.

4.2. Meeting 2—Requirement Management

4.2.1. Subject Matter

Students used the tools recommended for finding papers on their topic, summarizing them, explaining key concepts in the papers, and deriving the conclusions of those papers. They commented on the functionality of the tools, whether the performance was acceptable through comparison between their understanding and tool-generated summaries, and how to get them to work better.

Students independently discovered additional tools beyond those initially presented, including PIXLR for image generation, demonstrating autonomous tool exploration.

4.2.2. Drone Design

In applying the papers’ content to the drone design problem, the students concluded through systematic analysis that the drone requirements are reasonable. They did not apply GenAI to propose some applications of the topic they studied to the drone design task. During the seminar discussion and instructor feedback regarding potential applications, one student used ChatGPT to check the instructor’s comment and found that ChatGPT did not function well; he started a new chat and found a different functionality issue. Students proposed different ways to interrogate the GenAI, for example, asking what the underlying knowledge for its response is, and whether it can improve it.

While this meeting touched upon all of Bloom/Hershkovitz et al.’s proficiency levels (remembering, understanding, applying, analyzing, evaluating, and creating), it demonstrated that students’ proficiency is limited, but their openness to engage in experimentation and learning is significant.

4.3. Meeting 3—Adaptable Flexible Architecture

4.3.1. Subject Matter

The students used similar tools as the previous group, but in addition, they interrogated Perplexity to understand specific topics in the paper that were technically complex. The students asked: “How could the authors predict and consider the optimization process in those unknown modules that will be added later?” They received a decent summary of the approach and continued to refine their understanding. In addition, following a question stating that they did not understand the approach fully and would like to see an example, Perplexity generated a detailed example calculation. Another interesting example was the use of GenAI to analyze concepts presented in papers, such as by running a SWOT (Strengths, Weaknesses, Opportunities, Threats) [28] analysis on an approach proposed in one of the papers.

The students commented that they used GenAI after reading the papers to enable critical assessment of the results. In their presentations, the students elaborated on the details of using the GenAI tools and commented on their limitations following some interrogation. In addition to using GenAI tools, the nature of the task prompted students to look for other tools and explore them. While preparing for this meeting, one student found ResearchRabbit and explored its functionality.

4.3.2. Drone Design

The students used Perplexity to request a design of a drone that incorporates the topic theory and received a detailed sequence of steps that would allow them to design the drone. Following the process plan, Perplexity provided a general architecture for a plausible drone and a script for Mermaidchart to facilitate its drawing. The code generated was used in Mermaidchart for diagrammatic representation. Other uses included incorporating ideas from the paper in the drone design challenge. These uses did not necessarily need the drone requirements.

This meeting advanced beyond the previous in learning to use GenAI for practical purposes, including sequencing diverse tools. In this meeting, students demonstrated all of Bloom/Hershkovitz et al.’s levels; for example, their evaluation improved as they commented on the value and limitations of different GenAI tools.

4.4. Meeting 4—Complex Adaptive Systems

4.4.1. Subject Matter

The team presenting in this meeting used a richer set of tools to address the tasks; see Table 3. The students commented on the ability of the tools to answer questions related to the papers. They further used several tools to obtain answers to queries and compared their responses, which differed completely. Further, the students combined all papers to try to derive common themes, a more creative analysis of the research material.

The students noted gaps in the functioning of the tools, including a comparison between different GenAI tools. For example, one student noted that when asking questions about information that does not appear in papers, NotebookLM (using Gemini 1.5 Pro) would comment that this information does not exist in the papers, while ChatGPT would generate an answer anyway. Such comments demonstrate the students’ work process: reading the papers, forming their understanding, and then using different tools to understand their capabilities. This demonstrated improved evaluation skills.

The students interrogated several GenAI tools about gaps in their knowledge of the topics presented in the papers and received diverse answers.

Another query was to ask GenAI to imagine a system that included concepts from each of the papers, and one viable response—using a multi-agent system for disaster response—surprised the students. This shows that GenAI tools could be used to drive creative solutions.

4.4.2. Drone Design

The students used ChatPDF to address the drone design task. Using general prompts, they received some recommendations about implementing the ideas of the papers in the drone design challenge.

This meeting again demonstrated all of Bloom/Hershkovitz et al.’s levels with improved skills. Further, from students’ referencing previous meetings, it was clear that they took it as a baseline for improvement. Here, they improved upon the comparison between different tools, exposing a limitation of ChatGPT compared to NotebookLM (evaluation skill) and a creative use of GenAI to generate surprising results (create skill).

4.5. Meeting 5—Antifragility

4.5.1. Subject Matter

The students kept looking for additional GenAI tools by asking GenAI to recommend tools. When they found a tool with new functionality, such as PDF marking parts of text from which answers were extracted, they presented this functionality to the class.

4.5.2. Drone Design

To check the application of antifragility to the drone design problem, ChatGPT was asked to design a drone according to the requirements and then apply the concept of the paper to this design. The tool scored its proposal according to its antifragility. The student asked the tool to improve the antifragility score and received another design. The designs were not assessed independently by the students.

Another trial used Perplexity and Copilot to generate a detailed drone design and recommend how to improve its antifragility. These tools were selected following ChatGPT’s recommendation. In real practice, all the candidate designs could be compared and assessed, selecting the best for further analysis.

This meeting repeated the previous meetings’ insight, with the addition of having GenAI recommend other GenAI tools (know and evaluate skills).

4.6. Meeting 6—Industry 4.0, Predictive Maintenance

4.6.1. Subject Matter

The students used the tools as in previous presentations. They further conducted a comparison between two tools, ChatPDF (version 1.5) and ChatGPT-4o, after reading the papers themselves. They found that concerning topic challenges, ChatPDF generated its answer from all the papers, and ChatGPT only from a section in the papers called challenges. ChatGPT provided a better critical analysis than ChatPDF and a better description of the papers’ contributions.

4.6.2. Drone Design

ChatGPT was used to recommend implementing the papers’ ideas in drone design. The use of specific prompts led to particular results being unusable. Detailing the prompts and extended dialogue led to answering whatever the students asked, starting with which sensors to use, whether special expertise is required of people working with such drones, and how they can be trained.

Asking ChatGPT to evaluate its recommendations and the recommendations in a paper started with rating the paper as higher than its own. Further questions regarding potential deficiencies in the paper’s recommendation led to a revised equal evaluation of the two recommendations. This demonstrates that all answers by GenAI tools need to be critically assessed. Nevertheless, answers, even if incorrect, point to the relevance of issues that we might not have considered without the dialogue. In addition, contrasting the paper recommendation with another source, such as GenAI, required a deeper analysis of the paper content, thereby improving its understandability.

Asking ChatGPT whether Gemini or Copilot could do a better job than itself did not yield an explicit answer, but it commented on the benefits and limitations of using these tools. When asked to estimate the quality of the answers these tools might provide, ChatGPT gave them lower estimates than it gave to its own recommendation.

This meeting presents a greater sophistication in using GenAI to evaluate GenAI results and to create results (evaluate and create skills).

4.7. Meeting 7—Human Machine Interface/Society 5.0

4.7.1. Subject Matter

In this meeting, the students used similar but also many new GenAI tools in their study. One student created the flowchart in Figure 3 to represent the sequence of tools he used. This marks an explicit representation of a flowchart of a complex process rather than a chaotic use of a mix of tools.

4.7.2. Drone Design

The students used PDF to load their papers and asked PDF to design a drone that combines the concepts. PDF summarized the papers. The students asked PDF again to design a drone and received a proper response with some components mentioned in the papers. Asking to provide the best solution led to selecting items from one paper only, and subsequent prompts to integrate insights from both led to specifying a drone with aspects from both papers and the rationale to propose them. The students learned that an extended and persistent dialogue may lead to obtaining relevant answers and that using GenAI allows them to integrate insight from multiple papers, improving their comprehension and supporting critical assessment of their content.

In the seminar, we queried PDF about the price of the drone it designed. The response after specifying the breakdown of the cost led to USD 4770. We asked to adjust the price to a budget of USD 3000, and the response provided a different design with a lower cost that did not fit the requested budget; following this, PDF initiated another cycle of redesign, leading to USD 2550.

This quick interactive exercise was instrumental in showing the ease of using the tools and the constant opportunity to exercise many what-if scenarios. This is a new form of sophistication in using GenAI tools (create skills).

4.8. Meeting 8—Fast Processes

4.8.1. Subject Matter

The three students visually described their three different processes, including regular tools such as Google Translate (Figure 4, Figure 5 and Figure 6). In Figure 6, the student incorporated graphs depicting the student’s perception of the effort it takes and the resulting understanding that he obtained in four ways. The best understanding and most effort came from reading the papers, and the least effort and mediocre understanding came from listening to the GenAI summary. Such process descriptions and analyses mark an explicitly documented understanding of GenAI tools. Such documentation provides the basis for further development [24,29].

4.8.2. Drone Design

To develop fast processes for the drone design problem, the students wanted to input three papers and the requirements as a prompt and ask for a process plan. Several tools were tried, but most failed because they could not handle three papers. Two tools passed the initial screening, and one was added from the failing group: NotebookLM, Elicit, and SCISPACE. The students wanted to understand which tool is preferred for a similar future task. They asked ChatGPT to compare the tools’ answers to their query about the process plan development task using the Pugh Controlled Convergence (PCC) tool [30]; see a summary of the process in Figure 7.

ChatGPT’s answer appears in Figure 8, but it did not use PCC to derive it. “SCISPACE appears to be the most balanced and effective method based on the Pugh analysis, excelling in multiple critical areas such as initial design, prototyping, risk assessment, and stakeholder involvement, with a total score of 6. Elicit follows closely with strong feature integration and validation processes. NotebookLM provides a solid approach but falls slightly short in continuous testing and stakeholder involvement.”

The students wanted to obtain the output as performed in the PCC (Figure 9), and they asked, “Now do the Pugh table like this:” and inserted the picture in Figure 9.

The ChatGPT results are shown in Figure 10 and include a textual recommendation to use SCISPACE; consequently, the students presented the results of this tool only. This example marks progress in realizing that AI tools can be trained to use and execute known design tools. It was interesting to observe that the GenAI tool could interpret the picture and use its content and general knowledge to execute PCC. This exercise demonstrated students’ resilience in using GenAI. At the end of the seminar, students were asked to build upon this presentation in the next two meetings.

This meeting presents a creative, methodical use of GenAI tools (incorporating a small study that demonstrates advanced apply, analyze, and evaluate skills) and a skill outside the scope of the taxonomy—teaching or developing GenAI tools.

4.9. Meeting 9—MBSE and AI

This was intended to be an interesting meeting, as the topic was broadly digitalization in systems engineering and, specifically, the use of AI and model-based systems engineering (MBSE) [31] in systems engineering.

4.9.1. Subject Matter

The study of the papers resembled previous sessions, with tools interrogated to provide insight into the topics. The students presented graphic summaries of all the tools they used, like those in Figure 4 and Figure 6.

4.9.2. Drone Design

When the students applied the topics of their papers to the drone design problem, they received verbal descriptions, but they wanted to obtain further MBSE models of the drone proposed by GenAI. It was impossible to generate such diagrams with common GenAI tools, so with the help of GenAI, the students found a workaround: asking ChatGPT to provide a textual description in a format that could be used by PlantUML to display diagrams. The students requested activity, block, and requirements diagrams; received the description; and moved to PlantUML to obtain partially satisfactory results. They did not try to improve the input by asking ChatGPT to provide better-looking diagrams. We discussed in class the potential of not stopping before trying to get the tools to deliver what we want, for example, by providing feedback to ChatGPT about the graphic deficiencies and giving instructions on how to correct the diagrams. Through a quick interaction with ChatGPT, it might be able to respond to such instructions.

Another student used another recommendation of ChatGPT, PlantText, which uses PlantUML for its display. Since he presented a paper that dealt with improved modeling using domain knowledge and such knowledge improved modeling, he used a similar approach in designing the drone. The student generated diagrams in two ways, one with the regular approach and one by simply using the following prompt: “Remake functional analysis using NLP enhancement and create https://www.planttext.com/ diagram—make sure you improve the text based on buzzwords from the web.” The idea was to find words termed “buzzwords” relevant to the topic and use them to enhance the responses. Besides the new diagrams, ChatGPT also commented, “These enhanced requirements and diagrams use modern buzzwords and best practices to create a more comprehensive and clear representation of your drone system.” At the seminar, the students asked the audience to distinguish between the diagrams created in two different ways: with and without “buzzwords”. It was obvious that the diagrams that had more details were generated following this prompt; see Figure 11. The student commented that, based on his experience, the diagrams were quite good as a starting point. The student further explored the same process with Copilot and presented the results. The content and format of diagrams generated by the two tools differed in many ways. The student commented that a tool or manual modeling could use the output of several tools as an ensemble.

The third student was disappointed by the performance of several diagramming tools he found. It turned out from the error messages they produced that all employed the PlantUML (version V1.2024.6) tool. The student decided to generate his own tool for modeling code or systems with MBSE.

He trained ChatGPT with XML descriptions of diagrams created by the draw.io tool. He focused on class diagrams that are easiest to model and are the foundations for all MBSE modeling. He drew typical diagrams in draw.io, took their XML code, and trained ChatGPT to understand them. The process took about 4 h to get ChatGPT to understand the different XML commands. The next step was testing ChatGPT to see whether it could generate diagrams from textual descriptions. This is a simple and necessary step in verifying the process.

The next step was to ask AI to take C or C++ code and transform it into diagrams to verify the output of coding. Given the effort spent on the training, the student thought he could go a step further and create a tool for his professional work. When building software applications, one starts with such diagrams, generates code, and tests it. If one could take the code and transform it back into models and compare the input with the output models, one would have a way to validate the code. The student checked this ability and found that his trained ChatGPT could generate diagrams that could be compared with the initial diagrams. The student commented that he can now use the tool built in the seminar in his professional work.

This meeting demonstrated that GenAI tools could be developed for a desired purpose. One must have some proficiency in different tools to be able to use them efficiently, but this might not be necessary soon. Moreover, this process demonstrated significant proficiency with GenAI beyond Bloom/Hershkovitz et al.’s taxonomy—enhancing GenAI tools.

Two other general observations demonstrated in this meeting are the improvement of results by using general concepts or best practices through a simple prompt (creative use of GenAI—create skill) and by breaking down general queries into a collection of simple queries (apply and create skill).

4.10. Meeting 10—Cyber–Physical Systems

4.10.1. Subject Matter

The students examined whatever tools were mentioned and demonstrated in previous students’ presentations and looked for additional sources, such as TikTok, to find new recommendations. They found SlideSpeak for generating slides from large files.

4.10.2. Drone Design

The students asked ChatGPT to employ cybersecurity in drone design. One student was an expert in the subject and was able to comment on the quality of the response. The results derived by GenAI were found to be reasonable.

4.11. Summary of GenAI Use at the Seminar

Students employed numerous tools in the seminar, both free and paid versions of the tools. While there are differences in the performance of these versions, this evaluation was not the purpose of their use, but rather, learning about the technology. The students found most of the tools they used on their own, including by asking other tools, such as ChatGPT, for recommendations:

• AskPDF
• ChatGPT
• ChatPDF
• ChatUML
• Copilot
• Claude
• Designer (Microsoft)
• DiagrammingAI
• DrLambda
• Elicit
• Gamma
• Gemini
• Grammarly
• HyperWrite
• Ideogram
• Mermaidchart
• NaturalReader
• NotebookLM
• OpenArt
• PDF
• PDFAid
• Perplexity
• PIXLR
• PlantText
• PlantUML
• Presentation
• PromptPerfect
• QuillBot
• ResearchRabbit
• SCISPACE
• SlideSpeak
• Sharly
• Speechify
• SummaryPlus
• TLDRThis

A breakdown of the tools used for each purpose in the different meetings is shown in Table 3. The collection of tools shows enrichment as the seminar unfolded. Note that the precise version of the tools used depends on the date they were used (between June and August 2024), but it is unimportant to the results of this paper. The conclusions would apply to tools being used today.

Students used GenAI for diverse purposes: summarizing, criticizing, and pointing to potential future work related to the topics they presented. By reading the papers themselves and forming their own opinion, students observed the limitations of the tools and realized that they could use GenAI as a second opinion that supported their reasoning. Many students reiterated these remarks.

Some of the students’ presentations included quick tutorials on using tools; some presented the prompts they used, compared the performance of tools, and shared their experience in a particular task using various tools.

During the eighth meeting, one of the students presenting at this meeting prepared a list of AI tools mentioned in the seminar and a description of the tool prepared by ChatGPT. He shared this list with the seminar participants.

Students formed processes of using tools for different purposes, as shown before. The availability of explicit process representations provides anchoring for new elaborations and improvements.

Finally, at the ninth meeting, one student, proficient in programming, generated his own AI tool by training ChatGPT and building a process to deliver the desired result. He commented that he can use this in his professional work.

Students disclosed whether they had experience with their presentation topic. It seemed that for their presentation, some students selected a topic that was familiar to them in general, although the research paper documented new state-of-the-art developments beyond common practice. Consequently, the students could reflect better on the paper content and the use of GenAI. This prior knowledge improved the experience of the other attendees. The students reported that it was critical to continuously check the results of GenAI tools as the dialogues with them unfolded. The tools’ value was good in generating new responses, potentially de-fixating students’ thinking, but less in providing quality responses that could be relied on.

Altogether, the seminar outcomes not only demonstrated significant skill development in using GenAI but also critical thinking, creativity, and presentation skills. Concerning the hypotheses, the use of GenAI improved students’ proficiency with the tools, and it can be concluded that the PBL element further improved this proficiency. Further, engaging with the disciplinary knowledge in the design challenge improved understanding. Meetings 5 and 6 included a comparison between alternative implementations of the knowledge, allowing for reflection on the knowledge; Meeting 7 integrated knowledge from three sources in the implementation, demonstrating knowledge integration, and Meeting 9 included incorporating general knowledge in addition to the papers, effectively increasing its breadth and scope.

The qualitative evidence presented in this section validates the hypotheses. Clearly, “engaging with GenAI in doing research and design activities improves student proficiency in using GenAI.” Further, “engaging with GenAI in design activity related to advanced disciplinary knowledge improves its understanding and use.” This was clear from explicit experiences in Meetings 6 and 7, where students contrasted and integrated insight from two or more papers and were able to identify their differences and complementarities. Similar experiences were reported in other meetings but are not mentioned explicitly in this paper.

5. Survey

To complement the qualitative evidence of skill development and knowledge integration, quantitative survey data were collected to assess student’ perceptions of their GenAI literacy progression. This data provided both validation of observed learning patterns and guidance for the remaining seminar sessions. A short survey was administered to query about the literacy levels defined by the TAU AI Learning Community [19]. The full questionnaire is available in Supplementary Material. These are slightly different from Bloom/Hershkovitz et al.’s levels, shown in Figure 1. At the eighth meeting of the seminar, students were asked to report on their perceptions of GenAI before the seminar started and at the present meeting. Participation in the questionnaire was voluntary, anonymous, and presented no risk to the participants; consequently, no ethical approval was necessary. The questionnaire was administered through Google Forms. Fourteen students (out of twenty-six) answered the questionnaire. While a 54% response rate is acceptable in education surveys and others (e.g., see [32]), the sample size and response rate limit generalizability; nevertheless, the survey data provide valuable supplementary evidence supporting the qualitative findings. The results appear in

Table 4. Results of the survey: perceptions about GenAI literacy.

Perception Pre-Seminar	Perceptions at Class 8th
(a)—Rank your familiarity level with GenAI that could be used for particular tasks before the seminar	Rank your familiarity level with GenAI that could be used for particular tasks after the seminar
(b)—Rank your update level with GenAI technology innovations before the seminar	Rank your update level with GenAI technology innovations after the seminar
(c)—Rank your understanding level of deriving the best outcome from GenAI before the seminar	Rank your understanding level of deriving the best outcome from GenAI after the seminar
(d)—Rank your capability level to implement prompts leading to the most desired outcomes from GenAI before the seminar	Rank your capability level to implement prompts leading to the most desired outcomes from GenAI after the seminar
(e)—Rank your capability level to employ GenAI ethically in addressing a task before the seminar	Rank your capability level to employ GenAI ethically in addressing a task after the seminar
(f)—Rank your capability level to compare the results of GenAI tools used for particular tasks before the seminar	Rank your capability level to compare the results of GenAI tools used for particular tasks after the seminar
(g)—Rank your capability level to validate the results of GenAI compared to other sources and prior knowledge before the seminar	Rank your capability level to validate the results of GenAI compared to other sources and prior knowledge after the seminar
(h)—Rank your capability level to best address the given challenges with GenAI before the seminar	Rank your capability level to best address the given challenges with GenAI after the seminar

.

Statistical validation was conducted using GenAI tools (ChatGPT and Copilot) following the reflexive practice principle [27]. After initial verification concerns, both tools provided detailed calculations confirming paired t-test results with Bonferroni correction. All comparisons yielded p-values below 0.0002, indicating statistically significant improvements across all literacy dimensions. This analytical process itself demonstrated effective GenAI validation strategies taught in the seminar.

Table 5. Results of the survey: difference between post and pre-condition for each response and results of a corrected paired t-test.

	Question	(a)	(b)	(c)	(d)	(e)	(f)	(g)	(h)
Response #
1		6	6	6	8	6	7	6	6
2		6	6	7	5	8	5	4	7
3		1	2	2	3	0	1	1	2
4		5	2	4	−2	1	5	2	5
5		3	0	2	3	3	2	3	2
6		5	3	4	2	4	1	4	4
7		5	0	5	7	7	7	7	−1
8		4	2	3	5	3	4	3	4
9		2	0	2	2	3	2	3	4
10		3	5	7	3	0	3	3	0
11		7	7	7	7	8	7	7	6
12		3	3	4	1	0	3	1	3
13		4	2	6	6	3	3	4	5
14		4	4	4	4	4	4	4	4
Average differences		4.14	3.00	4.50	3.86	3.57	3.86	3.71	3.64
SD differences		1.66	2.32	1.87	2.71	2.82	2.11	1.90	2.27
p-value		1.95 × 10−7	0.000162	3.01 × 10−7	6.96 × 10−5	0.000194	5.86 × 10−6	2.91 × 10−6	2.24 × 10−5

provides the difference between the levels of proficiency in the post- and pre-conditions. For example, the 6 in cell [1, (a)] shows that student 1 listed 8 in the post-condition familiarity level of GenAI and 2 in the pre-condition, leading to a difference of 6. The null hypothesis is that the post-condition is not different than the pre-condition. All p-values are well below what is considered significant. Besides manual calculations, these results were confirmed with Statistics Kingdom (https://www.statskingdom.com/paired-t-test-calculator.html; accessed on 21 September 2025).

The sources of learning were the instructor’s slides during the first class of the seminar, self-learning, learning from other students’ presentations, and learning from preparing the seminars. Students were asked to report on the importance of sources from which they learned about GenAI. The results in

Table 6. Results of the survey: sources of learning.

Rank your learning level about GenAI from the lecturer’s presentation in the first meeting	Rank your learning level about GenAI from self-learning
Rank your learning level about GenAI from other students’ presentations	Rank your learning level about GenAI from the presentation you prepared for the seminar

show that the relative contribution of the sources was in the aforementioned order. No statistical tests were conducted to support this observation.

Finally, the students were asked about skill transfer into personal use, use at work, and use in other courses.

Table 7. Results of the survey: transfer of literacy.

Is your seminar experience leading you to use GenAI in other courses?	Is your seminar experience leading you to use GenAI for personal challenges?
Is your seminar experience leading you to use GenAI in your professional work?

shows that there is a transfer of GenAI skills. The degree of transfer also makes sense. The positive experience in the class transfers easily to other courses; personal challenges are next in the transfer of skills; and transfer to professional work exists but is less profound, perhaps due to the responsibility necessary in professional practice.

At the end of the survey, the students could remark on their experience in an open-ended comment. Six responses were obtained:

• In my opinion, this was an excellent experience!
• The seminar and its structure are excellent. I used and knew about GenAI before, but the course strengthened nice things for me.
• Amazing! Only at the expense of what does it come?
• Great and interesting
• The focus of the seminar made me learn a lot about the different GenAI and enriched me a lot in the field.
• Excellent seminar, I learned a lot, I lacked significant knowledge in the field and today I filled in the gaps, I am sure that if there is a course where we would need AI services, I will know how to get the most out of it unlike the period just before the course, a must for every master’s degree student in my personal opinion.

Note that the survey was conducted during the eighth meeting. Conducting it after the end of the seminar would have strengthened the positive outcomes regarding the skill improvements and probably the skill transfer to other uses.

6. Discussion and Conclusions

6.1. General Discussion

By applying ASIT design theory within a reflexive practice framework [27], this study addressed the apparent contradiction between teaching GenAI skills and maintaining disciplinary learning objectives. The integrated approach enabled simultaneous development of GenAI proficiency and disciplinary knowledge. Students developed shared vocabulary for GenAI discourse and demonstrated progression across all of Bloom/Hershkovitz et al.’s taxonomy levels, with evidence of enhanced creativity emerging through advanced applications. Using GenAI for research is critical for professional work, as engineers continuously deal with documents and documentation. Moreover, the drone design task demonstrated to students the power of GenAI more profoundly, as the task represents the actual practice of systems engineers. The two challenges of learning GenAI and designing the drone enticed students to spend more effort than they usually spend on seminar assignments. Their effort paid off in improved skills and knowledge comprehension.

Moreover, although not part of the survey, it was clear from the students’ discussions that their understanding of the disciplinary knowledge that they studied and presented at the seminar was improved by using this knowledge on the drone design challenge. Instead of reading the material in the paper they were expected to present, they read and presented a few other papers to strengthen their understanding, and they had to apply the paper content in a design task. Without GenAI, such implementation would be prohibitive in a seminar because it would take significantly more time than is expected; GenAI tools allowed students to experience a much more advanced skill and internalize the material better. Consequently, not only did students gain proficiency in GenAI tools, but they also improved their disciplinary knowledge and research skills.

It was clear that the instructor’s slides at the beginning of the course only initiated the students’ curiosity. As more students presented how they used the tools and their results, subsequent students were driven to perform better. Students built on the experience of former presenters, explored additional tools, and improved how they used the tools and their way of presenting them. Each presentation made subsequent presenters push the boundaries of their exploration as if it were a competition.

Altogether, the students gained significant skill development at all proficiency levels and demonstrated it through their presentations. Their documented experiences in the form of video, presentations, and GenAI interaction documentation bootstrapped learning within the seminar by being used by others in the future.

The students used multiple tools; compared between tools; asked tools to recommend, compare, and select between tools; taught tools about design tools (Pugh Controlled Convergence); and trained tools with data representing diagrams for MBSE to generate specialized tools. The students’ use of GenAI was far beyond any expectations.

Students used tools for whatever difficulty they encountered as instructed and by learning from their peers, including querying about difficult concepts; asking for examples of calculations; comparing the results presented in papers and by the tool itself; commenting about the poor answers of the tools; and, in general, not settling on the first response.

It is clear from the students’ presentations that this class had side benefits. The challenges encouraged students to explore new territories, leading to improved self-learning, critical thinking, creativity, and presentation skills. This is supported by the evidence summarized in the concluding paragraphs of the meeting reports.

6.2. Evaluation of Research Hypotheses

The evidence collected in this research supports the hypotheses as follows.

6.2.1. Hypothesis 1 (H1): Engaging with GenAI in Research and Design Activities Improves Student Proficiency in Using GenAI

The evidence strongly supports H1 through multiple convergent sources:

• Progressive sophistication in tool usage from basic applications (Meeting 2) to advanced customization (Meeting 9).
• Development of systematic validation processes and multi-tool integration workflows (Meetings 7 and 8).
• Students teaching GenAI design tools (Meeting 8) and creating custom GenAI tools for specialized applications (Meeting 9).
• Statistically significant improvements across all GenAI literacy dimensions (p < 0.0002).

6.2.2. Hypothesis 2 (H2): Engaging with GenAI in Design Activity Related to Advanced Disciplinary Knowledge Improves Its Understanding and Use

H2 is supported by substantial qualitative evidence:

• Students’ ability to integrate multiple papers’ concepts through GenAI-assisted design (Meetings 6–7).
• Enhanced critical assessment capabilities when evaluating GenAI outputs against the academic literature (Meeting 3, 4, 6–8).
• Successful application of systems engineering principles through the drone design challenge (all meetings).

6.3. System-Level Educational Implications

The findings inform broader educational design in several ways:

• CDIO Integration: This approach aligns naturally with CDIO (Conceive–Design–Implement–Operate) engineering education principles [10], suggesting scalability across engineering curricula without requiring complete course redesign. Such interventions could also provide significant value if incorporated only in selected courses throughout education.
• Selective Implementation: Not all courses require full GenAI integration; strategic application in courses with design components may yield optimal cost–benefit ratios.
• Policy Considerations: Institutional policies should support faculty development in GenAI pedagogy while maintaining academic integrity standards.
• The educational framework demonstrated here appears transferable to other engineering contexts, though implementation may require adaptation for different student populations, particularly those with limited professional experience. The core principle—combining GenAI-enhanced learning with immediate practical application—may be broadly applicable across engineering education, potentially requiring only modest curricular adjustments to achieve substantial learning benefits.

6.4. Best Practices for GenAI Integration

Based on this educational intervention, several evidence-based practices emerged:

• Progressive Skill Development: Structure learning experiences to build from basic tool familiarity toward advanced customization.
• Authentic Task Integration: Embed GenAI use within meaningful, discipline-relevant challenges rather than isolated exercises.
• Peer Learning Facilitation: Create opportunities for students to learn from each other’s successes and failures.
• Systematic Validation Training: Explicitly teach and reinforce critical assessment of GenAI outputs.
• Curiosity development: Encourage students to ask GenAI about any aspect they are not certain about.
• Documentation and Reflection: Require students to document their processes and reflect on tool effectiveness.
• Multi-Tool Exploration: Encourage experimentation with diverse GenAI tools rather than reliance on single platforms.
• Meta-Cognitive Application: Use GenAI tools to evaluate and select other GenAI tools, developing sophisticated decision-making capabilities.

6.5. Limitations and Future Research

Several methodological limitations must be acknowledged: single-site implementation limits transferability; cultural specificity may affect generalizability; and single-researcher analysis introduces potential bias despite triangulation efforts. In addition, the survey cannot serve as a single validation source but only as supplementary evidence. The limited sample size (14 out of 26 students) limits the generalizability of the quantitative findings; however, the extremely high statistical significance (p < 0.0002) supports the observed perceptual shift, which is further corroborated by the rich qualitative data. Further, the consistency of findings across multiple data sources and replication in the subsequent semester support the robustness of key conclusions. Future research should examine implementation across diverse educational contexts and employ multi-researcher analytical approaches.

6.6. Final Note

The findings suggest that GenAI integration in engineering education need not be an additional burden but can enhance existing pedagogical approaches while developing essential 21st-century skills. The “antifragile” concept applied here—where challenges become sources of strength—may serve as a model for educational adaptation to emerging technologies. Rather than viewing GenAI as a threat to traditional educational approaches, strategic integration can strengthen both technological literacy and disciplinary understanding.