Using Generative AI to Support UX Design Students in Web Development Courses

Abstract

This work explores the integration of Generative AI (GenAI) tools into web development educational settings, with a focus on enhancing the user experience (UX) design process and supporting students with limited technical backgrounds. The democratization of GenAI has allowed non-technical users to engage in the creation of computing applications. However, its adoption among UX-focused learners remains limited. In this study, we propose an innovative instructional approach designed to facilitate the effective use of GenAI tools in web development courses for UX designers. This approach is based on incorporating interaction mechanisms that allow students to prompt GenAI tools using an incremental strategy. Moreover, this scaffolding process enables the definition of benchmarks that can be used as a reference for the development tasks proposed and their assessment by comparing the students’ outcomes with the benchmarks provided. The results obtained from applying the proposed approach in a web development course using GitHub Copilot environments show its potential to address the challenges UX design students face in these educational settings.

1. Introduction

Large Language Models (LLMs) are emerging as influential platforms in the development of computing applications, and the use of Generative Artificial Intelligence (GenAI) tools is becoming increasingly common among software practitioners [1]. These tools have enabled more effective and faster software development by accelerating most of the phases of this process [2,3,4,5]. In this context, the incorporation of these platforms and tools plays a crucial role in software-development-related educational settings [6], highlighting their ability to generate code and assess students’ perceptions about how they can be helped with their programming tasks. Nevertheless, examinations have revealed some pitfalls regarding the use of LLMs as coding assistants for computer science students [7].

There are multiple studies related to the educational application of GenAI in CS (or computing education) disciplines [8,9,10], and Raihan et al. [11] recently performed a systematic review of the use of LLMs in computer science education. Moreover, GenAI tools have been deployed in several computing areas, ranging from introductory programming [12] to more specific fields such as software engineering education [13] and database administration courses [14]. In particular, attention has been paid to novice programmers who use LLM-based code generators [15], including the benefits and harms they can bring about, and GenAI’s impact on novice programmer metacognition aspects [16]. However, the democratization of Generative AI tools has not yet translated into widespread adoption by non-technical users for developing computing applications. This is the case for UX (user experience) design students taking web development (WD) courses, the main target in our research. These students often face issues mainly precipitated by technical barriers, limited integration of design tools, and concerns around creative control [17].

In some cases, UX designers turn to Low-Code Development Platforms (LCDPs) [18], which offer easy-to-use visual environments, allowing these students to develop software applications without a solid programming background. Moreover, cooperation between UX designers and developers has long been promoted [19], and their collaboration has been boosted to improve efficiency in transformation processes from design to development through LCDPs [20]. The intention behind the different perspective considered in this work is to provide UX design students with tools enabling them to actively participate in the development process. This aspect is particularly important for students enrolled in creative technology courses whose instructional settings combine design skills with a basic knowledge of end-user development.

End-user development has long been linked with web applications [21], and Barricelli et al. [22] carried out a systematic mapping study concerning its relationships with end-user programming and software engineering. There are multiple aspects to be considered in this context. Recently, Paternó [23] addressed the contribution of AI technologies to the development of end-user applications. Assessing the impact of these technologies, particularly concerning the integration of GenAI when teaching WD courses, is one of the main goals of this work. The aim of such integration is genuine synergy between web design knowledge and development programming practices, thereby empowering students to leverage GenAI tools and facilitate this convergence. In line with this research idea, three main questions are proposed:

• RQ1: How can GenAI tools be effectively used in or integrated into web development courses tailored to UX designers?
• RQ2: What new interaction mechanisms can be explored to foster the practical application of these tools in web development educational settings?
• RQ3: How can these GenAI-based interactive learning settings be assessed and evaluated?

The remainder of the work is structured as follows. The first section reviews related work in consideration of the research questions listed above. Section 3 describes the materials and methods deployed in the current research, including a description of the target courses and the methods used to apply GenAI tools in this context. Section 4 reveals the results obtained after applying the proposed methods to the target courses. Section 5 provides a discussion on these results in consideration of the three research questions addressed in this work. Finally, some conclusions and ideas for further work are provided.

2. Related Works

As mentioned earlier, web applications have a strong relationship with end-user development [21], especially when geared towards users with low levels of programming skills. In this context, LLM platforms and GenAI tools can provide invaluable support for UX designers interested in developing these types of applications. The use of LLMs for end-user website generation has been endorsed [24], with emphasis placed on their ability to generate websites by abstracting away concerns about the underlying code. Furthermore, the impact of GenAI on front-end development processes has been thoroughly examined [25], revealing that it promotes automated code generation and dynamic content creation [26]. In this vein, the WebDiffusion tool [27] allows web developers to harness Generative AI to develop text and images. Web Sculptor [28] is another example of a GenAI-based development framework that improves code generation in web applications.

Despite the general success of these AI-enabled design support tools, several obstacles have been identified regarding UX practitioners. The first gap was observed by Lu et al. [29], who stated that many AI-based design tools are based on graphical interface elements, neglecting a design-thinking-oriented view. Additionally, some issues regarding overreliance and perceived lack of control over the automated code generated have been reported [30]. Hsiao & Tank [17] sought to apply GenAI tools in assisting the user experience design process, combining image prototype generators with prompts that allowed interaction with ChatGpt’s DALL·E, moving from abstract to more specific requirements. The idea of using incremental prompts was also exploited by Calo & de Russis [24], who employed LLMs to allow users to refine their prompt outcomes based on predefined templates when generating websites. A similar strategy was incorporated in Kodless [31], an LLM-based tool designed to automatically build web applications through an iterative prompt refinement process. These methods of organizing iterative prompts can be extremely useful in WD educational settings, and they are in line with the mechanisms proposed for interacting with GenAI tools in this study.

There are several initiatives relevant to our research focus, i.e., assessing the impact of GenAI tools in WD educational settings. York [32] evaluated ChatGPT’s use as a GenAI tool capable of assisting UX and WD students in three main areas: ideation, design, and coding. This evaluation focused on examining prompts, which could be used in these areas in a UX course context. The results of this evaluation were based on judging the quality of prompt outcomes, lacking a quantitative assessment. Lively et al. [33] were also interested in integrating GenAI tools into web design education by enhancing students’ aesthetic and creative abilities. In this case, the authors assessed the artifacts generated by students along with a collection of their impressions derived from the application of GenAI tools. In a similar way, students were asked to assess improvements in their critical thinking skills in a web programming module using this kind of tool [34]. A self-evaluation questionnaire was employed to collect students’ opinions after each practice activity, with the results showing limited critical-thinking improvement. The use of Generative AI by students as a support tool in an advanced web development course was studied in [35]. The authors analyzed the use of GenAI-based prompts in different educational tasks, ranging from ideation processes and explaining concepts to helping students generate code. They also accounted for the results after the formulation of prompts, differentiating their types together with the outcomes obtained.

3. Materials and Methods

3.1. Case Study

We conducted a case study on a Web Applications (WA) course during the 2nd semester of the academic year 2024–2025. This course is part of a bachelor’s degree titled “Design and Creative Technologies” in the Arts School at UPV (Universitat Politècnica de València). The Degree in Design and Creative Technologies focuses on the professional application of artistic processes in cultural industries, emphasizing the creation, production, and commercialization of creative content. It prepares students to develop innovative goods and services in the creative sector by endowing them with a deep understanding of visual communication and functional and aesthetic design solutions, as well as through the integration of various techniques, including emerging technologies. In this context, students assume the role of UX designers by applying their artistic and technological skills to create user-centered digital experiences. It is important to note the combination of these artistic and technological skills, including those linked with coding and development capabilities, which enable designers to complement their creative profiles.

That is the case for the WA course, which was taught by lecturers in two different computing departments: one teaching programming language topics and the other addressing technological aspects in web infrastructure issues. Furthermore, this WA course was scheduled in the third year of the four-year degree mentioned earlier, as shown in Figure 1, which displays a timeline of the course and its relationships with other courses. Figure 1 shows the temporal evolution of courses relevant to the development of web applications. During the first year, there was a compulsory course called Programming Basics that introduced some basic notions of programming using Processing [36] as the language for creative coding. Processing, which is based on the Java language/framework, provides a free graphics library and integrated development environment (IDE) tailored to electronic arts and visual design. It also allows non-expert programmers to work with the fundamentals of computer programming in a visual context.

In the second year, the Interactive Media course helped students acquire knowledge and skills required for interacting with narrative artistic elements using web technologies such as HTML5, CSS, and JavaScript and physical art installations based on Arduino components. In the 3rd year, the students were taught several elective courses dealing with the creation of web artifacts from different perspectives. During the first semester, the User Interface Design and Web Design courses provided a front-end perspective focused on visual aspects and introducing UI (User Interface) notions. The Web Applications (WA) course took place during the second semester and included topics related to back-end aspects that complemented the previously presented front-end perspective in the Web Design course (considered a recommended course). In the 4th year, there was a final compulsory course that promoted the integration of web and XR (extended reality) technologies into the production of interactive UX (user experience) projects. Naturally, other courses were taken over the course of this degree. The courses mentioned in Figure 1 were strongly connected with the WA course, which is the main research target.

Figure 1 also provides a graphical representation of the WA course curriculum, which was divided into three main parts. The first part introduced some basic topics on technologies used to maintain the server infrastructure required to host a web application and some examples of these applications, highlighting the role of Content Management Systems (CMSs) such as WordPress or Joomla [37]. The second part consisted of a practical teaching strategy involving students in individual activities, whose corresponding assignments needed to be submitted by the dates shown in Figure 1. These proposed activities covered different aspects of the front- or back-end views in a web application. For example, a global perspective of both views could be achieved by building a sample WordPress website. Other activities required the students to write PHP scripts that input data via visual interfaces and connected to a database in order to store and manage these data items (back-end). The third part of the WA course was oriented towards compelling the students to work collaboratively via a final web project. The intervention in this case study involved a cohort of 21 students who participated in the application of GenAI tools during a 2nd-semester course. The following subsections outline the ways these tools were integrated as part of the teaching methods used in the course, the interaction framework for applying GenAI tools in specific course tasks, and, finally, the data analysis instruments used to verify the outcomes yielded.

3.2. Teaching Methods

The WA course, part of the Design and Creative Technologies Degree since 2021, was taught via a traditional teaching approach combining theoretical topics related to web applications with development activities in lab sessions. Last year, degree students started working with ChatGPT-4, which has become a popular AI tool for solving technical problems related to implementing complex PHP scripts or managing specific database operations. The lecturers, who are also the authors of this study, decided to adapt their teaching methods in order to integrate Generative AI techniques while maintaining the original structure of the course. Individual activities, scheduled as shown in Figure 1, had precise instructions, including a rubric document that contained a set of associated assessment criteria. These types of activities represented a suitable scenario for incorporating GenAI tools and guiding students in their application. For example, the Template-Session task (the 3rd individual activity) required the students to write a PHP script allowing a user to log in to a web page. Once the user signed in, he or she could initialize a PHP session to acquire a private view of the page. This strict sequence of steps could be structured as a prompt template, helping the students to interact with GenAI tools in an incremental way and leading them through a controlled learning path.

Moreover, the rubric associated with each task represented a way of informing the students about the steps to be followed during a task’s completion and how these steps would be assessed and scored.

Table 1. Template task rubric.

Assessment Criteria/Scores	A 100–80%	B 80–60%	C 60–40%	D 40–0%
Page structure	The page is complete and well structured based on the right template.	The page is not complete, but it is appropriately structured.	The page is not complete, and its code is not sufficiently structured.	The page is not complete, and its code lacks a template-based structure.
Multimedia content	Suitable multimedia resources have been used.	The number of multimedia resources is sufficient but could be improved.	Relevant multimedia resources are lacking.	No multimedia content has been included.
Graphical interface	The graphical interface is clear and offers unified content display in public mode.	The graphical interface is correct, but the public content display has some flaws.	The graphical interface is correct, but the public content display is not adequate.	The graphical interface is poorly designed.
Private access	The code allows user identification and access to the main website page.	The code allows user identification, but the user name is not shown on the page.	The code allows user identification, but the main page is not connected.	The code does not allow user identification or access to other pages.
Session management	The page uses sessions to show user ID during private access and includes a logout function.	The page uses sessions to show user ID during private access but lacks a logout feature.	The page uses sessions for private access, but user ID is not shown.	Sessions are not used.
Content interaction	The identified user can add and view comments on content items after logging in.	The identified user can add comments after logging in, but these comments cannot be viewed.	The identified user can add limited comments after logging in.	No special interaction occurs after logging in.

shows a sample rubric attached to the Template-Session task and displaying six assessment criteria, which were graded from A (the highest score) to D (the lowest score). The first three criteria were linked with the “public view,” and they were used to grade questions related to the completion of template page structures, the number of relevant multimedia resources, or the visual display of the provided graphical interface. The second set of assessment criteria referred to the sign-in/login identification process, the management of private session information once the user has been identified, or the user’s ability to interact with some content items on a web page. Following an established sequence of steps, the students not only had an accurate idea of their assessments but also an appropriate guide with which to formulate AI-based iterative prompts, serving as one of the innovative teaching methods proposed in this work.

The second part of the teaching methodology employed in the current work was based on group activities. This type of activity invited the students to collaborate on the development of website projects, which could include some of the functionalities previously implemented through individual activities. Project tasks could encompass typical UX stages, ranging from initial ideation and the analysis of its potential target users to final testing and project evaluation. The intermediate stages dealt with tasks such as designing the project information architecture, prototyping user web interfaces in the form of wireframes or high-fidelity mockups, or coding and developing different aspects such as User Interface (UI) components related to the front-end side or access to information stored in a database or multimedia resources in the back-end server. Some of these UX stages have already been scrutinized with the assistance of the Generative AI tools [38]; this study focuses on the potential of these tools to support the implementation of complex programming tasks.

At this juncture, it is important to highlight that innovative teaching methods may encounter challenges when addressing the development of web projects with moderate to high complexity, even when supported by Generative AI technologies. The next subsections describe some of the tools and instruments employed to support the use of these technologies and evaluate their application in the context of web development courses. This context entailed confronting the challenge of using these technologies in complex software projects that usually involve the cooperation of multiple actors. However, a collaborative use of GenAI tools is beyond the scope of the current research, but it should be considered for future work.

3.3. GenAI Interaction Framework

In line with the introduction of Generative AI tools and the support they could offer in addressing our research questions, a framework named AI4WD (Artificial Intelligence for Web Development) was established. In a previous work [38], a prompt-engineering strategy based on the exchange of Voila AI-assistant messages [39] was formulated to facilitate UX ideation processes and prototype design tasks. In this research, a similar strategy was employed by providing task instructions, which should help students generate iterative prompts in order to develop certain web functionalities. The purpose of AI4WD was not to deliver a specific sequence of prompts so as to automatically produce a website project, but to help students generate their own prompts that could interact with tools such as ChatGPT-4 or GitHub Copilot. Moreover, the instruments included in AI4WD enabled us to assess the suitability of the generated prompts and monitor their outcomes.

The idea of using GenAI techniques and tools for effective SW development has received widespread attention (see the Section 1). Li et al. [4] advocated for the use of ChatGPT along with prompt-engineering techniques for rapid code development. They noted that current LLM applications “lack guidance for practical software development process” and proposed an environment that was structured as three main services: (i) inputting the original prompt containing the main functional requirements and including references to some technological components; (ii) configuring this original prompt; and (iii) employing a self-debugging optimization process to improve the results. This proposal is somewhat similar to the one offered in the AI4WD framework, despite some differences in how the sequences of required prompts are built. Originally, a skeleton of the prompt template was provided to the students to assess their knowledge of web UI concepts. They were invited to use ChatGPT to complete this skeleton with their own prompt contributions, test the outcomes obtained, and return them to the lecturers, as described in the Section 4. This preliminary step was considered very useful for gaining an initial insight into the students’ perspectives, although ChatGPT demonstrated that, at least in the free version, it lacked mechanisms with which to easily export and manage the outcomes produced. Instead, an alternative GenAI tool (GitHub Copilot) was evaluated for incorporation in the AI4WD context. GitHub Copilot (GC) [40] is a code completion and automatic programming tool that assists developers via Visual Code editors. Within these editors acting as IDEs (integrated development environments), GC can be integrated with several LLMs through plugin and extension mechanisms. Figure 2 shows a screenshot that displays the use of GC with the chat extension [41], represented on the left side of the image, to ask prompts in a programming context (right side).

The students, who were initially unaware of GC, rapidly realized how easy it was to use and quickly discovered its capacity to generate code automatically. The first observation consisted of confirming the need to assist Creative Design students in the formulation of prompts to guide web development tasks. The students usually employed the “trial and error” technique, constituting a major reason for facilitating the preparation of prompt templates, serving as one of the main AI4WD pillars. After a short tutorial on the GC features (particularly those related to chat prompt mechanisms), the students were invited to carefully read a set of task instructions (e.g., those related to the Template-Session task) and organize the sequence of prompts they should formulate to accomplish the task. These task instructions were important not only for guiding the students in their academic work but also for providing a sort of benchmark that could be used for comparison with the prompt sequences submitted by the students. This benchmark summarized the instructions provided for each task and served as a useful analysis instrument. In this sense, GC’s ability to export/import chat logs in JSON format [42] was invaluable for tracking and processing student interactions, another fundamental pillar within the AI4WD framework.

3.4. Data Analysis

We obtained chat logs from GC–student interactions, composed of a variety of data corresponding to the user prompts formulated and their resulting outcomes in the form of generated pieces of code or technical reports explaining their generation. The analysis of these logs was extremely useful in uncovering the details of human–AI interactions, but the logs had to be cleaned and processed beforehand. Figure 3 shows the different stages of the entire analysis process, starting with the filtering of chat information by using Python 3.9 scripts to convert the chat logs into MD (MarkDown) notation to allow improved display and extracting student requests via regex expressions [43]. After the filtering stage, the logs were translated to English and summarized for subsequent processing. R and Python scripts were then used to load a udpipe (trainable pipe) model [44] that allowed for the annotation of the incoming text and the extraction of terms such as nouns and verbs. This stage played a crucial role in analyzing the actions discussed by the students (through verbs) and the objects (nouns) of such actions.

Once nouns and verbs were extracted as lemmatized terms, we retrieved a corpus of tokenized documents to build DTM (Document–Term Matrix) structures [45] in order to create a vector-space representation of these documents. These structures were scored by using TF-IDF (Term Frequency-Inverse Document Frequency) measures to retrieve the most meaningful terms and obtain a compact DTM. After these processing steps, LSA (Latent Semantic Analysis) procedures [46] were applied to achieve several relevant outcomes, such as a graphical display of term relationships and their semantic similarity. We used LSA, a statistical technique that models word usage patterns to measure semantic relationships and compare the degree of similarity between different pieces of text through these relationships [47]. A Python script [48] was adapted to compute this semantic similarity value by comparing logs produced by the students against the proposed benchmark in each experiment case.

In addition to these text analytics techniques, other methods were employed in assessing the outcomes obtained, such as usability tests and user questionnaires. The usability tests were oriented towards evaluating certain aspects of the web pages designed by the students [49], while the user questionnaires were prepared to collect the students’ opinions regarding the use of GenAI tools in their courses.

4. Results

Several experiments were performed in order to apply the methods and tools introduced in the previous section. First, the integration of GenAI techniques into the proposed WA course was examined. A second batch of experiments processed student interactions gathered as chat.json logs in different stages of the WA course. Finally, a comprehensive evaluation of student scores, usability tests, and student opinions was implemented.

4.1. GenAI Integration

The first set of experiments was used to assess the integration of GenAI mechanisms into key aspects of instructional tasks within the WA course. One of the most interesting aspects for creative design students was the elaboration of visual interfaces based on templates and their implementation as website pages. This question was also challenging since further stages would require the students to develop more examples of webpages and connect their content items to database records.

Figure 4 shows an example of a generic mockup based on templatetemo designs (https://templatemo.com/, accessed on 28 May 2025) that displays a sample web homepage composed of two main parts. In the upper area, the header contains a “Logo” title on the left side and a horizontal navigation menu with options such as Home, Browse, How it Works, Contact, and Pages, along with a user icon (enabling sign-in actions). The central part of the Homepage features a Hero section with a main title and a subtitle complemented by a search component and some card samples. The students could download this mockup image in a graphic format. The first experiment conducted to test the use of GenAI tools involved providing the students with a prompt template skeleton with some relevant web design aspects they were tasked with completing.

Table 2. Design prompt templates.

Category	Description
Preamble	I have a screenshot of a UI design, and I want you to analyze it and create a detailed html document based on the following criteria.
Structure	Identify the specific components within the provided design, such as [to be completed], and break down the structure of each component using aspects such as [to be completed]. Focus on individual component layouts rather than large sections like headers or footers.
Style	Describe the visual styles, including [to be completed]. Mention any noticeable visual elements that could be included in the current design.
Interaction	Identify any potential interactions or animations, such as [to be completed]. Specify what should happen when the identified elements are hovered over or clicked or focused on.
Content	Provide details about the content within the design, including [to be completed with any visible or potential data items]. Describe attributes, such as [to be completed], of these data items.

shows a list of aspects the students could use to prompt ChatGPT, a well-known GenAI tool.

Figure 5 shows a plot of four bar charts that display the frequency of terms categorized by the items defined in Table 2. These charts were obtained from the responses of nine students who experimented using the proposed prompt skeleton and completed the main web components described in the template. Regarding the Structure category, the most mentioned term by the students was Card, which represented a suitable unit for organizing content items. In the case of the Style category, the students were mostly concerned with the visual appearance of buttons, and Focus was selected as the favorite concept for capturing user attention. The last category was Content, in which Text was the recurrent term.

In further experiments, we used this starting point to provide a sample UI specification from the mockup displayed in Figure 4 that the students could use to complete their course tasks. In these new experiments, GenAI techniques were applied based on using the GC tool to develop several tasks (either individual or group-led) and gather information from student interactions. The first set of tasks was proposed to help the students develop a front-end UI view based on PHP templates, and it was complemented by a “sign-in” login procedure. Furthermore, the students were invited to work with back-end mechanisms concerning database connections or access operations to retrieve data using SQL expressions. A second set of tasks dealt with group activities that involved linking UI components with data items. These kinds of activities were more sophisticated since they required a higher degree of coordination between the students in charge of developing front-end modules and those involved in the database (back-end) access task. They also represented a more challenging task from the perspectives of GenAI integration and the preparation of suitable benchmarks for analysis. The next subsection describes the experiments conducted to collect data logs from the students’ interactions with the GC Chatchat in these task examples.

4.2. GenAI Interaction

A second set of results was obtained by gathering the outcomes derived from interactions between the students and the GenAI mechanisms (GitHub Copilot-assisted). These outcomes were based on two data sources: (i) the set of development artifacts produced by the students during their learning activities and (ii) the collection of chat logs derived from the GenAI interactions and exported as JSON files. These AI-based learning activities were a form of support for task completion (scheduled in Figure 1), and the rate of completion of these tasks was also computed. Before reviewing the associated academic results, a careful analysis of chat logs was performed, beginning with the visualization of extracted terms (e.g., nouns and verbs) and their relationships. Figure 6 shows an example of a graph displaying the relationships between the verbs represented in the prompt benchmark (green points) and the entity nouns involved (blue points) for the Template-Session task benchmark (see Appendix A). The resulting graphs displayed two different areas. The first, located in the upper -left corner, showed a dense network of concepts with generic entities such as “html”, “php”, or “function (located on the right-hand side of this area) and references to web components such as “header”, “section”, “navigation”, and “menu”. The second main area was located in the lower part of Figure 6, displaying more sparsely distributed relationships among entities that are more specific in the context of the target task, such as “session”, “username”, and “login”. Importantly, this type of graph could be useful as a support for comparing the relationship patterns exhibited by student–AI interactions (through their associated chats) and the task benchmark, a visual reference of which is displayed in Figure 6.

However, this graphical representation was difficult to use as an instrument for measuring the quantitative distance between the provided benchmark (which, in this case, is associated with the Template-Session task) and the outcomes obtained by the students in their chat logs. Instead, alternative instruments were used, such as those reported in the Data Analysis subsection (see Figure 3). For example, the computed term frequencies gathered from student chats were compared with the terms found in the task benchmark. This process is depicted in Figure 7, which shows the distribution of differences in the frequency of terms identified as lemmas by comparing the frequency of lemmas in student chats against the selected terms in the Template-Session task benchmark. This comparison is divided by lemma POS (Part of Speech) entities, such as nouns (displayed in blue) and verbs (in green). The accumulated frequency values for verbs did not show remarkable differences in student-based outcomes compared to the provided benchmark. However, notable discrepancies emerged for terms such as “session” or, especially, “login” with high negative accumulation (fewer than −20 occurrences), indicating that this particular concept was underestimated or neglected by the students in their chats.

4.3. Final Assessment of GenAI Outcomes

In previous experiments, an initial attempt to analyze students’ interactions with GenAI tools like GC was made. The analysis of term frequency in a specific task provided a first impression of how close the prompts submitted by students were in comparison with those provided as benchmark references. A second instrument, based on evaluating the semantic similarity of student chat logs against provided benchmarks, could contribute to a more accurate measurement. In this case, two examples of tasks were assessed in order to analyze the percentage of semantic similarity and the task completion rate. The first task consisted of the individual activity comprising assessing Template-Session concepts (already mentioned in previous sections), while the second one referred to a group activity based on defining an initial UI to access a prototype of project data items without a database connection. Figure 8 shows two charts comparing semantic similarity with task completion percentages in both activities. In the individual activity case (a), there were 11 student answers showing values of semantic similarity ranging from 62% to 86%. However, the level of correlation between the similarity and completion percentages was rather low (approximately 21%). The second analyzed case, in Figure 8b (a group-led activity with seven answers), displayed closer semantic similarity values (exceeding 80%), with a correlation rate of 60% between the similarity and completion rates.

Two additional group activities were analyzed concerning the work involving database items. The first activity was completed by seven groups that exhibited good performance in basic database operations, but the corresponding computed semantic similarity was quite irregular, with low values ranging from 30 to 60%. The second proposed activity was based on employing several PHP scripts to develop a sort of database administrator based on multiple CRUD (Create/Read/Update/Delete) operations. This was a new activity for the WA course and difficult to complete without the help of GenAI tools since it required a high level of expertise in database management. The students involved in this activity spent several hours completing it as a kind of final project/test, and the analysis of their chat logs revealed the difficulties these students encountered. These chat logs contained multiple messages indicating several error issues, and student interactions involving the management of these issues were difficult to track. This observation confirmed the general impression of the use of GenAI tools, which require careful prompt organization in order to yield effective results, especially as task complexity grows.

As the final part of this assessment stage, two questionnaires were used to collect feedback from users. The first one was administered to students in the Interactive Communication Projects (ICP) course (a fourth-year course) who examined web prototypes developed by WA groups. There were more students (about forty) in this course, and their projects were unrelated to those worked on by the WA students. Seven web projects were tested by 27 students (with a range of three to five evaluations per project). Figure 9a displays a bar chart that shows the results for four of the assessment criteria used for testing the web prototypes developed. These results were obtained by averaging the ICP students’ answers, revealing assessment values close to 80% in three categories, although the Interaction category received a lower score. The full list of questions is available in Appendix B. A second questionnaire (see Appendix C) was used to gather the WA students’ perceptions regarding the application of GenAI tools in their course. The bar chart in Figure 9b displays the average answers from fourteen students in four assessment categories that measured the suitability of applying GenAI, the ease of producing prompts in the course context, and the usefulness of and trust levels regarding the GenAI outcomes obtained. None of the assessed categories reached 80% agreement, and only 62% of the students, on average, provided positive opinions regarding the suitability of GenAI use during the course.

5. Discussion

This section reports on the various ways Generative AI tools could impact a set of learning activities within web development courses. In this case, the courses were tailored to students whose affinity for design was stronger than that for computing, so they required special consideration for their creative profiles and limited programming skills. The aspects discussed are organized according to the previously introduced research questions.

The first question (RQ1) deals with the way GenAI-based technology could be effectively used in or integrated into WD courses for designers. As mentioned in the Introduction, multiple studies have addressed the use of these technologies in several web development areas, although these studies mainly focused on ideation and prototyping stages [32]. As shown in the questionnaire results in Figure 9, the Creative Design students analyzed in this study often expressed their concerns about the impact of AI applications on these areas, but they were also fully aware of the benefits of using this kind of technology in terms of the dissemination of ideas or their rapid prototyping, particularly for web development, which involves close collaboration between designers and programmers, requiring their combined efforts. The students in the Creative Design program realized that their designs needed “to be moved forward”, and WD courses represented a good chance to assume a new hybrid role. This hybridization phenomenon regarding web designers and developers has revealed multiple challenges and opportunities [50]. In this case, the degree curriculum taught to students during the first years included Programming Basics and Interactive Media courses, providing them with a sound background of basic programming skills and HTML/CSS knowledge. However, they usually lack deeper notions related to database management, asynchronous access to web content (AJAX), and the usage of widespread API services. These educational programs currently lack information on these advanced programming technologies, but the deployment of GenAI tools could help provide students with a suitable introduction to innovative web mechanisms. In this study, we tested the use of the GitHub Copilot (GC) environment with respect to students who had limited practice with ChatGPT or Copilot tools. Non-expert students explored the possibilities of this kind of environment using their chat capabilities in technical tasks related to the creation of “workplaces” integrating multiple CSS files and HTML documents, pieces of PHP code to dynamically generate and organize web pages, or working with SQL statements to retrieve database items. GC also provided invaluable help in explaining pieces of generated code [51] or fixing programming issues that appeared during the development process in small-scale tasks. Moreover, the instructors reported that the GC tool led to a reduction in the frequency of minor technical inquiries among the students, thereby freeing them to help the students in other areas more aligned with a creative profile. As the students were improving their programming abilities, they requested involvement in projects with a higher level of complexity. Therefore, there was a continuous challenge regarding the integration of GenAI tools in this development context and the need to promote the application of these tools in completing more complex tasks. The projects also required close collaboration between front-end designers and back-end programmers, so new tools may be required to support such collaboration.

Before dealing with this collaboration challenge, we considered a second research question (RQ2) concerning the determination of what new interaction mechanisms could be considered in a GenAI context, with the students requesting prompts to develop increasingly more complex tasks. As mentioned before, the Creative Design students in this study were well aware of the potential of tools such as ChatGPT or Copilot, but they occasionally interacted with them in a disordered or unsystematic manner. They usually submitted prompts without a predefined plan, and one of the main goals of this study was to guide the students toward a more systematic and rigorous approach. Accordingly, an investigation into the creative skills required for prompt engineering should be considered to deal with this situation [52]. Experiments with GC Chats in individual WD tasks were conducted by providing the students with a set of instructions rather than a detailed list of “mechanical” prompts to be submitted. This idea was complemented by an incremental method to develop pieces of code by adding functionalities in a given order while avoiding a rigidly constrained plan. Importantly, the combination of the development instructions provided and the use of incremental methods helped to build benchmarks that could be used as a reference for comparison with further interactions. The analysis of the results gathered from the chat logs revealed that the students started to provide their prompts using a more organized strategy that was closer to the proposed benchmarks. The studied outcomes, such as the graphical display of web topic maps derived from these student interactions or the frequency counts of the used terms, indicated at least the willingness of students in this direction. Interestingly, the application of semantic similarity techniques highlighted that the students followed the instructions provided to formulate prompts, helping them achieve their academic endeavors. Alternative ways of analyzing the closeness between student prompts and benchmark proposals were explored, such as knowledge-based text analysis methods [53], but their values were less representative than those returned by the corpus LSA-based techniques. Nevertheless, the problem of comparing the semantic similarity of texts persists, and new opportunities could emerge with the use of LLMs to address this issue [54]. The way GenAI–user interactions could improve students’ performance in web or other programming areas is another challenging field, and more research should be carried out to address it, particularly when collaborative tasks are involved.

The final research question (RQ3) focused on how the use of these GenAI-based interactive learning technologies could be evaluated. As mentioned before, there were many AI–student interactions, which were measured in several ways, aimed at either evaluating individual or group-led learning activities. These interactions were gathered from GC Chat export capabilities, though employing manual procedures entailed a significant time investment. Automatic methods of collecting such chat logs should be implemented in order to obtain a more effective way of facilitating their evaluation. Despite the paucity of chat logs collected in each activity, they allowed us to observe how the students interacted with GenAI tools such as GC and helped them to solve the tasks they were assigned. In the case of individual tasks, semantic similarity measures offered a method of comparing GC Chat outcomes with a given benchmark, highlighting the moderately high values of the matching percentages obtained (about 78%, on average, for the Template-Session task). A group-based activity was also evaluated using this semantic matching method, yielding even higher percentages (close to 90%, on average), although it was difficult to differentiate each group member’s interactions. Thus, researchers should pay attention to collaboration mechanisms, which enabled a more meaningful evaluation of these GenAI–student-group interaction patterns. Furthermore, correlation indices between semantic similarity and task completion percentages were measured. In the case of the Template-Session task, the correlation index was rather low (about 21%), highlighting the difficulty of establishing a direct link between GenAI–student interactions and students’ academic performance. This research question thus inspires reflection on the potential split between the activities assisted by GenAI that could be evaluated using a formative assessment perspective and other complementary activities based on summative methods. Ethical issues regarding the impact of GenAI tools could be incorporated to determine the extent to which academic grades were based on AI-assisted tasks. In addition to discussion about formative vs. summative assessments, other evaluation criteria could be considered to analyze the intervention of GenAI tools. The first impression inspired by this course experience is that the students needed less time to finish their tasks, indicating, at least partially, the GenAI tool’s effectiveness. Furthermore, the decrease in the number of technical questions the students asked during these tasks also indicated greater autonomy in their learning processes. A more systematic evaluation of these aspects could be performed in order to measure student productivity. Finally, qualitative measurement methods, such as assessing the quality of web projects developed by students, should also be considered. In this sense, the usability tests used in collaboration with students from other courses were an interesting instrument for acquiring an initial assessment of this project’s quality.

6. Conclusions

This work presents several important aspects regarding the use of GenAI tools to support UX design students in web development courses. We focused on a case study in a Web Applications (WA) course for students pursuing a Design and Creative Technologies degree, who were primarily UX designers with a combination of artistic and technological skills. In this context, the adoption of tools such as GitHub Copilot and ChatGPT represents a remarkable step toward democratizing access to programming resources and fostering their inclusion in different instructional settings. However, the full pedagogical potential of these tools has yet to be realized, especially in areas in which creativity, user experience aspects, and basic programming skills are central to the learning process. This latency was evident in the WA course, which, during the first semester of 2025, registered twenty-one students who participated in a set of individual and group-based activities dealing with web development topics.

One of the primary focuses of this work was exploring how these students were able to incorporate this type of GenAI tool through their instructional pathways. GenAI integration played a crucial role in supporting students in coding and developing web artifacts related to front- and back-end topics. Most of the students were familiar with ChatGPT technologies, facilitating the integration of new tools such as GitHub Copilot and their chat plugins integrated with the Visual Code Editor. They realized GC’s potential to generate code in response to simple prompts, even for organizing a set of files in which the generated code could be easily structured. The outcomes obtained by the students in the form of chat.json logs demonstrate seamless GenAI integration during learning activities. At least for individual, small-scale tasks, the observations in these logs show how GC provided significant help with code generation, explaining code, and fixing programming issues. The instructors also noticed a significant decrease in the number of minor technical questions asked by the students, suggesting increased autonomy in their learning processes. Nevertheless, challenges remain in applying GenAI for more complex tasks, such as in group activities, and the need to articulate AI-supported collaboration mechanisms between front-end designers and back-end programmers has become evident.

Regarding the introduction of innovative interaction mechanisms, an ordered application of these mechanisms in the form of prompt skeletons and instruction benchmarks has revealed multiple benefits that can guide some of the proposed instructional activities. GenAI played a pivotal role in scaffolding the learning experience through several stages in the web development process, helping the students deal with increasingly complex tasks. For example, the students experienced a reduction in cognitive overload when managing multiple design ideas and exhibited the ability to build web prototypes and test database operations in a faster, yet controlled, manner. An analysis of student–AI chat logs, including graphical displays of term relationships and frequency accounting of the terms, indicated that the students have begun to adopt a more organized prompting strategy closer to the proposed benchmarks. Semantic similarity techniques further highlighted that the students followed the instructions provided, aiding them in completing their academic tasks. Moreover, the use of such benchmarks provided a suitable instrument for assessing GenAI’s impact on students’ activities. Several measurement techniques were used to assess and evaluate this impact, such as counting the frequency of selected terms and computing the semantic similarity between student prompts and the benchmarks provided. These measurements were combined with a review of task completion rates as well as questionnaires submitted to students in a fourth-year course, who tested the usability of the prototypes developed by the WA groups. This combination of results confirmed the suitability of the current instructional approach and the benefits of using GenAI in this educational setting. Nevertheless, open issues regarding the study of users’ interactions with GenAI tools should be taken into account, including issues of overreliance on these tools or a lack of reflection, constituting pitfalls into which students could fall when overusing these tools.

In future work, we plan to extend the use of GenAI tools to new courses dealing with web topics ranging from basic design concepts to more advanced development platforms. These experiences could include new programming methods, including those based on agile methodologies, and incorporate developer platforms like GitHub, potentially helping to foster better collaborative skills. Consequently, the application of GenAI tools that can be shared should be examined, as they are required to fulfill such teamwork requirements. Future research should focus on collaboration mechanisms to allow a more meaningful evaluation of GenAI–student group interaction patterns. In this sense, automatic methods of collecting chat logs should be employed to make these logs’ evaluation easier and more effective. Additional research is required in order to reveal how GenAI–student interactions can improve students’ performance in web development or other programming areas. Therefore, a more systematic evaluation of these interactions should include student productivity metrics or incorporate measurements assessing the quality of web projects developed by students.

In conclusion, integrating GenAI into design-oriented web development courses is both feasible and potentially transformative. With careful pedagogical framing, collaborative task design, and critical engagement, GenAI can serve not just as a coding assistant but also as a catalyst for broader student participation in developing web artifacts, especially among learners traditionally marginalized by technical barriers.