Integration of UX Design Guidelines in the Requirements Engineering Lifecycle for Generative AI Solutions

Abstract

This research arose from the lack of methodologies that systematically integrate user experience (UX) guidelines into the requirements engineering phase of generative artificial intelligence-based solutions, especially in educational environments with technological gaps and diverse stakeholders. A practice based on the Essence standard graphic notation was designed, which guided four key activities: understanding UX guidelines through stakeholder needs, specifying and aligning requirements with verification criteria, component testing, and user validation. To this end, a set of techniques and artifacts was defined. The experimental validation, employing a between-subjects design, utilized standardized questionnaires (SUS and SUPR-Q) and nonparametric analyses. The results show that the experimental group achieved significantly better scores in perceived complexity and ease of use, as well as greater consistency, demonstrating the effectiveness of the proposed practice.

1. Introduction

Integrating artificial intelligence (AI) into educational technology has presented transformative opportunities and significant challenges. Over the past decade, advances in generative artificial intelligence (GenAI) have enabled systems to produce high-quality text, images, and other media outputs with minimal human intervention [1,2]. These systems—from language models such as GPT-3 and GPT-4 to multimodal applications—have increasingly penetrated educational settings, promising personalized content, adaptive learning environments, and efficient administrative support. However, while technical capabilities have advanced, the early stages of the development lifecycle often overlook critical aspects of UX design and human-centered approaches [3]. This oversight can compromise the quality, trustworthiness, and adoption of interactive systems based on GenAI, particularly in educational contexts where diverse stakeholder needs and socio-economic constraints are pronounced [4].

The intersection of GenAI and UX represents a nascent yet crucial area of study [5]. On the one hand, technical innovation in GenAI continues at a breakneck pace; on the other hand, the methodological frameworks for integrating UX principles within the requirements engineering process remain underdeveloped [6]. Historically, the field of human–computer interaction (HCI) has underscored the importance of iterative design processes that incorporate user feedback early and throughout the lifecycle [7,8]. Simultaneously, the emerging discipline of HCAI emphasizes that artificial intelligence should augment human capabilities rather than replace them [9,10]. The convergence of these perspectives necessitates a rigorous approach that captures technical requirements and embeds UX guidelines to ensure that the resulting GenAI solutions are both functionally robust and emotionally and cognitively aligned with users’ expectations.

In educational settings, this integration becomes even more imperative. Academic institutions, especially those in regions facing digital divides—such as the Valle del Cauca in Colombia—must contend with limited access to state-of-the-art digital resources and infrastructural disparities [11]. Incorporating UX guidelines in the early design phases can facilitate the development of inclusive, accessible, and adaptive systems that mitigate these challenges while enhancing pedagogical practices. For instance, tailored GenAI solutions can support administrative processes, improve instructional delivery, and offer personalized learning. Furthermore, in educational settings where infrastructure limitations, such as limited connectivity or scarce cloud computing resources, hinder the adoption of GenAI solutions, implementing Edge systems with generative capabilities presents a practical alternative [12]. By allowing local processing of generative models, these approaches can boost system independence, lower latency, protect privacy, and improve operational resilience, especially in rural or disconnected areas [12,13,14].

Despite the rapid technological advancements in GenAI, a substantial gap remains in the systematic integration of UX design principles within the requirements engineering (RE) phase of system development. In many cases, the development of GenAI solutions in education is driven by technical imperatives without a concurrent focus on how end-users—students, teachers, and administrative staff—interact with and perceive these systems. Despite technically innovative solutions, the lack of established practices for embedding UX guidelines can result in systems that are often misaligned with user needs, leading to issues such as low trust, inadequate usability, and ultimately, poor adoption [15,16].

The underlying challenge is twofold. First, a comprehensive set of UX design guidelines must be defined and specifically adapted to GenAI environments. This involves rethinking traditional UX methodologies to account for AI outputs’ probabilistic and generative nature and the inherent uncertainties associated with machine-generated content [17]. Second, there is the methodological challenge of integrating these guidelines into the RE process—a phase traditionally dominated by technical and functional specifications. The primary objective of this research was to develop, implement, and validate a practice for incorporating UX design guidelines into the RE lifecycle for GenAI solutions. Considering the research question: How can user experience (UX) be systematically integrated into the requirements engineering phase of GenAI-based solutions, especially in educational contexts with technological gaps and diverse stakeholders? This study outlines a set of interconnected actions aimed at enhancing UX design in GenAI systems, from a practice described in the graphical notation language of Essence [18]. The presented work’s methodological contribution is the creation of a novel Essence-based RE practice for GenAI, filling a recognized gap in how UX design principles are integrated early in AI system development.

This article is organized as follows: Section 2 presents the background. Section 3 describes the materials and methods applied in this research. Section 4 presents the results and discussion, and finally, Section 5 presents the conclusions and future work.

2. Background

Generative artificial intelligence (GenAI) refers to a class of models that leverage deep learning techniques to generate content—be it text, images, audio, or video—based on patterns learned from vast datasets [1,19]. These systems have evolved from early experiments in machine learning to sophisticated models such as GPT-3/4, Llama, and others that exhibit remarkable proficiency in natural language understanding and generation [2]. GenAI offers transformative possibilities in educational contexts, such as automated content creation, personalized tutoring, intelligent assessment, and adaptive learning experiences [20].

However, to the same extent that GenAI offers opportunities, it also poses risks associated with reliability, bias, and the overall quality of user interactions in educational settings. Several studies have documented the challenges of ensuring AI-generated outputs align with pedagogical goals and ethical standards [21,22]. In particular, these systems’ lack of an integrated approach to UX design has been highlighted as a major contributor to user mistrust and suboptimal learning experiences. As educational stakeholders increasingly rely on digital technologies to bridge the education gap, the need for both functionally robust and user-friendly systems becomes paramount.

UX design encompasses the entire spectrum of human–computer interactions, focusing on how users perceive and interact with technology. Traditional UX principles emphasize usability, accessibility, and esthetic quality [23]. In the context of AI-driven systems, however, UX design must also address the unique challenges posed by the probabilistic nature of AI outputs and these systems’ dynamic, adaptive behavior [24].

Recent literature has begun to explore the intersection of AI and UX. Researchers have argued that as AI systems become more autonomous and decision-making processes more opaque, it is crucial to design interfaces that foster transparency, trust, and user control [9,25]. The principles of HCAI advocate for an approach where AI serves as an augmentation of human capabilities rather than a replacement, thus necessitating interfaces that are intuitive, supportive, and aligned with human values [26,27]. Moreover, Value Sensitive Design (VSD) studies have underscored the importance of integrating ethical considerations into the design process. VSD provides a framework for embedding human values—such as privacy, autonomy, and fairness—into technology design from the outset [15,28]. This is particularly relevant in educational environments, where the stakes involve the effectiveness of learning and the protection and empowerment of vulnerable populations.

Requirements engineering is a critical phase in the development lifecycle where system specifications are defined, and user needs are articulated. Traditional approaches to requirements engineering have focused on functional and non-functional specifications without a deep integration of UX considerations [29,30]. Recent advances suggest that embedding UX guidelines at this early stage can result in systems that are more responsive to user needs and better aligned with stakeholder expectations, especially for GenAI solutions [17,31].

Empirical research on the integration of UX principles in GenAI-driven systems is emerging. Liu and Siau examined the effects of GenAI integration on user interaction in metaverse environments [32], noting significant improvements in user satisfaction when transparency and contextual adaptation were prioritized. Similarly, studies comparing control groups with and without UX interventions in AI systems have demonstrated that systems designed with integrated UX guidelines tend to perform better in terms of usability, trust, and engagement [33].

In the educational domain, several case studies have reported the benefits of UX-focused design in improving learning outcomes and facilitating technology adoption. For instance, initiatives involving interactive multimedia experiences have shown that well-designed interfaces can increase student engagement and foster a positive attitude towards digital learning tools [34]. Yet, these studies highlight that many GenAI applications remain underutilized or even rejected by end-users without a systematic approach that embeds UX in the foundational stages of system development [35].

Despite the promising advances in both GenAI and UX design, a clear research gap exists regarding the systematic integration of UX guidelines into the RE phase for GenAI systems [36], particularly in contexts characterized by digital inequities, such as education in emerging economies. The literature suggests that the early inclusion of UX considerations can improve system performance and greater user satisfaction [37]; however, concrete methodologies and validated practices remain scarce [3].

3. Materials and Methods

3.1. Methodology

The procedure followed in designing the practice for incorporating UX design guidelines into the development of GenAI-based assistants within the RE lifecycle was based on the stages of the design science research methodology (DSRM) [38]. Specifically, the DSRM framework of Peffers et al. [39] was adapted to the following six stages to design the practice:

• The identification of a problem associated with the incorporation of UX design guidelines in the development of GenAI assistants.
• The formulation of an objective related to the application of a set of UX design guidelines for the development of systems based on GenAI.
• The design of a solution that enables UX design guidelines to be applied in GenAI system development through a practice based on Essence, the standard notation language in software engineering [18]
• The application of the practice for the inclusion of UX design in the design phase of a GenAI system.
• The evaluation of the results achieved through the application of the practice.
• The socialization of the results achieved through the application of the practice.

Figure 1 illustrates the described stages.

This process initially involved conducting a state-of-the-art review to position the problem and its application context accurately [3]. Subsequently, an objective was formulated to define a set of UX design guidelines for developing GenAI assistants. However, this definition alone did not sufficiently clarify its practical application throughout development [17]. Therefore, the next step involved constructing an artifact that integrated these guidelines within a development framework. This effort led to the design of a practice that incorporates UX design into requirements engineering for the development of GenAI-mediated solutions.

3.2. Practice for the Inclusion of UX Design in the Development of GenAI Systems

3.2.1. Principles of Practice

As a result of the above-described needs, a practice was designed to guide the integration of UX design guidelines into GenAI-based systems. This practice follows a process that supports the analysis, definition, application, specification, testing, and validation of the various guidelines through the set of recommendations embedded in each one, all within the RE lifecycle. It was deemed appropriate to embed this practice within the lifecycle, as it addresses the guidelines through stakeholder needs while analyzing, discussing, and negotiating the solution’s requirements. In this way, the aim is to ensure that any process involving the analysis, discussion, and potential application of UX design guidelines is always approached from a perspective that frames the process through the lens of stakeholder needs. This enables consideration of how UX design guidelines can contribute to a more satisfactory fulfillment of stakeholders’ needs, expectations, and motivations regarding the GenAI-based solution. For the design of this practice, the graphical notation language from the software engineering standard Essence [40] was employed.

An important element underlying the design of this practice is grounded in the principles of the HCAI discipline [41]. Specifically, it is essential to adopt a lifecycle associated with HCAI. In this regard, several recognizable approaches can be found in various frameworks proposed in the literature [10,26,42,43] that align with HCAI principles. Nevertheless, it is also evident that many AI-based solution development projects have failed due to weak consideration of human factors [44]. This failure sometimes stems from unmet user needs, the lack of user experience design, or the absence of AI explainability, among other factors.

Within the scope of this study, particular attention has been given to the framework proposed by Xu [6], as a consistent approach to an HCAI lifecycle. This proposal is structured around three fundamental dimensions: (1) users, which focuses on understanding and prioritizing the needs, experiences, and expectations of end users in the design and development of AI systems; (2) ethics, which ensures that the development and deployment of AI are carried out ethically and responsibly; and (3) technology, which encompasses the development and implementation of AI solutions that are technically viable and advanced.

Building on the references above, particularly those that address ethical and moral inclusion factors from an HCAI perspective, a practice was conceived for incorporating UX guidelines into the process that steers the development of GenAI-based systems. This practice focuses on analyzing and identifying factors associated with UX design guidelines, which can be considered in light of stakeholder needs and integrated into the RE lifecycle.

Considering the previously discussed elements, the objective of this practice is centered on the specification, implementation, verification, and validation of UX design guidelines for developing GenAI-based systems. This practice takes as its reference point the established set of UX design guidelines for GenAI [17]. It aims to conduct a process of analysis and definition of the UX design principles to be considered in the system’s development as the outcome of a collaborative analysis and discussion process with stakeholders. Using patterns from the graphical notation language of the software engineering Essence standard [40], Figure 2 illustrates the various components encompassed by the practice for specifying UX design guidelines in GenAI.

Figure 2 presents several important principles that warrant clarification, as they form the foundational basis of the designed artifact:

• The practice is founded on the Requirements Alpha of the Essence standard. This is due to its lifecycle-oriented approach, which begins with understanding the UX design guidelines for GenAI from the perspective of identified needs. It then progresses through the alignment of these guidelines with the system’s requirements and verification criteria, culminating in end-user validation of the GenAI-based system.
• The understanding of UX design guidelines for GenAI, based on user needs, is inspired by the foundational principle of this proposal, which is aligned with the HCAI discipline [17]. This principle holds that UX design guidelines exist to enhance users’ perception of the benefits that the system can provide through its solutions, always aiming to exceed their expectations. Conversely, the guidelines are not intended to act as elements that constrain, limit, or restrict the needs expressed by users.
• Work teams that adhere to UX design guidelines in developing GenAI-based system solutions are responsible for establishing the interrelation between design guidelines, user needs, requirements, and verification criteria for the components that comprise the system as a solution. They must be capable of ensuring the traceability of the guidelines through the implemented requirements and conducting their verification at the level of each solution component.
• Component-level verification of UX guidelines for GenAI is essential, as it is important to clarify that their scope extends beyond aspects solely related to the user interface (UI). The impact of UX design guidelines for GenAI may influence considerations such as fine-tuning large language models to address issues related to bias, lack of transparency, privacy, ethical dilemmas, prejudice, and discrimination, among others. They also encompass elements relevant to prompt engineering and user interface design. Therefore, component-level verification is considered critical to ensure the effective traceability of the correct application and implementation of the guidelines across the different abstraction levels of the GenAI system, where they were applied.
• Finally, advancing toward effective user validation is crucial in determining whether the application and implementation of the GenAI system’s UX design guidelines yield tangible benefits for the solution’s users.

3.2.2. Activities

Based on the principles described above,

Table 1. Activities included in the UX-GenAI practice.

Code	Activities	Description
A1	Understanding UX design guidelines for GenAI through stakeholder needs.	This activity is based on two principles: the first relates to the prior effort required to understand stakeholder needs regarding the solution and comprehend UX guidelines through the lens of user needs. The second principle assumes that the stakeholder is a non-expert in GenAI, possessing limited knowledge about the possibilities that a solution based on this technology can offer.
A2	Integration of UX Design Guidelines for GenAI into the Specification of Requirements and Their Verification Criteria.	This activity should lead to the integration of system requirements and their verification criteria with the UX guidelines, generating an alignment that enables the visualization of how the implementation of various UX design guidelines for GenAI is supported through the requirements and their associated verification criteria.
A3	Verification of Design Artifacts Associated with UX Guidelines for GenAI at the Component Level.	The UX design guidelines for GenAI systems have a significant scope across various levels of system abstraction. Various aspects of these guidelines can be considered, for instance, at the level of large language model design, prompt engineering, user interface design, and other relevant dimensions that may depend on the characteristics and scope of the solution development project.
A4	Validation of UX Design Guidelines for GenAI within the System.	This activity involves the validation process associated with the GenAI UX design guidelines, conducted with the solution’s potential users.

presents the activities comprising the UX-GenAI practice and their corresponding descriptions.

3.2.3. Techniques

Table 2. Techniques that enable the execution of activities in the UX-GenAI practice.

Code	Techniques	Recommended Steps	Recommended Tools
TEUX1	Ideation and Adaptation of UX Design Guidelines for GenAI Aligned with Need.	- Conduct a brainstorming session based on the UX design guidelines for GenAI, considering the identified needs. - Identify the factors associated with the UX design guidelines for GenAI that are linked to stakeholder needs. - Identify potential factors associated with the UX design guidelines for GenAI that may indirectly influence the fulfillment of user expectations based on their discussed needs. - Organize and classify the preliminarily identified direct and indirect factors according to the UX design guidelines for GenAI. - Communicate to stakeholders the outcomes achieved through the ideation and adaptation process.	Toolkit for the Application of UX Design Guidelines in GenAI-Based Systems.
TEUX2	Roadmap of Needs and UX Design Guidelines for GenAI.	- Analyze the ideation and adaptation process of the UX design guidelines for GenAI based on the identified needs. - Create a map that interrelates the UX design guidelines for GenAI with each defined need. - Document the factors associated with the UX design guidelines for GenAI that will be considered in the solution. - Present to stakeholders the results of adapting the UX design guidelines for GenAI based on user needs.	Preliminary Matrix of Factors vs. Needs.
TEUX3	Alignment Between Requirements, Verification Criteria, and UX Design Guidelines.	- Perform traceability of the factors associated with the UX design guidelines for GenAI that must be aligned with the system requirements based on their prior association with user needs. - Verify potential factors associated with the UX design guidelines for GenAI that were initially linked to user needs but are currently not aligned with system requirements. - Define the final set of factors associated with the UX design guidelines for GenAI to be included within the scope of the specified system requirements. - Establish the verification criteria through which the proper implementation of the factors associated with the UX design guidelines for GenAI will be demonstrated. - Produce the alignment among requirements, guidelines, factors, and verification criteria. - Share the alignment matrix with stakeholders.	Alignment Matrix of Requirements, UX Design Guidelines for GenAI, Factors, and Verification Criteria.
TEUX4	Verification Test: Component–Requirement–UX Design Guideline.	- Define a test plan to verify the UX design guidelines for GenAI, ensuring consistency with the system requirements and components. - Conduct verification tests for the guidelines at the component level. - Identify failures and inconsistencies in the requirements and make the necessary adjustments to ensure that the implemented component meets the guidelines. - Document the results associated with the verification tests of the factors linked to the UX design guidelines for GenAI.	Software Verification Tools Adapted to UX Design Guidelines for GenAI.

presents the set of techniques that enable the execution of the described activities.

3.2.4. Work Route

The activities and techniques associated with the practice for incorporating UX design guidelines into the lifecycle of GenAI-based system development are also reflected in the roadmap. The illustration presented in Figure 3 depicts the activity flow within the scope of the practice, aiding understanding, especially among individuals who are not experts in software engineering. The activity highlighted in yellow suggests that the execution of the roadmap should begin with that activity. In contrast, the one in red is proposed as the final activity in the execution flow. It is important to note that although the roadmap suggests a sequential order for conducting the activities—to provide structure and coherence to the integration process of UX design guidelines in GenAI-based systems—the underlying process model of the practice is inherently iterative. This characteristic is essential and aligns with the principles of Human-Centered Design, a foundational element inferred from the framework proposed by Xu et al. [6] to support the emerging discipline HCAI.

3.2.5. States

The UX Design Guidelines for GenAI sub-alpha (see Figure 2) are intended to guide the UX design effort within the RE lifecycle. Accordingly, following the guidelines established by the Essence standard [18],

Table 3. States and checklist items corresponding to the UX guidelines for GenAI sub-alpha.

Alpha	States	Checklist Items
UX Guidelines for GenAI	Recognized	- The team has identified the need to establish a set of UX design guidelines for the new GenAI-mediated system. - A preliminary review has been conducted of the factors associated with UX design guidelines that should be prioritized based on the identified needs of the new GenAI-mediated system. - The team has discussed and agreed upon the foundational UX design factors in the context of the needs related to the new GenAI-mediated system. - Stakeholders are aware of and understand the UX design factors that will potentially be considered within the scope of the GenAI-mediated system’s needs. - An information base of UX design guidelines is defined and prioritized to specify the GenAI system requirements.
	Integrated	- The team has completed a preliminary list of requirements, embedding the agreed-upon UX design factors. - Alignment between prioritized requirements and UX design guidelines has been completed. - Alignment conflicts between requirements and UX guidelines have been resolved. - The team and stakeholders understand the benefits expected from implementing UX guidelines. - The team and stakeholders are aware of and accept the expected deliverables resulting from implementing UX design guidelines within the system requirements.
	Verified	- A list of system components mediated by UX design elements resulting from applying the guidelines has been completed. - The team has finalized and approved the UX design verification and testing plan for each system component. - Results of component-level verification tests for UX guidelines have been documented and are well understood. - Adjustments to components based on the UX guideline verification results have been completed. - The team and stakeholders are aware of and accept the adjustments made to components to meet the UX design guidelines for GenAI.
	Validated	- A completed list of requirements and UX guidelines to be validated with users in the GenAI system is available. - A dedicated team has been assigned to work with users to validate UX guidelines in the GenAI system. - Stakeholders have completed and approved a plan for validating UX guidelines in the GenAI system. - Results of UX guideline validation tests in the system have been documented and are well understood. - Adjustments to the GenAI system have been made and documented. - Users have confirmed and accepted the UX guidelines and system requirements for GenAI.

presents the sub-alpha states and main checklist items.

3.3. Design of the Evaluation Process

The evaluation was structured as a between-subjects controlled experiment, comparing an experimental group (which applied the integrated UX guidelines within the requirements lifecycle) against a control group (which followed the conventional development process without those guidelines). Such a between-subjects design is widely regarded in UX/HCI research for establishing causal impact since random assignment of participants to a single condition prevents learning or transfer effects between system versions and thus minimizes bias. We adhered to classical methodological recommendations for randomization and variable control in HCI experiments, ensuring that groups were equivalent at baseline. This approach has been applied in other studies that validate the use of GenAI in educational contexts [45].

Furthermore, the study adopted a quantitative approach to capture objective measures of user experience. Standardized UX questionnaires were applied, including the ten-item System Usability Scale (SUS), one of the most extensively validated instruments for perceived usability in interactive systems [46], and an adapted eight-item Standardized UX Percentile Rank Questionnaire (SUPR-Q), which holistically measures usability, trust, esthetics, and loyalty within an educational context. These quantitative metrics provided objective and comparable usability scores, allowing for a rigorous evaluation of the impact of the UX guidelines on user interaction with the GenAI assistant.

3.3.1. Validation Design

The validation followed a between-subjects experimental design comparing two distinct approaches to GenAI system development. The study evaluated three primary dimensions: usability, user experience, and utility, aiming to determine whether the assistant fulfilled the defined use cases, proved intuitive for different user profiles, and provided significant value in terms of efficiency and the quality of information delivery.

A quantitative approach was prioritized, specifically adapted to address the particularities of GenAI-based systems. The design prioritized flexibility and accessibility in the evaluation process, accommodating the participating users’ institutional requirements and availability constraints.

The experimental design contemplated two differentiated groups to evaluate the impact of the implemented UX guidelines:

• Control group (n = 8): Users without prior participation in the assistant’s definition and appropriation phase who interacted with a version that partially implemented the UX design guidelines.
• Experimental group (n = 7): Users who participated in the definition and appropriation process, utilizing a version of the assistant that fully implemented the UX design guidelines defined through the proposed UX-GenAI practice.

3.3.2. Experimental Setup

The evaluation was structured around three primary use cases defined for the educational context: (1) definition of concepts associated with educational management, (2) guides for completing processes in document management systems, and (3) suggestions for feedback in institutional self-evaluation processes. Six specific tasks were designed to comprehensively cover these use cases, from conceptual explanations to procedural guidance and a comparative analysis of institutional data.

Two distinct versions of the GenAI assistant were implemented in this study. The experimental version incorporated the complete set of UX design guidelines as defined through the UX-GenAI practice, including scope understanding sessions, bias detection mechanisms, adaptable open models, contextual help features, benefit determination processes, source identification capabilities, information persistence functions, and result validation mechanisms. The control version implemented only a subset of these guidelines, specifically adaptable open models, source identification, and information persistence, representing a conventional development approach without systematic UX integration.

Participants were recruited from the stakeholder community, including administrative staff, educators, and technical personnel who would be the end-users of the GenAI assistant. The participants previously agreed to participate in system development and evaluation activities for the institution’s technology integration initiative. Before the evaluation sessions, all participants provided explicit verbal consent for participation and recording, as documented in the session recordings.

3.3.3. Data Collection Instruments

Post-interaction survey: An online questionnaire was designed by adapting validated instruments, including the System Usability Scale (SUS) [47], and the Standardized User Experience Percentile Rank Questionnaire (SUPR-Q) [48], incorporating specific questions to evaluate GenAI-based systems. The adapted instrument assessed usability, satisfaction, perceived utility, and trust in the GenAI assistant.

All original data used in this study are publicly available in the GitHub repository https://github.com/juancamiloed1227/ux-guidelines-integration-data (accessed on 21 July 2025), enabling replication and independent verification of the results.

4. Results and Discussion

4.1. Statistical Results

The statistical analysis aimed to evaluate the differences between the Control and Experimental groups across multiple user experience dimensions assessed using Likert scale items (1 to 5). Given the ordinal nature of the data and limited sample size, normality assumptions were not assumed. Therefore, non-parametric methods were employed [49]. Descriptive statistics (mean, median, standard deviation, and coefficient of variation) were computed for each group to explore central tendency and variability. To formally test group differences, the Mann–Whitney–Wilcoxon test was employed for each item.

Additionally, boxplots were generated to visualize the distribution of scores across groups. All analyses were performed using R version 4.5.1. The significance level of α = 0.05 was adopted.

To compare two independent groups with ordinal data, the non-parametric Mann–Whitney–Wilcoxon test was applied [50]. This test assesses whether the probability differs from 0.5, indicating that the two distributions are not identical. The statistic was computed by either summing the indicator functions overall observation pairs or by summing the ranks assigned to the Experimental group. Under the null hypothesis, the expected value and variance are calculated using variance adjustments that account for ties, as commonly found in Likert-type scales. For sufficiently large samples, the distribution can be approximated using a standard normal distribution to compute the p-value. This test is particularly suitable for ordinal data because it does not require assumptions of normality or equal variances, making it more robust than the independent samples t-test.

Table 4. Descriptive statistics by group.

Statistic	Control	Experimental
Mean	3.99	4.60
Median	4.00	5.00
Standard deviation (SD)	0.96	0.75
Coefficient of variation (CV %)	24.1%	16.3%

presents the descriptive statistics of user evaluations for the Control and Experimental groups. The Experimental group achieved a higher mean score (4.60) and median (5.00) compared to the Control group (mean = 3.99, median = 4.00), indicating a more favorable overall perception. Additionally, the Experimental group showed lower variability, as evidenced by a smaller standard deviation (0.75 vs. 0.96) and a reduced coefficient of variation (16.3% vs. 24.1%), suggesting greater consistency in responses.

Figure 4 shows the distribution of Likert-scale scores (1 to 5) for each item by group (Control vs. Experimental). The Experimental group exhibited higher medians across all items, including Ease of Use (Median = 5), Recommendation (5), and User Trust (5), compared to the Control group medians of 4 in most categories. Items such as Task Speed and Response Clarity showed larger interquartile ranges in the Control group, suggesting greater variability in perceived performance. These patterns indicate that participants in the Experimental group consistently rated the system more positively and with less dispersion.

The Likert-scale analysis revealed notable differences in perceived usability between the Control and Experimental groups (see Figure 5). Participants in the Experimental group consistently reported higher levels of agreement across key dimensions such as Ease of Use, Frequent Use, Information Value, and Recommendation, with 100% indicating “Agree” or “Strongly Agree” in most categories. In contrast, the Control group displayed more dispersed responses, including a higher proportion of neutral and negative evaluations in items such as Perceived Complexity, Task Speed, and Response Clarity.

4.2. Analytical Discussion

Table 5. Comparison of item scores by group.

Variable	p-Value	Effect Size (r)	95% CI Lower	95% CI Upper	Significant
Perceived Complexity	0.0177	−0.613	−1.12	−0.107	Yes
Ease of Use	0.0437	−0.521	−1.03	−0.015	Yes
User Trust	0.1548	−0.367	−0.873	0.139	No
Task Speed	0.2055	−0.327	−0.833	0.179	No
Quick Learning	0.2261	−0.313	−0.819	0.194	No
Information Value	0.3112	−0.261	−0.768	0.245	No
Response Accuracy	0.3218	−0.256	−0.762	0.25	No
Frequent Use	0.3219	−0.256	−0.762	0.25	No
Recommendation	0.3219	−0.256	−0.762	0.25	No
Response Clarity	0.7136	−0.095	−0.601	0.411	No

presents the results of the Mann–Whitney–Wilcoxon test comparing the Control and Experimental groups across each of the evaluated features of the AI assistant. According to the results, the variables Perceived Complexity (p = 0.0177) and Ease of Use (p = 0.0437) showed statistically significant differences between groups (p < 0.05). These findings suggest that the experimental group perceived the assistant as less complex and easier to use compared to the control group. This difference may be attributed to factors such as pedagogical interventions, prior experience, or a more favorable interaction environment.

To complement the p-values and provide a more comprehensive interpretation of the findings, we calculated the nonparametric effect size $r=Z/\sqrt{n}$ for each comparison, along with 95% confidence intervals. For Perceived Complexity, the effect size was moderate to large (r = −0.613; 95% CI: [–1.12, –0.11]), while Ease of Use showed a medium effect (r = −0.521, 95% CI: [–1.03, –0.015]). These confidence intervals, computed under the assumption of simple random sampling, reflect the uncertainty around the magnitude of the observed effects. In scenarios involving complex sample designs, such as stratified or clustered structures, confidence intervals can be adjusted using the design effect (DEFF); however, in the present study, no such design correction was necessary due to the randomized assignment.

The remaining variables—User Trust, Task Speed, Quick Learning, Information Value, Response Accuracy, Frequent Use, Recommendation, and Response Clarity—did not show statistically significant differences (p > 0.05). Although some of these dimensions suggest visual tendencies between groups, the statistical evidence is insufficient to claim systematic differences. Nonetheless, these variables could be revisited in future studies with larger sample sizes or by using multivariate models that incorporate interactions among features.

The results suggest that the experimental condition had a specific positive effect on the perceived usability of the assistant, mainly in terms of simplicity and ease of use. These dimensions are crucial in the adoption of technologies and in improving the overall user experience.

Overall, the results highlight the importance of targeted pedagogical strategies and user-centered design in shaping favorable perceptions of AI-based educational tools. The observed improvements in usability perceptions—especially regarding cognitive simplicity and operational ease—are particularly relevant for increasing user adoption, engagement, and trust. Future work should investigate the longitudinal effects, interactions between multiple usability dimensions, and potential mediating variables such as digital literacy or motivation. These insights are essential for informing the iterative development of intelligent educational systems and maximizing their impact in diverse learning environments.

4.3. Limitations

Although the study provides valuable initial evidence, it is essential to acknowledge the limitations imposed by the small sample size (n = 15). While reduced samples are common in HCI and UX research, they constrain the statistical power of the analyses and limit the generalizability of the findings. The results should therefore be interpreted with caution, as they represent formative evidence rather than definitive conclusions. This limitation highlights the need for future studies to involve larger and more diverse participant groups, which would enhance the robustness of statistical inferences and enable broader conclusions about the effectiveness of integrating UX guidelines into GenAI system development.

Although the UX-GenAI practice showed significant improvements in perceived complexity and ease of use, eight of the ten evaluated dimensions (including user trust, task speed, information value, response accuracy, and response clarity) did not present significant differences between groups. This highlights a limitation of the study, suggesting that the effectiveness of the practice may be restricted to certain aspects of usability rather than producing broad improvements across all dimensions.

A notable limitation is the absence of prior studies that explicitly integrate user experience guidelines into the requirements engineering phase for generative AI systems in educational contexts. This prevents direct comparisons with equivalent research. However, rather than a methodological flaw, this gap represents a research opportunity: it confirms the emergent nature of the field and positions our proposal as an initial empirical effort to open a necessary line of inquiry. Accordingly, our results should be considered not only in light of analogous studies in related domains but also as a starting point that invites future work to replicate, contrast, and extend the proposed practice across different contexts and with larger samples.

Long-term evaluation is crucial for understanding how user perception and UX guideline effectiveness evolve as users develop familiarity with GenAI systems. Longitudinal studies of 6 to 12 months could reveal adoption patterns, changes in user trust, and the sustainability of initially observed user experience improvements. Beyond usability perceptions, it is crucial to evaluate whether UX improvements translate into tangible benefits, such as improved task performance, increased user retention, or enhanced learning outcomes, in educational contexts. This requires the development of impact metrics specific to GenAI systems that extend beyond the traditional usability measures.

5. Conclusions and Future Work

This study presented a practice for integrating UX guidelines into the requirements engineering lifecycle of GenAI-based solutions. The validation results demonstrate the contribution of the proposed practice.

The validation demonstrated that the application of UX guidelines through the proposed practice resulted in statistically significant improvements in two critical usability dimensions: perceived complexity (p = 0.0177) and ease of use (p = 0.0437). Participants in the experimental group perceived the GenAI assistant as less complex and easier to use than those in the control group. Additionally, the experimental group showed higher overall scores (4.60 vs. 3.99) and lower response variability (CV = 16.3% vs. 24.1%), suggesting a more consistent and favorable user experience.

From a methodological perspective, this work contributes to the emerging field of HCAI by providing a concrete and applicable artifact: the UX-GenAI practice. This practice enables development teams to integrate UX considerations from the early phases of GenAI system development, thereby addressing a gap in the literature. In this approach, the technical aspects of solutions must be articulated in conjunction with a user-centered design. This is especially important when designing solutions based on GenAI technologies, which have the potential to strongly impact human–computer interaction, either positively or negatively.

The findings of this study open multiple directions for future research that can expand both the theoretical scope and the practical applicability of UX integration in GenAI systems. Future research should extend validation to multiple contexts, helping evaluate the proposed guidelines generally, beyond the educational domain.

Future research should explore how the UX-GenAI practice can be integrated into broader frameworks for ethical and responsible AI, such as Value Sensitive Design (VSD), to ensure that user experience improvements do not compromise key aspects, including transparency, fairness, and privacy.

A relevant future line of research is the application of the UX-GenAI practice in the development of generative assistants deployed on Edge infrastructure. This architecture would enhance accessibility, safeguard the privacy of sensitive data, and enable functional autonomy in educational institutions with intermittent connectivity. Evaluating the user experience in Edge-GenAI solutions represents a natural and necessary extension of this study.