Abstract
Course designers routinely build online content with an intended level of user experience (UX) and gamification quality in mind, but whether learners’ affective responses actually match this intention is rarely measured directly. This study examines this question using a facial emotion recognition (FER) instrument integrated into Moodle that continuously records facial-expression confidence scores as students interact with course material. Twenty-five undergraduate students interacted, in a repeated-measures design, with four scenarios deliberately built along a low-to-high UX–gamification progression: E1 (low UX, low gamification), E3 (low UX, high gamification), E7 (high UX, low gamification), and E9 (high UX, high gamification). The single hypothesis tested was that the facial emotions detected by the instrument would be similar to the UX and gamification levels intentionally designed in each scenario. Sad was the dominant detected expression in the two low-UX scenarios (E1 and E3), and happy was dominant in the two high-UX scenarios (E7 and E9); a study-defined affective-valence indicator rose monotonically with the designed progression, from (E1) to (E3), (E7), and (E9). A Friedman test confirmed that there were significant differences across scenarios (, , Kendall’s ), and five of six Bonferroni-adjusted Wilcoxon comparisons were significant (the exception being E1 vs. E3), showing that the correspondence tracked the designed UX level more closely than the designed gamification level. These results support viewing facial-expression-based affective patterns as usable, HCI-oriented signals of whether learners actually perceive the design quality a course was built with.
1. Introduction
Course designers make deliberate choices about navigation, visual structure, content segmentation, and motivational elements such as points, badges, and progress indicators, all with an intended effect on the learner’s experience. Whether this intended effect is actually perceived, however, is usually inferred only indirectly through completion rates, grades, or post-hoc questionnaires that learners fill out after the interaction is already over. None of these instruments capture how the learner was responding while the interaction was happening, which leaves a gap between what a course was designed to feel like and the amount of evidence showing that it did indeed have the intended effect.
Facial emotion recognition (FER) offers a way to observe part of this response as it happens: webcam-based computer-vision models can continuously estimate facial-expression confidence scores without interrupting the learner’s task. When used this way, FER output should be interpreted as a probabilistic indicator of facial-expression patterns, not as a direct readout of internal psychological states; even so, if a course was deliberately built with a low or high level of UX and gamification quality, a useful question is whether the facial-expression patterns detected during interaction move in the direction that the design intended.
This study investigates this question directly using an FER instrument integrated into Moodle. It is browser-based, runs inside the course page, and continuously logs facial-expression confidence scores linked to the participant, scenario, and time. Twenty-five undergraduate students interacted with four Moodle scenarios that were purpose-built along a low-to-high UX–gamification progression: E1, E3, E7, and E9. A single hypothesis, presented in Section 4.1, was tested: the facial emotions detected during interaction will be similar to the UX and gamification levels intentionally designed in each scenario. The contribution of this paper is this design-evaluation evidence: a demonstration that the detected affect matches, at least for the UX dimension, the quality a course was deliberately designed to have.
2. Related Work
2.1. UX in Learning Management Systems
UX is a central dimension of LMS quality: learners interact not only with instructional content but with navigation, visual hierarchy, multimedia, activities, and feedback. Studies of Moodle and comparable platforms have linked UX characteristics to engagement, satisfaction, motivation, and acceptance [1,2], and heuristic evaluation, the User Experience Questionnaire, AttrakDiff, and learner-centered methods have consistently indicated that navigation clarity, instructional organization, and visual structure are decisive design factors [3,4,5]. Nearly all of these methods, however, rely on explicit, retrospective learner input rather than on signals captured continuously during interaction [6,7,8]. Eye-tracking is a further example of such a continuously captured signal, used to study visual attention and scanning behavior and, more recently, combined with physiological measures to evaluate learning experience directly [9]. This study instead treats UX as a manipulated design variable whose effect is read off a different continuous signal: facial-expression-based affect.
2.2. Gamification and Affective Engagement in Online Learning
Gamification embeds game-like mechanisms, such as points, badges, progress indicators, challenges, and motivational feedback, into non-game environments to shape participation and motivation, and prior work generally reports that there is a positive relationship between gamification and engagement [10,11]. This relationship is not unconditional, though: several studies argue that gamification only works when aligned with instructional goals and learner characteristics as opposed to being bolted on as isolated interface elements [12,13], and emotional engagement itself has been shown to depend on the surrounding interaction design rather than on gamification elements in isolation [14,15,16,17]. This raises a specific open question that motivated the present design: do the same gamification elements produce a different detectable affective pattern depending on the UX level they are embedded in? This is exactly the E1-vs-E3 and E7-vs-E9 contrast built into this study.
2.3. Facial Emotion Recognition as a Design-Evaluation Signal
FER has been used in educational settings mainly as an engagement- or emotion-monitoring tool layered onto a learning environment [7,18,19], sometimes combined with gaze, EEG, or other interaction signals to increase reliability [6,8], and is increasingly embedded in adaptive systems that respond to the learner’s affective state [20,21]. FER is used far less often as a comparative instrument for design evaluation itself, that is, as a way of checking whether an interface built to be better is actually interpreted as better in the facial-expression signal it produces rather than merely tracking generic engagement or emotion over time. Predictions from any such a pipeline remain sensitive to pose, lighting, and individual differences and should be interpreted cautiously [22].
2.4. Research Gap and Positioning
UX research, gamification research, and FER research each address part of this picture, but the specific question of whether FER-detected affect actually mirrors an intentional, multi-level UX–gamification design has received little direct empirical attention. This study addresses that gap with a controlled, repeated-measures comparison across four purpose-built scenarios, evaluated against a single hypothesis about correspondence between designed and detected patterns.
3. Materials: The FER Instrument
This study is one part of a broader Moodle-based research program, shown in Figure 1: a content layer, UX, gamification, and facial-emotion-recognition signals feeding a prompt set intended to guide generative-AI content creation, with an adaptive layer as future work. The present paper occupies the UX, gamification, and facial-emotion-recognition portion of that picture, examining how the last of these relates to the first two; the generative-AI prompt step itself is not evaluated here.
Figure 1.
Research focus within the broader Moodle-based architecture: UX, gamification, and facial emotion recognition.
The instrument used to collect the affective data analyzed in this paper is a facial emotion recognition (FER) component that was built directly into Moodle rather than operated as an external tool. Implemented in JavaScript with face-api.js and TensorFlow.js, it runs in a client-side fashion in the learner’s browser once webcam access is authorized and chains three pre-trained models, TinyFaceDetector, FaceLandmark68Net, and FaceExpressionNet, to sample facial-expression confidence scores roughly every 100 ms across seven categories (happy, neutral, sad, angry, fearful, surprised, and disgusted). Each reading is logged as a JSON record (confidence scores, timestamp, scenario, and participant) on a remote server, so the full interaction history can be reconstructed per participant and scenario; readings above a 0.90 confidence threshold additionally trigger an image capture used for manual inspection. The design, engineering validation, and functional testing of this instrument are reported separately; here, it is used only as a measurement tool, and its outputs are treated throughout as probabilistic facial-expression indicators rather than as direct measurements of internal emotional states.
4. Experimental Design
4.1. Goal and Hypothesis
The goal was to determine whether facial-expression-based affective patterns detected while learners interact with Moodle content correspond to the UX and gamification quality intentionally built into this content. This yielded a single hypothesis:
Hypothesis 1.
The facial emotions detected by the FER instrument are similar to the UX and gamification levels intentionally designed for each scenario (E1, E3, E7, and E9).
4.2. The Four Scenarios
A conceptual UX × gamification matrix (low/medium/high) defined the design space; the four corner points were selected such that they would provide the sharpest possible contrast—E1 (low UX, low gamification), E3 (low UX, high gamification), E7 (high UX, low gamification), and E9 (high UX, high gamification)—as shown in Figure 2. Intermediate points were left out to keep sessions short enough to avoid fatigue while still spanning the full intended range.
Figure 2.
The four corner configurations selected from the UX × gamification design space. Checkmarks (✓) mark the four selected corner scenarios (E1, E3, E7, E9); crosses (×) mark the intermediate configurations not implemented in this study.
E1 was the low-UX, low-gamification baseline; E3 added gamification while keeping UX low; E7 raised UX while keeping gamification low; E9 combined both at their high levels, with all four scenarios sharing the same topic, learning objectives, and core content (Table 1). ChatGPT (GPT-5.5, high reasoning effort; OpenAI, San Francisco, CA, USA) supported early prototyping only; final interfaces (Figure 3) were built and reviewed manually in Moodle.
Table 1.
Design characteristics intentionally varied across the four scenarios. Checkmarks (✓) indicate the element is present in that scenario; crosses (×) indicate it is absent.
Figure 3.
The four designed interfaces: (a) E1, low UX/low gamification; (b) E3, low UX/high gamification; (c) E7, high UX/low gamification; (d) E9, high UX/high gamification. Red boxes highlight the instructional, UX, and gamification elements implemented in each scenario.
4.3. Participants and Procedure
In accordance with the THEDRE research process [23], 25 undergraduate software engineering/programming students (mean age: ≈21) completed all four scenarios in the course “Las Maravillas del Ecuador” on EducaLab.EC in a fixed order, i.e., E1, E3, E7, and E9 (deliberately matching the low-to-high design progression being tested), spending roughly 45 min per scenario; independently; and without instructor intervention, while the FER instrument logged facial-expression confidence scores throughout. Because scenario order and design progression were confounded by construction, order effects are discussed as a limitation in Section 6. A broader post-interaction questionnaire was collected under the same protocol but is not analyzed here.
4.4. Turning Detections into a Design-Comparable Score
Detection covers seven facial-expression categories, but two were set aside for the comparison reported here: neutral, which was removed because its sheer prevalence would swamp any aggregate signal, and surprise, which was removed because its valence is ambiguous without further context. The remaining five, , were used to compute, for participant u in scenario s, a mean confidence per category and, from those means, a single affective-valence score,
an operational measure built for this comparison, not a validated psychological scale. With 25 participants each contributing one valence value per scenario, differences across the four scenarios were tested with a Friedman test (Kendall’s W), followed by Bonferroni-corrected Wilcoxon signed-rank pairwise tests () provided that a significant omnibus result was obtained. Dominant-category counts (Table 2) were computed the same way, per record and per participant, by taking the highest-scoring category in E.
Table 2.
Detected facial-expression dominance versus the designed UX–gamification progression. Bold values in the “Participant H/S/A/F/D” column indicate the most frequent per-participant dominant category in that scenario.
5. Results
5.1. Detected Patterns Track the Designed Progression
Table 2 shows that sad was the dominant detected expression in both low-UX scenarios (E1, E3) and that happy was dominant in both high-UX scenarios (E7, E9), according to mean confidence and record-level frequency alike. The same shift appears on a participant-by-participant basis: happy-dominant participants rose from 5 in E1 to 18 in E9, while sad-dominant participants fell from 15 to 5. Affective valence rose monotonically along the same E1→E3→E7→E9 path the scenarios were designed on, from to , , and .
Figure 4 and Figure 5 show, respectively, the raw temporal confidence traces and the per-scenario aggregates behind these numbers, which are included for descriptive context rather than as independent evidence.
Figure 4.
Temporal evolution of detected facial-expression confidence during interaction with each scenario. The horizontal axis shows time from the start of the session in minutes, and the vertical axis the confidence score in the range 0 to 1. Facial expressions are happy (green), sad (blue), angry (red), disgusted (violet), and fearful (yellow).
Figure 5.
Mean facial-expression confidence aggregated by scenario.
5.2. Statistical Confirmation
A Friedman test on affective valence confirmed the pattern was not incidental: , , Kendall’s . Table 3 reports the six Bonferroni-corrected Wilcoxon pairwise comparisons that follow up this omnibus result.
Table 3.
Post-hoc Wilcoxon signed-rank comparisons of affective valence between scenarios.
Every comparison that crosses the UX boundary (any E1/E3 scenario against any E7/E9 scenario) was significant, with large effects ( to ); E7 vs. E9, which differs only in gamification at the high-UX level, was also significant (). The single exception was E1 vs. E3 (, ), which differs only in gamification at the low-UX level.
5.3. Synthesis: Where the Correspondence Holds
Taken together, the descriptive dominance pattern, the monotonic rise in valence, and the pairwise significance pattern all point in the same direction: the detected affective signal tracks the E1→E3→E7→E9 progression the scenarios were designed on, which is the evidence of H1. The correspondence is not uniform across both design dimensions, however. Every comparison involving a UX change was significant, so the detected signal reliably distinguished low-UX from high-UX scenarios. Gamification’s effect was only detectable within the high-UX pair (E7 vs. E9) and not within the low-UX pair (E1 vs. E3), so, on the basis of this evidence, gamification’s contribution to the detected affective pattern is conditional on UX rather than independent of it. H1 is therefore supported overall, most strongly for the UX dimension and more narrowly for the gamification dimension.
6. Threats to Validity and Future Work
Scenario order (E1, E3, E7, E9 for every participant) was deliberately matched to the design progression under test, which is what makes the comparison interpretable, but it also confounds order with condition: learning or familiarization effects occurring in that same direction cannot be ruled out from this design alone, and future replications should test a counterbalanced or randomized order against this one. Each scenario bundled multiple design characteristics at once (Table 1), so the E1-vs.-E3 result shows that gamification alone did not produce a detectable shift within the low-UX condition, not that no single gamification element could ever matter; isolating individual elements is left to future factorial designs. The sample was a single course and 25 students at one institution, and the FER instrument’s pre-trained models were not independently benchmarked on this population, so confidence scores may be affected by pose, lighting, or webcam conditions; readings are treated as probabilistic rather than exact throughout. The 100-ms sampling rate also produces temporally correlated records, which is why all inference used participant-scenario aggregates.
Future work should replicate this comparison with counterbalanced scenario order, larger and more varied samples, and a factorial manipulation that separates UX and gamification more finely than the four corner points used here, to test directly whether gamification’s apparent dependence on UX level (Section 5.3) holds in a design where every combination of levels is observed.
7. Conclusions
This study asked whether facial-expression-based affective patterns, captured continuously by a Moodle-integrated FER instrument, correspond to the UX and gamification quality intentionally designed into four Moodle scenarios. They do, for the most part: sad expressions dominated the two low-UX scenarios, and happy expressions dominated the two high-UX scenarios; affective valence rose monotonically along the same low-to-high progression the scenarios were built on ( to ), and this pattern was statistically confirmed (Friedman , , Kendall’s , with five of six Bonferroni-adjusted Wilcoxon comparisons being significant), supporting H1.
The one area where the correspondence broke down is informative in its own right: adding gamification without also raising UX quality (E1 vs. E3) produced no statistically detectable shift in the affective signal, while every UX-crossing comparison, and the E7-vs.-E9 gamification comparison at the high-UX level, did. This suggests that gamification’s contribution to a detectable positive affective pattern may depend on the UX level it is embedded in rather than acting as an independent lever, which is a design-relevant finding in itself and a natural target for the factorial follow-up work described in Section 6. More broadly, these results support viewing facial-expression-based affective patterns as workable, HCI-oriented signals for checking whether a course’s UX and gamification design is actually perceived the way it was intended to be, complementing, rather than replacing, questionnaire-based and behavioral evaluation methods.
Author Contributions
Conceptualization, H.O.-C.; methodology, H.O.-C.; software, H.O.-C. and J.G.-L.; validation, H.O.-C., O.G.B.-Q. and J.G.-L.; formal analysis, H.O.-C.; investigation, H.O.-C.; resources, H.O.-C.; data curation, H.O.-C.; writing—original draft preparation, H.O.-C.; writing—review and editing, H.O.-C., O.G.B.-Q. and J.G.-L.; visualization, H.O.-C.; supervision, O.G.B.-Q. and J.G.-L. All authors have read and agreed to the published version of the manuscript.
Funding
This work was funded by Universidad Politécnica Salesiana under the project “Plataforma de aprendizaje adaptativo basado en inteligencia artificial generativa, gamificación y experiencia de usuario para la personalización del aprendizaje en entornos virtuales”.
Institutional Review Board Statement
The study involved human participants interacting with Moodle-based learning scenarios and facial emotion recognition data. Ethical review and approval were waived for this study because the experiment was conducted for educational research purposes, involved minimal risk, and the collected data were anonymized and used only for academic analysis.
Informed Consent Statement
Informed consent was obtained from all participants. Participants were informed about the webcam-based FER procedure and the use of anonymized interaction data for academic research.
Data Availability Statement
Participant-level FER and interaction data are not publicly available due to privacy and ethical restrictions. Aggregated data may be made available from the corresponding author upon reasonable request.
Acknowledgments
The authors thank FabLab EPN for supporting the development and integration of the FER module within Moodle (version 4.3.12), EducaLab.EC, for supporting deployment of the experimental course, and the students who participated in the study. ChatGPT (GPT-5.5, high reasoning effort; OpenAI) was used during scenario prototyping and for language editing and text refinement. All outputs were reviewed and validated by the authors, who take full responsibility for the manuscript.
Conflicts of Interest
The authors declare no conflicts of interest.
Abbreviations
The following abbreviations are used in this manuscript:
| UX | User Experience |
| LMS | Learning Management System |
| FER | Facial Emotion Recognition |
| H5P | HTML5 Package |
| JSON | JavaScript Object Notation |
| EPN | Escuela Politécnica Nacional |
References
- Marinescu, I.A.; Dobre, C.; Băjenaru, L.; Petre, I. Towards a Longitudinal Understanding of Mobile User Experience: The Moodle App Case. Stud. Inform. Control 2026, 35, 109–120. [Google Scholar] [CrossRef] [Scilit]
- Azevedo, R.; Lucena, K.; Castro, A.; Gadelha, B. Understanding the Influence of Collaborative Tasks on User Experience in Moodle. J. Braz. Comput. Soc. 2026, 32, 817–838. [Google Scholar] [CrossRef] [Scilit]
- Siahaan, R.D.; Kusumawardani, S.S.; Hidayah, I. Heuristics Evaluation of Moodle E-Learning Using Specialized Educational Usability Indicators. In Proceedings of the 2025 9th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE); IEEE: Banyumas, Indonesia, 2025; pp. 317–322. [Google Scholar] [CrossRef] [Scilit]
- Saleh, A.M.; Abuaddous, H.Y.; Alansari, I.S.; Enaizan, O. The Evaluation of User Experience on Learning Management Systems Using UEQ. Int. J. Emerg. Technol. Learn. (iJET) 2022, 17, 145–162. [Google Scholar] [CrossRef] [Scilit]
- Segovia-García, N. Propuesta de mejora en el diseño de interfaz y experiencia de usuario (UX) en Moodle: Valoración del alumnado. Edutec. Rev. Electrón. Tecnol. Educ. 2022, 82, 199–216. [Google Scholar] [CrossRef] [Scilit]
- Bahri, N.E.; Itahriouan, Z.; Jamil, M.O. Emotion-Aware Education Through Affective Computing and Learning Analytics: Insights from a Moroccan University Case Study. Digital 2025, 5, 45. [Google Scholar] [CrossRef] [Scilit]
- Pabba, C.; Kumar, P. An intelligent system for monitoring students’ engagement in large classroom teaching through facial expression recognition. Expert Syst. 2022, 39, e12839. [Google Scholar] [CrossRef] [Scilit]
- Qi, Y.; Zhuang, L.; Chen, H.; Han, X.; Liang, A. Evaluation of Students’ Learning Engagement in Online Classes Based on Multimodal Vision Perspective. Electronics 2023, 13, 149. [Google Scholar] [CrossRef] [Scilit]
- Rodriguez-Lopez, E.; Alvarez-Rodriguez, F.; Muñoz-Zavala, A. UXEEGET: A framework for evaluating learning experience with EEG and Eye Tracking. Univers. Access Inf. Soc. 2026, 25, 41. [Google Scholar] [CrossRef] [Scilit]
- Babu, S.S.; Moorthy, A.D. Application of artificial intelligence in adaptation of gamification in education: A literature review. Comput. Appl. Eng. Educ. 2024, 32, e22683. [Google Scholar] [CrossRef] [Scilit]
- Zourmpakis, A.I.; Kalogiannakis, M.; Papadakis, S. Adaptive Gamification in Science Education: An Analysis of the Impact of Implementation and Adapted Game Elements on Students’ Motivation. Computers 2023, 12, 143. [Google Scholar] [CrossRef] [Scilit]
- Bachiri, Y.A.; Mouncif, H.; Bouikhalene, B. Artificial Intelligence Empowers Gamification: Optimizing Student Engagement and Learning Outcomes in E-learning and MOOCs. Int. J. Eng. Pedagog. (iJEP) 2023, 13, 4–19. [Google Scholar] [CrossRef] [Scilit]
- Ghai, A.; Tandon, U. Integrating gamification and instructional design to enhance usability of online learning. Educ. Inf. Technol. 2023, 28, 2187–2206. [Google Scholar] [CrossRef] [Scilit]
- Liu, X.; Ardakani, S.P. A machine learning enabled affective E-learning system model. Educ. Inf. Technol. 2022, 27, 9913–9934. [Google Scholar] [CrossRef] [Scilit]
- Ordoñez-Calero, H.; Bravo-Quezada, O.G.; Galindo-Losada, J. Toward an Integrated Architecture for Personalized Learning: Combining Generative AI, Gamification, and UX. In Proceedings of the Information and Communication Technologies—13th Ecuadorian Conference, TICEC 2025, Proceedings; Berrezueta, S., Gualotuña, T., Fonseca C., E.R., Rodriguez Morales, G., Maldonado-Mahauad, J., Eds.; Springer: Cham, Switzerland, 2026; pp. 379–393. [Google Scholar] [CrossRef] [Scilit]
- Alam, M.I.; Malone, L.; Nadolny, L.; Brown, M.; Cervato, C. Investigating the impact of a gamified learning analytics dashboard: Student experiences and academic achievement. J. Comput. Assist. Learn. 2023, 39, 1436–1449. [Google Scholar] [CrossRef] [Scilit]
- Rodrigues, A.A.; de Souza Santos, F.; Gama, S.P. The Emotionality Tool: Evaluating Usability with Facial Emotions Analysis. Int. J. Human–Computer Interact. 2025, 41, 14903–14918. [Google Scholar] [CrossRef] [Scilit]
- Suen, H.Y.; Hung, K.E. Enhancing learner affective engagement: The impact of instructor emotional expressions and vocal charisma in asynchronous video-based online learning. Educ. Inf. Technol. 2025, 30, 4033–4060. [Google Scholar] [CrossRef] [Scilit]
- Wang, Q.; Wen, Y.; Quek, C.L. Engaging learners in synchronous online learning. Educ. Inf. Technol. 2023, 28, 4429–4452. [Google Scholar] [CrossRef] [Scilit]
- Belluano, P.L.L. Clustering-Based Adaptive UX in E-Learning Systems: Aligning Microservices with the 4C Framework. J. Appl. Data Sci. 2025, 6, 2436–2448. [Google Scholar] [CrossRef] [Scilit]
- Perez-Aranda, J.; Medina-Claros, S.; Urrestarazu-Capellán, R. Effects of a collaborative and gamified online learning methodology on class and test emotions. Educ. Inf. Technol. 2024, 29, 1823–1855. [Google Scholar] [CrossRef] [Scilit]
- Nassr, R.M.; Aldossary, A.A.; Nasir, H.M. Students’s intention to use emotion-aware virtual learning environment: Does a lecturer’s interaction make a difference? Malays. J. Learn. Instr. 2021, 18, 183–218. [Google Scholar] [CrossRef] [Scilit]
- Mandran, N.; Dupuy-Chessa, S. Supporting experimental methods in information system research. In Proceedings of the 2018 12th International Conference on Research Challenges in Information Science (RCIS); IEEE: Nantes, France, 2018; pp. 1–12. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.




