Multimodal Technologies and Interaction

25 pages, 2939 KiB

Open AccessReview

by Jakub Swacha

Multimodal Technol. Interact. 2025, 9(6), 59; https://doi.org/10.3390/mti9060059 - 11 Jun 2025

Video games come in many genres. Although the popularity of games that belong to different genres is the subject of various research and industry reports, so far, there have been no studies investigating their popularity in research papers. This paper addresses this gap [...] Read more.

Video games come in many genres. Although the popularity of games that belong to different genres is the subject of various research and industry reports, so far, there have been no studies investigating their popularity in research papers. This paper addresses this gap with an analysis of bibliographic data sourced from Scopus, spanning 45 years since the emergence of the topic till today and covering nine widely recognized genres: Action, Puzzle, Rhythm, Role-Playing, Simulation, Sports, Shooter, Strategy, and Traditional. The obtained results not only reveal the current popularity of these video game genres but also illustrate its change over time and geographic distribution as well as highlight the most impactful papers referring to the respective genres and their topics, providing a number of footholds for future studies, including regarding the identified disparities in the research interest in some genres and the number of available games belonging to them, the fluctuations in the relative popularity of the respective genres, and the disparities in the share of research output dedicated to video game genres in the total research output of different countries. Full article

► Show Figures

Figure 1

27 pages, 1540 KiB

Open AccessArticle

Designing Inclusive and Adaptive Content in Moodle: A Framework and a Case Study from Jordanian Higher Education

by Lamis F. Al-Qora’n, Julius T. Nganji and Fadi M. Alsuhimat

Multimodal Technol. Interact. 2025, 9(6), 58; https://doi.org/10.3390/mti9060058 - 10 Jun 2025

Abstract

Blended learning has introduced a more accessible and flexible teaching environment in higher education. However, ensuring that content is inclusive, particularly for students with learning difficulties, remains a challenge. This paper explores how Moodle, a widely adopted learning management system (LMS), can support [...] Read more.

Blended learning has introduced a more accessible and flexible teaching environment in higher education. However, ensuring that content is inclusive, particularly for students with learning difficulties, remains a challenge. This paper explores how Moodle, a widely adopted learning management system (LMS), can support inclusive and adaptive learning based on Universal Design for Learning (UDL) principles. A 16-week descriptive exploratory study was conducted with 70 undergraduate students during a software engineering fundamentals course at Philadelphia University in Jordan. The research combined weekly iterative focus groups, teaching reflections, and interviews with 16 educators to identify and address inclusion barriers. The findings highlight that the students responded positively to features such as conditional activities, flexible quizzes, and multimodal content. A UDL-based framework was developed to guide the design of inclusive Moodle content, and it was validated by experienced educators. To our knowledge, this is the first UDL-based framework designed for Moodle in Middle Eastern computing and engineering education. The findings indicate that Moodle features, such as conditional activities and flexible deadlines, can facilitate inclusive practices, but adoption remains hindered by institutional and workload constraints. This study contributes a replicable design model for inclusive blended learning and emphasizes the need for structured training, intentional course planning, and technological support for implementing inclusivity in blended learning environments. Moreover, this study provides a novel weekly iterative focus group methodology, which enables continuous course refinement based on evolving students’ feedback. Future work will look into generalizing the research findings and transferring the findings to other contexts. It will also explore AI-driven adaptive learning pathways within LMS platforms. This is an empirical study grounded in weekly student focus groups, educator interviews, and reflective teaching practice, offering evidence-based insights on the application of UDL in a real-world higher education setting. Full article

(This article belongs to the Special Issue Innovative Theories and Practices for Designing and Evaluating Inclusive Educational Technology and Online Learning)

► Show Figures

Figure 1

15 pages, 2437 KiB

Open AccessArticle

The Impacts of Incorporating Virtual Reality and Data Gloves in Exergames on Intrinsic Motivation in Upper-Extremity Assessments: A Study in a Young and Healthy Group

by He Kunze, Noppon Choosri and Supara Grudpan

Multimodal Technol. Interact. 2025, 9(6), 57; https://doi.org/10.3390/mti9060057 - 9 Jun 2025

Abstract

Virtual reality (VR) technology has shown potential as a viable tool for rehabilitation. VR is a well-recognized technology that creates immersive experiences to enhance engagement and encourage more effective participation in activities. In the current study, it has been shown that using a [...] Read more.

Virtual reality (VR) technology has shown potential as a viable tool for rehabilitation. VR is a well-recognized technology that creates immersive experiences to enhance engagement and encourage more effective participation in activities. In the current study, it has been shown that using a standard VR system setup can effectively increase participant motivation for various rehabilitation applications. However, there is a research gap in terms of participant motivation, relating to the intervention of integrating data gloves into VR to improve visibility in hand tracking for rehabilitation. This study presents and assesses an integrated approach utilizing VR and data glove technology to evaluate upper extremity function in a young, healthy population, comparing this to traditional methods. Participants’ intrinsic motivation was measured using the Intrinsic Motivation Inventory (IMI). The findings indicate that the combined immersive environment outperforms conventional practice in most aspects. Therefore, this research also sheds light on the fact that a data glove is promising technology in rehabilitation applications that can augment positive experiences while having no adverse effects on the VR system. Full article

► Show Figures

Figure 1

24 pages, 698 KiB

Open AccessSystematic Review

Multiple Ways in Which Video Games Make Education Inclusive: A Systematic Review of Cognitive Enhancement for Neurodivergent Learners

by Martina Basciano, Elisa Bisagno and Alessia Cadamuro

Multimodal Technol. Interact. 2025, 9(6), 56; https://doi.org/10.3390/mti9060056 - 7 Jun 2025

Abstract

Children with neurodevelopmental disorders, such as ADHD, ASD and SLDs, often face challenges in executive functioning that can impact their inclusion and participation in educational and social experiences. Digital technologies, in particular video games, are becoming increasingly more integrated into children’s lives and [...] Read more.

Children with neurodevelopmental disorders, such as ADHD, ASD and SLDs, often face challenges in executive functioning that can impact their inclusion and participation in educational and social experiences. Digital technologies, in particular video games, are becoming increasingly more integrated into children’s lives and are receiving attention as tools to support cognitive development in these populations. This systematic review analyses the effects of video games on Working Memory (WM) and Executive Functions (EFs) in neurodivergent individuals, to evaluate their effectiveness as training tools and their impact in terms of inclusion. Following the PRISMA guidelines, 25 peer-reviewed studies published between 2013 and 2025 were analysed. The interventions included action-video games, exergames, serious games and ad hoc video game training with a pre- and post-assessment in neurodiverse participants. The results indicate that action-video games and exergames show promise in enhancing EFs, while serious games and ad hoc video game training seem to support WM. Despite a few contrasting results, overall, video games are emerging as promising tools of inclusive education thanks to their interactive, customisable and socially empowering nature, especially significant for neurodiverse children. The discussion will depict multiple ways in which video games can make education more inclusive for these populations. Full article

(This article belongs to the Special Issue Innovative Theories and Practices for Designing and Evaluating Inclusive Educational Technology and Online Learning)

► Show Figures

Figure 1

15 pages, 693 KiB

Open AccessArticle

Evaluating a UDL-Based Digital Escape Room for Bullying Awareness in Secondary Education

by Yosra Achab-Moukram, Paula Rodríguez-Rivera, José Manuel Aguilar-Parra, José M. Rodríguez-Ferrer, Javier Rodríguez-Moreno and Ana Manzano-León

Multimodal Technol. Interact. 2025, 9(6), 55; https://doi.org/10.3390/mti9060055 - 6 Jun 2025

Abstract

This study evaluates the effectiveness of a digital escape room designed according to the Universal Design for Learning (UDL) principles as an educational tool to raise awareness about school bullying and cyberbullying in lower secondary education. Using a quasi-experimental design with 137 students [...] Read more.

This study evaluates the effectiveness of a digital escape room designed according to the Universal Design for Learning (UDL) principles as an educational tool to raise awareness about school bullying and cyberbullying in lower secondary education. Using a quasi-experimental design with 137 students (66 in the control group and 71 in the experimental group), the gamified activity was compared to a traditional expository intervention. Empathy was measured using the Brief Basic Empathy Scale (BES-B), and bullying-related knowledge and experiences were assessed with the SPB-A scale. The escape room integrated key UDL principles, including multiple means of representation (e.g., video, images, text), action and expression (interactive tasks and puzzles), and engagement (narrative-based missions and collaborative challenges). The results indicate significant improvements in knowledge about behaviors related to school bullying among participants who completed the escape room. Although changes in affective and cognitive empathy were not statistically significant, positive trends were observed in the experimental group. The immersive, collaborative, and accessible nature of the activity positions the escape room as a promising tool within broader strategies for bullying prevention. Full article

(This article belongs to the Special Issue Video Games: Learning, Emotions, and Motivation)

► Show Figures

Figure 1

27 pages, 6914 KiB

Open AccessArticle

A New Serious Game (e-SoundWay) for Learning English Phonetics

by Alfonso Lago-Ferreiro, María Ángeles Gómez-González and José Carlos López-Ardao

Multimodal Technol. Interact. 2025, 9(6), 54; https://doi.org/10.3390/mti9060054 - 4 Jun 2025

Abstract

This paper presents the design and evaluation of e-SoundWay, a cross-platform serious game developed to improve English phonetic competence through a multimodal and narrative-driven approach. While the platform is specifically tailored to meet the needs of Spanish-speaking learners, it is adaptable for [...] Read more.

This paper presents the design and evaluation of e-SoundWay, a cross-platform serious game developed to improve English phonetic competence through a multimodal and narrative-driven approach. While the platform is specifically tailored to meet the needs of Spanish-speaking learners, it is adaptable for a wider range of English as a Foreign Language (EFL) users. e-SoundWay offers over 600 interactive multimedia minigames that target three core competencies: perception, production, and transcription. Learners progress along a gamified version of the Camino de Santiago, interacting with characters representing diverse English accents. A mixed-methods evaluation combining pre- and post-tests with a user experience questionnaire revealed statistically significant improvements across all domains, particularly in perception. Reduced post-test variability indicated more equitable learning outcomes. User satisfaction was high, with 64% of participants reporting satisfaction with their phonetic progress and 91% stating they would recommend the platform. These findings highlight the educational effectiveness, accessibility, and motivational value of e-SoundWay, reinforcing the role of serious games and multimodal technologies in delivering inclusive and engaging pronunciation instruction. Full article

(This article belongs to the Special Issue Video Games: Learning, Emotions, and Motivation)

► Show Figures

Figure 1

25 pages, 5444 KiB

Open AccessArticle

Evaluating the Usability and Engagement of a Gamified, Desktop, Virtual Art Appreciation Module

by Sitong Li, Nurfaradilla Mohamad Nasri and Helmi Norman

Multimodal Technol. Interact. 2025, 9(6), 53; https://doi.org/10.3390/mti9060053 - 3 Jun 2025

Abstract

Traditional art appreciation instruction relies heavily on textbooks, slides, or videos, limiting student engagement and immersion. To address this issue, this study proposes a desktop VR (GDVR) art appreciation module based on a gamification approach. Unlike traditional VR art learning environments, the GDVR [...] Read more.

Traditional art appreciation instruction relies heavily on textbooks, slides, or videos, limiting student engagement and immersion. To address this issue, this study proposes a desktop VR (GDVR) art appreciation module based on a gamification approach. Unlike traditional VR art learning environments, the GDVR module combines real-time feedback and gamification elements to increase students’ motivation and understanding of information. This study used focus group interviews to evaluate the usability of the GDVR module, as well as student engagement. In addition, on-screen observations have been adopted to capture student interaction behavior and navigation patterns, providing greater insight into usability. Forty Chinese middle school students participated, and the data were analyzed thematically. The results show that the GDVR module demonstrates stable technical performance, intuitive navigation, and a high level of immersion. Moreover, most students find it more engaging than traditional methods, noting that the real-time feedback system significantly enhanced their engagement with and understanding of the material. Furthermore, this study highlights the practical application potential of utilizing low-cost, desktop-based virtual reality systems in the context of middle school art appreciation. Finally, the study acknowledges its limitations and provides recommendations for future research to further refine and expand the application of GDVR in the field of art education. Full article

► Show Figures

Figure 1

18 pages, 6478 KiB

Open AccessArticle

BabloXR: An Authoring Tool for Developing WebXR Educational Applications

by María Lucía Barrón-Estrada, Ramón Zatarain-Cabada, Manuel Alberto Sotelo-Rivas, María Blanca Ibáñez and Víctor Manuel Bátiz-Beltrán

Multimodal Technol. Interact. 2025, 9(6), 52; https://doi.org/10.3390/mti9060052 - 29 May 2025

Abstract

Different areas of education have undergone a significant transformation due to the rapid progress of extended reality in recent years. Nonetheless, existing eXtended Reality (XR) authoring tools often require advanced programming skills, limiting their accessibility to educators and students. To address this challenge, [...] Read more.

Different areas of education have undergone a significant transformation due to the rapid progress of extended reality in recent years. Nonetheless, existing eXtended Reality (XR) authoring tools often require advanced programming skills, limiting their accessibility to educators and students. To address this challenge, we introduce BabloXR, an intuitive authoring tool that leverages Blockly to enable visual, code-free programming for creating educational XR environments. BabloXR was evaluated through a user study involving teachers and students who successfully designed their own XR learning environments. The results indicate that users of BabloXR were able to create complete and detailed scenarios, with usability testing revealing high ease of use, positive user experience, and strong interest in extended reality. Full article

(This article belongs to the Special Issue Innovative Theories and Practices for Designing and Evaluating Inclusive Educational Technology and Online Learning)

► Show Figures

Figure 1

32 pages, 646 KiB

Open AccessArticle

Methods and Findings in the Analysis of Alignment of Bodily Motion in Cooperative Dyadic Dialogue

by Zohreh Khosrobeigi, Maria Koutsombogera and Carl Vogel

Multimodal Technol. Interact. 2025, 9(6), 51; https://doi.org/10.3390/mti9060051 - 27 May 2025

Abstract

This research analyses the temporal flow of motion energy (ME) in dyadic dialogues using alternating lagged correlation tests on consecutive windows and also Granger causality (GC) tests. This research considers both alternatives of lagged values, those of the more dominant party preceding those [...] Read more.

This research analyses the temporal flow of motion energy (ME) in dyadic dialogues using alternating lagged correlation tests on consecutive windows and also Granger causality (GC) tests. This research considers both alternatives of lagged values, those of the more dominant party preceding those of the less and vice versa (with relative dominance independently determined) and labels the resulting lagged windows according to the category of correlation (CC) that holds (positive, negative or none, if the correlation is not significant). Similarly, windows are labeled in relation to the significance of GC (one party causing the other, mutual causation, or no causation). Additionally, occurrences of gestures or speech within windows by an interlocutor whose ME precedes are identified. Then, the ME GC labels are compared with labels derived from simple lagged correlation of ME values to identify whether GC or CC is more efficacious in highlighting which participant independent observers classify as the more dominant party, potentially the “leader” for the conversation. In addition, the association between speech, gestures, dominance, and leadership is explored. This work aims to understand how the distributions of these labels interact with independent perceptions of dominance, to what extent dominant interlocutors lead, and the extent to which these labels “explain” variation in ME within any dialogue. Here, the focus is on between speakers dynamics. It shows dominant speakers have measurable influence on their conversation partners through bodily ME, as they are more likely to lead motion dynamics, though moments of mutual influence also occur. While GC and lagged correlation both capture aspects of leadership, GC reveals directional influence, whereas correlation highlights behavioural alignment. Furthermore, ME contrast during speaking and interaction of ME and gestures indicate that bodily movement synchronisation is shaped not only by dominance but also by gesture types and speaking states: speech affects leadership more than gestures. The interactions highlight the multimodal nature of conversational leadership, where verbal and nonverbal modalities interact to shape dialogue dynamics. Full article

► Show Figures

Figure 1

21 pages, 1796 KiB

Open AccessArticle

A Study of NLP-Based Speech Interfaces in Medical Virtual Reality

by Mohit Nayak, Jari Kangas and Roope Raisamo

Multimodal Technol. Interact. 2025, 9(6), 50; https://doi.org/10.3390/mti9060050 - 26 May 2025

Abstract

Applications of virtual reality (VR) have grown in significance in medicine, as they are able to recreate real-life scenarios in 3D while posing reduced risks to patients. However, there are several interaction challenges to overcome when moving from 2D screens to 3D VR [...] Read more.

Applications of virtual reality (VR) have grown in significance in medicine, as they are able to recreate real-life scenarios in 3D while posing reduced risks to patients. However, there are several interaction challenges to overcome when moving from 2D screens to 3D VR environments, such as complex controls and slow user adaptation. More intuitive techniques are needed for enhanced user experience. Our research explored the potential of intelligent speech interfaces to enhance user interaction while conducting complex medical tasks. We developed a speech-based assistant within a VR application for maxillofacial implant planning, leveraging natural language processing (NLP) to interpret user intentions and to execute tasks such as obtaining surgical equipment or answering questions related to the VR environment. The objective of the study was to evaluate the usability and cognitive load of the speech-based assistant. We conducted a mixed-methods within-subjects user study with 20 participants and compared the voice-assisted approach to traditional interaction methods, such as button panels on the VR view, across various tasks. Our findings indicate that NLP-driven speech-based assistants can enhance interaction and accessibility in medical VR, especially in areas such as locating controls, easiness of control, user comfort, and intuitive interaction. These findings highlight the potential benefits of augmenting traditional controls with speech interfaces, particularly in complex VR scenarios where conventional methods may limit usability. We identified key areas for future research, including improving the intelligence, accuracy, and user experience of speech-based systems. Addressing these areas could facilitate the development of more robust, user-centric, voice-assisted applications in virtual reality environments. Full article

► Show Figures

Figure 1

12 pages, 683 KiB

Open AccessArticle

Integrated Hyperparameter Optimization with Dimensionality Reduction and Clustering for Radiomics: A Bootstrapped Approach

by S. J. Pawan, Matthew Muellner, Xiaomeng Lei, Mihir Desai, Bino Varghese, Vinay Duddalwar and Steven Y. Cen

Multimodal Technol. Interact. 2025, 9(5), 49; https://doi.org/10.3390/mti9050049 - 21 May 2025

Cited by 1

Abstract

Radiomics involves extracting quantitative features from medical images, resulting in high-dimensional data. Unsupervised clustering has been used to discover patterns in radiomic features, potentially yielding hidden biological insights. However, its effectiveness depends on the selection of dimensionality reduction techniques, clustering methods, and hyperparameter [...] Read more.

Radiomics involves extracting quantitative features from medical images, resulting in high-dimensional data. Unsupervised clustering has been used to discover patterns in radiomic features, potentially yielding hidden biological insights. However, its effectiveness depends on the selection of dimensionality reduction techniques, clustering methods, and hyperparameter optimization, an area with limited exploration in the literature. We present a novel bootstrapping-based hyperparameter search approach to optimize clustering efficacy, treating dimensionality reduction and clustering as a connected process chain. The hyperparameter search was guided by the Adjusted Rand Index (ARI) and Davies–Bouldin Index (DBI) within a bootstrapping framework of 100 iterations. The cluster assignments were generated through 10-fold cross-validation, and a grid search strategy was used to explore hyperparameter combinations. We evaluated ten unsupervised learning pipelines using both simulation studies and real-world radiomics data derived from multiphase CT images of renal cell carcinoma. In simulations, we found that Non-negative Matrix Factorization (NMF) and Spectral Clustering outperformed the traditional Principal Component Analysis (PCA)-based approach. The best-performing pipeline (NMF followed by K-means clustering) successfully identified all three simulated clusters, achieving a Cramér’s V of 0.9. The simulation also established a reference framework for understanding the concordance patterns among different pipelines under varying strengths of clustering effects. High concordance reflects strong clustering. In the real-world data application, we observed a moderate clustering effect, which aligned with the weak associations to clinical outcomes, as indicated by the highest AUROC of 0.63. Full article

(This article belongs to the Special Issue Artificial Intelligence in Medical Radiation Science, Radiology and Radiation Oncology)

► Show Figures

Figure 1

11 pages, 12478 KiB

Open AccessArticle

Computer Vision-Based Obstacle Detection Mobile System for Visually Impaired Individuals

by Gisel Katerine Bastidas-Guacho, Mario Alejandro Paguay Alvarado, Patricio Xavier Moreno-Vallejo, Patricio Rene Moreno-Costales, Nayely Samanta Ocaña Yanza and Jhon Carlos Troya Cuestas

Multimodal Technol. Interact. 2025, 9(5), 48; https://doi.org/10.3390/mti9050048 - 18 May 2025

Abstract

Traditional tools, such as canes, are no longer enough to subsist the mobility and orientation of visually impaired people in complex environments. Therefore, technological solutions based on computer vision tasks are presented as promising alternatives to help detect obstacles. Object detection models are [...] Read more.

Traditional tools, such as canes, are no longer enough to subsist the mobility and orientation of visually impaired people in complex environments. Therefore, technological solutions based on computer vision tasks are presented as promising alternatives to help detect obstacles. Object detection models are easy to couple to mobile systems, do not require a large consumption of resources on mobile phones, and act in real-time to alert users of the presence of obstacles. However, existing object detectors were mostly trained with images from platforms such as Kaggle, and the number of existing objects is still limited. For this reason, this study proposes to implement a mobile system that integrates an object detection model for the identification of obstacles intended for visually impaired people. Additionally, the mobile application integrates multimodal feedback through auditory and haptic interaction, ensuring that users receive real-time obstacle alerts via voice guidance and vibrations, further enhancing accessibility and responsiveness in different navigation contexts. The chosen scenario to develop the obstacle detection application is the Specialized Educational Unit Dr. Luis Benavides for impaired people, which is the source of images for building the dataset for the model and evaluating it with impaired individuals. To determine the best model, the performance of YOLO is evaluated by means of a precision adjustment through the variation of epochs, using a proprietary data set of 7600 diverse images. The YOLO-300 model turned out to be the best, with a mAP of 0.42. Full article

► Show Figures

Figure 1

27 pages, 1259 KiB

Open AccessArticle

The Influence of the Labeling Effect on the Perception of Command Execution Delay in Gaming

by Duy H. Nguyen and Peter A. Kara

Multimodal Technol. Interact. 2025, 9(5), 47; https://doi.org/10.3390/mti9050047 - 15 May 2025

Abstract

Gaming is one of the largest industries of digital entertainment. Modern gaming software may be susceptible to command execution delay, which may be caused by various factors, such as insufficient rendering capabilities or limited network resources. At the time of writing this paper, [...] Read more.

Gaming is one of the largest industries of digital entertainment. Modern gaming software may be susceptible to command execution delay, which may be caused by various factors, such as insufficient rendering capabilities or limited network resources. At the time of writing this paper, the utilized advances in gaming are often accompanied by brief descriptions when communicated to the users. While such descriptions may be compressed into a couple of words, even a single word may impact user experience. Due to the cognitive bias induced by the labeling effect, the impact of such a word may actually be more significant than what the user genuinely perceives. In this paper, we investigate the influence of the labeling effect on the perception of command execution delay in gaming. We carried out a series of subjective tests to measure how the word “optimized” affects gaming experience. The test variables of our experiment were the added delay between command and execution, the speed of the game, and the label that was assigned to gaming sequences. The test participants were tasked to directly compare gaming sequences with the different labels assigned: “optimized” and “not optimized”. In every comparison, both sequences had the same objective characteristics; only the label differed. The experiment was conducted on single-input and continuous-input computer games that we developed for this research. The obtained results indicate that for both of these input types, the labeling effect has a statistically significant impact on perceived delay. Overall, more than 70% of the subjective ratings were affected by the assigned labels. Moreover, there is a strong correlation between the amount of delay and the effect of cognitive bias. The speed of the game also affected the obtained results, yet statistically significant differences were only measured between the slowest and the fastest gameplay. Full article

► Show Figures

Figure 1

15 pages, 3211 KiB

Open AccessArticle

Optimizing HUD-EVS Readability: Effects of Hue, Saturation and Lightness on Information Recognition

by Xuyi Qiu

Multimodal Technol. Interact. 2025, 9(5), 46; https://doi.org/10.3390/mti9050046 - 14 May 2025

Abstract

Enhanced Vision System (EVS) offers a display advantage that conventional devices lack, enabling interface information to be overlaid on real-world imagery. However, information overload, especially in complex environments, can reduce the recognizability of important information and impair decision-making. This study investigates a dual [...] Read more.

Enhanced Vision System (EVS) offers a display advantage that conventional devices lack, enabling interface information to be overlaid on real-world imagery. However, information overload, especially in complex environments, can reduce the recognizability of important information and impair decision-making. This study investigates a dual color-coding strategy to optimize the recognizability of Primary Information (PI) and Secondary Information (SI) in Head-Up Display–Enhanced Vision System (HUD-EVS) against complex backgrounds. The results show that adjusting the hue, saturation, and lightness of SI affects the recognizability of both PI and SI. Specifically, certain saturation (20% or 80%) and lightness (60%) combinations should be avoided to ensure PI prominence and maintain sufficient recognizability for SI. These findings provide insights for designing color-coding strategies for EVS, enhancing the recognizability of information on mobile devices. Full article

► Show Figures

Figure 1

34 pages, 10532 KiB

Open AccessArticle

Personalized and Timely Feedback in Online Education: Enhancing Learning with Deep Learning and Large Language Models

by Óscar Cuéllar, Manuel Contero and Mauricio Hincapié

Multimodal Technol. Interact. 2025, 9(5), 45; https://doi.org/10.3390/mti9050045 - 14 May 2025

Abstract

This study investigates an Adaptive Feedback System (AFS) that integrates deep learning (a recurrent neural network trained with historical student data) and GPT-4 to provide personalized feedback in a Digital Art course. In a quasi-experimental design, the intervention group (n = 42) [...] Read more.

This study investigates an Adaptive Feedback System (AFS) that integrates deep learning (a recurrent neural network trained with historical student data) and GPT-4 to provide personalized feedback in a Digital Art course. In a quasi-experimental design, the intervention group (n = 42) received weekly feedback generated from model predictions, while the control group (n = 39) followed the same program without this intervention across four learning blocks or levels. The results revealed (1) a cumulative effect with a significant performance difference in the fourth learning block (+12.63 percentage points); (2) a reduction in performance disparities between students with varying levels of prior knowledge in the experimental group (−56.5%) versus an increase in the control group (+103.3%); (3) an “overcoming effect” where up to 42.9% of students surpassed negative performance predictions; and (4) a positive impact on active participation, especially in live class attendance (+30.21 points) and forum activity (+9.79 points). These findings demonstrate that integrating deep learning with LLMs can significantly improve learning outcomes in online educational environments, particularly for students with limited prior knowledge. Full article

(This article belongs to the Special Issue Innovative Theories and Practices for Designing and Evaluating Inclusive Educational Technology and Online Learning)

► Show Figures

Figure 1

20 pages, 1108 KiB

Open AccessArticle

LLMs in Education: Evaluation GPT and BERT Models in Student Comment Classification

by Anabel Pilicita and Enrique Barra

Multimodal Technol. Interact. 2025, 9(5), 44; https://doi.org/10.3390/mti9050044 - 12 May 2025

Abstract

The incorporation of artificial intelligence in educational contexts has significantly transformed the support provided to students facing learning difficulties, facilitating both the management of their educational process and their emotions. Additionally, online comments play a vital role in understanding student feelings. Analyzing comments [...] Read more.

The incorporation of artificial intelligence in educational contexts has significantly transformed the support provided to students facing learning difficulties, facilitating both the management of their educational process and their emotions. Additionally, online comments play a vital role in understanding student feelings. Analyzing comments on social media platforms can help identify students in vulnerable situations so that timely interventions can be implemented. However, manually analyzing student-generated content on social media platforms is challenging due to the large amount of data and the frequency with which it is posted. In this sense, the recent revolution in artificial intelligence, marked by the implementation of powerful large language models (LLMs), may contribute to the classification of student comments. This study compared the effectiveness of a supervised learning approach using five different LLMs: bert-base-uncased, roberta-base, gpt-4o-mini-2024-07-18, gpt-3.5-turbo-0125, and gpt-neo-125m. The evaluation was carried out after fine-tuning them specifically to classify student comments on social media platforms with anxiety/depression or neutral labels. The results obtained were as follows: gpt-4o-mini-2024-07-18 and gpt-3.5-turbo-0125 obtained 98.93%, roberta-base 98.14%, bert-base-uncased 97.13%, and gpt-neo-125m 96.43%. Therefore, when comparing the effectiveness of these models, it was determined that all LLMs performed well in this classification task. Full article

► Show Figures

Figure 1

22 pages, 5933 KiB

Open AccessArticle

Education 4.0 for Industry 4.0: A Mixed Reality Framework for Workforce Readiness in Manufacturing

by Andrea Bondin and Joseph Paul Zammit

Multimodal Technol. Interact. 2025, 9(5), 43; https://doi.org/10.3390/mti9050043 - 9 May 2025

Abstract

The rapid emergence of Industry 4.0 technologies has transformed manufacturing, requiring a workforce skilled in automation, data-driven decision-making, and process optimisation. While traditional education includes structured formats such as lectures and tutorials, it may not always equip graduates with the hands-on expertise demanded [...] Read more.

The rapid emergence of Industry 4.0 technologies has transformed manufacturing, requiring a workforce skilled in automation, data-driven decision-making, and process optimisation. While traditional education includes structured formats such as lectures and tutorials, it may not always equip graduates with the hands-on expertise demanded by modern industrial challenges. This study presents a Mixed Reality (MR)-based educational framework that promotes interactive experiences to enhance students’ engagement with and understanding of Industry 4.0 concepts, aiming to bridge the skills gap through immersive Virtual Learning Factories (VLFs). The framework was developed using a mixed-methods approach, combining qualitative feedback with quantitative benchmarking. A proof-of-concept MR application was developed and tested at the (Anonymised), simulating Industry 4.0 scenarios in an engineering education context to validate the framework. The findings indicate that MR-based learning improved students’ engagement with the academic content, leading to better knowledge retention and deeper conceptual understanding. The students also demonstrated enhanced problem-solving, process optimisation, and adaptability compared to traditional methods. The immersive nature of MR provided an interactive, context-rich environment that fostered active learning. This research highlights MR’s potential as a transformative educational tool, aligning academic training with industry needs. Future research is recommended to evaluate the framework’s scalability and long-term effectiveness. Full article

► Show Figures

Figure 1

25 pages, 1517 KiB

Open AccessArticle

Towards Structured Gaze Data Classification: The Gaze Data Clustering Taxonomy (GCT)

by Yahdi Siradj, Kiki Maulana Adhinugraha and Eric Pardede

Multimodal Technol. Interact. 2025, 9(5), 42; https://doi.org/10.3390/mti9050042 - 3 May 2025

Abstract

Gaze data analysis plays a crucial role in understanding human visual attention and behaviour. However, raw gaze data is often noisy and lacks inherent structure, making interpretation challenging. Therefore, preprocessing techniques such as classification are essential to extract meaningful patterns and improve the [...] Read more.

Gaze data analysis plays a crucial role in understanding human visual attention and behaviour. However, raw gaze data is often noisy and lacks inherent structure, making interpretation challenging. Therefore, preprocessing techniques such as classification are essential to extract meaningful patterns and improve the reliability of gaze-based analysis. This study introduces the Gaze Data Clustering Taxonomy (GCT), a novel approach that categorises gaze data into structured clusters to improve its reliability and interpretability. GCT classifies gaze data based on cluster count, target presence, and spatial–temporal relationships, allowing for more precise gaze-to-target association. We utilise several machine learning techniques, such as k-NN, k-Means, and DBScan, to apply the taxonomy to a Random Saccade Task dataset, demonstrating its effectiveness in gaze classification. Our findings highlight how clustering provides a structured approach to gaze data preprocessing by distinguishing meaningful patterns from unreliable data. Full article

► Show Figures

Figure 1

14 pages, 1689 KiB

Open AccessArticle

Evaluating the Effectiveness of Tilt Gestures for Text Property Control in Mobile Interfaces

by Sang-Hwan Kim and Xuesen Liu

Multimodal Technol. Interact. 2025, 9(5), 41; https://doi.org/10.3390/mti9050041 - 29 Apr 2025

Abstract

The objective of this study is to verify the usability of gesture interactions such as tilting or shaking, rather than conventional touch gestures, on mobile devices. To this end, a prototype was developed that manipulates the text size in a mobile text messaging [...] Read more.

The objective of this study is to verify the usability of gesture interactions such as tilting or shaking, rather than conventional touch gestures, on mobile devices. To this end, a prototype was developed that manipulates the text size in a mobile text messaging application through tilt gestures. In the text input interface, three types of tilt gesture interaction methods (‘Shaking’, ‘Leaning’, and ‘Acceleration’) were implemented to select the text size level among five levels (extra-small, small, normal, large, and extra-large). Along with the gesture-based interaction methods, the conventional button method was also evaluated. A total of 24 participants were asked to prepare text messages of specified font sizes using randomly assigned interaction methods to select the font size. Task completion time, accuracy (setting errors and input errors), workload, and subjective preferences were collected and analyzed. As a result, the ‘Shaking’ method was generally similar to the conventional button method and superior to the other two ‘Leaning’ and ‘Acceleration’ methods. This may be because ‘Leaning’ and ‘Acceleration’ are continuous operations, while ‘Shaking’ is an individual operation for each menu (font size level). According to subjective comments, tilting gestures on mobile devices can not only be useful if users take the time to learn them, but also provide ways to convey intentions with simple text. Although tilting gestures were not found to significantly improve text editing performance compared to conventional screen touch methods, the use of motion gestures beyond touch on mobile devices can be considered for interface manipulations such as app navigation, gaming, or multimedia controls across diverse applications. Full article

► Show Figures

Figure 1

Journal Description

Multimodal Technologies and Interaction

Latest Articles

Journal Menu

Journal Browser

Highly Accessed Articles

Latest Books

E-Mail Alert

News

Topics

Conferences

Special Issues

Further Information

Guidelines

MDPI Initiatives

Follow MDPI