Virtual Worlds

Egocentric hand gesture recognition is vital for natural human–computer interaction in augmented and virtual reality (AR/VR) systems. However, most deep learning models struggle to balance accuracy and efficiency, limiting real-time use on wearable devices. This paper introduces a Two-Stream Temporal Shift Module Transformer Fusion Network (TSMTFN) that achieves high recognition accuracy with low computational cost. The model integrates Temporal Shift Modules (TSMs) for efficient motion modeling and a Transformer-based fusion mechanism for long-range temporal understanding, operating on dual RGB-D streams to capture complementary visual and depth cues. Training stability and generalization are enhanced through full-layer training from epoch 1 and MixUp/CutMix augmentations. Evaluated on the EgoGesture dataset, TSMTFN attained 96.18% top-1 accuracy and 99.61% top-5 accuracy on the independent test set with only 16 GFLOPs and 21.3M parameters, offering a 2.4–4.7× reduction in computation compared to recent state-of-the-art methods. The model runs at 15.10 samples/s, achieving real-time performance. The results demonstrate robust recognition across over 95% of gesture classes and minimal inter-class confusion, establishing TSMTFN as an efficient, accurate, and deployable solution for next-generation wearable AR/VR gesture interfaces.

Along with the rapid advancement of Virtual Reality (VR) and the metaverse, interest in this technology has surged among game developers and in fields such as education and healthcare. VR has enabled the rise in immersive, gamified activities, whether for rehabilitation, therapy, or learning. Additionally, VR and Motion Capture (MoCap) have allowed developers to create further accessibility features for end-users with special needs. However, the excitement of using new technology often does not align with the end user’s use cases. The over-reliance on cutting-edge hardware can negatively impact most end users who lack access to such expensive tools. To this end, we conducted an inclusivity-focused study that enables learners to practice ASL in an immersive and engaging way using only head- and controller-based tracking. Our approach replaces full-body MoCap with Inverse Kinematics (IK) and simple controller mappings for upper-body pose and hand-gesture recognition, providing a low-cost, reproducible alternative to costly setups.

Technologies such as XR (Extended Reality), in the form of VR (Virtual Reality), AR (Augmented Reality) and MR (Mixed-Reality), are being researched for their potential to support higher education. XR offers novel opportunities for improving understanding and engagement of computer science (CS) courses, abstract and algorithmic thinking and the application of knowledge to solve problems with computers. This narrative literature review aims to report the state of XR adoption in the university CS education context by studying pedagogical benefits, representative cases, challenges, and future research work. Recent case studies have demonstrated that VR innovations are supportive of algorithm and data structure visualization, AR in programming and circuit analysis contextualization, and MR in bridging the experimental practice on virtual with real hardware within computer labs. The potential of XR to enhance engagement, motivation, and complex content understanding has already been researched. However, ongoing obstacles remain such as the high cost of hardware, technical issues in practicing scalable content, restricted access for students with disabilities, and ethical considerations over privacy and data protection. This review also presents XR, not as a substitute for traditional pedagogy, but as an additive tool that, in alignment with well-defined curricular objectives, may enhance CS learning. If it overcomes these deficiencies and progresses appropriate inclusive evidence-based practices, XR has the potential to play a powerful role in the future of computer science education as part of the digital learning ecosystem.