ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners

Amiruzzaman, Stefanie; Batchu, Raga Mouni; Amiruzzaman, Md; Ngo, Linh; Dewan, M. Ali Akber

doi:10.3390/computers15050299

Open AccessArticle

ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners

by

Stefanie Amiruzzaman

^1,*

,

Raga Mouni Batchu

²

,

Md Amiruzzaman

²

,

Linh Ngo

²

and

M. Ali Akber Dewan

^3,*

¹

Department of Languages and Cultures, West Chester University, West Chester, PA 19383, USA

²

Department of Computer Science, West Chester University, West Chester, PA 19383, USA

³

School of Computing and Information Systems, Faculty of Science and Technology, Athabasca University, Athabasca, AB T9S 3A3, Canada

^*

Authors to whom correspondence should be addressed.

Computers 2026, 15(5), 299; https://doi.org/10.3390/computers15050299

Submission received: 17 February 2026 / Revised: 26 April 2026 / Accepted: 27 April 2026 / Published: 7 May 2026

(This article belongs to the Special Issue Advances in Game-Based Learning, Gamification in Education and Serious Games)

Download

Browse Figures

Versions Notes

Abstract

Digital learning tools for American Sign Language (ASL) often lack the interactive depth necessary to engage learners effectively. This paper introduces a novel, browser-based word search game designed to facilitate ASL vocabulary familiarization through gamified interaction. The system employs a two-tier architecture consisting of a React-based frontend and a Flask-based backend. At its core, the application integrates a lightweight, skeleton-based Isolated Sign Language Recognition (ISLR) model, utilizing a Stacked Transformer-based Spatial-Temporal Attention Network to enable real-time webcam-based word entry during the configuration phase. This model, trained on the WLASL-100 dataset, achieves a Top-5 test accuracy of 88.48% with an average model inference latency of 141 ms, enabling real-time webcam input without proprietary hardware. Furthermore, we implement a constraint-satisfaction puzzle generation algorithm that achieves a 100% success rate in creating interlocked, multi-directional grids. Our results demonstrate that merging computer vision with pedagogical game mechanics provides an accessible, high-performance tool for the Deaf and Hard-of-Hearing (DHH) community, bridging the gap between static instruction and active linguistic practice.

Keywords: american sign language; gamified learning; word search puzzle; computer vision; accessibility; educational technology

Share and Cite

MDPI and ACS Style

Amiruzzaman, S.; Batchu, R.M.; Amiruzzaman, M.; Ngo, L.; Dewan, M.A.A. ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners. Computers 2026, 15, 299. https://doi.org/10.3390/computers15050299

AMA Style

Amiruzzaman S, Batchu RM, Amiruzzaman M, Ngo L, Dewan MAA. ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners. Computers. 2026; 15(5):299. https://doi.org/10.3390/computers15050299

Chicago/Turabian Style

Amiruzzaman, Stefanie, Raga Mouni Batchu, Md Amiruzzaman, Linh Ngo, and M. Ali Akber Dewan. 2026. "ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners" Computers 15, no. 5: 299. https://doi.org/10.3390/computers15050299

APA Style

Amiruzzaman, S., Batchu, R. M., Amiruzzaman, M., Ngo, L., & Dewan, M. A. A. (2026). ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners. Computers, 15(5), 299. https://doi.org/10.3390/computers15050299

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

ASL Recognition and Game-Based Interaction: A Machine Learning—Driven, Gamified and Accessible Vocabulary Learning System for Deaf Learners

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI