Towards a Lightweight Arabic Sign Language Translation System
Abstract
1. Introduction
2. Related Studies
3. Sign Language Datasets
4. Feature Extraction Step
5. Model Architecture
Model Training
6. Results and Discussion
6.1. Qualitative Analysis of Testing Samples
6.2. Real-Time Performance and Model Stability
6.3. Comparing the Model’s Performance with Other Models
7. Ablation Study
8. Conclusions
Author Contributions
Funding
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Chadha, S.; Kamenov, K.; Cieza, A. The world report on hearing, 2021. Bull. World Health Organ. 2021, 99, 242. [Google Scholar] [CrossRef]
- General Authority for Statistics. Health and Disability Statistics. Saudi Census Statistics 2022. 2022. Available online: https://www.stats.gov.sa/en/w/%D8%A7%D9%84%D8%B5%D8%AD%D8%A9-%D9%88-%D8%A7%D9%84%D8%A5%D8%B9%D8%A7%D9%82%D8%A9?category=417709&tab=436327 (accessed on 21 August 2025).
- Kahlon, N.K.; Singh, W. Machine translation from text to sign language: A systematic review. Univers. Access Inf. Soc. 2023, 22, 1–35. [Google Scholar] [CrossRef]
- ZainEldin, H.; Gamel, S.A.; Talaat, F.M.; Aljohani, M.; Baghdadi, N.A.; Malki, A.; Badawy, M.; Elhosseini, M.A. Silent no more: A comprehensive review of artificial intelligence, deep learning, and machine learning in facilitating deaf and mute communication. Artif. Intell. Rev. 2024, 57, 188. [Google Scholar] [CrossRef]
- Alayed, A. Machine Learning and Deep Learning Approaches for Arabic Sign Language Recognition: A Decade Systematic Literature Review. Sensors 2024, 24, 7798. [Google Scholar] [CrossRef] [PubMed]
- Von Agris, U.; Zieren, J.; Canzler, U.; Bauer, B.; Kraiss, K.-F. Recent developments in visual sign language recognition. Univers. Access Inf. Soc. 2008, 6, 323–362. [Google Scholar] [CrossRef]
- Mohandes, M.; Deriche, M.; Liu, J. Image-based and sensor-based approaches to Arabic sign language recognition. IEEE Trans. Hum. Mach. Syst. 2014, 44, 551–557. [Google Scholar] [CrossRef]
- Tharwat, G.; Ahmed, A.M.; Bouallegue, B. Arabic sign language recognition system for alphabets using machine learning techniques. J. Electr. Comput. Eng. 2021, 2995851, 2021. [Google Scholar] [CrossRef]
- El Kharoua, R.; Jiang, X. Deep learning recognition for arabic alphabet sign language rgb dataset. J. Comput. Commun. 2024, 12, 32–51. [Google Scholar] [CrossRef]
- Al-Barham, M.; Alsharkawi, A.; Al-Yaman, M.; Al-Fetyani, M.; Elnagar, A.; SaAleek, A.A.; Al-Odat, M. RGB Arabic alphabets sign language dataset. arXiv 2023, arXiv:2301.11932. [Google Scholar] [CrossRef]
- Alnabih, A.F.; Maghari, A.Y. Arabic Sign Language letters recognition using vision transformer. Multimed. Tools Appl. 2024, 83, 81725–81739. [Google Scholar] [CrossRef]
- Latif, G.; Mohammad, N.; Alghazo, J.; AlKhalaf, R.; AlKhalaf, R. Arasl: Arabic alphabets sign language dataset. Data Br. 2019, 23, 103777. [Google Scholar] [CrossRef]
- Hassan, M.A.; Ali, A.H.; Sabri, A.A. Enhancing communication: Deep learning for Arabic sign language translation. Open Eng. 2024, 14, 20240025. [Google Scholar] [CrossRef]
- Al Ahmadi, S.; Mohammad, F.; Al Dawsari, H. Efficient YOLO-based deep learning model for arabic sign language recognition. J. Disabil. Res. 2024, 3, 20240051. [Google Scholar] [CrossRef]
- Batnasan, G.; Gochoo, M.; Otgonbold, M.-E.; Alnajjar, F.; Shih, T.K. ArSL21L: Arabic Sign Language Letter Dataset Benchmarking and an Educational Avatar for Metaverse Applications. In Proceedings of the 2022 IEEE Global Engineering Education Conference (Educon), Tunis, Tunisia, 28–31 March 2022; pp. 1814–1821. [Google Scholar]
- Ibrahim, N.B.; Selim, M.M.; Zayed, H.H. An automatic arabic sign language recognition system (ArSLRS). J. King Saud Univ. Inf. Sci. 2018, 30, 470–477. [Google Scholar] [CrossRef]
- Mosleh, M.A.A.; Assiri, A.; Gumaei, A.H.; Alkhamees, B.F.; Al-Qahtani, M. A bidirectional Arabic sign language framework using Deep Learning and fuzzy matching score. Mathematics 2024, 12, 1155. [Google Scholar] [CrossRef]
- Aly, S.; Aly, W. DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition. IEEE Access 2020, 8, 83199–83212. [Google Scholar] [CrossRef]
- Shanableh, T.; Assaleh, K.; Al-Rousan, M. Spatio-temporal feature-extraction techniques for isolated gesture recognition in Arabic sign language. IEEE Trans. Syst. Man Cybern. Part B 2007, 37, 641–650. [Google Scholar] [CrossRef]
- Bencherif, M.A.; Algabri, M.; Mekhtiche, M.A.; Faisal, M.; Alsulaiman, M.; Mathkour, H.; Al-Hammadi, M.; Ghaleb, H. Arabic sign language recognition system using 2D hands and body skeleton data. IEEE Access 2021, 9, 59612–59627. [Google Scholar] [CrossRef]
- Al-Hammadi, M.; Muhammad, G.; Abdul, W.; Alsulaiman, M.; Bencherif, M.A.; Mekhtiche, M.A. Hand Gesture Recognition for Sign Language Using 3DCNN. IEEE Access 2020, 8, 79491–79509. [Google Scholar] [CrossRef]
- Al-Hammadi, M.; Muhammad, G.; Abdul, W.; Alsulaiman, M.; Bencherif, M.A.; Alrayes, T.S.; Mathkour, H.; Mekhtiche, M.A. Deep learning-based approach for sign language gesture recognition with efficient hand gesture representation. IEEE Access 2020, 8, 192527–192542. [Google Scholar] [CrossRef]
- Al-Hammadi, M.; Bencherif, M.A.; Alsulaiman, M.; Muhammad, G.; Mekhtiche, M.A.; Abdul, W.; Alohali, Y.A.; Alrayes, T.S.; Mathkour, H.; Faisal, M.; et al. Spatial attention-based 3d graph convolutional neural network for sign language recognition. Sensors 2022, 22, 4558. [Google Scholar] [CrossRef]
- Oulad-Naoui, S.; Ben-Abderrahmane, H.; Chagha, A.; Cherif, A. An LSTM-Based System for Dynamic Arabic Sign Language Recognition. In Proceedings of the International Conference on Emerging Intelligent Systems for Sustainable Development (ICEIS 2024), Aflou, Algeria, 26–27 June 2024; pp. 313–323. [Google Scholar]
- Luqman, H. ArabSign: A Multi-Modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition. In Proceedings of the 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), Waikoloa Beach, HI, USA, 5–8 January 2023; pp. 1–8. [Google Scholar]
- Shanableh, T. Two-stage deep learning solution for continuous Arabic Sign Language recognition using word count prediction and motion images. IEEE Access 2023, 11, 126823–126833. [Google Scholar] [CrossRef]
- Algethami, N.; Farhud, R.; Alghamdi, M.; Almutairi, H.; Sorani, M.; Aleisa, N. Continuous Arabic Sign Language Recognition Models. Sensors 2025, 25, 2916. [Google Scholar] [CrossRef] [PubMed]
- Alsulaiman, M.; Faisal, M.; Mekhtiche, M.; Bencherif, M.; Alrayes, T.; Muhammad, G.; Mathkour, H.; Abdul, W.; Alohali, Y.; Alqahtani, M.; et al. Facilitating the communication with deaf people: Building a largest Saudi sign language dataset. J. King Saud Univ. Inf. Sci. 2023, 35, 101642. [Google Scholar] [CrossRef]
- Saudi Association for Hearing Impairment. Saudi Sign Language Library. Available online: https://sshi.sa/ (accessed on 21 August 2025).
- Lugaresi, C.; Tang, J.; Nash, H.; McClanahan, C.; Uboweja, E.; Hays, M.; Zhang, F.; Chang, C.L.; Yong, M.G.; Lee, J.; et al. Mediapipe: A framework for building perception pipelines. arXiv 2019, arXiv:1906.08172. [Google Scholar] [CrossRef]
Type | Landmarks |
---|---|
Each Hand | 35 (21 hand landmarks + 5 position features + 4 angle features + 5 curl features) |
Pose | 25 (21 upper body landmarks + 2 position features + 2 velocity features) |
Training Mode | Experiment ID | Training Set | Validation Set | Testing Set |
---|---|---|---|---|
Signer-dependent mode | Exp_1 | All signers, repetitions: 1 | 10% from the training set | All signers, repetitions: 4 |
Exp_2 | All signers, repetitions: 1, 2 | |||
Exp_3 | All signers, repetitions: 1, 2, 3 | |||
Signer-Independent mode | Exp_4 | All repetitions of signers 1–5 | All repetitions of signers 26–32 | |
Exp_5 | All repetitions of signers 1–10 | |||
Exp_6 | All repetitions of signers 1–15 | |||
Exp_7 | All repetitions of signers 1–20 | |||
Exp_8 | All repetitions of signers 1–25 |
Mode | Experiment ID | Size of Training Data | Accuracy |
---|---|---|---|
Signer-dependent mode effect of number of repetitions | Exp_1 | All S, one R | 92.9% |
Exp_2 | All S, two R | 96.4% | |
Exp_3 | All S, three R | 97.7% | |
Signer-independent mode effect of number of signers | Exp_4 | 5 S, three R | 46.7% |
Exp_5 | 10 S, three R | 60.3% | |
Exp_6 | 15 S, three R | 84.7% | |
Exp_7 | 20 S, three R | 87.0% | |
Exp_8 | 25 S, three R | 90.7% |
Signer ID | One Frame from the Input Video | The Extracted Features | Prediction Results |
---|---|---|---|
32 | Class: Five Confidence: 98.3% Sequence length: 12 frames | ||
29 | Class: Family Confidence: 97.0% Sequence length: 10 frames | ||
28 | Class: Injection Confidence: 93.5% Sequence length: 12 frames |
Performance Criteria | Value |
---|---|
Model Size | 2.43 MB |
Training Parameters | 635,174 |
Single Sample Latency (Intel Core i7) | 11.97 ms |
Estimated FPS (Intel Core i7) | 83.6 |
Single Sample Latency (Raspberry Pi 5) | 38.36 ms |
Estimated FPS (Raspberry Pi 5) | 26.1 |
Training Mode | Experiment ID | Training Set | Validation Set | Testing Set | Accuracy | Accuracy [20] |
---|---|---|---|---|---|---|
Signer-dependent mode | Exp_9 | All signers, repetitions 1,2,3 | All signers, repetition 5 | All signers, repetition 4 | 98.38% | 89.62% |
Signer-independent mode | Exp_10 | Signers 1, 2, 10–31 | Signers 3–9, 40 | 32–39 | 96.22% | 88.09% |
Model | Test Accuracy | Total Number of Parameters |
---|---|---|
Baseline | 90.7% | 635,174 |
Baseline (no spatial) | 88.5% | 718,886 |
Baseline (no temporal) | 81.5% | 91,173 |
Baseline (no attention) | 90.4% | 602,149 |
Baseline (LSTM temporal instead of GRU) | 90.1% | 800,038 |
Baseline (GRU unidirectional) | 89% | 305,958 |
Model | Test Accuracy | Total Number of Parameters |
---|---|---|
Baseline | 90.7% | 635,174 |
Baseline (256 hidden dimensions) | 91.1% | 2,384,166 |
Baseline (64 hidden dimensions) | 88.2% | 178,470 |
Model | Test Accuracy | Total Number of Parameters |
---|---|---|
Baseline | 90.7% | 635,174 |
Baseline (1 GRU layer) | 89.8% | 338,726 |
Baseline (3 GRU layers) | 90.2% | 931,622 |
Baseline (2 spatial layers) | 90.8% | 651,942 |
Baseline (1 classifier layer) | 91.2% | 639,782 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Algabri, M.; Mekhtiche, M.; Bencherif, M.A.; Saeed, F. Towards a Lightweight Arabic Sign Language Translation System. Sensors 2025, 25, 5504. https://doi.org/10.3390/s25175504
Algabri M, Mekhtiche M, Bencherif MA, Saeed F. Towards a Lightweight Arabic Sign Language Translation System. Sensors. 2025; 25(17):5504. https://doi.org/10.3390/s25175504
Chicago/Turabian StyleAlgabri, Mohammed, Mohamed Mekhtiche, Mohamed A. Bencherif, and Fahman Saeed. 2025. "Towards a Lightweight Arabic Sign Language Translation System" Sensors 25, no. 17: 5504. https://doi.org/10.3390/s25175504
APA StyleAlgabri, M., Mekhtiche, M., Bencherif, M. A., & Saeed, F. (2025). Towards a Lightweight Arabic Sign Language Translation System. Sensors, 25(17), 5504. https://doi.org/10.3390/s25175504