A Comprehensive Review of Deep Learning Approaches for Video-Based Sign Language Recognition: Datasets, Challenges and Insights

Abstract

This study presents a comprehensive review of more than 100 research papers on sign language recognition (SLR) published between 2020 and 2026. The analysis focuses on deep learning approaches applied to video-based SLR, including spatiotemporal feature extraction, temporal modeling, attention mechanisms, motion-based representations, hybrid frameworks, transfer learning methods and other methods. Particular attention is given to how these methods model spatiotemporal dynamics and capture subtle gesture characteristics in sign language communication. The review highlights several recent developments, such as the introduction of specialized datasets, the emergence of real-time recognition systems, and the integration of multimodal fusion strategies. At the same time, persistent challenges remain, including data scarcity in low-resource sign languages, limited linguistic standardization of datasets, and insufficient model interpretability. The findings underline the importance of developing scalable and generalizable models capable of handling diverse datasets and user variability. The distinct contributions of this review are fourfold: (1) a comprehensive synthesis of over 100 studies published between 2020 and 2026, covering the full spectrum of deep learning architectures for video-based SLR; (2) a structured six-category taxonomy enabling systematic cross-architectural comparison; (3) a comprehensive focus on low-resource sign languages, which remain underrepresented in the existing literature; and (4) a critical analysis of the current benchmark landscape for low-resource sign languages, identifying key gaps and outlining strategic directions for future dataset development. These contributions are intended to guide further research toward more robust, inclusive, and universally applicable SLR systems.

1. Introduction

SLR systems are fundamental technological tools in creating an inclusive information society by providing equal access to digital resources for people with hearing and speech impairments. As part of this challenge, the issue of video-based SLR is scientifically relevant, particularly due to the need for a special approach that takes into account the unique vocabulary, grammar, and ethnocultural features of the language, which are not fully covered by current universal models [1,2,3,4,5].

Sign language is a rich and complex means of communication, involving spatiotemporal hand movement, facial expression, and body position. For its effective recognition, it is necessary to integrate modern computer vision with Natural Language Processing (NLP) techniques, and take into account the limited computing resources available for applications on mobile devices and in cloud environments [6,7,8,9,10].

In the current technological landscape, the development of SLR systems is gaining particular importance as artificial intelligence and computer vision become central tools for building inclusive communication solutions. These systems serve as critical interfaces between deaf and hearing communities, enabling access to digital services and social participation for people with hearing and speech impairments. These systems for people with hearing and speech impairments perform the function of communication with the outside world. SLs represent complex and distinctive linguistic systems with their own structure, grammar, and expressive mechanisms.

Video recognition in sign language faces various challenges, including the complexity of the data collection and annotation process, the uncertainty in the signal, and the underdevelopment of the architecture that is balanced for context and the use of deep learning architectures capable of effectively capturing contextual information and adapting to the specific characteristics of sign languages [11,12,13,14,15]. Addressing these challenges requires advanced video stream analysis techniques, including object detection, motion tracking, and sequential classification, which form the technical foundation of modern SLR systems [16,17,18,19,20]. Figure 1 demonstrates the architecture of the collection processing system used for sign language video recognition.

This review examines studies related to SLR and explores the importance of developing “inclusive” models that support low-resource languages, as well as enabling their use in international SLs. The implementation of SLR systems opens the way to inclusive growth in the fields of education, employment, and social communication [21,22,23,24]. This review will review current approaches to video recognition in SL, with a focus on the low-resource Kazakh Sign Language (KSL), and suggest areas for further research and development. The development of SLR systems requires a complex architecture that includes video stream processing, feature extraction, and classification components. Solutions for this architecture should be tailored to the unique characteristics of SL, taking into account spatial and temporal aspects, as well as cultural features.

As shown in Figure 2, the structure of the sign language video recognition system includes several main components:

1. Data Capture Module. The data capture module contains devices that capture a video stream, such as a high-resolution camera. The main requirement is the ability to accurately capture human hand movements and facial expressions, as well as body shape.

2. Pre-Processing Module. The pre-processing stage aligns frames, removes noise, restores data, and extracts regions of interest (e.g., hand gestures and facial expressions) within them, which is crucial for reducing computational overhead and improving the quality of data sent to subsequent processing stages.

3. Feature Extraction Module. Deep learning techniques such as convolutional neural networks (CNNs) can be used to extract spatial and temporal features. For example, a three-dimensional convolutional neural network (3D CNN) can be used for motion analysis in combination with a Long Short-Term Memory (LSTM) or a Gated Recurrent Unit (GRU).

Spatiotemporal representations are typically learned using deep neural network architectures. Convolutional layers capture spatial patterns across frames, while short-term relationships are modeled using cyclic mechanisms such as LSTM or GRU modules. In dynamic gesture recognition, three-dimensional convolutional neural networks (3D CNNs) are often integrated with cyclic layers for synchronous encoding of motion dynamics and sequential context.

4. Classification Module. This module performs gesture analysis and thoroughly interprets the sifted feature images. Transformer-based architectures model continuous sub-constraints in sequences, allowing them to effectively encode contextual relationships for subsequent classification.

5. Integration and Output Module. A special module for displaying gestures allows the user to initiate the solution of expected gestures in the text or voice version, which increases user satisfaction and adaptability.

A highly reliable SLR and accurate detection capability require a comprehensive integration of all components [4]. Despite the public recognition of inclusive communication, efforts in this area face significant obstacles, including variable levels of language complexity, insufficient resources, and limited hardware capabilities Therefore, a design that provides high reliability must provide a balance between computational power, quality assurance, integrity of values, and availability across a wide range of deployment scenarios.

The study provides a comprehensive overview of current deep learning strategies used in SLR, focusing on their methodological peculiarities and inherent limitations. The present study synthesizes the results of further research conducted after the video data, with a view to identifying current research directions and incomplete solutions to current research directions and technical challenges.

The present review also contributes a structured analytical framework for evaluating deep learning approaches across low-resource sign language contexts, with KSL serving as a representative case study. This paper outlines future research directions in SLR, emphasizing the need for generalizable, computationally efficient, and linguistically inclusive models capable of performing across diverse datasets and real-world deployment conditions.

As shown in

Table 1. Comparison of the present review with closely related survey articles.

Dimension	Al Abdullah et al. [18]	Violet et al. [20]	Present Review
Publication year	2024	2025	2026
Papers covered	∼80	∼60	103
Period covered	Up to 2023	Up to 2024	2020–2026
Focus on video-based SLR	Partial	Partial	Exclusive
CNN/recurrent neural network (RNN) coverage	Yes	Yes	Yes, comprehensive
Transformer-based models	Limited	Limited	Yes, comprehensive
Graph convolutional network (GCN)-based models	Not covered	Limited	Yes, comprehensive
Self-supervised/transfer learning	Not covered	Not covered	Yes, covered
Structured architectural taxonomy	Not provided	Not provided	Yes, six-category
Low-resource sign languages	Not addressed	Not addressed	Yes, dedicated focus
Systematic PRISMA methodology	No	No	Yes
Bias assessment	No	No	Yes
Integrative discussion with roadmap	No	No	Yes

, the present review differs from prior surveys in several substantive respects. While existing reviews provide useful overviews of CNN and RNN-based approaches, none of them provide comprehensive coverage of Transformer and GCN-based architectures for video-based SLR, nor do they address low-resource sign languages or follow a formal systematic methodology with explicit inclusion criteria and bias assessment. The proposed six-category taxonomy was derived inductively from the systematic analysis of 103 included studies rather than adopted from prior classification schemes, and reflects the specific architectural landscape of video-based SLR as it has evolved between 2020 and 2026. This taxonomy integrates temporal modeling capacity, multimodal fusion capability, and real-world applicability as organizing principles, i.e., the criteria that are absent from the categorical frameworks used in prior reviews. Together, these distinctions substantiate the originality of the present work and its contribution to the systematic understanding of deep learning approaches for video-based SLR.

Several systematic reviews on deep learning-based SLR have been published in recent years [18]; however, most existing reviews either focus narrowly on isolated gesture recognition, cover only a limited architectural scope, or do not address low-resource sign languages. The present review distinguishes itself in three key respects. First, it provides a unified six-category taxonomy that systematically organizes approaches ranging from spatial feature extraction to self-supervised multi-stream learning, enabling structured cross-architectural comparison. Second, it explicitly addresses the challenge of low-resource SLR with a dedicated focus on KSL as a representative underrepresented case. Third, the formal systematic methodology includes explicit inclusion and exclusion criteria, a four-stage PRISMA-compliant filtering process, and bias assessment that ensures reproducibility and transparency that distinguish the present work from narrative surveys in the field.

2. Methodology

This review adopts a structured, systematic methodology to ensure transparency, reproducibility, and comprehensive coverage of recent advances in video-based SLR using deep learning techniques. The overall approach is aligned with established systematic review practices and is designed to provide a consistent and unbiased analysis of the literature.

A systematic search was conducted across four major scientific databases: Scopus, Web of Science, IEEE Xplore, and Google Scholar. The search was performed in January 2026 and covered publications from January 2020 to January 2026 in order to capture the most recent developments in the field. The search queries were applied to titles, abstracts, and keyword fields using combinations of terms related to sign language recognition, gesture recognition, deep learning, and video-based analysis. Specifically, the following query structure was used: (“sign language recognition” OR “gesture recognition”) AND (“deep learning” OR “neural network” OR “convolutional neural network” OR “transformer” OR “recurrent neural network”) AND (“video” OR “video-based” OR “continuous”). In addition, targeted searches were conducted to ensure adequate coverage of specific methodological directions, including graph convolutional networks, attention-based architectures, continuous recognition systems, and approaches designed for low-resource sign languages.

To ensure methodological consistency, explicit inclusion and exclusion criteria were applied. Studies were included if they were published in peer-reviewed journals or conference proceedings within the specified time frame, focused on video-based SLR using at least one deep learning component, reported quantitative performance metrics such as accuracy, Word Error Rate (WER), BLEU score, or mean Average Precision (mAP), and were available in English as full-text publications. Studies were excluded if they addressed only static image-based gesture recognition without temporal modeling, relied exclusively on traditional machine learning methods without deep learning, lacked sufficient methodological detail, or represented duplicate or superseded versions of previously published work.

The study selection process followed a PRISMA-compliant multi-stage filtering procedure. Initially, 1050 records were identified across all databases. After the removal of approximately 320 duplicate entries, 730 unique records remained. In the screening stage, titles and abstracts were reviewed, resulting in the exclusion of 540 studies that were not directly related to video-based SLR or did not involve deep learning techniques. The remaining 190 articles were subjected to full-text evaluation, during which 87 studies were excluded due to insufficient methodological rigor, lack of accessibility, or misalignment with the scope of this review. Ultimately, 103 studies were retained for detailed analysis. The full selection process is illustrated in a PRISMA flow diagram (Figure 3).

For each selected study, relevant information was systematically extracted, including publication metadata, the sign language addressed, the type of deep learning architecture employed, datasets used, evaluation metrics, reported performance, and identified limitations. Based on this information, the studies were categorized according to the taxonomy proposed in Section 3, enabling a structured comparison of existing approaches.

To reduce selection bias, the search and screening procedures were conducted independently by two authors, with any disagreements resolved through discussion. Nevertheless, certain limitations remain. In particular, a degree of publication bias is expected, as many studies report relatively high performance metrics, which may reflect a tendency to publish positive results. Additionally, no formal risk-of-bias assessment tool was applied, as existing instruments are primarily designed for clinical research contexts. Instead, methodological quality was evaluated based on criteria such as dataset scale, experimental design, clarity of evaluation protocols, and reproducibility of results.

Overall, the adopted methodology ensures a transparent and reproducible framework for analyzing the current state of deep learning approaches in video-based SLR.

3. Taxonomy of Deep Learning Approaches for SLR

This review provides a systematic analysis of video-based SLR technologies, focusing on systems designed to support people with hearing and speech impairments. This review also focuses on modern deep learning architectures, highlighting key industry challenges such as high tagging variability, lack of annotated datasets, and significant computational complexity.

To ensure a coherent and structured analysis of the literature, existing approaches are divided into major technological paradigms. These include methods for acquiring spatial and spatiotemporal objects, approaches for modeling time sequences, Transformer-based and focusing architectures, graph-based spatial and temporal modeling methods, hybrid full SLR and sign language translation systems, and transfer learning strategies and other auxiliary methods.

This comprehensive collection of publications ensures coverage of recent industry impacts and contributions, and was selected based on rigorous criteria of relevance and quality. This systematic classification allows for a critical evaluation of current achievements, identification of methodological trends, and identification of challenges and promising areas open to future research.

Bubble position reflects temporal modeling capacity (vertical axis) and data efficiency (horizontal axis) (See Figure 4). Dashed arrows indicate the main evolutionary progression across architectural generations, from spatial feature extraction approaches toward hybrid end-to-end systems.

3.1. Spatial and Spatiotemporal Feature Extraction Approaches

One of the most widely used approaches for video-based SLR is the use of deep learning architectures to obtain spatial and spatiotemporal features. This methodology is effective in mitigating the problem of insufficient data and improving the representation of features similar to those that can be extracted from higher levels of visual abstraction [25].

Dynamic gesture recognition in American Sign Language (ASL) has been widely studied, particularly with 3D CNN-based approaches that have achieved good results. These models can classify up to 100 words in real time with high accuracy, although efficiency is often limited by the volume and diversity of available datasets [26]. Malaysian Sign Language (MSL) has been shown to achieve over 90% accuracy using the Convolutional Block Attention Module-ResNet (CBAM-ResNet) architecture. However, the results appear to be highly dependent on dataset quality, with over-classification and generalization decreasing in more complex and invisible scenarios [27].

Comparative studies of CNN and 3D CNN models have shown that 3D CNNs offer the best performance, achieving 91% accuracy in Trinidadian and Tobago Sign Language (TTSL) and 83% accuracy on the ASL dataset. Despite these good results, the evaluation of resilience to dataset and environmental changes has not been adequately considered, limiting the practical application of such systems in real-world interactive environments [28].

Building on previous research, this study [29] presents a deep learning-based SLR system aimed at reducing communication barriers in the deaf community. This system is based on a CNN architecture adapted to recognize the five basic signs of ASL. Furthermore, it supports a variety of input formats, including video sequences and real-time camera data, making the system more flexible in use. Despite these advantages, the proposed approach has primarily been evaluated under controlled conditions. Its performance on larger and more diverse datasets, as well as its robustness in real-world settings, has not yet been fully explored. This limitation highlights the need for further practical validation and scalability analysis.

A study of Afaan Oromo Sign Language (AOSL) achieved 92.98% accuracy using a pipeline with pre-processing, Gabor features, and a ResNet-50 classifier [30]. However, most studies still focus on isolated gestures, and generalization across datasets remains limited. This highlights the need for improved temporal modeling and adaptive representations for practical use.

Despite the strong benchmark performance reported across many CNN- and 3D CNN-based SLR systems, these architectures exhibit several structural limitations that affect their applicability in real-world environments. Most convolutional approaches primarily rely on appearance-level spatial representations, making them sensitive to illumination variation, background clutter, camera viewpoint changes, and signer-specific visual characteristics. Furthermore, although 3D CNNs partially capture short-term temporal dynamics, their ability to model long-range contextual dependencies remains limited due to the localized nature of convolutional operations. As a result, many models demonstrate high performance under controlled laboratory conditions but experience noticeable degradation in signer-independent and cross-dataset evaluation settings. These limitations explain the subsequent shift in SLR research toward recurrent, attention-based, and Transformer-driven architectures designed to improve temporal reasoning, contextual understanding, and generalization across diverse signing conditions.

Across the reviewed studies, CNN and 3D-CNN architectures demonstrate strong performance on controlled, single-dataset benchmarks, with accuracy figures frequently exceeding 90%. However, a consistent pattern emerges: performance degrades substantially when models are evaluated on unseen signers or cross-dataset scenarios. The primary reason is that convolutional features remain sensitive to appearance-level variations—background clutter, illumination changes, and signer-specific morphology—which are not adequately addressed by spatial convolution alone. Furthermore, most 3D-CNN models reviewed here were trained and evaluated on datasets of limited scale and homogeneity, which inflates reported accuracy and masks generalization weaknesses. These observations suggest that spatial feature extraction approaches require complementary temporal modeling and domain adaptation strategies to achieve robust real-world performance, particularly for low-resource sign languages where dataset diversity is inherently limited.

3.2. Temporal Sequence Modeling

Architectures with RNNs and LSTMs are commonly used in SLR systems to model time-dependent behavior. This preserves the ordinal context of movement along with video signals and provides a comprehensive explanation of the dynamics of the gesture process. For example, in experiments conducted with the IISL2020 database containing 11 characters of Indian Sign Language (ISL), a 97% accuracy was achieved by combining LSTM and GRU layers [31]. However, since these results are obtained with relatively simple and repetitive phrases, the effectiveness of this model has not been fully demonstrated in complex or continuous speech sequences. Similarly, a multimodal system developed for Mexican Sign Language (MSL) used LSTM and GRU layers to analyze temporal dynamics. This system combines hand, body, and facial functions with an OpenCV AI Kit with Depth (OAK-D) camera to measure depth (see Figure 5) [32]. However, such architectures still require further research into their potential for adaptation to other language groups or natural communication conditions.

The PoseNet-LSTM architecture achieved 98% accuracy in real-time text translation [33], but by integrating OpenCV, MediaPipe, and LSTM tools to recognize short phrases in Indian Sign Language (ISL), we achieved 99.17% accuracy in training and 97.78% accuracy in testing [34]. However, the fact that these models can perform poorly when motion overlaps and is constrained by short circuits is one of the issues that needs to be addressed. In summary, these works demonstrate the effectiveness of recurrent architectures for temporal modeling, while also exposing limitations in robustness, scalability, and generalization to complex real-world gestures.

In the context of Arabic Sign Language (ArSL) [35], keypoint representations are used, suggesting that pre-processing is crucial for model accuracy and achieving an accuracy of over 88.5%. In a recent study [36], a MediaPipe LSTM-based ASL recognition system was proposed, which accurately recognized three words, indicating reliable performance for very constrained tasks, but insufficient evidence for continuous or large-vocabulary SLR.

Recurrent architectures such as LSTM and GRU have demonstrated strong sequential modeling capabilities, achieving high accuracy on constrained tasks with short, well-defined phrase sequences. However, a fundamental limitation shared across these models is their tendency to compress entire gesture sequences into fixed-size hidden states, which inevitably leads to information loss for longer or more complex signing sequences. This compression bottleneck becomes particularly problematic in continuous SLR, where individual signs overlap and temporal boundaries are ambiguous. Additionally, the high accuracy figures reported in several studies, including 99.17% on ISL, are typically obtained under controlled laboratory conditions with limited vocabulary and a small number of signers, which does not reflect the complexity of real-world signing environments.

From a broader architectural perspective, recurrent sequence modeling represented a major advancement over purely spatial CNN-based approaches because it enabled explicit temporal reasoning across sign sequences. However, the reviewed studies consistently indicate that recurrent architectures encounter scalability limitations as sequence complexity increases. In tasks involving continuous SLR that are characterized by long signing streams, overlapping gestures, rapid motion transitions, and signer variability, recurrent hidden-state compression frequently results in information loss and unstable temporal alignment. Furthermore, the sequential processing nature of these models imposes limitations on parallelization efficiency and increases inference latency, consequently reducing their suitability for real-time deployment scenarios. These structural constraints were a significant motivation for the transition to attention-driven and Transformer-based architectures. These architectures provide direct modeling of long-range temporal dependencies without reliance on compressed sequential memory representations.

3.3. Transformer-Based and Attention-Driven SLR

Modern research is actively developing new approaches that allow systems not only to recognize individual gestures, but also to translate entire sentences in real time. The authors of [37] proposed a novel Transformer architecture jointly trained to handle both Continuous Sign Language Recognition (CSLR) and sign language translation (SLT). The system employed the Connectionist Temporal Classification (CTC) loss function, which connects recognition and translation tasks without requiring explicit temporal alignment. This design significantly improved performance on the challenging RWTH-PHOENIX-Weather-2014T dataset. The suggested networks outperformed both video-to-speech translation models and gloss-to-speech translation models, in some cases boosting translation quality by more than half, achieving Bilingual Evaluation Understudy with 4-gram (BLEU-4) scores that increased from 9.58 to 21.80.

The Transformer encoder and decoder utilize a scaled dot product attention mechanism, which is formulated as follows:

$$\mathrm{Attention}(\mathrm{Q},\mathrm{K},\mathrm{V})=\mathrm{softmax}\left({\displaystyle \frac{\mathrm{Q}{\mathrm{K}}^{\mathrm{T}}}{\sqrt{{\mathrm{d}}_{\mathrm{k}}}}}\right)\mathrm{V}$$

(1)

The formulas presented here describe the scaled dot product attention mechanism, which is a core component of both encoding and decoding in the Transformer architecture, where $Q$ and $K$ denote the query, key, and value representations, respectively, and $d_k$ represents the key dimensionality scaling factor.

To further enhance representation capacity, multi-head attention is applied, defined as

$$\mathrm{MultiHead}(\mathrm{Q},\mathrm{K},\mathrm{V})=\mathrm{Concat}({\mathrm{head}}_1,\dots ,{\mathrm{head}}_{\mathrm{h}}){\mathrm{W}}^{\mathrm{O}}$$

(2)

where each head is computed as

$${\mathrm{head}}_i=\mathrm{Attention}({\mathrm{QW}}_i^Q,{\mathrm{KW}}_i^K,{\mathrm{VW}}_i^V)$$

In this formulation, $h$ denotes the number of parallel attention heads, while ${\mathrm{W}}_i^Q,{\mathrm{W}}_i^K$, and ${\mathrm{W}}_i^V$ represent the learnable projection matrices for the query, key, and value representations, respectively.

By computing $h$ independent attention functions in parallel, the multi-head mechanism enables the model to jointly attend to information from different representation subspaces at different temporal positions. In the context of video-based SLR, this property is particularly valuable: individual attention heads can specialize in capturing distinct aspects of gesture articulation, such as handshape trajectories, upper-body posture transitions, and facial expression dynamics. Since sign language meaning is conveyed simultaneously across multiple articulators, this parallel multi-channel attention provides a more complete and semantically grounded representation of the sign sequence than single-head attention alone.

$$\mathrm{Decoder}\mathrm{Output}=\mathrm{softmax}\left({\displaystyle \frac{{\mathrm{Q}}_{\mathrm{d}\mathrm{e}\mathrm{c}}{\mathrm{K}}_{\mathrm{e}\mathrm{n}\mathrm{c}}}{\sqrt{{\mathrm{d}}_{\mathrm{k}}}}}\right){\mathrm{V}}_{\mathrm{e}\mathrm{n}\mathrm{c}}$$

(3)

where $Q_{dec}$ denotes the decoder query representations, while $K_{enc}$ and $V_{enc}$ correspond to the key and value representations obtained from the encoded video features. The scaling factor $\sqrt{d_k}$ stabilizes gradient magnitudes during training by preventing the dot products from growing excessively large in high-dimensional spaces.

This cross-attention mechanism enables the decoder to dynamically align predicted glosses or words with the most informative temporal regions of the input sign sequence. Such alignment has been demonstrated to enhance contextual translation consistency and strengthen the modeling of long-range dependencies between gesture phases and linguistic units in continuous SLR tasks. In practice, cross-attention significantly improves sequence-level recognition and translation quality. However, the global attention operation also increases computational cost and memory requirements, which remain important limitations for real-time and low-resource deployment scenarios.

$$\mathcal{L}=-{\displaystyle \sum _{t=1}^T\log P(y_t}|y_{<t},X)$$

(4)

where $T$ denotes the number of output tokens in the target sequence, $y_t$ represents the target token at position $t$, and $X$ corresponds to the encoded video feature sequence.

Minimizing $\mathcal{L}$ during training encourages the model to assign progressively higher probability mass to the correct output tokens at each decoding step, effectively penalizing confident incorrect predictions more heavily than uncertain ones. This loss function trains both the recognition and translation components jointly in an end-to-end manner, creating a unified optimization objective that connects visual feature extraction, temporal modeling, and sequence generation. Lower values of $\mathcal{L}$ correspond to closer alignment between the predicted probability distributions and the reference translation sequences, which empirically correlates with improved BLEU scores and reduced Word Error Rate (WER) on standard SLR benchmarks such as RWTH-PHOENIX-Weather-2014T [38]. For more accurate and reliable recognition of sign language [39], studies take into account not only hand gestures, but also body movements and facial expressions.

In [40], a multi-branch architecture was proposed to address the computational complexity and limitations of Transformer-based models for Korean SL. The model integrates CNN and Transformer features, lightweight Multi-Head Self-Attention (MHSA), and a Grain module with 3×3 convolutions, achieving 89.0% accuracy on a 77-label dataset and 98.3% on a laboratory dataset while reducing computational cost.

Within this multi-branch Transformer-based architecture, a scaled dot product attention mechanism is applied, defined as

$$\mathit{Attention}(Q,K,V)=soft\max \left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(5)

where $Q$, $K$ and $V$ denote the query, key, and value representations, respectively, and $d_k$ represents the scaling factor associated with the key dimensionality. This approach estimates the pairwise similarity of all temporal features, which are then converted into attention weights that can be normalized by the softmax function, allowing the model to prioritize the most important parts of the feature sequence. In practice, this allows Transformers to capture long-term dependencies between gesture components, including hand gestures, body posture changes, and contextual coarticulation within cues that span sequences. Unlike other iterative architectures, the attention-based model directly links to distant frames, making CSLR very effective.

$$P(y|x)=soft\max (W{h}_{cls}+b)$$

(6)

where $h_{cls}$ denotes the global sequence representation produced by the Transformer encoder, while $W$ and $b$ correspond to the learnable classification parameters.

The softmax function converts the linear classification scores into normalized probability values across all label classes, allowing the model to assign a confidence score to each candidate label class, enabling probabilistic decisions during inference.

From a representation learning perspective, the Transformer encoder learns context-sensitive embeddings that capture the temporal gesture evolution across sign sequences, and the classification layer maps these embeddings to discrete sign label probabilities.

Although Transformer-based architectures have demonstrated high performance in SLR tasks, several challenges remain: most Transformer-based SLR models rely on large labeled datasets, limiting their applicability to resource-scarce SLs.

In SLR, Transformer methods rely on visual cues and skeletal coordinate sequences rather than accurately modeling the geometric relationships in the joints of the hand, which resemble dynamic variations in joint angles. Transformers are traditionally weighted like all temporal cues, which reduces the sensitivity to subtle local motion that is essential for visual cue separation. Therefore, the inclusion of geometric features and adaptive temporal attention in the Transformer circuit model provides a promising direction for robustness and generalization in SLR systems.

Although Transformer-based architectures enable integrated end-to-end training of SLR and translation, several limitations remain. First, Transformer models require large annotated datasets, which are difficult to obtain for many SLs. Second, the models rely primarily on visual token embeddings extracted from raw video features, which may still contain redundant background information. A third obstacle is that the trans-architecture directly ignores the geometric structure of the arms (e.g., the corners of the frame). Furthermore, since the mechanism of transient attention is the same for all tokens, there is a risk of incomplete activation of the subtle dynamics of finger movements. For this reason, combining geometric properties with a flexible focusing mechanism to enhance the reliability and adaptability of the system remains an unresolved problem.

A cross-attention (CA) module was proposed in [41] to process multimodal data such as RGB and optical flow in a single system for the purpose of analyzing complex gestures. Although this proposed approach reduces computational costs and improves recognition quality, it does not solve the problem of maintaining a balance between model complexity and speed. While the Contrastive Visual–Textual SLR (CVT-SLR) system described in [42] outperforms both single-modal and multimodal methods, its reliance on pre-trained models may limit its flexibility in new databases or new domains.

To facilitate CSLR, the Inter-Gloss Attention (IIGA) module was used in [43]. Although this method performs well in terms of WER, it requires careful pre-processing steps such as background removal to work effectively, which means that the system remains sensitive to the interference from the real environment.

Also, in [44], the authors proposed a Relative Spatial Transformer (RST) architecture that encodes spatial relative relationships instead of absolute coordinates. While this solution improves the model’s ability to distinguish spatial relationships, its scalability to larger vocabularies or specific use cases remains an open question. In summary, although Transformer-based SLR models offer high accuracy and the ability to analyze complex sequences, their high computational requirements, dependence on database quality, and inability to fully handle subtle geometric features such as finger gestures remain major challenges for researchers.

Transformer-based architectures signify a substantial conceptual shift in the domain of SLR research, wherein they supersede conventional compressed sequential memory mechanisms, instead favoring direct global attention modeling across the entire sign sequences. This enables improved contextual reasoning, stronger long-range temporal dependency modeling, and more effective integration of multimodal information compared to recurrent architectures. The reviewed studies demonstrate that Transformers consistently achieve superior benchmark performance in continuous SLR and sign language translation tasks, particularly on large-scale datasets with complex temporal structure.

Nevertheless, there are several significant limitations that have yet to be satisfactorily addressed. Firstly, Transformer models generally necessitate substantial annotated datasets and considerable computational resources, which limits their applicability to low-resource sign languages and edge-device deployment scenarios. Secondly, it has been demonstrated that attention mechanisms in themselves do not inherently encode geometric relationships between finger joints and body articulators. This has the effect of limiting sensitivity to subtle fine-grained motion patterns that are linguistically important in sign communication. Thirdly, the elevated parameter complexity of numerous Transformer-based systems has been demonstrated to increase the cost of training and to reduce the efficiency of inference in real-time applications. The aforementioned limitations have led to a surge of interest in graph-based representations, lightweight attention mechanisms, and hybrid multimodal architectures. These architectures are designed to enhance geometric awareness, computational efficiency, and robustness under real-world signing conditions.

3.4. Graph-Based Spatial–Temporal Modeling for SLR

GCN can effectively model the spatial and temporal patterns in a sign language sequence by analyzing the coordinates in the skeletal joints. In the pose-based word-level sign language recognition (WSLR) framework, spatial and temporal features are separated in [45], spatial dependence is determined by GCN, and temporal relationships are modeled using the Bidirectional Transformer Encoder (BERT). While this approach improves prediction accuracy by approximately 5% on the WLASL dataset compared to conventional pose-based methods, its generalization to diverse datasets and complex real-world conditions remains underexplored.

Complex video backgrounds and the challenges posed by multiple lighting scenarios were addressed with structures based on multiple frames. The multi-stream Sign Language Image Deformation Network (SL-IDN) is used for study by researchers [46], with 27 main nodes across four feature streams, and offering strong generalization across WLASL, AUTSL, and CSL datasets, yet performance on large-scale, unconstrained videos has not been fully validated. With a low computational resource requirement during comparison, multi-branch attention models that can incorporate spatial and temporal graph features for dynamic hand movements with deep neural representations have achieved accuracies of 94.12%, 92.00%, and 97.01% on the MSRA, DHG, and SHREC’17 datasets, respectively [47]. However, these high accuracies often rely on preprocessed or idealized skeleton data, which may limit robustness in noisy or occluded environments.

The lightweight pipelines incorporated into the residual GCN containing constrained configurations effectively model the spatial and temporal dependencies, reduce computational cost, and maintain high performance, which yielded an accuracy rate of 27.62% in WLASL-300, 26.97% in WLASL-1000, and 100% in LSA-64 [48]. Despite these results, performance degradation is evident on large-scale and complex datasets, indicating that scalability is a major limitation.

To better capture the local topology of the arm and wrist, HAND-aware graph convolutional networks (HA-GCNs) outperformed previously presented models in AUTSL by using subgraphs of all joints and bones, providing a drop-graph function that included the dataset [49]. Graph convolution focusing networks (GCARs) improved their performance by integrating multi-stage graph scrolling and focusing mechanisms [50]. Similarly, models including a spatial–temporal graph convolutional network (ST-GCN), BiLSTM, and 3D pose parameterization showed competitive results on the RWTH-PHOENIX-2014 and Chinese datasets (Split I) [51].

However, these approaches remain computationally intensive, often requiring careful hyperparameter tuning and limiting their real-time use. Lightweight models based on handcrafted frameworks and residual GCNs showed superior results on the WLASL and MINDS-Libras datasets, while reducing computational cost [52]. This implies a trade-off between model complexity, accuracy, and computational efficiency, which must be carefully balanced depending on the application context. GCN-based recognition is advancing automated translation systems by shifting the focus from raw pixel analysis to structured pose and motion analysis. However, performance heavily relies on idealized skeletons, and combining multiple models increases complexity and the risk of over-classification. These limitations require the combination of graphical representations and additional modules to enhance robustness, capture spatiotemporal dependencies, and generalize to invisible signers.

GCN-based approaches offer a fundamentally different representational paradigm compared to pixel-based methods: by explicitly modeling the skeletal graph structure of the human body, they achieve greater robustness to background noise and illumination variation. The reviewed studies confirm that GCNs consistently outperform pose-based baselines on datasets such as WLASL and AUTSL when skeleton data quality is high. However, this strength becomes a vulnerability in real-world conditions: GCN performance degrades significantly when pose estimation fails due to the occlusion, fast motion, or low image resolution, because erroneous skeletal inputs propagate directly into graph convolution. Furthermore, most reviewed GCN models rely on idealized, preprocessed skeleton data that may not be available in deployment settings. The computational cost of multi-stream GCN architectures also limits their applicability in real-time and mobile scenarios. These trade-offs suggest that GCNs are most effective as a complementary component within hybrid architectures, rather than as standalone solutions.

From a system-level perspective, GCN-based approaches address one of the central weaknesses of appearance-driven SLR models by explicitly modeling the structural relationships between body joints, hand articulators, and motion trajectories. This representation significantly improves robustness to background variability, illumination changes, and certain forms of visual noise that commonly affect RGB-based methods. As a result, graph-based representations have become increasingly important for signer-independent recognition and multimodal fusion pipelines.

Nevertheless, the reviewed studies also reveal several persistent challenges. The effectiveness of GCN architectures remains highly dependent on the quality and stability of pose estimation, meaning that errors caused by occlusion, motion blur, low-resolution video, or inaccurate skeletal extraction directly propagate into graph representations and reduce recognition reliability. In addition, many multi-stream GCN systems introduce considerable computational overhead due to the simultaneous processing of spatial, temporal, and multimodal graph structures. These constraints limit scalability and complicate deployment in real-time and mobile-assistive systems. Consequently, recent research trends increasingly explore lightweight graph architectures, adaptive graph attention mechanisms, and hybrid Transformer–GCN frameworks that attempt to balance structural awareness with computational efficiency and temporal modeling capacity.

3.5. Hybrid End-to-End SLR and Translation Models

Sign language interpretation models tend to be computationally expensive and have an insufficient understanding of the global context. The Transformer–CNN hybrid approach combines the local functions of a CNN with the global representation of a Vision Transformer (ViT) to effectively capture subtle gestures and achieve 99.97% accuracy, 110 Frames Per Second (FPS), and just 5.0 Gigabytes of Giga Floating-Point Operations Per Second (GFLOP) on Alphabet’s ASL dataset [53]. However, these results are often obtained on controlled datasets, and the model’s robustness under real-world variations, such as occlusion or lighting changes, remains unclear.

Kondo et al. [54] have recently presented their work on the efficiency of capturing angular features present in Japanese SLR cameras and have compared the results of CNN and ViT-based architectures, as shown in Figure 6.

Let $p_i$, $p_j$, and $p_k$ $\in$$R^3$ denote the 3D coordinates of finger joints extracted using the MediaPipe pose estimation framework. The angular feature $\theta$ represents the angle formed at the joint $p_j$ by the vectors connecting adjacent joints:

$$\theta =\arccos \left({\displaystyle \frac{(p_i-p_j)\times (p_k-p_j)}{\parallel p_i-p_j\parallel \parallel p_i-p_j\parallel }}\right)$$

(7)

where $p_i$, $p_j$ and $p_k$ denote the 3D coordinates of adjacent finger joints, and $\theta$ represents the angular relationship between the corresponding skeletal vectors. The vector of values used as a divisor is used to smooth the values and to ensure that they are stable over the current measurement and the distance between the shots. Here, the function is a normalized dot product of the function, which converts the angle parameter measured in radians into the format of the view. These angle parameters provide a compact and reliable description of finger movements, and are less sensitive to changes in translation and illumination than when using raw coordinate symbols. The resulting angle feature vectors are stored in a temporary location and are introduced as input tokens through the ViT encoder feature in the SLR pipeline.

Despite the achievements described above, several limitations remain unresolved across current SLR approaches. RGB-based methods remain susceptible to background noise and illumination changes, while skeleton-based methods partially mitigate these issues but rely on raw Cartesian joint coordinates that do not capture geometric relationships between finger joints. The angular feature representations offer a more compact and robust alternative, as they are invariant to translation and scale changes, making them better suited for real-world deployment. Using a skeleton, these methods partially eliminate the limitations of using joint coordinates in visualization, which are often characterized by not taking into account the geometric relationships of the finger joints, and use raw (unprocessed) Cartesian coordinates.

The angular difference representations given by the coordinates of the joints in the skeleton represent a compact form of the movement of the joints of the hand, and the model, in turn, produces semantically meaningful descriptions. These characteristics are not inherent in the transmission, zoom adaptation, and changes in lighting conditions, factors that make them suitable for use in real-world SLR systems.

Several research questions remain open in this area. First, the integration of angular skeletal representations with Transformer-based architectures for contextual modeling remains limited. Second, the deep temporal modeling of angular motion dynamics in sign sequences has not been fully addressed. Third, systematic comparative studies evaluating angular feature implementations across multiple deep learning architectures under controlled experimental conditions are lacking. These gaps represent concrete directions for future investigation. Second, the problem of deep temporal modeling of sign language sequences in angular motion dynamics, which is focused on frame-level classification performance, has not yet been fully addressed in the current study. Third, comparative studies that can assess how similar angular feature implementations are used in many deep learning architectures under controlled experimental conditions remain lacking.

Research on hybrid architectures involving convolutional spatial feature extraction and global contextual modeling in the Transformer when implemented on geometric skeleton representations is limited. And the generalization efficiency of SLR systems that rely on angular features in real-world writing situations, such as signers and datasets, is still an open question.

Subsequent research should systematically investigate Transformer-based and hybrid deep learning architectures that incorporate angular skeletal features, with a focus on temporal sequence modeling, robustness to real-world variation, and computational efficiency for practical SLR deployment.

Lip-reading-based Audio–Visual Speech Recognition (AVSR) models using an Active Shape Model (ASM) and CNN reduced WER by 6.59% and reached 95% lip-reading accuracy [55]. A hybrid CNN–BiLSTM–GAN pipeline using MediaPipe enabled recognition, translation, and gesture generation, achieving a classification accuracy above 95% and a BLEU of 38.06 [56]. TwoStream-SLR/SLT models with dual visual encoders achieved state-of-the-art SLR/SLT performance [57]. Nevertheless, many hybrid models still use static modality fusion or simple temporal aggregation, ignoring adaptive weighting of manual and non-manual features, which limits their robustness in natural sign language communication.

A hybrid deep learning system combining CNN-based spatial features, manual keypoint priors, LSTM temporal modeling, and Transformer-based long-range dependencies was proposed for Kannada SL, which achieved 97.6% training, 96.75% validation, and 81% testing accuracy on a new medical domain dataset and outperformed traditional CNN-LSTM and manual keypoint (HKP)-LSTM baselines [58]. A multi-scale context-aware network (MSCA-Net) with multi-scale motion and temporal modules achieved the highest CSLR results [59]. Specialized models improved ASL question recognition (98.91% accuracy) [60], boosted efficiency with spiking neural networks [61], or enabled fast Sistem Isyarat Bahasa Indonesia (SIBI)-to-Indonesian translation (96–99% accuracy) [62].

Region-specific systems included Uighur SLR using improved You Only Look Once version 7 (YOLOv7) [63], Ethiopian Sign Language (EthSL) with 8.82% WER [64], and Azerbaijani Sign Language (AzSL), achieving 94% accuracy [65]. Lightweight detection models like YOLOv8 [66], and Single-Shot MultiBox Detector (SSD) with MobileNet V2 [67], and Dual-branch Shuffle Attention Mechanism–You Only Look Once version X (DSA-YOLOX) [68] improved real-time CSLR with reduced cost. Advanced fusion methods such as Multi-Scaled feature Fusion–Euclidean Transformer (MSF-ET) [69] and Dual-Stream Full-Hand networks [70] set new benchmarks on CSL and Ankara University Turkish Sign Language (AUTSL) datasets.

Additional studies explored a Long-term Recurrent Convolutional Network (LRCN) for ASL [71] and introduced a multimodal ArSL dataset with MobileNet-LSTM, achieving 99.7% dependent-mode accuracy [72], as shown in Figure 7.

To formally represent the multimodal feature integration, the fused feature vector is defined as

$$F_{fusion}=[F_{manual}||F_{nonmanual}]$$

(8)

where $F_{manual}$ and $F_{nonmanual}$ denote the feature representations extracted from manual and non-manual signing components, respectively. Vector concatenation denoted by the operator || produces a unified multimodal feature vector, integrating manual and non-manual information. If the dimensionality of manual features is $d_m$ and non-manual features is $d_n$, the fused vector resides in $R^{d_m+d_n}$. This finding opens the way for a recognition model to learn about cross-modality dependencies between hand gestures and facial features, which is important for sign language in this semantic sense, which is transmitted over multiple channels. However, it is not possible to predict that these two modalities can contribute similarly to the causes of all-encompassing sign language gestures.

$$P(y|x)=soft\max (W{F}_{fusion}+b)$$

(9)

where $F_{fusion}$ denotes the fused multimodal representation, while $W$ and $b$ correspond to the learnable classification parameters. The fused feature vector provides a joint representation of manual and non-manual linguistic information. Following the linear classifier, the softmax function transforms its output into a normalized probability for all feature classes, which generates a visual representation of the features that represent facial features or head movements that is sensitive to the semantic differences in the model. During training, this output layer is typically optimized using a cross-entropy loss, allowing the model to learn discriminative multimodal feature representations for SLR.

Hybrid multimodal architectures currently represent the dominant research direction in advanced SLR because they combine the complementary strengths of multiple representational paradigms, including CNN-based spatial extraction, Transformer-based contextual modeling, graph-based structural reasoning, and multimodal fusion of manual and non-manual linguistic cues. The reviewed studies consistently demonstrate that such systems achieve the strongest benchmark performance, particularly in continuous SLR and sign language translation tasks involving complex temporal semantics.

However, the increasing architectural complexity of hybrid systems gives rise to a number of significant practical challenges. Firstly, the integration of multiple feature streams substantially increases computational cost, memory consumption, and training requirements, which limits deployment feasibility on mobile and embedded assistive platforms. Secondly, many multimodal fusion strategies rely on static weighting mechanisms that assume equal importance of all modalities. However, the contribution of facial expressions, hand motion, pose dynamics, and contextual information varies significantly across sign languages and communication scenarios. Thirdly, it is evident that a significant proportion of reported benchmark results are obtained under controlled experimental conditions using relatively homogeneous datasets. This may result in an overestimation of real-world generalization capability. These observations suggest that future progress in SLR will depend not only on increasing architectural complexity but also on developing adaptive fusion mechanisms, lightweight deployment-oriented models, and more ecologically valid evaluation protocols capable of reflecting realistic signing environments.

3.6. Transfer of Training and Other Methods

Transfer learning, feature enrichment, and auxiliary strategies are used to improve the accuracy of SLR. Enriching WSLR models with subtitle-aligned features and external memory for class centroids was proposed in [73]. While this improves recognition, the reliance on external annotations may limit application to domains without aligned subtitles, which highlights the reliance on leading datasets. The ChaLearn LAP Large-Scale Signer-Independent Isolated SLR Challenge (CVPR 2021) [74] evaluated multimodal approaches including RGB, depth, pose estimation, transfer learning, ensembles, and spatiotemporal models on the AUTSL dataset (226 gestures, 36K videos). Although the accuracy exceeded 96%, similar gestures, authenticity, and interpretation issues remained, highlighting the need for balanced datasets and contextual attention.

The research in [75] introduced Sign Hidden-Unit BERT (SHuBERT) (Figure 8), a self-supervised encoder trained on 1000 h of ASL videos, extending SHuBERT to multi-stream visual input (hands, face, body). Using MediaPipe keypoints, DINOv2 features, and K-means clustering, SHuBERT achieved state-of-the-art performance in ASL translation (How2Sign, OpenASL, Few-shot Learning Evaluation for Universal Representation of Sign Languages (FLEURS-ASL) and isolated recognition (ASL Citizen, SEM-LEX)).

To formally define the self-supervised multi-stream cluster prediction objective, the loss function can be expressed as

$${\mathcal{L}}_{cluster}=-{\displaystyle \sum _{s=1}^S{\displaystyle \sum _{i=1}^N{q}_i^{(s)}}\log }{p}_i^{(s)}$$

(10)

where $S$ denotes the number of modality streams, $q_i^{(s)}$ represents the pseudo-label assignment probability for sample $i$ in stream $s$, and $p_i^{(s)}$ corresponds to the predicted cluster distribution produced by the model.

This self-supervised objective enables the learning of shared latent representations across multiple feature streams without requiring extensive manual annotation. In the context of video-based SLR, such learning strategies are particularly valuable for low-resource sign languages where large-scale labeled datasets are difficult to obtain. However, cluster-based self-supervised objectives primarily capture global semantic structure and may insufficiently model fine-grained temporal gesture dynamics and subtle finger articulation patterns required for continuous SLR tasks.

$${\mathcal{L}}_{consistency}={\displaystyle \sum _{s=1}^S{\displaystyle \sum _{t=1}^S||z^{(s)}}-z^{(t)}}|{|}_2^2$$

(11)

where $z^{(s)}$ and $z^{(t)}$ denote latent feature embeddings extracted from different modality streams.

The objective minimizes the Euclidean distance between embeddings from different streams corresponding to the same video segment. This encourages the model to generate gesture representations that are independent of a particular modality, reducing its dependence on a single source of information. Maintaining consistency between different data streams ensures consistent SLR across different writing methods, changing illumination and partial occlusion. By storing additional information appropriate for the model’s modality, it creates alignment through hidden embeddings across multiple threads to provide the same semantic identity for all. Although the dependence on the sign language datasets identified in the self-directed learning model has been reduced, there are limitations to the methods for clustering across multiple streams. Specifically, they cannot adequately capture the temporal dynamics of sign language movements during task performance, which involves static clustering, which is more commonly used in these methods. Furthermore, cluster-based self-supervised learning typically focuses on the semantics of the entire video, which may not be able to adequately model fine-grained finger movements or short-term movement patterns.

Additionally, cross-stream consistency constraints often assume that the importance of all modality streams is uniform, which may not reflect real-world sign language scenarios where certain modalities dominate depending on the gesture type. Therefore, an adaptive multi-stream self-supervised learning framework incorporating temporal attention mechanisms and fine-grained geometric feature modeling remains an open research topic.

Research in [76] proposed an integrated Isolated Sign Language Recognition (ISLR) pipeline that combines target data augmentation, regression heads, and an Intersection-Over-Union (IoU)-balanced classification loss. To improve generalizability and explainability, research in [77] developed an American Sign Language Knowledge Graph (ASLKG) from 12 expert sources. A neuro-symbolic model trained on the ASLKG achieved an ISR accuracy of 91%, an unseen semantic feature prediction accuracy of 14%, and a YouTube–ASL topic classification accuracy of 36%.

Subspace-based and multimodal fusion methods [78,79] improved training accuracy, but non-visual generalization and interlingual translation accuracy reveal persistent issues with domain shift, class confusion, and dependence on high-quality multimodal inputs. Although transfer learning and self-referential strategies offer advantages, their effectiveness depends on the quality of the learning data, which requires careful validation and dataset management that includes accurate and contextually relevant key assumptions.

Transfer learning and self-supervised approaches represent the most promising direction for addressing data scarcity in low-resource SLR. Models such as SHuBERT demonstrate that pre-training on large unlabeled sign language video corpora can achieve competitive performance with significantly less labeled data. However, the effectiveness of transfer learning depends critically on the domain proximity between the source and target sign languages: features learned from ASL may not transfer effectively to structurally distinct languages such as KSL, which differs in handshape inventory, spatial grammar, and non-manual marker usage. Furthermore, cluster-based self-supervised objectives typically optimize for global video-level semantics, which may inadequately capture the fine-grained temporal dynamics of individual phonological features. These observations highlight the need for language-specific pre-training corpora and temporally aware self-supervised objectives as priorities for future research in low-resource SLR.

4. Overview and Contribution to Datasets

The WLASL dataset has been expanded to include over 2000 words performed by more than 100 sign artists. As one of the largest publicly available resources for word-level gesture recognition in ASL, it enables experimentation with diverse deep learning approaches, including holistic visual perception and two-dimensional human pose estimation. However, despite its scale, the dataset may still be limited in participant diversity and environmental conditions, which can impact model generalization. The proposed Pose-based Time Graph Convolution Networks (Pose-TGCNs) have been shown to improve recognition performance [80].

A scalable method for collecting data for gesture recognition in continuous videos is proposed. Using substandard broadcast subtitles in conjunction with a keyword detection methodology, gesture exemplars were automatically localized, resulting in a lexicon comprising 1000 characters from 1000 h of video footage. The British Sign Language 1000-class (BSL-1K) dataset, a substantial collection of BSL data, is accessible to the public and has been demonstrated to serve as a robust foundation for pre-learning models of other SLs, the Multilingual Sign Language Dataset (MSASL), and WLASL [81]. Due to its public accessibility and structured organization, this dataset is valuable for replicating experiments and conducting comparative analyses, but it may still lack detailed spatiotemporal annotations.

Two Arabic Sign Language datasets, ArASL2018 and ArASL2021, were presented in the study in [82], which cover gestures in simple and complex situations, respectively. Both datasets are publicly available and contribute to research in the field of assistive technologies. Nevertheless, the relatively small size compared to ASL datasets may limit deep learning model scalability.

The largest database of Saudi SL (SSL) has been created, comprising 293 signs, 33 SLs, and 145,035 samples across 10 different thematic areas [83]. However, the paper does not mention dataset availability, which significantly limits reproducibility and the broader scientific impact.

A large-scale dataset of ASL with annotations of six Recognizing Phonological Properties (RPPs) has been partially released [84]. Graph neural networks applied to skeletal features demonstrate promising recognition of these properties, including unseen patterns. However, restricted data access limits external validation and benchmarking.

A large-scale dataset of Turkish Sign Language (TSL) is available to the public. This dataset contains more than 22,000 individual videos, including 744 unique Turkish texts created by six native sign language specialists [85]. Its structured design supports cross-linguistic pre-training, but the limited number of performers may constrain the dataset’s variability.

A multimodal corpus has been created to improve the understanding of religious aspects by deaf people using ArSL. The dataset includes audio signals and 262 Arabic Sign Language videos with text written by two experts [86], but it is not publicly available.

The Temporal Lift Pool (TLP) method has been improved in [87] for the aggregation of time series features. Experiments show that it has higher accuracy compared to traditional methods; however, public access to the datasets used is limited, which limits its general applicability. The ASL Citizen dataset [88] is a public and valuable research tool, while the Slovo dataset for Russian Sign Language [89] contains 20,000 Full HD videos covering 1000 classes created by 194 performers. Although this dataset allows for high accuracy in gesture classification, it is necessary to consider possible errors due to the homogeneity of participant selection and recording conditions.

Extending 3D sign language resources, SignAvatars provides a multi-narrative dataset of 70,000 videos and 8.34 million frames [90]. The main advantage of including ASL and HamNoSys gestures with the multimodal structure of the dataset is that they facilitate the interpretation and creation of 3D sign language. However, the high complexity of the data can make it difficult to train standard models and increase storage requirements.

The AUTSL dataset contains 226 gestures performed by 43 participants and contains 38,336 RGB-D images [91]. The diversity of backgrounds and the complexity of poses make it a valuable benchmark dataset, but a significant decrease in accuracy to 62.02% is observed in user-independent estimation, which indicates the difficulty of generalizing the model. Despite the limitations of a widely used dataset such as AUTSL, this issue highlights the need to develop specialized corpora for specific sign languages with limited resources. In this regard, the creation of our own dataset is an important step to improve the recognition quality and increase the generalization ability of the models. The next section describes the development process of our developed KSL dataset.

Collected Dataset for KSL

The Kazakh Sign Language Video Dataset (KSLVD) [92] was developed to address the critical absence of sentence-level resources for KSL, which remains one of the most underrepresented sign languages in the deep learning literature. The dataset comprises 18,000 augmented video sequences representing 200 linguistically validated sentence-level phrases, recorded by three native KSL signers under naturalistic conditions with intentional variability in background, lighting, and signer position. Three native speakers of KSL, all early bilinguals and graduates of specialized education programs for the deaf, conducted the recordings in accordance with ethical requirements after obtaining written informed consent and granting copyright agreements. The feature extraction was performed using MediaPipe, yielding 225 skeletal keypoints per frame; sequences were normalized to 50 frames. In comparison to the FluentSigners-50 dataset [93], which contains 173 lexical items, KSLVD provides broader vocabulary coverage and greater syntactic diversity at the sentence level (see

Table 2. Comparative analysis of selected sign language datasets.

Reference	Dataset	Classes	Videos	Strengths	Limitations
[80]	WLASL	2000+	21,083	Large-scale, diverse signers, ASL	Limited environmental diversity
[91]	AUTSL	226	38,336	RGB-D, multimodal, varied backgrounds	62% signer-independent accuracy
[81]	BSL-1K	1000	1000 h	Scalable co-articulated SL, mouthing cues	Limited spatiotemporal annotations
[85]	Bosphorus Sign22k	744	22,000+	Large Turkish SL, cross-lingual pre-training	Limited number of signers (6)
[88]	ASL Citizen	2731	83,399	Community-sourced, high signer diversity	Isolated signs only
[89]	Slovo	1000	20,000	Full HD, Russian SL, 194 signers	Participant homogeneity
[93]	FluentSigners-50	173	43,250	Signer-independent KSL benchmark	No sentence-level, limited vocabulary
[92]	KSLVD	200	18,000	Sentence-level KSL, linguistic validation	Not publicly available, limited size

).

The dataset is currently not publicly available; however, it represents a structured linguistic resource for KSL research, and its detailed development methodology is described in a companion publication [92].

5. Evaluation Metrics

A comprehensive assessment of the performance of SLR systems consists of a set of indicators that quantitatively characterize the recognition accuracy, error rate, generalization ability of the model, and its performance in real time. Accuracy is the main indicator that reflects the proportion of correctly classified gestures [94,95]. In addition, to determine the practical applicability of the system, its generalization ability, tolerance to input data distortion, and the quality of contextual interpretation are additionally taken into account.

SLR methods have shown high accuracy rates in various language contexts. For example, a South Indian SL classification method based on optical flow and an Inflated-3D model achieved an average accuracy of 0.8709 when subsequently translated into Kannada SL in real time [96]. A model for ArSL using two-dimensional (2D) keypoints achieved 98.39% accuracy for dynamic gestures and 88.89% accuracy for static gestures [97]. Using a CNN with the CBAM module for Malaysian Sign Language has achieved an accuracy rate exceeding 90% [98], while for Egyptian Sign Language, CNN and CNN-LSTM models achieved 90% and 72% accuracy, respectively [99]. To develop comparative analysis, new datasets for Indian Sign Language, INSIGNVID for ISL [100], as well as specialized corpora for Hong Kong SL [101], KSL [102], Bengali SL [103], and Bangla SL [104], have been proposed. These resources have contributed to the development of robust models. However, differences in dataset size, annotation quality, and writing conditions lead to heterogeneity, which can reduce the robustness of comparative analysis.

Modern architectures have shown significant improvements in recognition performance. In particular, 3D CNN-based models have achieved 97.5%, 99.75%, and 98% accuracy for Tamil, ISL, and ASL, respectively [105]. The CNN-LSTM architecture achieved 97% accuracy in a study [106], while the SKResNet-TCN model achieved 100% accuracy on the LSA64 dataset [107]. ViT and Transformer achieved 99.53% accuracy [108]. An ArSL recognition system with a CNN-LSTM–Self-Attention Multilayer Perceptron (CNN-LSTM-SelfMLP) achieved 87.69% on a segmented video dataset in [109]. An event-based dataset for event-based cameras was presented in [110], where a convolutional neural network achieved up to 77% accuracy. In [111], an LSTM model for Indian SL achieved 96% accuracy on the training set and 87% on the test set, while in [112], a ResNet-LSTM model for Argentine SL achieved 86.25% accuracy. Despite such high performance, many models show significant performance degradation in user-independent settings, which highlights the need for further research into generalization issues in this area.

The work in [113] proposed a method for dynamically adjusting the sampling rate to improve recognition accuracy in mobile environments, increasing performance to 83.64% (Top 5) and 66.54% (Top 1).

Other regional studies confirmed this trend. An ASL recognition system achieved 94% accuracy [114]. Moroccan SL recognition with a 3D CNN reached 99.6% [115]. ASL gestures and non-manual markers from RGB-D achieved 92.88% accuracy [116]. Experiments on the Mexican SL recognition keypoint sign signal dataset showed that the proposed arm-movement approach achieved 85.78% accuracy using six keypoints [117]. Chinese SL with optimized CNN-CB reached 94.88% [118]. Filipino SL with LSTM and MediaPipe reached 94% [119]. Arabic SL recognition with CNN-RNN reached 98% [120]. The study in [121] proposed a Mexican SL recognition system based on an RNN using spatial tracking of hands and facial expressions, achieving 0.93 accuracy in offline mode. Finally, multilingual systems are also emerging. A Visual Geometry Group (VGG)-19 network-based model achieved 99.9% accuracy for English signs and supported translation into Indian regional languages [122]. However, such high performance often depends on dataset-specific conditions and may not transfer effectively across languages or domains.

Overall, modern ISLR systems report recognition rates ranging from 87% to 100%. These results confirm the effectiveness of deep learning and multimodal technologies for both isolated and regional SL.

In continuous recognition tasks where it is necessary to interpret gestures as a sequence of glosses, the WER becomes a critical indicator of the average error per word, reflecting the discrepancy between the predicted and reference sequences. In [123], a new method for detecting semantic boundaries based on reinforcement learning was proposed for poorly controlled CSLR, which allows for explicit alignment of video frames with gesture words. The method achieves a WER of 25.5% on the CSL Split II and RWTH-PHOENIX-Weather 2014 datasets. In [124], the Self-Mutual Knowledge Distillation (SMKD) method for CSLR was proposed, which simultaneously optimizes visual and contextual modules, avoiding the disadvantages of error backpropagation and addressing the CTC “spike effect” problem. In [125], the Visual Alignment Constraint (VAC) was introduced, improving the feature extractor, reducing overfitting, and making networks competitive in end-to-end setups. Despite these improvements, WER remains sensitive to alignment errors and the quality of gloss annotations, which limits robustness in real-world scenarios.

The study in [126] suggests an automatic annotation algorithm based on a Bi-Gated Recurrent Unit (BGRU) and Connectionist Temporal Classification (CTC), which reduces WER by 0.3% compared to modern SLR methods. Building on Transformer-based CSLR models, two additional constraints—spatial attention guided by keypoints and sentence embedding consistency—enabled strong performance on PHOENIX-2014, PHOENIX-2014-T, and SLR datasets [127]. The research in [128] introduces TransASL, an end-to-end real-time recognition and translation system using a wearable device to capture manual and non-manual markers, reaching a WER of 8.3% at the word level and 7.1% at the sentence level for 80 ASL words and 40 sentences.

A Self-Emphasizing Network (SEN) was presented in [129], highlighting informative spatial regions and discriminative frames, and achieving top accuracy on PHOENIX14, PHOENIX14-T, CSL-Daily, and CSL with minimal overhead. Two additional CSLR constraints were introduced in [130], a keyword-driven spatial attention module and sentence embedding consistency, which improved results on PHOENIX-2014, PHOENIX-2014-T, and CSL. The study in [131] proposes F2DCNet, a fully two-dimensional convolutional network for CSLR, relying only on 2D CNNs to extract spatiotemporal features and showing competitive performance on large datasets.

Article [132] suggests a multisensory CSLR framework combining RGB video, inertial measurement unit (IMU), and Surface Electromyography (sEMG) signals with BiLSTM and CTC, achieving a WER of 10.3% when modalities are fused. In [133], TCN is proposed as a hybrid network modeling spatiotemporal information from trajectories and correlated regions, yielding strong results on PHOENIX14, PHOENIX14-T, CSL, and CSL-Daily. By combining a gesture dictionary, an isolated recognition model, and a sliding window method, the study advanced the first steps toward online CSLR [134]. Finally, ref. [135] introduces a cross-modal complementation approach that generates pseudo video–text pairs by cross-modal editing, helping to resolve discrepancies between CTC and WER losses. However, multimodal approaches introduce additional challenges related to data synchronization, sensor availability, and system complexity.

Modern CSLR research demonstrates a steady downward trend in WER through the use of Transformers, bi-recurrent networks, attention mechanisms, and multimodal integration. These advances improve the alignment of video sequences with language units, mitigate CTC limitations, and enhance semantic interpretation of gestures, allowing recent systems to achieve WER below 10%, which moves CSLR closer to practical interactive communication.

The mean Average Precision (mAP) metric is actively used in tasks where it is necessary not only to classify but also to localize gestures in the video time interval. Lightweight and real-time architectures are increasingly used in SLR tasks, especially for mobile and embedded applications. An Android application was developed for two-way communication in Bahasa Isyarat Malaysia (BIM). It recognizes BIM letters using MobileNet, achieving 99.75% accuracy, and BIM gestures using SSD-MobileNet-V2 Feature Pyramid Network Lite (FPNLite), achieving 61.60% accuracy [136]. An automatic Bangla SL (BSL) recognition system was developed for Jetson Nano, where YOLOv7-Tiny achieved the best real-time performance compared to Detectron2 and EfficientDet-D0 [137]. Another study introduced a model for both classification and correctness evaluation of gestures, combining flow-controlled features, 3D Conv, 2D deformable convolution, and attention mechanisms [138].

Real-time recognition of ASL letters (A–Z) was achieved using a pre-trained YOLOv8 model [139]. For Bengali gestures, an end-to-end system with YOLOv4-Tiny (99.7% mAP for 49 characters) and LSTM text generation (99.12% accuracy) was proposed [140]. A YOLOv5-based framework for Telugu SL recognition achieved strong performance with F1 of 90.5% and mAP of 98.1%, balancing accuracy and computational efficiency [141]. Finally, a comparative study on hand detection highlighted YOLOv8n and YOLOv8s, achieving 86.7% mAP on Oxford Hand and 98.9% on EgoHand after 100 epochs of training [142]. These studies highlight that the integration of lightweight architectures and optimization mechanisms can achieve a balance between accuracy (up to 99.75% mAP) and speed, which is particularly important in mobile applications and real-time systems focused on comprehensive communication and gesture learning.

The Spatial–Temporal Multi-Cue (STMC) Transformer [143] significantly improved glossary-to-text and video-to-text translation on PHOENIX-Weather-2014T (BLEU +5 and +7, respectively) and ASLG-PC12 (BLEU +16), but also demonstrated the limitations of glossary-based representations and suggested the need for end-to-end learning. The task-aware instruction network for sign language translation (TIN-SLT) network [144] introduced task-based learning and feature integration to the Transformer, surpassing previous best results with 1.65 BLEU-4 on PHOENIX-2014T and 1.42 BLEU-4 on ASLG-PC12. The large-scale OpenASL dataset [145] was introduced, along with techniques for handling real-world conditions, such as gesture search and multi-task learning. Rule-based pseudo-vocabulary generation was proposed in [146] as a simple method to improve SLT, achieving advanced results on PHOENIX-Weather 2014T and ASLG-PC12. An initial SLT benchmark using a dilated 3D-based Transformer on the broad-coverage How2Sign dataset was presented in [147].

Further research focused on multimodal and transfer learning approaches. The gloss attention SLT network (GASLT) model [148] combined RGB video and 3D pose data and improved BLEU by 18.39%. Transfer learning was explored in [149], where a Twin Residual Net outperformed multiple ResNet variants, achieving an average BLEU of 0.46. A comparative study [150] showed that ResNet-101 was the most effective backbone for video-based SLT in an LSTM framework. The study in [151] described that the quality of translation has a significant impact on the selection of algorithm optimization, activation function, and label smoothing, which is a key consideration in careful learning design. Performance on the PHOENIX-Weather 2014T dataset was improved by a graph-based dynamic multimodal SLT model proposed in [152] that integrates additional semantic knowledge at the word level. However, BLEU-based evaluation may not fully capture the semantic correctness and naturalness of sign language translation, especially in complex linguistic contexts.

Therefore, the selection of evaluation metrics must be carefully aligned with task-specific requirements, the characteristics of the captured body movements, and the desired balance between recognition accuracy and computational efficiency.

6. Comparative Analysis of Methods and Models

This section analyzes modern methods of SLR based on video data. Special attention is paid to the study of key models and approaches, their architectural features, accuracy, performance, and computational efficiency, structured within the framework of the proposed classification.

According to the proposed classification, the main random-access schemes are summarized for each category in Tables S1 and S2. To make a clear comparison of the various methods, we highlight the main goal, method, performance improvements, and limitations of each approach. Table 1 shows modern models and their performance in SLR tasks. Most of the methods demonstrate high accuracy and efficiency and solve specific tasks such as sentence translation or multimodal feature fusion. CBAM-3DResNet-based approaches achieve over 90% accuracy [27]; LSTM [34,35] models have shown great performance gains on specific data. The limitations of these methods include the lack of datasets, the complexity of processing hidden features, and the need for complex, large datasets.

Table S3 provides modern approaches to sign language tasks, including performance analysis, creation of new datasets, and architecture development. Most of the methods demonstrate high accuracy, for example, greater than 96% recognition on AUTSL [91], and successfully solve specialized tasks such as modeling phonological properties or developing multimodal systems. However, most solutions face limitations, such as the difficulty of distinguishing visually similar gestures, a lack of data, or the impact of uncontrolled conditions. These methods require further consideration of cultural aspects, improvement of interpretability of models, and ensuring equity in access to data.

Table S4 compares methods for providing complexity and individualization in SLR tasks. These methods, such as PoseNet-LSTM [33], have shown high accuracy. Many approaches take into account specific user needs, such as developing energy-efficient SLR applications or ensuring equal access to knowledge using ResNet-50 [30]. However, some methods face challenges such as the complexity of processing noisy data or the need to improve performance with limited computational resources. Ignoring these aspects may limit the development of complex and robust systems that can effectively operate in real-world situations.

Table S5 provides an analysis of the performance and robustness of various SLR methods, evaluating them in terms of accuracy, processing speed, real-time applicability, coverage, and noise immunity. Despite the high robustness of multimodal methods to noise, most are computationally intensive, which limits their practical application in real time. The Multi-Stream Graph-Based Deep Neural Network (SL-GDN) [46] model achieves high performance with relatively low computational load even on complex backgrounds, but its generalizability across different scenarios remains limited.

HA-GCN and multi-branch GCN architectures show more efficient representation of skeletal data in dynamic situations, improving spatiotemporal modeling, but may face scaling issues on large datasets. In practical applications, the Time–Frequency Network (TFNet) can handle complex scenarios such as the Chinese Continuous Sign Language Dataset (CE-CSL), but its performance is highly dependent on the quality and diversity of the training data. The AUTSL dataset demonstrates the benefits of multimodal approaches and contextual diversity, but also shows that existing methods still need improvement to fully generalize to new users and survey situations.

Table S5 presents the results of the evaluation of various methods aimed at improving the performance of SLR and SLT systems. The methods were compared on criteria such as accuracy, processing speed, real-time capability, coverage, and noise immunity. The analysis included the development of practical and inclusive solutions for 3D CNNs, cross-attention Transformers, and multimodal approaches that combine information from gestures, posture, and facial expressions. The methods under study may represent technological progress in terms of efficiency, noise immunity, and adaptability. However, challenges remain in reducing computational requirements and improving model interpretation, which represent key areas for future research.

7. Discussions

Recent advances in the field of video-based SLR have seen significant progress [153,154,155,156,157,158,159,160,161,162,163], largely due to developments in deep learning, multimodal data analysis, and the creation of specialized datasets. The systematic analysis conducted in this review across six architectural categories reveals several overarching patterns that transcend individual method comparisons and deserve explicit discussion.

A clear evolutionary trajectory emerges from the reviewed literature. Early CNN and 3D-CNN approaches established strong spatial feature extraction baselines but were fundamentally limited by their inability to model long-range temporal dependencies. Recurrent architectures such as LSTM and GRU addressed sequential modeling but introduced information bottlenecks through fixed-size hidden states, which caused performance degradation on longer and more complex signing sequences. Transformer-based models subsequently resolved the temporal dependency problem through attention mechanisms but introduced new challenges related to data requirements and computational cost. GCN-based approaches offered geometric robustness by explicitly modeling skeletal structure but remained vulnerable to pose estimation failures in uncontrolled environments. The hybrid architectures currently represent the state-of-the-art by combining the complementary strengths of these paradigms, yet they simultaneously amplify the data requirements of their constituent components. This pattern suggests that architectural progress in SLR has been largely driven by addressing the limitations of the preceding dominant paradigm rather than by a principled unified framework, and that a more productive direction would be the development of architectures that simultaneously address spatial precision, temporal modeling, geometric structure, and computational efficiency within a single coherent design.

It is important to note that this evolutionary progression also reflects a broader shift in SLR research from isolated benchmark optimization towards attempts to achieve robustness, scalability, and practical deployment feasibility under real-world communication conditions. As architectures become increasingly sophisticated, the central challenge is no longer limited to maximizing recognition accuracy but rather to achieving an effective balance between temporal modeling capacity, geometric precision, computational efficiency, and cross-domain generalization.

As shown in

Table 3. Cross-architectural comparison of deep learning approaches for video-based SLR.

Architecture	Accuracy Range	Computational Complexity	Robustness to Real-World Variation	Data Requirements	Suitable for Low-Resource SL	Real-Time Applicability
CNN/3D-CNN	83–99%	Medium	Low—sensitive to background and lighting	Medium	Limited	Yes (lightweight variants)
RNN/LSTM/GRU	88–99%	Low–Medium	Medium—degrades on long sequences	Low–Medium	Moderate	Yes
Transformer/Attention	85–98%	High	Medium–High	High	Limited without pre-training	Partial
GCN-based	88–100%	Medium–High	Medium—depends on pose estimation quality	Medium	Moderate	Partial
Hybrid End-to-End	90–99.97%	Very High	High	Very High	Low	Limited
Transfer/Self-supervised	80–95%	Medium	Medium–High	Low (labeled)	High	Partial

, a clear performance–complexity trade-off emerges across the six architectural categories reviewed. CNN- and RNN-based approaches offer the best balance of computational efficiency and real-time applicability, but their robustness to real-world variation remains limited, particularly in cross-dataset and signer-independent scenarios. Transformer-based models achieve higher accuracy on large benchmarks but require substantial annotated data and computational resources, limiting their direct applicability to low-resource sign languages. GCN-based approaches provide structural robustness through skeletal modeling but remain vulnerable to pose estimation failures in uncontrolled environments. Hybrid architectures represent the current state-of-the-art in terms of accuracy but amplify the data and computational requirements of their constituent components, creating a significant barrier to deployment in resource-constrained settings. Transfer learning and self-supervised approaches offer the most promising path for low-resource SLR, but their effectiveness depends critically on domain proximity between source and target languages. These trade-offs suggest that no single architectural paradigm currently satisfies all requirements for robust, generalizable, and computationally efficient real-world SLR (see Figure 9), and that future progress will depend on principled integration strategies rather than continued optimization of individual components in isolation.

It is suggested by these observations that future SLR systems should be evaluated not only according to benchmark accuracy metrics but also according to deployment-oriented criteria. These include, but are not limited to, inference latency, memory efficiency, robustness to environmental variability, and adaptability across signer populations and recording conditions. The evaluation principles outlined here are of particular importance for assistive communication systems intended for continuous real-world interaction in environments external to laboratory settings.

A persistent and systematic gap exists between reported benchmark performance and real-world applicability. The majority of reviewed studies report accuracy figures above 90% on standard datasets, yet practical deployment of SLR systems in real assistive communication contexts remains limited. This gap arises from three structural factors that are consistently present across the reviewed literature: most benchmarks are collected under controlled laboratory conditions that do not reflect natural signing environments; evaluation is typically performed in signer-dependent settings that mask the model’s inability to generalize to unseen users; and dataset-specific fine-tuning inflates reported metrics in ways that do not transfer across languages or domains. Future evaluations should prioritize signer-independent protocols and cross-dataset generalization benchmarks as standard reporting requirements across the field.

The three core challenges identified throughout this review are data scarcity, generalization, and computational complexity that deserve deeper analysis than a simple enumeration. The data scarcity in sign language research is not merely a quantitative problem of insufficient samples, but a structural problem rooted in the requirements of the data collection process itself. Unlike speech recognition, where large-scale data collection is relatively straightforward, sign language data collection requires native signers, specialized recording infrastructure, linguistically informed annotation protocols, and ethical frameworks for working with deaf communities. These requirements create a compounding barrier that disproportionately affects low-resource sign languages, including Kazakh, Ethiopian, Azerbaijani, and hundreds of others, where the community of native signers is small and institutional support is limited. The transfer learning and self-supervised pre-training offer partial mitigation, but their effectiveness depends on domain proximity between source and target languages, which cannot be assumed across structurally distinct sign language families. The generalization problem is similarly more complex than it appears: sign language production is inherently variable across signers, regional dialects, and communicative contexts, and this variability is not noise to be filtered out but linguistically meaningful variation that must be modeled explicitly. The computational complexity challenge creates a direct tension with the primary use case of SLR systems, namely real-time assistive communication on resource-constrained devices, and model compression, knowledge distillation, and efficient architecture design remain underexplored directions in the SLR literature.

Based on the systematic analysis conducted in this review, we identify the following priority research directions organized by time horizon. In the short term (1–2 years), the field would benefit most from standardization of evaluation protocols, specifically the adoption of signer-independent splits and cross-dataset evaluation as minimum reporting requirements, and from the development of linguistically validated benchmark datasets for underrepresented sign languages, with particular priority given to sign languages of Central Asia, Africa, and Southeast Asia. Additionally, the systematic evaluation of existing state-of-the-art architectures under cross-domain conditions, specifically across datasets collected in different recording environments, with different signer demographics, and for different sign language families, should be established as a standard validation requirement to provide more reliable estimates of real-world generalization capability. In the medium term (2–4 years), the most impactful directions include the development of large-scale multilingual sign language pre-training corpora enabling cross-lingual transfer learning, the integration of geometric skeletal representations with Transformer-based temporal modeling to address fine-grained motion modeling gaps, and the development of adaptive multimodal fusion strategies that dynamically weight manual and non-manual features rather than applying static equal-weight concatenation. In the long term (4+ years), the field needs end-to-end SLR systems capable of handling the full linguistic complexity of natural sign language, including non-manual markers, spatial grammar, and prosodic features, as well as participatory design frameworks that involve deaf communities as active partners in dataset creation, model evaluation, and system deployment.

A cross-cutting theme emerging from this review is the structural underrepresentation of low-resource sign languages in the deep learning literature. Of the 103 reviewed studies, the majority focus on ASL, CSL, or ArSLs, with only a small number addressing languages such as Kazakh, Ethiopian, or Azerbaijani Sign Language. This distributional imbalance reflects existing inequalities in access to assistive communication technology. The development of resources such as KSLVD, discussed in Section 4, represents one concrete step toward addressing this gap, but the broader challenge requires coordinated effort across linguistics, computer science, and deaf education communities. Future work should treat the geographic and linguistic distribution of studied sign languages as an explicit metric of field inclusivity, and funding bodies should prioritize research on underrepresented sign languages as a matter of equity in assistive technology development.

Deep learning methods have been shown to significantly improve SLR and translation performance, and the integration of multimodal architectures, Transformer-based models, and self-supervised learning strategies represents the most promising current direction. However, it is essential that proposed SLR methods increasingly take into account not only technical performance metrics but also real-world usability, linguistic validity, and equitable access across sign language communities. These considerations should guide the next generation of research in this field.

8. Conclusions

This review analyzes systematic deep learning approaches and their limitations. Furthermore, the lack of high-quality standardized datasets, especially for regional sign language variants, significantly complicates the development and validation of systems that handle large vocabularies and huge amounts of data. The KSLVD was developed as a standardized resource for sentence-level KSL recognition.

The obtained conclusions emphasize the need for further research aimed at improving algorithms, creating larger and more diverse datasets, as well as developing universal and user-difference-resistant models for the practical implementation of SLR technologies. We hope that the results of this review will be useful to future researchers, providing systematic information, references and inspiration for the development of new approaches and solutions in the field of SLR.