Towards Trustworthy Sign Language Translation System: A Privacy-Preserving Edge–Cloud–Blockchain Approach

Abstract

The growing Deaf and Hard-of-Hearing community faces communication challenges due to a global shortage of certified sign language interpreters. Therefore, developing efficient and secure sign language machine translation (SLMT) systems is essential. Current work addresses the accuracy of the sign language translation task. However, there is a need for an SLMT system that encompasses privacy, efficiency, translation accuracy, and Machine Learning development operations. This paper addresses this void by proposing a novel consent-aware privacy-preserving end-to-end edge, cloud, and blockchain integrated computing system. We evaluate the system by comparing the mostly used Encoder–Decoder Transformer and a lightweight Adaptive Transformer (ADAT), using two datasets: the most comprehensive sign language dataset RWTH-PHOENIX-Weather-2014T (PHOENIX14T), and MedASL, our newly developed medical-domain dataset. A comparative analysis of translation quality on PHOENIX14T shows that ADAT improves BLEU-4 by 0.02 absolute points and ROUGE-L by 0.11. On MedASL, ADAT gains 0.01 in BLEU-4 and 0.02 in ROUGE-L. For runtime efficiency on MedASL, ADAT reduces training time by 50% and lowers both edge–cloud and end-to-end system communication times by 2%.

1. Introduction

More than 430 million people are Deaf or Hard-of-Hearing (DHH) worldwide; this figure is expected to exceed 700 million by 2050 [1]. While DHH individuals communicate using sign language, there is a shortage of certified interpreters. For instance, the ratio of DHH individuals to interpreters in the United States is 4800:1 [2]. The situation is more severe in the least developed countries, where there are fewer interpreters [3]. This disparity underscores the need for automated and efficient sign language machine translation (SLMT) systems.

SLMT systems are Assistive Technologies (ATs) that help DHH individuals overcome communication barriers and foster inclusion and independence, particularly when a human interpreter is unavailable. These systems must be real-time, reliable, and secure as described in [4]. Developing such systems can save lives in natural and medical emergencies [5]. Consequently, building an accurate and real-time software-based SLMT system is crucial.

Several works investigate SLMT systems [6,7,8,9,10,11,12,13,14,15,16]. However, these systems consider only a few components of the building blocks of an end-to-end privacy-preserving, real-time, and efficient SLMT. In particular, most studies focus on optimizing translation quality without addressing deployment challenges such as runtime performance and user privacy. In this paper, we fill this gap by proposing an end-to-end SLMT system that integrates edge and cloud computing with blockchain-based consent management. We evaluate its performance in a real-world setting through a comparative analysis of the Encoder–Decoder Transformer [17] and a lightweight Adaptive Transformer (ADAT), based on [18] using the RWTH-PHOENIX-Weather-2014T (PHOENIX14T) [19] and MedASL datasets. We created MedASL to exemplify conversations between DHH people and a medical professional.

This proposed system considers deployment factors, including the various components required to build a real-time system, the computational costs of each component, and user privacy requirements. It also enables secure, traceable deployment through blockchain integration, which has not been previously explored in the SLMT literature. The proposed system integrates five components: (1) a sign language recognition (SLR) module to capture sign videos, (2) an artificial intelligence (AI)-enabled application that serves as a gateway between the user and the edge, (3) edge nodes to extract and preprocess keypoints from the sign videos for inference, (4) cloud servers to develop and deploy the SLMT AI model, and (5) an integrated blockchain–cloud computing architecture as in [20] to record and manage user consent for data collection, ensuring compliance with relevant regulations [21,22]. We address user privacy through consent mechanisms, which we categorize into (1) system-level consents in which the DHH individual allows systems administrators to access user data for system failure diagnosis and recovery, and (2) application-level consents to obtain users’ permission on data sharing and privacy management.

The main contributions of this paper are as follows:

• We propose an end-to-end edge, cloud, and blockchain computing integrated system for SLMT.
• We evaluate our proposed system in comparison with the most used Encoder–Decoder Transformer using the largest publicly available German sign language dataset, PHOENIX14T, and our new MedASL dataset.
• We conduct experiments and numerical analysis of our system’s performance in terms of Bilingual Evaluation Understudy (BLEU), Recall-Oriented Understudy for Gisting Evaluation (ROUGE), training time, translation time, and overall end-to-end system time.
• We deploy the Encoder–Decoder Transformer and ADAT models in a unified setup, demonstrating the feasibility of our proposed system for real-world applications.

The rest of the paper is organized as follows. Section 2 reviews the related work. Section 3 explains the proposed end-to-end edge–cloud–blockchain architecture for SLMT. Section 4 describes the materials and methods. Numerical experiments and comparative results are provided in Section 5. Section 6 discusses the responsible deployment of the proposed system. Finally, Section 7 concludes the paper.

2. Related Work

Several works applied machine and deep learning models for SLMT [6,7,8,9,10,11,12,13,14,15,16].

Table 1. Comparison between sign language machine translation systems in the literature.

Work	Sign Language(s)	Translation Formulation		Model			Deployment Environment			Consent		Runtime Measurement
		Type	Unit	Input Modality	Preprocessing	Algorithm	Edge	Cloud	Blockchain	System	Application	Training Time	Preprocessing Time	Inference Time	Communication Time
[6]	ASL	S2G, S2T	Words + Sentences	Keypoints	Smoothing and normalization	Hierarchical Bidirectional RNN	✓	✗	✗	✗	✓	✗	✗	✓	✗
[7]	ASL	S2G2T	Words + Sentences	RGB	Segmentation and contour extraction	Attention-based LSTM	✓	✗	✗	✗	✗	✗	✗	✗	✗
[8]	SSL	S2G	Words	RGB	Segmentation, contour extraction, ROI formation, and normalization	CNN + Tree data structure	✓	✗	✗	✗	✗	✗	✗	✓	✗
[9]	DGS, CSL	S2G, S2G2T	Sentences	RGB	Greyscale, resizing, and ROI formation	GCN + Transformer decoder	✓	✗	✗	✗	✗	✗	✗	✓	✗
[10]	DGS, CSL	S2G, S2G2T	Sentences	RGB	Cropping	Transformer	✗	✗	✗	✗	✗	✗	✗	✗	✗
[11]	ASL, BSL, JSL, KSL	S2G	Words	RGB	Resizing and segmentation	Attention-based CNN	✗	✗	✗	✗	✗	✗	✗	✗	✗
[12]	ASL, ArSL, KSL	S2G	Characters	RGB	Resizing	MLP	✗	✗	✗	✗	✗	✗	✗	✗	✗
[13]	ASL, TSL	S2G	Characters + Words	RGB	Frame extraction, grayscale conversion, normalization, background subtraction, and segmentation	Attention-based CNN + LSTM	✗	✗	✗	✗	✗	✗	✗	✓	✗
[14]	LSA	S2G	Words	RGB	Segmentation, rescaling	LSTM	✗	✗	✗	✗	✗	✗	✗	✗	✗
[15]	DGS, CSL	S2G	Sentences	Keypoints	Partwise partitioning, and graph construction	Graph Fourier Learning	✓	✗	✗	✗	✗	✗	✗	✓	✗
[16]	ASL	S2G	Words	Keypoints	ROI formation, hand cropping, rescaling, and padding	YOLO	✓	✓	✗	✗	✗	✓	✗	✓	✗
This Work	DGS, ASL	S2G2T	Sentences	Keypoints	Normalization, rescaling, and padding	Transformer-Based	✓	✓	✓	✓	✓	✓	✓	✓	✓

ASL: American Sign Language; ArSL: Arabic Sign Language; BSL: Bangla Sign Language; CSL: Chinese Sign Language; DGS: German Sign Language; JSL: Japanese Sign Language; KSL: Korean Sign Language; LSA: Argentine Sign Language; SSL: Sinhala Sign Language; TSL: Turkish Sign Language. CNN: Convolution Neural Network; GCN: Graph Convolution Network; LSTM: Long Short-Term Memory; RNN: Recurrent Neural Network; ROI: Region of Interest; ✓: included; ✗: not included.

summarizes these works by comparing translation formulation, input modality, preprocessing techniques, and learning algorithms. It also compares the deployment environments, including edge, cloud, and blockchain, as well as consent mechanisms and runtime measurements.

The table shows that current works focus on sign-to-gloss (S2G) translation [6,8,9,10,11,12,13,14,15,16], where a gloss is a written representation of a sign. In an end-to-end SLMT system, sign videos are translated either into gloss then text (S2G2T) or directly into text (S2T). Therefore, S2G, without text output, offers limited benefits for real-world applications [4]. In terms of input modalities, previous studies relied mainly on RGB video [7,8,9,10,11,12,13,14]. While these inputs capture rich visual information such as facial expressions, they are sensitive to environmental factors, including lighting, background, and signer features and appearance [23]. These variations can degrade the model’s performance and require extensive preprocessing and computationally intensive backbones for feature extraction compared to keypoints [24]. In addition, RGB inputs preserve identifiable features and contextual details that raise privacy concerns, such as re-identification and unintended data exposure [25]. In contrast, the use of keypoints in the proposed system retains abstract visual content, including skeletal landmarks and poses, while omitting identifiable details, hence mitigating privacy risks. This abstraction preserves essential spatiotemporal features for sign language understanding and consequently reduces computational cost [26].

The trade-offs between computational costs and privacy implications highly impact the deployment choices. This can be addressed by adopting architectures that integrate edge processing with cloud support and blockchain-based governance mechanisms. However, few works partially explore these architectures, focusing on the edge as an offloading component, which limits the system scalability. While many works employ edge devices [6,7,8,9,15,16], VisioSLR [16] extends this by incorporating cloud servers, and none integrate a privacy-preserving blockchain system.

The deployment environment influences runtime behavior, making it necessary to measure training, preprocessing, inference, and communication times. Few works report inference time [6,8,9,13,15,16] and only VisioSLR [16] measures training time. However, no works compute preprocessing and edge–cloud communication times. Moreover, despite the importance of user consent in privacy-sensitive applications, existing SLMT works do not implement system-level consent, whereas DeepASL [6] addresses application-level consent.

Due to the limited adoption of consent mechanisms in SLMT systems, we provide a cross-domain comparison of both consent mechanism applications in

Table 2. Comparison of consent applications across domains.

Work	Application Domain	Use Case	Consent		Data Collected
			Type	Purpose
[27]	Web browsing	Web browser crash reporting	System	Improve software reliability	Crash/bug-related traces, metadata, and user information
[28]	Operating system	Operating system crash reporting	System	Bug diagnosis and security analysis	Memory snapshots, CPU registers, user credentials, browsing history, cryptographic keys
[29]	Digital behavior	Willingness to share data for research	Application	Participate in research	GPS location, photos/videos (house and self), wearable sensor data
[30]	Critical incident management	Anonymous emergency reporting to authorities	Application	Share data with authorities	Geo-coordinates, status alerts, media (optional)
[31]	Blockchain-based reporting	Anonymous crime reporting to authorities	Application	Report criminal incidents	User data, embedded report data
[32]	Blockchain-based reporting	Anonymous crime reporting to authorities	Application	Report criminal incidents	Reported information and cryptographic keys
[33]	Digital health	Sharing personal health data with healthcare and research entities	Application	Control data access and sharing	Personal health, genomic, medications, mental health data
This Work	Assistive technology	Sign language translation	System and Application	Improve software reliability, bug diagnosis, enable private communication	Videos, system performance logs (optional)

. Prior works show that system-level consents preserve privacy and anonymity when reporting platform crashes, such as in browsers [27] and operating systems [28]. On the other hand, application-level consents anonymize personal data in contexts including research participation [29], emergency and crime reporting [30,31,32], and digital health data sharing [33].

In summary, prior works on SLMT focused on S2G translation using RGB video inputs, despite their computational overhead and privacy vulnerabilities. Keypoint representations offer a more efficient and privacy-preserving alternative. Nevertheless, comprehensive evaluations of deployment factors and runtime measurements in SLMT remain scarce. Similarly, while cross-domain works highlight the benefits of consent mechanisms for privacy-sensitive applications, such mechanisms remain unexplored in the current SLMT literature. In this work, we propose an end-to-end system for SLMT, integrating a camera, an AI-enabled application, edge nodes, cloud servers, and blockchain. To our knowledge, no existing SLMT work provides such a unified framework that evaluates translation quality and computational efficiency while addressing user data governance.

3. Proposed End-to-End System for Sign Language Translation

The overview of our proposed end-to-end edge–cloud–blockchain system for SLMT is presented in Figure 1. We explain the system components in the following subsections.

3.1. Sign Language Recognition Module

Sign language is based on visual cues, making video capture and feature extraction essential components for translation. The SLR module captures sign videos within a conversation through a camera and transmits them to the edge for keypoint extraction and translation. This enables direct communication between DHH and hearing individuals without relying on a human interpreter. The technical feasibility and operation reliability of the SLR module depend on the following specifications:

• Camera configuration: Higher resolution and frame rate per second improve fine-grained feature extraction [34], but increase computation and video uplink load [24]. Keypoint extraction mitigates this issue by reducing uplink demand and latency.
• Device computation: CPU, GPU, and RAM capacities determine the feasibility of capturing frames, extracting keypoints, and performing inference [35].
• Connectivity: Wi-Fi and 5G networks enable low-latency and high-throughput communication [36,37].
• Operating System (OS): The OS manages camera access, hardware accelerators, security, and user experience. User-driven access controls reduce unauthorized access. Hardware accelerators determine the feasibility of keypoint extraction and inference. Security policies enforce user privacy and implement tamper-evident safeguards, including integrity checks and tamper detection modules [38]. Permissions for camera, network, and storage are tied to user consent, ensuring legal compliance [39].

Our proposed system provides an accurate translation with low latency. CNN-based keypoint extraction achieves state-of-the-art accuracy [40]. Therefore, we recommend 1080p at 30 or 60 fps for video streaming, aligning with the public sign language datasets [4]. Privacy is ensured through encryption, secure access controls, and revocable consent, forming the first layer of protection in the overall SLMT.

3.2. AI-Enabled Sign Language Translation Application

The AI-enabled application serves as a user gateway to the system, acquiring sign video input and transmitting it to the edge. The interaction flow proceeds as follows: (1) enable Wi-Fi or 5G connectivity, (2) establish an authenticated session through credential-based login, (3) obtain user consent, (4) capture sign videos and extract keypoints for preprocessing and inference, and (6) receive and display the translation.

Translation is best performed using Transformer-based models [4,17], which surpass earlier CNN- and RNN-based approaches. This is due to the Transformer’s ability to capture long-range spatiotemporal dependencies, enabling parallel sequence processing and improving representation learning [17].

To ensure transparency, user consents are explained in spoken and sign languages, clearly defining the purpose, types of collected data, and retention periods. Once the user grants consent, the application generates a consent receipt, which is a blockchain-recorded metadata record confirming the user’s authorization for specific data processing under a defined policy version. It includes a unique consent identifier, consent scope, corresponding policy version, its validity period, timestamp, and digital signature verifying authenticity. Each obtained consent is bound to a policy and an audit log. A policy log is a metadata record that contains the consent scope, data retention period, and data storage and transmission rules [41]. An audit log is an immutable blockchain ledger entry that traces all consent, translation, and model-related transactions in the SLMT system [42]. Both logs contain only content-free metadata such as version identifiers, hashes, timestamps, and digital signatures, ensuring transparency and accountability without revealing any user identity or sign content.

Table 3. Consent types in the proposed AI-enabled sign language translation application.

Consent Type	Purpose	Required Access	Data Transmitted	Data Storage	Revocation Effect	Status
Application	Translation	Camera	Keypoints and metadata	None	Camera access denied, translation stops	Mandatory
Application	Improvement and retraining	Keypoints and raw sign videos	Raw videos, keypoints, and metadata	On cloud	Immediate, no future data sharing, per policy	Optional
System	Reliability improvement	Raw sign videos	Incident logs, raw videos, and metadata	On cloud	Immediate, no future data sharing, per policy	Optional

presents the consent types in our end-to-end system.

3.3. Edge Computing

Once the video streams are transmitted, the edge device performs keypoint extraction, preprocessing, and translation. Keypoint extraction is a crucial step as it reduces bandwidth consumption, communication overhead, and privacy risks [26]. The edge preprocesses keypoints by normalization, rescaling, and padding, before translating sign language into spoken language using a Transformer-based model. The resulting translation is returned to the application for display to the user.

When the user grants an application-level consent for model improvement, the edge forwards the raw videos, corresponding keypoints, and generated translations to the cloud. Under system-level consent, the edge transmits incident logs and associated sign videos to the cloud. These logs include system performance, inference latency, and incident traces. Independent of the consent, all data transmitted by the edge is encrypted and follows policy-defined rules.

The edge also records tamper-evident, content-free metadata on the blockchain, describing every translation and event. Each blockchain submission includes the following metadata: a unique translation session identifier, reference to the consent receipt, reference to the system certificate that defines the deployed model and its version, hash of the processed keypoints, hash of the translation, policy version, processed and retained data receipts, timestamp, and digital signature of the edge device ensuring authenticity. This signed record, known as a data receipt, is periodically transmitted to the blockchain through a secure gateway.

3.4. Cloud Computing

The edge communicates with the cloud only when the user grants application-level consent for model improvement and/or system-level consent. Under application-level consent, sign experts annotate uploaded sign videos for future model retraining whenever translation precision falls below predefined thresholds. Updated models are then rolled out to the edge. Under system-level consent, system administrators analyze incident logs and raw videos to diagnose and resolve reliability issues before deploying new updates. In both consent scenarios, the cloud generates a system certificate, a policy log, and a cloud digital signature. Moreover, the cloud appends the following metadata fields: a reference to the system certificate associated with the deployed model, the policy version, the cloud’s signature to ensure integrity, and a timestamp. These records are encrypted and transmitted to the blockchain to ensure auditability without exposing user content. Figure 2 illustrates the deep learning operations pipeline between the edge and cloud in our proposed system.

3.5. Blockchain

Ensuring the security and privacy of user data is a key requirement in SLMT system deployment. In particular, storing user data on infrastructures, such as the cloud, offers scalability but limits traceability of data access and policy enforcement, which is critical in healthcare, as demonstrated in [43]. To address this issue, our proposed system integrates a blockchain [44] that serves as a tamper-evident audit layer for user transactions, consents, data access and usage, system certificates, and policy logs. Within this layer, all records are linked through a chain of hashes and replicated across the edge and cloud, ensuring that any attempt to alter or remove data is immediately detectable. This distributed verification mechanism enhances confidentiality, integrity, and accountability while maintaining cloud scalability.

As illustrated in Figure 3, the process begins with the cloud generating the system certificate and the policy log, each containing a unique identifier, a version number, and a cryptographic hash. These are transmitted to the blockchain and the edge to establish a baseline of trust. When the user grants consent, the application creates a consent receipt and submits it to the blockchain. The blockchain verifies the system certificate, policy log, and consent receipt to ensure consistency and authenticity, then forwards verified references to the edge. The edge cross-verifies these records against the cloud to confirm alignment before starting the translation session.

During translation, the application streams the sign video to the edge for keypoint extraction and translation. The edge then hashes the processed keypoints and generated text, combines them with the verified references, creates a data receipt, and transmits it to the cloud. The cloud re-verifies the certificate and policy version, appends its own metadata, and submits the final transaction to the blockchain. Upon receipt, the blockchain validates all hashes, signatures, and certificate references, appends a new block, and commits it to the distributed ledger. The ledger stores only content-free metadata, forming a lightweight audit trail that ensures verifiability without exposing personal identifiers [20]. Verified entries are shared with the edge and cloud to maintain synchronized, policy-compliant operation [45].

Table 4. Participants, events, transactions, assets, and system effects in the blockchain layer for sign language translation.

Participant	Event	Transaction	Asset	Scope	System Effect
User (Deaf)	Authorize consent	Grant consent	Consent receipt	1. Translation 2. Model improvement 3. System diagnostics	By scope: 1. Enable translation 2. Allow data storage and access 3. Allow diagnostic uploads and access on the client
	Change consent scope(s)	Update consent	Consent receipt	Any of the above	Adjust allowed data processing, storage, and access on the client
	Withdraw consent	Revoke consent	Consent receipt	Any of the above	Stop processing under the revoked scope on the client
Sign Expert	Request viewing of consented sign videos	Request data access	Data-use receipt: access outcome (granted/denied)	Curation	Gate and log access on cloud
	Begin sign video annotation	Annotate data (if granted)	Access outcome	Curation	Annotation proceeds on cloud only if access is granted
System Administrator	System/model passes security/privacy/ evaluation checks	Deploy system/model	System certificate	Release system/model	Roll out new system/model on edge and cloud
	Policy/system updated, retrained model improved	Update system/model version	System certificate	Release new system/model version	Roll out updated system/model on edge and cloud
	Policy annulled, vulnerability detected, model performance degraded	Revoke system/model version	System certificate	Withdraw system/model version	Roll back to previous system/model version on edge and cloud
	Approved change in policy	Commit policy	Policy log	Policy	Enforce updated policy on edge and cloud
Auditor	Periodic review	Read audit trails	Audit logs	Governance and legal compliance	Verify compliance and integrity on the ledger

Notes: Consent entries include scope and validity window; revocation takes effect immediately, without affecting any previous data collection. All transactions are logged on the ledger for auditing.

summarizes the proposed blockchain layer, listing the participants, events, and the associated actions that affect the ledger transactions, log assets, scope, and system effects.

In summary, the proposed system integrates the SLR module, AI-enabled application, edge, cloud, and blockchain. This design enables scalability, privacy, and reliability while providing accurate and trustworthy sign language translation.

4. Materials and Methods

This section presents the experimental setup and numerical comparative analysis of our proposed system. To evaluate system performance, we employ two datasets covering distinct sign languages and domains, benchmarking the Encoder–Decoder Transformer, the widely adopted algorithm in SLMT, and its variant, ADAT. We assess the performance using BLEU [46] and ROUGE [47] for translation precision, as well as training time, translation time, edge–cloud communication time, and overall system time.

4.1. Datasets

We conduct experiments on PHOENIX14T [19], the most used benchmark dataset for SLMT, and MedASL. MedASL contains 2000 medical-related statements reflecting conversations between DHH and medical professionals. The recordings were collected in varied environments, including changes in lighting and background, to enhance robustness. Using a large, multi-signer dataset (PHOENIX14T) and a smaller, single-signer dataset (MedASL) allows comparative analysis of efficiency and generalization across different scales.

Table 5. Dataset characteristics.

Dataset	Sign Language	Domain	# Videos	Vocabulary Size		Sentence Length		# Signers	Resolution @fps
				Gloss	Text	Gloss	Text
PHOENIX14T	German	Weather	8257	1115	3004	32	54	9	210 × 260 @25 fps
MedASL	American	Medical	2000	1142	1682	10	16	1	1280 × 800 @30 fps

summarizes the characteristics of both datasets.

4.2. Data Preprocessing

We adopt a consistent preprocessing pipeline for training and inference. We use MediaPipe [48] to extract the following per-frame keypoints: 21 landmarks per hand, 468 face landmarks, 5 iris landmarks per eye, and 6 upper body pose landmarks. We then normalize and rescale these keypoints. Next, we concatenate frames belonging to the same sentence into a temporal sequence and zero-pad them for batch processing. During inference, we segment the input stream into sliding windows before passing them to the model.

Regarding the translation output, we construct a vocabulary that includes special tokens, such as start- and end-of-sequence (<SOS> and <EOS>) and <UNK> for unknown words. We then tokenize the text sequences, index each token, and pad them to a fixed length to enable batched training and evaluation.

During evaluation, we use the predefined train/dev/test sets of PHOENIX14T. For MedASL, we allocate 80% for 5-fold cross-validation and 20% for testing.

4.3. Model Development

We evaluate our proposed end-to-end system using two AI models: Encoder–Decoder Transformer [17] and ADAT [18].

4.3.1. Encoder–Decoder Transformer

The Encoder–Decoder Transformer maps sequences of visual features extracted from sign videos to sequences of spoken-language tokens. It is the most used and accurate model for SLMT due to its ability to capture long-range spatiotemporal dependencies, which are relationships between distant frames in a sign sequence and their corresponding linguistic units [4]. In particular, its encoder processes a sequence of sign input embeddings and generates feature representations that are fed into the decoder, which predicts the target text sequence. During decoding, cross-attention aligns the encoder outputs with the linguistic context at each step. This Encoder–Decoder interaction enables precise translation of sign video inputs into grammatically correct sentences. The attention mechanisms are presented in Equations (1)–(3).

$$Self-Attention=Sofmax\left({\displaystyle \frac{{\mathrm{Q}\mathrm{K}}^{\mathrm{T}}}{\sqrt{d_{\mathrm{k}}}}}\right)V$$

(1)

where Q, K, and V represent query, key, and value, respectively. $d_{\mathrm{k}}$ is the dimension.

$$Multi-Head Attention=Concat({Attention}_1\dots {Attention}_i)\times w^o$$

(2)

where $w^o$ is a learned weight that adds parameters in a single-head self-attention.

$$Cross-Attention(Q_d,K_e,V_e)=Sofmax({\displaystyle \frac{{\mathrm{Q}}_d{\mathrm{K}}_e^T}{\sqrt{d_{\mathrm{k}}}}})V_e$$

(3)

where ${\mathrm{Q}}_{\mathrm{d}}$ is the decoder query and ${\mathrm{K}}_{\mathrm{e}},{\mathrm{V}}_{\mathrm{e}}$ are from the encoder output.

4.3.2. Adaptive Transformer (ADAT)

ADAT modifies the Encoder–Decoder Transformer by integrating convolutional layers for localized feature extraction, LogSparse Self-Attention (LSSA) [49] to reduce the quadratic cost, and an adaptive gating mechanism [50] that balances short- and long-range dependencies.

The Encoder–Decoder Transformer’s self-attention computes pairwise attention scores between every pair of tokens in a sequence of frames, resulting in a quadratic computational cost. On the other hand, ADAT’s LSSA restricts each frame to attend only to a logarithmically spaced subset of preceding frames, sampled at exponentially increasing intervals. This structure preserves long-range temporal dependencies while reducing the complexity from $O\left(n^2\right)$ to $O\left(nlogn\right)$. Consequently, LSSA reduces redundant attention operations, making ADAT faster and more efficient for long sign sequences while maintaining contextual accuracy.

Equations (4) and (5) show LSSA and the gating mechanism, respectively.

$$LSSA(Q,K)=Sofmax\left({\displaystyle \frac{Q_{I_p^j}{K}_{I_p^j}^T}{\sqrt{\frac{d_k}{2}}}}\right)$$

(4)

where $I_p^j$ is the patch indices that a current patch $p$ can attend during a logarithmic computation from $j$ to $J+1$.

$$Gating=g·LSSA+(1-g)·GAP$$

(5)

where $g$ is the gate value computed via a gate neural network to adaptively weight between LSSA, i.e., long-range dependencies, and Global Average Pooling (GAP), i.e., short-range dependencies.

Prior studies have shown that the adaptive gating mechanism achieves comparable or superior translation quality to the Encoder–Decoder Transformer while significantly improving efficiency and scalability [18].

In summary, an accurate SLMT model is achieved by training on temporally ordered visual embeddings, ensuring that attention mechanisms learn spatial relationships between keypoints and temporal dynamics across the signing sequence.

4.4. Experimental Setup

The proposed system operates in two stages: (1) S2T translation, executed by the AI-enabled application in coordination with the edge device, and (2) SLMT development, which trains AI models on benchmark datasets and retrains them on consented, annotated user videos.

In the first stage, sign videos are captured at 30 frames per second using an Intel RealSense D455 camera, connected to the edge, represented by a desktop in our experiments. The videos are streamed to the application, and the edge device extracts keypoints via MediaPipe, applies preprocessing, and performs inference. In the second stage, the edge connects to a cloud server for model training. Retrained models are periodically deployed back to the edge for updated inference. Figure 4 illustrates the experimental setup, where a user signs in ASL in front of an Intel RealSense D455 camera connected to the edge device.

4.5. Experiments

We evaluate the translation quality and runtime efficiency of the proposed system under scenarios that mirror real-world operation. To assess translation quality, we compare generated translations to human-annotated references using BLEU [46] and ROUGE [47] scores. Applying these metrics across both datasets allows direct comparison between the Encoder–Decoder Transformer and ADAT architectures.

Regarding efficiency, we record the time required for model convergence during training and measure detailed runtime behavior during inference. At the beginning of inference, the edge device retrieves the most recent trained model from the cloud to stay synchronized with the latest deployment, defining the cloud-to-edge communication time. The edge then continuously captures video and processes it in short overlapping segments of 30 consecutive frames. Once the first segment is complete, the device begins generating translations, applying a brief warmup period to stabilize performance. At each translation step, the edge records preprocessing and inference times. The former covers feature extraction, normalization, and batching, while the latter measures the duration of sign language translation. After processing video segments, the edge uploads the extracted keypoints and corresponding translations to the cloud, defining the edge-to-cloud communication time.

We measure the overall end-to-end system time as perceived by the user. This interval spans from the first captured frame to the display of the translated text, including capture, preprocessing, inference, and edge–cloud communication. The system maintains synchronized clocks across both components to ensure timing consistency. These experiments provide the methodological basis for the runtime evaluation in Section 5.

5. Performance Evaluation

This section evaluates the models under study within a unified experimental setup, focusing on translation quality and runtime efficiency. We structure the evaluation into four components: the metrics used to assess translation and runtime performance, the hyperparameter tuning strategy, the hardware specifications for training and inference, and an in-depth numerical analysis of the results.

5.1. Evaluation Metrics

To ensure a comparable assessment of linguistic accuracy and system-level efficiency, we measure translation quality using BLEU and ROUGE. We use NLTK’s BLEU [46] and Rouge-score [51] Python 3.10 libraries. We assess runtime efficiency by reporting training, translation, communication, and end-to-end system times.

5.1.1. Bilingual Evaluation Understudy (BLEU)

We use a BLEU score with n-grams ranging from 1 to 4, as shown in Equations (6) and (7).

$$BLEU=BP ·e^{(\sum _{n=1}^N{w}_n\mathit{log}\left(p_n\right)}$$

(6)

where $p_n$ is the precision of n-grams, $w_n$ is the weight of each n-gram size, and $\mathrm{B}\mathrm{P}$ is the Brevity Penalty.

$$BP=\left\{\begin{array}{c}1, if G>r\\ e^{(1-{\displaystyle \frac{r}{G}})}, if G\le r\end{array}\right.$$

(7)

where $\mathrm{G}$ is the length of the generated translation. $r$ is the reference corpus length.

5.1.2. Recall-Oriented Understudy for Gisting Evaluation (ROUGE)

We assess the similarity between generated translations and their references using ROUGE-1 and ROUGE-L. ROUGE-1 measures unigram overlap, while ROUGE-L assesses the longest sequence of overlapping words. We normalize both metrics through precision and recall. ROUGE-L is calculated using Equations (8)–(10).

$${ROUGE}_{precision}={\displaystyle \frac{\sum _{n=1}^{N}LCS(G,R)}{\sum _{n=1}^N\left|G\right|}}$$

(8)

$${ROUGE}_{recall}={\displaystyle \frac{\sum _{n=1}^{N}LCS(G,R)}{\sum _{n=1}^N\left|R\right|}}$$

(9)

where $\mathrm{G}$ is the generated translation. $\mathrm{R}$ is the reference translation. $\mathrm{L}\mathrm{C}\mathrm{S}(\mathrm{G},\mathrm{R})$ is the length of the longest common subsequence between $\mathrm{G}$ and $\mathrm{R}$. $\left|·\right|$ is the number of tokens in the translation.

$$ROUGE-L={\displaystyle \frac{2 ·{ROUGE}_{precision}·{ROUGE}_{recall}}{{ROUGE}_{precision}+{ROUGE}_{recall}}}$$

(10)

5.1.3. Runtime Metrics

We measure the runtime efficiency by calculating the training time and the latency across training, preprocessing, inference, translation, edge–cloud communication, and the overall end-to-end system time, as presented in

Table 6. Runtime efficiency metrics, definitions, and equations.

Metric	Notation	Definition	Equation
Training time	$T_{\mathrm{train}}$	Time required for model convergence during training.	$T_{\mathrm{train}}=N_{epochs}\times T_{epoch}$
Preprocessing time	$T_{\mathrm{pre}}$	Time required to convert sign videos into keypoints and prepare them for inference.	Not applicable
Inference time	$T_{\mathrm{infer}}$	Time required for model inference once preprocessing is completed.	Not applicable
Translation time	$T_{\mathrm{translate}}$	Total user-perceived translation time.	${T_{\mathrm{translate}}=T}_{pre}+T_{\mathrm{infer}}$
Edge–cloud communication time 1	$T_{\mathrm{edge}2\mathrm{cloud}}$	Uplink latency for transmitting keypoints and videos from the edge to the cloud.	Not applicable
Cloud–edge communication time 2	$T_{\mathit{cloud}2\mathit{edge}}$	Downlink latency for transmitting trained models from the cloud to the edge.	Not applicable
Overall end-to-end system time	$T_{\mathit{end}2\mathit{end}}$	Total system latency.	$T_{\mathrm{end}2\mathrm{end}}=T_{\mathrm{translate}}+T_{\mathrm{edge}2\mathrm{cloud}}$

¹ Occurs under application-level consent for model improvement and/or system-level consent. ² Occurs periodically after model retraining.

. These measurements capture the backend operational overhead and the user-perceived latency.

5.2. Hyperparameter Tuning

We perform systematic hyperparameter tuning to achieve the best translation performance across both models. The search space includes optimization strategies, training setups, model architectures, and decoding choices. The final hyperparameters are selected based on validation BLEU-4 scores with early stopping.

Table 7. Hyperparameter search space and selected configurations.

Category	Hyperparameter	Configurations Studied	Selected Configurations
Optimization	Loss functions	Gloss: CTC loss, none Text: Cross-Entropy loss	Gloss: CTC loss Text: Cross-Entropy loss
	Label smoothing	0, 0.01, 0.05, 0.1	0.1
	Optimizer	Adam with β1 = 0.9 (default) or 0.99 and β2 = 0.999 (default) or 0.98	Adam with β1 = 0.9, β2 = 0.98
	Learning rate	5 × 10−5, 1 × 10−5, 1 × 10−4, 1 × 10−3, 1 × 10−2	5 × 10−5
	Weight decay	0, 0.1, 0.001	0
	Early stopping	Patience = 3, 5 Tolerance = 0, 1 × 10−2, 1 × 10−4, 1 × 10−6	Patience = 5 Tolerance = 1 × 10−6
	Learning rate scheduler	Constant with linear warmup until either 1000 steps have been taken or 5% of the total training steps are reached, whichever is larger.
Training setup	Gradient clipping	None, 0.1, 0.5, 1.0, 2.0, 3.0, 4.0, 5.0	5.0
	Epochs	30, 100	30
	Batch size	32	32
Model architecture	Embedding dimension	256, 512	512
	Encoder layers	1, 2, 4, 12	1
	Decoder layers	1, 2, 4, 12	1
	Attention heads	8, 16	8
	Feed-forward size	512, 1024	512
	Dropout	0, 0.1	0.1
Decoding	Gloss	Greedy or beam search	Greedy
	Text	Greedy or beam search	Beam search with a size of 5 with reranking using a 4 g KenLM * (interpolation weight 0.4, length penalty 0.2).

CTC: Connectionist Temporal Classification [52]. *: We build a KenLM model [53] using text tokenized with SentencePiece [54] into BPE [55] sub-word units to rerank the generated translation.

summarizes the hyperparameters investigated, the ranges explored, and the chosen settings.

5.3. Hardware Specifications

The evaluation was conducted using a distributed setup consisting of a capture unit, an edge device, and a cloud server. The capture unit provides lightweight video acquisition, the edge device handles preprocessing and inference, and the cloud server supports model development and data storage. This division reflects practical deployment, where time-sensitive tasks are executed closer to the user while high-compute workloads are offloaded to the cloud.

Table 8. Hardware specifications for performance evaluation.

Device	CPU	GPU	Storage	RAM	Connectivity	Operating System
Intel RealSense Camera	Vision Processor D4 (ASIC)	NA	2 MB	NA	USB Type-C	NA
Edge	1 x Intel Core i7-8700 at 4.20 GHz, 6 cores/CPU	NA	475 GB	8 GB	Ethernet	Windows
Cloud	2 x AMD EPYC 7763 at 2.45 GHz, 64 cores/CPU	2 x NVIDIA RTX A6000 48 GB/GPU	5 TB	1 TB	Wi-Fi, LTE *, Ethernet	Ubuntu

NA: not available; *: supported through external hotspot.

lists the specifications of each device. All software components are implemented using Python 3.10.

5.4. Experimental Results Analysis

This section presents a numerical evaluation of the Encoder–Decoder Transformer and ADAT in terms of translation quality and runtime efficiency.

5.4.1. Translation Quality

Table 9. Comparison of models for sign language translations (scores range between 0 and 1; higher being better). Best results are in bold.

Model	Dataset	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-1	ROUGE-L
Encoder–Decoder Transformer	PHOENIX14T	0.32	0.21	0.14	0.10	0.18	0.16
ADAT		0.33	0.23	0.16	0.12	0.28	0.27
Encoder–Decoder Transformer	MedASL	0.39	0.24	0.15	0.09	0.19	0.18
ADAT		0.43	0.26	0.16	0.10	0.20	0.20

compares the translation quality of the Encoder–Decoder Transformer and ADAT models using the PHOENIX14T and MedASL datasets. On PHOENIX14T, ADAT surpasses the Encoder–Decoder Transformer in all metrics, with +0.02 and +0.11 absolute points in BLEU-4 and ROUGE-L, respectively. Similarly, on MedASL, ADAT outperforms the Encoder–Decoder Transformer with +0.01 and +0.02 in BLEU-4 and ROUGE-L, respectively. This is because ADAT integrates LSSA, which retains key temporal segments across signing sequences, and an adaptive gating mechanism to balance short- and long-range dependencies.

ADAT shows larger performance gains than the Encoder–Decoder Transformer using ROUGE-L across datasets. This is because ADAT’s LSSA component enables preservation of global semantic consistency rather than focusing on word-level matches. BLEU-n measures exact overlaps of n-gram sequences; therefore, its smaller improvement reflects limited changes in local accuracy. In contrast, ROUGE-L evaluates the overall sentence structure and semantic alignment with the reference translation, which explains ADAT’s gains at the sentence level.

In addition, ADAT achieves +0.11 gain in ROUGE-L using PHOENIX14T versus +0.02 gain using MedASL. This is because PHOENIX14T has longer and more structured sentences, where maintaining global information order is essential. Conversely, MedASL sentences are shorter and more diverse, which limits the improvement in the overall sentence-level structure.

In summary, our experimental results demonstrate that while ADAT’s BLEU score gains are minimal, ROUGE-L score confirms its effectiveness in SLMT. In particular, ADAT is designed to preserve sentence-level organization and global semantic coherence. Therefore, it delivers translations that maintain the overall structure and meaning of the original message, making it a more effective model for real-world SLMT systems.

5.4.2. Runtime Efficiency

We assess the models’ runtime efficiency within the edge–cloud pipeline using the MedASL dataset. Figure 5 presents the model training time in both datasets.

Table 10. Comparison of sign language translation runtime and system performance. Best results are in bold.

Model	Preprocessing Time (ms)	Inference Time (ms)	Translation Time (ms)	Cloud-to-Edge Communication Time (ms)	Edge-to-Cloud Communication Time (ms)	Overall End-to-End System Time (ms)	Model Size (MB)	Keypoints Size (MB)
Encoder–Decoder Transformer	0.03	188.25	188.28	11,725.42	324.09	512.37	134.8	3
ADAT	0.02	183.75	183.77	11,517.81	312.85	496.62	104.9	3

ms: milliseconds.

presents the runtime and system performance for both models. Figure 6 illustrates ADAT’s relative improvements over the Encoder–Decoder Transformer.

The results indicate that training on PHOENIX14T takes longer than on MedASL due to its larger size and vocabulary. Within each dataset, ADAT consistently converges faster than the Encoder–Decoder Transformer, reflecting its reduced computational overhead. In particular, ADAT reduces training time by 1.6× on PHOENIX14T and by half on MedASL, due to its adaptive attention mechanism.

In real-world deployment, translation and communication times are critical constraints. In our experiments, ADAT slightly reduces translation time from 188.28 ms to 183.77 ms by enabling faster inference, resulting in a 1.02× relative improvement. It also achieves 1.02× speedup in cloud–edge communication. This is explained by ADAT’s 22% smaller model size compared to the Encoder–Decoder Transformer (104.9 MB vs. 134.8 MB), which reduces computational overhead. When considering the complete pipeline, ADAT delivers a 1.02× reduction in overall end-to-end time, showing that latency gains are consistent across the entire pipeline. These findings confirm that ADAT sustains efficiency advantages during training and inference in distributed environments.

Comparing ADAT with the Encoder–Decoder Transformer highlights the trade-off between efficiency and translation quality. ADAT improves sentence-level fluency with lower computational overhead due to its sparse attention and adaptive gating. However, the Encoder–Decoder Transformer achieves comparable translations but is more complex than ADAT due to quadratic attention computation.

In summary, ADAT is faster in training, translation, and data transfer. These characteristics make it well-suited for deployment in SLMT systems operating in distributed edge–cloud environments with blockchain integration.

6. Responsible Deployment of the Proposed System

The experimental results, presented and discussed in Section 5, demonstrate the technical feasibility of SLMT systems in an edge–cloud architecture. However, real-world SLMT deployment requires specific considerations, which we address in this section.

6.1. Ethical, Legal, and Data Rights Considerations

Sign language video recordings contain sensitive biometric and contextual information. As outlined in the Experimental Setup (Section 4.4), the deployed system captures high-resolution videos and extracts and stores the signer’s keypoints for inference and model retraining. Managing such data requires clear protocols for ownership, consent, and accountability, guided by international and national frameworks, such as the General Data Protection Regulation (GDPR) [21] and the UAE Federal Decree Law concerning the Protection of Personal Data (PDPL) [22].

Users must grant consent, which they fully understand and can withdraw at any time. These consents must be auditable to ensure accountability and transparency. The blockchain ledger, described in Section 3.5, provides an immutable record for each consent-related event [44]. Once logged, entries cannot be altered or erased; instead, any changes are appended as new records. This design preserves the whole history of modifications and prevents record tampering.

Furthermore, users should retain ownership of raw video recordings and extracted keypoints, while also having the right to withdraw. Moreover, the collected data must not be repurposed for unrelated applications without renewed consent. In addition, transparency is ensured through blockchain-based logs, providing verifiable and tamper-resistant records of data usage [20].

6.2. Security Threats and Mitigations

An edge–cloud–blockchain SLMT pipeline introduces security risks that must be addressed to ensure trustworthy deployment. The system processes data and metadata across its components. The data include raw sign videos, extracted keypoints, generated translations, and incident logs that the edge produces and, when users grant consent, transmits to the cloud. In the cloud, the system generates additional data, including annotated sign videos used for model retraining, trained model files, and diagnostic records. We classify the metadata into two categories: (1) consent-related metadata, which includes consent receipts, policy logs, and audit logs, and (2) system-generated metadata, which consists of system certificates, data receipts, hashes of processed keypoints and translations, policy versions, timestamps, and digital signatures.

All data and metadata are encrypted during transmission and storage using Advanced Encryption Standard (AES-256) [56] and Elliptic Curve Cryptography (ECC) [57]. AES-256 provides symmetric encryption, securing data through a shared encryption key known only to authorized components. ECC enables asymmetric key exchange and digital signatures. During key exchange, each component, i.e., edge, cloud, and blockchain, uses its private key along with the other component’s public key to compute an identical shared AES encryption key, without exposing it. These cryptographic mechanisms ensure confidentiality, integrity, and authenticity across all components of the distributed architecture. However, while encryption secures data when stored and transmitted, the system must also address broader security threats across the operational pipeline.

• Video tampering poses a significant risk, as it can lead to consent violations, model corruption, and unauthorized access during training and validation [58]. This can be mitigated by applying a cryptographic chain of hashing to the videos to ensure integrity, and by employing homomorphic encryption and secure computation over encrypted data, which allows processing without exposing personal data [59].
• Data manipulation, such as unauthorized alteration of consent receipts, policy records, or audit logs, threatens the reliability and accountability of the SLMT system. The blockchain layer mitigates this by making all committed records immutable, validating each new transaction through distributed consensus, and replicating the ledger across distributed nodes. Since all nodes retain synchronized copies, any inconsistency reveals manipulation and is automatically rejected. These mechanisms maintain consistent, tamper-evident records and preserve the integrity [44].
• Unauthorized access to stored data and trained models at the edge and cloud must be protected through strong encryption, multi-factor authentication, and role-based access control [60].
• Consent and policy misuse, such as using data beyond the approved scope, is reduced through a feedback loop [45], where edge and cloud continuously reference the latest consent receipts and policy logs before processing or updating models [61].
• Deanonymization risks from signer keypoints or metadata are reduced through local differential privacy methods, which introduce noise at the edge to prevent identity exposure [62].

In summary, the proposed SLMT system employs a layered set of security mechanisms that extend beyond encryption to ensure end-to-end trust. These measures preserve data reliability, regulatory compliance, and user confidentiality throughout capture, transmission, processing, and storage.

6.3. Minimum Requirements for Responsible Sign Language Translation System Deployment

The empirical evaluation and the ethical and security considerations discussed above define the following minimum requirements for deploying SLMT systems:

• Translation quality must be sufficient to support transparent and precise communication between DHH individuals and the broader society. Section 5.1 shows that ADAT consistently outperforms the Encoder–Decoder Transformer across all metrics, demonstrating the feasibility of achieving precise translation.
• Latency must be minimized to ensure an efficient end-to-end translation. Section 5.2 demonstrates ADAT’s reductions in inference and communication times in the edge–cloud setup.
• Generalization across signers and environmental conditions is essential to ensure inclusivity. Achieving this requires continuous data collection from different signers, domains, and environments.
• Data collection and model retraining must comply with international and national regulations, such as GDPR [21] and UAE PDPL [22]. These principles ensure that users have control over their data, can withdraw their consent, and determine how their data is used. The blockchain-based logging mechanism described in Section 3.5 provides a solid foundation for meeting these obligations and ensuring accountability.
• Security and robustness are critical for reliable deployment. As discussed in Section 6.2, proper safeguards must be integrated into the system design.
• Scalability must be maintained as the number of users and translation requests increases. Employing an efficient clustered lightweight blockchain as proposed in [63], where only representative nodes replicate and validate the ledger, sustains responsiveness as workloads grow. This approach reduces communication load, improves ledger synchronization across clusters, and provides high throughput.

In summary, integrating a precise AI model into an edge–cloud–blockchain architecture demonstrates that advances in translation and runtime efficiency must be accompanied by ethical, legal, and security safeguards. This alignment is essential for deployment in domains such as healthcare, where translation quality and data privacy are critical. By adhering to established regulations, incorporating blockchain-based auditability, and employing robust security protections, the proposed architecture serves as a blueprint for future SLMT technologies that strike a balance between technical effectiveness and responsible deployment. Beyond SLMT, this work also provides a transferable model for the responsible deployment of AI in other biometric domains, including speech and facial recognition.

7. Conclusions

In this work, we present a novel end-to-end sign language machine translation (SLMT) system built on an edge–cloud–blockchain architecture. The proposed system consists of a sign language recognition (SLR) module, an AI-enabled SLMT application, edge nodes, cloud servers, and a blockchain layer that provides immutable logging, ensuring compliance with data protection regulations.

Within this framework, we develop and evaluate the Encoder–Decoder Transformer and the Adaptive Transformer (ADAT) with respect to translation quality and runtime efficiency. We conduct a comparative analysis on RWTH-PHOENIX-Weather-2014T (PHOENIX14T) and the newly extended MedASL dataset. The results show that ADAT consistently outperforms the Encoder–Decoder Transformer in translation accuracy and achieves a twofold reduction in training time, with lower inference and communication latency.

Future work should expand multilingual sign language datasets that incorporate diverse signers across various domains, integrate multimodal inputs with advanced feature extraction to enhance semantic alignment, explore lightweight models for constrained edge devices, and extend blockchain frameworks to support cross-institutional governance and enable broader, transparent adoption across domains.