Spatio-Temporal Deep Learning with Adaptive Attention for EEG and sEMG Decoding in Human–Machine Interaction

Abstract

Electroencephalography (EEG) and surface electromyography (sEMG) signals are widely used in human–machine interaction (HMI) systems due to their non-invasive acquisition and real-time responsiveness, particularly in neurorehabilitation and prosthetic control. However, existing deep learning approaches often struggle to capture both fine-grained local patterns and long-range spatio-temporal dependencies within these signals, which limits classification performance. To address these challenges, we propose a lightweight deep learning framework that integrates adaptive spatial attention with multi-scale temporal feature extraction for end-to-end EEG and sEMG signal decoding. The architecture includes two core components: (1) an adaptive attention mechanism that dynamically reweights multi-channel time-series features based on spatial relevance, and (2) a multi-scale convolutional module that captures diverse temporal patterns through parallel convolutional filters. The proposed method achieves classification accuracies of 79.47% on the BCI-IV 2a EEG dataset (9 subjects, 22 channels) for motor intent decoding and 85.87% on the NinaPro DB2 sEMG dataset (40 subjects, 12 channels) for gesture recognition. Ablation studies confirm the effectiveness of each module, while comparative evaluations demonstrate that the proposed framework outperforms existing state-of-the-art methods across all tested scenarios. Together, these results demonstrate that our model not only achieves strong performance but also maintains a lightweight and resource-efficient design for EEG and sEMG decoding.

1. Introduction

A human–machine interface (HMI) is a framework designed to mitigate patient disability by facilitating connectivity and control between users and external rehabilitation devices [1]. Two widely used HMI paradigms are brain–computer interfaces (BCIs) and electromyography-based HMIs (EMG-HMIs) [2,3]. In both paradigms, non-invasive BCI and surface EMG-HMI (sEMG-HMI) systems have garnered significant attention due to their convenience, low risk, and relative simplicity compared to invasive methods in medical applications [4,5,6]. Non-invasive BCI techniques control robotic devices to facilitate motor recovery in patients with stroke or other neural injuries by measuring changes in cortical neuronal activity during motor imagery (MI) tasks [7]. And the sEMG-HMI system employs gesture recognition based on surface EMG signals to assist individuals with mobility impairments in controlling prosthetic devices by distinguishing signal patterns produced by muscle activations across different gestures [8]. However, the susceptibility of EEG and sEMG signals to various sources of interference poses significant challenges for accurate signal recognition. These contaminants, such as motion artifacts, ambient noise, and electromagnetic interference, adversely affect amplitude, temporal, and spectral features commonly used in assessment and control applications [9,10]. Additionally, physiological variability among individuals and constraints in computational resources further complicate the development of robust HMI systems [11,12].

Recent advances in deep learning have prompted researchers to explore diverse neural network architectures to enhance the classification accuracy of EEG and sEMG signals. Convolutional neural networks (CNNs) [13] excel at extracting spatially localized features, but their receptive fields are inherently limited, making it difficult to model long-range temporal dependencies that are essential for sequential biosignals. Recurrent neural networks (RNNs) [14], long short-term memory networks (LSTMs) [15], and bidirectional LSTMs (Bi-LSTMs) [16] are designed for sequence modeling, yet they often suffer from vanishing gradients, limited parallelization, and difficulty in learning long-term dependencies over noisy and variable-length input sequences. Transformer models, with their superior sequence-modelling capabilities, have also attracted attention in biosignal processing. However, conventional Transformer architectures [17] exhibit high computational complexity and typically demand extensive pre-training data before being adapted to recognition tasks. Given the prevalent data scarcity in the EEG and sEMG HMI paradigms, the direct application of Transformers in this domain remains constrained.

To address these challenges, this paper proposes SCGTNet, a lightweight spatio-temporal fusion Transformer model that integrates an adaptive spatial attention mechanism with a multi-scale temporal feature extraction strategy to enhance both classification accuracy and computational efficiency for EEG and sEMG signals. The main contributions of this study are as follows:

• We introduce a novel CNN–Transformer architecture, SCGTNet, for EEG and sEMG classification in human–computer interaction. By capturing short-term local features and long-term dependencies, SCGTNet improves overall classification accuracy.
• We design an adaptive spatial attention (ASA) module that dynamically re-weights channel features to enhance signal discriminability, focusing the model on salient channels while suppressing redundant information.
• We propose a multiscale temporal feature extraction module (MSC), which consists of a one-dimensional convolutional parallel, combined with gated recursive units (GRUs) to improve the model’s ability to learn complex temporal patterns.

We evaluate SCGTNet on the BCI IV 2a (EEG) and NinaPro DB2 (sEMG) datasets. Experimental results demonstrate that SCGTNet outperforms state-of-the-art methods in both classification accuracy and computational efficiency.

The remainder of this paper is organized as follows. Section 2 reviews related work on deep learning approaches for EEG and sEMG signal classification. Section 3 details the proposed adaptive spatial attention structure, multi-scale convolution module, and overall SCGTNet design. Section 4 describes the datasets, preprocessing procedures, and experimental setup, followed by ablation studies and comparative analyses. Finally, Section 5 and Section 6 discuss the results, conclude the paper, and outline directions for future work.

2. Related Works

In recent years, deep learning has facilitated significant advances in recognition applications within electroencephalography (EEG) [18] and surface electromyography (sEMG) [3]. Dose et al. [19] proposed a method for an EEG-based motor imagery (MI) brain–computer interface (BCI) system, employing a convolutional neural network (CNN) layer for generalized feature learning and dimensionality reduction, followed by a traditional fully connected (FC) layer for classification. Hajian et al. [20] introduced a CNN for gesture recognition that extracts spatial features by stacking multiple convolutional layers, using EMG spectrograms as input. However, the reliance on a fixed convolutional kernel hinders the method’s ability to dynamically adapt to rapid changes in neural signals across different motion states.

To overcome this limitation, Li et al. [21] proposed a neural network feature fusion algorithm for motor imagery brain–computer interfaces (MI-BCIs), in which convolutional neural networks (CNNs) and long short-term memory (LSTM) networks operate in parallel. In their design, the CNN branch employs convolutional layers followed by a flattening layer to extract spatial features, while the LSTM branch captures temporal dependencies. All extracted features are then merged within a fully connected layer to enhance classification accuracy. Bao et al. [22] similarly introduced a CNN–LSTM architecture; a CNN first derives deep features from sEMG spectrograms, and an LSTM-based sequential regression module subsequently estimates muscle kinematics. However, both approaches still incur information loss during feature fusion, and in practical scenarios, such as prolonged, multimodal rehabilitation training and exercise monitoring, interindividual variability and diversity of movement patterns place even greater demands on model robustness.

To address these challenges, Lee et al. [23] employed the short-time Fourier transform to efficiently extract time–frequency representations of EEG signals and incorporated a self-attention mechanism to selectively emphasize salient features. Wan et al. [24] developed EEG-Former, a hierarchical inflated convolutional architecture that simultaneously captures exponentially expanding receptive fields (dilation rates of 1, 3, and 9) by exploiting short-time (<200 ms) transient components of event-related potentials alongside their long-term correlations with rhythmic oscillations. However, both methods still rely on static parameterization for spatial channel weight assignment, limiting their ability to adaptively respond to real-time inputs.

For signals exhibiting time-varying characteristics, multi-scale convolution has become a focal point of recent research. Zhang et al. [25] introduced a multi-scale Inception module within the EEG-Inception model—an important non-invasive approach to motor imagery (MI) classification in brain–computer interface (BCI) studies—that employs convolutional kernels of varying sizes to extract local details and global patterns in parallel across time–frequency features. Ding et al. [26] subsequently proposed a parallel multi-scale convolutional architecture to enhance the classification accuracy of sEMG-based limb motion recognition by incorporating filters of different scales while accounting for muscle independence. However, these networks still depend on fixed kernel sizes in their underlying CNN architectures, which limits their capacity to dynamically adjust feature extraction weights.

Furthermore, the Transformer model has revolutionized the landscape of natural language processing (NLP) [27]. Its ability to process all data points in parallel significantly reduces training time. Transformers are highly scalable, and large-scale models can capture complex patterns in massive datasets, achieving state-of-the-art performance across a variety of tasks [28]. However, prevalent data scarcity in EEG- and sEMG-based human–machine interface (HMI) paradigms constrains the direct and independent application of Transformers in this domain [29].

Compared with existing approaches, the adaptive spatial attention–parallel convolution architecture proposed in this work dynamically integrates channel attention weights with multi-scale temporal features, thereby enhancing the fine-grained classification accuracy of EEG and sEMG signals while maintaining a lightweight design. This novel framework offers a promising direction for future EEG- and sEMG-based HMI research.

3. Materials and Methods

In this section, we present the main architecture of SCGTNet, followed by a detailed description of the datasets and preprocessing procedures.

SCGTNet is a lightweight neural network model inspired by the Temporal Fusion Transformer (TFT) architecture [30]. It integrates several deep learning components, including an adaptive spatial attention module, a multi-scale convolution module, GRU, Transformer multi-head attention, and a feed-forward network. By design, these modules enable the effective extraction of both local features and long-range temporal dependencies from time-series signals, leading to improved classification accuracy for electroencephalographic (EEG) and surface electromyographic (sEMG) signals. The overall architecture of SCGTNet is illustrated in Figure 1. The core structure comprises the following main modules:

3.1. Adaptive Spatial Attention Module

The variability in the contribution of different sensor channels to the classification task is a key aspect of this study. To address this, the adaptive spatial attention module employs a channel attention mechanism to dynamically adjust the feature importance of multi-channel time-series data. Initially, Global Average Pooling (GAP) and Global Maximum Pooling (GMP) are performed along the time dimension to generate two feature vectors. These vectors are then concatenated and fed into a fully connected layer, where the channel attention weights are produced via a Sigmoid activation function. The resulting weights are subsequently applied to the original input through broadcast multiplication, thereby enhancing or suppressing specific feature channels.

${\mu }_c$ denotes the global average pooling, which reflects the overall response strength of the channel. The input tensor is $X\in {\mathbb{R}}^{B\times T\times C}$, B is the batch size, T is the time_steps, C is the number of channels:

$${\mu }_c={\displaystyle \frac{1}{T}}\sum _{t=1}^T{X}_{b,t,c}\forall b\in [1,B],c\in [1,C]$$

(1)

${\nu }_c$ denotes global maximum pooling for capturing significant local features:

$${\nu }_c={max}_{1\le t\le T}{X}_{b,t,c}\forall b,c$$

(2)

Z is the vector obtained by splicing the results of global average pooling and global maximum pooling along the channel dimensions:

$$Z=[\mu ;\nu ]\in {\mathbb{R}}^{B\times 2C}$$

(3)

Getting attention weights W through the fully connected layer, $W_1\in {\mathbb{R}}^{2C\times r}$, $W_2\in {\mathbb{R}}^{r\times C}$, $\sigma$ is Sigmoid function, $\delta$ is ReLU activation, compression ratio r is implicitly set to 1 (due to no explicit dimensionality reduction):

$$W=\sigma (W_2·\delta (W_1·Z+b_1)+b_2)$$

(4)

The output of this module is a channel-weighted feature map:

$${\widehat{X}}_{b,t,c}=X_{b,t,c}·W_{b,c}$$

(5)

Figure 2 presents the design of the Adaptive Spatial Attention Module.

3.2. Multi-Scale Convolution Module

To capture features at various time scales, the multi-scale convolution module utilizes a parallel convolution structure that deploys three one-dimensional convolution kernels of different widths—specifically, 3, 5, and 7. This design enables the extraction of short-term (e.g., transient events), medium-term (e.g., periodic patterns), and long-term (e.g., trend changes) features along the time axis. Each branch yields the same number of feature maps, which are then concatenated to form a comprehensive multi-scale feature representation.

$Y^{\left(k\right)}$ is output for each branch, where the input tensor is $X\in {\mathbb{R}}^{B\times T\times C\times 1}$ (after reshape), and $W^{\left(k\right)}\in {\mathbb{R}}^{k\times 1\times 1\times F}$ is learnable parameters. The set of convolution kernel widths is $K=\{3,5,7\}$:

$$Y^{\left(k\right)}=\mathrm{Conv}2\mathrm{D}(X;W^{\left(k\right)},b^{\left(k\right)})\in {\mathbb{R}}^{B\times T\times C\times F},k\in K$$

(6)

$Y_{\mathrm{concat}}$ is the concatenated output:

$$Y_{\mathrm{concat}}=[Y^{\left(3\right)};Y^{\left(5\right)};Y^{\left(7\right)}]\in {\mathbb{R}}^{B\times T\times C\times 3F}$$

(7)

Figure 3 illustrates the architecture of the Multi-Scale Convolution Module.

3.3. GRU Encoder

GRUs (Gated Recurrent Units) are a recurrent neural network structure commonly used in sequence modelling to effectively capture temporal dependencies in sequences. Their gating mechanism consists of a reset gate and an update gate, enabling GRUs to manage long-term dependencies in data by selectively controlling the flow of information [31]. The reset gate governs the extent to which past information is discarded, while the update gate regulates the amount of new information that is integrated and carried forward, thereby ensuring effective temporal pattern learning [32].

The GRU update process is shown below, $\left\{h_t\right\}\in {\mathbb{R}}^{T\times d}$ is the input for previous module, $z_t$ is the control of historical status retention ratio (update gate), $r_t$ determines the effect of the previous state on the candidate state (reset gate):

$$z_t=\sigma (W_z·[h_{t-1},x_t])$$

(8)

$$r_t=\sigma (W_r·[h_{t-1},x_t])$$

(9)

$${\tilde{h}}_t=tanh(W·[r_t\odot h_{t-1},x_t])$$

(10)

$$h_t=(1-z_t)\odot h_{t-1}+z_t\odot {\tilde{h}}_t$$

(11)

Normalization GRU output $H\in {\mathbb{R}}^{B\times T\times d}$, with $\mu$ and $\sigma$ calculation along feature dimensions, and $\gamma$ and $\beta$ represent learnable parameters:

$$H_{\mathrm{norm}}=\gamma \odot {\displaystyle \frac{H-\mu }{\sqrt{{\sigma }^2+ϵ}}}+\beta$$

(12)

3.4. Multi-Head Attention

The Multi-Head Attention (MHA) mechanism is a powerful component in contemporary deep learning architectures, particularly when handling long sequential data. Its primary objective is to enable the model to adaptively focus on the significance of different parts of the input sequence by computing the interrelationships among its elements. In contrast to traditional single-head self-attention mechanisms, the multi-head approach computes several attention heads in parallel, thereby enhancing the model’s representational capacity and enabling it to disentangle complex dependencies [33].

This module is designed to enhance the model’s expressive capacity by capturing long-range dependencies between different segments of the input sequence. In particular, when processing time-series data, various positions within the sequence may contain critical contextual information that can be efficiently extracted through the application of multi-head attention.

The model employs a single Multi-Head Attention layer and the number of heads is 4. We set the key and value dimensions equally as dk = dv. Given a single head of attention $Q,K,V$, count as follows:

$$\mathrm{Attention}(Q,K,V)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$$

(13)

Multiple outputs spliced together and linearly transformed:

$$\mathrm{MultiHead}(Q,K,V)=\mathrm{Concat}(\mathrm{hea}{\mathrm{d}}_1,\dots ,\mathrm{hea}{\mathrm{d}}_h)W^O$$

(14)

We apply a residual connection from the preceding GRU output to the attention output, followed by Layer Normalization and Dropout. In implementation, we use TensorFlow’s built-in layers. All forward and backward computations—including the softmax nonlinearity and LayerNorm normalization—are handled by TensorFlow’s automatic differentiation. No manual gradient derivation or linear-gradient approximation is performed. The residual connection conceptually facilitates gradient flow, but the detailed Jacobian contributions from attention and normalization are computed by the framework.

3.5. Feed-Forward Network

Feedforward neural networks [34] (FFNs) not only improve feature abstraction but also provide the model with increased expressive degrees of freedom to learn intricate patterns through nonlinear transformations. The FFN layer typically comprises two fully connected layers utilizing swish activation functions for mapping and transforming the data. By employing residual connectivity and layer normalization, the FFN effectively mitigates training instability while augmenting the network’s overall representation and robustness.

Initially, a fully connected layer maps the input features to a higher-dimensional space, thus enhancing the nonlinear representational capability of the network. By expanding the network’s width, more complex features can be learned. To prevent overfitting, Dropout is applied after this layer, randomly discarding a portion of neurons to improve the model’s generalization ability. Subsequently, the features are processed by a second fully connected layer that reduces them back to their original dimensions, compressing the high-dimensional representations into a low-dimensional format suitable for classification. Finally, the output of the feedforward network is combined with the output of the multi-head attention module through residual connections and normalized. This operation facilitates smoother gradient propagation during training, thereby enhancing both the stability and efficiency of the training process. $W_1\in {\mathbb{R}}^{d\times 4d}$, $W_2\in {\mathbb{R}}^{4d\times d}$, which results in the formation of bottleneck structures:

$$\mathrm{FFN}\left(x\right)=W_2·\mathrm{Swish}(W_1·x+b_1)+b_2$$

(15)

3.6. Data and Preprocessing

3.6.1. BCI IV 2a Dataset

The BCI IV 2a database [35] is one of the most well-known and widely used publicly available datasets in the field of Brain–Computer Interface (BCI) research; it was released by a team of researchers from the Graz University of Technology in 2008 with the aim of providing a standardized electroencephalographic (EEG) signal classification task for the benchmark for EEG classification tasks and to facilitate the development of Motor Imagery (MI)-related algorithms. It recorded EEG data from nine subjects, each of whom performed two experiments (sessions), each consisting of six runs. In each run, subjects were required to complete 4 categories of motor imagery tasks, namely left hand, right hand, feet and tongue motor imagery. The task was repeated 12 times for each category, so each experiment contained a total of 288 trials (4 categories × 6 runs × 12 repetitions). Each trial flowed through three phases: cue, task, and rest. In the first phase, a fixation cross was displayed on the screen for 2 s. In the second phase, an arrow was displayed on the screen instructing the subject to perform a specific type of motor imagery for 4 s. In the final rest phase the screen was blank and lasted for a random period of 1.5 to 2.5 s. Data were acquired using 22 EEG electrodes and 3 EOG (electro-ocular) electrodes with a sampling frequency of 250 Hz. The EEG electrodes were arranged according to the international 10–20 system, covering the main motor areas of the brain.

EEG signal preprocessing aims to enhance the Sensorimotor Rhythm (SMR) and suppress artefacts. In this paper, only 22 EEG electrodes were used, the data from 3 EOG (electrooculogram) electrodes were excluded, and the preprocessing was divided into four steps. Firstly, the sampling rate was reduced to 125 Hz using a FIR anti-aliasing filter [36] (cut-off frequency 62.5 Hz, coefficient 128) to reduce the computational burden, $h\left(k\right)$ is the filter order, $N=129$:

$$x_{down}\left(n\right)=\sum _{k=0}^{N-1}h\left(k\right)·x(2n-k)$$

(16)

Subsequent elimination of the global potential offset with Average Reference [37]:

$$x_{ref}^i=x^i-{\displaystyle \frac{1}{22}}\sum _{j=1}^{22}{x}^j$$

(17)

The second stage used Common Spatial Pattern (CSP) filtering [38] to extract spatial discriminative features by maximizing the ratio of the two types of task variance. $X_i\in {\mathbb{R}}^{22\times 563}$ is the number of i in trial data, while $N_c$ is the number of attempts in label c. The first 3 pairs (6 in total) of spatial filters were selected, retaining 90% of the discriminative information:

$${\Sigma }_c={\displaystyle \frac{1}{N_c}}\sum _{i=1}^{N_c}{X}_i{X}_i^T$$

(18)

The Morlet wavelet transform [39,40] is then applied to the CSP filtered signal, $f_b=2$ is the bandwidth parameter, $f_c$ is center frequency. Wavelets are defined as:

$${\psi }_{f,t}\left(\tau \right)={\displaystyle \frac{1}{\sqrt{\pi f_b}}}{e}^{2i\pi f_c\tau }{e}^{-{\tau }^2/f_b}$$

(19)

Finally for constructing the model input, sliding window segmentation is used, setting the window length to 200 samples and the overlap rate to 50% to balance the timing resolution and computational efficiency. The final output dimension obtained is shown below, where $N_{frames}$ is the total number of frames, and 22 is the number of channels:

$$N_{frames}\times 200\times 22$$

(20)

Z-score normalized [41] $ϵ$ is used to prevent divide-by-zero errors:

$${\mu }_c=\mathbb{E}\left[x_c\right],{\sigma }_c=\sqrt{\mathbb{E}\left[{(x_c-{\mu }_c)}^2\right]},ϵ=1e^{-6}$$

(21)

$${\widehat{x}}_c={\displaystyle \frac{x_c-{\mu }_c}{{\sigma }_c+ϵ}}$$

(22)

3.6.2. NinaPro DB2 Dataset

The NinaPro DB2 [42] database is a standard dataset widely used in gesture recognition and electromyographic signal (EMG) research. The dataset was first released by Atzori et al. in 2012 and has been extended and improved in subsequent studies with the aim of facilitating the development of gesture recognition algorithms based on surface EMG signals. It contains EMG signal data from 40 healthy subjects, 29 males and 11 females, aged between 29 and 34 years. Each subject performed 50 different gestural movements, including finger, wrist and arm movements. These gestures covered common hand movements in daily life, such as clenching the fist, extending the palm, and pinching and fetching. During data acquisition, 12 surface EMG electrodes were used, which were placed on the forearms of the subjects to capture EMG signals from different muscle groups. In addition, the dataset contains synchronously acquired accelerometer data, which is used to provide spatial information of gesture movements. Its EMG signals were sampled at a frequency of 2 kHz, and each gesture movement was repeated six times, each lasting five seconds with a three-second rest period in between. This design allows the dataset to provide a sufficient sample size to support the training and validation of deep learning models. In addition, the dataset contains basic information about the subjects, such as age, gender, and hand habitus, which facilitates the study of the effect of individual differences on gesture recognition performance.

For data preprocessing of this dataset, this paper used a fourth-order Butterworth filter [43] to eliminate low-frequency baseline drift (<20 Hz) and high-frequency EMG noise (>450 Hz), where the cut-off frequency $f_c=[20,450]$, ordinal number $n=4$. The filtered signal retains the main energy of the muscle fiber action potential (80% of the energy is concentrated at 50–150 Hz):

$$H\left(s\right)={\displaystyle \frac{1}{\sqrt{1+{\left(\frac{s}{2\pi f_c}\right)}^{2n}}}}$$

(23)

Subsequently, in order to capture the transient features of muscle activity, a sliding window segmentation was used, with the window length set to 200 samples, corresponding to a 100 ms time window to satisfy the condition that 95% of the gesture activation time be less than 150 ms, and the overlap rate set to 50% (i.e., a step size of 100 samples), to balance the temporal resolution with computational efficiency. The final output dimension obtained is shown below, where $N_{frames}$ is the total number of frames, and 12 is the number of channels:

$$N_{frames}\times 200\times 12$$

(24)

Finally, to eliminate inter-individual signal amplitude differences, z-score normalization is performed for each channel [41], where $ϵ$ is to prevent divide-by-zero errors:

$${\mu }_c=\mathbb{E}\left[x_c\right],{\sigma }_c=\sqrt{\mathbb{E}\left[{(x_c-{\mu }_c)}^2\right]},ϵ=1e^{-6}$$

(25)

$${\widehat{x}}_c={\displaystyle \frac{x_c-{\mu }_c}{{\sigma }_c+ϵ}}$$

(26)

4. Experiments and Results

In this section, we first provide details of the experimental setup and evaluation procedures, and then present the results of the experiments conducted to validate the effectiveness of the proposed method.

4.1. Training and Assessment Settings

In the BCI IV 2a dataset, the data from all nine subjects were randomly divided into a training set (80%) and a validation set (20%). In the NinaPro DB2 dataset, experiments were conducted and the performance of the proposed model evaluated by partitioning each subject’s data into a training set (75%) and a validation set (25%). For both datasets, the model was trained using the commonly adopted cross-entropy loss function and the Adam optimizer with a learning rate of 0.001. The maximum number of epochs was set to 200 with a minimum batch size of 256, and an early stopping mechanism was employed to prevent overfitting when the validation performance ceased to improve significantly.

The cross-entropy loss used in this work is given by:

$$L_{CE}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\sum _{c=1}^C{y}_{ic}log\left({\widehat{y}}_{ic}\right)$$

(27)

where N is the number of samples, C is the number of classes, $y_{ic}$ represents the ground truth indicator (1 if class c is the correct label for sample i, 0 otherwise), and ${\widehat{y}}_{ic}$ denotes the predicted probability for class c of sample i.

To assess the performance of the proposed model, the evaluation metrics used include recall, accuracy, precision, and F1 score. In these metrics, TP (True Positive) represents the number of positive examples correctly identified by the model, TN (True Negative) denotes the number of negative examples correctly classified as negative, FP (False Positive) indicates the number of negative examples incorrectly classified as positive, and FN (False Negative) refers to the number of positive examples misclassified as negative:

$$Recall={\displaystyle \frac{TP}{TP+FN}}\times 100\%$$

(28)

$$Accuracy={\displaystyle \frac{TN+TP}{TN+TP+FN+FP}}\times 100\%$$

(29)

$$Precision={\displaystyle \frac{TP}{TP+FP}}\times 100\%$$

(30)

$$F1={\displaystyle \frac{2(recall\times precision)}{recall+precision}}$$

(31)

But for some prior methods, the original code or detailed implementation specifications were not publicly available to guarantee faithful reproduction. Therefore, for these methods we only report the published Accuracy values, and mark Recall, Precision, and F1 as “–” to indicate “not reported or unable to reproduce under identical settings.”

4.2. Ablation Study

To evaluate the contribution of each architectural component in SCGTNet, ablation experiments were carried out using BCI IV-2a as the primary benchmark and NinaPro DB2 for secondary validation. On BCI IV-2a, we systematically omitted the Adaptive Spatial Attention (ASA) block and the Multi-Scale Convolution (MSC) block in isolation. In addition, the MSC block was replaced with a conventional $3\times 1$ convolution to assess the advantage of the multi-scale design. The same protocol was subsequently applied to the NinaPro DB2 dataset to confirm the generalizability of these findings.

Table 1. Ablation study results for SCGTNet on the BCI IV-2a dataset. These results highlight the contribution of each component to overall performance.

Configuration	Accuracy	Recall	Precision	F1-Score
Without ASA	73.33%	72.45%	74.31%	73.36%
Without MSC	74.12%	76.47%	71.26%	73.77%
$3\times 1$ CNN	76.04%	73.26%	74.63%	73.94%
Full model	79.47%	79.33%	80.96%	78.13%

and Figure 4 summarize the results on the BCI IV-2a dataset. Removing the ASA block led to a 6.14 percentage point drop in overall accuracy and a 4.77 percentage point reduction in F1-score. That highlights the effectiveness of the model in modeling spatial correlations between channels. By focusing the network’s capacity on the most informative electrode pairs, ASA appears to enhance discriminability, especially in complex MI patterns. Exclusion of the MSC block produced a 5.35 percentage point decrease in accuracy and a 4.36 percentage point loss in F1, indicating that capturing temporal features at multiple resolutions is crucial for distinguishing overlapping event-related desynchronizations. When the MSC was replaced by a standard $3\times 1$ convolution, accuracy fell by 3.43 percentage points and F1 by 4.19 points. This suggests that while basic temporal filtering does help, the multi-scale design more effectively accommodates varying gesture durations and frequencies. These findings confirm that both ASA and MSC modules significantly enhance SCGTNet’s discriminative power, with ASA exerting the largest effect on this dataset.

Figure 5 presents the t-SNE embeddings of the learned feature vectors under each ablation setting on BCI IV-2a. In the full model (Figure 5a), clusters corresponding to LEFT, RIGHT, FEET, and TONGUE appear well separated, suggesting strong inter-class discrimination. Upon removing the ASA block (Figure 5b), the LEFT and RIGHT clusters show noticeable overlap, aligning with the decrease in F1 observed in Table 1. Similarly, the exclusion of the MSC block (Figure 5c) increases intra-class dispersion and reduces inter-class separation, which is also reflected in the lower F1. When the MSC is replaced by a standard $3\times 1$ convolution (Figure 5d), an intermediate level of overlap appears, consistent with its moderate F1 performance. These t-SNE visualizations are intended for qualitative inspection and, together with the quantitative results, highlight the roles of ASA and MSC in achieving robust and discriminative feature representations.

On the NinaPro DB2 dataset (

Table 2. Ablation study results for SCGTNet on the NinaPro DB2 dataset. These results highlight the contribution of each component to overall performance.

Configuration	Accuracy	Recall	Precision	F1-Score
Without ASA	83.28%	81.02%	84.90%	82.88%
Without MSC	81.72%	87.52%	85.78%	81.26%
$3\times 1$ CNN	81.15%	85.43%	86.46%	80.60%
Full model	85.87%	87.42%	83.19%	85.42%

, as shown in Figure 6), omission of the ASA block resulted in a 2.65 percentage point decline in accuracy and a 2.54 point drop in F1-score. Excluding the MSC block caused a 4.21 point reduction in accuracy and a 4.16 point decrease in F1. Substituting MSC with a $3\times 1$ convolution led to a further 4.78 point fall in accuracy and a 4.82 point loss in F1. These outcomes indicate that the comparatively smaller impact of ASA removal versus MSC removal implies that spatial attention is somewhat less critical for sEMG-based hand gesture recognition, whereas multi-scale temporal patterns remain key. These insights not only validate our architectural choices but also guide future refinements, e.g., exploring dynamic attention mechanisms or adaptive scale selection.

4.3. Hyperparameter Sensitivity Analysis

To investigate whether the fixed bandwidth parameter $f_b=2$ of the Morlet wavelet is optimal for extracting ERP features from the CSP-filtered EEG signal, a series of controlled experiments were conducted. Specifically, we varied the $f_b$ parameter within the range of $[1.5, 2.0, 2.5]$ and the obtained data were classified by the model under the same conditions to obtain the respective results.

As shown in

Table 3. Impact of Morlet Wavelet Bandwidth (fb) on BCI IV 2a classification performance. Accuracies, recalls, precisions, and F1-scores were measured under different bandwidth settings (fb = 1.5, 2.0, 2.5).

Number of fb	Accuracy	Recall	Precision	F1-Score
1.5	68.79%	67.04%	70.61%	68.74%
2	79.47%	79.33%	80.96%	78.13%
2.5	69.42%	68.04%	71.08%	69.48%

, the highest classification accuracy was achieved when $f_b=2$, confirming its suitability for our dataset and model configuration.

To evaluate the effect of different attention head counts on model performance, we varied $num\_heads\in \{1, 2, 4, 8\}$ for the single Multi-Head Attention layer. In each configuration, all other hyperparameters were kept identical. The models were trained and evaluated under the same protocol on the BCI IV 2a and Ninapro DB2 datasets. Results are summarized in

Table 4. Impact of multi-head attention head number on BCI IV 2a and Ninapro DB2 classification performance.

Dataset	num_heads	Accuracy	Recall	Precision	F1-Score
BCI IV 2a	1	74.41%	73.11%	75.16%	74.12%
BCI IV 2a	2	77.47%	78.24%	76.27%	77.24%
BCI IV 2a	4	79.47%	79.33%	76.96%	78.13%
BCI IV 2a	8	79.51%	80.09%	77.62%	78.84%
Ninapro DB2	1	80.23%	82.14%	79.16%	80.62%
Ninapro DB2	2	81.88%	83.37%	80.21%	81.80%
Ninapro DB2	4	85.87%	87.42%	83.19%	85.42%
Ninapro DB2	8	84.95%	86.02%	82.17%	84.05%

.

Table 4 shows that for BCI IV 2a, increasing head count from 1 to 4 yields consistent improvements, with a marginal further gain at 8 heads; however, on Ninapro DB2, performance peaks at 4 heads and decreases when using 8 heads. Therefore, $num\_heads=4$ is selected for the final model based solely on these performance results.

4.4. Comparison Study

To benchmark SCGTNet against existing methods, comparative experiments were conducted on BCI IV 2a (primary) and NinaPro DB2 (secondary) datasets. Classical CNN [44] and LSTM [45] architectures, as well as more advanced EEG models (EEG-Inception [25], MIN2Net [46], EEG-TCNet [47], TFT [30]) and sEMG methods (Wei et al. [48], Ding et al. [26], Zhang et al. [49], Hu et al. [50], TFT [30]) were included.

Table 5. Performance comparison of SCGTNet with other models on the BCI IV 2a dataset.

Model	Accuracy	Recall	Precision	F1-Score
CNN [44]	70.65%	70.31%	71.26%	70.78%
LSTM [45]	71.24%	74.33%	74.21%	74.27%
EEG-Inception [25]	74.76%	72.47%	71.69%	72.08%
MIN2Net [46]	75.87%	75.09%	74.16%	74.62%
EEG-TCNet [47]	75.53%	76.63%	75.31%	75.96%
TFT [30]	74.49%	75.54%	73.36%	74.43%
SCGT (Ours)	79.47%	79.33%	76.96%	78.13%

reports that SCGTNet achieved 79.47% accuracy, 79.33% recall, 76.96% precision, and 78.13% F1 score—surpassing the best baseline MIN2Net [46] at 3.60 pp and 3.51 pp, respectively. These gains reflect SCGTNet’s holistic design, which integrates graph-based channel relationships, multi-resolution temporal filtering, and end-to-end joint optimization—unlike prior methods that treat spatial filtering, temporal dynamics, or inter-channel dependencies in isolation. In contrast, inception-style CNNs separate spectral and temporal paths, and transformer-based approaches focus primarily on sequential dependencies; neither fully exploits the complementary information present across channels and scales. As a result, SCGTNet more effectively captures the complex spatiotemporal patterns characteristic of EEG motor imagery, leading to superior classification across all four classes.

Figure 7 presents the confusion matrix for SCGTNet on the BCI IV-2a dataset. It shows that all classes achieve high recognition rates, with the LEFT class reaching 81%, RIGHT 78%, FEET 75%, and TONGUE 79%. Misclassifications are predominantly between LEFT and RIGHT, indicating that residual overlap remains in lateralized patterns, but at a substantially lower rate than in baseline models. This uniform class-wise performance underscores SCGTNet’s balanced discriminative capability across motor imagery categories.

To assess the cross-subject generalization ability of SCGTNet, we further conducted leave-one-subject-out (LOSO) experiments on the BCI IV 2a dataset. Figure 8 shows the performance of SCGTNet and the baseline methods across nine subjects. SCGTNet achieved consistently higher accuracy and exhibited lower inter-subject variability compared to the other methods, indicating its robustness across subjects.

As shown in

Table 6. Performance comparison of SCGTNet with other models on the NinaPro DB2 dataset. Recall/Precision/F1 is marked ‘-’ for baselines that cannot be reproduced, indicating that the original article was not reported or that we were unable to reproduce it with the same settings.

Model	Accuracy	Recall	Precision	F1-Score
CNN [44]	70.28%	68.58%	72.03%	70.21%
LSTM [45]	73.13%	72.22%	74.57%	73.40%
Wei et al. [48]	83.7%	-	-	-
Ding et al. [26]	78.86%	-	-	-
Zhang et al. [49]	83.29%	-	-	-
Hu et al. [50]	84.84%	-	-	-
TFT [30]	83.64%	83.85%	80.06%	82.40%
SCGT (Ours)	85.87%	87.42%	83.19%	85.42%

, SCGTNet obtained 85.87% accuracy, 87.42% recall, 83.19% precision, and 85.42% F1-score—exceeding Wei et al. [48] (83.70%) and Hu et al. [50] (84.84%) by 2.17 pp and 1.03 pp in accuracy. The superior performance arises from SCGTNet’s unified framework that simultaneously models inter-electrode/muscle-site relations and captures both short-term transients and longer activations within a single network. Whereas traditional CNNs apply fixed-scale filters and LSTMs process sequential features without explicit channel interactions, and recent transformer- or inception-based sEMG methods decouple these concerns, SCGTNet’s end-to-end architecture learns to leverage all these aspects jointly. This design yields more robust gesture discrimination across diverse muscle patterns and subjects, as evidenced by uniformly high per-class recognition rates (>80%) in the confusion matrix (Figure 9).

Figure 9 presents the full 49-class confusion matrix for SCGTNet on NinaPro DB2. Despite the large gesture vocabulary, SCGTNet maintains an average per-class recognition rate above 80%, with the majority of gestures exceeding 85% accuracy. Remaining misclassifications occur mainly among anatomically or functionally similar movements (e.g., individual finger flexions), indicating that residual overlap persists in fine-grained muscle activations. Overall, the balanced class-wise performance across 49 gestures confirms SCGTNet’s robustness in large-scale sEMG classification tasks.

5. Discussion

The ablation experiments highlight the distinct contributions of the adaptive spatial attention and multi-scale temporal convolution modules. Notably, the multi-scale temporal convolution module yields greater improvements on NinaPro DB2, while the adaptive spatial attention module is more effective on BCI IV 2a. These results suggest that different biosignal datasets benefit from different model components, but overall, both modules enhance feature extraction and boost classification performance.

Compared to traditional CNN-, LSTM-, and Transformer-based models, SCGTNet achieves consistently higher accuracy on both datasets. This underscores its strength in capturing spatio-temporal patterns in fine-grained EEG and sEMG signals. Due to the lack of publicly available source code for some prior methods, certain comparisons are limited to published accuracies; we acknowledge that such comparisons may be influenced by differences in experimental setups and reporting standards.

Additionally,

Table 7. Total and trainable parameter counts and floating-point operations (FLOPs) of SCGTNet and comparison models on the BCI IV 2a dataset.

Model	Total Params	Trainable Params	No-Trainable Params	FLOPs
CNN [44]	11.5k	11.3k	0.2k	0.83M
LSTM [45]	18k	16.6k	1.4k	0.56M
TFT [30]	220k	217.6k	2.4k	3.50M
SCGT (Ours)	56k	55.4k	0.6k	1.12M

and

Table 8. Total and trainable parameter counts and floating-point operations (FLOPs) of SCGTNet and comparison models on the NinaPro DB2 Dataset.

Model	Total Params	Trainable Params	No-Trainable Params	FLOPs
CNN [44]	9.8k	9.6k	0.2k	1.25M
LSTM [45]	14.5k	14.1k	0.4k	2.87M
TFT [30]	180k	180k	1.8k	52M
SCGT (Ours)	48k	47.4k	0.6k	12M

show that SCGTNet requires fewer parameters and computations than traditional Transformer-based models, indicating its potential for deployment on resource-limited hardware. Its lightweight design and competitive accuracy make it a promising candidate for practical HMI scenarios such as rehabilitation training, wearable devices, and assistive control.

Despite these promising results, further work is needed to evaluate the model’s generalizability across diverse biosignals, including ECG, eye tracking, and particularly EEG–EMG fusion, as well as its scalability to longer or higher-frequency inputs. Future research will focus on rigorous real-time inference benchmarking under controlled hardware conditions, as well as hardware-aware optimization techniques (e.g., pruning, quantization) to bridge the gap between model design and efficient low-latency implementation. Integrating transfer learning, reinforcement learning, or other advanced techniques may also help extend its application scope and enhance adaptability.

6. Conclusions

In summary, this paper proposes SCGTNet, a lightweight multiscale spatio-temporal fusion Transformer for EEG-based motor imagery classification and sEMG-based gesture recognition. Experimental results on BCI IV 2a and NinaPro DB2 show that SCGTNet achieves state-of-the-art performance (79.47% and 85.87%, respectively), outperforming traditional CNN-, LSTM-, and Transformer-based methods. Future work will explore further optimization to enable its practical deployment in real-time human–machine interface scenarios.