Improved Graph Convolutional Network with Enriched Graph Topology Representation for Skeleton-Based Action Recognition

Abstract

Lately, skeleton-based action recognition has drawn remarkable attention to graph convolutional networks (GCNs). Recent methods have focused on graph learning because graph topology is the key to GCNs. We propose to align graph learning on the channel level by introducing graph convolution with enriched topology based on careful channel-wise correlations, namely the attentive channel-wise correlation graph convolution (ACC-GC). For the model to learn channel-wise enriched topologies, ACC-GC learns a shared graph topology spanning many channels and enhances it with careful channel-wise correlations. Encoding the intra-correlation between various nodes within each channel, boosting informative channel-wise correlations, and suppressing trivial ones generates attentive channel-wise correlations. Our enhanced ACC-GCN is created by substituting our ACC-GC for the GC in a standard GCN. Extensive experiments on NTURGB60 and Northwestern-UCLA datasets demonstrate that our proposed ACC-GCN performs comparably to state-of-the-art methods while reducing the computational cost.

1. Introduction

Action recognition, which has a wide range of applications in various areas, including video surveillance [1], human–robot interaction [2,3], and medical services [4], is one of the most alluring areas of computer vision. Due to several factors, such as (i) the vast variations in the visual and motion appearance of people and actions, (ii) the variations within classes of the same actions, and (iii) the similarities between classes of different actions, identifying the precise type of human motion in videos is particularly challenging.

Recent skeleton-based action recognition models mainly learn human action representations by exploiting deep learning networks [5,6,7,8,9]. Among the deep models, the graph convolutional network (GCN) has shown superior performance over handcrafted features, recurrent neural networks (RNNs) [10,11], and convolution neural networks (CNNs) [12,13,14]. The key factor contributing to the supremacy of GCN is its ability to depict intricate interrelationships and dependencies between nodes (joints) in the skeletal graph [15]. Graph topology, represented by the static adjacency matrix $A\in {\mathbb{R}}^{NxN}$ is the key to GCN. N represents the number of nodes (joints), and A describes the representation of physical connections between joints in the skeleton graph. For any two joints i, j in A, if the two joints are physically connected, then $A_{ij}$ = 1; otherwise, $A_{ij}$ = 0.

Recent GCN-based methods use an unfavorable static graph topology across all layers and input samples for skeleton-based action recognition. For instance, it impacts the modeling of the multilayered semantic information represented. Various tasks require cooperation between non-connected joints, which is missing from the static graph. Clapping and playing the piano depends on the coordination of the right- and left-hand joints, which are physically unconnected. Additionally, it is preferable for different samples connected to various action classes to have various topologies. Therefore, to improve the representative capability of GCN, a dependency link between non-connected joints must be drawn in a graph for each sample [16].

Due to their greater generalization capacity, creating dynamic graphs to get around the shortcomings of static graphs is drawing a lot of interest. There have been several attempts to use dynamic graphs for skeleton-based action recognition. For instance, the context-encoding network (CeN), a dynamic GCN, was presented to automatically learn the skeleton topology [14]. Along with learning contextual characteristics from the remaining joints, which are combined and integrated globally, CeN also learns the dependency between any two joints. As a result, several graphs are built for various input samples and graph convolutional layers with varying depths. In the same context, several approaches [12,13,17,18] learn different graphs for different samples by utilizing non-local operations [19]. Even though these approaches enhance the expressiveness power of the model, they are considered dynamic-shared graph topology approaches since the same topology is shared among different channels. Different channels capture different kinds of motion data. As a result, the correlations between joints in various channels are varied, which prevents the model from capturing more insightful features when applying the same topology to all channels.

Rarely are topology non-shared GCs-based methods for skeleton-based action recognition investigated. Among few works of non-shared graph topology are [20,21]. In [20], the authors import the idea of decoupling aggregation from CNN to GCN to increase the expressiveness of spatial aggregation in GCN. Specifically, they divide channels into (g) decoupling sets, with a trainable adjacent matrix for each set. Moreover, different graph topologies are learned via the channel-wise topology refinement graph, which aggregates features in various channels in [21].

In this paper, motivated by finding that each feature channel focuses on a specific pattern, we aim to align the graph learning on the channel level. We specifically suggest a graph convolution with an enriched graph topological structure for skeleton-based action recognition. A careful channel-wise enriched topology-based graph convolution is what we specifically suggest (ACC-GC). Our ACC-GC is designed to learn a shared graph topology over a range of channels and enhance it using careful channel-wise correlations, allowing the model to learn channel-wise enriched topologies. Each channel’s intra-correlation between several joints is encoded by ACC-GC, enhancing and suppressing useful channel-wise correlations. In this manner, the channel attention and the intra-correlation are tied in a single structure. While channel attention can improve the features from the standpoint of channels during the feature extraction stage, intra-correlation modeling enables the model to employ channel-wise information to update the shared graph, which is a parametrized adjacency matrix. The networks are compelled to find and improve the informative channel-wise features that collect distinct information due to dynamically learning various topologies for various channels. The principal contributions of this paper are as follows:

• This paper proposes aligning the graph learning on the channel level by introducing a graph convolution with an enriched topology based on attentive channel-wise correlations (ACC-GCs). By integrating ACC-GC in GCN, we obtain an enhanced GCN with an enriched topology representation (ACC-GCN) for skeleton-based action recognition.
• This paper explores the advantage of integrating our ACC-GC configuration over dynamic topology non-shared GCN-based models.
• By performing extensive experiments, we demonstrate the effectiveness of our proposed ACC-GCN using two large-scale human action recognition datasets, mainly the NTURGB60 and Northwestern-UCLA datasets.

The rest of this paper is organized as the following: Section 2 shows the most related work to the proposed work. The ACC-GCN method is presented in Section 3, and Section 4 illustrates the experimental setting. Finally, this work is concluded in Section 5.

2. Related Work

This section introduces the prominent works in the skeleton-based action recognition and the attention aspect. In detail, this section is divided into four subsections. First, we introduce deep neural networks (DNNs) based approaches for skeleton-based action recognition, followed by the graph convolutional network (GCN) based approaches, Transformer-based approaches, and finally, the attention to deep learning.

2.1. DNN-Based Approaches

Skeleton-based action recognition has received remarkable attention recently. Recent approaches which utilize deep neural networks (DNNs) [5,6] have shown better performance compared with traditional approaches that utilize hand-crafted features [22,23,24]. Deep neural networks models include the recurrent neural network, such as LSTM [25] and GRU [26], convolutional neural network (CNN) and graph neural network (GCN). RNN-based approaches are used to model the temporal dynamics of skeleton sequence by aggregating temporal information sequentially [10,27]. In contrast, CNN-based approaches model spatiotemporal information jointly and use a pseudo-image to model the skeleton data, where the three coordinates of each skeleton joint (x, y, z) are seen as three channels of the image [28,29,30]. These attempts to apply CNN to action recognition have certainly shown success. However, the loss of skeleton topological information occurs when the irregularly structured skeleton data are transformed into regularly structured images.

2.2. GCN-Based Models

GCNs are a family of DNNs designed to handle graphs with erratic structures, such as skeletal data. Recent GCN-based approaches [31,32,33], which have shown outstanding performance for skeleton-based action recognition, are those that model skeleton data as graphs. For instance, authors in [34] argued that the recent GCN-based models have paid less attention to embedding the intricacies of human behavior information into the latent representations of human action. Hence, a GCN-based model, namely InfoGCN, was proposed to fill this gap in the literature, where a learning framework combines a novel learning objective and an encoding method for skeleton-based action recognition. The authors built an information bottleneck-based learning objective to direct the model to learn informative but condensed latent representations.

Depending on whether the graph topology is dynamically adjusted during inference, GCN-based approaches are either static-based approaches or dynamic-based approaches. A fixed graph topology based on the physical connections of the human body is defined in static-based approaches, as proposed in [35]. In contrast, the topology is dynamically adjusted during inference with the help of the non-local operation in the dynamic-based approaches [13,18]. Each one of the previously mentioned categories can be shown as either topology-shared or topology non-shared approaches in Figure 1. In topology-shared approaches, the graph topology is shared among different channels, while in topology non-shared approaches, the topology is different in each channel.

2.3. Transformer-Based Methods

The joints of the skeleton sequence are modeled using the Transformer-based approaches by using the idea of self-attention [36]. Most techniques typically calculate the correlation of joints in the space and time domains to reduce the computational cost. For instance, the spatio-temporal cross (STAR)-Transformer [37] is a Transformer-based model that encodes two cross-modal features as a distinctive vector. By combining the input video and skeleton sequence, global grid tokens are output as video frames, and skeleton sequences are generated as joint map tokens.

2.4. Attention in Deep Learning

Attention, a complex cognitive function, has undoubtedly become one of the deep learning field’s most semantic ideas. It is inspired by human biological systems, which tend to highlight distinguishing features when dealing with large amounts of data. On an abstract level, humans tend to selectively focus on a subset of information when and where it is required while ignoring other information. For example, when viewing a scene, humans typically do not pay attention to all of the details. Instead, they concentrate on certain parts of the scenes as needed. Recently, various computer vision applications have extensively utilized different attention mechanisms [38,39,40,41,42,43]. Channel attention is a type of attention mechanism that has attracted remarkable attention in recent years. It allows the networks to learn what should be focused on. Squeeze-and-excitation (SE) [44] network is a channel-attention network that adaptively re-calibrates channel-wise feature responses by explicitly modeling inter-dependencies between channels. SE adds a content-aware mechanism to weight each channel adaptively. Selective kernel (SK) [45] utilizes many branches with different kernel sizes, which are then fused via Softmax to enhance the effectiveness of object recognition. Furthermore, a channel hard attention mechanism is proposed in [46]. By enabling the network to focus on its filters repeatedly, it uses the capability of sequential processing to improve classification performance.

3. Proposed Method

In the context of skeleton-based action recognition, we first review the spatial graph convolution in standard GCN in this section (Section 3.1). Then, we show the details of our enhanced graph convolution (Section 3.2). Finally, the overall model is introduced in Section 3.3.

3.1. GCN for Action Recognition Using Skeletons

The graph is initially depicted by vertices and edges as follows: G = (V, E, A). In the skeleton graph, the graph is composed of V joints that are connected using E edges. The adjacency matrix A encodes joint connections. The graph convolutional network (GCN) consists of several spatial graph convolutional blocks (SGC-blocks) and temporal convolutional blocks (TGC-blocks) that are both responsible for explicitly exploring both the spatial and temporal correlations between human joints. Specifically, SGC blocks can be formulated as follows:

$$Y=\sum _k^{k_v}{{\mathsf{\Lambda }}_k}^{-\frac{1}{2}}{A}_k{{\mathsf{\Lambda }}_k}^{-\frac{1}{2}}XW$$

(1)

The input feature maps and learnable kernels are denoted by X and W, respectively. According to [35], k represents the number of spatial configurations. A represents the adjacency matrix, while ${\mathsf{\Lambda }}_k$ is the diagonal degree matrix of A. The temporal convolutional blocks are traditional convolutional layers with a kernel size k of t × 1. Learning spatiotemporal features requires constructing a GCN that alternately stacks SGC-blocks and TGC-blocks.

3.2. Attention-Based Correlation-Driven Graph Convolution (ACC-GC)

Our attentive channel-wise enriched topology-based graph convolution (ACC-GC) takes $X\in {\mathbb{R}}^{CxTxV}$ as an input, where the symbols C, T, and V denote the number of channels, the number of frames and the number of joints, respectively. ACC-GC consists of four parts, mainly graph modeling, channel-wise enhancement, topology augmentation, and feature aggregation, as illustrated in Figure 2.

The concept behind our suggested architecture is to obtain various topologies in each channel to gather discriminative data crucial for overall human motion recognition. This can be done via several steps. First, we learn the intra-correlations between different joints within each channel to dynamically obtain channel-wise correlations. Then, we enhance informative channel-wise correlations and suppress trivial ones in the channel enhancement stage. The attentive channel-wise correlations from the enhancement stage are used to update the learned shared topology in the topology augmentation stage. The shared topology serves as a generalized correlation between joints and is essentially a parameterized adjacency matrix shared by all channels. We aggregate the features in each channel with the relevant topology after getting several graph topologies for each channel to produce the final result. The following is a detailed explanation of each part separately, along with Figure 3.

Graph modeling IIn this stage, we utilize two 1 × 1 2D convolution layers, mainly $\varphi$, and $\psi$, to reduce feature channels for efficiency as illustrated in Equations (2) and (3):

$$x_{r1}=\varphi \ast X,x\in {\mathbb{R}}^{C/rXTXV}$$

(2)

$$x_{r2}=\psi \ast X,x\in {\mathbb{R}}^{C/rXTXV}$$

(3)

Both $x_{r1}$ and $x_{r2}$ denote the channel-reduced features. (r) is the reduction ratio. In our experiments, we utilize r = 8 and 16.

Next, we utilize a global average pooling layer to compact the temporal information, as our goal is to model spatial features, and the temporal layouts are insignificant at this stage:

$$S_1=Pool\left(x_{r1}\right),S_2=Pool\left(x_{r2}\right)$$

(4)

where $S_1$ and $S_2$ are the tensors summarized over the temporal dimension. Then, we simultaneously calculate the correlations between pairs of joints in each channel. Given a pair of joints ($v_i$, $v_j$) and their corresponding features ($x_i$, $x_j$), the correlation $c(.)$ is represented by:

$$c(S_{1i},S_{2j})=\sigma (f(S_{1i},S_{2j}))$$

(5)

where f is a correlation function, and $\sigma$ is a nonlinear function. The experiments utilize two modeling methods (matrix multiplication and element-wise subtraction) as correlation functions and sigmoid and Tanh as non-linear functions. $C(.)$ now holds a non-linear transformation of these correlations as the channel-specific topological relationship between $v_i$ and $v_j$.

Channel Enhancement: In this stage, our GC is required to recognize and enhance the discriminative channel-wise correlations that capture specific information at this step that capture differentiated information. To achieve that, we generate channel-wise statistics using global average pooling (GAP) on the spatial dimension. Formally, a statistic $e\in {\mathbb{R}}^C$ is generated by shrinking the correlated tensor $c(.)$ through its spatial dimensions:

$$e=GAP\left(c\right(x\left)\right)$$

(6)

Then we utilize transformation on the features, obtaining transformed features to calculate the attentive weight vector, which is obtained by using the sigmoid function and a 1 × 1 convolution (conv):

$$A=\sigma (conv\ast e),A\in {\mathbb{R}}^{C/rx1x1}$$

(7)

To excite different channel-wise correlations, we perform channel-wise multiplication between the attentive weights (A) and the feature map (c) to re-weight c(.), obtaining the excited channel-wise correlations (z) tensor:

$$z=c+c·AA\in {\mathbb{R}}^{C/rxVxV}$$

(8)

Topology augmentation: Now, we conduct 1 × 1 conv to retain the original channel dimension. Next, we augment the parameterized graph (G) with the excited channel-wise correlations (z) resulting from the channel enhancement stage using the concatenation operation (Cat) as illustrated in Figure 3:

$$T=Concat(G,conv(z\left)\right)$$

(9)

where T denotes the channel-wise topologies.

Feature aggregation: Feature aggregation takes the channel-wise topologies $\left(T\right)$ and the transformed feature map $x_{trans}$ as inputs. Channel-wise topology T is the result of the topology augmentation stage, and $x_{trans}$ is obtained using a transformation function. We conduct a 1 × 1 conv $\left(\zeta \right)$ as a transformation function in order to transform the input feature into a high-level representation. We utilize $x_{trans}$ and $\left(T\right)$ to obtain a channel graph for each channel by aggregating features channel-wise. We implement the aggregation using batch matrix multiplication to obtain the final output feature.

3.3. Our Enhanced Graph Convolution Network

The proposed graph convolutional network comprises spatial-GC, represented by ACC-GC and temporal-GC stacked together in one unit. Our final model is composed of 10 units, and the numbers of the output channels for the ten units are (64, 64, 64, 64, 128, 128, 128, 256, 256, and 256). Our temporal-GC follows [13] in the architecture. However, only some branches are used for efficiency.

4. Experimental Settings

4.1. Datasets

Two benchmarks are used to evaluate the proposed approach: NTURGB60 [47], and Northwestern-UCLA [48].

NTURGB60: NTURGB is composed of 56,880 data samples conducted by 40 different subjects. Each skeleton frame consists of (x, y, z) coordinates of 25 joint points. Samples are captured using three Microsoft Kinect v2 cameras from different views. This work proposes two settings: Cross-Subject (X-Sub) and Cross-View (X-View). In Cross-Subject, the 40 subjects are divided into training and test sets. The training set includes 40,320 samples, while the testing set includes 16,560 samples. In Cross-View, the samples captured by camera one are dedicated to the testing set, and the samples that are captured using cameras 2 and 3 are dedicated to the training set. The training set includes 37,920 samples in cross-view, while the testing set includes 18,960 samples.

Northwestern-UCLA: This dataset consists of 1494 video clips from ten different classes. A different subject carries out each action. We use the same protocol as [20], with training data from the first two cameras and validation samples from the third camera.

4.2. Implementation Details

The experiments are conducted using the Pytorch platform [49] on a single RTX 2080 TI GPU. The model is trained for 70 epochs with an initial learning rate (lr) of 0.1, which decays by a factor of 0.1 at epochs 34 and 54. In detail, lr is set to 0.01 at epoch 34 and 0.001 from epoch 54 to the last epoch. As our classification loss function, we use the cross-entropy loss function, and stochastic gradient descent with momentum (0.9) is our optimization approach. For stability issues, we adopt a warmup strategy [50] in the first five epochs. We set the batch size to 64. For prepossessing, we follow [12].

4.3. Ablation Study

To highlight the effectiveness of our proposed ACC-GCN model, we conducted extensive experiments on the NTU60 dataset. First, we tested our model with different implementations to verify the effectiveness of each component of the model, as illustrated in Section 4.3.1. Moreover, we built our model with varying stream structures, mainly: the joint stream, the bone stream, the joint-motion, and bone-motion streams. Then, we calculated the accuracy of each model separately, as illustrated in Section 4.3.2.

4.3.1. Effect of the Model Components

Different channels frequently concentrate on various data patterns. We constructed five neural networks with varying configurations and run several experiments on the NTU60 to show how well attention-based channel-wise correlations can be used to construct channel-wise graph structures. C(.) and E(.) denote the type of correlation function (either matrix multiplication or element-wise subtraction) and whether the enhancement stage is conducted or not. Act(.) denotes the type of activation function.

We compared the results with our baseline, a shared graph convolutional-based topology structure [5]. For a fair comparison, we integrated the temporal-GC described in Section 3.3.

Table 1. Comparisons of the validation accuracy of ACC-GC with different configurations.

Model	C(.)	E(.)	Act(.)	Acc (%)	Param
Baseline	-	-	-	88.1	1.21 M
A	matrix multiplication	Yes	Tanh	89.3	1.55 M
B	element-wise sub	No	Tanh	89.5	1.46 M
C	element-wise sub	Yes	Sigmoid	89.4	1.55 M
D	element-wise sub	Yes	Tanh	89.7	1.55 M
E	element-wise sub	Yes	Relu	89.6	1.55 M

shows the performance gains achieved by our ACC-GCN with the cross-subject benchmark on the NTU RGB+D dataset. We first evaluated the strong baseline model. As mentioned in Table 1, our best ACC-GCN improves the accuracy over the baseline by 1.6 with fewer extra parameters. Further, the best result is obtained when using element-wise subtraction as a correlation function, conducting enhancement, and using Tanh as an activation function.

4.3.2. Effect of Multi-Stream Structure

Skeleton-based action recognition is one of the fields that benefit from the multi-stream structure [18,51,52], showing significant improvement in the recognition accuracy. In this section, we evaluate the importance of the multi-stream structure in our model. We introduce our model performance using four streams, mainly joint, bone, joint motion, and bone motion, following [51]. The same model is used for all the streams, and the final prediction is generated by fusing the weighted sum scores of the four streams, as shown in Figure 4.

In

Table 2. Comparisons of the validation accuracy of ACC-GCN with different modalities.

Modality	Top-1 (%)
Joint (J)	89.70
Joint-motion (JM)	87.87
Bone (B)	90.30
Bone-motion (BM)	87.45
J + B + JM + BM	92.0

, we present the result of the multi-stream structure using the best configuration from Table 1, which is model (D). As mentioned in Table 2, the contribution of each stream varies. Hence, we utilize a weighted sum to fuse the result of the four streams and obtain the final result. For example, assigning a high weight to the bone motion stream will result in performance loss because its contribution to the overall multi-stream structure is smaller than that of every joint and bone. We need to slightly reduce the weight of the bone motion stream to achieve better performance.

4.4. Visualize Learned Topology

The learned topologies on the NTU RGB+D dataset are shown in Figure 5. The graphs show that our augmentation strategy improves the common topology, resulting in more distinct features. Furthermore, the variety of refined channel-wise topologies demonstrates the capability of our method to learn unique topologies based on different motion data for different channels.

4.5. Improving over GCN-Based Models

This section explores the advantage of integrating the ACC-GC configuration over dynamic topology non-shared GCN-based models. In detail, we replaced the graph convolution (GC) in two graph-based action recognition models, mainly ST-GCN [35], and 2s-AGCN [18] with our ACC-GC configuration.

As shown in

Table 3. Comparisons of the validation accuracy of ACC-GC with different configurations.

Model	GFLOPs	Acc (%)	Original Accuracy
ST-GCN [35]	16.3	82.5	81.5
2s-AGCN [18]	37.3	89.6	88.5
Ours	1.9	92.0	-

, the accuracies increase in both recognition models by 1% and 1.1% in ST-GCN [35] and 2s-AGCN [18], respectively, when replacing the GCs by our ACC-GC configuration, which validates the effectiveness of the ACC-GC configuration.

4.6. Comparison with the State of the Art

On the NTURGB dataset, we contrast our ACC-GCN model with various cutting-edge baselines, as shown in

Table 4. Comparison of the validation accuracy with the state-of-the-art methods on the NTU-RGBD dataset. The best results are in blue, and the second best is in red.

Model	x-Sub (%)	x-View (%)	GFLOPs
Lie Group [53]	50.1	52.8	-
H-RNN [11]	59.1	64.0	-
PA-LSTM [47]	62.9	70.3	-
ST-LSTM+TS [3]	69.2	77.7	-
STA-LSTM [54]	73.4	81.2	-
Visualize CNN [55]	76.0	82.6	-
C-CNN+MTLN [29]	79.6	84.8	-
VA-LSTM [10]	79.2	87.7	-
ST-GCN [35]	81.5	88.3	16.3
DPRL [56]	83.5	89.8	-
AS-GCN [57]	86.8	94.2	-
DC-GCN+ADG [20]	90.8	96.6	25.7
4s-ShiftGCN [58]	90.7	96.5	10.0
2s-AGCN [18]	88.5	95.1	37.3
PA-ResGCN [59]	90.9	96.0	18.5
RA-GCN [60]	87.3	93.6	32.8
MS-G3D [13]	91.5	96.2	48.8
PSUMNet [61]	92.9	96.7	2.7
Our ACC-GCN	92.0	96.5	1.93

. We compare the data from UCLA and Northwestern in

Table 5. Comparison of the validation accuracy with the state-of-the-art methods on the Northwestern-UCLA dataset.

Model	Northwestern-UCLA Top-1 (%)	GFLOPs
Lie Group [62]	72.2	-
Actionlet ensemble [63]	76.0	-
H-RNN [11]	78.5	-
Ensemble TS-LSTM [64]	89.2	-
AGC-LSTM [9]	93.3	-
Shift-GCN [58]	94.6	-
2s AGC-LSTM	93.3	10.9
CTR-GCN	96.5	1.97
ACC-GCN (ours)	96.1	1.93

. On the NTU-RGB dataset, our model obtains top-1 accuracy of 92% and 96.5% in the x-sub and x-view settings, respectively. It achieves 96.1% on the Northwestern-UCLA dataset. Our results on both datasets exceed almost all the previous methods. In contrast, it is on par with the best method. Regarding the model complexity, our model is the less complex model with GFLOPs of 1.93G, while the state-of-the-art method utilizes 2.9G, which makes our model outperform the state-of-the-art in terms of complexity, resulting in a faster model. Similarly, on the Northwestern-UCLA dataset, our ACC-GCN outperforms the other methods regarding model complexity. Notably, our method is the first to apply channel enhancement alongside the dynamic modeling of the graph topology, which shows a strong learning ability according to our experiments, demonstrating the effectiveness of our approach.

5. Conclusions

The proposed ACC-GCN model can obtain a useful, informative feature representation to improve model accuracy. ACC-GCN aligns graph learning on the channel level by introducing a graph convolution with enriched topology based on careful channel-wise correlations. For the model to learn channel-wise enriched topologies, ACC-GC learns a shared graph topology spanning many channels and enhances it with detailed channel-wise correlations. Encoding the intra-correlation between various nodes within each channel, boosting informative channel-wise correlations and suppressing trivial ones generates attentive channel-wise correlations. As a result, the model is compelled to find and improve the instructive channel-wise characteristics that capture differentiated information. Our techniques also examined several graph topology representation configurations, which will serve as a roadmap for topology learning. Its state-of-the-art performance on two datasets demonstrates the suggested model’s efficacy. Finally, we believe that our idea and results open interesting new perspectives to design efficient and effective human action recognition models. However, we believe that the model complexity of GCN-based models, including ours, is somehow high, which would guide us to utilize Transformers in future work.