Dual-Channel Edge-Featured Graph Attention Networks for Aspect-Based Sentiment Analysis

Abstract

The goal of aspect-based sentiment analysis (ABSA) is to identify the sentiment polarity of specific aspects in a context. Recently, graph neural networks have employed dependent tree syntactic information to assess the link between aspects and contextual words; nevertheless, most of this research has neglected phrases that are insensitive to syntactic analysis and the effect between various aspects in a sentence. In this paper, we propose a dual-channel edge-featured graph attention networks model (AS-EGAT), which builds an aspect syntactic graph by enhancing the contextual syntactic dependency representation of key aspect words and the mutual affective relationship between various aspects in the context and builds a semantic graph through the self-attention mechanism. We use the edge features as a significant factor to determine the weight coefficient of the attention mechanism to efficiently mine the edge features of the graph attention networks model (GAT). As a result, the model can connect important sentiment features of related aspects when dealing with aspects that lack obvious sentiment expressions, pay close attention to important word aspects when dealing with multiple-word aspects, and extract sentiment features from sentences that are not sensitive to syntactic dependency trees by looking at semantic features. Experimental results show that our proposed AS-EGAT model is superior to the current state-of-the-art baselines. Compared with the baseline models of LAP14, REST15, REST16, MAMS, T-shirt, and Television datasets, the accuracy of our AS-EGAT model increased by 0.76%, 0.29%, 0.05%, 0.15%, 0.22%, and 0.38%, respectively. The macro-f1 score increased by 1.16%, 1.16%, 1.23%, 0.37%, 0.53%, and 1.93% respectively.

1. Introduction

Aspect-based sentiment analysis (ABSA) is an important task in natural language processing (NLP). ABSA is a fine-grained sentiment analysis task that is used to determine the sentiment polarity of individual components of a text (such as positive, negative, or neutral). As illustrated in Figure 1, the sentiment polarity of the aspect “food” is positive, but the sentiment polarity of the aspect “services” and “environment” is negative. As a result, as compared to traditional text sentiment analysis, ABSA can mine users’ more delicate sentiment expressions, analyze users’ distinct sentiment viewpoints on diverse goals, and provide decision-makers with more precise decision help.

Figure 1. Examples of ABSA tasks. Each aspect is classified into a corresponding sentiment polarity.

On the ABSA task, the early is mainly based on artificial feature engineering, artificial design, and feature selection, such as sentiment dictionary, dependency information, etc., using the maximum entropy, support vector machine, and other traditional classifiers for sentiment classification. Although good performance has been achieved, the shortcomings of poor generalization ability and high labor costs are gradually exposed.

In recent years, due to the rise of neural networks, convolutional neural network (CNN) and recurrent neural network (RNN) have been used to solve problems in ABSA. For instance, FAN [1] proposed CMN; Li [2] proposed TNet; Xue [3] proposed the GCAE model. Huang [4] proposed PF-CNN and PG-CNN. The TD-LSTM and TC-LSTM models were proposed by Tang [5]. The GRNN model was suggested by Zhang [6]. The H-LSTM model was proposed by Ruder [7].

Many classification tasks, including ABSA, have been completed using the pre-trained language model BERT [8]. As an illustration, Xu [9] utilized a different corpus to post-train BERT and showed how well it performed aspect extraction and ABSA. By creating supplementary sentences, Sun [10] transformed ABSA into a sentence pair classification task.

With the development of the attention mechanism, the attention mechanism has been widely used in this task, as it can make the model focus on a given aspect, to make a more accurate judgment of sentiment polarity. The ATAE-LSTM model was suggested by Wang [11], the AB-LSTM model by Yang [12], and the BILSTM-ATT model by Liu [13]. The AOA model was proposed by Huang [14] and the IAN model by Ma [15].

Because of their intrinsic capacity in specific aspects and semantic alignment of contextual words, attention mechanisms and CNN are commonly utilized in aspect sentiment analysis. However, because these models lack the mechanism to account for relevant syntactic constraints and distant word dependencies, they may recognize grammatically irrelevant contextual words as signals to judgment sentiments. Zhang [16] presented the ASGCN model to overcome this problem; Hou [17] proposed the SA-GCN model, and Xiao [18] proposed the AEGCN model.

The syntactic structure of sentences is used by graph convolution networks (GCNs) [19] and graph attention networks (GATs) [20] based on dependency trees. This is because syntactic dependence can build the relationship between words in a sentence, but sentences with poor syntactic dependence sensitivity cannot gain appropriate sentiment qualities via the dependency tree. Furthermore, existing ABSA methods typically embed aspect information into sentence representation to learn relevant sentiment features of specific aspects, but this method makes obtaining valuable sentiment features for aspects lacking obvious sentiment features difficult.

We have two issues that need to be resolved, as was previously discussed. (1) How to extract valuable sentiment features from aspects without obvious sentiment expression; (2) How to extract precise Sentiment features from sentences with low syntactic sensitivity. In this paper, we provide a fresh approach to the issues raised above. For the first issue, we create the aspect parsing graph by enhancing the context-dependent encoding of important aspect words and utilizing the emotional reciprocity between various context-related aspects. To solve the second issue, we create the semantic graph by applying the self-attention mechanism to extract the semantic link between words.

Based on this, we present a dual-channel edge-featured graph attention networks model (AS-EGAT). We improve the GAT model’s original attention mechanism by making the edge features a significant factor in the calculation of the weight coefficients by the attention mechanism to efficiently mine the edge features of the GAT model, which uses aspect syntactic graphs and semantic graphs as adjacency matrices. The following is a summary of our paper’s significant contributions:

• We create aspect syntactic graphs, which are strengthened by the mutuality of various aspects in the context and particular aspect words, as well as semantic graphs, which use self-attention to determine the semantic connections between each word in a phrase;
• We propose a dual-channel edge-featured graph attention networks model (AS-EGAT), which learns aspect sentiment features by modeling aspect features and semantic features in the context and fully mining the edge features;
• We conduct extensive experiments on six benchmark datasets. The experimental results show the effectiveness of the AS-EGAT model in ABSA.

2. Related Work

Many people are now employing CNN to solve the ABSA task because it has lately developed rapidly. A convolutional memory network (CMN) that incorporates the attention mechanism and that can simultaneously capture single words and multi-word expressions in sentences was proposed by Fan [1]. After noting the deficiencies of the attention mechanism and the reasons why CNN is poor at categorizing tasks, Li [2] proposed a new classification model, TNet, to assist the CNN feature extractor to identify sentiment features more precisely. This model utilized an approach strategy and scaled the convolutional layer’s input by using the location correlation between words and aspect words. The earliest research employed LSTM and attention mechanisms to predict the sentiment polarity of sentences. According to Xue [3], this kind of model is extremely sophisticated and requires a lot of training time. Based on this, they suggest the GCAE, a convolutional neural network model with a gate mechanism that is more effective, whose gate units can operate independently, and whose computation can be easily parallelized during training. Convolutional neural network parameterized filter (PF-CNN) and parameterized gated Convolutional Neural network (PG-CNN) were proposed by Huang [4]. They included aspect word information into CNN by using the parameterized filter gate structure.

The majority of cutting-edge techniques in numerous ABSA problems are built on RNNs for modeling because of their strong sequence learning capabilities. To further develop the LSTM, Tang [5] presented the target-dependent long-term memory neural network (TD-LSTM) and the target-association-based long-term memory neural network (TC-LSTM). The model treats a given target as a feature for ABSA and links it to contextual features. To capture sentence syntax and semantics as well as the interaction between aspectual words and nearby contextual words via Bi-RNN, Zhang [6] created two gated neural network models (GRNNs) (bidirectional RNN). By more effectively propagating the gradient, gated neural networks have been demonstrated to lower the deviation of standard recurrent neural networks at the end of the sequence. Ruder proposed a comment hierarchy model (H-LSTM) [7] that uses hierarchical bidirectional long and short-term memory neural networks to learn intra-sentence and inter-sentence relationships. The model integrates word input into the bidirectional LSTM at the sentence level, and the forward and backward LSTM’s final states are connected by aspect embedding and are fed back into the bidirectional LSTM at the comment level.

The attention mechanism’s core goal is to select the information that is more critical to the current task objective from a large amount of information and then use the attention mechanism to obtain the description of a specific aspect category in the text, to make a more accurate polarity judgment of sentiment, which is consistent with the goal of aspect level sentiment classification. Wang [11] was the first to introduce an attention mechanism to this task, proposing an attention-based LSTM model using target embedding (ATAE-LSTM), where an attention vector is set for each target on top of an LSTM network, forcing the model to attend to the important parts of the sentence. This method was shown to be an effective way to strengthen the neural model to attend to the relevant parts of the sentence. Two attention-based bidirectional LSTM were also suggested by Yang [12] to enhance classification performance. Liu [13] expanded the attention model by differentiating between the left and right contexts of a particular target. Huang [14] proposed an attentional overattentive neural network (AOA) that jointly learns the representation of aspects and sentences through AOA modules and automatically focuses on the important parts of the sentences in a study based on LSTM with a joint approach to model aspects and sentences. Following that, some interactive attention mechanisms were used on ABSA tasks. Ma [15] proposed an interactive attention network (IAN) that considers both target attention and contextual attention, employing two attention networks to detect important words from the target description and important words from the entire context.

Because of their intrinsic capacity to match aspect-specific and contextual word meanings, attentional mechanisms and convolutional neural networks are commonly employed for ABSA tasks. However, because such models lack ways to account for important syntactic restrictions and remote word dependencies, they may identify grammar-independent contextual words as cues for determining aspectual sentiment wrongly. To solve this issue, Zhang [16] suggested an aspect-specific sentiment classification framework (ASGCN) built on the dependency tree of sentences employing syntactic information and word dependencies. Hou [17] stated that GCN models based only on dependency trees are prone to parsing mistakes, thus they combined a self-attentive sequence model with the GCN model to create a novel GCN model based on selective attention (SA-GCN). According to Xiao [18], most techniques cannot adequately capture context semantic information and lack tools to explain important syntactic restrictions and distant lexical dependencies. An attentional coding-based graph convolutional network model (AEGCN) was proposed to address these issues. Zhang and Sun [21] proposed employing graph convolutional networks (GCN) to learn node representations from dependency trees and utilize them in conjunction with other characteristics to classify sentiment. Huang and Carley [22] employed graph attention networks (GAT) to intentionally build word dependencies for similar goals.

The aforementioned GAT-based model does not, however, extract the semantic features and does not take into account the specific aspects and the sentiment relationships between the aspects while creating the adjacency matrix. We suggest a strategy that combines aspect and semantic features with edge feature GAT based on the benefits of GAT in the ABSA challenge. To improve GAT’s capacity to precisely capture semantic relationships, we introduce an edge feature.

3. Related Methods

Figure 2 illustrates the architecture of the proposed AS-EGAT model, which primarily consists of two elements: (1) The edge feature graph attention network based on aspect syntactic analysis seeks to reinforce the syntactic graph through the context of particular aspect words and the interrelation of different aspects in the context to extract the sentiment features of aspects; (2) The edge feature graph attention network based on semantic features seeks to capture the semantic relationship between words in sentences through self-attention. To produce the sentiment properties of a specific aspect, the feature representations captured by the two components are then merged. We begin by supposing that a sentence has n words, two aspects, i.e.,

$$\begin{array}{c}\hfill s=\left\{w_1,w_2,\dots ,a_{11},a_{12},\dots ,a_{1p},\dots ,a_{21},a_{22},\dots ,a_{2p},\dots ,w_n\right\}\end{array}$$

(1)

where $w_i$ represents the i-th contextual word, and $a_{ij}$ denotes the j-th word of aspect i. Every instance has a sentence, one or more aspects, and one or more words for each aspect.

Figure 2. The architecture of the proposed AS-EGAT model.

First, we use the pre-trained BERT to initialize the word vector; each word embedding is a distributed representation of a word in the sentence; for a sentence with n words, we can obtain the corresponding embedding matrix $X=\left\{x_1,x_2,\dots ,x_n\right\}$ where $x_i\in R_m$ is the word embedding of $w_i$ and m is the dimension of the word vector.

3.1. Aspect Syntactic Graph

To emphasize specific aspects from the context and capture aspect-focused weight maps for that sentence, we calculate the relative position weights of each word and construct a weight map depending on the specific aspect.

$$\begin{array}{c}\hfill W_{i,j}=\left\{\begin{array}{ccc}& 1\hfill & \hfill if{W}_i\in \left\{a_i^s\right\}and{W}_j\in \left\{a_i^s\right\}\\ & \frac{1}{\left(\left|j-p^s\right|+1\right)}\hfill & \hfill if{W}_i\in \left\{a_i^s\right\}\\ & \frac{1}{\left(\left|i-p^s\right|+1\right)}\hfill & \hfill if{W}_j\in \left\{a_i^s\right\}\\ & 0\hfill & \hfill otherwise\end{array}\right.\end{array}$$

(2)

We merge aspect-focused aspect graphs with syntactic analysis dependency trees to produce aspect-focused syntactic dependency adjacency matrices to improve the syntactic reliance of context words and gain relationships between aspects and context words.

$$\begin{array}{c}\hfill G_{i,j}=\left\{\begin{array}{ccc}& 1+W_{i,j}\hfill & \hfill ifi=jor{W}_i,W_{j}inthedependencytreeofthesentence\\ & 0\hfill & \hfill otherwise\end{array}\right.\end{array}$$

(3)

Affective expressions in the sentences may not be readily apparent in some aspects, however. In other words, because these aspects’ sentiment dependencies are in some ways coupled to other aspects, aspect-focused syntactic analysis graphs may not be sufficient to discover accurate sentiment connections. To make use of the connections between different aspects in a sentence, we therefore enhance the aspect syntactic graph by including the relative graphs of other aspects in the aspect-focused adjacency matrix.

$$\begin{array}{c}\hfill A^{asp}=G+\frac{1}{l}\sum _{a\in \left\{a_i^o\right\}}\alpha *G_a\end{array}$$

(4)

$$\begin{array}{c}\hfill \alpha =\frac{1}{\left(\left|p^o-p^s\right|+1\right)}\end{array}$$

(5)

where $p^o$ of each $a\in \left\{a_i^o\right\}$ stands for the initial position of another aspect, and $a_i^o$ is a word set of other aspects of length l. We create the directionless adjacency matrix, or $A_{i,j}^{asp}=A_{j,i}^{asp}$, to enhance the dependency information of the input sentences.

3.2. Semantic Graph

The self-attentive process produces an attention matrix as an adjacency matrix. On the one hand, self-attention, which is more flexible than the synchronous strategy structure, can capture semantically relevant sentences for each word in the sentence. The semantic graph, on the other hand, may be applied to the sentiment that is insensitive to syntactic information.

The attention score of each pair of elements was determined in parallel by Self-Attention [23]. The self-attention layer in our proposed AS-EGAT model is used to compute the attention score matrix $A^{sem}\in {\mathbf{R}}^{n\times n}$. Then we take the attention scoring matrix $A^{sem}$ as the semantic module’s adjacency matrix, namely $A_{i,j}^{sem}=A_{j,i}^{sem}$, which can be stated as:

$$\begin{array}{c}\hfill A^{sem}=softmax\left(\frac{Q{W}^Q\times {\left(K{W}^K\right)}^T}{\sqrt{d}}\right)\end{array}$$

(6)

The weight matrices $W^Q$ and $W^K$ can be learned, whereas the matrices Q and K originate from the output of the BERT layer. The input node feature’s dimension is d, as well. To determine a sentence’s attention score matrix in our model, we solely employ a self-attention head.

3.3. Edge-Featured Graph Attention Networks

We improve the GAT model’s original attention mechanism by making the edge features a significant factor in the calculation of the weight coefficients by the attention mechanism to efficiently mine the edge features of the GAT model, which uses aspect syntactic graphs and semantic graphs as adjacency matrices.

$$\begin{array}{c}\hfill {\overrightarrow{h_i}}^{\prime }=\sigma \left(\frac{1}{k}\sum _{k=1}^k\sum _{j\in N_i}{\alpha }_{i,j}^k{W}^k\overrightarrow{h_j}\right)\end{array}$$

(7)

$$\begin{array}{c}\hfill {\alpha }_{i,j}=\frac{exp\left(LeakyReLU\left({\overrightarrow{a}}^T\left[W\overrightarrow{h_i}\parallel W\overrightarrow{h_j}\parallel W\overrightarrow{A_{i,j}}\right]\right)\right)}{{\sum }_{k\in N_i}exp\left(LeakyReLU\left({\overrightarrow{a}}^T\left[W\overrightarrow{h_i}\parallel W\overrightarrow{h_j}\parallel W\overrightarrow{A_{i,j}}\right]\right)\right)}\end{array}$$

(8)

where ${\overrightarrow{a}}^T\in {\mathbf{R}}^{2F^{\prime }}$ is the feedforward neural network parameter, $LeakyReLU\left(·\right)$ is the activation function of the feedforward neural network, ∥ is the concatenation of two features, $\overrightarrow{A_{i,j}}$ is the matching adjacency matrix, and this formula may compute the correlation degree with the neighboring nodes. The number of heads in the attention span of multiple heads is represented by the parameter K, adding and averaging yields the updated characteristic ${\overrightarrow{h_i}}^{\prime }$.

3.4. Training

The initial node of the aspect parsing EGAT layer is formed from the Bi-LSTM layer’s hidden representation, which embeds words as input:

$$\begin{array}{c}\hfill H=\left\{h_1,h_2,\dots ,h_n\right\}=Bi-LSTM\left(x\right)\end{array}$$

(9)

The aspect parsing EGAT layer’s final representation is $h^{asp}$. $h^{asp}$ provides the initial node for the EGAT layer based on the semantic feature, while $h^{sem}$ provides the final representation for the EGAT layer based on the semantic feature. To extract the relationship between aspect syntactic and semantic information, we merge these two final representations.

$$\begin{array}{c}\hfill \tilde{h}=h^{asp}+\lambda h^{sem}\end{array}$$

(10)

where $\lambda$ is the coefficient of semantic features. We employ aspect-specific masking to cover non-aspect representations, highlighting the important aspects of aspect words:

$$\begin{array}{c}\hfill {\tilde{H}}_{mask}=\left\{0,\dots ,{\tilde{h}}_{\tau },\dots ,{\tilde{h}}_{\tau +k-1},\dots ,0\right\}\end{array}$$

(11)

where ${\tilde{h}}_t$ is the representation of the t-th word that AS-EGAT learned, $\tau$ is the index at which a certain aspect first appears, and k is the length of that aspect. We then use a retrieving-based attention method, inspired by Zhang [16], to extract significant sentiment features from contextual representations of particular aspects:

$$\begin{array}{c}\hfill {\xi }_t=\sum _{i=1}^n{h_t^c}^T{\tilde{h}}_i=\sum _{i=\tau }^{\tau +k-1}{h_t^c}^T{\tilde{h}}_i\end{array}$$

(12)

$$\begin{array}{c}\hfill {\eta }_t=\frac{exp\left({\xi }_t\right)}{{\sum }_{i=1}^{n}exp\left({\xi }_t\right)}\end{array}$$

(13)

As a result, the following is the final representation of the input for specific aspects:

$$\begin{array}{c}\hfill y=softmax\left(W_{o}r+b_o\right)\end{array}$$

(14)

$$\begin{array}{c}\hfill r=\sum _{i=1}^n{\eta }_i{h}_i^c\end{array}$$

(15)

where $softmax\left(·\right)$ is the $softmax$ function to obtain the output distribution of the classifier.

Minimizing the cross entropy loss between the predicted distribution and the ground real distribution is the definition of the classifier training objective:

$$\begin{array}{c}\hfill L=-\sum _{i=1}^S\sum _{j=1}^C{\widehat{y}}_i^j·log\left(y_i^j\right)+\gamma {\parallel \Theta \parallel }_2\end{array}$$

(16)

where C is the classification number and S is the training scale. The actual sentiment distribution is $\widehat{y}$. The regularization term $L_2$ weight is $\gamma$. All trainable parameters are represented by $\Theta$.

4. Experiment

4.1. Dataset and Experimental Setup

In the ABSA, since the LAP14, REST15, REST16, and MAMS datasets are the most commonly used and popular benchmark datasets and have abundant sources, we chose them to test the universality of our model. The LAP14, REST15, and REST16 datasets are the laptop and restaurant review datasets commonly used in the SemEval 2014 (Laptop 14) [24], SemEval 2015 (Restaurant 15) [25], and SemEval 2016 (Restaurant 16) [26] tasks. The MAMS datasets [27] are multiple aspects. To further demonstrate the validity of our proposed model, we use the dataset provided by Mukherjee [28], which uses similar guidelines to label two new datasets from the field of e-commerce, namely T-shirts and Television [28]. Each sample consists of a comment sentence, an aspect (single or multiple words), and a corresponding aspect of emotional polarity (positive or neutral, or negative). The statistics information of these six datasets is shown in Table 1.

Table 1. The statistical information of all the datasets.

Dataset	Positive		Neutral		Negative
	Train	Test	Train	Test	Train	Test
LAP14	994	341	464	169	870	128
REST15	1178	439	50	35	382	328
REST16	1620	597	88	38	709	190
MAMS	3380	400	5042	607	2764	329
T-shirt	1122	270	50	16	699	186
Television	2540	618	287	67	919	257

In our experiment, each word was initialized into a 768-dimensional word vector using the BERT preprocessing model, and the $\gamma$ of the $L_2$ regularization coefficient was set to ${10}^{-5}$. With a learning rate of ${10}^{-3}$, Adam was employed as an optimizer to train the model. The scale of batch training was 16. We use a uniform distribution to initialize all of the W and b at random.

4.2. Comparative Model

We compared the proposed model (AS-EGAT) with the following models:

• TD-LSTM [5]: develops two target-dependent long short-term memory (LSTM) models, where target information is automatically taken into account.
• ATAE-LSTM [11]: proposes an Attention-based Long Short-Term Memory Network for aspect-level sentiment classification. The attention mechanism can concentrate on different parts of a sentence when different aspects are taken as input.
• MemNet [29]: introduces a deep memory network for aspect level sentiment classification. This approach explicitly captures the importance of each context word.
• IAN [15]: proposes the interactive attention networks (IAN) to interactively learn attentions in the contexts and targets, and generate the representations for targets and contexts separately.
• RAM [30]: adopts a multiple-attention mechanism to capture sentiment features separated by a long distance, so that it is more robust against irrelevant information.
• GCAE [3]: proposes a model based on convolutional neural networks and gating mechanisms, which is more accurate and efficient.
• MGAN [31]: proposes a fine-grained attention mechanism, which can capture the word-level interaction between aspect and context. It then leverages the fine-grained and coarse-grained attention mechanisms to compose the MGAN framework.
• AOA [14]: models aspects and sentences jointly and explicitly captures the interaction between aspects and context sentences.
• TNet-LF [2]: proposes a component to generate target-specific representations of words in the sentence, meanwhile incorporating a mechanism for preserving the original contextual information from the RNN layer.
• ASGCN-DT [16] and ASGCN-DG [16]: propose to build a Graph Convolutional Network (GCN) over the dependency tree of a sentence to exploit syntactical information and word dependencies.
• BERT [8]: designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers.
• CapsNet+BERT [27]: presents a new large-scale Multi-Aspect Multi-Sentiment (MAMS) dataset, in which each sentence contains at least two different aspects with different sentiment polarities.
• GIN+BERT [32]: adopts two attention-based networks to learn the contextual sentiment for the entity and attribute independently and interactively. Further, based on the interactive attentions learned from entities and attributes, the coordinative gate units are exploited to reconcile and purify the sentiment features for the aspect sentiment prediction.
• MIMLLN+BERT [33]: proposes a Multi-Instance Multi-Label Learning Network for Aspect-Category sentiment analysis (AC-MIMLLN), which treats sentences as bags, words as instances, and the words indicating an aspect category as the key instances of the aspect category.
• SGGCN+BERT [34]: proposes a mechanism to obtain the importance scores for each word in the sentences based on the dependency trees that are then injected into the model to improve the representation vectors for ABSA.
• DGEDT+BERT [35]: proposes a dependency graph enhanced dual-transformer network (named DGEDT) by jointly considering the flat representations learned from Transformer and graph-based representations learned from the corresponding dependency graph in an iterative interaction manner.
• R-GAT+BERT [36]: defines a unified aspect-oriented dependency tree structure rooted at a target aspect by reshaping and pruning an ordinary dependency parse tree.
• DeBERTa [37]: presents a new pre-trained language model, DeBERTaV3, which improves the original DeBERTa model by replacing mask language modeling (MLM) with replaced token detection (RTD), a more sample-efficient pre-training task.
• T-GCN+BERT [38]: proposes an approach to explicitly utilize dependency types for ABSA with type-aware graph convolutional networks (T-GCN), where attention is used in T-GCN to distinguish different edges (relations) in the graph, and attentive layer ensemble is proposed to comprehensively learn from different layers of T-GCN.
• DualGCN+BERT [39]: proposes a dual graph convolutional networks (DualGCN) model that considers the complementarity of syntax structures and semantic correlations simultaneously.
• GL-GCN [40]: proposes a novel aspect-based sentiment classification approach, i.e., Global and Local Dependency Guided Graph Convolutional Networks (GL-GCN).
• SenticGCN+BERT [41]: proposes a graph convolutional network based on SenticNet to leverage the effective dependencies of the sentence according to the specific aspect, called Sentic GCN.
• SEDC-GCN [42]: proposes a novel GCN based model, named the Structure-Enhanced Dual-Channel Graph Convolutional Network (SEDC-GCN).
• HGCN [43]: proposes a hybrid graph convolutional network (HGCN) to synthesize information from constituency tree and dependency tree, exploring the potential of linking two syntax parsing methods to enrich the representation.
• AEN+BERT [44]: proposes an Attentional Encoder Network (AEN) which eschews recurrence and employs attention based encoders for the modeling between context and target.
• STGNN-GRU [45]: proposes a graph Fourier transform based network with features created in the spectral domain. Fourier transform is used to switch to the frequency (spectral) domain where new features are created.
• LGCF-CDM [46] and LGCF-CDW [46]: propose a multilingual learning model based on the interactive learning of local and global context focus, namely LGCF. This model can effectively learn the correlation between local context and target aspects and the correlation between global context and target aspects simultaneously.

4.3. Evaluation Metric

In order to evaluate the performance of the model, Accuracy (Acc.) and Macro-F1 score (F1) is selected as the main evaluation metric. Accuracy can intuitively see the percentage of predicted correct results in the total sample. However, for the data with an unbalanced proportion of positive and negative samples, the category with a large proportion will affect the accuracy. The F1 score is the harmonic average of Recall and Precision, which is a comprehensive evaluation metric.

4.4. Experimental Results

Table 2 shows an overview of the experimental results for LAP14, REST15, REST16, and MAMS datasets using Acc. and F1 metrics. The experimental results show that our proposed AS-EGAT model is superior to the current state-of-the-art baselines. More specifically, Acc./F1 scores in our AS-EGAT model improved to 0.76%/1.16%, 0.29%/1.16%, 0.05%/1.23%, and 0.15%/0.37% compared to baseline models for LAP14, REST15, REST16, and MAMS.

Table 2. Performance comparison on different models on LAP14, REST15, REST16, and MAMS datasets. The best performance is bold-typed.

Model	LAP14		REST15		REST16		MAMS
	Acc.	F1	Acc.	F1	Acc.	F1	Acc.	F1
TD-LSTM (2016a) [5]	71.83	68.43	76.39	58.70	82.16	54.21	74.59	–
ATAE-LSTM (2016b) [11]	68.88	63.93	78.48	60.53	83.77	61.71	77.05	–
MenNet (2016b) [29]	70.64	65.17	77.31	58.28	85.44	65.99	64.56	–
IAN (2017) [15]	72.05	67.38	78.54	52.65	84.74	55.21	76.60	–
RAM (2017) [30]	74.49	71.35	79.98	60.57	83.88	62.14	–	–
GCAE (2018) [3]	71.98	68.71	77.56	56.03	83.70	62.69	77.59	–
MGAN (2018) [31]	75.39	72.47	79.36	57.26	87.06	62.29	–	–
AOA (2018) [14]	72.62	67.52	78.17	57.02	87.50	66.21	77.26	–
TNet-LF (2018) [2]	74.61	70.14	78.47	59.47	89.07	70.43	–	–
ASGCN-DT (2019) [16]	74.14	69.24	79.34	60.78	88.69	66.64	–	–
ASGCN-DG (2019) [16]	75.55	71.05	79.89	61.89	88.99	67.48	–	–
BERT (2019) [8]	77.59	73.28	83.48	66.18	90.10	74.16	80.62	80.77
CapsNet+BERT (2019) [27]	–	–	81.89	61.85	86.50	62.12	82.97	–
GIN+BERT (2020) [32]	–	–	83.96	66.03	89.47	74.87	–	–
MIMLLN+BERT (2020) [33]	–	–	82.76	65.10	88.12	73.05	–	–
SGGCN+BERT (2020) [34]	–	–	82.72	65.86	90.52	74.53	–	–
DGEDT+BERT (2020) [35]	79.80	75.60	79.89	61.89	88.99	67.48	–	–
R-GAT+BERT (2020) [36]	79.73	75.50	–	–	91.87	75.54	81.75	80.87
DeBERTa (2021) [37]	–	–	–	–	–	–	83.06	82.52
T-GCN+BERT (2021) [38]	–	–	85.26	71.69	92.32	77.29	83.68	83.07
DualGCN+BERT (2021) [39]	79.51	–	83.78	–	91.43	–	–	–
GL-GCN (2021) [40]	76.91	72.76	80.81	64.99	88.47	69.64	–	–
SenticGCN+BERT (2022) [41]	78.21	74.07	85.32	71.28	91.97	79.56	–	–
SEDC-GCN (2022) [42]	77.74	74.68	81.73	66.23	90.75	73.84	–	–
HGCN (2022) [43]	78.82	–	80.81	–	88.92	–	–	–
AS-EGAT+BERT (ours)	80.56	76.76	85.61	72.85	92.37	80.79	83.83	83.44

Table 3 shows an overview of the results of experiments on the T-shirt and Television datasets using Acc. and F1 metrics. Acc./F1 scores improved by 0.22%/0.53% and 0.38%/1.93% in our AS-EGAT model compared with baseline models for T-shirt and Television.

Table 3. Performance comparison on different models on T-shirt and Television datasets. The best performance is bold-typed.

Model	T-Shirt		Television
	Acc.	F1	Acc.	F1
BERT-SPC (2019) [8]	93.13	73.86	89.96	74.68
BERT-AEN (2019) [8]	88.69	72.25	87.09	67.92
ASGCN-DT (2019) [16]	92.01	71.86	84.96	69.15
ASGCN-DG (2019) [16]	91.59	73.09	85.27	69.40
ASGCN+BERT (2019) [16]	92.22	77.24	88.99	74.42
AEN+BERT (2019) [44]	88.69	72.25	87.09	67.92
STGNN-GRU (2022) [45]	–	–	89.73	75.73
LGCF-CDM (2022) [46]	93.22	76.73	–	–
LGCF-CDW (2022) [46]	93.64	77.23	–	–
AS-EGAT+BERT (ours)	93.86	77.76	90.34	76.61

This verifies the validity of AS-EGAT in ABSA. It is also worth noting that the performance of AS-EGAT is significantly better than that of previous models based on the graph convolution network and graph attention network, which fundamentally verifies the validity of the new solution of graph construction used in this work.

4.5. Ablation Study

To confirm the viability of our suggested strategy, we also carried out ablation tests to examine the effects of different modules in the AS-EGAT model. Table 4 presents the outcomes. It can be shown that (1) eliminating the “aspect syntactic features” module lowers the model’s performance, proving that adding the “aspect syntactic features” information to the model improves aspect-oriented sentiment analysis; (2) the module’s unsatisfactory performance on all datasets when “semantic characteristics” are removed is evidence that rich grammatical knowledge can decrease dependency parsing errors; (3) the “edge feature” module can improve the correlation of edges with a strong correlation, as shown by the fact that removing it causes the model’s performance to decline; (4) the model effect becomes worse when the “aspect syntactic features and side features” module is removed, demonstrating the module’s influence on the model; (5) the performance of the model becomes unsatisfactory when the “semantic feature and edge feature” module is removed, demonstrating the module’s significance to the model. Our AS-EGAT model delivers the finest performance overall.

Table 4. Experimental results of the ablation study.

Model	LAP14		REST15		REST16		MAMS
	Acc.	F1	Acc.	F1	Acc.	F1	Acc.	F1
AS-EGAT+BERT w/o A	80.25	76.02	85.24	71.02	91.56	80.41	83.01	82.38
AS-EGAT+BERT w/o S	79.47	76.15	84.87	69.05	91.56	73.52	83.46	83.05
AS-EGAT+BERT w/o E	78.06	73.83	85.16	71.04	91.23	74.77	83.53	82.92
AS-EGAT+BERT w/o A+E	79.15	75.01	85.06	65.81	90.91	76.73	83.68	83.04
AS-EGAT+BERT w/o S+E	79.47	76.42	85.06	71.15	90.91	73.38	83.76	83.39
AS-EGAT+BERT	80.56	76.76	85.61	72.85	92.37	80.79	83.83	83.44

4.6. Influence of the Coefficient of Semantic

We conducted trials with various $\lambda$ values to further examine the impact of the semantic module in the AS-EGAT model, and the outcomes are displayed in Figure 3. It can be seen that the datasets perform best when the $\lambda$ value reaches $1.0$. The curve swings significantly whether $\lambda$ is less than or more than $1.0$, indicating that the best semantic information may be recovered when $\lambda$ is equal to $1.0$.

Figure 3. Impact of semantic coefficients. Accuracy and Macro-F1 score based on different values of $\lambda$ is reported.

4.7. Analysis of Different Sentiment Polarity

Figure 4 shows the performance of the AS-EGAT model in different sentimental polarities. In the five datasets of LAP14, REST15, REST16, T-shirt, and Television, “positive” has the best performance, while “neutral” has the worst performance. The results were similar for the three sentimental polarities in the “MAMS” dataset. This phenomenon is related to the distribution of sentimental polarity in the dataset. The number of “neutral” training data in the “MAMS” dataset is higher than the other two sentimental polarities, while for the other five datasets, the number of “neutral” training data is the least. Furthermore, the model struggles to capture emotional information in “neutral” sentences, resulting in low performance.

Figure 4. Experimental results of different sentiment polarity.

4.8. Single and Multiple Aspects Analysis

In order to further analyze the effects of the AS-EGAT model in single and multiple aspects, we divided the test set into single and multiple aspects according to the number of aspects in the sentence for experiments, and the results are shown in Figure 5. According to the experimental results, it can be found that in LAP14, REST15, and MAMS datasets, single and multiple aspects accuracy rates are basically the same, while in REST16, T-shirt, and Television datasets, multiple aspects accuracy rates are slightly higher than single aspects accuracy rates. From this, we can see that the AS-EGAT model is more effective in multiple aspects.

Figure 5. Experimental results of single and multiple aspects.

4.9. Visualization

We exhibit two visual attention weight graphs in Figure 6 to demonstrate how the AS-EGAT model improves the performance of aspect sentiment analysis. As shown in Figure 6a, the weight value of the relevant terms in the modules of ‘aspect syntactic analysis’, ‘semantic analysis’, and ‘edge feature’ is less than that of AS-EGAT. As a result, the AS-EGAT proposed can pay more attention to important aspect aspects in order to extract the sentiment characteristics of specific aspects. The multi-faceted examples in Figure 6b show how different aspects of the same sentence receive varied weights of attention, demonstrating the efficiency of our model in extracting sentiment features from diverse aspects of the text.

Figure 6. The attention visualizations.

5. Conclusions

In this paper, we explore a new ABSA solution for constructing aspect syntactic analysis graphs and semantic dependency graphs. On this basis, an attention network (AS-EGAT) model based on aspect syntactic and semantic features is proposed to extract aspect-based sentiment features from the aspect and semantic perspectives. Therefore, the proposed AS-EGAT model can pay significant attention to keywords when processing multiple words, connect valuable sentiment features of related aspects when considering aspects without obvious sentiment expression, and obtain corresponding sentiment features by extracting semantic features for sentences insensitive to the syntactic dependency tree. Experimental results on six datasets show that the proposed AS-EGAT can outperform existing graph-based models and BERT-based models.

Notably, the main contribution of this paper is to increase sentence structure information by constructing syntactic analysis graphs and semantic dependency graphs, a general strategy that can be easily applied to these BERT and graph-based neural network methods. From the experimental data, we can find that the model has poor performance for sentences with “neutral” sentiment polarity. For this purpose, in future work, we will construct new graphs to capture the sentiment information of sentences with “neutral” sentiment polarity and continue investigating its validity in these BERT-based frameworks.