Supervised and Few-Shot Learning for Aspect-Based Sentiment Analysis of Instruction Prompt

Abstract

Aspect-based sentiment analysis (ABSA), which aims to extract aspects and corresponding opinions from sentences and determine aspect sentiment polarity, has been widely studied in recent years. Most approaches focus on the subtasks of ABSA and deal with them in the pipeline method or end-to-end method. However, these approaches ignore the semantic information of the labels and the correlation between the labels. In this study, we process various ABSA tasks in a unified generative framework and use instruction prompts to guide the generative model to learn the relationships between different sentiment elements, accurately identify the sentiment elements in sentences, and improve the performance of the model in few-shot learning. Experimental results on several benchmark datasets show that our approach achieves significant performance gains. Among them, for the aspect term extraction and sentiment classification task on the Laptop 14 dataset, our method improves the F1 score by 4.08% and 1.87% on fully supervised learning compared to the GAS model and PARA model, respectively. In few-shot learning, we can achieve 80% of the fully supervised learning performance using one-tenth of the dataset. Our method can effectively address the problem of data shortage in low-resource environments.

1. Introduction

Aspect-based sentiment analysis (ABSA) is a fine-grained task in sentiment analysis that aims to identify aspect terms (a), their corresponding opinion terms (o), and the sentiment polarity (s) expressed in a sentence, as illustrated in Figure 1.

As coarse-grained sentiment analysis is no longer suitable for people’s evolving needs, aspect-based sentiment analysis has garnered increasing attention in recent years. ABSA consists of several subtasks, namely Aspect Term Extraction (AE), Opinion Term Extraction (OE), and Aspect Sentiment Classification (ASC). These subtasks are strongly correlated with each other and are often combined to create a variety of compound tasks. For instance, Zhao et al. [1] proposed the Aspect Opinion Pair Extraction (AOPE) task, which aims to extract aspect terms and their corresponding opinion terms in pairs. Similarly, Peng et al. [2] proposed the Aspect Sentiment Triple Extraction (ASTE) task, which involves extracting both aspect terms and corresponding opinion terms from sentences and determining the aspect sentiment polarity.

Previous research on ABSA has primarily focused on single-task forms, where AE, OE, and ASC can only produce results for one specific type of task. Regarding the AE task, most studies treat it as a sequence labeling problem [3,4]. The OE task is typically considered an auxiliary task to the AE task. Since opinions and aspects often co-occur in a sentence, it is illogical to extract opinions alone without considering the corresponding aspects. The ASC task is a sentiment classification problem that is based on aspect terms [5,6]. In this task, sentiment judgments are made by considering the aspect terms and their contexts.

Most studies now utilize either an end-to-end approach or a pipeline approach to solve multiple tasks simultaneously. Taking end-to-end ABSA as an example, it aims to predict both the aspect terms mentioned in the text and their corresponding sentiment polarity.

There are several issues with existing methods for aspect-based sentiment analysis, including the following:

C1: The absence of interaction between different target labels often leads to reduced accuracy in compound tasks. In general, the majority of tasks rely on sequence labels for term extraction and utilize specific classification networks for sentiment classification. However, these methods ignore the correlation between different tasks. During training, the rich semantic information conveyed by different labels is often neglected, and the interaction between labels is not taken into account. For instance, when given the sentence “The service is rude, but the burgers are delicious”, it is easier to identify the sentiment of the aspect term “service” as “negative” if we consider its corresponding opinion term “rude”. If the model understands that “delicious” is an adjective used to describe food, it can easily identify the aspect-opinion pair “(burgers, delicious)”.

C2: Differences in training methods between pre-trained models and downstream tasks often make it challenging to leverage the knowledge of pre-trained models. This can result in forgetting of prior knowledge, making it difficult to recognize multiple sentiment elements. The pre-trained model uses a natural language form for self-supervised learning during training, which is different from the training form for downstream tasks. This makes it difficult to take full advantage of the knowledge already acquired by the model during pre-training. As an example, Zhang et al. [7] proposed to use a generative model for aspect-based sentiment analysis, where the extraction style was targeted in the ASTE task in the form of (aspect, opinion, sentiment polarity), which was different from the learning form in the pre-training stage of the model and produced errors.

C3: It is difficult to achieve good results with low resources, making it inefficient under restricted or limited datasets. To achieve satisfactory results, existing ABSA methods typically require at least several thousand data to fine-tune the model. In some areas, the lack of large annotated datasets and the high cost of manual data annotation pose challenges to using traditional ABSA methods. Therefore, there has been a growing focus on how to achieve good results with a small amount of data.

Inspired by the recent success of transforming various NLP tasks into text generation problems [8,9,10], we use a unified generation framework to solve various ABSA tasks without the need for model network structure design for specific tasks. We transform the ABSA task into a text generation problem to enhance the interaction of different target elements, which can avoid error propagation in compound tasks. Due to the variety of ABSA tasks, we focus on aspect sentiment classification (ASC), aspect term extraction and sentiment classification (AESC), and aspect sentiment triplet extraction (ASTE).

In this paper, our main contributions are as follows:

(1)

We transform the ABSA task into a unified generative task, eliminating the need to design specific network structures for each task. The connection between sentiment elements is strengthened in the compound task. For example, the F1 scores for the AESC and ASTE tasks are improved by 4.86% and 10.17%, respectively, on the Laptop 14 dataset.

(2)

We propose three instruction prompts to redesign the input and output formats of the model. This approach enables downstream tasks to adapt to the pre-trained model, reduces the errors between the pre-trained model and the downstream tasks, and improves the utilization of the pre-trained model.

(3)

We use the instruction prompt method for few-shot learning on ABSA tasks. Experimental results show that using only 10% of the original data can achieve 80% of the model performance compared to fully supervised learning. This can reduce annotation costs while obtaining good models.

(4)

We conducted experiments on three ABSA tasks using both fully supervised and few-shot learning approaches on four benchmark datasets. Our proposed approach outperformed the state-of-the-art in nearly all cases, demonstrating its effectiveness in improving ABSA performance. For example, our approach demonstrates improvements over the state-of-the-art in almost all cases when conducting fully supervised learning and few-shot learning experiments on three ABSA tasks across four benchmark datasets. On the Laptop 14 dataset, our fully supervised learning approach results in a 1.46% increase in the F1 score in the ASTE task, while our few-shot learning approach achieves an F1 score of 50.62%, which is 82% of the performance of the fully supervised learning model.

2. Related Work

Sentence-level sentiment analysis mainly determines the sentiment expressed in the entire text, and there is currently some research [11,12,13] in this area. However, coarse-grained sentiment analysis can no longer meet people’s needs, so there is a growing exploration of aspect-based sentiment analysis methods to extract finer-grained sentiment information. Early research work on aspect-based sentiment analysis could only identify single sentiment elements. For example, the aspect term extraction task aims to extract the aspect terms mentioned in a sentence [14,15]. The aspect sentiment classification task aims to predict the sentiment polarity of aspects in a sentence [16]. These tasks are all single tasks. A single sentiment element is insufficient to comprehend the complete aspect-level perspective. This requires not only extracting multiple sentiment elements but also understanding the correspondences and dependencies between the sentiment elements. In recent years, most studies have focused on the ABSA compound task, which acquires multiple sentiment elements simultaneously. For example, the Aspect Sentiment Triplet Extraction (ASTE) task requires the extraction of aspect terms, corresponding opinion terms and aspect sentiment, such as the (atmosphere, nice, positive) triplet extracted from the example in Figure 1.

To improve the performance of ABSA tasks, it has become common to use pre-trained models, which have been shown to achieve better results. The modeling paradigms commonly used in ABSA tasks include Sequence-level Classification (SeqClass), Token-level Classification (TokenClass), Machine Reading Comprehension (MRC), and Sequence-to-Sequence modeling (Seq2Seq), as shown in Figure 2. Each modeling paradigm serves as a generic framework for handling ABSA tasks, on which the researchers can design modifications to achieve better results. In addition, the complex nature of ABSA tasks often requires the use of the pipeline paradigm, where multiple models are combined to make the final prediction.

In recent years, the study of ABSA tasks has slowly shifted from single tasks to compound tasks. Compound tasks are relatively complex. ABSA compound tasks are usually solved using the pipeline method [2,17,18,19] or end-to-end method [20,21,22,23]. The pipeline approach is relatively easy to implement, requiring only the sequential connection of solutions for each subtask to obtain the final result. However, it suffers from the error propagation problem, i.e., errors generated by the earlier models propagate to the later models and affect the overall output results. The end-to-end method of solving ABSA tasks typically requires a deeper understanding of the tasks and models involved, as well as specific design of input and output formats. This can make it a more challenging approach compared to the pipeline method.

ABSA has correlation between different tasks. Existing work designs a variety of task-specific classification networks, making it difficult for models to adapt to each other with potential errors. Zhang et al. [7] transformed the ABSA task into a text generation problem and proposed two paradigms, namely annotation-style and extraction-style modeling, to solve Aspect Opinion Pair Extraction (AOPE), Unified ABSA (UABSA), Aspect Sentiment Triplet Extraction (ASTE), and Target Aspect Sentiment Detection (TASD) tasks with good results. Different ABSA tasks require different models and it is challenging to solve multiple tasks simultaneously with a single model. Yan et al. [10] proposed a unified generative framework that uses the BART generative model and pointer networks to solve the seven ABSA tasks. Due to the strong correlation between different tasks, Zhang et al. [24] proposed the Paraphrase modeling paradigm to generate sentiment elements in the form of natural language, which can fully explore the essence of sentiment elements.

The above methods require a large amount of labeled data for training. However, there are not enough labeled datasets in some fields. Few-shot learning or zero-shot learning is the ideal solution for completing ABSA tasks in low-resource scenarios. Since the advent of GPT-3 [25], prompt-tuning has received greater attention and a series of research works have emerged. Compared with standard fine-tuning, prompt-tuning can effectively stimulate the knowledge in the pre-trained model and achieve better performance in few-shot learning. Liu et al. [26] first summarized the research on prompt learning, which led to a great promotion of prompt learning. Unlike traditional learning methods, prompt learning is based on language models that directly model text probabilities, and adapts to new scenarios with zero-shot or few-shot data by defining prompt templates that guide model output and tap into model latent knowledge. The purpose of model fine-tuning is to adapt the model to the requirements of the downstream task in a way that differs from the model pre-training stage. The difference is that prompt learning aims to adapt the downstream task to the model to bridge the gap between pre-training and fine-tuning. Recent research in Pattern-Exploiting Training (PET) [27] has explored the use of semi-supervised learning, where the input data are transformed into a completion format to assist language models in understanding the given task. PET outperforms both supervised training and strong semi-supervised training methods in low-resource environments. The findings indicate that prompt learning can result in improved performance for pre-trained models in various NLP tasks, including text classification [28], relation classification [29], named entity recognition (NER) [30], and so on.

Prompt learning has demonstrated superior performance in NLP tasks, and we aim to apply it to ABSA tasks. To explore the potential benefits of prompt learning on the complex ABSA task, we apply this technique to both single and compound ABSA tasks and evaluate its effectiveness in few-shot learning scenarios with limited labeled data.

3. Methodology

Aspect-based sentiment analysis (ABSA) aims to identify aspect terms, corresponding opinion terms, and aspect sentiment polarity from the given text. This is demonstrated in Figure 1.

Figure 3. Overview of the ABSA generation framework using the ASTE task as an example.

Figure 3. Overview of the ABSA generation framework using the ASTE task as an example.

Given a text T,the ABSA task obtains $(a,o,s)$, which correspond to aspect term, opinion term, and sentiment polarity, respectively. The aspect term a and the opinion term o exist in the sentence T, where $a\in V_T$ and $o\in V_T$. $V_T$ contains all the words that appear in T. Sentiment polarity s takes values in the set of sentiment categories {positive, negative, neutral}. In the study, the verbalizer is used to convert the sentiment categories into words that are represented in the output, i.e., {‘positive’: ‘great’, ‘negative’: ‘bad’, ‘neutral’: ‘ok’}. Our goal is to use the generative model M to obtain the final target y. y contains all the desired ABSA elements, i.e., $(a,o,s)$.

In this paper, we train the generative model using fully supervised learning and few-shot learning in the form of instruction prompts to obtain the three sentiment elements of aspect terms, opinion terms, and aspect sentiment.

3.1. Input Transformation

To reduce the errors in pre-training models and downstream task fine-tuning, we propose three instruction prompt templates, named IPT-a, IPT-b, and IPT-c, respectively. Instruction prompts in ABSA can be considered as providing the model with explicit cue prompts, similar to a question-answer format. The prompts guide the model to focus on specific aspects of the input text, such as identifying aspect terms or extracting aspect sentiment, which can be learned through a process of question-answering. By providing these prompts, the model is encouraged to learn the relevant ABSA elements more effectively. We transform the raw text data into the format of instruction prompts. The instruction prompt template reformats the input and output formats, as shown in Figure 3. Examples of the conversions for each task, including ASC, AESC, and ASTE, can be found in

Table 1. Examples of three instruction prompt templates for ASC tasks.

Prompt Template Name	Prompt Template—Input	Prompt Template—Output	Example—Input	Example—Output	Target—Ouput
ASC-IPT-a	[T] What is the sentiment of the [a] in the above text?	The [a] is [s]	The keyboard is too slick. What is the sentiment of the keyboard in the above text?	The keyboard is bad.	[negative]
ASC-IPT-b	Given the text: [T] ([a]) What is the meaning of the above text?	Summary: the [a] is [s]	Given the text: The keyboard is too slick. (keyboard) What is the meaning of the above text?	Summary: the keyboard is bad.	[negative]
ASC-IPT-c	Given the text: [T] ([a]) Please briefly summarize the above text.	In summary, the [a] is [s]	Given the text: The keyboard is too slick. (keyboard) Please briefly summarize the above text.	In summary, the keyboard is bad.	[negative]

,

Table 2. Examples of three instruction prompt templates for AESC tasks.

Prompt Template Name	Prompt Template—Input	Prompt Template—Output	Example—Input	Example—Output	Target—Ouput
AESC-IPT-a	[T] What are the aspects and sentiments in the above text?	The [a] is [s]	But the staff was so horrible to us. What are the aspects and sentiments in the above text?	The staff is bad.	[staff, negative]
AESC-IPT-b	Given the text: [T] What is the meaning of the above text?	Summary: the [a] is [s]	Given the text: But the staff was so horrible to us. What is the meaning of the above text?	Summary: the staff is bad.	[staff, negative]
AESC-IPT-c	Given the text: [T] Please briefly summarize the above text.	In summary, the [a] is [s]	Given the text: But the staff was so horrible to us. Please briefly summarize the above text.	In summary, the staff is bad.	[staff, negative]

and

Table 3. Examples of three instruction prompt templates for ASTE tasks.

Prompt Template Name	Prompt Template—Input	Prompt Template—Output	Example—Input	Example—Output	Target—Ouput
ASTE-IPT-a	[T] What are the aspects and their corresponding opinions and sentiments in the above text?	The [a] is [o] so it is [s]	This place is incredibly tiny. What are the aspects and their corresponding opinions and sentiments in the above text?	The place is tiny so it is bad	[place, tiny, negative]
ASTE-IPT-b	Given the text: [T] What is the meaning of the above text?	Summary: the [a] is [o] so it is [s]	Given the text: This place is incredibly tiny. What is the meaning of the above text?	Summary: the place is tiny so it is bad	[place, tiny, negative]
ASTE-IPT-c	Given the text: [T] Please briefly summarize the above text.	In summary, the [a] is [o] so it is [s]	Given the text: This place is incredibly tiny. Please briefly summarize the above text.	In summary, the place is tiny so it is bad	[place, tiny, negative]

, respectively.

Aspect Sentiment Classification (ASC) is a task that aims to classify the sentiment polarity of a given aspect term [31,32]. Table 1 illustrates examples of ASC tasks under three different instruction prompt templates.

Table 1 illustrates that the input and output formats for the model are natural language text, which is similar to the pre-training stage of the model. This reduces the discrepancy between the pre-training stage and the fine-tuning stage. Since the output template is fixed, the model needs to learn the form of the output distribution, which places higher demands on the model. We format the output to obtain the desired result.

Aspect Term Extraction and Sentiment Classification (AESC) is a task in which both the aspect terms and their corresponding sentiment polarities are extracted from the given text [33,34,35]. Table 2 presents examples of AESC tasks formatted using three instruction prompt templates.

Aspect Sentiment Triplet Extraction (ASTE) is a complex task to extract sentiment triples (a, o, s) from text [7,10]. Table 3 shows examples of ASTE tasks under three instruction prompt templates.

Under the three prompt templates, when the text contains multiple targets, the output is separated by a special token [SSEP], similar to the approach proposed by Zhang et al. [24]. For example, given the text “The pizza is yummy and I like the atmoshpere.”, the output of the ASC task is “The pizza is great [SSEP] The atmoshpere is great”. The output format of other tasks is also similar.

3.2. Sequence-to-Sequence Learning

Given the input text T, we use the generative model $f(•)$ to generate the target sequence $y^{\prime }$, where $y^{\prime }$ is similar to the “Example—Input” column in Table 1, Table 2 and Table 3. Then, the corresponding sentiment elements are extracted from $y^{\prime }$ based on the task requirements, as shown in Figure 3.

We choose the pre-trained T5 model [36], which uses the encoder–decoder architecture of the original transformer. Given an input text T, the encoder first transforms it into a contextual encoding sequence e. Then, the decoder performs conditional probability modeling $p_{\theta }\left(y^{\prime }\right|e)$ according to e to obtain the target text $y^{\prime }$, where $\theta$ is the parameter.

At each time step, the output of the decoder at the current time is obtained from the output of the decoder at the previous time step and the output e of the encoder. Next, the probability distribution of the next token is obtained using softmax.

$$y_i^{\prime }=f_{dec}(e,y_{i-1}^{\prime })$$

(1)

$$p_{\theta }\left(y_{i+1}^{\prime }\right|e,y_i^{\prime })=softmax\left(W^T{y}_i^{\prime }\right)$$

(2)

where $y_i^{\prime }$ is the output of the decoder at the i-th time step; $f_{dec}$ is the decoding calculation; W is the trainable parameter.

In the training stage, we initialize $\theta$ with the pre-trained T5 model and then further fine-tune the model parameters to maximize $p_{\theta }\left(y^{\prime }\right|e)$.

In the inference stage, we use greedy search as the decoding strategy, and the token with the highest probability from the word list is selected as the next token at each time step. Finally, the output $y^{\prime }$ is parsed to obtain the target sentiment element. Specifically, we first determine whether there are multiple results based on the special token ‘[SSEP]’ and then parse the sentiment element for each result. If the output does not conform to the predefined output format, the sentiment element will not be fetched, representing a decoding failure, indicated by ’None’.

4. Experiments

4.1. Dataset

The current ABSA task contains four benchmark datasets, which include Laptop 14, Rest 14 [37], Rest 15 [38], and Rest 16 [39]. For each ABSA subtask, we use the datasets that have been partitioned in previous studies in this fully supervised learning study [7,24].

This study also utilizes instruction prompt for few-shot learning of ABSA tasks, which is a preliminary attempt. The ABSA task lacks a standard few-sample dataset. Since there are three different types of sentiment polarity, we randomly choose a small number of samples for few-shot learning so that the number of samples per sentiment polarity as much as possible satisfies the requirement of containing K samples.

This study also utilizes instruction prompt for few-shot learning of ABSA tasks, which is a preliminary attempt. The ABSA task lacks a standard few-sample dataset. Since there are three different types of sentiment polarity, we randomly choose a small number of samples for few-shot learning so that the number of samples per sentiment polarity as much as possible satisfies the requirement of containing K samples. The distribution of the few-sample datasets we constructed for the ASC task is shown in

Table 4. Number of sentences per dataset for ASC tasks.

(a) Distribution of the Laptop 14 and Rest 14 Few-Sample Datasets
	Laptop 14					Rest 14
	K = 5	K = 10	K = 20	K = 40	Full	K = 5	K = 10	K = 20	K = 40	Full
Train	20	35	68	129	1309	18	35	62	125	1796
Dev	24	33	64	123	150	23	31	67	128	180
Test	411					600
(b) Distribution of the Rest 15 and Rest 16 Few-Sample Datasets
	Rest 15					Rest 16
	K = 5	K = 10	K = 20	K = 40	Full	K = 5	K = 10	K = 20	K = 40	Full
Train	18	33	63	122	750	16	32	62	126	1114
Dev	23	37	62	82	82	19	36	65	118	118
Test	401					420

a,b. In the evaluation model stage, the same test set is used to evaluate few-shot learning and fully supervised learning.

4.2. Evaluation Metrics

We use F1 scores as the primary evaluation metric for all tasks. For compound tasks, a prediction is considered correct only when all sentiment elements are predicted correctly; otherwise, the prediction is considered incorrect.

4.3. Experiment Details

We employ the “t5-base” [36] pre-trained generative model. The optimizer is AdamW with a learning rate of 3e-5, and a linear learning rate warm-up strategy is applied to the learning rate simultaneously. Regarding the experimental parameters, the epoch is set to 5 in fully supervised learning and 15 in few-shot learning. In fully supervised learning the batch size is 16, and the maximum sequence length is set to 256. In few-shot learning with K = 5, 10, 20, and 40, we use batch sizes of 2, 2, 4, 8, respectively. The model decoding strategy uses greedy search. We use greedy search as the model decoding strategy.

4.4. Baselines and Models for Comparison

Most previous studies have treated ABSA tasks as natural language understanding tasks, solved in the pipeline method or end-to-end method. In this paper, we study the use of natural language generation to solve the ABSA task. We propose three prompt templates to explore the solution of few-shot learning on ABSA task. We choose Generative Aspect-based Sentiment analysis (GAS) and PARAPHRASE (PARA) model proposed by Zhang et al. [7,24] as comparison models. These two models use the same backbone model as we do, which ensures that the output is not influenced by the model size. Meanwhile, we choose the pipeline or end-to-end approach for testing.

The GAS model has two paradigms, namely the annotation-style method and the extraction-style method.

The PARA model is proposed to solve the Aspect Sentiment Quad Prediction (ASQP) task. We reformat the model inputs and outputs so that they can solve other ABSA tasks.

The selected ASC models include LCFS and A-ABSA.

LCFS-ASC [40]: LCFS-ASC is a sentiment classification model that uses dependent syntactic trees to determine local context.

A-ABSA [23]: A-ABSA allows to determine the context of aspect terms in an adaptive way and then to determine the aspect sentiment.

RACL and Dual-MRC are two of the chosen AESC models.

RACL [34]: Chen et al. proposed a Relation-Aware Collaborative Learning (RACL) framework that allows subtasks to work together in a stacked multilayer network through multitask learning and relational propagation mechanisms.

Dual-MRC [19]: Dual-MRC transforms the ABSA task into two machine reading comprehension problems.

The selected ASTE models include P-pipeline and Jet+BERT.

P-pipeline [2]: P-Pipeline uses a two-stage approach to extract aspect sentiment triples.

Jet+BERT [41]: Jet+BERT uses the BIOES annotation format to jointly extract triples from the text.

5. Results and Discussions

5.1. Main Results

The results of the ABSA task using fully supervised learning are shown in

Table 5. Results of fully supervised learning on Laptop 14 (a), Rest 14 (b), Rest 15 (c), Rest 16 (d). Each result in the table is the average of five consecutive runs. The best result for the current task is shown in bold. ‘-’ means no result.

(a) Laptop 14
Model	ASC	AESC	ASTE
LCFS	77.13	-	-
A-ABSA	77.3	-	-
RACL	-	63.4	-
Dual-MRC	-	65.94	-
Pipeline	-	-	42.87
Jet+BERT	-	-	51.04
GAS	86.36	66.72	53.48
PARA	86.31	68.93	59.75
IPT-a	85.53	70.8	61.21
IPT-b	86.36	69.05	60.42
IPT-c	86.56	70.36	60.45
(b) Rest 14
Model	ASC	AESC	ASTE
LCFS	80.31	-	-
A-ABSA	81.21	-	-
RACL	-	75.42	-
Dual-MRC	-	75.95	-
Pipeline	-	-	51.46
Jet+BERT	-	-	62.4
GAS	90.84	74.52	66.45
PARA	90.8	76.96	71.42
IPT-a	90.13	77.6	71.53
IPT-b	90.65	76.9	71.44
IPT-c	91.17	77.16	71.4
(c) Rest 15
Model	ASC	AESC	ASTE
RACL	-	66.05	-
Dual-MRC	-	65.08	-
Pipeline	-		42.87
Jet+BERT	-		51.04
GAS	90.57	64.45	53.48
PARA	91.79	66.96	59.75
IPT-a	91.48	67.98	61.21
IPT-b	91.74	67.87	60.42
IPT-c	91.94	68.47	60.45
(d) Rest 16
Model	ASC	AESC	ASTE
Pipeline	-	-	54.21
Jet+BERT	-	-	63.83
GAS	91	69.21	64.71
PARA	91.42	73.86	70.99
IPT-a	91.55	73.83	70.58
IPT-b	91.81	74.1	71.03
IPT-c	91.82	74.38	71.14

. It has been observed that constructing the target sequence in natural language form can effectively leverage the knowledge encoded in the pre-trained model and result in improved model performance. Our proposed three prompt templates allow for better utilization of the hidden knowledge stored in the pre-trained model, leading to a reduction in error during the fine-tuning stage compared to the pre-training stage. Our proposed method achieves the best performance on all tasks.

From the experimental results, our approach shows a significant improvement (up to 9.9%) over the pipeline approach or the end-to-end approach. This shows that the generative model has significant advantages for solving ABSA tasks.

On single tasks, the F1 scores remain almost identical across methods. On compound tasks, our method shows surprising advantages. In the AESC task, our method raises the F1 score on average by 4.08% and 1.13% in comparison to the GAS model and the PHARA model, respectively. In the ASTE task, our method shows improvements of 5.91% and 0.67% in F1 scores, on average, relative to the GAS and PHARA models, respectively.

As shown in Table 5, the model performs better on the Rest 14 dataset compared to the Laptop 14 dataset, indicating that it may lack expertise in the laptop domain. Restaurant domain data does not involve specialized knowledge, so the model performs better on restaurant data. On the compound task, the model performs worse on the Rest 15 dataset, with an average F1 score that is 10.03% lower than for other restaurant data. By looking at the Rest 15 dataset, we find that there are fewer sentences expressing ’neutral’ sentiment within the training set, and the model has difficulty learning ’neutral’ sentiment information accurately. Meanwhile, the sentences in the Rest 15 dataset involve multiple aspects and are relatively complex.

Prompt learning is a method to explore few-shot learning and zero-shot learning by leveraging the knowledge learned by the generative model during the pre-training stage. We apply prompt learning to three tasks for few-shot learning.

Table 6. Results of few-shot learning on Laptop 14, Rest 14, Rest 15, Rest 16. Each result in the table is the average of five consecutive runs. The best result for the current task is shown in bold.

Dataset	Task	Prompt Template	K = 5	K = 10	K = 20	K = 40
Laptop 14	ASC	IPT-a	73.8	77.67	80.08	83.19
		IPT-b	74.43	77.6	80.9	83.33
		IPT-c	75.32	78.52	81.26	84.73
	AESC	IPT-a	31.64	38.02	43.65	52.24
		IPT-b	26.31	32.09	40.55	47.47
		IPT-c	26.76	31.62	39.53	48.75
	ASTE	IPT-a	30.72	37.11	40.48	50.62
		IPT-b	30.83	35.24	39.25	49.15
		IPT-c	31.35	35.45	41.3	48.14
Rest 14	ASC	IPT-a	74.73	79.46	86.48	86.63
		IPT-b	76.6	81.8	87.76	88.05
		IPT-c	78.46	83.24	86.94	87.64
	AESC	IPT-a	44.79	50.71	58.27	61.96
		IPT-b	41.91	48.95	56.52	62
		IPT-c	44.81	47.97	57.48	62.52
	ASTE	IPT-a	44.52	48.08	54.51	58.54
		IPT-b	43.84	49.56	53.37	59.05
		IPT-c	43.66	49.64	53.64	59.25
Rest 15	ASC	IPT-a	73.65	79.03	83.48	89.34
		IPT-b	71.05	77.95	83.99	88.65
		IPT-c	70.96	76.89	84.41	87.37
	AESC	IPT-a	39.94	50.61	54.04	56.62
		IPT-b	35.54	45.8	49.01	53.19
		IPT-c	35.58	45.02	48.27	56.66
	ASTE	IPT-a	31.91	40.38	48.61	56.6
		IPT-b	34.11	39.1	47.18	57.26
		IPT-c	34.89	41.17	46.66	54.91
Rest 16	ASC	IPT-a	69.16	81.24	83.74	89.72
		IPT-b	69.49	81.52	84.74	88.87
		IPT-c	64.08	79.4	84.93	89.02
	AESC	IPT-a	41.61	52.01	55.87	60.64
		IPT-b	42.53	51.46	52.33	60.37
		IPT-c	40.94	50.36	54.17	60.68
	ASTE	IPT-a	37.11	43.55	53.32	62.78
		IPT-b	39.1	44.91	52.72	63.49
		IPT-c	39.46	45.06	54.13	63.51

displays the results of our proposed three prompt templates for few-shot learning.

In a single task, the model performs better on few-shot learning. On the compound task, the small number of samples makes it difficult for the model to learn its complex semantic information. This could be attributed to the fact that the model does not learn enough during the pre-training stage and is unable to effectively handle multiple tasks through fine-tuning with a limited number of samples. Table 6 demonstrates that the IPT-a and IPT-c prompt templates perform better. Compared with other prompt templates in AESC tasks, IPT-a on the Laptop 14 dataset improves the model performance by 4.35% on average. Overall, the IPT-a prompt template makes the model’s performance relatively stable across tasks.

Table 6 also shows that the performance of the model on the AESC and ATSC tasks varies by 2.17% at K = 5. As the value of K increases, the performance gap between the AESC and ATSC tasks becomes smaller. This indicates that as more data is provided, the model’s ability to learn various sentiment elements and the relationships between them improves, even for more complex tasks.

5.2. Error Analysis and Case Study

Prompt learning is proposed to perform few-shot learning and to solve natural language processing tasks in low-resource scenarios. The experimental results show that IPT-a performs best among the three prompt templates. The single task performs better with few-shot learning and mainly explores errors on the compound task. We analyze the errors that occur with few-shot learning on the Laptop 14 and Rest 14 datasets and classify the errors. In the experiment, we set K = 10 and take 100 sentence samples for each test set, respectively. We perform the training first and then the prediction. Other parameters are the same as described in Section 4.3.

The results of our test set indicate that the model is not effective in making judgments about “neutral” sentiment. The judgment of “neutral” sentiment is easily influenced by other sentiment words. This is a challenge for the ABSA task. We first analyze which error type occurs the most in the ABSA task and present the results in Figure 4. Based on the analysis, five different types of errors were identified and displayed in Figure 4a,c. The “a-error” error type refers to the aspect term that was incorrectly predicted; the “o-error” error type refers to the aspect term that is correctly predicted and the corresponding opinion term that is incorrectly predicted; the “s-error” error type refers to the aspect term that was correctly predicted and the aspect sentiment that is incorrectly predicted; the “no-pre” error type refers to the aspect term that is not predicted; the “par-error” error type refers to the model whose output distribution does not conform to a fixed form during the prediction stage. We observe in Figure 4 that the “no-pre” error type is a common error. Example 3 in Figure 4d is one of the many reasons for this error. The errors in aspect term prediction appear more frequently. This is mainly due to the fact that aspect terms are often composed of multiple words, such as in Example 2 in Figure 4b.

Since opinion terms in ASTE tasks are often not single words, such as the opinion term “not fix” in Example 1 in Figure 4d, it is easy to make mistakes. When the target sentiment element consists of multiple words, it can be challenging for the model to predict it accurately. The aspect term “drinks” in Example 2 of Figure 4d corresponds to the wrong opinion term and sentiment prediction. However, the sentiment expressed in the opinion term and the predicted sentiment are consistent. This indicates that the model has learned the correlation between opinion terms and aspect sentiment when extracting multiple sentiment elements.

We have shown great potential in exploring few-shot learning for ABSA tasks. It has been demonstrated that as the number of model parameters increases, the model’s capacity to learn and generalize, commonly known as “emergent capacity”, also improves [42]. The “emergent capability” of large-scale language models refers to their ability to perform tasks that are not available to small models, but can be unlocked as model size increases. The ability to perform tasks through zero-shot learning is an emergent phenomenon. We attempt to perform zero-shot learning for the ABSA task. We perform zero-shot learning exploration using the model of T5-11b, but the experimental results are disappointing. The recent introduction of ChatGPT has received a lot of attention from researchers. ChatGPT’s remarkable capability makes it possible to solve natural language processing tasks with zero training samples. We utilize ChatGPT to extract the sentiment elements from a given text, and the resulting output is depicted in Figure 5 below.

From the experimental results, it is clear that ChatGPT demonstrates better zero-shot learning ability in the ABSA task. However, it should be noted that the model is not open source and has limitations in various aspects.

6. Conclusions

In this paper, we use prompt learning to investigate three ABSA tasks. We introduce three types of instruction prompts for fully supervised learning and few-shot learning on the ASC, AESC, and ASTE tasks. Our proposed method exhibits superior performance compared to other state-of-the-art models in fully supervised learning across four benchmark datasets, while also demonstrating promising potential for few-shot learning. Our analysis of the types of model output errors shows that the ABSA task still has many outstanding issues to be addressed. We also perform zero-shot learning for the ASC task, but the results are disappointing. Zero-shot learning is still a huge challenge for ABSA tasks. To enhance the model’s semantic understanding of text, we will consider how to guide the model to delve deeper into its thoughts, focusing on fine-grained information. This can improve the model’s ability to address ABSA tasks under zero-shot conditions. Alternatively, considering transfer learning to enhance the model’s generalization ability can reduce the cost of manual annotation.