Fine-Grained Sentiment-Controlled Text Generation Approach Based on Pre-Trained Language Model

Abstract

Sentiment-controlled text generation aims to generate texts according to the given sentiment. However, most of the existing studies focus only on the document- or sentence-level sentiment control, leaving a gap for finer-grained control over the content of generated results. Fine-grained control allows a generated review to express different opinions toward multiple aspects. Some previous works attempted to generate reviews conditioned on aspect-level sentiments, but they usually suffer from low adaptability and the lack of an annotated dataset. To alleviate these problems, we propose a novel pre-trained extended generative model that can dynamically refer to the prompt sentiment, together with an auxiliary classifier that extracts the fine-grained sentiments from the unannotated sentences, thus we conducted training on both annotated and unannotated datasets. We also propose a query-hint mechanism to further guide the generation process toward the aspect-level sentiments at every time step. Experimental results from real-world datasets demonstrated that our model has excellent adaptability in generating aspect-level sentiment-controllable review texts with high sentiment coverage and stable quality since, on both datasets, our model steadily outperforms other baseline models in the metrics of BLEU-4, METETOR, and ROUGE-L etc. The limitation of this work is that we only focus on fine-grained sentiments that are explicitly expressed. Moreover, the implicitly expressed fine-grained sentiment-controllable text generation will be an important puzzle for future work.

1. Introduction

In recent years, Transformer-based pre-trained language models (LMs) have greatly improved the state-of-the-art of natural language processing tasks as well as natural language generation (NLG). Large-scale autoregressive Transformer models [1] that leverage large amounts of unannotated data and a simple log-likelihood training objective have achieved remarkable results in many text-generation tasks, such as machine translation, text summarization, and text style transfer. Meanwhile, for other real-world text-generation applications, such as review generation and essay writing, users prefer the generated text to be more controllable. However, since the LMs are trained on unannotated data, controlling attributes of generated text becomes difficult without modifying the model architecture to allow for extra input attributes or fine-tuning with attribute-specific data [2,3]. Therefore, some approaches, such as Plug-and-Play-Language-Models (PPLM) [4], control generated text through attribute models without changing the architecture or weights of pre-trained LMs. These models usually regard controllable text generation as generating tasks conditioned on the attributes, such as topic and sentiment at the sentence- or document-level, leaving a gap for finer-grained (e.g., aspect-level) control over the content of generated texts.

The fine-grained sentiment-conditioned text-generation task aims to automatically generate a highly relevant statement when given a series of fine-grained sentiments (e.g., aspect-opinion, aspect-sentiment) as input. Zang and Wan [5] first introduced the aspect-sentiment information to perform aspect-level sentiment-controllable review generation. They conducted conditional training by adopting a supervised method requiring a large dataset annotated with sentence-level aspect-sentiment labels. However, very few datasets provide such sufficient fine-grained labels, and it is also labor-intensive and time-consuming to conduct annotation on all data instances. Chen et al. [6] proposed a mutual learning framework leveraging large unlabeled data through interactive learning between the generator and the classifier. Besides the aspect-sentiment, aspect-opinion pairs also express aspect-level sentiment information. Therefore, inspired by them, in this work, we introduce the aspect-opinion information into the fine-grained sentiment-controllable text generation.

The aspect-opinion pairs represent the fine-grained sentiments that could be expressed within a review sentence, where the aspect term refers to the target of an opinion, and the opinion term refers to the sentimental words that describe the aspect term. For example, in the sentence of Figure 1, (”hotdog”, “better”) is an aspect-opinion pair, where “hotdog” is an aspect term, and “better” is an opinion term, together they form the backbone of fine-grained sentiment in the review text. Therefore, the aspect-opinion conditioned generation task aims to generate a review text X that correctly contains the sentiment information from n non-repeated aspect-opinion pairs (a, o${)}_{1:n}$. Most previous works [5,7,8] used the aspect-polarity pairs rather than the aspect-opinion pairs, and they used a straightforward data-to-text modeling approach, which is much more difficult due to the discrete and sparsity of the input data. To tackle this problem, relying on the natural characteristics of aspect-opinion pairs directly presented in sentences, our approach proposed a query-hint mechanism as a dynamic prompt strategy to guide the generation direction. Furthermore, in order to guarantee the quality of the generated results, in the generator, we incorporate a GPT-2 345M model [9] as the “super generator,” then by extending this state-of-the-art model with our proposed query-hint mechanism and our sentiment control loss function to guide the generating process toward the given controlling information. Moreover, to further enhance the generator’s performance, with the assistance of a classifier by extracting the fine-grained sentiments, we leveraged a large unlabeled dataset to train the generator. The experimental results demonstrate the effectiveness of these components.

Our Contributions:

• We propose our conditional generative model by extending a pre-trained state-of-the-art Transformer-based generative model with our introduced query-hint mechanism and sentiment control loss function to further guide the text generation at a finer-grained level.
• To better model a text-to-text schema, we introduce the aspect-opinion pair as the fine-grained sentiment unit to control the constrained text generation.
• Through employing an auxiliary classifier, we leverage a large unannotated dataset to re-train and fine-tune an end-to-end conditioned text generative model.

The remainder of this paper is organized as follows. Section 2 discusses the related works in controlled text generation, including the review generation and the aspect-level sentiment-controlled generation, which is less studied. Section 3 introduces our proposed approach that achieved finer-grained sentiment control in generation. In Section 4, the experimental settings are detailed, and evaluation metrics and results are also discussed to demonstrate the validity of our approach. Finally, we conclude this work in Section 5 while discussing future work.

2. Related Work

2.1. Controlled Text Generation

Recently, there has been many studies that aim to generate text conditioned on input attributes with neural networks. Some of the earlier efforts have studied this controlled text generation by training a conditional generative model [10,11] while fine-tuning pre-trained models with Reinforcement Learning (RL) [3] and training a Generative Adversarial Network [12] have also shown inspiring results. The Conditional-Transformer-Language (CTRL) model [2] is a recent approach that trains a language model conditioned on a variety of control codes (e.g., “Reviews” and “Legal” control the model to generate reviews and legal texts, respectively), which prepended meta-data to the text during generation. Although it uses a GPT-2-like architecture to generate high-quality text, the result is at the cost of fixing the control codes and training a very large model. PPLM [4] composed a pre-trained LM with attribute controllers guiding text generation toward the desired attribute. At the same time, its flexible design allows it to control the generating process through relatively small “pluggable” attribute models while keeping parameters in the LM fixed. Chan et al. [13] incorporated a pre-trained GPT-2 model with a Content-Conditioner (CoCon) to control the generated text under the guidance of target text content. Yu et al. [14] proposed a simple and flexible method, infusing attribute representations into a pre-trained unconditional LM without changing the LM parameters to achieve sentiment- and topic-controlled generation. Different from our fine-grained sentiment-controlled text-generation (FSCTG) task, these works focus on sentence-based sentiment and topic control in text generation. In the FSCTG task, the text-generation process is controlled by a series of fine-grained sentiments (e.g., aspect-opinion or aspect-sentiment).

2.2. Review Generation

Review generation [7,15], a generation task aiming to automatically generate review text, is a related area that generates reviews conditioned on the given information. While most of the previous approaches [7,8] have framed review generation as A2T (Attribute-to-Text problem), leaving a gap between attributes (e.g., user, product, and rating) and linguistic data. To tackle this problem, Kim et al. [16] proposed AT2T (Attribute-matched-Text-to-Text) by augmenting inductive biases of attributes with matching reference reviews to learn the rich representations of attributes.

2.3. Aspect-Level Sentiment Control

Nevertheless, most of these works only focus on sentence-level sentiments and ignore the aspect-level sentiment control, and very few researchers have studied generating reviews from fine-grained sentiments due to the lack of announced data. Zang and Wan [5] gave the first attempt to generate reviews from aspect-sentiment scores, which requires the reviews with sentence-level aspect-sentiment score annotations. This makes it impractical in real-world applications due to the lack of labeled data. To tackle this problem, Chen et al. [6] proposed a semi-supervised aspect-level sentiment-controllable review generation method, under their proposed mutual learning framework with the assistance of a classifier, it can take advantage of large-scale unlabeled data to achieve aspect-level sentiment control in review generation with few labeled data. Fei et al. [17] combined fine-grained sentiment classification and generation tasks as a joint dual learning system, strengthening the mutual connection of both tasks. To overcome the defect of sparsity and discrete nature brought by the input data in the data-to-text scheme, Yuan et al. [18] proposed a hierarchical template-transformer (HTT); they split the generation task into two corresponding pipeline subtasks, i.e., opinion phrase generation and review composition, which were jointly trained on the HTT. Although in different ways, they all trained an efficient end-to-end generative model. However, they did not attempt to dynamically adjust the attention weights during the model’s generation process since some contents (e.g., the completion of sentiment words generation) are informative to the global generation and need to be notified.

3. Method

In this section, we introduce our fine-grained sentiment-controllable text-generation task together with a conditional generative model named Aspect-level Sentiment Conditioner (AlSeCond), which was trained with both labeled and unlabeled data to learn a fine-grained sentiment review generator with the assistance of a classifier.

First, we give the formalization of our fine-grained sentiment-controllable text-generation task. Specifically, given the fine-grained sentiment units (i.e., aspect-polarities or aspect-opinions) as the input s, the model generates a target text X that covers the input sentiments. As a straightforward approach, as other studies have used [5,7,8], the data-to-text modeling can be much more difficult when compared with the text-to-text modeling due to the discrete and sparsity of the input data [17]. Therefore, in this work, we consider a translation of this task to the text-to-text formulation. More conveniently, given aspect and polarity, it is effortless to retrieve opinion phrases from aspect sentiment triplets (AST [19], i.e., the triplet of aspect, opinion, and sentiment polarity) extracted from the review text. This work, therefore, set s = {$(a_1,o_1)$, $(a_2,o_2)$, …, $(a_n,o_n)$} and aims to generate a review text X comprising m words (X = {$x_1$, $x_2$, …, $x_m$}), which presents each aspect phrase $a_i$ and its corresponding opinion phrase $o_i$ ($i\in \{1,2,\dots ,n\}$) properly.

In this task, we have a labeled dataset L and an unlabeled dataset U. In the labeled dataset L, each labeled datum $\ell \in L$ comprises a review text and a list of aspect-opinion phrase pairs s, i.e., $\ell =\langle X,s\rangle$, while in the unlabeled dataset U, each u ∈ U only contains a review text, i.e., u = 〈X〉.

In the following subsections, we first introduce our main framework for how to train a generator on both labeled and unlabeled datasets. Then, we explain our generator and classifier in detail.

3.1. Main Framework

To make full use of both the limited labeled dataset and the large unlabeled dataset, inspired by Chen et al. [6], in the case of a text generator G, our proposed method additionally employs a sentiment classifier C, which is incorporated to extract all aspect sentiment triplets (aspect, opinion, polarity) in each sentence through a sequence-labeling schema, thus yielding pseudo labels for the unlabeled dataset. We assume that the generator can enhance itself by leveraging a large dataset with pseudo labels predicted by the classifier.

In order to benefit from both the data size of the unlabeled dataset and the correctness of the labeled dataset, we train our model sequentially using these two datasets. Specifically, as shown in Figure 2, following Chen et al. [6], we adopt three steps to make full use of the large unlabeled dataset:

• Step 1: We train both our generator and classifier on a limited labeled dataset to get G0 and C0, respectively.
• Step 2: The C0 is then used to extract the fine-grained sentiments in the large unlabeled dataset, thus yielding the pseudo labels for the next step’s training.
• Step 3: Again, the generator is trained on the unlabeled dataset that is attached with pseudo labels. Finally, the generator is fine-tuned with the labeled dataset (used in Step 1) to receive the final generator G1.

As a result, we obtain an enhanced generator G1 trained on both the limited labeled dataset and the large unlabeled dataset.

3.2. Generator

Unconditional language models (LMs) are trained on the huge amount of unlabeled text data to optimize the probability of p($x_i$|$x_1$:$x_{i-1}$) in an auto-regressive manner [20,21] where $x_i$ is the next token and $x_1$:$x_{i-1}$ are the previous tokens. While in the controlled text generation, the conditional distribution p($x_i$|a, $x_1$:$x_{i-1}$) is optimized, where a is the attribute for the model to control the generation.

To make use of the LM pre-trained with large unlabeled datasets, we need to infuse attribute a into the unconditional distribution p($x_i$|$x_1$:$x_{i-1}$). What is more, the pre-trained Transformer-based language model GPT-2 [9] has demonstrated remarkable natural text generation in an auto-regressive manner in recent years. Thereby, to improve the generated texts’ quality, our generative model incorporates a pre-trained GPT-2 model as the “super-generator,” and we further use the fine-grained sentiment infusion blocks, which are stacked in the AlSeCond to extend this pre-trained state-of-the-art language model’s decoder blocks.

Essentially, the GPT-2 model is stacked with numerous Transformer-Decoder blocks, each consisting of layer normalization [22], multi-head self-attention [1], and position-wise feed-forward operations. Therefore, our AlSeCond blocks extend this kind of decoder block and incorporate a sentiment infusion operation together with our proposed query-hint mechanism to conditionally infuse the fine-grained sentiments into the next-token prediction process.

The sentiment infusion operation is performed inside the AlSeCond’s blocks. Figure 3 briefly illustrated how our AlSeCond model works. Specifically, the target fine-grained sentiment pairs s0 are appended sequentially as a prompt to the head of the regular sequence s1 to form the S. This special appended sequence S is then encoded to h ($h=[h^0;h^1]$, $h^0$, $h^1$ is the hidden representation of s0 and s1, respectively) through numerous AlSeCond blocks, thus $h_t^1$ self-attends to the hidden states of the regular sequence $h^1$ for previous t time steps and, further, all time steps of the fine-grained sentiment pairs $h^0$. Therefore, the sentiment representation $h^0$ is infused into the intermediate representation $h^1$ to control the next token logits (o) and hence the generation process.

Our AlSeCond’s block (illustrated in the pink block in Figure 4) is a special Transformer-Decoder block that incorporates our proposed query-hint mechanism to guide the controlled generation process. Specifically, for fine-grained sentiment-appended hidden states, $h=[h^0;h^1]$ ($h^0$ and $h^1$ are the hidden states for the sentiment and regular sequence, respectively.), its key, value, and a special hinted query matrix ($K,V,Q^{\prime }\in R^{(l_s+t)\times d}$, $l_s$, t is the length of the appended sentiments and regular sequence, respectively) are computed to perform a query-hinted self-attention. Furthermore, during the computation of the hinted query ($Q^{\prime }$) matrix, we infuse $K^0\in R^{l_s\times d}$, the sentiments’ part of K, into $Q^1\in R^{t\times d}$ at their corresponding time step as the query-hint:

$$\begin{array}{cc}\hfill & Q=[Q^0;Q^1]=h\times W_q^T,K=[K^0;K^1]=h\times W_k^T\hfill \\ \hfill & Q^{\prime }=[Q^0;{Q^{\prime }}^1],{Q^{\prime }}^1=f_{hint}(K^0,Q^1)\times W_{q^{\prime }}^T\hfill \end{array}$$

(1)

$$f_{hint}(K^0,Q^1)=Q^1+M_h\times \left[\begin{array}{c}Mean\left(K_{0:l_1}\right)\hfill \\ Mean\left(K_{l_1:l_2}\right)\hfill \\ \cdots \hfill \\ Mean\left(K_{l_{n-1}:l_n}\right)\hfill \end{array}\right]$$

where $f_{hint}(·)$ is our proposed function, it strategically allocated the sentiments’ representation to $Q^1$ as the query-hint information, and $M_h\in R^{t\times n}$ is an adjacency matrix, representing which sentiment pair should be hinted for each time step in $Q^1$, and n is the number of sentiment pairs, $l_a$ ($a\in \{1,2,\dots ,n\}$) is the end index of the a-th sentiment pair in S. As a result, we guide the text generation by infusing the sentiment information into the generation process through the query-hinted self-attention operation.

3.3. Query-Hint Mechanism

Since the distance from the prompt and the next-token prediction correlates negatively with the prompt’s influence [23], which makes it difficult to use a prompt to guide a non-adjacent piece of text, especially when the generation time step is far away from the prompt. In other words, prompt and regular sentences share equal importance, which is inadequate for prompt-based generative models because the prompt tokens propagate less dominant information to the next-token prediction as the sequence expands. Our idea is similar to Xia et al. [24], where the actual importance of information from different sentiment units is unequal to each token in a sentence, so they need to be attended to differently. Therefore, as mentioned in Section 3.2, we introduced a query-hint mechanism to further remind each generation time step about the following content. The main idea of this mechanism is to let the generation process understand what text to generate in order to catch the next sentiment text.

Specifically, for each general sentiment pair, its aspect and opinion phrases have their own corresponding subsequence to provide query-hints. As shown in Figure 5 (e.g., 1 to 1), a sentiment pair’s member starts query-hint at the beginning of the sentence or the end step of the previous sentiment pair and closes before its own full-presenting. The hinted steps form a “hint-unit” (framed in the red dotted line in Figure 5).

In the source sentences, however, there are also some sentiment pairs that share the same phrase either in aspect or opinion (e.g., (food, great), (drinks, great)). Therefore, in order to make query-hint consistent in the training and generation process, given n sentiment pairs that share the same aspect/opinion phrase, their query-hints are merged into one “hint-unit”. As shown in Figure 5 (e.g., 1 to n), within the “hint-unit”, each aspect/opinion phrase gives the query-hint sequentially.

Although our proposed strategy of query-hint in the training process is almost identical to the generation process, there is still a slight difference between them. During the training process, the corresponding time steps in the sentence are provided with query-hint according to the position of each sentiment information presented in the sentence. While in the generation process, since the part of the sentence that has not been generated is unknown, query-hint should be allocated according to the generated part of the sentence.

3.4. Loss Functions

Generation loss function: through an LM training objective, we train our conditional generative model with the general generating loss term conditioned on previous $x_{:t-1}$ and input sentiment information s:

$${\mathcal{L}}_G=-\sum _{t}log{\left[p\left({x_t}^{\prime }\right|s,x_{:t-1})\right]}_{I^x\left(x_t\right)}$$

(2)

where ${x_t}^{\prime }$ is the predicted token at time step t. $I^x(·)$ is the index function of a vector.

Sentiment control loss function: To encourage the generator to output texts incorporating the input sentiment information (phrases), we train the generator additional with our proposed sentiment-control loss function. The main idea of this loss function is to maximize the probability value of the one with the highest probability in terms of given aspect/opinion word from all the next-word predictions of a sentence. Specifically, for every aspect phrase a and opinion phrase o presented in the source text, the training loss is defined as:

$$\begin{array}{cc}\hfill & {\mathcal{L}}_{Senti}={\mathcal{L}}_a+{\mathcal{L}}_o\hfill \\ \hfill & {\mathcal{L}}_a=-\sum _a\sum _{t}log{\left[Q(x^{\prime },Mas{k}_{a,t})\right]}_{I^x\left(x_{a,t}\right)}\hfill \\ \hfill & {\mathcal{L}}_o=-\sum _o\sum _{t}log[Q{(x^{\prime },Mas{k}_{o,t)}]}_{I^x\left(x_{o,t}\right)}\hfill \\ \hfill & \begin{array}{cc}\hfill Q(x,Mask)& =Mask\odot p_{max}\left(x\right)+(1\oplus Mask)\times {\varphi }_{mean}\hfill \end{array}\hfill \\ \hfill & p_{max}\left(x\right)=MaxPooling\left([p\left(x_1\right|s,x_{:0});p\left(x_2\right|s,x_{:1});\cdots ;p\left(x_t\right|s,x_{:t-1})]\right)\hfill \end{array}$$

(3)

where ${\mathcal{L}}_a$ and ${\mathcal{L}}_o$ are the losses for aspect and opinion term inclusion, respectively. $Mas{k}_{a,t/o,t}$ is a one-hot vector with the size of $\mathcal{V}$ (vocabulary size), and only the element in the index of $a_t/o_t$ is 1. ${\varphi }_{mean}$ is a hyper-parameter controlling how much the prediction of aspect/opinion terms should be enhanced. $p_{max}(·)$ is a max-pooling operation with a kernel size of $l_t\times 1$ ($l_t$ is the length of the target text). ⊙ and ⊕ represent the element-wise product and XOR, respectively.

As a result, our final loss function comprehensively considers the loss of generation quality and the loss of sentiment control:

$${\mathcal{L}}_{total}={\lambda }_G{\mathcal{L}}_G+{\lambda }_{Senti}{\mathcal{L}}_{Senti}$$

(4)

where $\lambda$ values are hyper-parameters controlling how much the loss terms dominate the training.

3.5. Classifier

In this section, first, we give the task definition of Aspect Opinion Pair Extraction (AOPE), then we briefly introduced the model architecture of our sentiment classifier C.

The task of AOPE aims to extract aspect terms and their corresponding opinion terms as pairs [25,26,27]. This task can be defined as follows: Given a sentence with m words $X=\{x_1,x_2,\dots ,x_m\}$, the goal of this task is to extract all aspect-opinion pairs $\tau ={\left\{{(a,o)}_n\right\}}_{n=1}^{\left|\tau \right|}$ from X, where $\left\{{(a,o)}_n\right\}$ is an aspect-opinion pair presented in X and the notations a and o denote an aspect term and an opinion term, respectively.

For the overall architecture of our classifier, the two-dimensional interaction-based multi-task learning framework (2D-IMLF) is shown in Figure 6. Given an input sentence, two highly related works of the extraction task (aspect term extraction and opinion term extraction) are adopted to learn aspect-related and opinion-related features, respectively. Then, to capture different interactive features of aspect terms and opinion terms, a 2D interactive representation is obtained by tensor composition. Finally, the classifier model regards the AOPE task as a grid tagging problem and in the end, obtains the final results by applying a decoding algorithm [28].

As shown in Figure 6, we first use a group of CNN layers to encode the input sentence and get their hidden state:

$$\begin{array}{cc}\hfill & H_k^c=Conv1D_k\left(X\right)\hfill \\ \hfill & H_{*}^c=[H_1^c;H_2^c;\dots ;H_k^c]\hfill \\ \hfill & H^c=Conv1D_3\left(Conv1D_5\left(H_{*}^c\right)\right)\hfill \end{array}$$

(5)

where $k\in \{1,2,3,\dots \}$ represents the kernel size of an 1D-CNN. Then, a Bi-LSTM layer together with multi-head self-attention is incorporated to extract the context information from the sentences:

$$\begin{array}{cc}\hfill & H_t^l=BiLSTM(H_{t-1}^l,H_t^c)\hfill \\ \hfill & H_c=MultiHeadAttention\left(H^l\right)\hfill \end{array}$$

(6)

Afterward, we concatenate the hidden state $H_c$ with their transferring state $H_c^T$ to get a grid-formed feature. We then obtain the prediction probabilities of $P_a^c$ and $P_o^c$ for aspect and opinion terms, respectively, from the final logits P:

$$\begin{array}{cc}\hfill & {\widehat{O}}_c=[H_c;H_c^T]\hfill \\ \hfill & P=Linear\left({\widehat{O}}_c\right)\hfill \end{array}$$

(7)

Finally, by using a grid-formed tagging schema [28], we can easily obtain a series of aspect-opinion pairs.

4. Experiments

In this section, we first introduce datasets and settings in our experiment and then report the evaluation metrics and results.

4.1. Dataset and Settings

We conduct experiments on three real-world datasets, two labeled and one unlabeled; the statistics of the datasets are reported in

Table 1. Statistics of the labeled and unlabeled datasets. Note that “Val” is short for “Validation”, the ASTE-Data-V2-Rest is labeled with aspect, opinion, and polarity, while the MAMS-ASTA is labeled with only aspect and polarity.

Dataset		#Instance	#Positive	#Neutral	#Negative	Sentiment Form
ASTE-Data-V2-Rest	Train	2728	3490	241	1014	Aspect-Opinion-Polarity
	Val	668	841	76	248
	Test	1140	1497	120	376
MAMS-ASTA	Train	4297	3380	5042	2764	Aspect-Polarity
	Val	500	403	604	325
	Test	500	400	607	329
Yelp	-	1,160,546	-	-	-	-

. Moreover, the experimental settings are also listed in this subsection.

4.1.1. Labeled Dataset

We conduct experiments of aspect-opinion and aspect-polarity pairs of conditioned controllable text generation on English restaurant reviews with ASTE-Data-V2 from Xu et al. [29] and MAMS-ASTA from Jiang et al. [30], respectively.

ASTE-Data-V2 (https://github.com/xuuuluuu/SemEval-Triplet-data, accessed on accessed on 18 May 2022): From Xu et al. [29], is originally from SemEval Challenges [31,32,33], and contains both aspect and opinion labels in each review datum. Specifically, we union the 14Rest, 15Rest, and 16Rest included in the ASTE-Data-V2 as our labeled dataset.

MAMS-ASTA: From MAMS (https://github.com/siat-nlp/MAMS-for-ABSA, accessed on accessed on 14 May 2022) (Multi-Aspect Multi-Sentiment), ref. [30] is an aspect-level sentiment-labeled dataset. Wherein, each datum instance in MAMS-ASTA is labeled with at least two aspects and different sentiment polarities, while no opinion term is labeled. Therefore, by using our classifier to retrieve opinion phrases according to the original pairs of aspect-polarity, we also conduct aspect-level sentiment-controllable text generation on MAMS-ASTA.

4.1.2. Unlabeled Dataset

To ensure that the training data are in the relevant review domain, we use Yelp’s review dataset (https://www.kaggle.com/yelp-dataset/yelp-dataset, accessed on accessed on 18 May 2022) as the unlabeled dataset and filter out the sentences with a length greater than 150. Unlike the labeled datasets, the Yelp dataset did not contain fine-grained sentiment labels. Therefore, we only use the sentences in the unlabeled data and discard other items, including user information.

4.1.3. Experimental Settings

Generator: In the experiment, we train our AlSeCond model that extends from a pre-trained GPT-2 medium 345M model [9]. The AlSeCond’s blocks clone the GPT-2 Transformer blocks’ parameters and settings. To ensure the generator can compute the probability of (and also generate) any string, we apply Byte Pair Encoding (BPE) [34] for the inputs. The max generating length was set to 32. We tune the ${\lambda }_G$ together with ${\lambda }_{senti}$ to 1 and 8, respectively. Adam [35] is used for optimization, while the batch size is set to 16, and the learning rate is set to $5\times {10}^{-5}$. During the period of G0, the generator is trained with the labeled and pseudo-labeled dataset for 4 and 2 epochs, respectively. In the following G1, the generator is fine-tuned with the labeled dataset for 24 epochs. We apply the above steps to train our model on an RTX A4000 GPU for 20 h. Furthermore, the above steps are also applied to train other baseline models. We ran our model and all baselines five times to average the scores.

Classifier: Following GTS [28], we combine a 300-dimension domain-general embedding from pre-trained GloVe [36] and a 100-dimension domain-specific embedding trained with fastText [37] to initialize double word embeddings. We use Adam as the optimizer, and the learning rate is $5\times {10}^{-4}$. The batch size and dropout rate are set to 32 and 0.5, respectively. The number of hidden units in Bi-LSTM is set to 128.

4.2. Baselines

We compare with six baselines. PPLM [4] incorporates an attribute model BoW (bag of words) to steer a pre-trained GPT-2 model toward increasing the generating probability of the target words. In this baseline, the BoW is formed with the words contained in the target sentiment pairs. For HTT [18], we omit the process of opinion phrase generation and only use its results (i.e., sentiment pairs) to compose the review. Through prepending the task description before the input text, the state-of-the-art text-to-text model T5 [38] is pre-trained with a multi-task objective. Following this schema, we append the sentiment pairs into the prompt, thus forming: “generate a sentence with $a_1$ is $o_1$, …, $a_n$ is $o_n$.”, and fine-tune the model with the target sentence. Its coverage of the input sentiment pairs in the baselines serves as an upper bound. Moreover, we also fine-tune UniLM [39], UniLM-v2 [40], and BERT-Gen [40] in a similar sequence-to-sequence fashion with both the large unlabeled dataset and the limited labeled dataset.

4.3. Generated Quality Evaluation

To study the performance of these models in a diversified manner, we conduct evaluations on both the quality and sentiment coverage of the generated text.

4.3.1. Fluency and Diversity Evaluation

We conduct a fluency evaluation on the generated texts with some automatic metrics: BLEU [41], ROUGE [42], and METEOR [43], which compare the similarity between the generated text and ground truth based on n-gram matching. Moreover, the diversity of generations is also an important indicator. We measure diversity for the generated results with Dist-1,-2,-3 [44] scores and Self-Bleu [45].

Table 2. Results for the fluency and diversity evaluation. Note that “↑” means the higher the better, “↓” means the lower the better, “w/o” means “no”.

Dataset	Models	BLEU-3 (↑)	BLEU-4 (↑)	METETOR (↑)	ROUGE-L (↑)	Self-Bleu-4(↓)	Dist-1 (↑)	Dist-2 (↑)	Dist-3 (↑)
ASTE-Data-V2	PPLM	0.196	0.032	14.078	13.827	7.939	0.0841	0.4102	0.7180
	HTT	13.100	7.656	34.899	42.544	42.664	0.0525	0.2356	0.4113
	T5-base	21.246	13.216	29.007	41.092	22.580	0.1621	0.4725	0.6101
	T5-large	24.747	16.462	29.986	43.614	23.045	0.1721	0.4658	0.5934
	UniLM	33.093	27.486	46.808	52.582	20.334	0.1489	0.4961	0.6663
	BERT-Gen	32.693	28.050	45.223	45.162	24.149	0.1450	0.4957	0.6411
	UniLM-v2	32.159	27.525	45.107	44.514	22.830	0.1451	0.5060	0.6553
	AlSeCond	40.453	34.611	55.127	63.720	15.972	0.1610	0.5439	0.7073
	⌊ w/o sentiment loss	37.961	32.190	55.699	62.911	16.195	0.1552	0.5301	0.7028
	⌊ w/o query-hint	34.305	29.080	55.391	61.237	14.442	0.1551	0.5431	0.7264
	⌊ w/o unlabeled dataset	29.085	26.387	42.601	48.213	21.727	0.1444	0.4942	0.6628
MAMS-ASTA	HTT	2.279	0.412	17.193	23.197	51.373	0.0602	0.2271	0.4003
	T5-base	3.653	1.479	14.400	24.181	27.671	0.1299	0.3761	0.5541
	T5-large	4.212	1.767	15.180	25.828	27.626	0.1418	0.3761	0.5591
	UniLM	3.178	1.251	18.833	23.872	37.890	0.1032	0.3211	0.4878
	BERT-Gen	4.003	1.605	17.751	24.162	28.284	0.1284	0.4024	0.5778
	UniLM-v2	3.898	1.559	17.757	23.999	27.858	0.1255	0.3989	0.5796
	AlSeCond	5.159	2.113	19.736	31.738	13.714	0.1627	0.5085	0.6811
	⌊ w/o sentiment loss	4.944	1.999	23.734	31.302	14.112	0.1477	0.4978	0.7171
	⌊ w/o query-hint	4.208	1.635	23.661	29.497	10.835	0.1604	0.5538	0.7653
	⌊ w/o unlabeled dataset	3.458	1.026	20.761	28.924	15.787	0.1478	0.4728	0.6627

shows the fluency and diversity evaluation results by the automatic evaluations. From the results, we can observe that: (1) Compared with baseline models, our AlSeCond model extended from the GPT-2 achieves better performance in fluency evaluations. (2) Comparing results in diversity metrics, it can be observed that our AlSeCond model performs much better than the rest of the baselines in the MAMS-ASTA dataset, which means the results generated by our model are less like the template-generated text than that generated by other models.

4.3.2. Sentiment Evaluation

As to measure the quality of sentiment containment in the generated sentence and indicate whether the input sentiments are correctly expressed in the generated text, we employ two metrics: Coverage (Cov.), just like in Lin et al. [46], which is the average rate of input sentiment pairs presented in the generated texts. This metric includes Cov-a, Cov-o, and Cov-ao, representing the presenting rate of aspect, opinion, and aspect-opinion pairs, respectively. Accuracy (Acc.) is a rate indicating how many fine-grained sentiments are accurately expressed in the sentence, and it is evaluated by the external sentiment classifier [30] trained on MAMS-ASTA.

Table 3. Results for the sentiment evaluation. Note that Accuracy (Acc.) is a rate indicating how many fine-grained sentiments are accurately expressed in the sentence, and it is automatically evaluated by an external classifier.

Dataset	Models	Cov-a	Cov-o	Cov-ao	Acc.
ASTE-Data-V2	PPLM	0.3597	0.3642	0.1094	0.1761
	HTT	0.7689	0.7773	0.6050	0.6328
	T5-base	0.9563	0.9764	0.9403	0.7812
	T5-large	0.9633	0.9839	0.9508	0.7948
	UniLM	0.9513	0.9568	0.9182	0.7450
	BERT-Gen	0.9352	0.9343	0.8886	0.7521
	UniLM-v2	0.9438	0.9488	0.9087	0.7475
	AlSeCond	0.9824	0.9849	0.9734	0.7771
	⌊ w/o sentiment loss	0.9633	0.9649	0.9468	0.7683
	⌊ w/o query-hint	0.9412	0.9313	0.8966	0.7443
	⌊ w/o unlabeled dataset	0.8158	0.8841	0.7556	0.6306
MAMS-ASTA	HTT	0.7203	0.5123	0.3800	0.4532
	T5-base	0.9610	0.9147	0.9042	0.5734
	T5-large	0.9738	0.9453	0.9416	0.5698
	UniLM	0.9251	0.7821	0.7590	0.5883
	BERT-Gen	0.9438	0.8009	0.7807	0.6048
	UniLM-v2	0.9341	0.7515	0.7305	0.6310
	AlSeCond	0.9798	0.9588	0.9558	0.6267
	⌊ w/o sentiment loss	0.9318	0.8952	0.8825	0.6050
	⌊ w/o query-hint	0.8338	0.6811	0.6257	0.5447
	⌊ w/o unlabeled dataset	0.7829	0.7095	0.6325	0.5157

shows the results of sentiment coverage and accuracy for generated texts. It is worth noting that for a linguistically complicated sentence, its aspect-level sentiments are more difficult to be correctly predicted by the external classifier than a relatively simple sentence, so its sentiment accuracy may be lower than the actual situation. What is more, T5’s original seq2seq architecture allows it to generate texts that highly correspond to the input sequences. Hence its coverage and accuracy scores serve as an upper bound, although its generated results’ syntax is relatively simple and repetitive.

Comparing the above metrics results for all models on different datasets, we can observe that our model has stable advantages over both ASTE-Data-V2 and MAMS-ASTA, which indicates that our AlSeCond model has stronger adaptability. Additionally, Figure 7 presents the learning curves for fine-tuning all models with the labeled dataset, which also demonstrates the strong capabilities of our model compared to baselines.

4.4. Case Study

Figure 8 presents some generated cases from AlSeCond, HTT, T5, UniLM, BERT-Gen, and UniLM-v2. From the cases, we found that: AlSeCond tends to generate more linguistically complicated sentences, while the other baselines are more likely to focus on generating review texts that simply express the input information and less on the complexity of the expressions and the syntaxes.

5. Conclusions and Future Work

In this paper, we propose a fine-grained sentiment-controllable text-generation method based on the pre-trained language model and the auxiliary sentiment classifier that utilizes both the labeled and unlabeled dataset to reach the aspect-level sentiment control in text generation. Our proposed query-hint mechanism and fine-grained sentiment control loss function have greatly enhanced the generator in controlling the sentiment during the text-generating process. Experiments on real-world datasets have demonstrated our generator’s ability to generate aspect-level sentiment-controllable review statements with high quality and diverse syntax.

For future work, we will explore the controllable text generation for implicitly expressed fine-grained sentiments (e.g., in this sentence: “We had to constantly ask the waiter to top up water glasses.”, the reviewer had a negative opinion of the waiter although there is no related opinion phrase in the sentence.), since the query-hint mechanism proposed in this paper is only effective for explicitly expressed fine-grained sentiments.