IER-SMCEM: An Implicit Expression Recognition Model of Emojis in Social Media Comments Based on Prompt Learning

Abstract

Financial text analytics methods are employed to examine social media comments, allowing investors to gain insights and make informed financial decisions. Some emojis within these comments often convey diverse semantics, emotions, or intentions depending on the context. However, traditional financial text analysis methods relying on public annotations struggle to identify implicit expressions, leading to suboptimal performance. To address this challenge, this paper proposes an implicit expression recognition model of emojis in social media comments (IER-SMCEM). Firstly, IER-SMCEM innovative designs a data enhancement method based on the implicit expression of emoji. This method expands the pure text financial sentiment analysis dataset into the implicit expression dataset of emoji by homophonic replacement. Secondly, IER-SMCEM designs a prompt learning template to identify the implicit expression of emoji. Through hand-designed templates, large-scale language models can predict the true meaning that emojis are most likely to express. Finally, IER-SMCEM recovers implicit expression by choosing the predictions of models. Thus, the downstream financial sentiment analysis model can more precisely realize the sentiment recognition of the text with emoji by the recovered text. The experimental results indicate that IER-SMCEM achieves a 98.03% accuracy in semantically recovering implicit expressions within financial texts. In the task of financial sentiment analysis, the sentiment analysis model achieves the highest accuracy of 3.99% after restoring the true implied expression of the texts. Therefore, the model can be effectively applied to sentiment analysis or quantitative analysis.

1. Introduction

In recent years, financial text analytics has been extensively applied in the financial sector, including risk prediction, investment decision-making, and stock market trading [1,2,3,4]. As illustrated in Figure 1, the Stocktwits platform shows that on 19 September 2024, the share price of Avalanche Corporation (AVAX.X) experienced a significant increase [2]. Notably, the platform data indicate a strong positive correlation between stock price changes, investor sentiment, and overall chatter [3]. Consequently, financial text analytics can assist investors in more accurately predicting stock price trends by analyzing sentiment, offering substantial support for investment decisions.

Social media platforms have given rise to a new pattern of communication where market participants increasingly share their views and sentiments through emojis, as illustrated in

Table 1. Implicit expressions of emojis in financial commentary.

Content	Emojis	Semantics
Just bought the dip . Time to HODL and watch it go . Not financial advice, just pure vibes!	[: Hole; Downward] [: Diamond; Hands] [: Rocket; Moon] [: Rocket]	Endless Downwards; Diamond Hands; Soaring Stock Prices; Soaring Stock Prices;
Market be like: , but my portfolio’s like: . Guess we’re all just clowning around until the next bull run	[: Downwards] [: Clown; Flying Money] [: Bull; Flexed Biceps]	Downwards Trend; Juggled by the Market; Expecting the Market to Rebound

. Emojis amplify market volatility through sentiment contagion and algorithmic interference, which may lead to follow-up trading or manipulation risk. Elon Musk’s emoji (Shiba Inu) posted on Twitter has repeatedly triggered violent fluctuations in the price of the Dogecoin. The most typical event occurred on 3 April 2023, when he changed the official DOGE of Twitter from Bluebird to Shiba Inu, which caused the price of Dogecoin to soar by over 30% in a short time. Financial researchers are highly concerned about the significant impact of emojis on the market [4,5,6]. The research found that the sentiment analysis model can associate emojis with sentiment. The association can improve the accuracy of sentiment analysis [6]. Not only that, emojis can also soften the views of non-professional investors on bad news or enhance their views on good news [7]. To determine the meaning, sentiment, or intent of emojis in financial social media posts, researchers typically rely on human annotators. This process is often time-consuming and not scalable [8]. In social media communications, emojis can convey different meanings across trading communities, influenced by varying cultural backgrounds and market contexts [4]. In Section 3, we explore how these context-specific meanings of emojis vary across different market scenarios and communities. Due to the complex nature of these community-driven interpretations, traditional analysis methods based on manual annotations may misinterpret the actual meaning of these emojis in financial contexts [9]. This misinterpretation can hinder the effectiveness of financial text analysis models (e.g., financial sentiment analysis models and investment decision-making models).

To address the problem that the implicit expression of emojis can amplify the expression of sentiments, this paper proposes an implicit expression recognition model of emojis in social media comments (IER-SMCEM). First, the model is designed with the Implicit Emoji Enhancement Algorithm (IEEA) for data enhancement. IEEA is based on the homophone of the original text between the words and the emoji substitution, allowing the emoji to express different implicit expressions beyond public annotations in the comments. Then, this paper utilizes the Implicit Emoji Prediction Module (IEPM) to integrate the content of comments with implicit expressions into the prompt template. This data is then input into a pre-trained masked language model to predict the implicit expressions conveyed by emojis. The experimental results demonstrate that predicting implicit emoji expressions can significantly enhance the performance of traditional financial sentiment analysis models when dealing with implicit expressions. The experimental results indicate that IER-SMCEM achieves a 98.03% accuracy in semantically recovering implicit expressions within financial texts. The primary contributions of this paper are outlined below. Data and codes are available at https://github.com/junzhangNLP/MIDF, accessed on 24 April 2024.

• By applying IEEA to an existing publicly accessible dataset, we have created a sentiment analysis dataset for Chinese and English social comments. Because it is difficult to collect and process data in other languages, the target language of this study is based on Chinese and English.
• In this paper, IEPM is designed to address the challenge of recognizing implicit expressions of emojis in financial text analysis models.
• In this paper, we validate the positive impact of recognizing implicit expressions of emojis on financial text analysis tasks using a financial sentiment analysis model.

2. Related Work

From the perspective of semantic analysis technology, this section introduces the development process of technology and discusses the related research of emoji semantics. Then, this section discusses the influence of emojis on financial decision-making from the perspective of financial text analysis technology. In addition, this section discusses the situation of the financial dataset in detail. Based on the related research of datasets, we put forward the idea of constructing a new dataset.

2.1. Semantic Analysis Technology

Semantic analysis technology has primarily focused on document-level [10], sentence-level [11], and aspect-level [12] studies, with applications spanning social media, restaurant reviews, and product evaluations. In early research around 2003, systems based on logical expressions and syntactic rules were widely adopted. For instance, Alquézar et al. [13] proposed a method that employed handcrafted rules to structurally process text for identifying and learning context-sensitive language patterns. Park et al. [14] utilized logical expressions and syntactic rules to describe complex contextual relationships and introduced memory-based learning for more precise text chunking. Recently, researchers have explored a novel technique known as “prompt learning” within the text-mining domain for accurate textual information extraction. Liu et al. [15] introduced prompt learning as a groundbreaking paradigm in natural language processing. Central to prompt learning is the pivotal challenge of constructing prompt templates, as underscored by Liu X et al. [16], who proposed a P-tuning method that automatically searches for prompts in a continuous space. However, empirical evidence suggests that the effectiveness of this method may lag behind that of manual prompting [15,17]. In-depth investigations by numerous scholars have focused on developing superior manual prompt templates [18]. Schick et al. [19] emphasized the necessity of domain expertise and an understanding of language modeling capabilities for manually defining the mapping between words and tags. Similarly, Hu et al. [20] highlighted the importance of constructing a projection between the tag space and the tagged word space, known as verbalization, as the core step in prompt tuning. Building on these insights, this study adopts a manual approach to devise prompt templates.

With the development of social tools, emojis and pictures have become tools to express semantics. Many researchers have made contributions to the semantic extraction of emojis. The first emoji can be traced back to Japanese interface designer Kurita Satoshi [21], who created 176 unique graphic symbols for NTT DoCoMo devices in 1999. Today, 92% of the online population in the world uses emojis in their communication. With the popularity of emojis on social networks and other platforms, there is increasing interest in studying the functions of emojis [22]. Researchers have explored emojis with the function of expressing emotions, highlighting themes, decorating texts, adjusting mood, indicating identity, and attracting audiences [23]. Using these emoji functions, the researchers summarized the intention of using emoji on different platforms. In addition to these functions and intentions, researchers have also noticed the differences in the application scenarios of emojis, such as applications, culture, gender, labels, and platforms [24]. With the rise in popularity of emojis on social networks and other platforms, researchers have conducted extensive studies on their functions [25]. Lu et al. [26] explored how emojis express emotions, highlight themes, decorate text, adjust tone, indicate identity, and engage the audience. Hu et al. [22] summarized the intentions behind using emojis on various platforms based on these features. In addition to these functions and intentions, Chen et al. [27] highlighted differences in emoji usage across various application scenarios (e.g., apps, cultures, genders, hashtags, and platforms). However, much of the research on emoji functionality has relied on manual annotation for emoji interpretation. Given the richness and diversity of emoji semantics, there is an urgent need for automated tools to interpret these meanings. Zhou et al. [8] explored the use of ChatGPT (GPT-4o) for automated semantic annotation of emojis, as opposed to manual methods. They found that the semantic annotation of emojis is challenging due to their susceptibility to cultural differences, which complicates the interpretation process. In addition to semantic annotation of emojis, researchers have applied emoji information to downstream tasks such as emoji prediction and enhanced sentiment analysis [28]. Most existing emoji annotation methods rely on manual annotation. Although the emergence of LLMs has effectively alleviated this issue, its practical application remains limited due to the need for vast amounts of data and computational resources. In addition to the emoji function, some researchers have applied emoji information to downstream tasks, such as emoji prediction, improving sentiment analysis and predicting the loss of developers [29]. Recent evidence shows that emojis are also very useful for sentiment analysis [27]. Therefore, this paper considers emojis to be beneficial to financial sentiment analysis.

2.2. Financial Text Analysis and Dataset Challenge

Based on the research of semantic analysis, many researchers apply it to the financial field [30]. In the early stages of the research, sentences were first converted into vectors using embedding models or bag-of-word techniques such as GloVe, Word2Vec, and sentiment dictionaries [31]. These vectors were then classified using memory-based (recurrent) neural networks like RNNs, LSTMs, and GRUs [31,32]. In the subsequent phase, marked by the development of pre-trained models like BERT, the model achieved state-of-the-art performance in various tasks, including sentiment analysis, entity recognition, and text generation [33]. The textual data used for financial sentiment analysis include email communications, social media posts (e.g., tweets), corporate reports, and daily news [34]. Financial corpora are labeled through either manual annotation [7] or based on stock price [35]. Overall, there is one document-level, four sentence-level, two target-level, and one targeted aspect-level dataset. It is also observed that each public release of financial sentiment analysis benchmark dataset, particularly in open challenges such as FiQA Task 1 [36] and SemEval 2017 Task 5 [37] organized more recently, has promoted the research in FSA techniques. However, these mainstream datasets do not contain emojis. In fact, there are often many emojis in social media comments.

Due to the lack of labeled training datasets and domain adaptation, the sentiment analysis of financial reviews with emojis became more challenging. They considered sentiment analysis technology easy to fail for six types of reasons, such as subjunctive mood, rhetoric, dependence on opinions, etc. [38]. When studying the financial sentiment analysis dataset StockSen deeply, many emojis in the byte code were not converted correctly. In the original preprocessing step of the dataset, these codes are simply deleted from the corpus to fit into the machine learning model [39]. Emojis are not considered and used in general fields in financial sentiment analysis. To better realize financial sentiment analysis, this paper attempts to solve the problem of sentiment recognition of emojis in the financial field. Therefore, this paper first constructs the first implicit expression dataset of emojis by homophonic replacement in the aspect of the dataset. Secondly, this paper effectively enhances the task of financial sentiment analysis by restoring the true semantics of emojis.

3. Implicit Expression of Emojis

Social comments differ from formally regulated financial documents, as they often include semantically ambiguous emojis influenced by factors like cultural context [6,7,8,25,27]. Figure 2 illustrates a user comment about GME (GameStop Corp., an entertainment software company, Grapevine, TX, USA) on StockTwits. In early 2021, shares in the American video game retailer GME surged more than 700% in one week following the speculative involvement of individual investors, a move touted as investors piling in to buy stock they like [40]. This section explores how emojis are implicitly expressed in social media comments by analyzing recent posts about GME on StockTwits.

In Figure 2, the author demonstrates how implicit expressions are conveyed through emojis, capturing a range of semantics across different cultural contexts and application scenarios. The semantics of the overt annotations of the emojis in Figure 2 are compared with those expressed in the text, as shown in

Table 2. Public annotations and semantics of emojis.

Emojis	Public Annotations	Semantics
	[Rocket] [Chart Increasing]	Rapid Increase in Share Price
	[Gem Stone] [Raising Hands]	Diamond Hands
	[Banana]	Contempt
	[Rocket]	Significant Increase
	[Bar Chart] [Bullseye]	Precise Stock Price Forecast

.

The implicit expression of emojis effectively conveys the author’s emotional tendency, but recognizing these expressions presents significant challenges [6,7,8,9,25,27]. Implicit expressions can take various forms, such as homophones (). Different expressions can arise depending on the language, cultural background, or domain, making the automated annotation of implicit expressions in emojis particularly difficult [8]. Given the significant variations in cultural contexts across different domains, this chapter focuses on the implicit expressions communicated through emoji harmonics. In Table 2, emojis utilize harmonics (e.g., , ) to achieve implicit expressions.

Referring to the previous research on emojis, this paper defines the implicit expression of emojis as follows [7,8,9,27,41,42]. Emojis are defined as the implicit expression when emojis express a different meaning from the official expression in the text through homophonic or combination.

Table 3. The implicit expressions of emojis.

Content	Content with Emojis
I do not believe in the FINRA.	I do not in the FINRA.
It seemed like a bad idea.	It seemed like a bad .

offers illustrative examples of these implicit expressions enabled by harmonics. For more scientific purposes, this paper gives the following mathematical definition of the implicit expression of emojis. $M_{Meaning}=\{m_1,m_2,m_3,\cdots ,m_n\}$ represents all possible expressions of an emoji $e$, where $m$ denotes a semantic expression of the emoji. These public annotations form the set of shallow expressions, denoted as $M_{Shallow}=\left\{m_1,m_2,m_3,\cdots ,m_i\right\}$, where $M_{Shallow}\in M_{Meaning}$. However, emojis may also convey implicit expression diverging from the public annotation. These implicit expressions constitute the set of implicit expression $M_{Deep}=\left\{m_1,m_2,m_3,\cdots ,m_j\right\}$, where $M_{Deep}\in M_{Meaning}$ and M_Deep ∩ M_Shallow = ∅. The traditional model lacks the capacity to discern these deep semantics. $e$ can convey $M_{Deep}$ through a specific expression relation $f_{Relationship}$, as shown in Equation (1).

$$M_{Deep}=f_{Relationship}\left(m\right),m\in M_{Shallow}$$

(1)

This paper summarizes the process of implicit expression of emojis. Based on the process shown in Figure 3, humans can interpret the implicit expressions of emojis through harmonics. Financial professionals do have a different view on emojis compared to common people. However, traditional financial text analysis models typically focus only on the explicit annotations of emojis and struggle to grasp these implicit expressions [1,7]. This limitation adversely impacts the model’s performance when dealing with implicit emoji expressions.

4. Methods

This paper proposes IER-SMCEM, a model for recognizing implicit emoji expressions in social media comments based on prompt learning. IER-SMCEM comprises two main components: IEEA and IEPM. Firstly, IEEA leverages the implicit expressions of harmonics to map textual content into emojis. The input for IEPM consists of the IEEA-enhanced dataset $X=\left\{x_1,x_2,x_3,\cdots {,x}_n\right\}$. The text data $x_n$ from dataset $X$ is populated into the predefined template $T$. The prediction process in the model for each online financial text can be represented by Equation (2). M denotes the pre-trained language model, while $\theta$ represents the parameter set of the model. The model’s output is the probability $P$.

$$P=M(T\left(x_n\right),\theta )$$

(2)

For example, the model puts the input “I do not in the FINRA.” in the template “Related: [symbol]. Complete this sentence: [input] [mask].” Then, the input of the model is “ Related: bee leaf. Complete this sentence: I do not [mask] in the FINRA.” Finally, the prediction result of the model is “believe.” This paper introduces Bert’s workflow to explain how the model is predicted.

The pre-trained language model (PLM) receives input in the form of a sequence of tokens $W=\left\{w_1,w_2,w_3,\cdots ,w_n\right\}$, where n represents the length of the text. This step converts the text into a vector for input into the model. Usually, it consists of three kinds of word embeddings: token embedding, segment embedding, and position embedding. Firstly, the token embedding layer of the model converts the tokens into token embedding, denoted as $E_W$, as shown in Equation (3). $f_{token}$ indicates the process of conversion.

$$E_W=f_{token}\left(W\right)$$

(3)

Secondly, the segment embedding layer encodes the sentence to distinguish the segment embedding of the two sentences, as shown in Equation (4), where $E_A$ represents the segment embedding. The segment embedding process is denoted as $f_{Segment}$.

$$E_A=f_{Segment}\left(E_w\right)$$

(4)

Finally, the position embedding layer encodes the positional information of words into position embedding, as shown in Equation (5), where $E_n$ represents the position embedding. The position embedding process is denoted as $f_{Position}$.

$$E_n=f_{Position}\left(E_A\right)$$

(5)

Input the above vectors into the main model frame in the form of superposition and the coding and decoding structure based on Bi-transformer [33] can obtain all the predicted values of the possible words in the [mask] position. The vectors are entered into the Bi-transformer for calculation, as shown in Equation (6), where $f_{Bi-T}$ represents the Bi-transformer. $E_{Bi}$ represents the output of the Bi-transformer.

$$E_{Bi}=f_{Bi-T}\left({E_W,E_A,E}_n\right)$$

(6)

Finally, the model transforms the predicted value into a probability value of 0 to 1 through normalization operation. By choosing the prediction with a high probability value, the model can predict the true meaning of the emoji. The output is then normalized by the Softmax layer, as shown in Equation (7), where $f_{softmax}$ represents the Softmax layer. $W_{fd}$ represents the output of the Softmax layer.

$$W_{fd}=f_{softmax}\left(E_{Bi}\right)$$

(7)

Finally, the output marker layer converts the output to the corresponding encoding. Figure 4 shows the general structure of the model of the IER-SMCEM.

4.1. Implicit Emoji Enhancement Algorithm (IEEA)

The proposed IEEA is a method for embedding the implicit expressions of emojis into text. For the input text, IEEA first matches emojis based on harmonics using an emoji library, replacing well-matched words with corresponding emojis and masking the original word’s position. Then, the public annotation of the emoji, the masked text, and the actual expression are combined into the input format for the IEPM. It leverages harmonics to integrate emojis into an existing corpus. Figure 5 presents the principle of IEEA.

This paper employs homophones as the replacement logic for IEEA. The function $f_{clip}$ determines the position for replacing the emoji. By comparing the Levenshtein Distance between words and emojis one by one, $f_{clip}$ replaces the words with suitable emojis. Only a ratio of words in the sentence will be replaced by $f_{clip}$. As shown in Figure 5, “I believe in the FINRA.” is clipped into three parts by $f_{clip}$. $w_{text}$ is the remaining text, $w_{mask}$ is the pronunciation of the replaced word, and $w_{label}$ is the replaced word. IEEA retains the phonological features of the original text as $w_{mask}$. The expression of the original text is retained as the data label, $w_{label}$, and the remaining part of the original text is $w_{text}$. The process can be represented by Equation (8).

$$w_{text},{w_{mask},w}_{label}=f_{clip}\left(x_i\right)$$

(8)

The function $f_{tone}$ maps $w_{mask}$ within the data $x_i=[w_{text},{w_{mask},w}_{label}]$ to the corresponding emoji from set $E$. Subsequently, $w_{mask}$ is replaced with the expression of the mapped emoji $w_{symbol}$. The position of $w_{symbol}$ is marked as [MASK]. This replacement process is delineated by Equation (9). As shown in Figure 5, IEEA selects the replacement emoji by Levenshtein Distance in the emoji set. Replace $w_{mask}$ with the emoji’s annotation. Thus, enhanced $x_i$ is obtained.

$$w_{emoji}=f_{tone}\left(w_{mask},E\right)$$

(9)

Through Equation (9), IEEA achieved the construction of the dataset. Each data in the dataset consists of three parts: the marked text ($w_{text}$), the emojis in the text ($w_{symbol}$), and the true meaning of the emoji ($w_{label}$). IEEA reconstructed the content of $x_i$. Thus, the output of $x_i$ through the IEEA module is $x_i=[w_{text},{w_{symbol},w}_{label}]$.

4.2. Implicit Emoji Prediction Module (IEPM)

For the input text generated by IEEA with emojis, IEPM first embeds the text along with its corresponding public annotations into a predefined template. Next, IEPM leverages the mask prediction function of the pre-trained model to infer the expression of the masked token. The model computes the probability distribution at the mask position and selects the prediction with the highest probability of replacing the emoji in the original text. This process effectively identifies and interprets the implicit expression of the emojis, thereby improving the text’s accurate representation. Specifically, IEPM is utilized to extract sample features from the IEEA-enhanced dataset $X$. IEPM fine-tunes the model’s learning focus through the design of the input text. Figure 6 presents the model’s structure.

This paper takes one piece of data $x_i$ in $X$ as an example. The content in $x_i$ is spliced with the prompt template $T=“Relatedto\left[symbol\right].Completethissentence:\left[input\right].”$ The $\left[input\right]$ position in template $T$ is embedded by $w_{text}$. The $\left[symbol\right]$ position is embedded by $w_{symbol}$. Thus, the final output, $x_T$, is obtained, as shown in Equation (10).

$$x_T=f(x_i,T)$$

(10)

For input $x_T$, the word embedding model will predict the mask position in $x_T$. Therefore, this paper takes the output of the word embedding model as the prediction vector. The word embedding layer converts the $x_T$ into the corresponding word vector $E_T\in {\mathbb{R}}^N$, $N=768$. $N$ is the word vector dimension in the Bert model [33], as shown in Equation (11).

$$E_T=Embedding\left(x_T\right)$$

(11)

This work inputs $E_T$ into the pre-trained model $M$ with the model parameter $\theta$. Subsequently, the output of the model is transformed into the probability vector of the corresponding position using the function $f_{softmax}$. The final answer list $L_{Answer}$ of the model is obtained, as shown in Equation (12).

$$L_{Answer}=f_{softmax}\left(M\left(E_T,\theta \right)\right)$$

(12)

This work employs a stricter training requirement when training model M to pay attention to the expression of deep semantics. Therefore, in this paper, the model parameters are adjusted using the L1 loss. Equation (13) shows the loss function.

$$Loss(X,Y)={\displaystyle \frac{{\sum }_{i=1}^n\left|y_i-max\left(L_{Answer}\right)\right|}{n}}$$

(13)

5. Results

5.1. Dataset

In the realm of emoji semantic recognition, there is no dataset directly suited to the task addressed in this paper. To address it, we construct two datasets: IEEA-C for Chinese and IEEA-E for English. These datasets are created by enhancing the implicit expression of emojis using publicly available datasets, including SST-2 [43], IMDB [44], Weibo-100k [45], and ChnSentiCorp [46].

This paper adopts the Levenshtein Distance (LD) as the criterion for replacement. Specifically, emoji with an LD greater than 0.7 are targeted for replacement. Section 5.2 elaborates on the text’s ambiguity level under this condition.

Table 4. Distribution of IEEA-E samples.

Dataset	Train Set	Validation Set	Test Set
IEEA-E	14,394	3676	4410

presents the distribution of the IEEA-E dataset post-replacement.

This paper combines Chinese financial commentaries sourced from online platforms with publicly available datasets, Weibo_100k and ChnSentiCorp, to formulate the Chinese dataset IEEA-C.

Table 5. Example of IEEA-C sample table.

Content	Example1	Example2
Text	动资金 1	财务欺 2
Semantics	Working capital	Financial fraud

¹ Working fund. ² Financial fraud.

presents the example data from IEEA-C.

Table 6. Distribution of IEEA-C samples.

Dataset	Train Set	Validation Set	Test Set
IEEA-C	16,010	5113	5191

illustrates the dataset’s distribution.

5.2. Ambiguity Analysis

To assess the level of ambiguity [47] in sentences before and after data enhancement, this paper adopts the evaluation methodology outlined in Basile et al.’s [48] study. The evaluation metrics employed are detailed in

Table 7. Indicators of the degree of sentence ambiguity.

Score	Type	Range
Levenshtein Distance (↓) 1	distance	${\mathcal{D}}_{\mathrm{L}\mathrm{e}\mathrm{v}}(a,b)\in [0,max(\mid a\mid ,\mid b\mid \left)\right]$
InDel (↑)	similarity	${\mathcal{R}}_{\mathrm{I}\mathrm{D}}(a,b)\in \left[\mathrm{0,1}\right]$
Jaro–Winkler (↑)	similarity	${\mathcal{S}}_{\mathrm{J}\mathrm{W}}\left(a,b\right)\in \left[\mathrm{0,1}\right]$
Jaccard (↑)	similarity	$S_{\mathrm{J}\mathrm{a}\mathrm{c}}(a,b)\in \left[\mathrm{0,1}\right]$

¹ ↓ means the lower the value, the better. ↑ means the higher the value, the better.

, where a and b represent the sentences before and after enhancement, respectively.

Because the range of values for the four indicators is not uniform, this paper uses Equation (14) to map the range of variation from 0 to 1. The genuine value is $V$. The Normalization is $Map$.

$$Map={\displaystyle \frac{V-min\left(V\right)}{max\left(V\right)-min\left(V\right)}}$$

(14)

This paper investigates the relationship between the replacement ratio and the similarity. The replacement ratio ranges from 10% to 70%. Figure 7 and Figure 8 show the experimental results.

As shown in Figure 7 and Figure 8, all similarity metrics (InDel, Jaro–Winkler, Jaccard) exhibit a significant increase when substitutions are made using higher values of Levenshtein Distance (LD). Conversely, the distance metric (Levenshtein) decreases rapidly. The experiments demonstrated that higher LD values correlate with improved sentence similarity. With an increasing replacement ratio, metrics associated with sentence similarity decrease.

Table 8. Comparison of text before and after replacement.

Before Replacement	After Replacement
This show really is the Broadway American Idol. It has singing, the British Guy, A guy who’s sometimes nice, and a super-nice woman.	This show really is the Broadway American Idol. It has , the , a who’s sometimes , and a . 1

¹  is “singing”. is “British Guy”. is “guy”. is “nice”. is “super-nice woman”.

presents the text before and after the replacement. Guided by these experimental findings, this paper chooses an LD threshold of 0.7 and a replacement ratio of 0.4 to enhance the data. Therefore, in the example shown in Table 8, this paper replaces 40% of the words in the sentence with emojis that have an LD higher than 0.7 by comparing the LD between each word and the corresponding emoji.

5.3. Pre-Trained Models

In IEPM, IER-SMCEM utilizes the mask prediction function of pre-trained models. To evaluate how different pre-trained models affect the performance of IER-SMCEM, this paper employs BERT-base, BERT-large, and RoBERTa as the pre-trained language models (PLMs). The relevant parameters for these models are detailed in

Table 9. Parameters of the pre-trained model based on the corpus. (The static mask is a pattern of generating masks in the data preprocessing stage, while the dynamic mask is a pattern of generating masks randomly in real time during model training.).

Model	Model-Layer	Pre-Training Tasks
Bert-base-E	12-layer	MLM (static mask), NSP
Bert-large-E	24-layer	MLM (static mask), NSP
Roberta-E	12-layer	MLM (dynamic mask)
Bert-base-C	12-layer	MLM (static mask), NSP
Bert-large-C	24-layer	MLM (static mask), NSP
Roberta-C	12-layer	MLM (dynamic mask)

.

During the fine-tuning phase of the pre-trained model, we utilize its mask prediction function (MLM). The data is first tokenized to break the text into units that the model can process. In the process of tokenization, data is represented as vectors by the embedded layer. The text with [mask] tokens is then fed into the pre-trained model, where the mask prediction function predicts the masked positions and generates the most contextually appropriate emoticon or word. The hyperparameters for this phase are optimized based on the validation set, with the specific parameters detailed in

Table 10. The hyper-parameter setting of the model.

Hyper-Parameters	IEEA-C	IEEA-E
learning rate	1 × 10−5	2 × 10−5
training batch size	32	32
maximum context length	128	128
number of training epochs	20	20

.

5.4. Prompt Template

IER-SMCEM utilizes the $T_{\mathrm{E}\mathrm{M}\mathrm{J}-\mathrm{P}\mathrm{L}\mathrm{M}}$ template, which incorporates the relationship between public annotations and the implicit expressions of emojis. We devise template $T_{\mathrm{P}\mathrm{L}\mathrm{M}}$, which is devoid of semantic relation information of emojis. Additionally, we introduce an unprompted template, $T_{\mathrm{L}\mathrm{M}}$, to evaluate the role of prompts.

Table 11. English prompt template.

Type	${\mathit{T}}_{\mathbf{L}\mathbf{M}}$-E	${\mathit{T}}_{\mathbf{P}\mathbf{L}\mathbf{M}}$-E	${\mathit{T}}_{\mathbf{E}\mathbf{M}\mathbf{J}-\mathbf{P}\mathbf{L}\mathbf{M}}$-E
Prompt Template	[input] [mask]	Complete this sentence: [input] [mask]	Related: [symbol]. Complete this sentence: [input] [mask].
Example	I do not [mask] in the FINRA.	Complete this sentence: I do not [mask] in the FINRA.	Related: bee leaf. Complete this sentence: I do not [mask] in the FINRA.

and

Table 12. Chinese prompt template.

Type	${\mathit{T}}_{\mathbf{L}\mathbf{M}}$-C	${\mathit{T}}_{\mathbf{P}\mathbf{L}\mathbf{M}}$-C	${\mathit{T}}_{\mathbf{E}\mathbf{M}\mathbf{J}-\mathbf{P}\mathbf{L}\mathbf{M}}$-C
Prompt Template	[input] [mask]	请补全这个成语：[input] [mask]。 1	请补全这个成语：[input] [mask]。提示：与[symbol]有关。 2
Translation	[input] [mask]	Please complete this idiom: [input] [mask].	Please complete this idiom: [input] [mask]. Tip: It is related to [symbol].
Example	[mask]动资金	请补全这个成语： [mask]动资金	请补全这个成语：[mask]动资金。提示：与牛有关。

¹ “请补全这个成语：” means “Please complete this idiom:”. ² “提示：与[symbol]有关。” means “Tip: It is related to [symbol].”

present the templates employed for the experiments.

5.5. Evaluation

Several similarities exist between our work and that of XMTC: (1) both necessitate retrieving the accurate label from a broad spectrum of potential labels and (2) both demonstrate a long-tail impact on the labels. The task addressed in this paper involves recalling a single label, whereas the XMTC task involves recalling multiple labels. The goal of this study is to recover the implicit expression of emojis through a special case of extreme multi-label text classification (XMTC) [49]. We use $precision@k$ to evaluate the performance at the top of the result, a metric that has been widely used to evaluate XMTC methods [49]. Equation (15) shows the calculation for $P@k$.

$$P@k={\displaystyle \frac{1}{k}}\sum _{l=1}^k{\mathbf{y}}_{rank\left(l\right)}$$

(15)

where $\mathbf{y}\in \{\mathrm{0,1}{\}}^L,L\in \mathbb{R}$ is the true binary vector, and $rank\left(l\right)$ is the index of the $l$-th highest predicted label. We propose Equations (16) and (17) as an evaluation indicator based on Equation (18).

$$P=\sum _{l=1}^r{\mathbf{y}}_{rank\left(l\right)}$$

(16)

$$Accuracy={\displaystyle \frac{\sum _{n=1}^{N_D}{P}_T}{\sum _{n=1}^{N_D}{P}_T+\sum _{n=1}^{N_D}{P}_F}},where\left\{\begin{array}{c}{P}_T=1,P_F=0P=1\\ P_T=0,P_F=1P=0\end{array}\right.$$

(17)

where $r,r\in \mathbb{R}$, represents the size of the answer space when the answer space threshold is set to $\omega ,\omega \in \mathbb{R}$. $N_D,N_D\in \mathbb{R}$ represents the size of the dataset. To evaluate the size of $r$, this paper introduces a metric for assessing answer correlation, as shown in Equation (18).

$${\sigma }_{\mathrm{A}\mathrm{R}}={\displaystyle \frac{\sum _{n=1}^{N_D}{P}_{\mathrm{T}}}{r}}$$

(18)

In the task of XMTC, the precise retrieval of target labels presents significant challenges due to the enormous scale of the label space. Similar to the $P@k$ evaluation metric, the $P$ metric adopted in this study is also used to assess the accuracy of the model’s predictions within a given range. However, unlike $P@k$, which limits the evaluation scope to the top $k$ predictions, the $P$ metric extends the evaluation range to $r$. This adjustment holds substantial theoretical and practical significance. Specifically, in traditional XMTC tasks, each instance typically corresponds to multiple relevant labels, whereas the single-label prediction task focused on in this study exhibits notable differences. In XMTC, several labels need to be predicted from hundreds of labels. In the task, this paper needs to predict a word from tens of thousands of words. Moreover, some words have synonymous or similar expressions, which makes the task difficulty inconsistent. Given the inherent polysemy and context-dependency of emojis, employing the $P@k$ metric would introduce overly stringent evaluation criteria. This not only misaligns with the actual semantic expression characteristics of emojis but may also lead to a misjudgment of model performance. Therefore, the $P$ metric proposed in this study better accommodates the characteristics of emoji prediction tasks, providing a more reasonable standard for evaluating model performance.

This paper uses the commonly binary sentiment analysis metrics in the downstream sentiment analysis tasks. Metrics are shown by Equation (19). True Positive (TP) refers to the number of samples correctly predicted as positive, while True Negative (TN) represents the number of samples correctly predicted as negative. False Positive (FP) indicates the number of negative samples incorrectly predicted as positive, while False Negative (FN) denotes the number of positive samples incorrectly predicted as negative.

$$Acc={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(19)

5.6. Selection of Answer Space

To determine the most appropriate answer space size r across different datasets, this paper investigates the accuracy and answer relevance of various models on their respective language datasets.

Table 13. Answer space thresholds corresponding to answer space step.

Answer Space Step	1	2	3	4	5
IER-SMCEM-base-E	4	4.5	5	5.5	6
IER-SMCEM-Roberta-E	9	9.5	10	10.5	11
IER-SMCEM-large-E	7	7.5	8	8.5	9
IER-SMCEM-base-C	6.5	7	7.5	8	8.5
IER-SMCEM-Roberta-C	6.5	7	7.5	8	8.5
IER-SMCEM-large-C	6.5	7	7.5	8	8.5

shows the answer space thresholds represented by the answer space steps across different models. Due to the inconsistent size of the answer space for each model, this paper uniformly divides the answer space into five intervals to fairly evaluate model performance. Each interval represents an answer step. Since different models require varying thresholds for the answer space, as an optional parameter, this paper divides the answer space into five segments (answer space steps) for comparison purposes.

Figure 9 depicts the accuracy and answer correlation of the three models at the identical answer space step.

Based on the definition of answer correlation, a higher correlation indicates better model performance. IER-SMCEM-base-E demonstrates similar accuracy to the other two models while surpassing them in terms of answer correlation. Regarding accuracy, IER-SMCEM-large-C shows comparable performance to the other two models. When it comes to answer relevance, IER-SMCEM-large-C outperforms the other two models.

According to the definition of answer space, we select an answer space threshold with an answer space step of 3 to participate in subsequent model performance comparisons and application experiments.

5.7. The Semantic Recovery of Traditional Models

We utilize the $T_{\mathrm{L}\mathrm{M}}$ template to assess traditional models’ capacity in discerning the deep semantics of non-textual symbols. Figure 10 shows the experimental results on the IEEA-E.

The accuracy of Bert-large-E surpasses that of Bert-base-E, suggesting that having more deep layers is advantageous for the task of recognizing the implicit expression of emojis as it enables the model to more accurately learn sample features and achieve improved results. This conclusion also holds true for IEEA-C.

5.8. Experimental Results of Prompt Templates

We conduct two experiments using two different prompt templates: $T_{\mathrm{P}\mathrm{L}\mathrm{M}}$ and $T_{\mathrm{E}\mathrm{M}\mathrm{J}-\mathrm{P}\mathrm{L}\mathrm{M}}$. Figure 11 shows the experimental results.

When using the $T_{\mathrm{P}\mathrm{L}\mathrm{M}}$ template, IER-SMCEM-base-C demonstrates similar accuracy to that of IER-SMCEM-large-C. However, in the IEEA-E dataset, IER-SMCEM-base-E does not follow this trend because the English content in this dataset is less challenging. Compared with English, Chinese is richer in form. There is a similar S (Subject)-V (Verb)-O (Object) structure in Chinese. Generally, models with more complex structures perform better. Therefore, models with more intricate architectures tend to excel when handling simpler content.

After using the $T_{\mathrm{E}\mathrm{M}\mathrm{J}-\mathrm{P}\mathrm{L}\mathrm{M}}$ template, each model shows some improvement. This proves that embedding emoji information into the model’s input through templates can effectively improve the mode’s performance.

6. Case Study

The IER-SMCEM proposed in this paper is a semantic-driven model. Although the performance index of automation reflects the performance of the model to a certain extent, the performance of the model can be understood more deeply through intuitive manual analysis. Therefore, in this section, this paper selects two samples to evaluate the effectiveness of the model by comparing the semantic recovery effect. The case is shown in

Table 14. Example of case.

Original Data	Sentiment Label	Implicit Expressions	Analysis Results	IER-SMCEM	Analysis Results
I believe in the FINRA. Well actually, I forgot to mention that ‘diamond hands’ is typically ‘paper hands’ when faced with market fluctuations.	Positive	I in the FINRA. , I forgot to mention that ‘’ is typically when faced with market fluctuations.	Negative	I believe in the FINRA. Well actually, I forgot to mention that ‘diamond hands’ is typically ‘paper hands’ when faced with market fluctuations.	Positive
For the most part, I spy was an amusing lark that will probably rank as one of Murphy’s better performances in one of his lesser-praised movies.	Positive	For the most part, I was an amusing that will probably as one of Murphy’s performances in one of his lesser-praised movies.	Neutral	For the most part, I spy was an amusing lark that will probably rank as one of Murphy’s better performances in one of his lesser-praised movies.	Positive

.

In Table 14, this paper selected two cases and analyzed their sentiment. In the experimental setup, DistilRoBERTa [50] was used to respectively analyze the sentiment of the cases directly or after semantic recovery. By comparing the results, it can be seen that IER-SMCEM can effectively recover implicit expression. In the results of sentiment analysis, because emojis hide some positive sentiment words, direct sentiment analysis leads to errors. After semantic recovery by IER-SMCEM, the semantic expression of emoji is correct, and the correct sentiment analysis results are achieved. The experimental results show that it is very important to restore the implicit expression of emojis for downstream text analysis tasks.

7. Application

Previous research in finance has highlighted numerous links between investor sentiment and market [51,52]. To accurately identify investor sentiment, it is essential to extract the true content. The proposed IER-SMCEM focuses on recovering the true expressions of emojis. Therefore, this study employed common sentiment analysis tasks to validate the applicability of the recovered text for subsequent research tasks.

The performance of financial markets is significantly influenced by sentiment. Therefore, effectively extracting and quantifying sentiment from users and investors is a pressing issue in the financial field. Vamossy’s [53] research team has developed a tool that aids researchers in exploring the impact of sentiment on financial markets. By extracting sentiment from social media texts specifically related to finance, their research indicates that sentiment expressed by investors regarding specific firms can forecast daily price movements, highlighting the strong correlation between sentiment and market dynamics.

When facing implicit expressions, as shown in Figure 12 the performance of these models is inevitably affected to some extent.

The IER-SMCEM introduced in this paper helps sentiment analysis models manage the implicit expressions of emojis. To illustrate that IER-SMCEM is effective, this paper conducted experiments using standard datasets and widely used sentiment analysis models. After balancing the number of labels, we verified the robustness of the results with a k-fold CV (k = 5).

Table 15. Data distribution in the two datasets used for sentiment analysis.

Sentiment Analysis	IEEA-E	IEEA-C
Positive	11,240	13,157
Negative	11,240	13,157

shows the distribution of labels in the two datasets.

This paper selected five sentiment analysis models representing deep learning (i.e., TextCNN, BiLSTM, Bert, Bert-large, and DistilRoBERTa) [50] and two traditional machine learning models (i.e., CART [54], and XGB [55]).

Table 16. Impact of recovered text on traditional sentiment analysis models (English).

	IER-SMCEM-Base-E		IER-SMCEM-Large-E		IER-SMCEM-Roberta-E
Recover	False	True	False	True	False	True
TextCNN	70.37%	73.65%	69.66%	73.65%	70.37%	72.51%
BiLSTM	77.95%	80.04%	79.16%	80.39%	77.81%	79.47%
Bert	81.18%	81.85%	80.97%	82.10%	81.46%	81.68%
Bert-large	83.45%	85.79%	84.01%	86.50%	82.10%	83.02%
CART	61.87%	63.49%	61.87%	63.27%	61.87%	62.50%
XGB	69.52%	72.37%	69.52%	72.23%	69.52%	71.16%
DistilRoBERTa	84.23%	87.42%	84.73%	87.00%	87.28%	87.99%

and

Table 17. Impact of recovered text on traditional sentiment analysis models (Chinese).

	IER-SMCEM-Base-C		IER-SMCEM-Large-C		IER-SMCEM-Roberta-C
Recover	False	True	False	True	False	True
TextCNN	82.24%	84.61%	82.85%	83.69%	82.76%	83.59%
BiLSTM	81.07%	81.62%	83.65%	83.74%	83.46%	84.86%
Bert	81.39%	81.48%	80.96%	81.29%	81.62%	81.91%
Bert-large	79.41%	80.31%	78.28%	80.92%	80.26%	80.45%
CART	69.66%	71.05%	69.65%	70.91%	69.65%	71.19%
XGB	62.56%	64.14%	62.56%	63.95%	62.56%	64.19%
DistilRoBERTa	86.74%	87.07%	84.16%	87.92%	85.66%	86.46%

present experimental results.

Based on these experimental results, the IER-SMCEM proposed in this paper effectively enhanced the performance of downstream sentiment mining models. When mining social media information, models using the data recovered from this paper demonstrated excellent results. Experiments demonstrate that IER-SMCEM significantly enhances the model’s ability to analyze sentiment in the presence of implicit emoji expressions. Financial fraud can be effectively detected by analyzing corporate events and investor sentiment on social media. In future work, the accuracy of fraud detection can be enhanced by leveraging investor sentiment, with corporate events serving as the primary driver. In future works, we plan to explore applying financial sentiment analysis to detect potential financial fraud.

8. Conclusions and Future Works

This paper introduces an implicit expression recognition model for implicit expressions, addressing the challenge of accurately extracting valuable information. The model integrates relational information between public annotations and the implicit expressions of emojis, enhancing the model’s ability to recognize the implicit expressions.

Comparing the IER-SMCEM-base and IER-SMCEM-large models reveals that the more complex model structure yields better performance without prompts. For example, in Figure 10, the IER-SMCEM-large-E model achieves an accuracy rate of 79.29%, indicating that the increase in the number of hidden layers can improve the model’s learning ability. With the inclusion of prompt templates, the performance of a simple model can approach that of a complex model. For example, in Figure 11, the accuracy of IER-SMCEM-base-C with prompt templates reaches 97.10%. In Figure 11, IER-SMCEM-base-C achieves the highest accuracy of 98.03% in semantically recovering the implicit expressions within financial texts. Semantic reduction using this framework also significantly improves subsequent text-mining tasks such as sentiment analysis.

The semantics of the emoji recovery technology proposed in this paper, combined with downstream market analysis technology such as sentiment analysis, can be used as an early warning signal/indicator in the stock market (assuming the comments are reliable). Moreover, in other application fields that need social comment analysis, the method in this paper can be used to enhance the ability of other methods to recognize emojis. Although this study has achieved some progress in the semantic recovery of emojis, there are still some limitations. Firstly, the experiment relies on quantitative indicators (such as accuracy) and does not include human subjective evaluation (such as usability and manual evaluation). Therefore, different people will have different understandings of the true meaning of emojis. Secondly, data collection and experimental design are limited to specific languages (such as Chinese and English). Therefore, the universality of the conclusion in other cultures needs further verification. Finally, the language representation of the dataset is insufficient. This may introduce potential linguistic and socio-cultural deviations. To summarize, in future works, we will combine multilingual datasets and introduce human experts to enhance the reliability and generalization of the results.