Online-Review-Driven Products Ranking: A Hybrid Approach

Abstract

Online customer reviews (OCRs) are the real feelings of customers in the process of using products, which have great reference value for potential customers’ purchase decisions. However, it is difficult for consumers to extract helpful information from very large numbers of OCRs. To support consumers’ purchase decisions, this paper proposes a hybrid method to rank alternative products through OCRs. In this method, we use the fine-grained Bidirectional Encoder Representation from Transformers (BERT) model for aspect-level sentiment analysis (SA) and convert SA results of sub-criteria into a corresponding interval intuitionistic fuzzy number, accurately extracting customer satisfaction in OCRs and reducing the errors caused by different amounts of OCRs. Furthermore, in order to obtain the ranking results of products, the subjective and objective weights are combined to determine weight of feature. Subsequently, an improved interval intuitionistic fuzzy VIKOR method is proposed to rank mobile games. Finally, we conduct a case study and make some comparisons, which show that our method can reduce the complexity of accurately obtaining consumers’ personal preferences and help consumers make more accurate decisions.

1. Introduction

With the wide application of the Internet and social media, more and more people share their experiences of purchasing or using products on social networks. Through online customer reviews (OCRs), potential consumers can gain a deeper understanding of product performance and quality, which can help them to make better purchase decisions to a certain extent [1]. In addition, enterprise managers can better understand the quality of products, master users’ preferences and purchase desires for products to better improve product design and plan future development strategies of products [2]. Therefore, OCRs have become a significant information carrier of user needs [3].

However, with the surge in OCRs, the number of product categories and related OCRs has become enormous. Relevant product information has become more complicated and chaotic, which has exceeded the scope that consumers can accept and effectively process [4,5,6,7]. Therefore, this paper proposes a hybrid method that combines a fine-grained Bidirectional Encoder Representation from Transformers (BERT) model and weight fusion interval intuitionistic fuzzy number to solve the above problems.

Traditional sentiment analysis (SA) research only considers the local information of a text and does not consider the influence of global information on word vector training [8,9,10,11,12,13]. Only considering local information, it often obtains biased information, and leads to wrong decisions. Araci et al. [14] introduce the BERT model in financial SA, combining the global information of target words to conduct sentiment analysis. The classification result is 15% more accurate than the top result of the two financial sentiment analysis data sets. Therefore, this paper uses the fine-grained BERT model to learn a better text feature from the sub-feature layer of the product, changing the unidirectional training mode of traditional language models. By integrating the text’s upper and lower semantic information to encode and fine-tune the model’s hyperparameters, we can better understand the semantic information of sentences and articles.

The existing research of feature weight mainly divides the determination into two parts: (1) only use collective attribute weight [15,16]; and (2) only use personal attribute preference [17]. However, only collective attribute preference or individual attribute preference is used when calculating the feature weight, which makes it easy to lose individual preference information and renders the feature weight subject to external interference, resulting in large experimental errors. Therefore, both subjective preference and objective preference are considered when calculating the feature weight in this paper, and an improved VIKOR method is established based on this. While considering the collective preferences, it also refers to the objective preferences of potential consumers to better support the purchase decisions of potential consumers.

Nowadays, scholars have realized that, due to the complexity and uncertainty in OCRs, it is challenging to represent the information carried in OCRs with accurate real values. Therefore, a few studies have applied intuitionistic fuzzy set theory for product ranking problems [18,19]. Studies have constructed an intuitionistic fuzzy number (IFN) based on OCR sentiment analysis results. Furthermore, these studies have uses membership degree, non-membership degree and hesitation degree in the IFN to reflect the positive, negative and neutral attitudes of online reviews related to product features [20]. However, they did not consider the effect of the number of OCRs on the consistency of the decision data obtained from online reviews. The higher the number of OCRs is, the higher the consistency of decision data obtained from online reviews is; the lower the number of OCRs is, the lower the consistency of decision data obtained from OCRs is. Therefore, we construct an interval-valued intuitionistic fuzzy number to represent the product performance of alternative products and eliminate the influence of the number of OCRs on the consistency of decision data.

The objective of this paper is to propose a hybrid method based on the fine-grained BERT model, subjective weight and objective weight to rank products through OCRs. First, we classify target products to determine alternative products and obtain OCRs. Then, we extract the product features concerning consumers from a large number of OCRs and construct the evaluation system of the target, standard and sub-standard layers. After that, we use the fine-grained BERT model for SA and to score OCRs. Finally, according to the interval intuitionistic fuzzy theory, we use a constructed IFN to represent the feature performance of the alternative products and use the weight fusion VIKOR method to rank all the alternative products. So far, few studies have combined the BERT model with subjective and objective weights to deal with product ranking. Our method provides a new process for product ranking through OCRs in the future.

The main contributions of this paper are as follows:

• Classify products and analyze them at a more fine-grained level. Previous studies have rarely considered the classification of products, ignoring the error caused by product classification, which makes it easy to mislead consumers into making wrong decisions;
• Integrate subjective weight and objective weight into the VIKOR method, correct the deviation caused by the traditional method that only considers subjective or objective weight and help consumers choose more satisfactory and suitable products;
• Verify the effectiveness of the method via an example analysis. A mobile game case study shows that our method can reduce the complexity of accurately obtaining consumers’ personal preferences and help consumers make more accurate decisions.

The rest of this paper is structured as follows: In Section 2, the related literature is reviewed. Section 3 describes the product ranking problem and introduces the approach of this article. Section 4 describes case studies of mobile game ranking and how the proposed approach and sensitivity analysis can be used. Section 5 presents the conclusions and limitations of this study and puts forward future research directions.

2. Literature Review

This paper is related to two streams of the literature: (1) the studies on sentiment analysis, and (2) the studies on ranking products through OCRs.

2.1. Studies on Sentiment Analysis

Sentiment analysis is a process of analyzing, processing and extracting subjective text with sentiment colors by using natural language processing and text mining technology [21]; it covers many fields, such as natural language processing, text mining and machine learning. In recent years, it has become the primary concern in natural language processing and text mining [22,23,24].

According to the granularity of analysis objects, SA can be divided into three research levels: document-level, sentence-level and aspect-level [21]. The document-level identifies the sentiment polarity of the whole document, and it is considered the most basic task [25]. The sentence-level SA is regarded as a subtask of document-level SA, which identifies the sentiment polarity of each sentence in the document [26]. The aspect-level SA determines the corresponding sentiment intensity of each aspect from the fine-grained level of the document [27]. In order to obtain more accurate information from OCRs, aspect-level SA is utilized in this paper.

Li et al. [28], through a summary of multimodal emotional analysis research on existing multiple media channels, discussed future research trends and potential directions of multimedia emotional analysis. Anastasiei et al. [29], through an online survey, have proved that rational information is more persuasive than emotional information, which is conducive to a deeper understanding of the impact of rational and emotional information on consumer decision-making in social media. Kauffmann et al. [30] studied the impact of online reviews on brand image and positioning to classify customers and improve recommendation systems. Yuan et al. [31], based on real data from Amazon.cn, have revealed why useful comments can help consumers.

According to the existing SA research, the main sentiment classification methods are divided into three categories: lexicon-based [8,9,10], traditional machine learning [11,12] and deep learning [32,33,34]. However, lexicon-based SA technology cannot analyze sentiments in the contextual semantic environment of sentiment words, nor can it analyze the semantic similarity of the same word in a text [35]. This dramatically reduces SA accuracy and makes it challenging to process sentiment classification in long texts. Meanwhile, traditional machine learning SA technology cannot learn the sequential dependence of texts and analyze the semantic information of words in different contexts. Therefore, to ensure the accuracy of sentiment analysis results, this paper utilizes aspect-level SA technology based on fine-grained BERT in deep learning. Compared to traditional methods, this method has strong discrimination ability and feature self-learning ability, enabling it to automatically learn a text’s deep semantic and syntactic features. Through the fine-grained BERT model, connection with the target word’s context to semantic analysis can maintain a high effect even in long texts without data annotation.

2.2. Studies on Ranking Products through Online Reviews

Text mining of OCRs has been studied for a long time, but no research has combined it with fine-grained sentiment analysis and subjective and objective weights. Current studies mainly include [17,36,37,38,39].

Najmi et al. [36] proposed a new method of online product ranking combining OCR usefulness analysis, sentiment analysis and product brand ranking. Their dataset included 197 products in the TV and camera categories and 56,368 OCRs of Amazon-related products. Experiments show that this method can provide better product suggestions when inquiring with consumers about relevant products. Liu et al. [17] proposed a novel approach combining intuitionistic fuzzy theory with SA techniques. The method consisted of two stages: first, it used SA technology to determine online review sentiment orientation, and second, it used intuitionistic fuzzy theory to determine product ranking. This method developed an algorithm to classify the characteristics of each review as positive, negative or neutral. Next, the method transformed the identified emotional orientations into intuitionistic fuzzy numbers to represent the performance of the product features. Finally, it used a fuzzy PROMETHEE II method to calculate the final product ranking.

Wu et al. [37] proposed a comprehensive method for ranking products through online reviews. The method proposed a new framework and corresponding calculation rules based on intuitionistic fuzzy theory for expressing sentiment orientation and intensity. Then, the weight of features was determined through feature occurrence frequency and attention degree, so different features’ weights were more reasonable. Finally, it combined multi-attribute decision theory and the TODIM method to determine the final product ranking result. Chen et al. [38] proposed a framework for visualizing market structures via integrating online reviews with topic modeling, MDS and TOPSIS techniques. In the approach, they first separated the polarity of OCRs into positive and negative evaluations. Then, they extracted the critical attribute characteristics of the product from the positive and negative evaluations, respectively, and determined the weight matrix and significant criteria weight of each brand. Finally, they derived the market structure from the product attributes. Peng et al. [39] proposed a fuzzy MCDM method for evaluation and ranking products through OCRs. First, they calculated the similarity between words, regarded Chinese characters with word similarity greater than 0.5 as synonyms and found synonyms for all the words. Then, based on the frequency of each word and its synonyms, they determined the main characteristic criteria of the products. Finally, the fuzzy decision matrix of products was obtained, and they determined the ranking of all alternative products through the fuzzy PROMETHEE method.

This research has made significant contributions to SA and product ranking through online reviews. However, some aspects that need attention have not been considered in these studies. First, selecting alternative products was not classified, thus ignoring the differences among different products. In addition, few studies have paid attention to the combination of objective and subjective weights. Most studies only considered objective or subjective weight, which brings significant deviation to consumer decision-making. Finally, the construction of sentiment dictionaries and pre-training annotation data processing in existing studies is troublesome and consumes many human and material resources. Therefore, to solve these problems to support consumer decisions, we develop a method to select products by combining product classification, the fine-grained BERT model and weight fusion.

Table 1. Comparison of previous works with our proposed method.

Literature	Theories and Methods
	Classification of Products	Fusion Weight	Aspect-Level SA	Interval Intuitionistic Fuzzy Set	MCDM	BERT Model
[19]			✓		✓
[21]			✓
[27]			✓
[32]	✓		✓
[33]					✓
[34]			✓		✓
[35]					✓
[28]		✓	✓		✓
[29]				✓	✓
[30]						✓
[35]			✓			✓
[36]	✓				✓
Our method	✓	✓	✓	✓	✓	✓

shows the traditional approach compared with our method.

3. Research Method

The proposed method in this paper consists of three phases. The first phase classifies products to determine alternative products and crawl OCRs of alternative products. Next, the second phase extracts the aspects that concern consumers from the OCRs to construct a three-level evaluation system, using aspect-level SA to identify the sentiment orientations of sub-criteria through the fine-grained BERT model. The third and final phase constructs interval intuitionistic fuzzy numbers (IVIFNs) based on interval intuitionistic fuzzy sets theory to rank the alternative products with the improved IVIFN-VIKOR. The process of this proposed methodology is shown in Figure 1. We combine objective weight with subjective weight. The opinions of consumers who have purchased products also take into account the personal purchase preferences of potential consumers. In addition, the interval-valued intuitionistic fuzzy number is used to represent product performance, which reduces the impact of the number of online comments on the consistency of decision data and makes the results more accurate.

3.1. Collecting and Processing Online Reviews

3.1.1. Crawling Online Reviews Concerning Alternative Products

According to product attributes, main uses and use status, we classified products in order to determine alternative products. We classified products and determined each category’s most popular alternative products through a questionnaire survey. Then, we used a crawler written in Python 3.8 to crawl a large number of OCRs related to product features from three relevant websites.

3.1.2. Crawling and Preprocessing Data

To identify the sentiment orientations of the online reviews, it was necessary to preprocess the obtained online reviews. The preprocessing included two processes: (1) word segmentation and part-of-speech (POS) tagging and (2) stop words removal. Word segmentation recombined online reviews into word sequences according to certain norms. POS is the correct part of speech for each wordmark in the segmentation result. For example, after processing the Chinese “游戏画面不算差” (graphics are not bad) with word segmentation and POS, the result is “游戏画面/n, 不/v, 算/d，差/a” (i.e., graphics /n, are /v, not /d, bad /a). Where “/n”, “/v”, “/d” and “/a” denote “noun”, “verb”, “adverb” and “adjective”, respectively. Finally, we deleted the stop words. Deleting stop words deletes words that frequently appear in the text but do not carry important information so as to improve the efficiency and effect of sentiment analysis [17].

3.2. Using the Fine-Grained BERT Model for Sentiment Analysis

3.2.1. Extracting Product Aspect Features from Online Reviews

We combined word vectors and K-means clustering to extract product features from online reviews. First, we calculated the Term Frequency-Inverse Document Frequency (TF-IDF) values of OCRs, and selected N product keywords with high TF-IDF values. Then, we used word2Vec to transform the product keywords into word vector form, embedding the words in the high-dimensional word vector space. Finally, we used the K-means clustering method to cluster the product keywords, making the similarity within the group as large as possible, and the similarity between the groups as small as possible [40]. The feature values of the products were extracted from the clustering results. Furthermore, we calculated the information entropy of online reviews according to Equation (1):

$$E(H)=-{\displaystyle \sum _{i=1}^{n}P(h_i){\log }_2(P(h_i))},$$

(1)

where $H=\{h_1,h_2,h_3,\cdots ,h_n\}$ denotes a set of words that appear in a review and $P(h_i)$ denotes the frequency of the $i$th word in review, $i=1,2,\cdots ,n$.

3.2.2. Calculating the Sentiment Orientation

There were many reviews with semantic and scoring ambiguities in the initial OCRs. For example, online reviews containing “这款游戏的画质非常不好，我的体验感非常差” (“The picture quality of this game is very bad, my experience is very bad”) obtained a consumer scoring of 5. It was clear from the semantics of online reviews that consumers felt very poorly about the product they were reviewing, and yet they gave the product the highest possible score of 5 points. So, in the first step, we used Snow NLP to perform sentiment scoring on the initial reviews. We marked the sentiment values less than or equal to 0.5 as 0, and the sentiment values greater than 0.5 as 1. Then, we removed data with a scoring of 3 in the training data. We regarded reviews with scores of 1 and 2 as negative and marked them as 0. We regarded reviews with scores of 4 and 5 as positive and marked them as 1. Finally, we compared the score results with the sentiment scores and deleted the reviews with ambiguous scores. We used the remaining data as training data. We selected 20% of the training data as the test set and used the Adam optimizer to fine-tune the BERT-Base-Chinese model.

We input the data set into the trained BERT-Base-Chinese model. Through multiple Transformer layers, we calculated the similarity between a product feature word and other words in the same sentence and normalized the similarity results into weights through the softmax function. Then, we calculated the sentiment score of product features by integrating the value of the product feature word and the upper and lower words. Finally, we identified the sentiment orientation. Figure 2 shows the BERT model. The following describes the calculation of the Self-Attention mechanism.

Suppose the model input is $I$: $I=\{a_1,a_2,a_3,\cdots ,a_n\}$, which maps onto the continuous representation sequence $Z$: $Z=\{z_1,z_2,z_3,\cdots ,z_n\}$. We produced a linear mapping for each input input $a_i$ and multiplied it by the weight matrix $W^Q$, $W^K$ and $W^V$, respectively, to obtain the Query vector, Key vector and Value vector of each input (Briefly referred to as $Q$, $K$, and $V$ vectors):

$$Q=I\cdot W^Q$$

(2)

$$K=I\cdot W^K$$

(3)

$$\mathrm{V}=I\cdot W^V.$$

(4)

The similarity between vector $Q$ and vector $K$ was normalized via the softmax function. We then multiplied the calculation results generated by the softmax function by vector $V$ to obtain the continuous representation sequence $Z$. This is expressed by the following formula:

$$Z=Attention(Q,K,V)=softmax(\frac{Q\cdot K^T}{\sqrt{d_k}})\times V,$$

(5)

where $d_k$ is the dimension of vectors $Q$ and $K$.

The multi-attentional mechanism was adopted in the BERT model; that is, it used different self-attentional modules to calculate the semantic vectors of text in different semantic spaces. For example, the Transformer used eight attention headers, and each attention header was equipped with a Query, Key and Value weight matrix to calculate the semantic vector. Finally, multiple semantic vectors of each word were linearly combined to obtain the final semantic vector.

The process is expressed by the formula:

$$Z=MultiHead(Q,K,V)=Concat(hea{d}_0,hea{d}_1,\dots ,hea{d}_7)\cdot W^0,$$

(6)

where $hea{d}_i=Attention(Q_i,K_i,V_i)$, $i=1,2,\cdots ,n$.

We utilized the linear classifier to identify product features’ sentiment orientations. A product feature with a sentiment score less than or equal to 0.35 was considered negative, while a product feature with a sentiment score greater than 0.35 and less than or equal to 0.7 was considered neutral and one with a sentiment score greater than 0.7 was considered positive.

$S_{ij}^k=({\alpha }_{ij}^k,{\beta }_{ij}^k,{\gamma }_{ij}^k)$ denotes the sentiment orientation concerning feature $F_j$ in the $k$th online review of alternative product $P_i$. The value of $S_{ij}^k$ has four cases, i.e., $S_{ij}^k$ = (1,0,0), = (0,1,0), $S_{ij}^k$ = (0,0,1) and $S_{ij}^k$ = (0,0,0), where $S_{ij}^k$ = (1,0,0), $S_{ij}^k$ = (0,1,0), $S_{ij}^k$ = (0,0,1), respectively, denote that the $k$th online review of alternative product $P_i$ expresses the positive, neutral and negative sentiment orientations. Specially, $S_{ij}^k$ = (0,0,0) denotes that the sentiment orientation does not contain the relevant content of feature $F_j$ in the $k$th online review of alternative product $P_i$, $i=1,2,\cdots ,m$, $j=1,2,\cdots ,n$, $k=1,2,\cdots ,l$.

3.3. Measure Consumer Satisfaction and Preference Based on Interval Value Intuitionistic Fuzzy Number

3.3.1. Interval Value Intuitionistic Fuzzy Set

Definition 1

([41]). Let $X=\{x_1,x_2,\cdots ,x_n\}$ be a non-empty finite set, then an interval intuitive fuzzy set $X$ on $A$ is defined as:

$$A=\{<x,[{\mu }_A{(x)}^L,{\mu }_A{(x)}^U],[{\nu }_A{(x)}^L,{\nu }_A{(x)}^U]>|x\in X\},$$

(7)

where $[{\mu }_A{(x)}^L,{\mu }_A{(x)}^U]$ and $[{\nu }_A{(x)}^L,{\nu }_A{(x)}^U]$ represent the ordered interval pairs that element $X$ belongs to, respectively, the member interval and the non-member interval of set $A$.

In general, the ordered interval pair $([{\mu }_A{(x)}^L,{\mu }_A{(x)}^U],[{\nu }_A{(x)}^L,{\nu }_A{(x)}^U])$ composed of membership interval $[{\mu }_A{(x)}^L,{\mu }_A{(x)}^U]$ and non-membership interval $[{\nu }_A{(x)}^L,{\nu }_A{(x)}^U]$ is called the interval intuitive fuzzy number.

Definition 2

([42]). Let ${\alpha }_i=([a_i,b_i],[c_i,d_i])$, $(i=1,2,\cdots ,n)$ be a group of interval intuitionistic fuzzy numbers, as follows:

$$\mathrm{IIFWA}({\alpha }_1,{\alpha }_2,\cdots {\alpha }_n)=([1-{\displaystyle \prod _{i=1}^n{(1-a_i)}^{{\omega }_i},1-{\displaystyle \prod _{i=1}^n{(1-b_i)}^{{\omega }_i}}}],[{\displaystyle \prod _{i=1}^n{c}_i{}^{{\omega }_i}},{\displaystyle \prod _{i=1}^n{d}_i{}^{{\omega }_i}}]),$$

(8)

where ${\omega }_i$ is the weight of ${\alpha }_i(i=1,2,\cdots ,n)$ and ${\omega }_i\in [0,1]$ $(i=1,2,\cdots ,n)$, ${\displaystyle \sum _{i=1}^n{\omega }_i}=1$. $\mathrm{IIFWA}$ stands for interval intuitionistic fuzzy weighted arithmetic average operator.

Definition 3

([42]). Let ${\alpha }_1=([a_1,b_1],[c_1,d_1])$ and  ${\alpha }_2=([a_2,b_2],[c_2,d_2])$ be two arbitrary interval intuitionistic fuzzy numbers; the scoring functions of ${\alpha }_1$ and ${\alpha }_2$ are, respectively, $S({\alpha }_1)=\frac{(a_1+b_1-c_1-d_1)}{2}$, $S({\alpha }_2)=\frac{(a_2+b_2-c_2-d_2)}{2}$, and the exact functions of ${\alpha }_1$ and ${\alpha }_2$ are, respectively, $H({\alpha }_1)=\frac{(a_1+b_1+c_1+d_1)}{2}$ and $H({\alpha }_2)=\frac{(a_2+b_2+c_2+d_2)}{2}$, then:

(1):

If $S({\alpha }_1)<S({\alpha }_2)$, then ${\alpha }_1<{\alpha }_2$;

(2):

If: $S({\alpha }_1)=S({\alpha }_2)$, as follows:

If $H({\alpha }_1)<H({\alpha }_2)$, then ${\alpha }_1<{\alpha }_2$;

If $H({\alpha }_1)=H({\alpha }_2)$, then ${\alpha }_1={\alpha }_2$.

The distance formula of two interval intuitive fuzzy numbers $A=([a,b],[c,d])$ and $B=([a_1,b_1],[c_1,d_1])$ is:

$$d(A,B)=\sqrt{\frac{1}{4}[{(\mathrm{a}-{\mathrm{a}}_1)}^2+{(b-b_1)}^2+{(c-c_1)}^2+{(d-d_1)}^2]}.$$

(9)

3.3.2. Construct the Interval Value Intuitionistic Fuzzy Number of Product Features

Information entropy is a measure of the amount of information. Online reviews with higher entropy are more helpful to consumers and more critical. To determine the weight of online review $R_{ij}^k$ based on its information entropy, where $R_{ij}^k$ denotes the sentence concerning feature $F_j$ in the $k$th OCRs of alternative product $P_i$, $i=1,2,\cdots ,m$, $j=1,2,\cdots ,n$, $k=1,2,\cdots ,l$:

$${\omega }_{ij}^k=E_{ij}^k/{\displaystyle \sum _{k=1}^l{E}_{ij}^k}, i=1,2,\cdots ,m, j=1,2,\cdots ,n, k=1,2,\cdots ,l,$$

(10)

where ${\omega }_{ij}^k$ denotes the weight of review $R_{ij}^k$, $E_{ij}^k$ denotes the information entropy of review $R_{ij}^k$, $i=1,2,\cdots ,m,j=1,2,\cdots ,n$, $k=1,2,\cdots ,l$.

$N_{ij}^{pos}$, $N_{ij}^{neu}$ and $N_{ij}^{neg}$ represent the weighted frequencies of positive, neutral and negative online reviews concerning feature $F_j$ for product alternative $P_i$, respectively. $i=1,2,\cdots ,n$, $j=1,2,\cdots ,m$. $N_{ij}^{pos}$, $N_{ij}^{neu}$ and $N_{ij}^{neg}$ are computed as follows:

$$N_{ij}^{pos}={\displaystyle \sum _{k=1}^l{\omega }_{ij}^k{\alpha }_{ij}^k}, i=1,2,\cdots ,m, j=1,2,\cdots ,n, k=1,2,\cdots ,l,$$

(11)

$$N_{ij}^{neu}={\displaystyle \sum _{k=1}^l{\omega }_{ij}^k{\beta }_{ij}^k}, i=1,2,\cdots ,m, j=1,2,\cdots ,n, k=1,2,\cdots ,l,$$

(12)

$$N_{ij}^{neg}={\displaystyle \sum _{k=1}^l{\omega }_{ij}^k{\gamma }_{ij}^k}, i=1,2,\cdots ,m, j=1,2,\cdots ,n, k=1,2,\cdots ,l.$$

(13)

To construct intuitionistic fuzzy numbers: ${\overline{x}}_{ij}=({\mu }_{ij},v_{ij})$, Where ${\mu }_{ij}$ and $v_{ij}$,respectively, denote the support and opposition degrees of alternative product $P_i$ concerning feature $F_j$. ${\mu }_{ij}$ and $v_{ij}$ are computed as follows:

$${\mu }_{ij}=N_{ij}^{pos}/(N_{ij}^{pos}+N_{ij}^{neu}+N_{ij}^{neg}), i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(14)

$$v_{ij}=N_{ij}^{neg}/(N_{ij}^{pos}+N_{ij}^{neu}+N_{ij}^{neg}), i=1,2,\cdots ,m, j=1,2,\cdots ,n.$$

(15)

According to the method presented in [39], one can calculate ${\mu }_{ij}^L$, ${\mu }_{ij}^U$, $v_{ij}^L$ and $v_{ij}^U$ by intuitionistic fuzzy number ${\overline{x}}_{ij}=({\mu }_{ij},v_{ij})$ based on consistency interval estimation in probability theory. The calculation process is as follows:

$${\mu }_{ij}^L={\mu }_{ij}-z_{\theta /2}\times \sqrt{\left[{\mu }_{ij}(1-{\mu }_{ij})\right]/(N_{ij}^{pos}+N_{ij}^{neu}+N_{ij}^{neg})}, i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(16)

$${\mu }_{ij}^U={\mu }_{ij}+z_{\theta /2}\times \sqrt{\left[{\mu }_{ij}(1-{\mu }_{ij})\right]/(N_{ij}^{pos}+N_{ij}^{neu}+N_{ij}^{neg})}, i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(17)

$$v_{ij}^L=v_{ij}-z_{\theta /2}\times \sqrt{\left[v_{ij}(1-v_{ij})\right]/(N_{ij}^{pos}+N_{ij}^{neu}+N_{ij}^{neg})}, i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(18)

$$v_{ij}^U=v_{ij}+z_{\theta /2}\times \sqrt{\left[v_{ij}(1-v_{ij})\right]/(N_{ij}^{pos}+N_{ij}^{neu}+N_{ij}^{neg})}, i=1,2,\cdots ,m, j=1,2,\cdots ,n,.$$

(19)

In the above formulas, in order to eliminate the influence of the different number of online reviews, we introduced $z_{\theta /2}$ to express internal sentiment value based on consistency interval estimation in probability theory. $\theta$ represents the confidence level, and the value of $z_{\theta /2}$ can be found in the normal distribution table. However, ${\mu }_{ij}^U$ and $v_{ij}^U$ might not satisfy the definition of interval-valued intuitionistic fuzzy numbers that ${\mu }_{ij}^U+v_{ij}^U>1$. So, we must convert ${\mu }_{ij}^L$, ${\mu }_{ij}^U$, $v_{ij}^L$ and $v_{ij}^U$ using the following formulas:

$${\widehat{\mu }}_{ij}^L={\mu }_{ij}^L/max({\mu }_{ij}^U+v_{ij}^U), i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(20)

$${\widehat{\mu }}_{ij}^U={\mu }_{ij}^U/max({\mu }_{ij}^U+v_{ij}^U), i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(21)

$${\widehat{v}}_{ij}^L=v_{ij}^L/max({\mu }_{ij}^U+v_{ij}^U), i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(22)

$${\widehat{v}}_{ij}^U=v_{ij}^U/max({\mu }_{ij}^U+v_{ij}^U), i=1,2,\cdots ,m, j=1,2,\cdots ,n,.$$

(23)

Based on ${\mu }_{ij}^L$, ${\mu }_{ij}^U$, $v_{ij}^L$ and $v_{ij}^U$, we can construct the intuitionistic fuzzy number $a_{ij}=([{\widehat{\mu }}_{ij}^L,{\widehat{\mu }}_{ij}^U],[{\widehat{v}}_{ij}^L,{\widehat{v}}_{ij}^U])$ of alternative product $P_i$ concerning feature $F_j$. ${\mu }_{ij}^L$ and ${\mu }_{ij}^U$ denote the lower and upper limits of the membership degree of the attribute $F_j$ in alternative $P_i$, while $v_{ij}^L$ and $v_{ij}^U$ denote the lower and upper limits of the non-membership degree of the attribute $F_j$ in alternative product $P_i$, $i=1,2,\cdots ,m, j=1,2,\cdots ,n$.

3.3.3. Use Fuzzy Best–Worst Method (FBWM) to Calculate Product Attribute Weight

The FBWM integrates the traditional fuzzy preference relationship into the BWM, expands the precise integer used for preference comparison to fuzzy number to represent experts’ preference cognitions of alternative schemes and only uses a scale between 0.5 and 0.9 to compare and describe the two schemes. This situation is simpler for decision makers to directly ensure the accuracy and objectivity of information description [43]. The use of the FBWM model to measure index weight is mainly divided into five steps:

• Define the indicator set. A set of indicators $\{C_1,C_2,\dots ,C_n\}$ is used to describe the target object for evaluation;
• Select the best indicator and the worst indicator. The decision maker selects the best indicator $C_B$ and the worst indicator $C_W$ from the indicator set $\{C_1,C_2,\dots ,C_n\}$, according to the actual situation;
• Determine the importance of other indicators relative to the best indicators. Numbers 1–9 were used to represent the importance of other indicators relative to the best indicator, and a comparison vector $A_B=(a_{B1},a_{B2},\dots ,a_{Bn})$ was constructed, where $a_{Bn}$ represented the importance of the best indicator $C_B$ relative to the indicator $n$. A value of 1 means that $C_B$ and $C_n$ are equally important, and 9 means that $C_B$ is extremely important relative to $C_n$;
• Determine how important other indicators are relative to the worst indicators. Numbers 1 to 9 are used to represent the importance of other indicators relative to the worst indicator, and a comparison vector $A_W=(a_{1W},a_{2W},\dots ,a_{nW})$ is constructed, where $a_{nW}$ represents the importance of the worst indicator $C_W$ relative to the indicator $n$. A value of 1 means that $C_n$ and $C_W$ are equally important, and 9 means that $C_n$ is extremely important relative to $C_W$;
• Determine the index weight $w=(w_1,w_2,\dots ,w_n)$. A mathematical programming model is constructed and solved:

  $$\begin{array}{l}\min \underset{j}{\max }\{\left|\frac{w_B}{w_j}-a_{Bj}\right|,\left|\frac{w_j}{w_W}-a_{jw}\right|\}\\ \hspace{1em}\hspace{1em}s.t.\{\begin{array}{l}{\displaystyle \sum _{j=1}^n{w}_j=1}\\ w_j\ge 0,j=1,2,\cdots ,n\end{array}\end{array}$$

  (24)

3.3.4. Ranking Products Based on the Improved Interval Value Intuitionistic Fuzzy VIKOR Method

The attribute weights of traditional VIKORs do not consider the personal preferences of potential consumers. Therefore, we combined the consumer’s attribute preference $\widehat{W}=({\widehat{\omega }}_1,{\widehat{\omega }}_2,{\widehat{\omega }}_3,\cdots ,{\widehat{\omega }}_n)$ and collective attribute preference to improve the conventional VIKOR method.

First, we determined the positive-ideal solution $a_j^{*}$ and negative ideal solution $a_j^{-}$:

$$a_j^{*}=([\underset{i}{\max }{\widehat{\mu }}_{ij}^L,\underset{i}{\max }{\widehat{\mu }}_{ij}^U],[\underset{i}{\min }{\widehat{v}}_{ij}^L,\underset{i}{\min }{\widehat{v}}_{ij}^U]), i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(25)

$$a_j^{-}=([\underset{i}{\min }{\widehat{\mu }}_{ij}^L,\underset{i}{\min }{\widehat{\mu }}_{ij}^U],[\underset{i}{\max }{\widehat{v}}_{ij}^L,\underset{i}{\max }{\widehat{v}}_{ij}^U]), i=1,2,\cdots ,m, j=1,2,\cdots ,n.$$

(26)

Then, we computed the values $S_i$ and $R_i$:

$$S_i={\displaystyle \sum _{j=1}^n\left\{[r{\widehat{\omega }}_j+(1-r){\omega }_j]d(a_j^{*},a_{ij})\right\}/d(a_j^{*},a_j^{-})}, i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(27)

$$R_i=\underset{j}{\max }\left[\left\{[r{\widehat{\omega }}_j+(1-r){\omega }_j]d(a_j^{*},a_{ij})\right\}/d(a_j^{*},a_j^{-})\right], i=1,2,\cdots ,m, j=1,2,\cdots ,n,$$

(28)

where $r\in [0,1]$ denotes the importance of individual attribute preference, which is the result of decision makers weighing personal attribute preference and collective attribute preference.

Next, we computed the value $Q_i$:

$$Q_i=v\left[(S_i-S^{*})/(S^{-}-S^{*})\right]+(1-v)\left[(R_i-R^{*})/(R^{-}-R^{*})\right],$$

(29)

where $S^{*}=\underset{i}{\min }{S}_i$, $S^{-}=\underset{i}{\max }{S}_i$, $R^{*}=\underset{i}{\min }{R}_i$, $R^{-}=\underset{i}{\max }{R}_i$, $v\in [0,1]$ is identified as a weight for strategy of maximum group utility.

Finally, we ranked the alternatives, sorting by the values of $Q_i$ in decreasing order.

4. Case Study

4.1. Decision-Making Process

This paper selects mobile games from China’s leading mobile phone market for the case study. The reason why we chose mobile games as the research object is not only because mobile games have penetrated people’s lives and become the fastest-growing entertainment product in the field of mobile applications, but also because two-thirds of mobile game consumers provide feedback on online game user platforms [44].

We have divided mobile games into 11 categories by content: Strategy Game (STG), Role-playing Game (RPG), Racing Game (RCG), Puzzle Game (PUZG), Simulation Game (SIMG), First-person Shooting Game (FPS), Card Game (CAG), Music Game (MUG), Tabletop Role-playing Game (TRPG), Sports Game (SPG) and Multiplayer Online Battle Arena (MOBA). Each game category contains five of the most popular games. For example, the RPG category includes Journey to the West, Brawl Stars, Identity V, Onmyoji and Aotu World.

Table 2. Game classification.

Game Type	Name of the Game
STG	Clash of Clans	Lords Mobile	Boom Beach	Game Science	Late Shift
RPG	Journey to the West	Brawl Stars	Identity V	Onmyoji	Aotu World
RCG	QQ Speed	Need For Speed	Crazyracing Kartrider	Ace Race Overdrive	Asphalt 9 Legends
PUZG	Happy Elimination	Snake	Plants vs. Zombies 2	Angry Birds 2	Hearth Stone: Heroes of Warcraft
SIMG	Township	Mole’s World	Hay Day	Simcity Buildlt	Family Island
FPS	Call of Duty	Cross Fire	Game For Peace	Knives Out	Wefire
CAG	Clash Royale	Ark Nights	Happy landlords	Night of Full Moon	Battle Hand
MUG	Chameleon Run	QQ Dance	Piano Tiles 2	Rolling Sky	Dancing Line
TRPG	Werewolf	Killers of Three Kingdom	Armello	Pansdemic: the board game	Through the Ages
SPG	Street Basketball	Dream league Soccer	Ski Safari	Slam Dunk	FIFA Mobile
MOBA	Honor of Kings	Onmyoji Arena	Soul Blade	MARVEL Super War	Time and Space Call

shows a detailed breakdown of the games.

We took a questionnaire to select the most popular game from each category; the questionnaire was conducted online, with 11 single-choice questions to determine the most popular games in 11 categories. We distributed 515 questionnaires and collected 511 questionnaires, among which 280 male students and 231 female students participated. The results of the alternative game products determined by the questionnaire are: Boom Beach (P₁), Identity V (P₂), QQ Speed (P₃), Hearth Stone: Heroes of Warcraft (P₄), Mole’s World (P₅), Game For Peace (P₆), Clash Royale (P₇), QQ Dance (P₈) Killers of Three Kingdom (P₉), FIFA Mobile (P₁₀) and Honor of Kings (P₁₁). To ensure the rationality of the questionnaire results, we used Cronbach’s alpha to test the reliability of the questionnaire data. We used SPSS 26.0 to study the reliability of experimental data and calculated Cronbach’s alpha to be 0.842. The reliability coefficient value is higher than 0.8, indicating that the reliability is excellent [45].

We used Python 3.8 to write crawler software to obtain 94,395 online reviews of relevant mobile games from the Apple and Huawei mobile app stores, we preprocessed initial online reviews and deleted ambiguous reviews. After the original data were processed, the scores were compared with the scores calculated by SnowNLP, and 4608 online reviews whose semantics and scores differed in snow resolution were deleted to obtain a final dataset containing 89,787 online reviews.

We calculated the TF-IDF value of each OCR after processing and selected the words with high TF-IDF values in the reviews as feature candidates; that is, we selected the words representing the most complete online review content. We screened the candidate words into words with two characters or more, and deleted the words with low frequency and no apparent correlation with product features. After processing, we obtained 200 feature candidate words related to product feature attributes in each mobile game.

Then, we used the Word2Vec tool of Wikipedia Chinese training to calculate the word vector of each feature candidate word. We obtained each candidate word’s similarity by calculating the Euclidean distance of each feature vector and using the K-means clustering algorithm to cluster 200 feature candidate words. We set K to 5 based on recommendations from mobile game industry experts. After obtaining the K-means clustering results of 200 feature candidate words, we deleted those unrelated to feature attributes and SA. We combined them with the frequency of consumers’ reviews on relevant candidate features and selected the product features that most concerned consumers from the remaining candidate features. The product feature clustering results are shown in

Table 3. The product feature clustering results.

	F1	F2	F3	F4	F5
P1	Game loading stability	Game Graphics	Game vs. Mechanics	Game props	Game combat items
	Flash back, Disconnect, Loading	Graphics, Content, Design	Opponents, Upgrades, Mechanics	Architecture, Diamonds, Resources	Base, Task force, Carrier
P2	Game characters	Picture harmony	Game effects	Game atmosphere	Game matching mechanism
	Overseer, Butcher, Human	Map, skill	Golden, Purple, Knife air	Quality, Ethos, Actor	Match, Teammate, Balance
P3	Loading speed	Game scene	Racing mechanism	Racing props	Game touch
	Network, Login, Time	Mode, Screen, Map	Racing, System, Mechanism	Racing Cars, Clothes, props	Broken, Broken, Stuck
P4	Match balance	Card props	Gameplay mechanics	Client stability	Interface design
	Classes, Balance, Arenas	Cards, Coins	War chess, Turns, Adventures	Flashbacks, Drops, Freezes	Logins, Interfaces, Stores
P5	Game experience	Loading speed	Character modeling	Game scene	Server stability
	System, Customer Service	Loading, Login, Interface	Screen, Clothes, Modeling	Fishing, Planting vegetables, Scene	Flash back, Lag, Server
P6	Operation related	Game optimization	Anti-addiction Settings	Cheating penalty mechanism	Weapon props
	Network, Flashback, Stuck	Repair, Optimization, Improvement	Anti-addiction, Minors, Face recognition	Plugins, Matching, Sealing	Skin, Clothing, Quarterback
P7	Game balance	Card equipment	Character props	Game stability	Functional modules
	Fair, Balanced, Matching	Cards, Holy water	King, Cup, Treasure chest	Flashback, Offline, Optimization	Customer service, Functions
P8	Props raffle	Dance pictures and costumes	Game matching experience	Character dance movements	Game stability
	Lucky draw, Gift box, Explosion rate	Dazzle dance, Picture, Clothing	Experience, Customer service, Matching	People, Dance, Action	Flashback, Server, Offline
P9	General occupation	Skills and equipment	Combat special effects	Game scene	Gameplay
	General, Master	Skills, Equipment	Combat, Special effects	Pictures, Scenes	Simple, Playable
P10	Character action	Game content	Game optimization	Character design	Game experience
	Shooting, Passing, Long shot	Play, Attack, Rank	Flashback, Optimization, Stuck	Player, Movement, Fluency	Experience, System, Customer service
P11	Character design	Anti-addiction mechanism	Game fairness	Game experience	Run related
	Hero, Skin	Addiction prevention, Minors, Face recognition	Matching, Fairness, Actors	Hang up, Mentality	Flashback, Scoring

.

Then, we conducted SA of online reviews of mobile games using trained Bert models. We calculated the weighted frequency $N_{ij}^{pos}$ and $N_{ij}^{neg}$ of online reviews’ sentiment orientation, $i=1,2,\cdots ,11,j=1,2,\cdots ,5$. The weighted frequency of sentiment orientation is shown in

Table 4. The weighted frequency of sentiment orientation.

	F1		F2		F3		F4		F5
	$N_{i1}^{pos}$	$N_{i1}^{neg}$	$N_{i2}^{pos}$	$N_{i2}^{neg}$	$N_{i3}^{pos}$	$N_{i3}^{neg}$	$N_{i4}^{pos}$	$N_{i4}^{neg}$	$N_{i5}^{pos}$	$N_{i5}^{neg}$
P1	0.136	0.841	0.612	0.371	0.499	0.475	0.573	0.404	0.544	0.434
P2	0.556	0.394	0.732	0.236	0.683	0.269	0.565	0.397	0.510	0.448
P3	0.267	0.704	0.529	0.444	0.380	0.584	0.542	0.420	0.229	0.717
P4	0.312	0.655	0.354	0.599	0.181	0.783	0.065	0.918	0.042	0.947
P5	0.122	0.865	0.136	0.833	0.379	0.575	0.379	0.594	0.129	0.851
P6	0.206	0.771	0.401	0.550	0.359	0.590	0.315	0.646	0.436	0.523
P7	0.259	0.708	0.444	0.502	0.348	0.605	0.176	0.804	0.313	0.666
P8	0.294	0.662	0.711	0.263	0.213	0.746	0.653	0.302	0.123	0.839
P9	0.793	0.179	0.912	0.078	0.856	0.144	0.749	0.250	0.668	0.328
P10	0.343	0.616	0.345	0.618	0.158	0.828	0.458	0.514	0.204	0.777
P11	0.254	0.695	0.326	0.604	0.180	0.783	0.169	0.800	0.171	0.600

. We constructed the interval intuitionistic fuzzy number of features with formulas (17)–(20), and the data are shown in

Table 5. The interval intuitionistic fuzzy numbers of features.

	F1	F2	F3	F4	F5
P1	([0.0914,0.1551], [0.7276,0.7954])	([0.4983,0.6093], [0.2806,0.3096])	([0.3894,0.4113], [0.4711,0.4931])	([0.4736,0.5650], [0.3202,0.4109])	([0.4556,0.5556], [0.3296,0.4288])
P2	([0.4856,0.5448], [0.3355,0.3936])	([0.6516,0.7040], [0.1936,0.2439])	([0.6029,0.6614], [0.2214,0.2771])	([0.4771,0.5697], [0.3217,0.4131])	([0.4490,0.5036], [0.3805,0.4375])
P3	([0.1993,0.2877], [0.5977,0.6890])	([0.3495,0.4267], [0.4627,0.5405])	([0.2974,0.3967], [0.4831,0.5841])	([0.3312,0.4371], [0.4414,0.5488])	([0.1438,0.2749], [0.5845,0.7250])
P4	([0.2177,0.3445], [0.5253,0.6554])	([0.4893,0.5915], [0.2692,0.3689])	([0.6696,0.7428], [0.1290,0.1973])	([0.0495,0.0682], [0.8177,0.8386])	([0.0208,0.0539], [0.8351,0.8724])
P5	([0.0700,0.1495], [0.7386,0.8217])	([0.0847,0.1612], [0.7096,0.7928])	([0.2719,0.4111], [0.4480,0.5898])	([0.4636,0.6088], [0.2698,0.4131])	([0.1022,0.1300], [0.7523,0.7820])
P6	([0.1504,0.2346], [0.6778,0.7653])	([0.3350,0.4154], [0.4742,0.5558])	([0.2906,0.3815], [0.5062,0.5994])	([0.2527,0.3377], [0.5608,0.6484])	([0.3802,0.4357], [0.4621,0.5180])
P7	([0.2094,0.2453], [0.6038,0.6409])	([0.3587,0.4041], [0.4096,0.4729])	([0.2682,0.3433], [0.4937,0.5709])	([0.1291,0.1813], [0.6796,0.7342])	([0.2063,0.3439], [0.5160,0.6560])
P8	([0.2197,0.3256], [0.5577,0.6677])	([0.6223,0.6943], [0.2082,0.2781])	([0.1466,0.2486], [0.6368,0.7451])	([0.5468,0.6636], [0.2236,0.3363])	([0.0824,0.1457], [0.7421,0.8128])
P9	([0.6857,0.7403], [0.1346,0.1863])	([0.7984,0.8414], [0.0499,0.0908])	([0.7195,0.8205], [0.0784,0.1794])	([0.6508,0.6964], [0.2018,0.2474])	([0.5588,0.6429], [0.2526,0.3363])
P10	([0.2548,0.3743], [0.5033,0.6256])	([0.2818,0.3511], [0.5307,0.6016])	([0.1183,0.1711], [ 0.7318,07865])	([0.4004,0.4395], [0.4518,0.4910])	([0.1602,0.2143], [0.6845,0.7403])
P11	([0.5961,0.6935], [0.1892,0.2812])	([0.5335,0.5880], [0.2767,0.3290])	([0.1315,0.2021], [0.6890,0.7648])	([0.1077,0.2056], [0.6899,0.7943])	([0.4986,0.6151], [0.1141,0.2036])

. According to Formulas (21)–(22), we used interval intuitive fuzzy entropy to calculate the collective attribute preferences of consumers who have experienced the product, and the collective attribute weights are shown in

Table 6. The collective attribute weights.

	P1	P2	P3	P4	P5	P6	P7	P8	P9	P10	P11
${\mathsf{\omega }}_{i1}$	0.20	0.15	0.22	0.20	0.24	0.28	0.21	0.17	0.18	0.16	0.21
${\mathsf{\omega }}_{i2}$	0.19	0.31	0.28	0.14	0.19	0.11	0.15	0.25	0.19	0.18	0.19
${\mathsf{\omega }}_{i3}$	0.20	0.15	0.16	0.16	0.17	0.15	0.18	0.22	0.21	0.24	0.23
${\mathsf{\omega }}_{i4}$	0.20	0.18	0.15	0.25	0.20	0.23	0.24	0.16	0.22	0.2	0.22
${\mathsf{\omega }}_{i5}$	0.21	0.21	0.19	0.25	0.20	0.23	0.22	0.20	0.20	0.22	0.15

. The preferences given by customers are shown in

Table 7. The individual preference weights.

	P1	P2	P3	P4	P5	P6	P7	P8	P9	P10	P11
${\widehat{\mathsf{\omega }}}_{\mathrm{i}1}$	0.11	0.16	0.17	0.23	0.20	0.14	0.34	0.10	0.13	0.36	0.12
${\widehat{\mathsf{\omega }}}_{i2}$	0.30	0.27	0.22	0.13	0.14	0.30	0.26	0.31	0.18	0.20	0.20
${\widehat{\mathsf{\omega }}}_{i3}$	0.19	0.17	0.13	0.16	0.30	0.10	0.12	0.15	0.31	0.10	0.27
${\widehat{\mathsf{\omega }}}_{i4}$	0.15	0.28	0.15	0.17	0.19	0.30	0.21	0.30	0.23	0.14	0.20
${\widehat{\mathsf{\omega }}}_{i5}$	0.25	0.12	0.32	0.31	0.17	0.22	0.08	0.16	0.15	0.20	0.21

.

Then, given $r=0.5$, $v=0.5$, we used the improved VIKOR method to calculate the consumer’s $Q_i$. The results are as follows: $Q_1$ = 0.3206, $Q_2$ = 0.2388, $Q_3$ = 0.4127, $Q_4$ = 0.4718, $Q_5$ = 0.4383, $Q_6$ = 0.4043, $Q_7$ = 0.4349, $Q_8$ = 0.3221, $Q_9$ = 0.1515, $Q_{10}$ = 0.4229 and $Q_{11}$ = 0.3852. We determined the order of alternative game products based on the size of the $Q_i$ value to be: P4 > P5 > P7 > P10 > P3 > P6 > P11 > P8 > P1 > P2 > P9.

4.2. Sensitivity Analysis

In this research, the subjective and objective weight ratio $r$ and the majority importance degree $v$ of criteria significantly impacted the ranking results of products. We performed a sensitivity analysis to assess the influence of changes in $r$ and $v$ values on the ranking. The results are shown in Figure 3 and Figure 4.

As shown in Figure 3, the rankings of products P3, P4 and P11 gradually increase with the decrease in decision makers’ risk preference degree $v$ value. This indicates that, when decision makers pay more attention to the best interests, they are more satisfied with products P3, P4 and P11. At this time, decision makers are more satisfied with product P11, because product P11 increases more significantly with decision makers favoring group interests. Similarly, the rankings of P5, P6 and P10 increase with the increase in decision makers’ risk preference degree $v$. This means that decision makers are more satisfied with P5, P6 and P10 when they pay more attention to their minimal regrets. In addition, the rankings of products P1, P2, P8 and P9 are not affected by the change in $v$ value, which indicates that no matter whether decision makers pay more attention to the maximum benefit of all or the minimum personal regret, they have low satisfaction with P1, P2, P8 and P9.

Figure 4 shows the change in product ranking as the weight fusion coefficient $r$ of individual preference weight and collective attribute weight changes from 0 to 1. As shown from Figure 4, the rankings of products P1, P3, P4, P7 and P11 also improve with the increasing $r$ value. Similarly, the rankings of P5, P6 and P8 decrease as the $r$ value increases. In addition, the rankings of P2 and P9 are not affected by the $r$ value, and the rankings of P2 and P9 do not change no matter how the $r$r value changes. This indicates that both individual and collective preferences are dissatisfied with P2 and P9, which means that P2 and P9 are unsatisfactory for all customers. Due to ${\omega }_j=r{\widehat{\omega }}_j+(1-r){\tilde{\omega }}_j$, individual preference ${\widehat{\omega }}_j$ plays an increasingly important role with the increase in r. Customers’ personal preference for products has a greater impact on the final ranking result, and the decision result is highly subjective. On the contrary, the opinions of consumers who have purchased products have a greater impact on the final ranking results, and the decision results lack individuation. Hence, we combine objective weight with subjective weight. Considering not only the opinions of consumers who have bought products but also the personal preferences of consumers who want to buy products can help potential consumers choose more satisfactory and suitable products.

To summarize, different $v$ and $r$ values produce different sorting results. That is to say, the proportion of consumers’ personal attribute preference and collective attribute preference will greatly impact the decision makers’ product ranking and selection results. Therefore, combining consumers’ individual and collective attribute preferences is necessary when ranking products through online reviews.

4.3. Comparative Analysis

We selected five game features common to eleven kinds of mobile games and used the same experimental data comparative test to illustrate the rationality and correctness of the method in this paper. The five game features were picture, optimization, content, lag and experience. Using the same experimental data, the product ranking results are shown in Figure 5.

As can be seen in Figure 5, there are big differences in the ranking results of products. The reason for this is because the extraction of their common product features ignores the different game characteristics of different categories of mobile games, which are the indicators that consumers care about. As a result, traditional mobile game feature extraction methods extract the game feature results that often miss the most competitive aspects of the games. Therefore, this paper makes a fine-grained summary and grouping according to different categories of mobile games, and finally extracts the features that can more accurately show the features and attributes of mobile game products that affect consumers’ decisions.

We compared the proposed method with the previously designed ones by [46,47,48] to prove the reliability of the proposed method. Using the same data, the product ranking results are shown in

Table 8. Product ranking results.

Different Methods	Ranking Result
The proposed method	P4 > P5 > P7 > P10 > P3 > P6 > P11 > P8 > P1 > P2 > P9
The method proposed by [48]	P4 > P5 > P10 > P7 > P11 > P3 > P6 > P8 > P1 > P2 > P9
The method proposed by [46]	P4 > P5 > P6 > P10 > P7 > P8 > P1 > P11 > P3 > P2 > P9
The method proposed by [47]	P4 > P5 > P3 > P6 > P7 > P10 > P11 > P1 > P8 > P2 > P9

.

From the comparison of the ranking results shown in Table 8, we can see that the ranking results of different methods are similar. Compared with the ranking results of the other three methods, products P4 and P5 are ranked first and second, respectively. Products P2 and P9 are the last two products. Our method is similar to the result obtained by using the existing methods; the same product ranking part shows the effectiveness and rationality of our method, and the different product ranking part denotes the advantage of our method.

The first advantage of our method is that it constructs an interval value intuitionistic fuzzy number to evaluate product feature attributes. Sıcakyüz [48] combines the triangular fuzzy logarithm methodology of additive weights and the Fermatean fuzzy weighted aggregated sum product assessment method, but ignores the impact of the number of online reviews on the consistency of decision data. The fewer online reviews there are, the lower the consistency of decision data. Therefore, this paper adopts the form of an interval valued intuitionistic fuzzy number to eliminate the influence of the number of online reviews.

The second advantage of our methods is that it uses the fine-grained BERT model for sentiment analysis. Both [46] and [47] use the lexicon method to conduct sentiment analysis, so the product ranking results are different from [46,47]. We can capture longer-distance semantic information more efficiently and obtain bidirectional text information in the real sense through pre-training and fine-tuning the fine-grained BERT model.

The third advantage of our method is that it combines subjective and objective weights. Only subjective weight or objective weight is used in [46,47,48]. We calculate the collective preference weight of product attributes through interval intuitionistic fuzzy entropy and respect the individual preference of consumers’ attributes, making the ranking method more scientific and personalized.

4.4. Discussion

Before determining mobile game products, we classified different game products according to their characteristics, breaking away from the previous model of lumping all products into one category. The analysis of products at a more fine-grained level corrects the decision errors associated with lumping together different types of products.

We used a fine-grained BERT model for sentiment analysis on online reviews. The fine-grained BERT model had stronger feature extraction abilities that combined with entity naming to extract semantic information of text at a deeper level and improve the accuracy of sentiment analysis.

Our weights were composed of individual and collective preference weights. Consumers determined the weight value of individual preferences according to their personal preferences, and the weight values of collective preferences were calculated with intuitionistic fuzzy entropy. This considers the opinions of consumers who purchased products and the personal preferences of potential consumers. This method can help potential customers purchase satisfactory and more truly desirable products based on their personal preferences.

4.5. Research Hypothesis and Limitations

Our research applies to mobile game ranking issues and is highly portable and can be used with other processes with similar characteristics, such as online shopping, hotel booking, etc. However, there are some limitations to this study. When using a fine-grained BERT model for sentiment analysis, we directly analyzed all online reviews without considering inefficient and useless reviews in online reviews, which increases the complexity of the decision-making process. Therefore, the information load in online reviews needs to be considered in future studies. In addition, with the gradual diversification of the types and structures of information expressed by consumers, accurately describing the ambiguity and complex uncertainty in consumer information is an important research direction for the future.

5. Conclusions

Faced with massive amounts of online review data in the Internet era, it is a real problem to efficiently extract useful information. To this end, this paper proposes a hybrid method to solve this problem. The hybrid method of this paper consists of three stages: The first stage classifies products to identify alternative products and crawls online reviews related to alternative products. In the second stage, the features of game products that consumers care about are extracted through dimension reduction and clustering and a fine-grained BERT model is used for aspect-level sentiment analysis. The third stage identifies each feature’s interval intuitionistic fuzzy information and determines standard weights by combining the preference information of potential customers and experienced customers. Finally, it uses the improved VIKOR method to rank the alternative game products. Our method can reduce the complexity of accurately obtaining consumers’ personal preferences and help consumers make more accurate decisions.

In the face of a large number of OCRs, how to quickly and effectively help potential consumers and business operators to obtain information is a very practical problem. The hybrid method of this paper can provide potential consumers and business operators with useful information from very large numbers of OCRs to make better decisions. This new approach fully integrates the two types of weight, respecting the individual minds of potential consumers while also taking into account the thoughts of past consumers, thus helping consumers to make more authentic choices. In addition, the method proposed in this paper can also effectively help enterprise managers to accurately screen out helpful product information from OCRs, understand the market environment, provide references for enterprise promotion, market development and product development and help merchants to accurately position their products in fiercely competitive markets to gain competitive advantages.