Identification of Usefulness for Online Reviews Based on Grounded Theory and Multilayer Perceptron Neural Network
Abstract
:1. Introduction
2. Literature Review
2.1. Research on the Influencing Factors of the Usefulness of Online Reviews
2.2. Research on the Algorithm of Online Review Usefulness Recognition
3. A Method to Identify the Usefulness of Online Reviews
3.1. TF-IDF Method
3.2. Feature Extraction Based on Grounded Theory
- (1)
- Nkr: refers to the keyword retrieval number. According to the interview record of Question 3 in P2, “Generally, when I want to buy seasonal clothes, I will search several keywords, such as ‘2022 new women’s clothing’, ‘2022 winter extra thick’, and ‘blouse loose for women’, to find clothes suitable for me”;
- (2)
- Ncv: refers to the number of comments viewed. P14 indicates that he will browse the reviews of the target products that he wants to buy, especially for the reviews that contain the product features that he/she is interested in;
- (3)
- Npsp: refers to the number of other products to be viewed in the store where the target is located. The interview record for P16 for question 2 records that “when I find something I want to buy, I will click it to see the reviews and the stores, especially when I buy clothes. I will also check whether the style of the store is suitable for me and the reviews of other products in this store”;
- (4)
- Npp: refers to the number of similar products from the target to be viewed. P21 indicates that the price of the same product may be inconsistent in different stores, so browsing and comparison will be carried out in the selection of products;
- (5)
- Dp: refers to the number of days the product was added to the cart. P10 ‘s interview record for question 7 states that “I always buy some things with appropriate prices or possible needs, and it is more cost-effective to buy them together when there is a huge promotion activity”;
- (6)
- Drc: refers to the days between receiving goods and commenting. P24 indicates that generally, after confirming the receipt of goods, products will be reviewed after a period of experience;
- (7)
- Dfa: the days between the first comment and additional comments. According to the interview record for question 8 in P27, “Sometimes I review and sometimes I don’t. Sometimes I will make additional comments after a period of experience when I find that there is a big difference between my experience and that at the beginning”;
- (8)
- Nb: refers to the number of comments to be viewed. The interview record of P29 on question 5 records that “There are too many fake comments now, but relatively speaking, there are reference value for the more popular ones”;
- (9)
- Ng: refers to the number of likes on comments. The interview record of P8 for question 5 records that “It must be the number of likes of comments. Usually, I will also give a like to comments that are relevant and valuable for reference”.
3.3. MLP Neural Network
4. Experimental Design
4.1. Construction of Training Set and Test Set
4.1.1. Automatic Annotation Setting of Training Set
4.1.2. Selection of Training Set and Test Set
4.2. Evaluation Indicators
4.3. Result Analysis
4.3.1. Classification Results of MLP Model Based on Different Feature Combination
- (1)
- As a baseline system, the effectiveness (F1) of the MLP neural network model trained by the TF-IDF feature extraction method to identify the usefulness of online reviews is 59.2%. However, the MLP neural network model proposed is to train the multi-stage combined features from the consumer purchase decision-making process. The recognition effect (F1) reached 88.9%, compared with an increase of nearly 30%. The results show that the feature extraction method, considering the purchasing decision-making process of consumers, significantly improves the classification effect of the MLP classifier. The usefulness of online reviews can be identified not only from the comments themselves but also from the whole purchasing decision-making process of consumers, and the usefulness of online reviews can be identified through the characteristic indicators in each stage;
- (2)
- Both the classification effect of the classification model with the indexes of each stage trained separately and the other features of the multi-index stage trained after the removal of a single index decrease to a certain extent. Among them, by analyzing the recognition effects of classifier models at different stages of consumer decision-making, it can be seen that the influence of each stage on the usefulness of online reviews, in descending order, is: post-purchase behavior, product evaluation, purchase decision, information collection, and demand generation. In the product evaluation stage, Npsp (the number of other products to be viewed in the store where the target is located) has a greater impact on the identification results than Npp (the number of similar products to be viewed). In the post-purchase behavior stage, the degree of influence on the recognition results of the usefulness of online comments, in descending order, is: Nb (the number of comments to be viewed), Ng (the number of likes on comments), Dfa (the days between the first comment and additional comments), and Drc (the days between receiving goods and commenting).
4.3.2. Comparison between the Proposed Method and the Classical Method
- (1)
- In the selection of feature extraction methods, the overall recognition effect of the three classifiers trained by the TF-IDF method is poor, and they only perform well in the recall rate, while the , F1 value and their scores are low compared with those trained by the other two feature representation methods considering consumers’ purchase decisions. Since the TF-IDF feature representation method only starts from the perspective of text word frequency and the usefulness of online reviews in real situations is affected by many factors, it can be seen that the feature extraction method carried out in this paper from each stage of consumers’ purchase decisions has certain effectiveness;
- (2)
- In the comparison of the three feature extraction methods, the optimal feature combination has the best performance, and its recognition effect on the three different classifier models is excellent. For the SVM classifier model, the F1 value based on optimal feature combinations is between 4.3% and 15.4% higher than those based on all feature combinations and the TF-IDF method. For the Naive Bayes classifier model, the F1 value based on optimal feature combination is between 1% and 5.4% higher than those based on all feature combinations and the TF-IDF method. For the MLP neural network classifier model, the F1 value based on the optimal feature combination is 0.4% and 30.1% higher than those based on all feature combinations and the TF-IDF method. It can be concluded that the selection of the optimal feature combination significantly improves the recognition effect of the classifier.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Shin, S.; Chung, N.; Zheng, X.; Koo, C. Assessing the Impact of Textual Content Concreteness on Helpfulness in Online Travel Reviews. J. Travel Res. 2019, 58, 579–593. [Google Scholar] [CrossRef]
- Kostyra, D.; Reiner, J.; Natter, M.; Klapper, D. Decomposing the effects of online customer reviews on brand, price, and product attributes. Int. J. Res. Mark. 2016, 33, 11–26. [Google Scholar] [CrossRef]
- Purnawirawan, N.; Pelsmacker, P.D.; Dens, N. Balance and sequence in online reviews: How perceived usefulness affects attitudes and intentions. J. Interact. Mark. 2012, 26, 244–255. [Google Scholar] [CrossRef]
- Chen, H.N.; Huang, C.Y. An investigation into online reviewers’ behavior. Eur. J. Mark. 2013, 47, 1758–1773. [Google Scholar] [CrossRef]
- Cheung, C.M.K.; Lee, M.K.O. What drives consumers to spread electronic word of mouth in online consumer-opinion platforms. Decis. Support Syst. 2012, 53, 218–225. [Google Scholar] [CrossRef]
- Liu, X.; Wang, G.A.; Fan, W.; Zhang, Z. Finding Useful Solutions in Online Knowledge Communities: A Theory-Driven Design and Multilevel Analysis. Inf. Syst. Res. 2020, 31, 731–752. [Google Scholar] [CrossRef]
- Lee, S.G.; Trimi, S.; Yang, C.G. Perceived Usefulness Factors of Online Reviews: A Study of Amazon.com. J. Comput. Inf. Syst. 2018, 58, 344–352. [Google Scholar] [CrossRef]
- Malik, M.; Hussain, A. An analysis of review content and reviewer variables that contribute to review helpfulness. Inf. Process. Manag. 2018, 54, 88–104. [Google Scholar] [CrossRef]
- Ketron, S. Investigating the effect of quality of grammar and mechanics (QGAM) in online reviews: The mediating role of reviewer crediblity. J. Bus. Res. 2017, 81, 51–59. [Google Scholar] [CrossRef]
- Cheng, M.; Xin, J. What do Airbnb users care about? An analysis of online review comments. Int. J. Hosp. Manag. 2019, 76, 58–70. [Google Scholar] [CrossRef]
- Rullah, R.; Zeb, A.; Kim, W. The impact of emotions on the helpfulness of movie reviews. J. Appl. Res. Technol. 2015, 15, 359–363. [Google Scholar]
- Ren, G.; Hong, T. Examining the relationship between specific negative emotions and the perceived helpfulness of online reviews. Inf. Process. Manag. 2019, 56, 1425–1438. [Google Scholar] [CrossRef]
- Zhang, W.; Xu, H.; Wan, W. Weakness finder: Find product weakness from Chinese reviews by using aspects based sentiment analysis. Expert Syst. Appl. 2012, 39, 10283–10291. [Google Scholar] [CrossRef]
- BSparks, A.; So, K.K.F.; Bradley, G.L. Responding to negative online reviews: The effects of hotel responses on customer inferences of trust and concern. Tour. Manag. 2016, 53, 74–85. [Google Scholar] [CrossRef]
- Guo, B.; Zhou, S. What makes population perception of review helpfulness: An information processing perspective. Electron. Commer. Res. 2016, 17, 585–608. [Google Scholar] [CrossRef]
- Weimer, M.; Gurevych, I. Predicting the perceived quality of web forum posts. In Proceedings of the RANLP 2007, Bororets, Bulgaria, 27–29 September 2007; pp. 643–648. [Google Scholar]
- Li, S.T.; Pham, T.T.; Chuang, H.C. Do reviewers’ words affect predicting their helpfulness ratings? Locating helpful reviewers by linguistics styles. Inf. Manag. 2019, 56, 28–38. [Google Scholar] [CrossRef]
- Lee, P.J.; Hu, Y.H.; Lu, K.T. Assessing the helpfulness of online hotel reviews: A classification-based approach. Telemat. Inform. 2018, 35, 436–445. [Google Scholar] [CrossRef]
- Ou, W.; Huynh, V.N.; Sriboonchitta, S. Training attractive attribute classifiers based on opinion features extracted from review data. Electron. Commer. Res. Appl. 2018, 32, 13–22. [Google Scholar] [CrossRef]
- Chen, C.C.; Tseng, Y.D. Quality evaluation of product reviews using an information quality framework. Decis. Support Syst. 2011, 50, 755–768. [Google Scholar] [CrossRef]
- Hu, H.; Li, P.; Chen, Y. Biterm-based multilayer perceptron network for tagging short text. In Proceedings of the 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Siem Reap, Cambodia, 15–17 July 2015; pp. 212–217. [Google Scholar]
- Lee, S.; Choeh, J.Y. Predicting the helpfulness of online reviews using multilayer perceptron neural networks. Expert Syst. Appl. 2014, 41, 3041–3046. [Google Scholar] [CrossRef]
- Qader, W.A.; Ameen, M.M.; Ahmed, B.I. An Overview of Bag of Words; Importance, Implementation, Applications, and Challenges. In Proceedings of the 2019 International Engineering Conference (IEC), Erbil, Iraq, 23–25 June 2019; pp. 200–204. [Google Scholar]
- Ngo-Ye, T.L.; Sinha, A.P. The influence of reviewer engagement characteristics on online review helpfulness: A text regression model. Decis. Support Syst. 2014, 61, 47–58. [Google Scholar] [CrossRef]
- Yu, C.T.; Salton, G.M. On the construction of effective vocabularies for information retrieval. ACM SIGIR Forum 1973, 9, 48–60. [Google Scholar]
- Chen, R.Y.; Zheng, Y.T.; Xu, W. Secondhand seller reputation in online markets: A text analytics framework. Decis. Support Syst. 2018, 108, 96–106. [Google Scholar] [CrossRef]
- Kokkodis, M.; Ipeirotis, P.G. Reputation transferability in online labor markets. Manag. Sci. 2016, 62, 1687–1706. [Google Scholar] [CrossRef]
- Rout, J.K.; Singh, S.; Jena, S.K. Deceptive review detection using labeled and unlabeled data. Multimed. Tools Appl. 2017, 76, 3187–3211. [Google Scholar] [CrossRef]
- Zhang, W.; Du, Y.; Yoshida, T.; Wang, Q. DRI-RCNN: An approach to deceptive review identification using recurrent convolutional neural network. Inf. Process. Manag. 2018, 54, 576–592. [Google Scholar] [CrossRef]
- Noekhah, S.; Salim, N.B.; Zakaria, N.H. A novel model for opinion spam detection based on multi-iteration network structure. Adv. Sci. Lett. 2018, 24, 437–1442. [Google Scholar] [CrossRef]
- Le, Q.V.; Mikolov, T. Distributed representations of sentences and documents. In Proceedings of the 31st International Conference on Machine Learning, Beijing, China, 21–26 June 2014; Volume 23, pp. 1188–1196. [Google Scholar]
- Zhang, H.; Ji, D.; Ren, Y.; Zhang, H. Finding deceptive opinion spam by correcting the mislabeled instances. Chin. J. Electron. 2015, 24, 52–57. [Google Scholar]
- Zhang, Y.; Lin, Z. Predicting the helpfulness of online product reviews: A multilingual approach. Electron. Commer. Res. Appl. 2018, 27, 1–10. [Google Scholar] [CrossRef]
- Malik, M.; Hussain, A. Helpfulness of product reviews as a function of discrete positive and negative emotions. Comput. Hum. Behav. 2017, 73, 290–302. [Google Scholar] [CrossRef]
- Zheng, X.; Zhu, S.; Lin, Z. Capturing the essence of word-of-mouth for social commerce: Assessing the quality of online e-commerce reviews by a semi-supervised approach. Decis. Support Syst. 2013, 56, 211–222. [Google Scholar] [CrossRef]
Number | Interview Questions |
---|---|
1 | How many hours a week do you spend reading reviews on e-commerce platforms? What is the frequency? |
2 | Could you tell me something about your usual shopping process? |
3 | How do you select search keywords for products? |
4 | What shopping experience would you most like to see in a product review? |
5 | What metrics do you look for when reviewing reviews? |
6 | Do you compare the target products you want to buy? How does it compare? |
7 | Have you ever collected or added the products you finally purchased? |
8 | How long do you comment on the product firstly and additionally after receiving it? |
Feature Combination | Precision_uf | Recall_uf | F1 | Precision |
---|---|---|---|---|
TF-IDF | 0.647 | 0.546 | 0.592 | 0.599 |
PDP | 0.916 | 0.864 | 0.889 | 0.896 |
DG | 0.792 | 0.732 | 0.761 | 0.779 |
IC | 0.821 | 0.769 | 0.794 | 0.805 |
PE | 0.896 | 0.735 | 0.808 | 0.847 |
PE-Npsp | 0.875 | 0.724 | 0.792 | 0.832 |
PE-Npp | 0.885 | 0.728 | 0.799 | 0.84 |
PD | 0.845 | 0.723 | 0.779 | 0.805 |
PB | 0.912 | 0.851 | 0.88 | 0.889 |
PB-Drc | 0.894 | 0.847 | 0.87 | 0.876 |
PB-Dfa | 0.893 | 0.832 | 0.861 | 0.87 |
PB-Nb | 0.802 | 0.769 | 0.785 | 0.796 |
PB-Ng | 0.813 | 0.771 | 0.791 | 0.799 |
Feature Combination | Precision | Recall | F1 | Accuracy |
---|---|---|---|---|
NB | 0.913 | 0.764 | 0.832 | 0.843 |
NB, NG | 0.91 | 0.768 | 0.833 | 0.85 |
NB, NG, NPSP | 0.912 | 0.782 | 0.842 | 0.862 |
NB, NG, NPSP, DFA | 0.909 | 0.79 | 0.845 | 0.869 |
NB, NG, NPSP, DFA, NPP | 0.902 | 0.824 | 0.861 | 0.874 |
NB, NG, NPSP, DFA, NPP, Ncv | 0.911 | 0.832 | 0.87 | 0.881 |
NB, NG, NPSP, DFA, NPP, Ncv, Drc | 0.914 | 0.86 | 0.886 | 0.89 |
NB, NG, NPSP, DFA, NPP, Ncv, Drc, Dp | 0.914 | 0.872 | 0.893 | 0.9 |
NB, NG, NPSP, DFA, NPP, Ncv, Drc, Dp, Nkr | 0.916 | 0.864 | 0.889 | 0.892 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Hou, J.; Zhu, A. Identification of Usefulness for Online Reviews Based on Grounded Theory and Multilayer Perceptron Neural Network. Appl. Sci. 2023, 13, 5321. https://doi.org/10.3390/app13095321
Hou J, Zhu A. Identification of Usefulness for Online Reviews Based on Grounded Theory and Multilayer Perceptron Neural Network. Applied Sciences. 2023; 13(9):5321. https://doi.org/10.3390/app13095321
Chicago/Turabian StyleHou, Jiani, and Aimin Zhu. 2023. "Identification of Usefulness for Online Reviews Based on Grounded Theory and Multilayer Perceptron Neural Network" Applied Sciences 13, no. 9: 5321. https://doi.org/10.3390/app13095321
APA StyleHou, J., & Zhu, A. (2023). Identification of Usefulness for Online Reviews Based on Grounded Theory and Multilayer Perceptron Neural Network. Applied Sciences, 13(9), 5321. https://doi.org/10.3390/app13095321