Enhancing Explainable Recommendations: Integrating Reason Generation and Rating Prediction through Multi-Task Learning
Abstract
:1. Introduction
2. Related Work
2.1. CL-Based Rating Prediction
2.2. Explanation of Recommender Systems
2.3. Multi-Task Recommendation
3. Problem Definition and Acronyms
3.1. Problem Definition
3.2. Acronyms
4. Our Model
4.1. CL-Based Graph Convolutional Neural Network Encoder
4.1.1. Graph Convolution
4.1.2. Noise-Based Contrastive Learning
4.2. Transformer Decoder Based on Attention Mechanism
Algorithm 1 Transformer decoder. |
Input: , a sequence of token IDs, , a hidden state embedding matrix. Output: , where each column of is a distribution over the vocabulary. Hyperparameters: Parameters: includes all of the following parameters: , , token and positional embedding matrices. For : , multi-head attention parameters for layer l, see Equation (12), , two sets of layer-norm parameters, MLP parameters. , final layer-norm parameters. , the unembedding matrix.
|
4.2.1. Augmenting Input Information with Hidden Information from Neural Networks
4.2.2. Self-Attention-Based Transformer Layer
4.3. Multi-Task Learning Loss Function
5. Experimental Setup
5.1. Datasets
5.2. Baselines
5.3. Evaluation of Indicators
5.4. Experimental Settings
6. Results and Discussion
6.1. Rating Prediction Results
6.2. Reason Generation Results
6.3. Ablation Experiment
6.3.1. The Need for Hidden State-Based Information Enhancement
6.3.2. Enhancement from Contrastive Loss
6.3.3. Advantage of Using a Transformer
6.4. Hyperparameters in Multi-Task Learning
6.5. Computational Load Analysis
7. Conclusions and Future Work
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
Appendix A
Acronyms | Full Name |
---|---|
CL | Contrastive Learning |
InfoNCE | A form of Noise Contrastive Estimation |
GNN | Graph Neural Network |
GCN | Graph Convolutional Network |
RNN | Recurrent Neural Network |
DNN | Deep Neural Network |
MLP | Multilayer Perceptron |
LSTM | Long Short-Term Memory |
SGL | Self-supervised Graph Learning |
Xsimgcl | Extremely Simple Graph Contrastive Learning |
EFM | Explicit Factor Model |
AMF | Attentional Factorization Machine |
UAP | User Aspect Preference |
IAQ | Aspect Quality |
Att2Seq | Attribute-to-Sequence Method |
NRT | Neural Rating Regression with Abstractive Tips Generation |
GRU | Gated Recurrent Unit |
NETE | Neural Template Explanation Generation |
J3R | Joint Multi-task Learning of Ratings and Review Summaries |
EMER | Encoder–Decoder and MLP-based Explainable Recommendation |
CAML | Co-Attentive Multi-Task Learning |
MF | Matrix Factorization |
NCF | Neural Collaborative Filtering |
GCLTE | Our Graph Contrastive Learning with transformers within an Encoder–Decoder framework |
References
- Batmaz, Z.; Yurekli, A.; Bilge, A.; Kaleli, C. A review on deep learning for recommender systems: Challenges and remedies. Artif. Intell. Rev. 2019, 52, 1–37. [Google Scholar] [CrossRef]
- Jaiswal, A.; Babu, A.R.; Zadeh, M.Z.; Banerjee, D.; Makedon, F. A Survey on Contrastive Self-Supervised Learning. Technologies 2021, 9, 2. [Google Scholar] [CrossRef]
- Zhou, C.; Ma, J.; Zhang, J.; Zhou, J.; Yang, H. Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Singapore, 14–18 August 2021; ACM: New York, NY, USA, 2021; pp. 3985–3995. [Google Scholar] [CrossRef]
- Wu, J.; Wang, X.; Feng, F.; He, X.; Chen, L.; Lian, J.; Xie, X. Self-supervised Graph Learning for Recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Montreal, QC, Canada, 11–15 July 2021; ACM: New York, NY, USA, 2021; pp. 726–735. [Google Scholar] [CrossRef]
- Yu, J.; Yin, H.; Gao, M.; Xia, X.; Zhang, X.; Hung, N.Q.V. Socially-Aware Self-Supervised Tri-Training for Recommendation. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Singapore, 14–18 August 2021; ACM: New York, NY, USA, 2021; pp. 2084–2092. [Google Scholar] [CrossRef]
- He, K.; Fan, H.; Wu, Y.; Xie, S.; Girshick, R. Momentum Contrast for Unsupervised Visual Representation Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 9729–9738. [Google Scholar]
- Yu, J.; Xia, X.; Chen, T.; Cui, L.; Hung, N.Q.V.; Yin, H. XSimGCL: Towards Extremely Simple Graph Contrastive Learning for Recommendation. IEEE Trans. Knowl. Data Eng. 2024, 36, 913–926. [Google Scholar] [CrossRef]
- Hou, Y.; Yang, N.; Wu, Y.; Yu, P.S. Explainable recommendation with fusion of aspect information. World Wide Web 2019, 22, 221–240. [Google Scholar] [CrossRef]
- Zhang, Y.; Chen, X. Explainable Recommendation: A Survey and New Perspectives. Found. Trends Inf. Retr. 2020, 14, 1–101. [Google Scholar] [CrossRef]
- Wang, X.; Chen, Y.; Yang, J.; Wu, L.; Wu, Z.; Xie, X. A Reinforcement Learning Framework for Explainable Recommendation. In Proceedings of the 2018 IEEE International Conference on Data Mining (ICDM), Singapore, 17–20 November 2018; pp. 587–596. [Google Scholar] [CrossRef]
- Wang, X.; Wang, D.; Xu, C.; He, X.; Cao, Y.; Chua, T.S. Explainable reasoning over knowledge graphs for recommendation. Proc. Aaai Conf. Artif. Intell. 2019, 33, 5329–5336. [Google Scholar] [CrossRef]
- He, X.; Chen, T.; Kan, M.Y.; Chen, X. TriRank: Review-aware Explainable Recommendation by Modeling Aspects. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, VIC, Australia, 19–23 October 2015; pp. 1661–1670. [Google Scholar] [CrossRef]
- Wang, H.; Zhang, F.; Wang, J.; Zhao, M.; Li, W.; Xie, X.; Guo, M. RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Torino, Italy, 22–26 October 2018; pp. 417–426. [Google Scholar] [CrossRef]
- Balloccu, G.; Boratto, L.; Fenu, G.; Marras, M. Hands on Explainable Recommender Systems with Knowledge Graphs. In Proceedings of the 16th ACM Conference on Recommender Systems, Seattle, WA, USA, 18–23 September 2022; pp. 710–713. [Google Scholar] [CrossRef]
- Li, P.; Wang, Z.; Ren, Z.; Bing, L.; Lam, W. Neural Rating Regression with Abstractive Tips Generation for Recommendation. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Tokyo, Japan, 7–11 August 2017; pp. 345–354. [Google Scholar] [CrossRef]
- Zhu, J.; He, Y.; Zhao, G.; Bu, X.; Qian, X. Joint Reason Generation and Rating Prediction for Explainable Recommendation. IEEE Trans. Knowl. Data Eng. 2023, 35, 4940–4953. [Google Scholar] [CrossRef]
- Chen, Z.; Wang, X.; Xie, X.; Wu, T.; Bu, G.; Wang, Y.; Chen, E. Co-attentive Multi-task Learning for Explainable Recommendation. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence (IJCAI-19), Macao, Chian, 10–16 August 2019; pp. 2137–2143. [Google Scholar] [CrossRef]
- Peake, G.; Wang, J. Explanation Mining: Post Hoc Interpretability of Latent Factor Models for Recommendation Systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK, 19–23 August 2018; pp. 2060–2069. [Google Scholar] [CrossRef]
- Sun, P.; Wu, L.; Zhang, K.; Su, Y.; Wang, M. An Unsupervised Aspect-Aware Recommendation Model with Explanation Text Generation. ACM Trans. Inf. Syst. 2021, 40, 1–29. [Google Scholar] [CrossRef]
- Li, L.; Zhang, Y.; Chen, L. Personalized Prompt Learning for Explainable Recommendation. ACM Trans. Inf. Syst. 2023, 41, 1–26. [Google Scholar] [CrossRef]
- Lyu, Z.; Wu, Y.; Lai, J.; Yang, M.; Li, C.; Zhou, W. Knowledge Enhanced Graph Neural Networks for Explainable Recommendation. IEEE Trans. Knowl. Data Eng. 2023, 35, 4954–4968. [Google Scholar] [CrossRef]
- Li, L.; Zhang, Y.; Chen, L. Personalized Transformer for Explainable Recommendation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Virtual Event, 1–6 August 2021; pp. 4947–4957. [Google Scholar] [CrossRef]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is All You Need. In Proceedings of the NeurIPS, Long Beach, CA, USA, 4–9 December 2017; Volume 30. [Google Scholar]
- Kipf, T.N.; Welling, M. Semi-supervised classification with graph convolutional networks. arXiv 2016, arXiv:1609.02907. [Google Scholar]
- Gao, C.; Wang, X.; He, X.; Li, Y. Graph Neural Networks for Recommender System. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, Tempe, AZ, USA, 21–25 February 2022; pp. 1623–1625. [Google Scholar] [CrossRef]
- Yu, J.; Yin, H.; Xia, X.; Chen, T.; Li, J.; Huang, Z. Self-Supervised Learning for Recommender Systems: A Survey. IEEE Trans. Knowl. Data Eng. 2024, 36, 335–355. [Google Scholar] [CrossRef]
- You, Y.; Chen, T.; Sui, Y.; Chen, T.; Wang, Z.; Shen, Y. Graph Contrastive Learning with Augmentations. Adv. Neural Inf. Process. Syst. 2020, 33, 5812–5823. [Google Scholar]
- Yu, J.; Yin, H.; Xia, X.; Chen, T.; Cui, L.; Nguyen, Q.V.H. Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, 11–15 July 2022; ACM: New York, NY, USA, 2022; pp. 1294–1303. [Google Scholar] [CrossRef]
- Zhang, Y.; Lai, G.; Zhang, M.; Zhang, Y.; Liu, Y.; Ma, S. Explicit Factor Models for Explainable Recommendation Based on Phrase-Level Sentiment Analysis. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, Gold Coast, Australia, 6–11 July 2014; pp. 83–92. [Google Scholar] [CrossRef]
- Gatt, A.; Krahmer, E. Survey of the State of the Art in Natural Language Generation: Core Tasks, Applications and Evaluation. J. Artif. Intell. Res. 2018, 61, 65–170. [Google Scholar] [CrossRef]
- Dong, L.; Huang, S.; Wei, F.; Lapata, M.; Zhou, M.; Xu, K. Learning to Generate Product Reviews from Attributes. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, Valencia, Spain, 3–7 April 2017; pp. 623–632. [Google Scholar]
- Staudemeyer, R.C.; Morris, E.R. Understanding LSTM—A tutorial into Long Short-Term Memory Recurrent Neural Networks. arXiv 2019, arXiv:1909.09586. [Google Scholar]
- Cho, K.; van Merrienboer, B.; Gülçehre, Ç.; Bougares, F.; Schwenk, H.; Bengio, Y. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv 2014, arXiv:1406.1078. [Google Scholar]
- Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I. Improving Language Understanding by Generative Pre-Training. OpenAI Blog. 2018. Available online: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf (accessed on 1 January 2024).
- Dong, L.; Yang, N.; Wang, W.; Wei, F.; Liu, X.; Wang, Y.; Gao, J.; Zhou, M.; Hon, H.W. Unified Language Model Pre-training for Natural Language Understanding and Generation. arXiv 2019, arXiv:1905.03197. [Google Scholar] [CrossRef]
- Li, L.; Zhang, Y.; Chen, L. Generate Neural Template Explanations for Recommendation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual, 19–23 October 2020; pp. 755–764. [Google Scholar] [CrossRef]
- Avinesh, P.V.S.; Ren, Y.; Meyer, C.M.; Chan, J.; Bao, Z.; Sanderson, M. J3R: Joint Multi-task Learning of Ratings and Review Summaries for Explainable Recommendation. In Proceedings of the Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, 16–20 September 2019; Proceedings, Part III. Springer International Publishing: Cham, Switzerland, 2020; pp. 339–355. [Google Scholar] [CrossRef]
- He, X.; Deng, K.; Wang, X.; Li, Y.; Zhang, Y.; Wang, M. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual, 25–30 July 2020; pp. 639–648. [Google Scholar] [CrossRef]
- Chen, X.; Zhang, J.; Wang, L.; Dai, Q.; Dong, Z.; Tang, R.; Zhang, R.; Chen, L.; Wen, J.R. REASONER: An Explainable Recommendation Dataset with Multi-aspect Real User Labeled Ground Truths Towards more Measurable Explainable Recommendation. arXiv 2023, arXiv:2303.00168. [Google Scholar]
- Hou, Y.; Li, J.; He, Z.; Yan, A.; Chen, X.; McAuley, J. Bridging Language and Items for Retrieval and Recommendation. arXiv 2024, arXiv:2403.03952. [Google Scholar]
- Papineni, K.; Roukos, S.; Ward, T.; Zhu, W.J. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA, 6–12 July 2002; pp. 311–318. [Google Scholar] [CrossRef]
- Lin, C.Y. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the Text Summarization Branches Out, Barcelona, Spain, 25–26 July 2004; pp. 74–81. [Google Scholar]
- Lu, Z.; Xie, H.; Liu, C.; Zhang, Y. Bridging the Gap between Vision Transformers and Convolutional Neural Networks on Small Datasets. Adv. Neural Inf. Process. Syst. 2022, 35, 14663–14677. [Google Scholar]
RETER | NRT | LighGCN | SGL | EMER | GCLTE | |
---|---|---|---|---|---|---|
RMSE | 1.3173 | 1.1926 | 1.2158 | 1.2030 | 1.3147 | 1.1147 |
MAE | 1.0978 | 0.9484 | 1.0035 | 0.9617 | 1.1275 | 0.9189 |
RETER | NRT | LighGCN | SGL | EMER | GCLTE | |
---|---|---|---|---|---|---|
RMSE | 1.1269 | 1.0959 | 1.0981 | 1.0476 | 1.0954 | 1.0358 |
MAE | 0.9050 | 0.8592 | 0.8715 | 0.8179 | 0.8611 | 0.8065 |
RETER | NRT | LighGCN | SGL | EMER | GCLTE | |
---|---|---|---|---|---|---|
RMSE | 1.1269 | 1.1191 | 1.1194 | 1.0645 | 1.1191 | 1.0816 |
MAE | 0.9050 | 0.8891 | 0.8840 | 0.8432 | 0.8878 | 0.8476 |
Model | BLEU1 | BLEU4 | ROUGE_1/f | ROUGE_1/r | ROUGE_1/p | ROUGE_2/f | ROUGE_2/r | ROUGE_2/p |
---|---|---|---|---|---|---|---|---|
Att2seq | 20.5806 | 3.5184 | 22.5084 | 20.4895 | 26.0100 | 5.6244 | 5.2796 | 6.2503 |
RTER | 17.1456 | 2.1726 | 19.6311 | 14.8421 | 29.4247 | 4.8076 | 3.5978 | 7.3658 |
NRT | 18.5425 | 3.3937 | 20.6744 | 16.8437 | 27.9985 | 5.6636 | 4.7378 | 7.4232 |
EMER | 18.6114 | 2.1851 | 22.0573 | 17.3172 | 30.9540 | 4.8569 | 4.0701 | 6.1467 |
GCLTE | 20.8766 | 3.4942 | 22.7577 | 18.6529 | 27.1614 | 6.0271 | 5.2776 | 7.1744 |
Model | BLEU1 | BLEU4 | ROUGE_1/f | ROUGE_1/r | ROUGE_1/p | ROUGE_2/f | ROUGE_2/r | ROUGE_2/p |
---|---|---|---|---|---|---|---|---|
Att2seq | 17.5467 | 1.5001 | 18.9466 | 14.9544 | 34.0016 | 3.2864 | 2.5508 | 6.8905 |
RETER | 7.8509 | 0.3690 | 11.0091 | 6.2566 | 47.6637 | 0.9135 | 0.5177 | 3.9592 |
NRT | 13.5462 | 1.1856 | 16.1556 | 10.2688 | 38.6763 | 3.3596 | 2.1277 | 8.1914 |
EMER | 8.9918 | 0.7997 | 9.5330 | 5.2827 | 66.2802 | 1.9228 | 1.0618 | 14.1053 |
GCLTE | 21.4245 | 1.7071 | 22.2740 | 17.8240 | 30.0254 | 4.0968 | 3.3816 | 5.2516 |
Model | BLEU1 | BLEU4 | ROUGE_1/f | ROUGE_1/r | ROUGE_1/p | ROUGE_2/f | ROUGE_2/r | ROUGE_2/p |
---|---|---|---|---|---|---|---|---|
Att2seq | 12.8202 | 0.5419 | 18.4875 | 11.6949 | 44.9535 | 1.5134 | 0.9536 | 3.7028 |
RETER | 7.8509 | 0.3690 | 11.0091 | 6.2566 | 47.6637 | 0.9135 | 0.5177 | 3.9592 |
NRT | 11.4115 | 0.6270 | 16.0631 | 10.1750 | 38.9668 | 1.9961 | 1.2563 | 4.8961 |
EMER | 13.3753 | 0.7190 | 17.6375 | 11.6058 | 37.6826 | 2.0430 | 1.3805 | 4.0564 |
GCLTE | 21.1844 | 1.1768 | 23.2173 | 21.0422 | 26.1984 | 3.0755 | 2.9826 | 3.2088 |
Datasets | Model | RSME | MAE | BLEU1 | BLEU4 | ROUGE_1/f | ROUGE_2/f |
---|---|---|---|---|---|---|---|
Reasoner | GCLTE | 1.1147 | 0.9189 | 20.8766 | 3.4942 | 22.7577 | 6.0271 |
1.2098 | 1.0023 | 18.1429 | 3.0023 | 20.0818 | 4.9880 | ||
Improve | 0.09 | 0.08 | 0.15 | 0.16 | 0.13 | 0.21 | |
1.1189 | 0.9201 | 16.6446 | 2.0450 | 19.2808 | 4.7288 | ||
Improve | 0.01 | 0.01 | 0.25 | 0.71 | 0.18 | 0.27 | |
1.1145 | 0.9193 | 16.1304 | 2.8761 | 18.6526 | 3.7841 | ||
Improve | 0.01 | 0.01 | 0.29 | 0.21 | 0.22 | 0.59 | |
Music | GCLTE | 1.0358 | 0.8065 | 21.4245 | 1.7071 | 22.2740 | 4.0968 |
1.0967 | 0.8573 | 18.6119 | 1.2658 | 20.9199 | 3.1654 | ||
Improve | 0.06 | 0.6 | 0.15 | 0.35 | 0.06 | 0.29 | |
1.0454 | 0.8021 | 13.9831 | 0.7928 | 16.1360 | 1.8572 | ||
Improve | 0.01 | 0.01 | 0.53 | 1.15 | 0.38 | 1.20 | |
1.0397 | 0.8225 | 14.3898 | 1.1689 | 17.6789 | 3.3935 | ||
Improve | 0.01 | 0.02 | 0.49 | 0.46 | 0.26 | 0.21 | |
Video | GCLTE | 1.0816 | 0.8476 | 21.1844 | 1.1768 | 23.2173 | 3.0755 |
1.1193 | 0.8844 | 17.0556 | 1.0914 | 20.6873 | 2.6436 | ||
Improve | 0.03 | 0.04 | 0.24 | 0.08 | 0.12 | 0.16 | |
1.0817 | 0.8473 | 14.1067 | 0.8425 | 16.9328 | 1.7363 | ||
Improve | 0.01 | 0.01 | 0.50 | 0.40 | 0.37 | 0.77 | |
1.0901 | 0.8440 | 14.3178 | 0.6712 | 18.1082 | 1.8650 | ||
Improve | 0.01 | 0.01 | 0.48 | 0.75 | 0.28 | 0.65 |
Model | Total Time (s) | Epochs | Time/Epoch (s) | GPU Memory Usage (MiB) |
---|---|---|---|---|
NRT | 130.1 | 38 | 2.6 | 2429 |
PETER | 222.3 | 29 | 5.1 | 3243 |
GCLTE | 116.5 | 13 | 4.5 | 3807 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhu, X.; Xia, X.; Wu, Y.; Zhao, W. Enhancing Explainable Recommendations: Integrating Reason Generation and Rating Prediction through Multi-Task Learning. Appl. Sci. 2024, 14, 8303. https://doi.org/10.3390/app14188303
Zhu X, Xia X, Wu Y, Zhao W. Enhancing Explainable Recommendations: Integrating Reason Generation and Rating Prediction through Multi-Task Learning. Applied Sciences. 2024; 14(18):8303. https://doi.org/10.3390/app14188303
Chicago/Turabian StyleZhu, Xingyu, Xiaona Xia, Yuheng Wu, and Wenxu Zhao. 2024. "Enhancing Explainable Recommendations: Integrating Reason Generation and Rating Prediction through Multi-Task Learning" Applied Sciences 14, no. 18: 8303. https://doi.org/10.3390/app14188303
APA StyleZhu, X., Xia, X., Wu, Y., & Zhao, W. (2024). Enhancing Explainable Recommendations: Integrating Reason Generation and Rating Prediction through Multi-Task Learning. Applied Sciences, 14(18), 8303. https://doi.org/10.3390/app14188303