A Closer-to-Reality Model for Comparing Relevant Dimensions of Recommender Systems, with Application to Novelty
Abstract
:1. Introduction
1.1. Evaluation Metrics
1.2. Evaluation Process
- To propose a new model closer to reality, integrating accuracy and novelty, to evaluate the performance of recommendation algorithms;
- To propose a new perspective for comparing recommendation algorithms;
- To provide guidance on which algorithms to use in real-world recommendation applications involving both accuracy and novelty.
2. Related Work
2.1. Defining Novelty
2.2. Quantifying Novelty
2.3. Alternative Metrics
2.4. Data Splitting
3. Proposed Model
4. Experimental Methodology
4.1. Datasets
4.2. Evaluation Metrics
4.2.1. Accuracy
4.2.2. Novelty
- is a normalizing constant, fixed to ;
- is the novelty of item i in a particular context and can be measured in various ways (see Item Novelty paragraph);
- is the estimation of an item discovery depending on its position l in the recommendation list (see Item Discovery paragraph);
- is the item relevance meaning the interest of a user u for a specific item i (see Item Relevance paragraph).
Item Novelty
- Based on the number of user–item interactions observed among the interactions [54]:
- Based on the fraction of total users who rated item i [28]:
Item Discovery
Item Relevance
5. Algorithms
5.1. Classic Algorithms
5.1.1. Matrix Factorization (MF)
5.1.2. User-Based Collaborative Filtering (CF)
5.1.3. Latent Class Model (LC)
5.2. Novelty-Oriented Algorithms
5.2.1. Inverted Recommendations (IR)
5.2.2. Reranking (RR)
5.2.3. Exploriometer (XP)
6. Results
Analysis
7. Discussions
8. Conclusions and Future Work
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Jannach, D.; Resnick, P.; Tuzhilin, A.; Zanker, M. Recommender Systems—Beyond Matrix Completion. Commun. ACM 2016, 59, 94–102. [Google Scholar] [CrossRef]
- Jalili, M.; Ahmadian, S.; Izadi, M.; Moradi, P.; Salehi, M. Evaluating Collaborative Filtering Recommender Algorithms: A Survey. IEEE Access 2018, 6, 74003–74024. [Google Scholar] [CrossRef]
- Bobadilla, J.; Alonso, S.; Hernando, A. Deep Learning Architecture for Collaborative Filtering Recommender Systems. Appl. Sci. 2020, 10, 2441. [Google Scholar] [CrossRef] [Green Version]
- Ricci, F.; Rokach, L.; Shapira, B.; Kantor, P.B. Recommender Systems Handbook; Springer: Berlin/Heidelberg, Germany, 2011. [Google Scholar]
- Bobadilla, J.; Ortega, F.; Hernando, A.; Gutierrez, A. Recommender systems survey. Knowl. Based Syst. 2013, 46, 109–132. [Google Scholar] [CrossRef]
- Fouss, F.; Saerens, M. Evaluating Performance of Recommender Systems: An Experimental Comparison. In Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Sydney, Australia, 9–12 December 2008; pp. 735–738. [Google Scholar] [CrossRef] [Green Version]
- Shani, G.; Gunawardana, A. Evaluating Recommendation Systems. In Recommender Systems Handbook; Springer: Boston, MA, USA, 2011; pp. 257–297. [Google Scholar] [CrossRef]
- Konstan, J.A.; Riedl, J. Recommender systems: From algorithms to user experience. User Model. User Adapt. Interact. 2012, 22, 101–123. [Google Scholar] [CrossRef] [Green Version]
- Shafqat, W.; Byun, Y.C. Enabling “Untact” Culture via Online Product Recommendations: An Optimized Graph-CNN based Approach. Appl. Sci. 2020, 10, 5445. [Google Scholar] [CrossRef]
- NGuyen, L.; Hong, M.S.; Jung, J.; Sohn, B.S. Cognitive Similarity-Based Collaborative Filtering Recommendation System. Appl. Sci. 2020, 10, 4183. [Google Scholar] [CrossRef]
- Valcarce, D.; Bellogín, A.; Parapar, J.; Castells, P. On the Robustness and Discriminative Power of Information Retrieval Metrics for top-N Recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, BC, Canada, 2 October 2018; pp. 260–268. [Google Scholar] [CrossRef]
- del Olmo, F.H.; Gaudioso, E. Evaluation of recommender systems: A new approach. Expert Syst. Appl. 2008, 35, 790–804. [Google Scholar] [CrossRef]
- Candillier, L.; Meyer, F.; Boulle, M. Comparing state-of-the-art collaborative filtering systems. In Proceedings of the International Workshop on Machine Learning and Data Mining in Pattern Recognition, Leipzig, Germany, 18–20 July 2007; pp. 548–562. [Google Scholar]
- Lara-Cabrera, R.; Gonzalez-Prieto, A.; Ortega, F. Deep Matrix Factorization Approach for Collaborative Filtering Recommender Systems. Appl. Sci. 2020, 10, 4926. [Google Scholar] [CrossRef]
- Tan, Z.; He, L.; Wu, D.; Chang, Q.; Zhang, B. Personalized Standard Deviations Improve the Baseline Estimation of Collaborative Filtering Recommendation. Appl. Sci. 2020, 10, 4756. [Google Scholar] [CrossRef]
- Riley, P. Three Pitfalls to Avoid in Machine Learning; Springer: Berlin/Heidelberg, Germany, 2019; Volume 572, pp. 27–29. [Google Scholar] [CrossRef] [Green Version]
- Zhang, F.; Gong, T.; Lee, V.E.; Zhao, G.; Rong, C.; Qu, G. Fast algorithms to evaluate collaborative filtering recommender systems. Knowl. Based Syst. 2015, 96, 96–103. [Google Scholar] [CrossRef]
- Fouss, F.; Pirotte, A.; Renders, J.M.; Saerens, M. Random-walk computation of similarities between nodes of a graph, with application to collaborative recommendation. IEEE Trans. Knowl. Data Eng. 2007, 19, 355–369. [Google Scholar] [CrossRef]
- Bellogín, A.; Cantador, I.; Díez, F.; Castells, P.; Chavarriaga, E. An Empirical Comparison of Social, Collaborative Filtering, and Hybrid Recommenders. ACM Trans. Intell. Syst. Technol. 2013, 4, 1–29. [Google Scholar] [CrossRef] [Green Version]
- Cremonesi, P.; Koren, Y.; Turrin, R. Performance of Recommender Algorithms on Top-n Recommendation Tasks. In Proceedings of the Fourth ACM Conference on Recommender Systems, Barcelona, Spain, 26–30 September 2010; pp. 39–46. [Google Scholar] [CrossRef]
- Bellogín, A.; Castells, P.; Cantador, I. Statistical biases in Information Retrieval metrics for recommender systems. Inf. Retr. J. 2017, 20, 606–634. [Google Scholar] [CrossRef] [Green Version]
- Herlocker, J.L.; Konstan, J.A.; Terveen, L.G.; Riedl, J.T. Evaluating Collaborative Filtering Recommender Systems. ACM Trans. Inf. Syst. 2004, 22, 5–53. [Google Scholar] [CrossRef]
- Yao, S.; Huang, B. Beyond Parity: Fairness Objectives for Collaborative Filtering. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 4–9 December 2017; pp. 2925–2934. [Google Scholar]
- Elahi, M.; Abdollahpouri, H.; Mansoury, M.; Torkamaan, H. Beyond Algorithmic Fairness in Recommender Systems. In Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization, Utrecht, The Netherlands, 21–25 June 2021; pp. 41–46. [Google Scholar]
- Li, Y.; Ge, Y.; Zhang, Y. Tutorial on Fairness of Machine Learning in Recommender Systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Montreal, QC, Canada, 11–15 July 2021; pp. 2654–2657. [Google Scholar]
- Gravino, P.; Monechi, B.; Loreto, V. Towards novelty-driven recommender systems. Comptes Rendus Phys. 2019, 20, 371–379. [Google Scholar] [CrossRef]
- Nandal, N.; Rana, C. Techniques for Novelty in Recommender System. Int. J. Adv. Res. Comput. Sci. Softw. Eng. 2013, 3, 751–755. [Google Scholar]
- Kaminskas, M.; Bridge, D. Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems. ACM Trans. Interact. Intell. Syst. 2016, 7, 1–42. [Google Scholar] [CrossRef]
- Baeza-Yates, R.; Ribeiro, B.d.A.N. Modern Information Retrieval; Addison-Wesley: Harlow, UK, 1999. [Google Scholar] [CrossRef]
- Zhang, L. The definition of novelty in recommendation system. J. Eng. Sci. Technol. Rev. 2013, 6, 141–145. [Google Scholar] [CrossRef]
- Hurley, N.; Zhang, M. Novelty and Diversity in Top-N Recommendation—Analysis and Evaluation. ACM Trans. Internet Technol. 2011, 10. [Google Scholar] [CrossRef]
- Vargas, S.; Castells, P. Rank and Relevance in Novelty and Diversity Metrics for Recommender Systems. In Proceedings of the Fifth ACM Conference on Recommender Systems, Chicago, IL, USA, 23–27 October 2011; pp. 109–116. [Google Scholar] [CrossRef] [Green Version]
- Adamopoulos, P.; Tuzhilin, A. On Unexpectedness in Recommender Systems: Or How to Better Expect the Unexpected. ACM Trans. Intell. Syst. Technol. 2014, 5, 1–32. [Google Scholar] [CrossRef]
- Oh, J.; Park, S.; Yu, H.; Song, M.; Park, S.T. Novel Recommendation Based on Personal Popularity Tendency. In Proceedings of the 2011 IEEE 11th International Conference on Data Mining, Vancouver, BC, Canada, 11–14 December 2011; pp. 507–516. [Google Scholar] [CrossRef]
- Kapoor, K.; Kumar, V.; Terveen, L.; Konstan, J.A.; Schrater, P. “I like to Explore Sometimes”: Adapting to Dynamic User Novelty Preferences. In Proceedings of the 9th ACM Conference on Recommender Systems, Vienna, Austria, 16–20 September 2015; pp. 19–26. [Google Scholar] [CrossRef]
- Zhou, T.; Kuscsik, Z.; Liu, J.G.; Medo, M.; Wakeling, J.R.; Zhang, Y.C. Solving the apparent diversity-accuracy dilemma of recommender systems. Proc. Natl. Acad. Sci. USA 2010, 107, 4511–4515. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Liu, J.G.; Shi, K.; Guo, Q. Solving the accuracy-diversity dilemma via directed random walks. Phys. Rev. E 2012, 85. [Google Scholar] [CrossRef] [Green Version]
- Shi, L. Trading-off Among Accuracy, Similarity, Diversity, and Long-tail: A Graph-based Recommendation Approach. In Proceedings of the 7th ACM Conference on Recommender Systems, Hong Kong, China, 12–16 October 2013; pp. 57–64. [Google Scholar] [CrossRef]
- Belém, F.M.; Batista, C.S.; Santos, R.L.T.; Almeida, J.M.; Gonçalves, M.A. Beyond Relevance: Explicitly Promoting Novelty and Diversity in Tag Recommendation. ACM Trans. Intell. Syst. Technol. 2016, 7, 1–34. [Google Scholar] [CrossRef]
- Yin, H.; Cui, B.; Li, J.; Yao, J.; Chen, C. Challenging the Long Tail Recommendation. Proc. VLDB Endow. 2012, 5, 896–907. [Google Scholar] [CrossRef] [Green Version]
- Maksai, A.; Garcin, F.; Faltings, B. Predicting Online Performance of News Recommender Systems Through Richer Evaluation Metrics. In Proceedings of the 9th ACM Conference on Recommender Systems, Vienna, Austria, 16–20 September 2015; pp. 179–186. [Google Scholar] [CrossRef]
- Ziegler, C.N.; McNee, S.M.; Konstan, J.A.; Lausen, G. Improving Recommendation Lists Through Topic Diversification. In Proceedings of the 14th International Conference on World Wide Web, Chiba, Japan, 10–14 May 2005; pp. 22–32. [Google Scholar] [CrossRef] [Green Version]
- Ribeiro, M.T.; Ziviani, N.; Moura, E.S.D.; Hata, I.; Lacerda, A.; Veloso, A. Multiobjective Pareto-Efficient Approaches for Recommender Systems. ACM Trans. Intell. Syst. Technol. 2015, 5, 53–73. [Google Scholar] [CrossRef]
- Vanchinathan, H.P.; Nikolic, I.; De Bona, F.; Krause, A. Explore-Exploit in Top-N Recommender Systems via Gaussian Processes. In Proceedings of the 8th ACM Conference on Recommender Systems, Foster City, Silicon Valley, CA, USA, 6–10 October 2014. [Google Scholar]
- Steck, H. Training and Testing of Recommender Systems on Data Missing Not at Random. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, 24–28 July 2010; pp. 713–722. [Google Scholar] [CrossRef]
- Steck, H. Item Popularity and Recommendation Accuracy. In Proceedings of the Fifth ACM Conference on Recommender Systems, Chicago, IL, USA, 23–27 October 2011; pp. 125–132. [Google Scholar] [CrossRef]
- Shi, Y.; Karatzoglou, A.; Baltrunas, L.; Larson, M.; Oliver, N.; Hanjalic, A. CLiMF: Learning to Maximize Reciprocal Rank with Collaborative Less-is-more Filtering. In Proceedings of the Sixth ACM Conference on Recommender Systems, Dublin, Ireland, 9–13 September 2012; pp. 139–146. [Google Scholar] [CrossRef]
- Said, A.; Bellogín, A. Comparative Recommender System Evaluation: Benchmarking Recommendation Frameworks. In Proceedings of the 8th ACM Conference on Recommender Systems, Foster City, Silicon Valley, CA, USA, 6–10 October 2014. [Google Scholar]
- Gunawardana, A.; Shani, G. A survey of accuracy evaluation metrics of recommendation tasks. J. Mach. Learn. Res. 2009, 10, 2935–2962. [Google Scholar] [CrossRef]
- Zhang, M.; Hurley, N. Avoiding Monotony: Improving the Diversity of Recommendation Lists. In Proceedings of the 2008 ACM Conference on Recommender Systems, Lausanne, Switzerland, 23–25 October 2008; pp. 123–130. [Google Scholar] [CrossRef]
- Salton, G. Automatic Information Organization and Retrieval; McGraw-Hill: Maidenhead, NY, USA, 1968. [Google Scholar] [CrossRef]
- Wang, Y.; Wang, L.; Li, Y.; He, D.; Chen, W.; Liu, T.Y. A Theoretical Analysis of NDCG Ranking Measures. J. Mach. Learn. Res. 2013, 8, 1–30. [Google Scholar]
- Breese, J.S.; Heckerman, D.; Kadie, C. Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, Madison, WI, USA, 24–26 July 1998; pp. 43–52. [Google Scholar] [CrossRef]
- Han, J.; Yamana, H. A Survey on Recommendation Methods Beyond Accuracy. IEICE Trans. Inf. Syst. 2017, 100, 2931–2944. [Google Scholar] [CrossRef] [Green Version]
- Koren, Y.; Bell, R.; Volinsky, C. Matrix Factorization Techniques for Recommender Systems. Computer 2009, 42, 30–37. [Google Scholar] [CrossRef]
- Nguyen, H.; Dinh, T. A Modified Regularized Non-Negative Matrix Factorization for MovieLens. In Proceedings of the 2012 IEEE RIVF International Conference on Computing Communication Technologies, Research, Innovation, and Vision for the Future, Ho Chi Minh City, Vietnam, 27 February–1 March 2012; pp. 1–5. [Google Scholar] [CrossRef]
- Wu, Z.; Tian, H.; Zhu, X.; Wang, S. Optimization Matrix Factorization Recommendation Algorithm Based on Rating Centrality. In Data Mining and Big Data; Tan, Y., Shi, Y., Tang, Q., Eds.; Springer International Publishing: Cham, Swizterland, 2018; pp. 114–125. [Google Scholar] [CrossRef] [Green Version]
- Zhao, X.; Niu, Z.; Chen, W. Opinion-Based Collaborative Filtering to Solve Popularity Bias in Recommender Systems. In Database and Expert Systems Applications; Decker, H., Lhotská, L., Link, S., Basl, J., Tjoa, A.M., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 426–433. [Google Scholar]
- Dunham, M. Data Mining: Introductory and Advanced Topics; Prentice Hall: Hoboken, NJ, USA, 2003. [Google Scholar]
- Hofmann, T.; Puzicha, J. Latent Class Models for Collaborative Filtering. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence, Stockholm, Sweden, 31 July–6 August 1999; pp. 688–693. [Google Scholar] [CrossRef]
- Fouss, F.; Francoisse, K.; Yen, L.; Pirotte, A.; Saerens, M. An experimental investigation of kernels on graphs for collaborative recommendation and semisupervised classification. Neural Netw. 2012, 31, 53–72. [Google Scholar] [CrossRef] [PubMed]
- Cheung, K.W.; Tsui, K.C.; Liu, J. Extended latent class models for collaborative recommendation. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 2004, 34, 143–148. [Google Scholar] [CrossRef]
- Vargas, S.; Castells, P. Improving Sales Diversity by Recommending Users to Items. In Proceedings of the 8th ACM Conference on Recommender Systems, Foster City, Silicon Valley, CA, USA, 6–10 October 2014; pp. 145–152. [Google Scholar] [CrossRef]
- Adomavicius, G.; Kwon, Y. Improving Aggregate Recommendation Diversity Using Ranking-Based Techniques. IEEE Trans. Knowl. Data Eng. 2012, 24, 896–911. [Google Scholar] [CrossRef]
- Chatzicharalampous, E.; Christos, Z.; Vakali, A. Exploriometer: Leveraging Personality Traits for Coverage and Diversity Aware Recommendations. In Proceedings of the 24th International Conference on World Wide Web, Florence, Italy, 18–22 May 2015; pp. 1463–1468. [Google Scholar] [CrossRef]
- Ekstrand, M.D.; Kluver, D.; Harper, F.M.; Konstan, J.A. Letting Users Choose Recommender Algorithms: An Experimental Study. In Proceedings of the 9th ACM Conference on Recommender Systems, Vienna, Austria, 16–20 September 2015; pp. 11–18. [Google Scholar] [CrossRef]
X | nov | Y | p | Z | ||
---|---|---|---|---|---|---|
1 | 1 | 1 | 1 | |||
2 | ||||||
2 | 1 | 1 | ||||
2 | ||||||
2 | 1 | 1 | 1 | |||
2 | ||||||
2 | 1 | 1 | ||||
2 | ||||||
3 | n/a | 1 | 1 | |||
2 |
Algorithms | Abb. | Parameters | Values |
---|---|---|---|
Matrix Factorization | MF | Latent Factor f | 40 |
User-Based Collaborative Filtering | CF | Number of neighbors k | 20 |
Latent Class Model | LC | Latent Class z | 20 (ML)—15 (BC) |
Inverted Recommendations | IR | Number of neighbors k | 20 |
Reranking | RR | Ranking Threshold | MaxScore/2 |
Exploriometer | XP | Number of neighbors k | 20 |
Book-Crossing | ||||||||||||||||||||||
Accuracy | EPC | |||||||||||||||||||||
NDCG | 1.1.1 | 1.1.2 | 1.2.1 | 1.2.2 | 2.1.1 | 2.1.2 | 2.2.1 | 2.2.2 | 3..1 | 3..2 | ||||||||||||
MF | 1.000 | 1 | 0.073 | 5 | 0.063 | 5 | 0.074 | 5 | 0.060 | 5 | 0.038 | 5 | 0.041 | 5 | 0.044 | 5 | 0.047 | 5 | 0.044 | 5 | 0.046 | 5 |
CF | 0.716 | 2 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 |
LC | 0.550 | 3 | 0.211 | 4 | 0.155 | 4 | 0.211 | 4 | 0.149 | 4 | 0.153 | 4 | 0.115 | 4 | 0.178 | 4 | 0.136 | 4 | 0.178 | 4 | 0.135 | 4 |
IR | 0.443 | 4 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 |
RR | 0.226 | 5 | 0.291 | 3 | 0.707 | 3 | 0.292 | 3 | 0.709 | 3 | 0.205 | 3 | 0.543 | 2 | 0.228 | 3 | 0.587 | 2 | 0.228 | 3 | 0.586 | 2 |
XP | 0.000 | 6 | 0.709 | 2 | 0.719 | 2 | 0.708 | 2 | 0.716 | 2 | 0.560 | 2 | 0.538 | 3 | 0.602 | 2 | 0.583 | 3 | 0.602 | 2 | 0.582 | 3 |
MovieLens | ||||||||||||||||||||||
Accuracy | EPC | |||||||||||||||||||||
NDCG | 1.1.1 | 1.1.2 | 1.2.1 | 1.2.2 | 2.1.1 | 2.1.2 | 2.2.1 | 2.2.2 | 3..1 | 3..2 | ||||||||||||
MF | 1.000 | 1 | 0.322 | 4 | 0.257 | 4 | 0.323 | 4 | 0.256 | 4 | 0.249 | 5 | 0.191 | 4 | 0.249 | 5 | 0.191 | 4 | 0.249 | 5 | 0.191 | 4 |
CF | 0.913 | 3 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 |
LC | 0.965 | 2 | 0.305 | 5 | 0.197 | 5 | 0.305 | 5 | 0.198 | 5 | 0.296 | 4 | 0.185 | 5 | 0.296 | 4 | 0.185 | 5 | 0.297 | 4 | 0.184 | 5 |
IR | 0.775 | 4 | 0.625 | 2 | 0.380 | 2 | 0.624 | 2 | 0.379 | 2 | 0.731 | 2 | 0.394 | 2 | 0.731 | 2 | 0.393 | 2 | 0.731 | 2 | 0.392 | 2 |
RR | 0.000 | 6 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 |
XP | 0.593 | 5 | 0.365 | 3 | 0.308 | 3 | 0.365 | 3 | 0.307 | 3 | 0.301 | 3 | 0.234 | 3 | 0.301 | 3 | 0.234 | 3 | 0.301 | 3 | 0.237 | 3 |
Book-Crossing | MovieLens | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Classical | Classical | ||||||||||||||||
MF | 1.000 | 1 | 0.739 | 4 | 0.000 | 6 | 0.000 | 6 | MF | 1.000 | 1 | 1.000 | 1 | 0.317 | 3 | 0.113 | 5 |
CF | 0.716 | 2 | 0.889 | 2 | 0.579 | 4 | 0.555 | 4 | CF | 0.913 | 3 | 0.413 | 3 | 0.000 | 6 | 0.000 | 6 |
LC | 0.550 | 3 | 1.000 | 1 | 0.319 | 5 | 0.195 | 5 | LC | 0.965 | 2 | 0.909 | 2 | 0.474 | 2 | 0.369 | 3 |
IR | 0.443 | 4 | 0.629 | 5 | 1.000 | 1 | 1.000 | 1 | IR | 0.775 | 4 | 0.223 | 4 | 0.281 | 4 | 0.421 | 2 |
RR | 0.226 | 5 | 0.838 | 3 | 0.970 | 2 | 0.874 | 2 | RR | 0.000 | 6 | 0.067 | 5 | 1.000 | 1 | 1.000 | 1 |
XP | 0.000 | 6 | 0.000 | 6 | 0.643 | 3 | 0.864 | 3 | XP | 0.593 | 5 | 0.000 | 6 | 0.170 | 5 | 0.358 | 4 |
Book-Crossing | MovieLens | ||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Classical | Classical | ||||||||||||||||
MF | 0.044 | 5 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | MF | 0.238 | 5 | 0.276 | 5 | 0.313 | 5 | 0.267 | 5 |
CF | 0.000 | 6 | 0.229 | 5 | 0.149 | 5 | 0.190 | 5 | CF | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 | 0.000 | 6 |
LC | 0.159 | 4 | 0.240 | 4 | 0.198 | 4 | 0.202 | 4 | LC | 0.258 | 4 | 0.313 | 4 | 0.387 | 4 | 0.283 | 4 |
IR | 1.000 | 1 | 0.803 | 2 | 0.885 | 2 | 0.825 | 2 | IR | 0.602 | 2 | 0.801 | 2 | 0.932 | 2 | 0.880 | 2 |
RR | 0.309 | 3 | 0.304 | 3 | 0.246 | 3 | 0.282 | 3 | RR | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | 0.650 | 3 |
XP | 0.588 | 2 | 1.000 | 1 | 1.000 | 1 | 1.000 | 1 | XP | 0.287 | 3 | 0.548 | 3 | 0.741 | 3 | 1.000 | 1 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Fouss, F.; Fernandes, E. A Closer-to-Reality Model for Comparing Relevant Dimensions of Recommender Systems, with Application to Novelty. Information 2021, 12, 500. https://doi.org/10.3390/info12120500
Fouss F, Fernandes E. A Closer-to-Reality Model for Comparing Relevant Dimensions of Recommender Systems, with Application to Novelty. Information. 2021; 12(12):500. https://doi.org/10.3390/info12120500
Chicago/Turabian StyleFouss, François, and Elora Fernandes. 2021. "A Closer-to-Reality Model for Comparing Relevant Dimensions of Recommender Systems, with Application to Novelty" Information 12, no. 12: 500. https://doi.org/10.3390/info12120500