Next Article in Journal
A Computation-Efficient Group Key Distribution Protocol Based on a New Secret Sharing Scheme
Previous Article in Journal
Interactivity in Cybermedia News: An Interview with Journalists in Colombia, Peru, and Ecuador
Previous Article in Special Issue
Optimizing Parallel Collaborative Filtering Approaches for Improving Recommendation Systems Performance
Article Menu
Issue 5 (May) cover image

Export Article

Open AccessArticle

Sequeval: An Offline Evaluation Framework for Sequence-Based Recommender Systems

1
Politecnico di Torino, Corso Duca degli Abruzzi 24, 10129 Turin, Italy
2
Istituto Superiore Mario Boella, Via Pier Carlo Boggio 61, 10138 Turin, Italy
3
EURECOM, Campus SophiaTech, 450 Route des Chappes, 06410 Biot, France
4
LINKS Foundation, Via Pier Carlo Boggio 61, 10138 Turin, Italy
*
Author to whom correspondence should be addressed.
Information 2019, 10(5), 174; https://doi.org/10.3390/info10050174
Received: 15 April 2019 / Revised: 30 April 2019 / Accepted: 7 May 2019 / Published: 10 May 2019
(This article belongs to the Special Issue Modern Recommender Systems: Approaches, Challenges and Applications)
  |  
PDF [473 KB, uploaded 10 May 2019]
  |  

Abstract

Recommender systems have gained a lot of popularity due to their large adoption in various industries such as entertainment and tourism. Numerous research efforts have focused on formulating and advancing state-of-the-art of systems that recommend the right set of items to the right person. However, these recommender systems are hard to compare since the published evaluation results are computed on diverse datasets and obtained using different methodologies. In this paper, we researched and prototyped an offline evaluation framework called Sequeval that is designed to evaluate recommender systems capable of suggesting sequences of items. We provide a mathematical definition of such sequence-based recommenders, a methodology for performing their evaluation, and the implementation details of eight metrics. We report the lessons learned using this framework for assessing the performance of four baselines and two recommender systems based on Conditional Random Fields (CRF) and Recurrent Neural Networks (RNN), considering two different datasets. Sequeval is publicly available and it aims to become a focal point for researchers and practitioners when experimenting with sequence-based recommender systems, providing comparable and objective evaluation results. View Full-Text
Keywords: evaluation framework; offline evaluation; sequence; sequence-based recommender systems; recommender systems; metrics evaluation framework; offline evaluation; sequence; sequence-based recommender systems; recommender systems; metrics
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Monti, D.; Palumbo, E.; Rizzo, G.; Morisio, M. Sequeval: An Offline Evaluation Framework for Sequence-Based Recommender Systems. Information 2019, 10, 174.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Information EISSN 2078-2489 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top