Next Article in Journal
Category Maps Describe Driving Episodes Recorded with Event Data Recorders
Previous Article in Journal
Introduction to MAchine Learning & Knowledge Extraction (MAKE)
Open AccessArticle

Learning to Teach Reinforcement Learning Agents

Department of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece
Borealis AI, University of Alberta, CCIS 3-232, Edmonton, AB T6G 2M9, Canada
Author to whom correspondence should be addressed.
Mach. Learn. Knowl. Extr. 2019, 1(1), 21-42;
Received: 19 September 2017 / Revised: 17 November 2017 / Accepted: 1 December 2017 / Published: 6 December 2017
In this article, we study the transfer learning model of action advice under a budget. We focus on reinforcement learning teachers providing action advice to heterogeneous students playing the game of Pac-Man under a limited advice budget. First, we examine several critical factors affecting advice quality in this setting, such as the average performance of the teacher, its variance and the importance of reward discounting in advising. The experiments show that the best performers are not always the best teachers and reveal the non-trivial importance of the coefficient of variation (CV) as a statistic for choosing policies that generate advice. The CV statistic relates variance to the corresponding mean. Second, the article studies policy learning for distributing advice under a budget. Whereas most methods in the relevant literature rely on heuristics for advice distribution, we formulate the problem as a learning one and propose a novel reinforcement learning algorithm capable of learning when to advise or not. The proposed algorithm is able to advise even when it does not have knowledge of the student’s intended action and needs significantly less training time compared to previous learning approaches. Finally, in this article, we argue that learning to advise under a budget is an instance of a more generic learning problem: Constrained Exploitation Reinforcement Learning. View Full-Text
Keywords: machine learning; reinforcement learning; transfer learning; action advice; machine teaching machine learning; reinforcement learning; transfer learning; action advice; machine teaching
Show Figures

Figure 1

MDPI and ACS Style

Fachantidis, A.; Taylor, M.E.; Vlahavas, I. Learning to Teach Reinforcement Learning Agents. Mach. Learn. Knowl. Extr. 2019, 1, 21-42.

Show more citation formats Show less citations formats

Article Access Map by Country/Region

Back to TopTop