Next Article in Journal
Symmetrical Motifs in Young Children’s Drawings: A Study on Their Representations of Plant Life
Previous Article in Journal
Evaluation of the Influencing Factors on Job Satisfaction Based on Combination of PLS-SEM and F-MULTIMOORA Approach
Previous Article in Special Issue
Pixel-Value-Ordering based Reversible Information Hiding Scheme with Self-Adaptive Threshold Strategy
Article Menu
Issue 1 (January) cover image

Export Article

Open AccessArticle
Symmetry 2019, 11(1), 25; https://doi.org/10.3390/sym11010025

Reusing Source Task Knowledge via Transfer Approximator in Reinforcement Transfer Learning

College of Intelligence Science and Technology, National University of Defense Technology, Changsha 410073, China
*
Authors to whom correspondence should be addressed.
Received: 20 November 2018 / Revised: 20 December 2018 / Accepted: 22 December 2018 / Published: 29 December 2018
(This article belongs to the Special Issue Information Technology and Its Applications 2018)
Full-Text   |   PDF [597 KB, uploaded 29 December 2018]   |  

Abstract

Transfer Learning (TL) has received a great deal of attention because of its ability to speed up Reinforcement Learning (RL) by reusing learned knowledge from other tasks. This paper proposes a new transfer learning framework, referred to as Transfer Learning via Artificial Neural Network Approximator (TL-ANNA). It builds an Artificial Neural Network (ANN) transfer approximator to transfer the related knowledge from the source task into the target task and reuses the transferred knowledge with a Probabilistic Policy Reuse (PPR) scheme. Specifically, the transfer approximator maps the state of the target task symmetrically to states of the source task with a certain mapping rule, and activates the related knowledge (components of the action-value function) of the source task as the input of the ANNs; it then predicts the quality of the actions in the target task with the ANNs. The target learner uses the PPR scheme to bias the RL with the suggested action from the transfer approximator. In this way, the transfer approximator builds a symmetric knowledge path between the target task and the source task. In addition, two mapping rules for the transfer approximator are designed, namely, Full Mapping Rule and Group Mapping Rule. Experiments performed on the RoboCup soccer Keepaway task verified that the proposed transfer learning methods outperform two other transfer learning methods in both jumpstart and time to threshold metrics and are more robust to the quality of source knowledge. In addition, the TL-ANNA with the group mapping rule exhibits slightly worse performance than the one with the full mapping rule, but with less computation and space cost when appropriate grouping method is used. View Full-Text
Keywords: artificial neural networks; probabilistic policy reuse; reinforcement learning; transfer approximator; transfer learning artificial neural networks; probabilistic policy reuse; reinforcement learning; transfer approximator; transfer learning
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Cheng, Q.; Wang, X.; Niu, Y.; Shen, L. Reusing Source Task Knowledge via Transfer Approximator in Reinforcement Transfer Learning. Symmetry 2019, 11, 25.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Symmetry EISSN 2073-8994 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top