Next Article in Journal
A Fair Power Allocation Approach to OFDM-Based NOMA with Consideration of Clipping
Next Article in Special Issue
A Deep Reinforcement Learning-Based Power Resource Management for Fuel Cell Powered Data Centers
Previous Article in Journal
A Novel Non-Isolated Three-Port Bidirectional DC/DC Converter for Photovoltaic Electric Scooter Charging Stations
Previous Article in Special Issue
Research and Implementation of Intelligent Decision Based on a Priori Knowledge and DQN Algorithms in Wargame Environment

Episodic Self-Imitation Learning with Hindsight

Department of Bioengineering, Imperial College London, London SW7 2AZ, UK
Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK
Author to whom correspondence should be addressed.
Electronics 2020, 9(10), 1742;
Received: 12 September 2020 / Revised: 15 October 2020 / Accepted: 16 October 2020 / Published: 21 October 2020
(This article belongs to the Special Issue Deep Reinforcement Learning: Methods and Applications)
Episodic self-imitation learning, a novel self-imitation algorithm with a trajectory selection module and an adaptive loss function, is proposed to speed up reinforcement learning. Compared to the original self-imitation learning algorithm, which samples good state–action pairs from the experience replay buffer, our agent leverages entire episodes with hindsight to aid self-imitation learning. A selection module is introduced to filter uninformative samples from each episode of the update. The proposed method overcomes the limitations of the standard self-imitation learning algorithm, a transitions-based method which performs poorly in handling continuous control environments with sparse rewards. From the experiments, episodic self-imitation learning is shown to perform better than baseline on-policy algorithms, achieving comparable performance to state-of-the-art off-policy algorithms in several simulated robot control tasks. The trajectory selection module is shown to prevent the agent learning undesirable hindsight experiences. With the capability of solving sparse reward problems in continuous control settings, episodic self-imitation learning has the potential to be applied to real-world problems that have continuous action spaces, such as robot guidance and manipulation. View Full-Text
Keywords: deep reinforcement learning; hindsight experience replay; imitation learning; exploration deep reinforcement learning; hindsight experience replay; imitation learning; exploration
Show Figures

Figure 1

MDPI and ACS Style

Dai, T.; Liu, H.; Anthony Bharath, A. Episodic Self-Imitation Learning with Hindsight. Electronics 2020, 9, 1742.

AMA Style

Dai T, Liu H, Anthony Bharath A. Episodic Self-Imitation Learning with Hindsight. Electronics. 2020; 9(10):1742.

Chicago/Turabian Style

Dai, Tianhong, Hengyan Liu, and Anil Anthony Bharath. 2020. "Episodic Self-Imitation Learning with Hindsight" Electronics 9, no. 10: 1742.

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

Back to TopTop