A Survey on Population-Based Deep Reinforcement Learning

Long, Weifan; Hou, Taixian; Wei, Xiaoyi; Yan, Shichao; Zhai, Peng; Zhang, Lihua

doi:10.3390/math11102234

Open AccessReview

A Survey on Population-Based Deep Reinforcement Learning

by

Weifan Long

¹

,

Taixian Hou

¹,

Xiaoyi Wei

¹,

Shichao Yan

¹,

Peng Zhai

^1,2,3,*

and

Lihua Zhang

^1,4,5,*

¹

Academy for Engineering and Technology, Fudan University, Shanghai 200433, China

²

Ji Hua Laboratory, Foshan 528251, China

³

Engineering Research Center of AI and Robotics, Ministry of Education, Shanghai 200433, China

⁴

Institute of Meta-Medical, Fudan University, Shanghai 200433, China

⁵

Jilin Provincial Key Laboratory of Intelligence Science and Engineering, Changchun 130013, China

^*

Authors to whom correspondence should be addressed.

Mathematics 2023, 11(10), 2234; https://doi.org/10.3390/math11102234

Submission received: 31 March 2023 / Revised: 7 May 2023 / Accepted: 8 May 2023 / Published: 10 May 2023

(This article belongs to the Special Issue Recent Advances in Swarm Intelligence Algorithms and Their Applications)

Download

Browse Figures

Versions Notes

Abstract

Many real-world applications can be described as large-scale games of imperfect information, which require extensive prior domain knowledge, especially in competitive or human–AI cooperation settings. Population-based training methods have become a popular solution to learn robust policies without any prior knowledge, which can generalize to policies of other players or humans. In this survey, we shed light on population-based deep reinforcement learning (PB-DRL) algorithms, their applications, and general frameworks. We introduce several independent subject areas, including naive self-play, fictitious self-play, population-play, evolution-based training methods, and the policy-space response oracle family. These methods provide a variety of approaches to solving multi-agent problems and are useful in designing robust multi-agent reinforcement learning algorithms that can handle complex real-life situations. Finally, we discuss challenges and hot topics in PB-DRL algorithms. We hope that this brief survey can provide guidance and insights for researchers interested in PB-DRL algorithms.

Keywords: reinforcement learning; multi-agent reinforcement learning; self play; population play

Share and Cite

MDPI and ACS Style

Long, W.; Hou, T.; Wei, X.; Yan, S.; Zhai, P.; Zhang, L. A Survey on Population-Based Deep Reinforcement Learning. Mathematics 2023, 11, 2234. https://doi.org/10.3390/math11102234

AMA Style

Long W, Hou T, Wei X, Yan S, Zhai P, Zhang L. A Survey on Population-Based Deep Reinforcement Learning. Mathematics. 2023; 11(10):2234. https://doi.org/10.3390/math11102234

Chicago/Turabian Style

Long, Weifan, Taixian Hou, Xiaoyi Wei, Shichao Yan, Peng Zhai, and Lihua Zhang. 2023. "A Survey on Population-Based Deep Reinforcement Learning" Mathematics 11, no. 10: 2234. https://doi.org/10.3390/math11102234

APA Style

Long, W., Hou, T., Wei, X., Yan, S., Zhai, P., & Zhang, L. (2023). A Survey on Population-Based Deep Reinforcement Learning. Mathematics, 11(10), 2234. https://doi.org/10.3390/math11102234

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

A Survey on Population-Based Deep Reinforcement Learning

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI