Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface
1
Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China
2
Department of physiology, Feinberg School of Medicine, Northwestern University, Chicago, IL 60611, USA
3
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China
4
Advanced Innovation Center for Intelligent Robots and Systems, Beijing Institute of Technology, Beijing 100081, China
*
Author to whom correspondence should be addressed.
Sensors 2020, 20(19), 5528; https://doi.org/10.3390/s20195528
Received: 27 August 2020 / Revised: 15 September 2020 / Accepted: 22 September 2020 / Published: 27 September 2020
(This article belongs to the Special Issue Wearable Sensor for Activity Analysis and Context Recognition)
Background: For the nonstationarity of neural recordings in intracortical brain–machine interfaces, daily retraining in a supervised manner is always required to maintain the performance of the decoder. This problem can be improved by using a reinforcement learning (RL) based self-recalibrating decoder. However, quickly exploring new knowledge while maintaining a good performance remains a challenge in RL-based decoders. Methods: To solve this problem, we proposed an attention-gated RL-based algorithm combining transfer learning, mini-batch, and weight updating schemes to accelerate the weight updating and avoid over-fitting. The proposed algorithm was tested on intracortical neural data recorded from two monkeys to decode their reaching positions and grasping gestures. Results: The decoding results showed that our proposed algorithm achieved an approximate 20% increase in classification accuracy compared to that obtained by the non-retrained classifier and even achieved better classification accuracy than the daily retraining classifier. Moreover, compared with a conventional RL method, our algorithm improved the accuracy by approximately 10% and the online weight updating speed by approximately 70 times. Conclusions: This paper proposed a self-recalibrating decoder which achieved a good and robust decoding performance with fast weight updating and might facilitate its application in wearable device and clinical practice.
View Full-Text
Keywords:
intracortical brain–machine interface; reinforcement learning; adaptive decoder; transfer learning
▼
Show Figures
Figure 1
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Zhang, P.; Chao, L.; Chen, Y.; Ma, X.; Wang, W.; He, J.; Huang, J.; Li, Q. Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface. Sensors 2020, 20, 5528. https://doi.org/10.3390/s20195528
AMA Style
Zhang P, Chao L, Chen Y, Ma X, Wang W, He J, Huang J, Li Q. Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface. Sensors. 2020; 20(19):5528. https://doi.org/10.3390/s20195528
Chicago/Turabian StyleZhang, Peng; Chao, Lianying; Chen, Yuting; Ma, Xuan; Wang, Weihua; He, Jiping; Huang, Jian; Li, Qiang. 2020. "Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface" Sensors 20, no. 19: 5528. https://doi.org/10.3390/s20195528
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
Search more from Scilit
