Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback
Abstract
:1. Introduction
2. Materials and Methods
2.1. System Description
2.2. Control Architecture
2.3. Human Subjects
2.4. Experimental Paradigm
2.5. EEG Recording and Processing
2.6. Grasp Force Detection and Biofeedback System
2.7. Task Design
3. Experiment Results
3.1. Task Success Rate of Cursor Control Training
3.2. The ERD/ERS Phenomenon
3.3. Trajectory of Robotic Arm
3.4. Online Control Task
4. Discussion
5. Conclusions and Future Work
Author Contributions
Funding
Conflicts of Interest
References
- Wolpaw, J.R.; Birbaumer, N.; McFarland, D.J.; Pfurtscheller, G.; Vaughan, T.M. Brain–computer interfaces for communication and control. Clin. Neurophysiol. 2002, 113, 767–791. [Google Scholar] [CrossRef]
- Van Gerven, M.; Farquhar, J.; Schaefer, R.; Vlek, R.; Geuze, J.; Nijholt, A.; Ramsey, A.; Haselager, P.; Vuurpijl, L.; Gielen, S.; et al. The brain–computer interface cycle. J. Neural Eng. 2009, 6, 041001. [Google Scholar] [CrossRef]
- Romano, D.; Donati, E.; Benelli, G.; Stefanini, C. A review on animal-robot interaction: From bio-hybrid organisms to mixed societies. Biol. Cybern. 2019, 113, 201–225. [Google Scholar] [CrossRef]
- Soekadar, S.R.; Witkowski, M.; Gómez, C.; Opisso, E.; Medina, J.; Cortese, M.; Vitiello, N. Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia. Sci. Robot. 2016, 1, 3296. [Google Scholar] [CrossRef] [Green Version]
- Liu, J.; Abd-El-Barr, M.; Chi, J.H. Long-term Training with a Brain-Machine Interface-Based Gait Protocol Induces Partial Neurological Recovery in Paraplegic Patients. Neurosurgery 2016, 79, N13–N14. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Xia, B.; Maysam, O.; Veser, S.; Cao, L.; Li, J.; Jia, J.; Xie, H.; Birbaumer, N. A combination strategy based brain–computer interface for two-dimensional movement control. J. Neural Eng. 2015, 12, 046021. [Google Scholar] [CrossRef] [PubMed]
- Minati, L.; Yoshimura, N.; Koike, Y. Hybrid control of a vision-guided robot arm by EOG, EMG, EEG biosignals and head movement acquired via a consumer-grade wearable device. IEEE Access 2016, 4, 9528–9541. [Google Scholar] [CrossRef]
- Meng, J.; Zhang, S.; Bekyo, A.; Olsoe, J.; Baxter, B.; He, B. Noninvasive electroencephalogram based control of a robotic arm for reach and grasp tasks. Sci. Rep. 2016, 6, 38565. [Google Scholar] [CrossRef] [Green Version]
- Casey, A.; Azhar, H.; Grzes, M.; Sakel, M. BCI controlled robotic arm as assistance to the rehabilitation of neurologically disabled patients. Disabil. Rehabil. Assist. Technol. 2019, 1–13. [Google Scholar] [CrossRef]
- McMullen, D.P.; Hotson, G.; Katyal, K.D.; Wester, B.A.; Fifer, M.S.; McGee, T.G.; Harris, A.; Johannes, M.S.; Vogelstein, R.J.; Ravitz, A.D.; et al. Demonstration of a semi-autonomous hybrid brain–machine interface using human intracranial EEG, eye tracking, and computer vision to control a robotic upper limb prosthetic. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 784–796. [Google Scholar] [CrossRef] [Green Version]
- Hotson, G.; McMullen, D.P.; Fifer, M.S.; Johannes, M.S.; Katyal, K.D.; Para, M.P.; Armiger, R.; Anderson, W.S.; Thakor, N.V.; Wester, B.A.; et al. Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject. J. Neural Eng. 2016, 13, 026017. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Fernández-Rodríguez, Á.; Velasco-Álvarez, F.; Bonnet-Save, M.; Ron-Angevin, R. Evaluation of switch and continuous navigation paradigms to command a brain-controlled wheelchair. Front. Neurosci. 2018, 12, 438. [Google Scholar] [CrossRef] [PubMed]
- Lazarou, I.; Nikolopoulos, S.; Petrantonakis, P.C.; Kompatsiaris, I.; Tsolaki, M. EEG-Based Brain–Computer Interfaces for Communication and Rehabilitation of People with Motor Impairment: A Novel Approach of the 21st Century. Front. Hum. Neurosci. 2018, 12, 14. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- He, Y.; Eguren, D.; Azorín, J.M.; Grossman, R.G.; Luu, T.P.; Contreras-Vidal, J.L. Brain–machine interfaces for controlling lower-limb powered robotic systems. J. Neural Eng. 2018, 15, 021004. [Google Scholar] [CrossRef]
- Xu, B.G.; Song, A.G.; Zhao, G.P.; Liu, J.; Xu, G.Z.; Pan, L.Z.; Yang, R.H.; Li, H.J.; Cui, J.W. EEG-modulated robotic rehabilitation system for upper extremity. Biotechnol. Biotechnol. Equip. 2018, 32, 795–803. [Google Scholar] [CrossRef] [Green Version]
- Delisle-Rodriguez, D.; Cardoso, V.; Gurve, D.; Loterio, F.; Romero-Laiseca, M.A.; Krishnan, S.; Bastos Filho, T. System based on subject-specific bands to recognize pedaling motor imagery: Towards a BCI for lower-limb rehabilitation. J. Neural Eng. 2019, 16, 056005. [Google Scholar] [CrossRef] [Green Version]
- Escolano, C.; Antelis, J.M.; Minguez, J. A telepresence mobile robot controlled with a noninvasive brain–computer interface. IEEE Trans. Syst. Man. Cybern. Part B Cybern. 2012, 42, 793–804. [Google Scholar] [CrossRef] [PubMed]
- Qiu, S.; Li, Z.; He, W.; Zhang, L.; Yang, C.; Su, C.Y. Brain-Machine Interface and Visual Compressive Sensing-Based Teleoperation Control of an Exoskeleton Robot. IEEE Trans. Fuzzy Syst. 2017, 25, 58–69. [Google Scholar] [CrossRef] [Green Version]
- Zhao, S.; Li, Z.; Cui, R.; Kang, Y.; Sun, F.; Song, R. Brain–machine interfacing-based teleoperation of multiple coordinated mobile robots. IEEE Trans. Ind. Electron. 2017, 64, 5161–5170. [Google Scholar] [CrossRef]
- Lo, C.C.; Chien, T.Y.; Chen, Y.C.; Tsai, S.H.; Fang, W.C.; Lin, B.S. A wearable channel selection-based brain-computer interface for motor imagery detection. Sensors 2016, 16, 213. [Google Scholar] [CrossRef] [Green Version]
- Bousseta, R.; El Ouakouak, I.; Gharbi, M.; Regragui, F. EEG based brain computer interface for controlling a robot arm movement through thought. Innov. Res. Biomed. Eng. 2018, 39, 129–135. [Google Scholar] [CrossRef]
- Li, Y.; Zhou, G.; Graham, D.; Holtzhauer, A. Towards an EEG-based brain-computer interface for online robot control. Multimed. Tools Appl. 2016, 75, 7999–8017. [Google Scholar] [CrossRef]
- Xu, B.G.; Song, A.G.; Zhao, G.P.; Xu, G.Z.; Pan, L.Z.; Yang, R.H.; Li, H.J.; Cui, J.W.; Zeng, H. Robotic neurorehabilitation system design for stroke patients. Adv. Mech. Eng. 2015, 7, 1687814015573768. [Google Scholar] [CrossRef]
- Acharya, J.N.; Hani, A.; Cheek, J.; Thirumala, P.; Tsuchida, T.N. American Clinical Neurophysiology Society Guideline 2: Guidelines for Standard Electrode Position Nomenclature. J. Clin. Neurophysiol. 2016, 33, 308–311. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Schalk, G.; McFarland, D.J.; Hinterberger, T.; Birbaumer, N.; Wolpaw, J.R. BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 2004, 51, 1034–1043. [Google Scholar] [CrossRef]
- Wolpaw, J.R.; McFarland, D.J. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. USA 2004, 101, 17849–17854. [Google Scholar] [CrossRef] [Green Version]
- De Nunzio, A.M.; Dosen, S.; Lemling, S.; Markovic, M.; Schweisfurth, M.A.; Ge, N.; Graimann, B.; Falla, D.; Farina, D. Tactile feedback is an effective instrument for the training of grasping with a prosthesis at low-and medium-force levels. Exp. Brain Res. 2017, 235, 2547–2559. [Google Scholar] [CrossRef]
- Patel, G.K.; Dosen, S.; Castellini, C.; Farina, D. Multichannel electrotactile feedback for simultaneous and proportional myoelectric control. J. Neural Eng. 2016, 13, 056015. [Google Scholar] [CrossRef]
- Wu, C.C.; Song, A.G.; Ling, Y.; Wang, N.; Tian, L. A control strategy with tactile perception feedback for EMG prosthetic hand. J. Sens. 2015, 2015, 869175. [Google Scholar] [CrossRef] [Green Version]
- Xu, X.N.; Song, A.G.; Ni, D.J.; Li, H.J.; Xiong, P.W.; Zhu, C.C. Visual-haptic aid teleoperation based on 3-D environment modeling and updating. IEEE Trans. Ind. Electron. 2016, 63, 6419–6428. [Google Scholar] [CrossRef]
- Romano, D.; Benelli, G.; Stefanini, C. Encoding lateralization of jump kinematics and eye use in a locust via bio-robotic artifacts. J. Exp. Biol. 2018, 222, jeb187427. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Romano, D.; Benelli, G.; Hwang, J.S.; Stefanini, C. Fighting fish love robots: Mate discrimination in males of a highly territorial fish by using female-mimicking robotic cues. Hydrobiologia 2019, 833, 185–196. [Google Scholar] [CrossRef]
- Nishinoma, H.; Ohno, K.; Kikusui, T.; Nagasawa, M.; Tsuchihashi, N.; Matsushita, S.; Mikayama, T.; Tomori, S.; Saito, M.; Murayama, M.; et al. Canine Motion Control Using Bright Spotlight Devices Mounted on a Suit. IEEE Trans. Med. Robot. Bionics 2019, 1, 189–198. [Google Scholar] [CrossRef]
- Polverino, G.; Karakaya, M.; Spinello, C.; Soman, V.R.; Porfiri, M. Behavioural and life-history responses of mosquitofish to biologically inspired and interactive robotic predators. J. R. Soc. Interface 2019, 16, 20190359. [Google Scholar] [CrossRef] [Green Version]
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Xu, B.; Li, W.; He, X.; Wei, Z.; Zhang, D.; Wu, C.; Song, A. Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback. Electronics 2020, 9, 174. https://doi.org/10.3390/electronics9010174
Xu B, Li W, He X, Wei Z, Zhang D, Wu C, Song A. Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback. Electronics. 2020; 9(1):174. https://doi.org/10.3390/electronics9010174
Chicago/Turabian StyleXu, Baoguo, Wenlong Li, Xiaohang He, Zhiwei Wei, Dalin Zhang, Changcheng Wu, and Aiguo Song. 2020. "Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback" Electronics 9, no. 1: 174. https://doi.org/10.3390/electronics9010174
APA StyleXu, B., Li, W., He, X., Wei, Z., Zhang, D., Wu, C., & Song, A. (2020). Motor Imagery Based Continuous Teleoperation Robot Control with Tactile Feedback. Electronics, 9(1), 174. https://doi.org/10.3390/electronics9010174