Competitive Learning in a Spiking Neural Network: Towards an Intelligent Pattern Classifier
Abstract
:1. Introduction
2. Models and Methods
3. Results
3.1. Spiking Neurons as Electromyographical (EMG) Features Extractors
3.2. Learning and Selective Response of a Single Neuron
3.3. EMG Patterns Classification Problem as an Example of Unsupervised Learning in Spiking Neuron Networks (SNN)
3.4. SNN Supervised Learning
4. Discussion
- (i)
- Hebbian learning (in the current work, through triplet-based STDP);
- (ii)
- synaptic competition or competition of inputs (in the current work, through synaptic forgetting); and
- (iii)
- neural competition or competition of outputs (in the current work, through lateral inhibition).
Supplementary Materials
Author Contributions
Funding
Conflicts of Interest
References
- Roche, A.D.; Rehbaum, H.; Farina, D.; Aszmann, O.C. Prosthetic myoelectric control strategies: A clinical perspective. Curr. Surg. Rep. 2014, 2, 44. [Google Scholar] [CrossRef]
- Hahne, J.M.; Biebmann, F.; Jiang, N.; Rehbaum, H.; Farina, D.; Meinecke, F.C.; Muller, K.-R.; Parra, L.C. Linear and nonlinear regression techniques for simultaneous and proportional myoelectric control. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 269–279. [Google Scholar] [CrossRef]
- Patel, Y.; Nageswaran, S. Myoelectric Controlled Thumb. In Proceedings of the 2018 3rd International Conference for Convergence in Technology (I2CT), Pune, India, 6–8 April 2018; pp. 1–5. [Google Scholar]
- Lima, A.A.M.; Araujo, R.M.; de Santos, F.A.G.; Yoshizumi, V.H.; de Barros, F.K.H.; Spatti, D.H.; Liboni, L.H.B.; Dajer, M.E. Classification of Hand Movements from EMG Signals using Optimized MLP. In Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil, 8–13 July 2018; pp. 1–7. [Google Scholar]
- Roldan-Vasco, S.; Restrepo-Agudelo, S.; Valencia-Martinez, Y.; Orozco-Duque, A. Automatic detection of oral and pharyngeal phases in swallowing using classification algorithms and multichannel EMG. J. Electromyogr. Kinesiol. 2018, 43, 193–200. [Google Scholar] [CrossRef] [PubMed]
- Ullah, K.; Kim, J.H. A mathematical model for mapping EMG signal to joint torque for the human elbow joint using nonlinear regression. In Proceedings of the 2009 4th International Conference on Autonomous Robots and Agents, Wellington, New Zealand, 10–12 Feburary 2009; pp. 103–108. [Google Scholar]
- Ngeo, J.G.; Tamei, T.; Shibata, T. Continuous and simultaneous estimation of finger kinematics using inputs from an EMG-to-muscle activation model. J. Neuroeng. Rehabil. 2014, 11, 122. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chen, J.; Zhang, X.; Cheng, Y.; Xi, N. Surface EMG based continuous estimation of human lower limb joint angles by using deep belief networks. Biomed. Signal. Process. Control. 2018, 40, 335–342. [Google Scholar] [CrossRef]
- Alghofaily, B.; Ding, C. Meta-Feature Based Data Mining Service Selection and Recommendation Using Machine Learning Models. In Proceedings of the 2018 IEEE 15th International Conference on e-Business Engineering (ICEBE), Xi’an, China, 12–14 October 2018; pp. 17–24. [Google Scholar]
- McCulloch, W.S.; Pitts, W. A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 1943, 5, 115–133. [Google Scholar] [CrossRef]
- Widrow, B.; Lehr, M.A. 30 years of adaptive neural networks: Perceptron, Madaline, and backpropagation. Proc. IEEE 1990, 78, 1415–1442. [Google Scholar] [CrossRef]
- Haykin, S. Neural Networks. A Comprehensive Foundation, 2nd ed.; Prentice Hall: Upper Saddle River, NJ, USA, 1998. [Google Scholar]
- Izhikevich, E.M. Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting; The MIT Press: Cambridge, MA, USA, 2007. [Google Scholar]
- Paugam-Moisy, H.; Bohte, S. Computing with Spiking Neuron Networks BT—Handbook of Natural Computing; Rozenberg, G., Bäck, T., Kok, J.N., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 335–376. ISBN 978-3-540-92910-9. [Google Scholar]
- Quiroga, Q.R.; Panzeri, S. Principles of Neural Coding; Taylor & Francis Group: Boca Raton, FL, USA, 2013; ISBN 9781439853313. [Google Scholar]
- Chiolerio, A.; Chiappalone, M.; Ariano, P.; Bocchini, S. Coupling resistive switching devices with neurons: State of the art and perspectives. Front. Neurosci. 2017, 11, 70. [Google Scholar] [CrossRef] [Green Version]
- Mikhaylov, A.N.; Morozov, O.A.; Ovchinnikov, P.E.; Antonov, I.N.; Belov, A.I.; Korolev, D.S.; Sharapov, A.N.; Gryaznov, E.G.; Gorshkov, O.N.; Pigareva, Y.I.; et al. One-Board design and simulation of double-layer perceptron based on metal-oxide memristive nanostructures. IEEE Trans. Emerg. Top. Comput. Intell. 2018, 2, 371–379. [Google Scholar] [CrossRef]
- Llinas, R.R. I of the Vortex: From Neurons to Self; The MIT Press: Cambridge, MA, USA, 2001. [Google Scholar]
- Buzsáki, G. Rhythms of the Brain; Oxford University Press: Oxford, UK, 2006. [Google Scholar]
- Lobov, S.; Kazantsev, V.; Makarov, V.A. Spiking neurons as universal building blocks for hybrid systems. Adv. Sci. Lett. 2016, 22, 2633–2637. [Google Scholar] [CrossRef]
- Lobov, S.; Mironov, V.; Kastalskiy, I.; Kazantsev, V. A spiking neural network in semg feature extraction. Sensors 2015, 15, 27894–27904. [Google Scholar] [CrossRef] [Green Version]
- Gupta, I.; Serb, A.; Khiat, A.; Zeitler, R.; Vassanelli, S.; Prodromakis, T. Real-time encoding and compression of neuronal spikes by metal-oxide memristors. Nat. Commun. 2016, 7, 12805. [Google Scholar] [CrossRef] [PubMed]
- Gater, D.; Iqbal, A.; Davey, J.; Gale, E. Connecting Spiking Neurons to a Spiking Memristor Network Changes the Memristor Dynamics. In Proceedings of the 2013 IEEE 20th International Conference on Electronics, Circuits, and Systems, Abu Dhabi, UAE, 8–11 December 2013. [Google Scholar]
- Hebb, D.O. The Organization of Behavior: A Neuropsychological Theory; Wiley: Hoboken, NJ, USA, 1949. [Google Scholar]
- Brown, T.H.; Kairiss, E.W.; Keenan, C.L. Hebbian synapses: Biophysical mechanisms and algorithms. Annu. Rev. Neurosci. 1990, 13, 475–511. [Google Scholar] [CrossRef]
- Kohonen, T. An introduction to neural computing. Neural Netw. 1988, 1, 3–16. [Google Scholar] [CrossRef]
- Kohonen, T. Self-organized formation of topologically correct feature maps. Biol. Cybern. 1982, 43, 59–69. [Google Scholar] [CrossRef]
- Sjöström, P.J.; Turrigiano, G.G.; Nelson, S.B. Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 2001, 32, 1149–1164. [Google Scholar] [CrossRef] [Green Version]
- Bi, G.; Poo, M. Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 1998, 18, 10464–10472. [Google Scholar] [CrossRef]
- Markram, H.; Lübke, J.; Frotscher, M.; Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science 1997, 275, 213–215. [Google Scholar] [CrossRef] [Green Version]
- Morrison, A.; Diesmann, M.; Gerstner, W. Phenomenological models of synaptic plasticity based on spike timing. Biol. Cybern. 2008, 98, 459–478. [Google Scholar] [CrossRef] [Green Version]
- Song, S.; Miller, K.D.; Abbott, L.F. Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 2000, 3, 919. [Google Scholar] [CrossRef]
- Izhikevich, E.M. Simple model of spiking neurons. IEEE Trans. Neural Netw. 2003, 14, 1569–1572. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Rubin, J.; Lee, D.D.; Sompolinsky, H. Equilibrium properties of temporally asymmetric hebbian plasticity. Phys. Rev. Lett. 2001, 86, 364–367. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Pfister, J.-P.; Gerstner, W. Triplets of spikes in a model of spike timing—dependent plasticity. J. Neurosci. 2006, 26, 9673–9682. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Masquelier, T.; Guyonneau, R.; Thorpe, S.J. Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS ONE 2008, 3, e1377. [Google Scholar] [CrossRef]
- Masquelier, T.; Thorpe, S.J. Unsupervised learning of visual features through spike timing dependent plasticity. PLOS Comput. Biol. 2007, 3, e31. [Google Scholar] [CrossRef]
- Masquelier, T.; Guyonneau, R.; Thorpe, S.J. Competitive stdp-based spike pattern learning. Neural Comput. 2008, 21, 1259–1276. [Google Scholar] [CrossRef]
- Lobov, S.; Simonov, A.; Kastalskiy, I.; Kazantsev, V. Network response synchronization enhanced by synaptic plasticity. Eur. Phys. J. Spec. Top. 2016, 225, 29–39. [Google Scholar] [CrossRef]
- Lobov, S.A.; Zhuravlev, M.O.; Makarov, V.A.; Kazantsev, V.B. Noise enhanced signaling in stdp driven spiking-neuron network. Math. Model. Nat. Phenom. 2017, 12, 109–124. [Google Scholar] [CrossRef]
- Shamsin, M.; Krilova, N.; Bazhanova, M.; Kazantsev, V.; Makarov, V.A.; Lobov, S. Supervised and unsupervised learning in processing myographic patterns. J. Phys. Conf. Ser. 2018, 1117, 12008. [Google Scholar] [CrossRef]
- Lobov, S.A.; Mironov, V.I.; Kastalskiy, I.A.; Kazantsev, V.B. Combined use of command-proportional control of external robotic devices based on electromyography signals. Sovrem. Tehnol. Med. 2015, 7, 30–38. [Google Scholar] [CrossRef] [Green Version]
- Lebedev, A.E.; Solovyeva, K.P.; Dunin-Barkowski, W.L. The Large-Scale Symmetry Learning Applying Pavlov Principle BT—Advances in Neural Computation, Machine Learning, and Cognitive Research III; Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 405–411. [Google Scholar]
- Clopath, C.; Büsing, L.; Vasilaki, E.; Gerstner, W. Connectivity reflects coding: A model of voltage-based stdp with homeostasis. Nat. Neurosci. 2010, 13, 344. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Bienenstock, E.L.; Cooper, L.N.; Munro, P.W. Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex. J. Neurosci. 1982, 2, 32–48. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tyukin, I.; Gorban, A.N.; Calvo, C.; Makarova, J.; Makarov, V.A. High-dimensional brain: A tool for encoding and rapid learning of memories by single neurons. Bull. Math. Biol. 2019, 81, 4856–4888. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Gorban, A.N.; Makarov, V.A.; Tyukin, I.Y. The unreasonable effectiveness of small neural ensembles in high-dimensional brain. Phys. Life Rev. 2019, 29, 55–88. [Google Scholar] [CrossRef]
- Lobov, S.A.; Krylova, N.P.; Anisimova, A.P.; Mironov, V.I.; Kazantsev, V.B. Optimizing the speed and accuracy of an emg interface in practical applications. Hum. Phys. 2019, 45, 145–151. [Google Scholar] [CrossRef]
- Bohte, S.M.; Kok, J.N.; Poutré, H. La Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 2002, 48, 17–37. [Google Scholar] [CrossRef] [Green Version]
- Neftci, E.O.; Mostafa, H.; Zenke, F. Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal. Process. Mag. 2019, 36, 51–63. [Google Scholar] [CrossRef]
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lobov, S.A.; Chernyshov, A.V.; Krilova, N.P.; Shamshin, M.O.; Kazantsev, V.B. Competitive Learning in a Spiking Neural Network: Towards an Intelligent Pattern Classifier. Sensors 2020, 20, 500. https://doi.org/10.3390/s20020500
Lobov SA, Chernyshov AV, Krilova NP, Shamshin MO, Kazantsev VB. Competitive Learning in a Spiking Neural Network: Towards an Intelligent Pattern Classifier. Sensors. 2020; 20(2):500. https://doi.org/10.3390/s20020500
Chicago/Turabian StyleLobov, Sergey A., Andrey V. Chernyshov, Nadia P. Krilova, Maxim O. Shamshin, and Victor B. Kazantsev. 2020. "Competitive Learning in a Spiking Neural Network: Towards an Intelligent Pattern Classifier" Sensors 20, no. 2: 500. https://doi.org/10.3390/s20020500
APA StyleLobov, S. A., Chernyshov, A. V., Krilova, N. P., Shamshin, M. O., & Kazantsev, V. B. (2020). Competitive Learning in a Spiking Neural Network: Towards an Intelligent Pattern Classifier. Sensors, 20(2), 500. https://doi.org/10.3390/s20020500