A New Spike Membership Function for the Recognition and Processing of Spatiotemporal Spike Patterns: Syllable-Based Speech Recognition Application

Abstract

This paper introduces a new spike activation function (SPKAF) or spike membership function for fuzzy adaptive neurons (FAN), developed for decoding spatiotemporal information with spikes, optimizing digital signal processing. A solution with the adaptive network-based fuzzy inference system (ANFIS) method is proposed and compared with that of the FAN-SPKAF model, obtaining very precise simulation results. Stability analysis of systems models is presented. An application to voice recognition using solfeggio syllables in Spanish is performed experimentally, comparing the methods of FAN-step activation function (STEPAF)-SPKAF, Augmented Spiking Neuron Model, and Augmented FAN-STEPAF-SPKAF, achieving very good results.

1. Introduction

Neural biological processes of communicating information between cortical neurons are performed by spikes, encoding and decoding data transmission for speech recognition and images. The processing of neural information from real-world stimuli and the classification of spike patterns is one of the main objectives for the area of cell physiology. This positions the recognition of the spike pattern sequences for the encoding and decoding of neural information; in this context, a fuzzy system is proposed for the decoding or recognition of spike patterns that, upon exceeding a threshold, activate a neuronal response.

Biological-inspired models of neurons have been proposed, including the models by McCulloch and Pitts, Hodgkin and Huxley [1,2], Gerstner and Kistler, Izhikevich, and Ramírez-Mendoza [3,4,5]. The theory of fuzzy logic such as that by Zadeh [6] has been very successful for decades, and models of fuzzy neurons, such as that from Gupta [7,8,9], have emerged, some of them with a spike response such as those by Ramírez-Mendoza et al. and Zhang [10,11,12].

Related Works

Some of the main characteristics of the human brain are learning and memory and recognizing voices, images, and objects. Classic and fuzzy neuron models have been configured in neural networks or systems for different applications, based on spiking neurons, some with axonal delays, which are mentioned below.

In [13], the authors present the firing neurons to extract perceptual information from a time series of images with firing patterns of the spiking neurons based on competitive learning, a method for the control of visual perception by predicting an associated robot. Spike patterns generated in a time domain based on temporal rules, spatiotemporal patterns, an advanced computational model, and spiking neurons for single spike encoded pattern recognition for neural information processing and learning applied to digit recognition are featured in [14].

Tapson [15] describes a neural network synthesis method to generate synaptic connectivity and process time-coded neural signals, allowing the user to define axonal and dendritic delays, as well as synaptic transfer functions, to compute dendritic weights. They used leaky integrator, nonlinear leaky integrator, alpha functions, resonant dendrites, and fixed axonal or dendritic delays for voice digit recognition. A multineural spike pattern detection configuration is presented in [16]. The structure is capable of autonomously implementing online learning and the recognition of parallel spike sequences, mapping a spatiotemporal stimulus in a multidimensional, temporal, and feature space. The scheme uses a leaky integrate-and-fire (LIF) with latency neuron model and a self-tuning mechanism with a time-amplitude conversion operated by the spike latency feature and is applied to experimental data obtained from a motor-inhibitory cognitive task as a method of classification.

Spiking Neural Networks (SNN) are designed for the recognition of hand gesture patterns using a surface electromyography signal [17]. In [18], the authors propose a curiosity-based learning method for SNN based on the LIF neuron model and an estimation of samples during the learning process to deal with spatiotemporal information and the time-consuming computation for the data set of hand-written digit recognition. A method with deep convolutional SNN training with input spike events and spike-based backpropagation learning is proposed in [19]. The SNN components are LIF neurons, which generate bipolar spikes [–1,1], for image recognition tasks. Verma [20] developed an algorithm for the future generation of image frames with ANFIS, a neural network based on the Takagi–Sugeno fuzzy inference system, on space–time framework, obtaining successful results.

Temporal spike learning is presented in the auditory, the visual, and the motor control information processing of the brain. It is essentially applied in neuroprosthetics, along with applications for fast, real-time recognition and control of the sequence of related processes. Temporal coding of information in SNN uses the exact time of spikes. Every spike and its timing are important [21]. Learning rules have been applied to train neurons to associate spatiotemporal spike patterns, or spike pattern recognition, for example, optical character recognition, and coding methods to obtain spike patterns from images. Spiking neurons process spikes or action potentials, such as biological neurons for the transmission of information and cognitive abilities such as recognition, in neural systems. Information from external stimuli to the neuron is represented by spikes, called neural coding. There is evidence that neural systems transmit information by the timing of the spikes, producing a temporal code. A complex external stimulus can fire different sensory neurons to generate a spike at different times, creating a spatiotemporal representation of the stimulus. To process spatiotemporal spike patterns, learning algorithms have been developed: the tempotron rule, the perceptron rule, and the precise-spike-driven rule [22].

SNN processes the input information and produces fast and accurate recognition of a particular pattern. Two schemes can be distinguished, the rate coding and the temporal coding. Rate coding or the rate of firing communication explains that the intensity of stimuli increases the rate of actions. Temporal encoding in spike training carries pattern information faster than rate encoding. An iris data set is taken to classify patterns using temporal encoding learning effectively and efficiently [23]. A sparse temporal coding method for processing spatial and temporal information is developed in [24]. A spike-timing-dependent back-propagation-based SNN classifier achieves high efficiency of parallel computing for feature extraction. Image classification and speech recognition applications show the SNN model yields state-of-the-art accuracy.

In [25], an Augmented Spiking Neuron Model and synaptic learning rules are proposed, with spike coefficients in addition to spike latencies to process and learn patterns of augmented spikes. The temporal-based approach effectively improves the accuracy of visual and acoustic pattern recognition tasks. Activation functions or membership functions with different forms, such as those by Sigmoid, Gaussian, Parabola, and Sigmoid–Gaussian, among others, have been developed in Zhao and Bose, Basterretxea et al., Xie et al., and Ramirez-Mendoza et al. [26,27,28,29,30,31,32].

A new spike activation function (SPKAF) for fuzzy neurons with dendritic inputs at unipolar context [0,1] and bipolar spike response at [–1,1] is proposed here. This article proposes a solution for a fuzzy neural system decoding a sequence of a spatiotemporal spike pattern that arrives at a neural system and leads to the conduction of other neurons. If encoded information corresponds to a spatiotemporal spike pattern determined or referenced, then the threshold of the neuron is exceeded, and a spike is generated. SPKAF is applied to the system to generate spikes and decode the information.

Solution methods based on an adaptive network-based fuzzy inference system (ANFIS) and fuzzy adaptive neuron (FAN)-SPKAF are proposed for pattern recognition problem. The ANFIS method consists of fuzzy inference rules and the FAN learning algorithm [33] to train the parameters or weights. Instead of sums at the last layers of the system, ANFIS uses a fuzzy logical operator, MAX. The FAN-based method includes the fuzzy logical operator MIN.

The main contribution of this work is the new SPKAF to perform pattern recognition and spike-time-encoded information. The FAN-SPKAF solution method comprises a FAN, axonal delays of previous neurons for the input channels of spike pattern sequences to be decoded, fuzzy logical operators, and the SPKAF. Among the advantages of the FAN-SPKAF are the processing of information with digital pulse trains, prior to the generation of analog spikes, and that the information is precisely encoded because the duration and shape of each analog spike generated based on the digital pulse trains is constant.

The problem of voice recognition using solfeggio syllables in Spanish is solved by comparing the methods of FAN-activation function step-type (STEPAF)-SPKAF, the Augmented Spiking Neuron Model, and Augmented FAN-STEPAF-SPKAF. This article is divided into the following sections: Section 2 presents the methods, a description of the FAN model with a Gupta Integrator [7,8,9], and the new SPKAF model for fuzzy neurons from Ramírez-Mendoza et al. [5,10,11,29,32,33,34,35]. Section 3 describes the models and algorithms of fuzzy systems based on ANFIS and FAN-SPKAF for the processing of spike-time-encoded information and pattern recognition. The FAN-STEPAF-SPKAF model is applied to voice recognition based on solfeggio syllables in Spanish and compared with the Augmented Spiking Neuron Model and Augmented FAN-STEPAF-SPKAF methods.

Section 4 develops the model analysis for systems of Section 3.1, Section 3.2 and Section 3.3, with nonlinear systems fuzzy modeling, fuzzy system training, and stability analysis. Section 5 presents the simulation results of the comparison of methods for the fuzzy systems models applied to the recognition of spatiotemporal spike patterns and voice recognition using solfeggio syllables in Spanish. Section 6 describes the conclusions and discussion.

2. Methods

2.1. Fuzzy Adaptive Neurons

The sigmoid activation function (SAF) was proposed for FAN, for unipolar FAN, and another for bipolar in [5,10,11,33]. This article presents a new SPKAF also for fuzzy neurons. The unipolar FAN model with SAF is briefly presented below.

For unipolar inputs at interval [0,1], with the time variable $k$, the synaptic operation of inputs $z_{in j}\left(k\right)$ and adaptive weights $w_{in j}\left(k\right)$ is presented in (1), the aggregation operation in (2), and the nonlinear operation with threshold in (3) and (4). The somatic operations of the FAN and the error $e\left(k\right)$ are described in (1)–(5):

$${\tilde{V}}_{min j}\left(k\right)=\mathit{min}\left(z_{in j}\left(k\right),w_{in j}\left(k\right)\right)$$

(1)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N{\tilde{V}}_{min j}\left(k\right)$$

(2)

$${\tilde{V}}_{out}\left(k\right)=\mathit{max}\left({\tilde{V}}_{max}\left(k\right),V_{threshold}\left(k\right)\right)$$

(3)

$${\tilde{y}}_{SAF}\left(k\right)=\frac{1}{1+e^{\left(-min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right)\cdot a+b\right)}}$$

(4)

$$e\left(k\right)={\tilde{y}}_{SAF ref}\left(k\right)-{\tilde{y}}_{SAF}\left(k\right)$$

(5)

2.2. Spike Activation Function Model

Different membership functions and approximations of activation functions have been proposed for various applications, such as seismic modeling of accelerograms and control, among others [26,27,28,29,30,31,32]. Following is the development of a spike membership function for unipolar inputs $z_{in j}$, shown in Figure 1.

The spike membership function is as follows:

$${\tilde{V}}_{out \gamma }\left(k\right)=min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right)$$

(6)

$${\tilde{y}}_{SPKAF}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}$$

(7)

3. Models for the Processing of Spike-Time-Encoded Information and Pattern Recognition

Models and theoretical analysis of fuzzy neural systems based on ANFIS and FAN-SPKAF methods for pattern recognition of spatiotemporal spike sequences are developed below. The spike pattern input reference and the input spike sequences are generated with FAN-SPKAF.

Memory is implemented by identifying axonal delays of the reference input spike sequences coming from other neurons, such that the maximum delay is that of the input spike with the maximum axonal delay and can be compared at the same time with the other input spikes of the other input channels.

One spike from each channel is compared or presented at the system at a time. This process is performed as many times as the highest number of subsequent spikes in the same channel, and only if it matches the reference input spike sequence, then fuzzy systems based on ANFIS, or the FAN-SPKAF and MIN operator will generate a spike.

3.1. ANFIS Model

The fuzzy system consists of a stage with axonal delays for the reference spike sequences from the previous neurons and an ANFIS with SAF unipolar membership functions. A MAX operator is used instead of sums, and a gain is added (see Figure 2).

3.1.1. Algorithm

If we have the reference spike pattern sequences of the input channels, the maximum spike delay ${\Delta k}_{ref input spike max}$ of those spike sequences is determined. The fuzzy neural system can generate an output or spike until after that maximum delay time ${\Delta k}_{ref input spike max}$, $y_{ANFIS spike}\left(k<{\Delta k}_{ref input spike max}\right)=0$.

All spikes of the input sequences to the neural system will be processed at that instant ${k+\Delta k}_{ref input max}$; therefore, an axonal delay will be given to each of the spikes of the input sequences of the input channels, ${\Delta k}_{i j axon delay}={\Delta k}_{ref input max}-{\Delta k}_{i j input spike}$, where $i=1,2,\cdots ,m$; $m$ is the number of channels and $j=1,2,\cdots ,n$; $n$ is the number of spikes at the reference input sequences of each channel.

3.1.2. Adaptive Network Based Fuzzy Inference System (ANFIS)

ANFIS is based on the Takagi–Sugeno fuzzy inference system [21]. In this inference system, fuzzy IF–THEN rules use a learning algorithm. There are five input channels with spike sequences, $x_1,x_2,x_3,x_4,x_5$, in the fuzzy inference system that describes a Sugeno fuzzy model with the following inference rules: $\forall k\ge {\Delta k}_{ref input max}$,

$$\begin{gathered} Rule 1: If{x}_1\left(k-{\Delta k}_{1 j axon delay}\right) \mathrm{i}\mathrm{s} A_1 then y_1\left(k\right)={MAX(p}_1\cdot x_1\left(k-{\Delta k}_{1 j axon delay}\right),q_1) \\ Rule 2: If x_2\left(k-{\Delta k}_{2 j axon delay}\right) \mathrm{i}\mathrm{s} A_2 then y_2\left(k\right)={MAX(p}_2\cdot x_2\left(k-{\Delta k}_{2 j axon delay}\right),q_2) \\ Rule 3: If x_2\left(k-{\Delta k}_{2 j axon delay}\right) \mathrm{i}\mathrm{s} A_3 and x_3\left(k-{\Delta k}_{3 j axon delay}\right) \mathrm{i}\mathrm{s} A_4 and x_4\left(k-{\Delta k}_{4 j axon delay}\right) \mathrm{i}\mathrm{s} A_5 then y_3\left(k\right) \\ = MAX(p_3\cdot x_2\left(k-{\Delta k}_{2 j axon delay}\right),{q_3\cdot x_3\left(k-{\Delta k}_{3 j axon delay}\right),r_3\cdot x_4\left(k-{\Delta k}_{4 j axon delay}\right),s}_3) \\ Rule 4: If x_3\left(k-{\Delta k}_{3 j axon delay}\right) \mathrm{i}\mathrm{s} A_6 and x_5\left(k-{\Delta k}_{5 j axon delay}\right) \mathrm{i}\mathrm{s} A_7 then y_4\left(k\right) \\ = {MAX(p}_4\cdot x_3\left(k-{\Delta \mathrm{k}}_{3 j axon delay}\right),q_4\cdot x_5\left(k-{\Delta \mathrm{k}}_{5 j axon delay}\right),r_4) \\ Rule 5: If x_3\left(k-{\Delta k}_{3 j axon delay}\right) \mathrm{i}\mathrm{s} A_8 and x_5\left(k-{\Delta k}_{5 j axon delay}\right) \mathrm{i}\mathrm{s} A_9 then y_5\left(k\right) \\ = {MAX(p}_5\cdot x_3\left(k-{\Delta k}_{3 j axon delay}\right),q_5\cdot x_5\left(k-{\Delta k}_{5 j axon delay}\right),r_5) \end{gathered}$$

Layer 1: Every node could be adaptive with a node membership function, from (4):

$$O_l^1= {\mu }_{A_l}\left(x_i\right)$$

(8)

$${\mu }_{A_l}\left(x_i\right)={\mu }_{SAF{A}_l}\left(x_i\right)$$

(9)

$${\mu }_{SAF{A}_l}\left(x_i\right)=\frac{1}{1+e^{\left(-min\left(\gamma ,\max \left({MAX}_{j=1}^N\min \left(x_i\left(k-{\Delta k}_{i j axon delay}\right),w_i\left(k\right)\right),V_{threshold}\left(k\right)\right)\right)\cdot a+b\right)}}$$

(10)

where $w_i\left(k\right)=1;l=1,2,\cdots ,9$; $i=1,2,\cdots ,5$.

Layer 2: Each node is fixed, and outputs are the product of all incoming inputs:

$$O_i^2={\omega }_i={\mu }_{A_l}\left(x_i\right)\cdot {\mu }_{A_{l+m}}\left(x_i\right)\cdot \dots \cdot {\mu }_{A_{l+m}}\left(x_i\right)$$

(11)

where $m=1,2,\cdots ,8;l=1,2,\cdots ,9$; $i=1,2,\cdots ,5$.

Layer 3: Each node is fixed, and outputs are the ratio:

$$O_i^3={\stackrel{-}{\omega }}_i=\frac{{\omega }_i}{{\omega }_1+{\omega }_2+\cdots +{\omega }_5}$$

(12)

where $i=1,2,\cdots ,5$.

Layer 4: Each node is adaptive with a node function:

$$O_i^4={\stackrel{-}{\omega }}_i\cdot y_i\left(k\right)={\stackrel{-}{\omega }}_i\cdot MAX(p_i\cdot x_i,{q_i\cdot x_i,r_i\cdot x_i,s}_i)$$

(13)

where $i=1,2,\cdots ,5$.

Layer 5: Each node is adaptive with a node function,

$$O_{spike}^5=overall output=\sum {\stackrel{-}{\omega }}_i\cdot y_i\left(k\right)=\frac{\sum {\omega }_i\cdot y_i\left(k\right)}{\sum {\omega }_i}$$

(14)

$$y_{spike ANFIS}\left(k\right)=O_{spike}^5$$

(15)

Adjusting the amplitude with a gain,

$$y_{spike ANFIS}\left(k\right)=O_{spike}^5\cdot g_1$$

(16)

where $i=1,2,\cdots ,5$; ${\stackrel{-}{\omega }}_i$ is a normalized firing strength and $p_i,{q_i,r_i,s}_i$ are adaptive parameters.

3.2. FAN-SPKAF Model

The model of a fuzzy system is presented with the new SPKAF developed in Section 2 for the recognition of spike-time-encoded sequence patterns. If the reference input spike pattern exceeds the SPKAF threshold, then the FAN output is activated, generating a spike.

The fuzzy system consists of a stage with axonal delays for the reference spike sequences from the previous neurons, a fuzzy logic operator MIN, a comparator, and a FAN-SPKAF with threshold; a gain and an adjustment constant are also added. If the dendrite input exceeds the threshold, the FAN-SPKAF activates with the input spike pattern, thereby generating an output spike, the FAN-SPKAF response, as shown in Figure 3.

After a spike is obtained from the fuzzy operator MIN, it is fuzzified or scaled from the bipolar interval [–1,1] to the unipolar interval [0,1], with a comparator taking the positive part of the spike. A unipolar ramp is generated, and then it is limited to the unipolar interval by means of a trimmer, thereby obtaining the input to the dendritic synapse of the fuzzy neuron. If the potential of the soma exceeds the threshold, the SPKAF of the FAN generates a spike, thus performing the recognition of spatiotemporal spike patterns.

3.2.1. Algorithm

If we have the reference spike pattern sequences of the input channels, the maximum spike delay ${\Delta k}_{ref input spike max}$ of those spike sequences is determined. The fuzzy neural system can generate an output or spike until after the maximum delay time ${\Delta k}_{ref input spike max}$, $y_{ANFIS spike}\left(k<{\Delta k}_{ref input spike max}\right)=0$.

All spikes of the input sequences to the neural system will be processed at the instant ${k+\Delta k}_{ref input max}$; therefore, an axonal delay will be given to each of the spikes of the input sequences of the input channels, ${\Delta k}_{i j axon delay}={\Delta k}_{ref input max}-{\Delta k}_{i j input spike}$, where $i = 1,2,\cdots ,m$; $m$ is the number of channels and $j = 1,2,\cdots ,n$; $n$ is the number of spikes at the reference input sequences of each channel.

3.2.2. Fuzzy Adaptive Neuron with Spike Activation Function (FAN-SPKAF)

The FAN uses a nonlinear somatic threshold operation, an activation function, or a membership function. In Section 3, the development of the spike shaped SPKAF was presented. Once axonal delays are given to the input spike sequences, a MIN operator is applied. If all the spikes of the input spike pattern sequences are present, then the MIN operator responds with one spike. Subsequently, a comparator with zero is used to obtain the positive part of the spike or zero. This result is the input to the FAN-SPKAF with fixed weights. Every time the FAN-SPKAF receives the positive part of a spike, it emits a bipolar spike.

$$z\left(k\right)={MIN}_{i=1}^N{x}_i\left(k-{\Delta k}_{i axon delay}\right)$$

(17)

where $i=1,2,\cdots ,5$.

If $z\left(k\right)>0 then$

$${\tilde{V}}_{min}\left(k\right)=\min \left(z\left(k\right),w\left(k\right)\right)$$

(18)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N{\tilde{V}}_{minj}\left(k\right)$$

(19)

$${\tilde{V}}_{out}\left(k\right)=\max \left({\tilde{V}}_{max}\left(k\right),V_{threshold}\left(k\right)\right)$$

(20)

$${\tilde{V}}_{out\gamma }\left(k\right)=min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right)$$

(21)

$${\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}$$

(22)

Adjusting the amplitude with a gain,

$${\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}\cdot g_2$$

(23)

else${\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=0$.

3.3. FAN-STEPAF-SPKAF Model for Syllable-Based Speech Recognition Application

For speech pattern recognition, a differentiation can be made between the recognition of individual speakers and the recognition of the vowels, consonants, and syllables of one or more speakers. Here, we present the SPKAF applied to the recognition of voice patterns to identify a syllable or musical note, using the first musical notes of a classical melody.

DO, RE, MI, FA, SOL, LA, and SI define the musical notes in Spanish; in English, they are C, D, E, F, G, A, and B. A voice sample was taken from a Spanish-speaker who read the first musical notes of Friederich von Schiller/Ludwig van Beethoven’s “Ode to Joy”.

The voice has different characteristics, articulation, tone, timbre, and volume. Algorithms of the speaker-dependent and speaker-independent type based on fuzzy logic have been presented for the recognition of speech patterns [36]. The main problems are (1) fluctuation of pattern length (intonation) caused by variance of utterance speed, and (2) frequency caused by differences of the vocal tract (timbre). Articulation is made by changing vocal tract, and then the information required is added at the sound wave. In this article, the speaker-dependent type is presented, the pattern recognition method samples the voice of the operator syllable by syllable. An SPKAF application developed in this article is shown.

Other methods have been applied to pattern recognition, for example, the approximate reasoning method for pattern recognition, which consists of fuzzy implications and a composition inference rule [37].

Voice qualities are different for each speaker and use temporal structures, articulation precision, vocal effort, and type of phonation. Vocal effort has to do with sound quality. For phonation, it is necessary to separate the quality of the voice from the linguistic content; the study is based on the phonemes [38].

In [39], the authors study the types of syllables in a language, vowel–consonant, consonant–vowel, long vowel–vowel, consonant–consonant, and the frequency of the syllable patterns.

Speech pattern recognition [40] has been applied in speech conversion methods for people with articulation disorders, based on spectral conversion using non-negative matrix factorization with very good results.

Different approaches to membership functions have been proposed in recent years. A new membership function for uncertain functions is developed in [41], based on cuts determined by an integral membership function, and defined by a continuous auxiliary function design, depending on the uncertainty of the function. In [29], a new approach of the sigmoid membership function proposed for FAN and fuzzy systems is developed using the Newton binomial, with exponential series to obtain its Laplace transform, with application to control systems.

The syllable-based pattern recognition information, for the application proposed here, is contained in the form of magnitudes or articulation of the voice, rather than frequency or timbre. The short-time Fourier transform (STFT) is obtained with a Matlab^® command. The maximum operator is applied to the magnitudes, a FAN-STEPAF [10,11,29,33], then a FAN-SPKAF, and a comparator with the syllable pattern of spikes to obtain the recognized spike speech pattern, as shown in Figure 4.

From a speech sample $x\left(k\right)$, the STFT is obtained with magnitude and phase:

$$STFT\left\{x\left(k\right)\right\}=X\left[n,k\right]={\sum }_{m=k}^{k+N-1}x\left[m\right]\cdot w[m-k]\cdot e^{-i\cdot 2\cdot \pi \cdot n\cdot (m-k)/N}$$

(24)

where $n=0,1,\cdots ,N-1$ is the frequency index.

From $X\left[n,k\right]$, the absolute value of the magnitude is obtained:

$$X_{mag}\left[n,k\right]=\left|X\left[n,k\right]\right|$$

(25)

Then the MAX operator is applied to each segment:

$$X_{MAX mag}\left[k\right]={MAX(X}_{mag}\left[n,k\right])$$

(26)

Via membership or STEPAF of the FAN [10,11,29,33], square pulses at [0,1] are obtained:

$$z\left(k\right)=X_{MAX mag}\left[k\right]$$

(27)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N\mathrm{m}\mathrm{i}\mathrm{n}(z_j\left(k\right),w_j\left(k\right))$$

(28)

$${\tilde{y}}_{step}\left(k\right)=\left\{\begin{array}{c}1 if {\tilde{V}}_{max}\left(k\right)\ge V_{threshold}\left(k\right)\\ 0 if {\tilde{V}}_{max}\left(k\right)<V_{threshold}\left(k\right)\end{array}\right.$$

(29)

where $\mathrm{N}=1,w\left(k\right)=1,V_{threshold}\left(k\right)$ is the triangular shaped threshold input.

From (1)–(3), (6) and (7), we can obtain,

If ${\tilde{y}}_{step}\left(k\right)>0 THEN$

$${z_{in}\left(k\right)=\tilde{y}}_{step}\left(k\right)$$

(30)

$${\tilde{V}}_{min j}\left(k\right)=\min \left(z_{in j}\left(k\right),w_{in j}\left(k\right)\right)$$

(31)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N{\tilde{V}}_{min j}\left(k\right),N=1$$

(32)

$${\tilde{V}}_{out}\left(k\right)=\max \left({\tilde{V}}_{max}\left(k\right),V_{threshold}\left(k\right)\right),V_{threshold}\left(k\right)=0$$

(33)

$${\tilde{V}}_{out \gamma }\left(k\right)= min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right),\gamma =1$$

(34)

$${\tilde{y}}_{SPKAF}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}$$

(35)

ELSE ${\tilde{y}}_{SPKAF}\left(k\right)=0$

Note: all spikes have the same shape and period.

If  ${\tilde{y}}_{SPKAF}\left(k\right)={\tilde{y}}_{SPKAF SI}\left(k\right) THEN$

$${\tilde{y}}_{SPKAF IDENTIFIED}\left(k\right)={\tilde{y}}_{SPKAF}\left(k\right)$$

(36)

ELSE ${\tilde{y}}_{SPKAF IDENTIFIED}\left(k\right)=0$ where ${\tilde{y}}_{SPKAF SI}\left(k\right)$ represents the spikes in the voice pattern of the syllable or musical note SI of solfeggio in Spanish.

To compare the FAN-STEPAF-SPKAF model, the methods of Section 3.4 and Section 3.5 are presented.

3.4. Augmented Spiking Neuron Model for Syllable-Based Speech Recognition Application

Syllable-based pattern recognition information is obtained based on the maximum of the STFT magnitudes and the Augmented Spiking Neuron Model method [25].

The reference spike pattern of the SI syllable ${\tilde{y}}_{ASNM SI}$ is compared with the first spikes of the melody ${\tilde{y}}_{ASNM}$. The input samples of the same interval of the SI syllable to be identified are stored in memory, and these are compared with the reference pattern. The mean squared error (MSE) is determined, and if the MSE is greater than a user-defined MSE, then the syllable was identified and it continues with the next sample; if not, then the pattern recognition is null, and the same sequence is carried out, advancing one more sample until the pattern recognition of the whole melody ${\tilde{y}}_{ASNM IDENTIFIED}$ is concluded, similar to Figure 4.

From (26) and (29) and [25], we can determine the following:

$${\tilde{y}}_{in}\left(k\right)={\tilde{y}}_{step}\left(k\right)\cdot X_{MAX mag}\left[k\right]$$

(37)

$$V_{Kernel}\left(k-k_i^j\right)=c_0\cdot \left[exp\left(-\frac{k-k_i^j}{{\kappa }_m}\right)-exp\left(-\frac{k-k_i^j}{{\kappa }_s}\right)\right]$$

(38)

$$\begin{gathered} V_{ASNM}\left(k\right)={\sum }_{i=1}^N{w}_i\cdot {\sum }_{k_i^j<k}{c}_i^j{\cdot V}_{Kernel}\left(k-k_i^j\right)-\vartheta \cdot \\ {\sum }_{k_s^j<k}exp\left(-\frac{k-k_s^j}{{\kappa }_m}\right) \end{gathered}$$

(39)

$$∆w_i=\left\{\begin{array}{c}\eta \cdot {\displaystyle \sum _{k_i^j<k_{max}}}{c}_i^j{\cdot V}_{Kernel}\left(k-k_i^j\right), if P^{+}error>0\\ -\eta \cdot {\displaystyle \sum _{k_i^j<k_{max}}}{c}_i^j{\cdot V}_{Kernel}\left(k-k_i^j\right), if P^{-}error<0\\ 0, otherwise\end{array}\right.$$

(40)

Substituting (37) in (38) and (39), considering $w_i=1$ and $\vartheta =0$ in (41) and (42),

$$V_{Kernel}\left({\tilde{y}}_{in}\left(k\right)\right)=q_0\cdot \left[exp\left(-\frac{{\tilde{y}}_{in}\left(k\right)}{{\kappa }_m}\right)-exp\left(-\frac{{\tilde{y}}_{in}\left(k\right)}{{\kappa }_s}\right)\right]$$

(41)

$${\tilde{y}}_{ASNM}\left(k\right)=c_i{\cdot V}_{Kernel}\left({\tilde{y}}_{in}\left(k\right)\right)$$

(42)

$$i=1$$

(43)

while $i\le Q$

$$MSE=\frac{1}{M}\cdot {\sum }_{i=1}^M{\left(\{{\tilde{y}}_{ASNM SI}\left(k_1\right)\cdots {\tilde{y}}_{ASNM SI}\left(k_M\right)\}-\left\{{\tilde{y}}_{ASNM}\left(k_i\right)\cdots {\tilde{y}}_{ASNM}\left(k_{i+M-1}\right)\right\}\right)}^2$$

(44)

IF $MSE>rTHEN$

$$\{{\tilde{y}}_{ASNM IDENTIFIED}\left(k_i\right)\cdots {\tilde{y}}_{ASNM IDENTIFIED}\left(k_{i+M-1}\right)\}=\{{\tilde{y}}_{ASNM}\left(k_i\right)\cdots {\tilde{y}}_{ASNM}\left(k_{i+M-1}\right)\}$$

(45)

$$i=i+M-1$$

(46)

ELSE

$${\tilde{y}}_{ASNM IDENTIFIED}\left(k_i\right)=0$$

(47)

$$i=i+1$$

(48)

End

End

3.5. Augmented FAN-STEPAF-SPKAF Model for Syllable-Based Speech Recognition Application

Spike patterns recognition of the SI syllable in a melody is performed by obtaining the maximum of the STFT magnitudes and the Augmented FAN-STEPAF-SPKAF method [29,30,31,32,33,34,35].

Based on the reference spike pattern ${\tilde{y}}_{SPKAF SI}$ of the SI syllable, it is compared with the first spikes of the melody ${\tilde{y}}_{SPKAF}$. The input samples of the same interval of the SI syllable to be identified are stored in memory and compared with the reference pattern. If the MSE is less than a user-defined MSE, then the syllable was identified and it continues with the next sample; if not, then the pattern recognition is null, and the same sequence is carried out, advancing one more sample until the pattern recognition of the whole melody ${\tilde{y}}_{ASPKAF IDENTIFIED}$ is concluded, similar to Figure 4.

From (26) and (29), we can determine the following:

$${\tilde{y}}_{sawtooth}\left(k\right)={\tilde{y}}_{step}\left(k\right)\cdot X_{MAX mag}\left[k\right]\cdot y_{triangular}\left(k\right)$$

(49)

$${z_{in}\left(k\right)=\tilde{y}}_{sawtooth}\left(k\right)$$

(50)

$${\tilde{V}}_{min j}\left(k\right)=\min \left(z_{in j}\left(k\right),w_{in j}\left(k\right)\right)$$

(51)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N{\tilde{V}}_{min j}\left(k\right),N=1$$

(52)

$${\tilde{V}}_{out}\left(k\right)=\max \left({\tilde{V}}_{max}\left(k\right),V_{threshold}\left(k\right)\right),V_{threshold}\left(k\right)=0$$

(53)

$${\tilde{V}}_{out \gamma }\left(k\right)=min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right),\gamma =1$$

(54)

$${\tilde{y}}_{ASPKAF}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}+c_3$$

(55)

$$i=1$$

(56)

while $i\le Q$

$$MSE=\frac{1}{M}\cdot {\sum }_{i=1}^M{\left({\tilde{y}}_{ASPKAF SI}\left(k_1\right)\cdots {\tilde{y}}_{ASPKAF SI}\left(k_M\right)\}-\left\{{\tilde{y}}_{ASPKAF}\left(k_i\right)\cdots {\tilde{y}}_{ASPKAF}\left(k_{i+M-1}\right)\right\}\right)}^2$$

(57)

IF$MSE>r THEN$

$${\{\tilde{y}}_{ASPKAF IDENTIFIED}\left(k_i\right)\cdots {\tilde{y}}_{ASPKAF IDENTIFIED}\left(k_{i+M-1}\right)\}=\{{\tilde{y}}_{ASPKAF}\left(k_i\right)\cdots {\tilde{y}}_{ASPKAF}\left(k_{i+M-1}\right)\}$$

(58)

$$i=i+M-1$$

(59)

ELSE

$${\tilde{y}}_{ASPKAF IDENTIFIED}\left(k_i\right)=0$$

(60)

$$i=i+1$$

(61)

End

End

4. Model Analysis

4.1. Nonlinear Systems Fuzzy Modeling

According to the study presented in [31] for the stability analysis of adaptive neurons, it is considered that a nonlinear system in discrete time can be represented as

$$y\left(k\right)=\mathsf{\Phi }\left[u\left(k-1\right),u\left(k-2\right),\dots ,u\left(k-n\right)\right]$$

(62)

where $\mathsf{\Phi }\left(·\right)$ is a nonlinear function, the system dynamics $u\left(k\right)$ and $y\left(k\right)$ are the input and output of the system, and $n$ is a number. Multivariable NARMA (nonlinear autoregressive-moving average) form,

$$Y\left(k\right)=\mathsf{\Phi }\left[\mathrm{H}\left(k\right)\right]$$

(63)

where $H\left(k\right)={\left[U\left(k-d_t\right),\dots \right]}^T$

$$U\left(k\right)={\left[u\left(k\right)\right]}^T$$

(64)

$$Y\left(k\right)={\left[y\left(k\right)\right]}^T$$

(65)

where $d_t$ is time delay. To analyze the system models presented in Section 3 and avoid some problems, such as slow convergence and difficulty designing parameters, we consider, from (7) and (64), and Figure 1,

$$u\left(k\right)={\tilde{y}}_{SPKAF}\left(k\right)$$

(66)

$$x\left(k\right)=u\left(k\right)$$

(67)

The subsections below present the analysis of fuzzy systems from Section 3.1.1, Section 3.2.1, and Section 3.3.

4.1.1. Analysis of ANFIS Model

If we normalize the input $U\left(k\right),U\left(k-d_t\right)$ and reference output $y_{spike ref}\left(k\right)$ of the unknown system (3.1.1) into [0, 1], then,

$$R^i: \mathrm{IF} h_1<0 \mathrm{or} h_2<0 \mathrm{or} \cdots h_n<0$$

then ${\widehat{h}}_1=max{(h}_1,r_e),{\widehat{h}}_2=max{(h}_2,r_e),\cdots ,{\widehat{h}}_n=max{(h}_n,r_e)$, where $r_e\in \mathbb{R}$.

We use the somatic Gupta-type aggregation (MAX operator) and the following fuzzy IF–THEN rules for the unknown system (3.1.1.1):

$$\begin{gathered} {Rule}^i: If{x}_j\left(\mathrm{k}-{\Delta \mathrm{k}}_{j l axon delay}\right) \mathrm{i}\mathrm{s} A_m and x_{j+1}\left(\mathrm{k}-{\Delta \mathrm{k}}_{j+1 l axon delay}\right) \mathrm{i}\mathrm{s} A_{m+1}\cdots and x_{j+n}\left(\mathrm{k} \\ -{\Delta \mathrm{k}}_{j+n l axon delay}\right) \mathrm{i}\mathrm{s} A_{m+n} \\ then y_i\left(k\right)= MAX(p_i\cdot x_j\left(\mathrm{k}-{\Delta \mathrm{k}}_{j l axon delay}\right),{q_i\cdot x_{j+1}\left(\mathrm{k}-{\Delta \mathrm{k}}_{j+1 l axon delay}\right),\cdots ,r_i\cdot x_{j+n}\left(\mathrm{k}-{\Delta \mathrm{k}}_{j+n l axon delay}\right),s}_i) \end{gathered}$$

(68)

$${\mu }_{A_m}\left(x_j\right)={\mu }_{SAF{A}_m}\left(x_j\right)$$

(69)

$$O_j^1={\mu }_{A_m}\left(x_j\right)$$

(70)

$$O_j^2={\omega }_j={\mu }_{A_m}\left(x_j\right)\cdot {\mu }_{A_{m+1}}\left(x_j\right)\cdot \dots \cdot {\mu }_{A_{m+n}}\left(x_j\right)$$

(71)

where $j=1,2,\cdots ,5;m=1,2,\cdots ,9;n=1,2,\cdots ,N$.

$$O_j^3={\stackrel{-}{\omega }}_j=\frac{{\omega }_j}{{\omega }_1+{\omega }_2+\cdots +{\omega }_5}$$

(72)

$$O_j^4={\stackrel{-}{\omega }}_j\cdot y_j\left(k\right)={\stackrel{-}{\omega }}_j\cdot MAX(p_j\cdot x_j,{q_j\cdot x_j,r_j\cdot x_j,s}_j)$$

(73)

Because of the fuzzy unipolar system $ϵ$ [0,1], the following synaptic operation, aggregation operation, and nonlinear operation are also the somatic operations:

$${\tilde{V}}_{min i}\left(\mathrm{k}\right)=\min \left(x_j\left(\mathrm{k}-{\Delta \mathrm{k}}_{j l axon delay}\right),w_j\left(k\right)\right)$$

(74)

$${\tilde{V}}_{\max j}\left(\mathrm{k}\right)={MAX}_{i=1}^N{\tilde{V}}_{min i}\left(\mathrm{k}\right)$$

(75)

$${\tilde{V}}_{out j}\left(\mathrm{k}\right)=\max \left({\tilde{V}}_{max j}\left(\mathrm{k}\right),V_{threshold j}\left(k\right)\right)$$

(76)

$${\tilde{y}}_{SAFj}\left(k\right)=\frac{1}{1+e^{\left(-min\left(\gamma ,{\tilde{V}}_{out j}\left(k\right)\right)\cdot a+b\right)}}$$

(77)

$$\begin{gathered} O_{spike}^5 = overall output=\sum {\stackrel{-}{\omega }}_j\cdot {\tilde{y}}_j\left(k\right) \\ =\sum {\stackrel{-}{\omega }}_j\cdot {MAX(p}_j\cdot x_j,{q_j\cdot x_j,r_j\cdot x_j,s}_j)=\frac{\sum {\omega }_j\cdot {\tilde{y}}_j\left(k\right)}{\sum {\omega }_j} \end{gathered}$$

(78)

$${\tilde{y}}_{spike ANFIS}\left(k\right)=O_{spike}^5$$

(79)

$${\tilde{y}}_{spike ANFIS}\left(k\right)=O_{spike}^5\cdot g_1$$

(80)

$$e_{spike ANFIS}\left(k\right)=y_{spike ANFIS ref}\left(k\right)-{\tilde{y}}_{spike ANFIS}\left(k\right)$$

(81)

where $j=1,2,\cdots ,5$.

Based on the above operations, the unknown nonlinear system can be expressed by

$${\tilde{Y}}_{SAF 1 j}\left(k\right)=\mathsf{\Phi }\left[\widehat{H}\left(k\right),W_1\left(k\right)\right]$$

(82)

$${\tilde{Y}}_{SAF 2 j}\left(k\right)=\mathsf{\Phi }\left[\widehat{H}\left(k\right),W_2\left(k\right)\right]$$

(83)

$${\widehat{Y}}_{ANFIS}\left(k\right)=\mathsf{\Phi }\left[\widehat{H}\left(k\right),W_1\left(k\right)\right]·\widehat{H}\left(k\right)·W_2\left(k\right)$$

(84)

where $W_1\left(k\right)=\left[1,\cdots ,1\right]$ is the proposed fixed weights, $W_2\left(k\right)=\left[p_j,{q_j,r_j,s}_j\right]$ is the weights to be obtained, and $\mathsf{\Phi }\left(·\right)$ is the nonlinear function corresponding to the membership functions of the fuzzy system.

4.1.2. Analysis of FAN-SPKAF model

For the nonlinear system of Section 3.2.1, with input $U\left(k\right),U\left(k-d_t\right)$, and reference output or pattern $y_{spike ref}\left(k\right)$,

$$z\left(k\right)={MIN}_{i=1}^N{x}_i\left(k-{\Delta k}_{i axon delay}\right)$$

(85)

where $i=1,2,\cdots ,5$.

$If z\left(k\right)>0 then$

$${\tilde{V}}_{min}\left(k\right)=\min \left(z\left(k\right),w\left(k\right)\right)$$

(86)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N{\tilde{V}}_{minj}\left(k\right)$$

(87)

$${\tilde{V}}_{out}\left(k\right)=\max \left({\tilde{V}}_{max}\left(k\right),V_{threshold}\left(k\right)\right)$$

(88)

$${\tilde{V}}_{out \gamma }\left(k\right)=min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right)$$

(89)

$${\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}$$

(90)

Adjusting the amplitude with a gain,

$${\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out \gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}\cdot g_2$$

(91)

$$else {\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=0$$

$$e_{spike FAN-SPKAF IDENTIFIED}\left(k\right)=y_{spike FAN-SPKAF ref}\left(k\right)-{\tilde{y}}_{spike FAN-SPKAF IDENTIFIED}\left(k\right)$$

(92)

Remark 1.

FAN-SPKAF uses the same learning algorithm for unipolar fuzzy systems,$\in$[0,1], developed for FAN-SAF in [16,31,33]. Because the spikes are bipolar, to normalize them to the unipolar interval, a max operator and a zero reference or a comparator with square output pulses $\in$[0,1] could be used. The spike period and the absolute refractory period [3], are considered constant.

Based on the above operations, the nonlinear system can be expressed by

$${\widehat{Y}}_{FAN-SPKAF}\left(k\right)=\left[\mathsf{\Phi }\left[\widehat{H}\left(k\right),W\left(k\right)\right],0\right]$$

(93)

where $W\left(k\right)=\left[1,\cdots ,1\right]$ is proposed fixed weights and $\mathsf{\Phi }\left(·\right)$ is the nonlinear function corresponding to the membership functions of the fuzzy system.

4.1.3. Analysis of FAN-STEPAF-SPKAF Model

From Section 3.3, the nonlinear system with input ${\tilde{y}}_{SPKAF}\left(k\right)=[U\left(k\right),U\left(k-d_t\right)]$ and reference output or pattern ${{\tilde{y}}_{SPKAF SI}\left(k\right)=y}_{spike ref}\left(k\right)$, where ${\tilde{y}}_{SPKAF SI}\left(k\right)$ is a spike’s voice pattern of the syllable or musical note SI of solfeggio in Spanish.

From a speech sample $x\left(k\right)$, the short-time Fourier transform (STFT) is obtained with magnitude and phase:

$$STFT\left\{x\left(k\right)\right\}=X\left[n,k\right]={\sum }_{m=k}^{k+N-1}x\left[m\right]\cdot w[m-k]\cdot e^{-i\cdot 2\cdot \pi \cdot n\cdot (m-k)/N}$$

(94)

$$X_{mag}\left[n,k\right]=\left|X\left[n,k\right]\right|$$

(95)

$$X_{MAX mag}\left[k\right]={MAX(X}_{mag}\left[n,k\right])$$

(96)

$$z\left(k\right)=X_{MAX mag}\left[k\right]$$

(97)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N\mathrm{m}\mathrm{i}\mathrm{n}(z_j\left(k\right),w_j\left(k\right))$$

(98)

$${\tilde{y}}_{step}\left(k\right)=\left\{\begin{array}{c}1 if {\tilde{V}}_{max}\left(k\right)\ge V_{threshold}\left(k\right)\\ 0 if {\tilde{V}}_{max}\left(k\right)<V_{threshold}\left(k\right)\end{array}\right.$$

(99)

where $\mathrm{N}=1,w\left(k\right)=1,V_{threshold}\left(k\right)$ threshold signal triangular shaped.

IF ${\tilde{y}}_{step}\left(\mathrm{k}\right)>0 THEN$

$${z_{in}\left(k\right)=\tilde{y}}_{step}\left(\mathrm{k}\right)$$

(100)

$${\tilde{V}}_{minj}\left(k\right)=\min \left(z_{inj}\left(k\right),w_{inj}\left(k\right)\right)$$

(101)

$${\tilde{V}}_{max}\left(k\right)={MAX}_{j=1}^N{\tilde{V}}_{minj}\left(k\right),N=1$$

(102)

$${\tilde{V}}_{out}\left(k\right)=\max \left({\tilde{V}}_{max}\left(k\right),V_{threshold}\left(k\right)\right),V_{threshold}\left(k\right)=0$$

(103)

$${\tilde{V}}_{out\gamma }\left(k\right)=min\left(\gamma ,{\tilde{V}}_{out}\left(k\right)\right),\gamma =1$$

(104)

$${\tilde{y}}_{SPKAF}\left(k\right)=\frac{c_1}{1+e^b-a\cdot e^b\cdot {\tilde{V}}_{out\gamma }\left(k\right)+a^2\cdot e^b\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^2-a^3\cdot e^b\cdot c_2\cdot {{\tilde{V}}_{out \gamma }\left(k\right)}^3}$$

(105)

ELSE ${\tilde{y}}_{SPKAF}\left(k\right)=0$

Note: all spikes have the same shape and period.

IF ${\tilde{y}}_{SPKAF}\left(k\right)={\tilde{y}}_{SPKAF SI}\left(k\right) THEN$

$${\tilde{y}}_{SPKAF IDENTIFIED}\left(k\right)={\tilde{y}}_{SPKAF SI}\left(k\right)$$

(106)

ELSE ${\tilde{y}}_{SPKAF IDENTIFIED}\left(k\right)=0$

$$e_{SPKAF IDENTIFIED}\left(k\right)={\tilde{y}}_{SPKAF SI}\left(k\right)-{\tilde{y}}_{SPKAF IDENTIFIED}\left(k\right)$$

(107)

Remark 2.

FAN-SPKAF uses the same learning algorithm for unipolar fuzzy systems, $\in$ [0,1], developed for FAN-SAF in [16,31,33]. Because the spikes are bipolar, to normalize them to the unipolar interval, a max operator and a zero reference or a comparator with square output pulses $\in$ [0, 1] could be used. The spike period and the absolute refractory period [3], are considered constant.

Therefore, the nonlinear system can be expressed by

$${\widehat{Y}}_{step}\left(k\right)=\mathsf{\Phi }\left[\widehat{X}\left(k\right),W_1\left(k\right)\right]$$

(108)

$$\widehat{H}\left(k\right)=\mathsf{\Phi }\left[{\widehat{Y}}_{step}\left(k\right),W_2\left(k\right)\right]$$

(109)

$${\widehat{Y}}_{SPKAF}\left(k\right)=\left[\widehat{H}\left(k\right),0\right]$$

(110)

where $W_1\left(k\right)=W_2\left(k\right)=\left[1,\cdots ,1\right]$ is proposed fixed weights and $\mathsf{\Phi }\left(·\right)$ is the nonlinear function corresponding to the membership functions of the fuzzy system.

4.2. Fuzzy System Training

Model (84) is based on input–output data of the system to be identified, and (83) applies the learning algorithm for FAN method. The fuzzy neural modeling main idea is to find the values of $W\left(k\right)$, such that output $\widehat{Y}\left(k\right)$ of the proposed model (84) can follow the $Y\left(k\right)$ output of the nonlinear plant. The identification error is defined as follows:

$$e\left(k\right)=Y\left(k\right)-\widehat{Y}\left(k\right)$$

(111)

FAN with spike, sigmoid, and step-type activation function share the same learning method, developed in [16,31,33], and presented below for unipolar fuzzy systems:

$$W\left(k+1\right)=W\left(k\right)+\mathsf{\Gamma }\left(k\right)·E\left(k\right)·H\left(k\right)$$

(112)

$$\mathsf{\Gamma }\left(k+1\right)=\mathsf{\Gamma }\left(k\right)+\mathsf{\Gamma }\left(k\right)·E\left(k\right)·H\left(k\right)$$

(113)

$$0<\mathsf{\Gamma }\left(k\right)\le 1,0\le W\left(k\right)\le 1$$

(114)

$$w\left(k+1\right)=w\left(k\right)+\gamma \left(k\right)·e\left(k\right)·h_{in}\left(k\right)$$

(115)

$$\gamma \left(k+1\right)=\gamma \left(k\right)+\gamma \left(k\right)·e\left(k\right)·h_{in}\left(k\right)$$

(116)

4.3. Stability Analysis

The stability of the models proposed in Section 4.1 depends on the convergence of the learning algorithm of the FAN; therefore, the stability analysis is performed using (83) of the model (84), which develops similarly to the one presented in [31]:

$$\mathrm{Plant}: y=\mathsf{\Phi }\left[H\left(k\right),W^{*}\right]$$

(117)

$$\mathrm{Model}: \widehat{y}=\mathsf{\Phi }\left[H\left(k\right),W\left(k\right)\right]$$

(118)

$$\mathrm{Error}: y-\widehat{y}=\mathsf{\Phi }\left[H\left(k\right)·\left(W^{*}-W\left(k\right)\right)\right]$$

(119)

$$Y\left(k\right)=\mathsf{\Phi }\left[H\left(k\right),W^{*}\right]+\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]+d\left(t\right)$$

(120)

$$\widehat{Y}\left(k\right)=\mathsf{\Phi }\left[H\left(k\right),W\left(k\right)\right]$$

(121)

$$e\left(k\right)=\mathsf{\Phi }\left[H\left(k\right),W^{*}\right]+\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]-\mathsf{\Phi }\left[H\left(k\right),W\left(k\right)\right]$$

(122)

$$e\left(k\right)=\mathsf{\Phi }\left[H\left(k\right),\tilde{W}\left(k\right)\right]+\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]$$

(123)

$$e\left(k\right)=\mathsf{\Phi }\left[H\left(k\right),\tilde{W}\left(k\right)\right]+{\mu }_d\left(k\right)$$

(124)

where $\tilde{W}\left(k\right)=W^{\mathrm{*}}-W\left(k\right)$, and ${\mu }_d\left(k\right)=\mathsf{\Phi }\left[H_d\left(k\right),W_d^{\mathrm{*}}\right]$.

We assume plants (83) and (84) are bounded-input–bounded-output (BIBO)-stable, i.e., $y\left(k\right)$ and $h\left(k\right)$ in (83) are bounded. The membership function $\mathsf{\Phi }\left(·\right)$ is bounded. The following theorem provides the stability analysis for nonlinear system modeling with the fuzzy system.

Theorem 1.

If the unknown nonlinear system (62) is modeled by the fuzzy system (84), the membership functions are updated by (112)–(116), and then the modeling error $e\left(k\right)$ is bounded. The normalized identification error,

$$E_N\left(k\right)=\frac{W_N\left(k+1\right)-W_N\left(k\right)}{{\mathsf{\Gamma }}_N\left(k\right)·H\left(k\right)}$$

(125)

satisfies the following average performance:

$$\underset{T\to \infty }{\lim }\sup \frac{1}{T}\sum _{k=1}^T{‖E_N\left(k\right)‖}^2\le {max}_k\left[{‖\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]‖}^2\right]$$

$$\underset{T\to \infty }{\lim }\sup \frac{1}{T}{\sum }_{k=1}^T{‖E_N\left(k\right)‖}^2\le {\stackrel{-}{\mu }}_d$$

(126)

Proof: 

For unipolar systems with values in [0,1], the conditions for $W\left(k+1\right)$ and $\mathsf{\Gamma }\left(k+1\right)$ are

$$\begin{gathered} \mathrm{IF} W\left(k+1\right)>1 \mathrm{THEN} W_N\left(k+1\right)=1. \\ \mathrm{IF} W\left(k+1\right)<0 \mathrm{THEN} W_N\left(k+1\right)=0. \\ \mathrm{ELSE} W_N\left(k+1\right)=W_N\left(k\right)+∆W_N\left(k\right). \\ \mathrm{IF} \mathsf{\Gamma }\left(k+1\right)>1 \mathrm{THEN} {\mathsf{\Gamma }}_N\left(k+1\right)=1. \\ \mathrm{IF} \mathsf{\Gamma }\left(k+1\right)<0 \mathrm{THEN} {\mathsf{\Gamma }}_N\left(k+1\right)=n. \\ \mathrm{ELSE} {\mathsf{\Gamma }}_N\left(k+1\right)={\mathsf{\Gamma }}_N\left(k\right)+∆{\mathsf{\Gamma }}_N\left(k\right). \end{gathered}$$

where $0<n\le 1$.

Therefore,

$${\mathsf{\Gamma }}_N\left(k+1\right)={\mathsf{\Gamma }}_N\left(k\right)+{\mathsf{\Gamma }}_N\left(k\right)·E\left(k\right)·H\left(k\right)$$

(127)

$$W_N\left(k+1\right)=W_N\left(k\right)+{\mathsf{\Gamma }}_N\left(k\right)·E\left(k\right)·H\left(k\right)$$

(128)

We selected a positive defined scalar $L_k$, defined as

$$L_k={‖\tilde{W}\left(k\right)‖}^2$$

(129)

where $‖·‖$ denotes the Euclidean norm.

By the updating law (126), we have

$$\tilde{W}\left(k+1\right)=\tilde{W}\left(k\right)+\mathsf{\Gamma }\left(k\right)·E\left(k\right)·{H\left(k\right)}^T$$

(130)

Using the inequalities,

$$‖q+r‖\le ‖q‖+‖r‖,‖q·r‖=‖q‖·‖r‖$$

For any $q$ and $r$, by using (130) and $0<{\mathsf{\Gamma }}_N\left(k\right)\le \mathsf{\Gamma }\left(k\right)\le 1$,

$$\begin{gathered} ∆L_k=L_{k+1}-L_k \\ ={‖\tilde{W}\left(k\right)+\mathsf{\Gamma }\left(k\right)·E\left(k\right)·{H\left(k\right)}^T‖}^2-{‖\tilde{W}\left(k\right)‖}^2 \\ =2‖\mathsf{\Gamma }\left(k\right)·E\left(k\right)·{H\left(k\right)}^T·\tilde{W}\left(k\right)‖+{‖\mathsf{\Gamma }\left(k\right)·E\left(k\right)·{H\left(k\right)}^T‖}^2 \\ ={‖\mathsf{\Gamma }\left(k\right)‖}^2·{‖\mathrm{E}\left(k\right)‖}^2·{‖{H\left(k\right)}^T‖}^2+2‖\mathsf{\Gamma }\left(k\right)·E\left(k\right)·{H\left(k\right)}^T·{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]\right),{H\left(k\right)}^T\right]‖ \\ ={‖\mathsf{\Gamma }\left(k\right)‖}^2·{‖\mathrm{E}\left(k\right)‖}^2·{‖{H\left(k\right)}^T‖}^2+2‖\mathsf{\Gamma }\left(k\right)‖·‖\mathrm{E}\left(k\right)‖·‖{H\left(k\right)}^T‖·‖{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]\right),{H\left(k\right)}^T\right]‖ \\ ∆L_k\le {\zeta \left(k\right)·‖\mathrm{E}\left(k\right)‖}^2+\delta \left(k\right)·‖\mathrm{E}\left(k\right)‖·‖{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]\right),{H\left(k\right)}^T\right]‖ \\ ∆L_k\le {\zeta \left(k\right)·‖\mathrm{E}\left(k\right)‖}^2+\delta \left(k\right)·‖\mathrm{E}\left(k\right)‖·‖{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-{\mu }_d\right),{H\left(k\right)}^T\right]‖ \end{gathered}$$

(131)

where $\zeta \left(k\right)$ and $\delta \left(k\right)$ are defined as,

$$\begin{gathered} \zeta \left(k\right)={{‖\mathsf{\Gamma }\left(k\right)‖}^2·‖{H\left(k\right)}^T‖}^2 \\ \delta \left(k\right)=2‖\mathsf{\Gamma }\left(k\right)‖·‖{H\left(k\right)}^T‖ \end{gathered}$$

Because

$$n\min \left({\tilde{w}}_i^2\right)\le L_k\le n\max \left({\tilde{w}}_i^2\right)$$

where $n\min \left({\tilde{w}}_i^2\right)$ and $n\max \left({\tilde{w}}_i^2\right)$ are ${\mathcal{K}}_{\mathrm{\infty }}-functions$; ${\zeta \left(k\right)·‖\mathrm{E}\left(k\right)‖}^2$ is a ${\mathcal{K}}_{\mathrm{\infty }}-function$; and $\delta \left(k\right)·‖\mathrm{E}\left(k\right)‖·‖{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-{\mu }_d\right),{H\left(k\right)}^T\right]‖$ is a $\mathcal{K}-function$. Thus, $L_k$ admits an ISS-Lyapunov function [31], and the dynamic of the identification error is input-to-state stable.

From (120) and (129) we know $L_k$ is the function of $E\left(k\right)$ and $\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]$. The INPUT and the STATE correspond to both terms of (131). However, usually, $\mathsf{\Phi }\left[H_d\left(k\right),W_d^{*}\right]\ll \mathsf{\Phi }\left[H\left(k\right){,W}^{*}\right]$.

Because the INPUT is bounded and the dynamic is ISS, therefore the STATE $\mathrm{E}\left(k\right)$ is bounded.

Applying the bounded conditions for $W_N\left(k+1\right)$ and ${\mathsf{\Gamma }}_N\left(k+1\right)$, Equation (131), from 1 up to T and using ${0<L}_T$ and $L_1$ is a constant,

$$\begin{gathered} {\zeta }_N\left(k\right)·\left(\sum _{k=1}^T{‖E_N\left(k\right)‖}^2\right)+{\delta }_N\left(k\right)·\left(\sum _{k=1}^T‖E_N\left(k\right)‖\right)·‖{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-{\stackrel{-}{\mu }}_d\right),{H\left(k\right)}^T\right]‖{\le L}_T-L_1 \\ {\zeta }_N\left(k\right)={{‖{\mathsf{\Gamma }}_N\left(k\right)‖}^2·‖{H\left(k\right)}^T‖}^2 \\ {\delta }_N\left(k\right)=2‖{\mathsf{\Gamma }}_N\left(k\right)‖·‖{H\left(k\right)}^T‖ \\ {\zeta }_N\left(k\right)·\left(\sum _{k=1}^T{‖E_N\left(k\right)‖}^2\right)\le L_T-L_1-{\delta }_N\left(k\right)·\left(\sum _{k=1}^T‖E_N\left(k\right)‖\right)·‖{\mathsf{\Phi }}^{-1}\left[\left(E\left(k\right)-{\stackrel{-}{\mu }}_d\right),{H\left(k\right)}^T\right]‖ \end{gathered}$$

(126) is established. □

Remark 3.

It is complex to obtain high modeling accuracy for the classical fuzzy neural networks due to the difficulty in deciding the hyperparameters of the fuzzy neural systems. However, our fuzzy system with adaptive neurons has fewer hyperparameters to be chosen, and we prove that modeling error converges to the zone${\stackrel{-}{\mu }}_d$.

Remark 4.

If the fuzzy system (2) could match the nonlinear plant (1) exactly (${\mu }_d\left(k\right)=0)$, i.e., we could find the best membership function ${\Phi }_H$ and $W^{*}$such that the nonlinear system could be written as $Y\left(k\right)={\Phi }_H\left[H\left(k\right),W^{*}\right]$, the same learning algorithm makes the identified error$‖E\left(k\right)‖$asymptotically stable:

$$\underset{K\to \infty }{\lim }‖\mathrm{E}\left(k\right)‖=0$$

(132)

Remark 5.

The normalization of learning rates in (120) and (124) are time-varying to insure the stability of identification error. Because the initial condition does not need any previous information, the time-varying learning rates usually are robust.

5. Results of the Simulation of the Fuzzy Systems

5.1. Recognition of Spatiotemporal Spike Patterns

For each reference spike input pattern or sequence that could be presented to the system, axonal delays of each spike and channel are identified. Subsequently, the pattern recognition of the reference spike sequences is performed with the ANFIS and FAN-SPKAF methods of different inputs to the system consisting of noise spike sequences. The spike pattern reference sequences, five input channels, are shown in Figure 5.

The initial conditions for ANFIS method are as follows:

○

Weights $ϵ$ [0,1],

$$p_{i ANFIS}\left(k\right)=0,q_{i ANFIS}\left(k\right)=0,r_{i ANFIS}\left(k\right)=0,s_{i ANFIS}\left(k\right)=0.$$

○

Learning factor, fixed value, ${\gamma }_{FAN}\left(k\right)=1$.

○

$a=11,b=6,g_1=9.$

The initial conditions for FAN-SPKAF method are as follows:

○

Weights $ϵ$ [0,1],

$$w_{in FAN-SPKAF}\left(k\right)=1$$

○

Learning factor, fixed value, ${\gamma }_{FAN}\left(k\right)=1$.

○

Threshold, fixed value, $V_{threshold FAN}\left(k\right)=0$.

○

$a=2,b=2,c_1=0.09,c_2=0.5,g_2=0.4.$

The output of the MIN operator, together with the spike response of the ANFIS and FAN-SPKAF methods, are shown in Figure 6.

Figure 7a–e describes the five noisy input spike sequences, with target spike patterns indicated by dotted lines. Figure 7f presents the result of the MIN operator, the target pattern of the fuzzy system, and the output of the FAN-SPKAF. We can notice that the error is practically null.

The results obtained in Figure 7 based on the FAN-SPKAF method, compared to the spike response of ANFIS and other methods [15], are quite precise because with the ANFIS method the response presents additional spikes, while with the FAN-SPKAF method the pattern recognition is successful, and the spikes coincide with the reference sequences for the recognition of spatiotemporal spike patterns, thus reducing the error.

5.2. Syllable-Based Spike Pattern Speech Recognition

Taking a sample of each syllable of the speech of each one of the musical notes of solfeggio in Spanish language, the first notes of “Ode to Happiness” are composed. Subsequently, the musical notes SI are identified. Environmental noise is considered null.

5.2.1. Results of the FAN-STEPAF-SPKAF Method

In Figure 8, the results obtained from the simulation of the fuzzy system for syllable-based spike pattern speech recognition are shown. The spike pattern of the musical note SI ${\tilde{y}}_{SPKAF SI}\left(k\right)$ is the syllable to be identified from the speech sample in Figure 8a. From Figure 8a the FAN-STEPAF is applied to the speech sample, obtaining the pulse train proportional in duration and frequency to the amplitude of the voice sample. Then the SPKAF generates the spike trains, shown in Figure 8b.

Figure 8d shows the spike patterns of the SI syllable ${\tilde{y}}_{SPKAF SI}\left(k\right)$. With the fuzzy system of Figure 4, the results show that the recognition was performed successfully. Although in second 14–15 it is observed that the model recognized three continuous SI, there is only one Spanish speaker; to obtain three overlapping and consecutive SI, three Spanish speakers are required simultaneously. In Figure 8, ${\tilde{y}}_{SPKAF IDENTIFIED}$ shows two additional spikes because the musical notes SI and LA coincide at time and magnitude around second 14.7.

5.2.2. Results of the Augmented Spiking Neuron Model Method

Normalized magnitude to [0,1] of the melody with musical notes of solfeggio is shown in Figure 9a; the train of square pulses (37) are shown in Figure 9b; the train of spikes based on the Augmented Spiking Neuron Models (41) and (42) are shown in Figure 9c; and in Figure 9d the recognized pattern of the syllable SI with methods (43)–(48) is shown.

The results obtained with the ASNM method after signal processing of the amplitude of the speech sample applying the STFT and (24)–(26), Figure 9, are satisfactory because a more accurate pattern recognition is obtained, and, therefore, the error is reduced.

5.2.3. Results of the Augmented FAN-STEPAF-SPKAF Method

Figure 10a shows the normalized magnitude of the melody with musical notes of solfeggio; Figure 10b shows the square pulse train proportional in frequency to the amplitude of the normalized magnitude of the melody; Figure 10c shows the train of ${\tilde{y}}_{sawtooth}$ (49) after signal processing with STFT of the normalized magnitude of the melody; Figure 10d shows the train of spikes based on the Augmented FAN-STEPAF-SPKAF model (55); and Figure 10e shows the recognized pattern of the syllable SI with (56)–(61).

The FAN-STEPAF-SPKAF model was augmented by including an input ${\tilde{y}}_{sawtooth}$ and spike coefficients to obtain augmented spikes, successful pattern recognition, and much more precise results in Figure 10. Errors were further reduced.

5.3. MSE

Table 1. MSE.

MODEL	MSE	Precision (%)
FAN-STEPAF-SPKAF	1.0823 × 10−4	99.99
Augmented Spiking Neuron Model	3.6915 × 10−4	99.96
Augmented FAN-STEPAF-SPKAF	1.6543 × 10−7	99.999983457

presents the MSE and Figure 11 the error of spike pattern recognition of the results obtained from the FAN-STEPAF-SPKAF, Augmented Spiking Neuron Model, and Augmented FAN-STEPAF-SPKAF methods.

The comparison of the errors and the percentage of precision of coincidence of syllable pattern recognition shows that the Augmented Spiking Neuron Model method has the lowest percentage of precision of coincidence, and the FAN-STEPAF-SPKAF and Augmented FAN-STEPAF-SPKAF methods have the lowest MSE; therefore, in Figure 11 and Table 1, the superiority of the proposed Augmented FAN-STEPAF-SPKAF method is observed.

6. Conclusions and Discussion

The proposed fuzzy systems for the recognition of spatiotemporal spike patterns ANFIS and FAN-SPKAF were compared. The ANFIS method achieved good results but with noise. Stability analysis of the system model was performed with great success. The results obtained with the FAN-SPKAF method exceeded expectations because the noise decreased.

FAN-SPKAF produces a spike response, recognizing the spike pattern of the SI syllable of the reference sequence of the melody with solfeggio musical notes. To generate a spike response, we could also opt for the spike response model developed in [5,10,11]; however, the new SPKAF is a simpler model.

The fuzzy system with the FAN-STEPAF-SPKAF model for syllable-based speech recognition developed in this article produced very good results, considering optimal conditions, null ambient noise, and constant speech characteristics such as articulation, timbre, and tone. Comparing the Augmented Spiking Neuron Model and Augmented FAN-STEPAF-SPKAF methods, the latter model has the advantage that pattern recognition is performed with a minimum of MSE, although all methods require a reference pattern.

For future work, it is proposed to identify the corresponding axonal delays with the FAN and its application in control [29,30,35], unmanned aerial vehicles (UAV) [34,42], and modeling of seismic events [31], among other areas of knowledge [5,43,44,45,46,47,48,49,50,51,52,53,54].