A Spiking Neural Network Based on the Model of VO₂–Neuron

Abstract

In this paper, we present an electrical circuit of a leaky integrate-and-fire neuron with one VO₂ switch, which models the properties of biological neurons. Based on VO₂ neurons, a two-layer spiking neural network consisting of nine input and three output neurons is modeled in the SPICE simulator. The network contains excitatory and inhibitory couplings, and implements the winner-takes-all principle in pattern recognition. Using a supervised Spike-Timing-Dependent Plasticity training method and a timing method of information coding, the network was trained to recognize three patterns with dimensions of 3 × 3 pixels. The neural network is able to recognize up to 10⁵ images per second, and has the potential to increase the recognition speed further.

1. Introduction

Artificial neural networks (ANNs), created by analogy with biological neural systems, are used to resolve various tasks, such as classification, clustering, and pattern recognition [1,2,3,4]. The main element of ANN is a neuron that may have several inputs and one output, and neurons can be connected in different ways, depending on the network architecture [5,6]. The main task of a neuron is to convert input signals to output signal using an activation function [5]. In the history of ANN, three generations of the networks are usually distinguished. The first generation includes simple forward and backward connection networks that operate with binary data and stepwise activation functions [7]. The second generation includes multilayer networks of direct and reverse distribution, operating with rational numbers with continuous activation functions [7]. The third generation of ANN (spiking neural networks (SNN)) uses biosimilar models of neurons that take into account not only the magnitude of the signals arriving at the input, but also the signals’ temporal distribution [7,8].

There is a large number of SNNs, which are used to solve practical tasks and are based on mathematical models of neurons (Integrate-and-Fire, Izhikevich, Hodgkin-Huxley) [9,10,11,12]. Such SNNs use the resources of computers, video cards, and field-programmable gate arrays to emulate the network operation [9,10,11,12,13]. Although such SNNs currently provide impressive performance results [14], any emulation loses hardware implementation in performance and energy efficiency [15,16]. Therefore, the development of SNNs based on microelectronic elements attracts the active attention of researchers [16,17,18,19,20,21,22]. One of the most frequently used functional elements of SNN is a memristor [16], which is used for implementing customizable weights and as a functional element of a neuron. The weights are adjusted during the network training, and, in electric networks, it is implemented by changing the impedance of the lines connecting the outputs and inputs of neurons. A multi-stable resistive memory cell is an ideal object for implementing a wide SNN functionality, due to the possibility of changing the resistance over a wide range of values. However, the application of a resistive memory cell as a bi-stable element with an off state (inactive neuron) and on state (active neuron) is not an optimal solution because of the high probability of its resistance modification during the operation [23,24].

In the current study, for the manufacturing of artificial neurons, we propose to use elements with a stable S-shaped I – V characteristic, such as switches based on transition metal oxides with a metal-insulator transition [25,26,27]. Implementations of neuron models on the VO₂ switch are described in References [28,29,30,31,32,33,34]. However, a few SNN implementations using such neurons have been proposed so far. Models of VO₂ neurons can be divided into two groups. The first group is an integrate-and-fire model of a neuron [32,33,35], which has three main states: the accumulation of action potential state due to charging the capacitor, the spike generation state, when the capacitor is discharged, and the VO₂ switch goes into a highly conductive state, and the inactivity state of a neuron. The discharge time of the capacitor is treated as a post firing refractory period [30,32], and the initiation of the second pulse is impossible at that time due to shunting of the low resistance of the switch. The second group of models covers neuron circuits that include inductance, and the possibility of generating a burst mode, which is similar to the FitzHugh-Nagumo and FitzHugh-Rinzel models [34,35].

We propose a leaky integrate-and-fire (LIF) circuit for a neuron based on a VO₂ switch that can implement excitatory and inhibitory couplings. Based on VO₂ neurons, in the SPICE simulator, the operation of a two-layer SNN network consisting of nine input and three output neurons was modeled. An image in the form of a 3 × 3 matrix is fed to the network input, and, at the output, one of the three neurons is activated with a certain input pattern, and this neuron suppresses the remaining output neurons according to the winner-take-all (WTA) principle [36]. The coding of information in the proposed network is performed by setting the delay time of the spikes in the input layer relative to the zero time moment (time to the first spike) [37]. Network training is performed according to the spike time-dependent plasticity (STDP) scheme [14,16,19,20,21,38]. As a result, a model SNN, based on VO₂ neurons, which allows pattern recognition, is presented and investigated in this study.

2. SNN Modeling Method

2.1. VO₂ Neuron Model

The VO₂-neuron model is created on the basis of the LIF neuron model, which is widely used due to the simplicity of implementation and the possibility of generating biosimilar spikes [39]. Its main element is a bi-stable two-electrode VO₂ switch [25,26,27]. The operation principle of the switch is based on the metal-insulator phase transition in VO₂ films, which happens near the transition temperature T_th ~ 340 K. The critical temperature T_th in the film is achieved due to the Joule heating effect when passing a current, which leads to a sharp abrupt change in the resistance [27]. In addition to the thermal effect, when modeling electric switching, the effect of the electric field on the concentration of charge carriers is taken into account [40,41]. The model I – V characteristic of the VO₂ switch corresponds to the experimental I – V characteristic (Figure 1), measured in our previous work [25], on a planar switch with a channel size of 2.5–3 μm, and a VO₂ film thickness of ~ 250 nm, with a current limiting resistor of 250 Ω connected in series.

Figure 1 demonstrates the dependence of the switch current I_sw on the voltage V_sw supplied to the switch. Reaching the threshold switching voltages V_th = 5.6 V and holding voltage V_h = 2.2 V, the switch passes from a high-resistance state to a low-resistance state and vice versa. The high-resistance and low-resistance branches of the I – V characteristic are approximated by linear dependencies on the voltage V_sw with resistance values R_off ~ 14 kΩ and R_on ~ 300 Ω, respectively.

To conduct SPICE simulations of the VO₂ neuron, a standard voltage-controlled switch was used with parameters corresponding to the experimental I – V characteristics (R_off, R_on, V_th, and V_h).

The electrical circuit of the VO₂ neuron is shown in Figure 2. The neuron model has n inputs, and one output V_out. Resistances $R_{\mathrm{w}}^1\dots R_{\mathrm{w}}^{\mathrm{n}}$ play the role of a synaptic weights between neurons. The smaller the resistance, the more the signal from the i-th input affects the neuron. The spikes coming from the inputs through the resistances are accumulated on the C_sum capacitance, by charging it with the cumulative charge. The charge from the C_sum capacitor gradually flows through the resistance R_in. C_sum capacitance voltage is an effective input signal that affects the current state of a neuron. The supply voltage V_dd is selected so that the VO₂ switch stays in the off state in the absence of input signals. The most clear way to achieve this condition is to set the voltage V_dd less than the switching voltage V_th. In this model, the inactive state of the neuron corresponds to the switched off VO₂ switch when it is in the subthreshold mode (V_sw <V_th). To activate the neuron, the switch should be turned on by setting the voltage on the switch to V_sw≥V_th. To achieve this, the supply voltage V_dd must have negative values, and the spikes supplied to the input must have a positive polarity.

To activate a neuron, the voltage across the capacitance C_sum should increase to a threshold value V_{c_th}, which depends on V_dd, the resistance of the switch in the off state R_off, and the values of the resistors R_s and R_in. After the switch is turned on, its resistance decreases to R_on, which leads to the discharge of the capacitance C_c through the resistances R_in and R_out. The capacitance C_c serves as a reservoir of charge, which is necessary for generating a spike when a neuron is activated. When C_c is discharged, a spike of positive polarity is generated at the output V_out of the neuron. By connecting the outputs of some neurons with the inputs of other neurons, SNNs with excitatory coupling can be obtained. The resistance R_s is load resistance and sets the operating current through the switch. R_s is selected in a way that the VO₂ switch turns off after discharging the capacitance C_c. In fact, the neuron circuit is tuned to generate a single current spike through the VO₂ switch, i.e., generate a single spike at the output.

Figure 3a presents the electrical circuit of a VO₂ neuron. The pulses from the voltage generator are supplied to the input of the neuron, and the output is connected to the input stage of the subsequent neuron (to simulate the output load of the neuron). The circuit modeling was performed in the LTspice XVII simulation software. Resistance and capacitance values: R_{w_1} = 500 Ω, R_{w_2} = 1 kΩ, R_s = 700 Ω, R_in = 1 kΩ, R_out = 10 kΩ, C_sum = 1 nF, and C_c = 10 nF. Supply voltage V_dd = −5.75 V. A pulse of positive polarity with an amplitude of 2 V and a duration of 0.3 μs is supplied from the generator. Figure 3b depicts the oscillograms of the input V_in and output V_out voltages, as well as the voltages V_c at the capacitance C_sum, which demonstrates the spikes’ dynamics.

The threshold voltage at the C_sum capacitance, required to initiate the output spike, is V_{c_th} ~ 0.33 V (dashed line in Figure 3b). After turning on the VO₂ switch, the capacitance C_c starts to discharge, and it leads to the appearance of a leading edge and a spike with a voltage amplitude of ~ 3.2 V. After turning on the switch, a decrease in the voltage V_c to negative values is associated with active recharging of the C_sum capacitor through an open switch due to the negative voltage on C_c capacity and V_dd power supply. The spike duration is ~ 170 ns, which is determined by the discharge time of the capacitance C_c until the moment, when the voltage at the switch V_sw is not less than V_h. The trailing edge of the pulse appears when the switch goes off. The duration of the output spike can be significantly longer than the duration of the initiating pulse.

The VO₂-neuron model is able to demonstrate various properties of real neurons, such as spike latency, subthreshold oscillations, refractory period, threshold behavior, and spike frequency adaptation [28,39].

For example, Figure 4a demonstrates that the higher the amplitude of the input pulse exists, the smaller the time delay between the leading edges of the input and output pulses remains, called spike latency. With a pulse amplitude of 2 V, the latency between the input and output signals is 140 ns, and with a pulse amplitude of 1 V, the latency reaches 440 ns. Therefore, the amplitude and duration of the input pulse, required to initiate the spike, can lie in a wide range. However, when the amplitude of the input pulse is less than V_{c_th}, the initiation of the output pulse does not occur.

If the input pulse is sufficiently long, several spikes can be obtained at the output of the circuit. Figure 4b demonstrates the response of a VO₂ neuron to a pulse with an amplitude of 1 V and a duration of 3.6 μs, which forms five spikes at the output. The latter mode resembles the occurrence of oscillations when an excitation signal is applied (subthreshold oscillations). The delay between the spikes T_r, called the refractory period, is approximately 630 ns and is determined by the charging time of the capacitor C_sum to voltage V_{c_th}. The refractory period depends on the amplitude of the pulse. For example, at a pulse amplitude of 2 V, the period T_r is 300 ns. In addition, the refractory period T_r is slightly increasing (see the values indicated in Figure 4b), because of the small increase in V_{c_th} from spike to spike, since the capacitance C_c does not have time to charge to its original values. This increase in the time period between the spikes under constant exposure is similar to biological neurons (spike frequency adaptation) [28,39].

If the delay between the spikes is less than the refractory period, the neuron generates only one spike. Figure 4c demonstrates two input pulses with an amplitude of 1 V and a delay of 300 ns, and the neuron generates a spike only for the first input pulse.

To implement the wide functionality of neural networks, in addition to excitation connections, the possibility to add inhibitory connections is required. Inhibitory connections are widely used in the SNN output layer to implement the WTA rule. Such connections allow the first spike-generated neuron to deactivate all other related neurons using the inhibitory connections. As a result, only one neuron, which is associated with a recognized class, is activated. Figure 5a demonstrates a diagram of two neurons interconnected via capacitances C_inh = 10 nF, which act as inhibitory connections. The capacitance and resistance values correspond to the single neuron circuit shown in Figure 3a, with the exception of R_in = 200 Ω and R_out = 200 Ω. Due to the presence of C_inh capacities, upon activation of one of the neurons and the discharge of its capacitance C_c, the voltage on the capacitance C_c of an inactive neuron decreases. In this case, the first (in time) activated neuron will suppress all other neurons connected to it by inhibitory connections. Namely, in such a group of neurons, the WTA rule is implemented. In order to trace the activation of neurons in this circuit, it is convenient to monitor the current I_sw and voltage V_sw on two switches (Figure 5b). The delay between the supplied pulses V_{in_1} and V_{in_2} is 2 μs. When the first pulse V_{in_1} arrives at the first switch (Figure 5b), the switch turns on, the current I_{sw_1} increases sharply, and the on mode lasts for ~ 4.2 μs. Switching on occurs because the voltage V_{sw_1} reaches the threshold value V_th (Figure 5b). After turning on the first switch, the voltage V_{sw_1} drops sharply and it leads to a decrease in voltage V_{sw_2} on the second switch, as the signal is transmitted through the capacitors C_inh. The second pulse arriving at the input of the second neuron (V_{in_2}) does not activate it, because the voltage V_{sw_2} does not reach the threshold value (V_{sw_2} <V_th). The activation of the first neuron inhibits the activation of the second neuron. If an excitation pulse is applied to the second neuron after deactivation of the first neuron, then the second neuron will go into an active mode.

To determine the activity of neurons in the output layer, the most appropriate solution would be to convert the I_sw current pulses into output voltage pulses. However, this solution requires additional external circuits. As neurons are connected by C_inh capacities, voltages, taken from R_out resistors, are correlated, and it is not advisable to use them. Schematically, as activity markers, the voltages V_{c_1} and V_{c_2} can be used, and their dynamics are shown in Figure 5c. When the first neuron is activated, the voltage V_{c_1} drops sharply due to the recharging of the capacitor C_sum, which forms a strong pulse of negative polarity, and the positive pulse V_{c_2} is weakly expressed on the inactive neuron.

2.2. SNN Architecture

For pattern recognition problems, various SNN architectures are used, which differ in the number of layers and in the way neurons are connected [14,16]. One of the simplest SNN architectures is a two-layer network (Figure 6a), where image information is supplied to the input (first layer), and one of the neurons associated with a certain class of images is activated at the output (second layer) [42,43,44]. Each of the first layer neurons is connected to each neuron of the second layer through excitatory connections. The connection strength between each pair of neurons is specified through synaptic weights, which can vary among themselves. All neurons of the output layer are interconnected by inhibitory connections. When applying signals to the first layer neurons, they are activated and transmit an excitation effect to the second layer. The neuron of the output layer, which is activated first, sends an inhibitory signal on all other output neurons. This prevents their activation. In this way, the WTA rule is implemented, when data is classified by defining the only active neuron in the output layer. The activation speed of the output layer neurons depends on the input signals and synaptic weights between the particular output neuron and each neuron from the input layer. For the correct pattern recognition, during the network training, it is necessary to correctly set the synaptic weights for each group of the output neuron on the input neurons.

Figure 6b presents a coupling diagram of neurons of the input and output layers of a pulsed neural network for image classification. Each element of the input layer corresponds to one pixel of the image. Therefore, to solve the problem of classifying images with a size of 3 × 3 pixels, nine neurons in the input layer are required. The number of output neurons depends on the number of patterns that the network is supposed to recognize. In this study, we will demonstrate the classification of images using three patterns, so the number of neurons in the output layer will be three. The input and output layers of the SNN are connected using synaptic weights, implemented through the resistances R_{w_i, j}, where i is the number of the input neuron and j is the number of the output neuron. The resistance values R_{w_i, j} will change during training. Memristors [16], where resistance can be adjusted, are often used as resistances in the circuits. In this study, we do not consider a circuit implementation that allows the change of resistances R_{w_i, j} during the training process. Instead, we assume to have control over the elements’ resistances. The range of resistance values R_{w_i, j} varies from 1.5 kΩ to 2.5 kΩ.

A signal from the generator, which encodes information about the color of the pixel, is supplied to the input of each input layer neuron. In this study, we use eight-bit grayscale images, so the information encoded by the generator reflects a gray scale, where the black pixel corresponds to the number 0 and the white pixel corresponds to the number 255. The generator is connected to the input layer neuron using an excitation connection. Then, all nine neurons of the input layer are connected by excitation connections to the three neurons of the output layer, which forms 9 × 3 = 27 connections.

All output neurons are interconnected by inhibitory connections, which are implemented by connecting to the inhibitory bus using the capacitance C_inh = 10 nF.

The remaining elements, depicted in Figure 6b, have the following ratings: R_w = 500 Ω, R_s = 700 Ω, R_{in_i} = 1 kΩ, R_{in_j} = 200 Ω, R_{out_i} = 10 kΩ, R_{out_j} = 200 Ω, C_sum = 1 nF, and C_c = 10 nF. The supply voltage of all neurons is V_dd = −5.75 V.

2.3. SNN Training

Before considering the network training algorithm, it is necessary to determine the method of information coding. A large number of information coding methods for SNN has been defined: rate coding, rank coding, time to first spike, latency coding, phase coding, population coding, and others [37]. Typically, two-layer neural networks, used to classify images, apply the rate coding method [43,45,46]. However, in the current study, we use the time to the first spike method [37]. This coding method requires fewer spikes for a single recognition act, and, as a result, less energy is spent on the circuit operation, as most of the energy is spent on generating spikes.

The information coding is performed as follows. The signals from the generators arrive on the first layer of the neural network with a delay Δt relative to the start time of the circuit t = 0, and the delay Δt determines the brightness of the image pixel. The value Δt = 0 corresponds to brightness 0 (black color), and the maximum delay Δt_max = 2 μs corresponds to brightness 255 (white color). The signal from the generator is a rectangular pulse with an amplitude of 2 V and a duration of 0.3 μs. The delay time is counted relative to the leading edge of the pulse.

The network training process is based on the standard STDP mechanism [14,16,19,20,21,38]. This mechanism is an implementation of the Hebbian learning rule and causes a change in synaptic weight depending on the delay Δt_in-out between pre-synaptic and post-synaptic spikes [45]. The traditional rule is an exponential function [45], which depends on Δt_in-out. However, various studies use the simplified versions [42,47], which significantly facilitate the calculations, while maintain the main ideas of the SPDT method. In this study, we use the function presented in Figure 7, where the form is given in the SNN training papers [42,45,47]. Since an increase in synaptic weight corresponds to a decrease in resistance R_{w_i, j}, the function is inverted in relation to the axes of an ordinate. Resistance decreases, if the post-synaptic spike (from the neuron in the output layer) arrives with a delay in the range of 0 to 0.5 μs after the pre-synaptic spike (from the neuron in the input layer). In other cases, the resistance R_{w_i, j} increases.

Typically, the STDP-based training procedure is used in SNN with unsupervised learning [19,20,21,42,44]. In this case, when training input data is supplied, the output neurons are randomly associated with input data patterns. Nevertheless, there are studies on SNN training mechanisms that implement supervised learning [48,49,50]. In these studies, the input pattern is forcibly assigned to a specific output neuron using back error propagation algorithms. In our study, we tried to implement a simplified approach that allows us to implement supervised learning. During the training, the supply voltage V_dd was set to be non-zero only at one of the three output neurons (see

Table 1. An example of the supply voltage setting V_dd of the output layer, using the supervised learning method in SNN training.

The Class of the Image, Fed to the SNN Input	The Voltage Vdd of the Output Neuron No. 1, V	The Voltage Vdd of the Output Neuron No. 2, V	The Voltage Vdd of the Output Neuron No. 3, V
Pattern 1	−5.75	0	0
Pattern 2	0	−5.75	0
Pattern 3	0	0	−5.75

). The remaining neurons are forcibly electrically deactivated. They do not emit spikes, and it causes all the associated weights R_{w_i, j} to increase. During network training (Table 1), power at the output neuron No. 1 is present (V_dd ≠ 0) only when “Pattern 1” images are inputted. When “Pattern 2” and “Pattern 3” images are supplied, the voltage V_dd is zero.

The SNN training algorithm consists of the steps listed in Figure 8. First, arbitrary resistances R_{w_i, j} are set in the range from 1.5 kΩ to 2.5 kΩ. Second, the iterative process of changing the resistances R_{w_i, j} begins. Initially, one of the patterns, that the network should be trained to express, is arbitrarily selected (the number of patterns should be equal to the number of the output layer neurons). In accordance with the pattern and the information coding scheme, the pulse delays are set to be supplied from the generators to the input layer neurons. Then, in accordance with Table 1, the V_dd values of the output neurons are set. Next, the circuit modelling starts in the SPICE simulator. Based on the simulation results, delays between pre-synaptic and post-synaptic spikes Δt_in-out are calculated, and ΔR_{w_i, j} are calculated using the resistance change function (Figure 7). After that, the new values of R_{w_i, j} are set, and, if the values are outside the range of 1.5 kΩ – 2.5 kΩ, R_{w_i, j} is set equal to the nearest border value. The training cycle is repeated, until all the R_{w_i, j} values stop changing.

3. Results

Three patterns for training with a dimension of 3 × 3 pixels are presented in Figure 9. The patterns have the same number of black and white pixels. If the number of black pixels is different, then normalization by color intensity can be applied to obtain more accurate results [51].

Figure 10 illustrates the resistance values R_{w_i, 1}, R_{w_i, 2}, and R_{w_i, 3} between all input neurons and three output neurons before and after network training. Resistance values are grouped by nine pieces according to the number of output neurons.

Before training, the resistances R_{w_i, j} were randomly generated in the range from 1.5 kΩ to 2.5 kΩ. Then, the SNN training procedure was performed for ~100 cycles, described in detail in Chapter 2.3, using the input patterns in Figure 9. As a result, the distribution of resistances for each output neuron began to correspond to the pattern assigned to each output neuron. This distribution is an expected result for two-layer networks operating, according to the WTA mechanism [44]. If the training patterns were a set of images distributed by classes (for example, numbers written in different handwriting), then the distribution of weights would be averaging all patterns belonging to the same class. Such a training outcome is observed in the studies using the MNIST database [42].

Analysis of the trained network reveals the following results. If patterns from the training set were input, then, as expected, the neurons corresponding to the associated pattern were activated at the SNN output.

Furthermore, images corresponding to distorted patterns in which pixel color intensities were randomly changed were inputted. The results of the SNN operation with the number of the activated output neuron are presented in Figure 11. The first three images corresponding to the distorted patterns from the training set were correctly classified by SNN. The first image corresponds to “Pattern 3,” the second image corresponds to “Pattern 1,” and the third image corresponds to “Pattern 2.”

When inputting the fourth image, which is a highly distorted template from the training set (“Pattern 3”), none of the output neurons were activated, and it could be interpreted as an undefined result. Inactivity of the output neurons reflects that the voltage on the capacitors C_sum of the output neurons never reaches the threshold values V_c < V_{c_th}.

This uncertainty could be avoided by reducing the time range for spikes coding Δt_max (see Section 2.3). For example, if Δt_max = 2 μs is reduced by half to Δt_max = 1 μs, then the fourth image in Figure 11 is correctly identified as “Pattern 3.” The pulses have shorter time intervals, and the voltage on the integrating capacitance C_sum is more likely to increase to the threshold value V_{c_th}.

4. Discussion

When modelling neurons, on the one hand, we could strive for greater bio-similarity of a neuron, as implemented in some models (FitzHugh-Nagumo, Izhikevich, Hodgkin-Huxley) [35,39], or, on the other hand, we could try to minimize the number of electrical components in the circuit to contribute to its miniaturization in practical implementation. The integrate-and-fire neuron model, which we propose, permits, on the one hand, to obtain a number of properties observed in biological systems (Section 2.1), and, on the other hand, it contains only one switching element and a power source, in contrast to more complex models [28]. In a number of studies of neurons based on VO₂, silicon semiconductor devices (field effect transistor, diode) are used as additional circuit elements [22,29]. This imposes certain restrictions on the compatibility of technological processes for manufacturing the silicon and non-silicon parts of the circuit. This drawback is avoided in the presented neuron model, where all circuit elements (switch, resistors, and capacitors) are manufactured using vanadium oxides of various stoichiometry (VO, VO₂, V₂O₃, and V₂O₅).

All the results presented in the current study were obtained by modeling the VO₂ neuron using numerical methods in the LTspice simulator. However, the I–V characteristic of the VO₂ switch corresponding to the experimental data was used in the model (see Section 2.1). Therefore, a discussion of the physical mechanisms that affect the I–V characteristics is of great importance for predicting the areas of practical application and comparing the characteristics of neuron models.

An increase in the ambient temperature T₀ leads to a decrease in the threshold voltage V_th, and at T₀~ T_th, the effect of electrical switching will be suppressed, because the VO₂ channel will achieve a highly conductive state [41,52,53]. It imposes a limitation on the use of a VO₂ neuron, where the operability will be limited at T₀ > T_th. The value of T_th is not high, and reaches T_th ~ 340 K (67 °C) and creates the question - how to increase T_th? A good overview of the T_th modulation methods by doping with various elements is presented in Reference [54]. For example, Cr doping can increase T_th by 10 °C [55]. An alternative way to increase T_th is to use other materials with an S-type I–V characteristic. NbO₂–based structures, having T_th ~ 1070 K, demonstrate electrical switching up to temperatures of T₀ ~ 300 °C [56], and can be used in the presented neuron model. The model is invariant to the use of other materials with the effect of electrical switching, and the main requirement is the presence of an S-type I–V characteristic. VO₂-based structures are a good model object and are often used in neural circuits. Nevertheless, the task of finding switching structures with a wide temperature range is a promising endeavour.

Another problem is the variation of V_th with the temperature. Therefore, if it is necessary to stabilize the operation of a neuron, it is necessary to come up with additional thermal compensation schemes.

To optimize the circuit presented in Figure 2, we propose to exclude the capacitor C_sum from the circuit. When switching a VO₂ switch by rectangular pulses, there is an effect of a time delay of switching on the switch. An inverse dependence of the time delay on the pulse amplitude is associated with the thermal heating of the switching channel to the phase transition temperature of the metal insulator. The physics of the electrical switching process is described in detail in Reference [27]. If pulse durations and coding time intervals Δt_max are used within the delay times of the switching, the pulse integration effect can be implemented without C_sum capacity, and will be caused by the heat accumulation in the region of the switching channel. A similar idea to use heat storage in the switch region to accumulate action in a neuron was proposed in Reference [31]. The role of capacitors in the oscillator circuit is discussed in a number of sources [25,57,58], and the oscillations can be obtained without an integrating capacitor, while being only due to the effects of heat storage. The study of the effect of temperature integration of input pulses could be the subject of future research.

The coupling between the switches in the network can be implemented not only by electrical coupling through resistors R_{w_i, j}, but, as described in our previous studies, can be organized through the thermal coupling of the switches [59,60]. The development of spike neural networks with thermal coupling could be the subject of further research.

The reduction of SNN classification uncertainty, when applying a highly distorted template (the right image in Figure 11), by reducing the time interval for coding spikes Δt_max, has its own limitations. The current SNN model does not take into account the effect of the turn-on and turn-off delay of the switch described above. For example, the turn-on time of the VO₂ switch, using our input signal amplitudes, does not exceed 10 ns [25], while the turn-off time can be much longer (hundreds of nanoseconds). In the time scale of the current SNN model, operating in microsecond intervals, by taking into account the effect of the turn-on delay, does not affect the results of the SNN operation. However, the turn-on and turn-off times can vary significantly, when using other switches, resistors, and resistances. It should be taken into account when designing SNN.

An important characteristic of the network is the pattern recognition time [61]. The SNN architecture, which we propose, provides recognition time of the coding interval order, corresponding to 2–3 μs. After the recognition is completed, the system requires ~ 7–8 μs to reach the initial state, caused by the recharging of the capacities. Therefore, the current SNN is able to recognize up to 10⁵ images per second, and its performance can be increased by reducing the capacitance rating and scaling the VO₂ switches [25]. The implemented method of information coding allows the use of single spikes. It does not only reduce the power consumption compared to the networks using rate coding, but minimizes the time to perform one image recognition operation.

At the end of this section, we present a comparison table of neurons with other proposed neuron devices. The neurons in

Table 2. Comparison of neurons with other proposed neuron devices.

Device	Neuron Type Material/Platform	Active Element Size (a) and Neuron Area (Sneuron)	Spike Amplitude (Vspike), Peak Power (Pmax), Duration, (Δtspike) and Energy per Spike (Espike)	Integration and Threshold Mechanism, Threshold Voltage of the Active Element Vth	SNN with Object Recognition, Coding Mechanism
VO2 (current study)	Leaky Integrate and Fire Vanadium Dioxide (VO2)	a ~ 3 μm	Vspike = 3.2 V Δtspike~500 ns Pmax~37 mW Espike~ 18 nJ	Capacitor charging, Switching effect when reaching Vth, Vth(VO2)~5.6 V	Time to first spike
Oxide neuron [35]	Piecewise linear FitzHugh-Nagumo, FitzHugh–Rinzel Vanadium Dioxide (VO2), Niobium oxide (NbO)	a ~ 3 μm	Vspike~3.5V Δtspike~100 μs Pmax~72 mW Espike~ 7 μJ	Capacitor charging and energy of inductance magnetic field, switching effect when reaching Vth, Vth(VO2)~ 5.6 V Vth(NbO2)~ 0.9 V	-
Stochastic VO2 neuron [33]	Integrate and fire Vanadium Dioxide (VO2)	a ~ 100 nm	Vspike~0.5 V Δtspike~4 μs Pmax~12 μW Espike~50 pJ	Capacitor charging, switching effect when reaching Vth, Vth(VO2)~ 1.7 V	Rate coding
CMOS neuron [62]	Leaky Integrate and fire CMOS	a ~ 90nm Sneuron= 442 μm2	Vspike = 0.6 V Δtspike~3 ms Espike = 0.4 pJ	Capacitor charging. Reset using comparator, Vth~ 0.6 V	-
CMOS neuron [63]	Simplified Morris - Lecar model CMOS	a ~ 65 nm Sneuron = 35 μm2	Vspike = 112 mV Δtspike~18 μs Espike = 4 fJ	Capacitor charging and discharging through transistors, Vth~ 112mV	-

are divided into two groups: neurons based on silicon (CMOS) technology and neurons based on VO₂ switches. The main parameters of the neurons are the size of the active element and the energy consumption. Although the silicon neurons have an advantage in these parameters, in our study, the spikes duration is of the least importance. Another advantage of the VO₂ neuron compared to the CMOS neuron is, apparently, the high noise level of the current channel that leads to the stochastic behavior of the neuron described in References [33,34]. This property allows the network to escape local minima and reach the global minimum of the error surface.

The spike amplitude, power, and energy consumption of the VO₂ neuron depend on the threshold switching voltage V_th, if only the energy release on the VO₂ switch is taken into account. The main technological parameters affecting the V_th value are the resistivity in the insulator phase ρ_off and the contact geometry [25,32]. In Reference [25], we obtained an equation for approximating V_th.

$$V_{\mathrm{th}}=\frac{\sqrt{\lambda \cdot {\rho }_{off}\cdot (T_{\mathrm{th}}-T_0)}}{\sqrt{d}}\cdot a^{\beta }$$

(1)

where d is the thickness of the VO₂ film, a is the inter-electrode distance, λ is the heat-transfer coefficient, and β is the exponential coefficient that determines the effective area of the heated zone in the inter-electrode gap (β < 1).

By decreasing the value of a, the structures with reduced V_th can be obtained. Using Equation (1) and the current value when the structure is turned on, estimated as I_on = V_th/R_on (where R_on = ρ_on/d), we can propose an equation for the maximum power per spike (λ = 35 W/m ∙ K, ρ_off = 4 ∙ 10⁻² Ω ∙ m, ρ_on = 4 ∙ 10⁻⁴ Ω ∙ m, β = 0.56, T_th = 340 K, T₀ = 300 K [25]):

$$P_{\max }=V_{\mathrm{th}}\cdot I_{\mathrm{on}}=\lambda \cdot \frac{{\rho }_{\mathrm{off}}}{{\rho }_{\mathrm{on}}}\cdot (T_{\mathrm{th}}-T_0)\cdot a^{2\beta }$$

(2)

By reducing the size of the inter-electrode distance a and the ratio (ρ_off/ρ_on), we would significantly reduce the value of P_max. For example, at a = 1 µm, the maximum power P_max is ~ 26 mW, while at a = 100 nm, the maximum power P_max drops to 2 mW.

E_spike can be estimated by multiplying the spike duration by the maximum power, E_spike~ P_max⋅Δt_spike. In Reference [25], we modelled the switch on and switch off durations of switches, which determine the minimum Δt_spike values. We demonstrated that the durations decrease with decreasing a. Therefore, it is possible to predict a significant decrease in E_spike with a decrease in the size of switching elements. The estimates are the following: E_spike ~ 6.4 nJ at a = 1 μm (Δt_spike = 240 ns), and E_spike ~ 105 pJ at a = 100 nm (Δt_spike = 52 ns).

The last column in Table 2 demonstrates that the majority of the previous studies gives only the model of VO₂ neuron itself. In the current study, we present the simple neural network that is capable of pattern recognition, using the timing method of information coding, which has a clear energy advantage over the firing rates coding method [33].

5. Conclusions

In the current study, we present the new model of an LIF neuron based on one switching VO₂ element. The neuron circuit was modeled in the LTspice program, and, for the component emulating the switch, a voltage-controlled key was used, which the I – V characteristic corresponded to experimental data. During the simulation, the VO₂-neuron model demonstrates biosimilar properties, such as spike latency, subthreshold oscillations, refractory period, threshold behavior, and spike frequency adaptation. A two-layer SNN was designed to allow pattern recognition. The coupling between the neurons of the input and output layers was implemented using excitatory connections, and, inside the output layer, the coupling used inhibitory connections. This architecture led to the activation of only one output neuron associated with the most similar pattern, according to the WTA principle. As an example, we studied the network that had nine input and threeoutput neurons, which was trained to recognize three patterns (3 × 3 pixels). A timing method of information coding was used, where the color intensity of the pixel was determined by the time delay between the spikes. The training was conducted using the supervised SPDT method, taking into account the time delay of pre-synaptic and post-synaptic spikes. To analyze the operation of the trained network, the images of distorted patterns from the training set were sent to the network input, and the images were correctly recognized in most cases. The network is capable of recognizing up to 10⁵ images per second, and the classification process is highly dependent on the time parameters of the network and the effect of electrical switching. Network architecture has the potential for further scaling, which increases the speed of recognition and miniaturization of the components. In the future, we plan to continue the work toward optimization of both the neuron circuit and the network architecture for classifying images from standardized databases [64].