Implementation of Physical Reservoir Computing in a TaO_x/FTO-Based Memristor Device

Abstract

As one of the solutions to overcome the current problems of computing systems, a resistive switching device, the TiN/TaO_x/fluorine-doped tin oxide (FTO) stacked device, was fabricated to investigate its capability in embodying neuromorphic computing. The device showed good uniformity during the resistive switching phenomenon under time and cycle-to-cycle dependent switching, which may be due to the oxygen reservoir characteristics of the FTO bottom electrode, storing oxygen ions during resistive switching and enhancing the device property. Based on the uniform switching phenomenon of the TiN/TaO_x/FTO device, the pulse applications were performed to seek its ability to mimic the biological brain. It was revealed that the volatile and non-volatile nature of the device can be altered by controlling the pulse stimuli, where strong stimuli result in long-term memory while weak stimuli result in short-term memory. To further investigate the key functions of the biological brain, various learning rules such as paired-pulse facilitation, excitatory postsynaptic current, potentiation and depression, spike-rate dependent plasticity, and spike-time dependent plasticity were tested, with reservoir computing implemented based on the volatile characteristic of the TiN/TaO_x/FTO device.

1. Introduction

The current computing system, based on the von Neumann architecture, is encountering limitations due to the bottleneck between the processor and the memory [1,2]. Additionally, the increasing volume of data in modern computer processing has led to the emergence of various alternative architectures. One such architecture, neuromorphic computing, which seeks to emulate the biological brain, is gaining attention due to its energy-efficient parallel data processing capabilities [3,4]. In the pursuit of applications for neuromorphic systems, researchers have explored memristors, including magnetic random-access memory, resistive random-access memory (RRAM), ferroelectric random-access memory, and phase-change random-access memory, to assess their potential for mimicking synaptic behavior [5,6,7,8,9,10]. Among these devices, RRAM stands out as a promising next-generation nonvolatile memory for neuromorphic applications due to its straightforward fabrication process, high scalability, low-power operation, and rapid switching speed [11,12,13,14,15,16,17]. RRAM is a two-terminal device with a metal–insulator–metal structure, where the insulator typically consists of transition metal oxides such as NiO_x, HfO_x, AlO_x, ZrO_x, TiO_x, and TaO_x [18,19,20,21,22,23]. TaO_x-based RRAM devices, in particular, have exhibited superior characteristics in previous studies, including high endurance (>10¹⁰ cycles), sub-nanosecond switching speeds (<1 ns), a high dielectric constant ($ϵ$ = 25), and compatibility with current complementary metal-oxide semiconductor processes [24,25,26,27,28,29,30]. Additionally, the low absolute value of Gibbs free energy for the redox reaction of TaO_x contributes to its good stability [31].

The resistive switching phenomenon in transition metal oxide-based RRAM devices occurs due to the formation and rupture of a conducting filament [32]. When a voltage is applied to the top electrode, reduction takes place under an electric field, causing the separation of oxygen vacancies (V_o⁺) and oxygen ions (O²⁻). The accumulation of oxygen vacancies forms a conducting filament, leading to a significant current flow and a switch from a high-resistance state to a low-resistance state. Conversely, when an opposite bias is applied, oxidation occurs, reuniting the separated oxygen vacancies and ions, causing the filament to rupture, and the device returns to the high-resistance state. Recent studies have indicated that an oxygen reservoir layer, forming at the interface between the transition metal oxide and the metal electrode, can enhance switching properties through its oxygen storage capacity [33,34,35,36,37]. FTO is one of the electrodes acting as an oxygen reservoir, offering the advantages of affordability [38]. Furthermore, research has highlighted the additional benefits of the FTO electrode, including high thermal stability and processability with lower sheet resistance, making it highly suitable for scalable device fabrication and electrode applications [38,39].

To achieve neuromorphic computing, it is crucial that the device emulates the biological brain. The function of the human brain relies on the strength of connections between post- and pre-synapses, where chemical reactions facilitate information exchange [40,41]. Moreover, human memory can be categorized into short-term memory and long-term memory. Short-term memory exhibits synaptic plasticity, lasting from milliseconds to minutes, while long-term memory endures for hours or even days, with transitions occurring from short-term memory to long-term memory through repetitive activity [42,43]. Synaptic behaviors characteristic of long-term memory and short-term memory can be replicated through pulse applications, such as potentiation and depression, excitatory postsynaptic current (EPSC), paired-pulse facilitation (PPF), and more [44,45,46,47]. Furthermore, studies have shown that certain high-order synaptic plasticity states can be monitored using Hebbian learning rules [48,49], including spike-time dependent plasticity (STDP) and spike-rate dependent plasticity (SRDP), adding complexity to synapse emulation [50,51].

Additionally, due to the volatile nature of some memristor devices, reservoir computing is often implemented [52,53,54,55,56]. Reservoir computing is a computational framework suitable for processing temporal and sequential data with low power consumption [54]. Two essential factors for implementing reservoir computing are nonlinearity and volatile characteristics [57]. As the reservoir system’s high-dimensional output depends on time-dependent input without relying on previous inputs, non-linear data conversion within the reservoir layer plays a critical role [58,59].

In this paper, we present the bipolar resistive switching characteristics of the TiN/TaO_x/FTO device. The FTO electrode functions as an oxygen reservoir, resulting in good uniformity over 300 DC cycles and 10⁴ s, attributed to its oxygen ion storage capability. Moreover, by applying pulse train-induced potentiation and depression, we tested the Modified National Institute of Standards and Technology database image recognition system, achieving favorable recognition accuracy through training and testing. We also emulated biological synapses using learning tools such as PPF, EPSC, STDP, and SRDP. Finally, we explored additional applications in reservoir computing systems, revealing various potential uses for the TiN/TaO_x/FTO memristive device.

2. Materials and Methods

The TiN/TaO_x/FTO device was fabricated as follows. Before fabrication, a commercially available 2.2 mm thick FTO electrode on a transparent glass substrate was prepared for its use as a bottom electrode. The surface of the FTO electrode was cleaned using acetone and isopropyl alcohol. Then, a 10 nm thick TaO_x film was deposited on the FTO electrode using radio frequency reactive sputtering using a Ta target (99.99% purity). The sputtering gas was a mixture of Ar and O₂, at 20 and 6 sccm each. Additionally, the power and the pressure of the fabrication process were 150 W and 5 mTorr. Then, by depositing the TiN top electrode on a circular shadow mask with a diameter of 100 µm, the top electrode TiN was acquired. The TiN top electrode was deposited using DC reactive sputter, with a gas mixture of Ar (19 sccm) and N₂ (1 sccm). The pressure and power were 5 mTorr and 350 W. The electrical properties of TiN/TaO_x/FTO were investigated through a Keithley 4200-SCS semiconductor parameter analyzer (Keithley Instruments, Cleveland, OH, USA) and a 4225-PMU pulse measuring unit (Keithley Instruments, Cleveland, OH, USA). The top TiN electrode was biased while grounding the bottom FTO electrode.

3. Results and Discussion

Figure 1a shows the fabrication process flow of the TiN/TaO_x/FTO device. Figure 1b illustrates the schematic representation of the TiN/TaO_x/FTO device fabricated on a glass substrate. The electrical behavior of TiN/TaO_x/FTO is elucidated in Figure 1c–f. Before achieving the desired switching, the device undergoes an initial breakdown process termed “forming,” which facilitates the transition from its initial resistance state to the low-resistance state [60]. As depicted in Figure 1c, the forming process occurs at a voltage of −3 V, and notably it does not require any current compliance. This behavior might be attributed to the FTO electrode, as previous research has indicated compliance-free switching in FTO-based RRAM devices [61,62,63]. Following the forming process, the devices exhibit a bipolar resistive switching phenomenon at voltages of −3 V and 3 V. Furthermore, Figure 1d presents the current–voltage (I-V) curves resulting from 100 DC cycle switches in the TiN/TaO_x/FTO device. It is evident that the device maintains its self-compliance feature without undergoing a harsh breakdown process, exhibiting good uniformity across cycles. In Figure 1e, we present endurance data, where resistance values are recorded at −0.1 V. Additionally, Figure 1f showcases retention data, with resistance values also measured at −0.1 V. These data illustrate that the device sustains its low-resistance state and high-resistance states for 300 cycles and 10⁴ s, respectively, demonstrating favorable device characteristics and potential relevance in the context of neuromorphic computing emulation.

Building upon the findings presented above and drawing from the conduction mechanisms established in prior investigations of TaO_x-based resistive switching [64,65,66,67], we present a conduction model centered on oxygen ion migration, as depicted in Figure 2. When a negative bias is applied to the TiN top electrode, reduction processes unfold, as described in the following Equation (1) [65,68]:

$$O\stackrel{Reduction}{\to }{V}_O^{2+}+O^{2-}$$

(1)

Under the influence of the electric field, the oxygen ions (O²⁻) and oxygen vacancies (V_o²⁺) undergo separation, with oxygen ions being repelled away from the top electrode. This repulsion results in the creation of defects, specifically oxygen ions, within the material. These defects accumulate steadily under the continued application of electrical stress (−3 V), culminating in the formation of a conductive filament that establishes a connection between the bottom and top electrodes. This conductive filament facilitates a substantial current flow, thereby reducing the device’s resistance. Consequently, the resistance state of the device transitions from a high-resistance state to a low-resistance state, as illustrated in Figure 2a. Conversely, when a positive bias is applied to the TiN electrode, it induces the attraction of oxygen ions back towards the top electrode. During this migration, an oxidation process takes place, as shown in Equation (2):

$$O\stackrel{Oxidation}{\leftarrow }{V}_O^{2+}+O^{2-}$$

(2)

In this process, oxygen ions combine with electrons to form oxygen molecules. As a consequence, the previously separated oxygen ions and vacancies recombine. This recombination event leads to the rupture of the conductive filament, causing the device to revert to its high-resistance state, as depicted in Figure 2b.

In our exploration of the potential applications of the TiN/TaO_x/FTO device, we employed pulse schemes to examine its conductance response. One of the widely recognized techniques for assessing the current behavior of this device involves conducting experiments related to potentiation and depression [69]. The pulse scheme used to acquire this behavior is illustrated in Figure 3a. As depicted, we applied a sequence of 50 consecutive set and reset pulses to the TiN/TaO_x/FTO device in order to observe a gradual change in conductance. The pulse parameters for both set and reset operations included an amplitude of −2.5 V and a width of 50 µs for set pulses, and 2.7 V and 70 µs for reset pulses. After each set and reset pulse, we followed up with read pulses of −0.1 V and 50 µs to assess the conductance change. The results of this pulse application are presented in Figure 3b. Furthermore, during the pulse applications, the device exhibited short-term memory characteristics, as depicted in Figure 3c. When a single pulse of −2.5 V and 500 µs was applied to the device, followed by a 5 ms read pulse, the current value experienced a decrease after a certain period. This decline in current can be interpreted as a process akin to “forgetting” in the human brain, mirroring the tendency of short-term memory in the biological brain to fade over time [70]. Moreover, it is worth noting that the transformation of short-term memory into long-term memory is attainable by regulating the number of input pulses [71]. In Figure 3d, a set pulse sequence of −2.5 V and 500 µs was iterated four times, emulating the rehearsal process observed in the biological brain. Notably, after the fourth set pulse, a diminished current decay was evident, signifying the transformation of short-term memory into long-term memory.

To test further neuromorphic applications, the modified National Institute of Standards and Technology Database pattern recognition system based on deep neural network was tested using Python in Google Collab. For the training and testing process, 28 × 28-pixel handwritten number images were converted from the potentiation and depression data of Figure 3b. In this context, the pixel values increase or decrease compared to the preceding image, following the pattern of potentiation or depression, respectively. Therefore, when asymmetric graphs for potentiation and depression are obtained, training is carried out with uncleared images. Conversely, the more symmetric the potentiation and depression graphs, the clearer the images, and consequently higher accuracies can be achieved. The structure of the deep neural network consists of three layers: the input, hidden, and output layers, as shown in Figure 4a. The calculated recognition accuracy is illustrated in Figure 4b, where after 10 epochs a maximum accuracy of 94.84% was obtained.

PPF is a function utilized to observe short-term memory characteristics in synaptic devices. As illustrated in Figure 5a, twin paired pulses were administered to the device with varying intervals between them. When the interval time was brief, the device retained the memory of the preceding current response, resulting in a more substantial current output upon the second pulse. Conversely, if the interval time was sufficiently long, the device tended to forget the previous response, leading to a similar current response. To assess PPF, twin pulses of −2.5 V and 1 ms were applied, while the interval time ranged from 10 µs to 20 ms, enabling the observation of current responses. The outcomes of PPF are presented in Figure 5b. The PPF index, defined by Equation (3), quantifies the relationship:

$$PPF Index\left(\%\right)=\frac{I_2}{I_1}\times 100$$

(3)

where I₁ and I₂ represent the current responses after the first and second pulses, respectively. Additionally, EPSC was explored to investigate the device’s amplitude and interval-dependent characteristics. To obtain EPSC, pulses of varying quantities, ranging from 1 to 50, were administered to the TiN/TaO_x/FTO device, as illustrated in Figure 5c. Read pulses of −0.1 V and 1 ms followed the variable set pulses to monitor current changes. The amplitude-dependent EPSC response is displayed in Figure 5d, demonstrating that larger amplitudes result in more significant EPSC responses. The interval-dependent EPSC response is depicted in Figure 5e, revealing that a more substantial EPSC response is achieved with shorter pulse intervals. These EPSC response findings indicate that the devices exhibit favorable input-current responses, making them capable of facile output modulations.

Moreover, we conducted tests involving Hebbian learning rules, one of which is STDP. STDP explores the relationship between synaptic weight (ΔW) and spike timing (Δt). Thanks to the straightforward two-terminal structure of RRAM, it offers the advantage of easily emulating the architectural principles of biological synapses, where the pre-and post-synaptic elements are mimicked by the top and bottom electrodes, as depicted in Figure 6a [72]. In this configuration, synaptic information migrates between the pre- and post-synapse modulated by the synaptic weight, akin to the growth of the conductive filament within the insulating layer. This filament connects the top and bottom electrodes, facilitating the flow of current. In STDP, two primary conditions exist. When the pre-synapse precedes the post-synapse in spike timing (Δt > 0), synaptic weight is enhanced, resulting in potentiation. Conversely, when the post-synapse precedes the pre-synapse in spike timing (Δt < 0), the synaptic weight decreases, leading to depression [65]. The relationships between synaptic weight (ΔW) and spike time (Δt) are formulated in Equations (4) and (5):

$$\mathrm{\Delta }t=t_{pre}-t_{pose}$$

(4)

$$\mathrm{\Delta }W \left(\%\right)=\frac{G_f-G_i}{G_i}\times 100$$

(5)

where t_pre, t_post, G_f, and G_i represent the spike firing time pre-synapse, the spike firing time post-synapse, conductance after pulse application, and conductance before pulse application, respectively. To emulate spike firing, a pulse train comprising voltage levels of 1.5 V, −2.5 V, −2.3 V, −2.1 V, −1.9 V, and −1.7 V was applied at different time points, as indicated in the inset of Figure 6b. The results are depicted in Figure 6b, showcasing a gradual change in conductance reminiscent of long-term potentiation (LTP) and long-term depression (LTD). Furthermore, we also conducted experiments involving another Hebbian learning rule known as SRDP. In the context of SRDP, we applied a series of 10 consecutive set pulses, each with a voltage of −2.5 V and a duration of 1 ms, to the TiN/TaO_x/FTO device. These pulses were administered with varying pulse intervals between each set, spanning five different conditions: 1 µs, 10 µs, 100 µs, 1 ms, and 10 ms. As illustrated in Figure 6c, the results demonstrate a favorable SRDP response. The SRDP index is mathematically expressed in Equation (6):

$$SRDP Index \left(\%\right)=\frac{I_n}{I_i}\times 100$$

(6)

where I_n and I_i represent the current response of the initial state and after pulse applications, respectively.

Finally, based on the coexistence of short-term memory and long-term memory of the TiN/TaO_x/FTO device, reservoir computing was implemented. Reservoir computing is a computational paradigm designed for the processing of temporal and sequential data [52]. By implementing physical reservoir computing, complex tasks can be performed, including time-series forecasting, pattern recognition, and pattern generation [73,74,75]. Figure 7a represents the reservoir computing system, which consists of three layers: the input, reservoir, and output layers. The input layer accepts and encodes the incoming time-dependent data into the resistance values of the RRAM cells within the reservoir. Then, the reservoir layer functions as a dynamic and recurrent computational medium, converting input signals into high-dimensional output through nonlinear transformation and information mixing. Lastly, the output layer generates the desired output or prediction based on the information of the reservoir layer. A computational system designed to encode 4-bit data by applying time-dependent pulse train inputs to five devices is illustrated in Figure 7b. When an electrical pulse is applied as input to the different devices, it induces a change in the device’s current state. Hence, due to the short-term memory characteristics of the device, for the next input, the current state of the device will return to its former state, preventing the device from being influenced by previous inputs. The change in current states can be figured out through read pulses following the inputs, enabling its representation of 4-bit data. Figure 7c represents the [1010] state of the device, which is one of sixteen states ranging from [0000] to [1111]. The “1” state of reservoir computing was obtained by applying a set pulse of −2.5 V, 1 ms to the device, with a read pulse of −0.1 V following. For the “0” state, only read pulses were applied. The interval between each following state was modulated as 20 ms, to ensure that the former input had minimal impact on the prior output. Thus, by applying “1”, “0”, “1”, and “0” pulse sequentially with appropriate intervals, state [1010] was obtained. Using “1” and “0” states, 16 different states of reservoir computing are shown in Figure 7d. The “1” and “0” can be distinguished through the current value between 1.82 and 2.59 µA. Furthermore, the letter “K” can be made by combining 4-bit data from five different devices. The black pixel represents “1”, while the white pixel represents “0”. As a result, by merging [1001], [1010], and [1100], the letter “K” consisted of 5 × 4 pixels, as depicted in Figure 7e.

4. Conclusions

In summary, we fabricated a TiN/TaO_x/FTO device and investigated its electrical characteristics for neuromorphic applications. The device exhibited bipolar switching for 300 cycles with uniform resistance states. The self-compliance behavior during cycle-repetition may be due to the FTO electrode, which is helpful in avoiding permanent device breakdown. Furthermore, during the pulse applications, the coexistence of short-term memory and long-term memory was observed through current decay and rehearsal. Due to this coexistence, various functions such as potentiation and depression, PPF, EPSC, STDP, and SRDP were mimicked, showing favorable characteristics in emulating biological synapses. Lastly, based on the short-term memory behavior of the device, reservoir computing was implemented. We believe that these results of various implementations of TiN/TaO_x/FTO devices depict various applications for the future neuromorphic system.