Memristors for the Curious Outsiders

Abstract

We present both an overview and a perspective of recent experimental advances and proposed new approaches to performing computation using memristors. A memristor is a 2-terminal passive component with a dynamic resistance depending on an internal parameter. We provide an brief historical introduction, as well as an overview over the physical mechanism that lead to memristive behavior. This review is meant to guide nonpractitioners in the field of memristive circuits and their connection to machine learning and neural computation.

1. Introduction

The present review aims at providing a structured view over the many areas in which memristor technology is becoming popular. As in many other hyped topics, there is a risk that most of the activity we see today will dissipate into smoke in the coming years. Hence, we have carefully selected a set of topics for which we have experience and we believe will remain relevant when memristors move out of the spotlight. After a general overview on memristors, we provide a historical overview of the topic. Next, we present an intuitive and a mathematical view on the topic, which we trust is needed to understand why anybody would consider alternative forms of computation. Thus, experts in the fields might find this article slow-paced.

The memristor was introduced as a device that “behaves somewhat like a nonlinear resistor with memory” [1], the resistance of the component depends on the history of the applied inputs: voltage or current. Different curves of the applied input elicit a different dynamic response and final resistance of the memristor. In addition, if we remove the input after certain time, leave the component alone, and come back to use it, the device will resume its operation from a resistance very similar to the one in which we left it; that is, they act as non-volatile memories. Furthermore, memristors also react to the direction of the current, i.e., they have polarity. This polarity is shown explicitly on the symbol of the memristor used in electronic diagrams: a square signal line (as opposed to a zigzag line for the resistor), inside an asymmetrically filled rectangle, as in Figure 1.

The interplay between the response behavior and the non-volatility of the device defines its usability either as a storage device or for more involved purposes such as neuromorphic computing. The present article gravitates around the fact that memristors are electronic components which behave similarly to human neuronal cells, and are an alternative building block for neuromorphic chips. Memristors, unlike other proposed components with neural behavior, can perform computation without requiring complementary metal–oxide–semiconductor (CMOS) if not for readout reasons.

After the experimental realization of a memristor by Strukov et al., which brought them to their current popularity, Leon Chua wrote an article titled “If it’s pinched it’s a memristor” [2]. The title refers to the fact that the voltage drop across the device shows a hysteresis loop pinched to the zero of the voltage–current axes when controlled with a sinusoidal voltage or current. The claim by Chua is that a device which satisfies this property must necessarily be a memristor. Chua also proved that such a device cannot be obtained from the combination of nonlinear capacitors, inductors and resistors [1,2,3]. It has been shown, however, that this property is a modeling deficiency for redox-based resistive switches [4].

Resistive switching was of interest even before the 2008 article. For instance, the review of of Waser and Aono shows that Titanium Dioxide had been in the radar for non-volatile memories for decades. Nevertheless, one can identify a clear explosion of interest after the 2008 publication. Next, we give a list of established and new companies developing memristors technology using a variety of compounds. The list is given without a particular order as these companies work on various types of compounds and type of memristors (for instance both resistive random-access memory (ReRam) and phase-change material (PCM)). Specifically, we are aware of: HP, SK Hynix, IBM, IMEC, Fujitsu, Samsung, SMIC, Sharp, TSMC, NEC, Panasonic, Macromix, Crossbar Inc., Qimonda, Ovonyx, Samsung, Intel, KnowM, 4DS Memories Ltd., Global Foundaries, Western Digital (before called SanDisk), Toshiba, Macronix, Nanya, NEC, Rambus, ST Microelectronics, Winbond, Adesto Technologies Corporation, HRL Laboratories LLC, and Elpida.

Going into the many physical mechanisms that make a memristive device work does require a deep knowledge of material properties. For example, the hysteretic behavior can be caused by Joule heating, as shown in an example taken from macroscopic granular materials [5]. In addition, hysteresis is a phenomenon that is common in nanoscale devices and it can be derived using Kubo response theory [6]. Kubo response theory is used to calculate the correction to the resistance induced by a time dependent perturbation: this is the formal framework to address resistive switching due to either electrical, thermal or mechanical stresses in the material. We can classify physical mechanisms that lead to memristive behavior in electronic components in four main types:

• Structural changes in the material (PCM like): in these materials, the current or the applied voltage triggers a phase transition between two different resistive states;
• Resistance changes due to thermal or electric excitation of electrons in the conduction bands (anionic): in these devices, the resistive switching is due to either thermally or electrically induced hopping of the charge carriers in the conducting band. For instance, in Mott memristors, the resistive switching is due to the quantum phenomenon known as Mott insulating-conducting transition in metals, which changes the density of free electrons in the material.
• Electrochemical filament growth mechanism: in these materials, the applied voltage induces filament growth from the anode to the cathode of the device, thus reducing or increasing the resistance;
• Spin-torque: the quantum phenomenon of resistance change induced via the giant magnetoresistance switching due to a change in alignment of the spins at the interface between two differently polarized magnetic materials.

These mechanisms above are truly different in nature, and whilst not the only ones considered in the literature, are the most common ones. We provide a technical introduction to these mechanisms in Appendix A for completeness.

The primary application of memristors, as we will show in this article, is towards neuromorphic computing. The word neuromorphic was coined by Carver Mead [7] to describe analog circuits which can mimic the behavior of biological neurons. In the past several years, the field has experienced an explosive development in terms of manufacturing neuromorphic novel chip architectures which can reproduce the behavior of certain parts of the brain circuitry. Among the components used in neuromorphic circuits, we consider memristors whose behavior resembles the one of a certain type of neurons. The analogies between biological neuronal systems and electronic circuits are manifold: conservation of charge, thresholding behavior, and integration to mention just a few. For instance, diffusion of calcium across the membrane can be mapped to diffusion in electronic components, and a computational role associated to the circuit design. In addition, it is known that the brain is power efficient: roughly twenty percent of an individual’s energy is spent on brain activity, and this is roughly around 10 W. Besides their role in biological neural models (Section 6), we also discuss theoretical approaches to memristive circuits and their connection to machine learning (Section 4 and Section 5).

The connection to machine learning is complemented with an overview on analog computation (Section 5.1). Historically, the very first (known) computer was analog. It is (approximately) 2100 years old, and it has been found in a shipwreck off the cost of the island of Antikythera at the beginning of the past century [8] (only recently, it has been understood as a model of planet motion). Despite our roots in analog computation, our era is dominated by digital computers. Digital computers have been extremely useful at performing several important tasks in computation. We foresee that future computers will likely be a combination of analog (or quantum analog) and digital (or quantum digital) computing chips. At the classical level, several analog computing systems have been proposed. Insofar, most of the proposed architectures are based on biological systems, whose integration with CMOS can be challenging, and this is the reason why analog electronic components are seen as promising alternatives [9,10].

Digital computation has been dominated by the von Neumann architecture in combination with CMOS technology. Within this architecture, we find two types of memory: Random Access Memory (RAM), a volatile and quick memory in which computation is performed, and non-volatile Hard Disk (HD) for long-term storage of information. The neat separation between memory (RAM and HD) and computing (the processors or central processing unit (CPU)) is a key feature of computation using von Neumann architecture. It requires that the data is split into packets, transferred to the CPU where computation is performed, and then repeated until the full computation task has been completed. As far as our understanding goes, this separation is not present in the brain, in which memory and processing happen within the same units. Several proposed architectures that mimic this property have been proposed, most of them based on memristors [11]. This type of computation is referred to as memcomputing.

This article is organized as follows: We first provide a historical introduction to memristors, as it is understood by the authors (Section 2). We then provide the key ideas behind the technology with simple mathematical models (Section 3). Albeit separated from the main text, we have provided an introduction to the main technology and physical principles underlying memristors in Appendix A. We then focus our gaze on the description and use of memristors both for data storage (Section 4) and data processing (Section 5): the former is the current target for marketing the technology, the second is believed to be the main application of memristors in the long run. The similarity between memristors and neurons allows the implementation of machine learning on chip via Memristor/FPGA (field-programmable gate array) interfaces using crossbar arrays, we cover this topic in Section 4.1. We have dedicated Section 5.1 to overview the fundamental topic of analog computation, followed by a brief recapitulation of machine learning techniques that can be seamlessly used with memristors (Section 5.2–Section 5.3). We discuss in Section 6 computation as an epiphenomenon of memristor dynamics and in connection with CMOS. We close the article with some remarks that should help in the discussion of the topics covered.

2. Brief History of Memristors

The earliest known occurrence of a “memristive” behavior in circuit components is in the study, by Sir Humprey Davy’s, of arc (carbon) lamps [12], dated to the late 19th century. Another example, which was key to the discovery of the radio, is the coherer [13,14]. The coherer was invented by Branly [15] after the studies of Calzecchi-Onesti, and inspired Marconi’s radio receiver [16]. Branly’s coherer serves as perfect homemade memristor [5], as it simply requires either a (fine) metallic filling or some metallic beads, and it falls within the Physics discipline of electrical properties of granular media. Coherers are very sensitive to magnetic fields, and thus can act as a radio receiver and as we describe in Appendix A. They do posses a typical hysteric behavior which is associated with memristive components. At the simplest level of abstraction, a memristor is a very peculiar type of nonlinear resistance with an hysteresis (e.g., they possess an internal memory). Currently, the discussion is focused not only on the application of this technology to novel computation and memory storage, but also on the fundamental role of memristors in circuit theory [17]. While this discussion is important, we first discuss why memristive behavior is not uncommon in analog computation. We thus use the analog of hydraulic computers.

Hydraulic computers were built in Russia during the 1930s before valves and semiconductor were invented. The hydraulic computers designed by the Russian scientist Vladimir Lukyanov were used to solve differential equations; other models like the Monetary National Income Analogue Computer (or MONIAC) [18], would be later employed by the Bank of England to perform economic forecasts.

The idea behind these computers was to use hydraulics to solve differential equations. As Kirchhoff laws can be stated for any conservative field, we have that the pressure drop in a loop is equal to the actions of pumps in that loop. The conservation of mass is equivalent to conservation of charge, and to Kirchhoff’s second law: in the steady state, the mass of water entering a node must be equal to the mass flowing out. Ohm’s law is equivalent to Poiseuille’s law in the pipes: a porous material in the pipes, or a constriction, is equivalent to a resistance. A material that expands when wet, like a sponge, will increase the resistance to the flow as it absorbs the water. This is equivalent to a higher resistance which depends on the amount of water that flowed thought the system, i.e., memristor-like. The sponge, however, does not have a polarity, while memristors do, depending on the physical mechanism which induces the switching. Other memristor-like systems can be built with other mechanical analogs [19,20], plants and potato tubes [21] or slime moulds [22,23] just to name a few.

The modern history of memristors is tied to the work of Leon Chua. The first time the word memristor (as an abbreviation for memory resistor) appeared was in the now celebrated Chua 1971 article [1]. During the 1960s, Leon Chua worked extensively on the mathematical foundations of nonlinear circuits. When he moved to Berkeley, where he currently is a Professor, he had already won several awards such as the IEEE ( Institute of Electrical and Electronics Engineers) Kirchhoff Award. The definition of memristor was made clearer in a second paper with his student at the time, Sung Mo Kang [3]. The second work was an important generalization of the notion of memristor and is the one we used in the present paper. Chua and Sung Mo Kang introduced the notion of “memristive device”: a resistance which depends on a state variable (or variables), which is sufficient to describe the physical state (resistance) of the device at any time. The component defined by Chua, and then by Chua and Sung Mo Kang, is a resistance whose value depends on some internal parameter, which in turn has to evolve dynamically according either to current or voltage. Implicitly, one needs to also define the relationship between the resistance and the state variable, which characterizes the device resistance. In the analogy with hydraulic computers, the state variable represents by the density of holes in the sponge as a function of time, while the resistance is the amount of traversable area. The 1976 and the 1971 papers were mostly mathematical and formal, without any connection to the physical properties of a real device. The 1976 paper also introduced the fact that, if one controls the device with a sinusoidal voltage, then one should observe Lissajous figures in the current–voltage (I–V) diagram of the device. It also established that any electronic component that displayed a pinched hysteresis in the I–V diagram has to be a memristor. The eager reader will find more details on the devices in the rest of the paper.

For the 30 years after the work by Chua and Sung Mo Kang, the field of memristors was basically non-existent. During the mid 2000s, Strukov, Williams and collaborators, working at Hewlett-Packard Labs (Palo Alto, CA, USA) were studying oxide materials and resistive switching. The physics of resistive switching has been known since the early 1970s, with the introduction of Phase-Change Materials. The physical origin of resistive switching was well studied, albeit not fully understood [24,25,26]. The idea that memristors could be experimentally realized became popular with the article of Strukov et al. at HP Labs.

3. Mathematical Models of Memristors

In this section, we provide a general mathematical description of memristive systems followed by the details of the memristor model with linear memory dynamics proposed by Strukov et al. This is one of the simplest memristors models that captures the behaviors relevant for technological applications of memristive systems.

The fundamental passive electric components are the resistors, capacitors and inductors. These components couple the voltage difference v applied between their ports with the current i flowing through, with the following differential relationships:

$$\begin{array}{c}\begin{array}{cc}\hfill \mathrm{Resistor}:& \mathrm{d}v=R\mathrm{d}i;\hfill \\ \hfill \mathrm{Capacitor}:& \mathrm{d}q=C\mathrm{d}v,\mathrm{charge}:\mathrm{d}q=i\mathrm{d}t;\hfill \\ \hfill \mathrm{Inductor}:& \mathrm{d}\Phi =L\mathrm{d}v,\mathrm{flux}:\mathrm{d}\Phi =v\mathrm{d}t.\hfill \end{array}\end{array}$$

(1)

These relations introduce the resistance R, the capacitance C, and the inductance L. The ideal memristor was initially and abstractly formulated as the flux–charge relationship:

$$\mathrm{d}\Phi =M\mathrm{d}q,$$

(2)

which, in order to be invertible, must have M defined either in terms of $\Phi$, or q, or it must be a constant. The latter case coincides with the resistor, while if M is defined in terms of $\Phi$, then we have a voltage-controlled memristor, if it is defined in terms of q it is a current-controlled memristor. For the sake of simplicity we do not make a difference between current-controlled and charge-controlled; this is an abuse of the language. The units of the coupling M are Ohms, as can be seen by the units of the quantities it relates, which justifies the notion of memristor at least as a nonlinear resistor. However, according to Chua, only nonlinear resistors showing a pinched hysteresis loop, i.e., $V=0\to I=0$, are classified as memristors. An example is shown in Figure 2. Further classifications (ideal, extended, generic, see [2]) can be added to a device depending on other properties of the current–voltage (I–V) diagram that are observed experimentally when the device is controlled with a sinusoidal voltage at a certain frequency. For example, if for high frequencies the I–V becomes a straight line (recovery of pure linear resistance, no hysteresis), the device is a generic memristor, under Chua’s nomenclature. General memristors might be locally-active, showing regions where the resistance changes sign [27].

Many systems, not necessarily electronic ones, can fulfill these properties. Physical systems with memristor-like behavior are denominated memristive systems, and are described by the following non-autonomous (input-driven) dynamical system:

$$\begin{array}{c}\begin{array}{cc}\hfill \frac{\mathrm{d}x}{\mathrm{d}t}& =F(x,u),\hfill \\ \hfill y& =H(x,u)u,\hfill \end{array}\end{array}$$

(3)

where $x$ is a vector of internal states, y a measured scalar quantity and u a scalar magnitude controlling the system. The pair $(y,u)$ is usually voltage–current or current–voltage, hence the second equation is simply Ohm’s law for the voltage–current relationship. In the first case (current-controlled system), the function $H(x,u)$ is the called memristance, and it is called memductance in the second case (voltage-controlled). The function $H(x,u)$ is assumed to be continuous, bounded, and of constant sign. In order to rule out locally active devices, it is necessary to further require that $H(x,u)$ is monotonic in u. This leads to the zero-crossing property or pinched hysteresis: $u=0\Rightarrow y=0$. The states $x$ can be many physical states other than charge, e.g., the internal temperature of a granular material. The minimum number of internal states on which $H(x,u)$ depends is called the order of the memristive system.

The reader should not overlook the difference between an ideal memristor (a mathematical model), memristive systems (also a mathematical model), and a memristor (a physical device). Ideal memristors are models of devices in which the internal parameter controlling the resistance is the charge, which is linked to the flux. Memristive systems generalize this behavior, and the internal parameters can be other physical quantities, or combination of physical quantities, even if they do not include flux or charged particles. The latter is better attuned to many memristors, i.e., physical realizations of memristive behavior.

Among the (large) class of memristive systems, the model proposed by Strukov et al. is appealing due to its simplicity. This model, shown in Equations (4) and (5), captures the basic properties of memristors that are relevant for the understanding of the device and for its technological applications.

Equations (4) and (5) model the behavior of a current-controlled Titanium Dioxide memristor [28], also known as the HP-memristor:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}w\left(t\right)}{\mathrm{d}t}& =\alpha w\left(t\right)-\frac{1}{\beta }I\left(t\right),\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill \frac{V\left(t\right)}{I\left(t\right)}& =R\left(w\left(t\right)\right):=R_{\mathrm{on}}\left[1-w\left(t\right)\right]+R_{\mathrm{off}}w\left(t\right),\hfill \end{array}$$

(5)

$$\begin{array}{cc}\hfill w\left(0\right)& =w_0\to R\left(w_0\right)=R_0,0\le w\left(t\right)\le 1,\hfill \end{array}$$

(6)

where $I\left(t\right)$ is the current flowing in the device at time t. The quantities $R_{\mathrm{on}}\le R_{\mathrm{off}}$ in the parametrization of the resistance R in terms of the adimensional variable $w\left(t\right)$ (called memory), define the two limiting resistances. The parameters $\alpha$ (with units s${}^{-1}$) and $\beta$ (with units s${}^{-1}$A${}^{-1}$), define the dynamic properties of the memristor. The parameter $\alpha \ge 0$, sometimes called volatility, indicates how fast the resistance drifts towards $R_{\mathrm{off}}$ in the absence of currents [29]. Since the resistance function depends on a single state w, this is a first order memristive system. Although this model has several drawbacks when it comes to its connection to the physical behavior of ion migration in the conductor [6,17,19,30,31,32], it will be the reference for most derivations and discussions in this article, as it captures several key properties of a generic memristor.

We begin with the case with $\alpha =0$, i.e., a non-volatile memristor. We solve the system of equations above using a zero mean small amplitude driving voltage $V\left(t\right)$ which is such that, for all times $0<w\left(t\right)<1$, i.e., the memristor should not saturate at any time [33]:

$$R\left(w\left(t\right)\right)\frac{\mathrm{d}w\left(t\right)}{\mathrm{d}t}=\frac{\mathrm{d}}{\mathrm{d}t}\left((R_{\mathrm{off}}-R_{\mathrm{on}})\frac{1}{2}{w}^2\left(t\right)+R_{\mathrm{on}}w\left(t\right)\right)=-V\left(t\right),$$

(7)

from which we derive, if we define $\xi =\frac{R_{\mathrm{off}}-R_{\mathrm{on}}}{R_{\mathrm{on}}}$ and integrate over time:

$$\left(\frac{\xi }{2}{w}^2\left(t\right)+w\left(t\right)\right)=\left(\frac{\xi }{2}{w}^2\left(t_0\right)+w\left(t_0\right)\right)+{\int }_0^{t}V\left(\tau \right)\mathrm{d}\tau ,$$

(8)

whose solution is given by

$$w\left(t\right)=\frac{\sqrt{2\left(\frac{\xi }{2}{w}^2\left(t_0\right)+w\left(t_0\right)+{\int }_0^{t}V\left(\tau \right)\mathrm{d}\tau \right)\xi +1}-1}{\xi }.$$

(9)

This equation fulfills the fundamental property of a memristor: it has the zero crossing property (pinched hysteresis loop). Furthermore, it shows properties of generic memristors: the loop tilts to the right for driving signals with increasing frequencies, becoming a straight line for sufficiently high frequencies. To see the latter, consider $V\left(t\right)=V_{0}cos\left(\omega t\right)$, as the frequency $\omega \to \infty$. For this voltage, the integral in Equation (8) goes to 0, and $w\left(t\right)\to w\left(t_0\right)$. This implies a constant resistance, i.e., the memristor becomes a resistor for high frequencies.

In this model for $\alpha =0$, we have that $w\left(t\right)\sim q\left(t\right)$, where $q\left(t\right)$ is the charge in the conductor. For a titanium dioxide thin film, it was shown in [28] that

$${\beta }^{-1}\sim \frac{{\mu }_e{R}_{\mathrm{on}}}{D},$$

(10)

and thus

$$R\left(q\right)\approx R_{\mathrm{off}}\left(1-\frac{{\mu }_e{R}_{\mathrm{on}}}{D^2}q\right),$$

(11)

where ${\mu }_e$ is the electron mobility in the film, $R_{\mathrm{off}}$ is the resistance of the undoped material (for instance titanium oxides, ∼100 Ω), and D is a characteristic length of the film. From the micrometer to the nanometer, the value of $\beta$ grows by a factor of ${10}^6$, which is why the memristance in this material is a nanoscale effect.

Memristors are also referred to as resistive switches, because when $\frac{R_{\mathrm{off}}}{R_{\mathrm{on}}}\gg 1$ and $\beta$ is small enough, the memristor can be quickly driven from one state to the other via a current (or voltage) inversion. The operation is often referred to a $SET$ or $RESET$ in the technical literature, depending on the operation one is interested in, and it allows the use of non-volatile memristors as bits.

In the volatile case, i.e., $\alpha >0$, the memristor does not retain its memory state. When a volatile memristor is not activated via an external forcing, the memristor drifts to its insulating state, $R(t=\infty )=R_{\mathrm{off}}$. That is, if $I\left(t\right)=0$, then the internal memory is simply given by $w\left(t\right)=w_0{e}^{\alpha t}$, where $w_0$ is the initial condition. This behavior was first experimentally observed in [34]. Volatility is discussed again in Section 5.4.

Following the distinction between memristive and memristors we have discussed, when $\alpha \ne 0$, the parameter w cannot be uniquely identified with the charge, hence the model does not correspond to a bona fide memristor. In fact, for a single component the solution of:

$$\frac{\mathrm{d}w\left(t\right)}{\mathrm{d}t}=\alpha w\left(t\right)-\frac{1}{\beta }\frac{\mathrm{d}q\left(t\right)}{\mathrm{d}t}$$

(12)

is given by

$$w\left(t\right)=\frac{e^{\alpha t}}{\beta }{\int }_1^t{e}^{-\alpha \tau }\frac{d}{d\tau }q\left(\tau \right)d\tau +w_0{e}^{\alpha t},$$

(13)

which is a composite function that includes the charge.

The numerical model of the memristor allows us to study their theoretical capabilities as well as simulating large networks of these devices. From the point of view of numerical simulations, the dynamics at the boundaries of w need to be stable, or alternatively, w be constrained in $[0,1]$. In general, the latter does not scale well in terms of runtime, and for this reason it is often useful to bound the internal states of the memristor model via the introduction of window functions [28,35,36,37,38,39,40]. Since we are not aware of any systematic validations of windowing functions with real devices, we believe they should be considered tricks useful for large simulations. It is common to use windowing techniques based on polynomials; for an extensive review, we refer the reader to [38]. Polynomial windowing functions that reproduce nonlinear dopants drift can be introduced via the so-called Joglekar window function $F\left(x\right)$ [36,40], such that $F\left(1\right)=F\left(0\right)=0$, which generalizes the evolution of $w\left(t\right)$ as

$$\frac{\mathrm{d}w\left(t\right)}{\mathrm{d}t}=\alpha w\left(t\right)-F\left(w\right)\frac{1}{\beta }I\left(t\right),$$

(14)

where the window function can take the form $F\left(w\right)=1-{(2w-1)}^{2p}$, with p a positive integer. Depending on the type of memristor model, different window functions might provide a better approximation of the nonlinear effects near the boundaries of the memory. For an overview of the various physical mechanisms that can be involved, we refer the reader to Appendix A.

Numerical simulations of memristive networks can be achieved with different methods. Efficient ones include the SPICE methodology [41,42] and the more recent Flux-Charge method [43]. Furthermore, general solvers for Differential-algebraic system of equations (e.g., SUNDIALS solvers [44]) can also be used for networks with a few hundred components, without major loss of performance. For some networks (without capacitance or inductance), it is possible to derive a monolithic system of ordinary differential equations for the evolution of the network states, which already include the nonlocal algebraic constraints imposed by Kirchhoff’s laws [45,46]. The choice of the method is largely based on the questions to be addressed by the simulations, which often results in a trade-off between generality of the solver (e.g., flexible memristor model) and performance.

4. Memristors for Storage

A memristor can be used as a digital memory of at least one bit. The simplest way to achieve this is to use the memristor as a switch. If the memristor is non-volatile, we can set its memristance to the value $R_{\mathrm{on}}$ or $R_{\mathrm{off}}$ using a voltage or current pulse, and associate these resistances with the state of a bit. We stress that non-volatility is essential for memory applications because a volatile memristor, i.e., one that drifts back to a high resistance state autonomously, would require a permanent source of current/voltage to keep it in the low resistance state. Volatility will, in general, render memristive memory worthless in terms of energy consumption. This is one of the reasons why volatility is engineered out of the devices meant for storage applications.

In order to illustrate the mechanism of flipping a bit, consider again the non-volatile memristor model described by Equation (4) (i.e., $\alpha =0$) connected to a constant voltage source $V_{\mathrm{write}}$. Solving the differential equation for $w\left(t\right)$ gives:

$$w\left(t\right)=\sqrt{a+b{V}_{\mathrm{write}}t},$$

(15)

where the coefficients $a,b$ depend on the parameters, but not on the input voltage. Thus, by controlling the sign of the input voltage $V_{\mathrm{write}},$ we can switch the resistance from $R_{\mathrm{off}}$ to $R_{\mathrm{on}}$ and vice versa (the flip of a bit). The switching happens within a characteristic time $\tau$:

$$\sqrt{a+b{V}_{\mathrm{write}}\tau }=1\to \tau \le \frac{1}{b{V}_{\mathrm{write}}}.$$

(16)

Hence, to flip the bit, we need to apply $V_{\mathrm{write}}$ for at least this amount of time. To read the bit, we apply a voltage $V_{\mathrm{read}}\ll V_{\mathrm{write}}$ (and optionally for period of time shorter than $\tau$) and compute the resistance via the measurement of the resulting current.

Equation (15) applies only to the idealized memristor described by the model in Equations (4) and (5). This model might not be valid for real devices which will show a different dynamic response to input voltages or currents. However, the idea of controlling and reading a memristor bit with pulsed inputs remains the same. Figure 3 shows the response of a real TiO${}_2$ memristor to write voltages (SET and RESET).

The fact that it is possible to write and read the state via signal pulses allows for advantageous scaling of power consumption and bit density (see [48] for details). As we discuss below, it is possible to use crossbar arrays with memristors to increase the density of components. The density of components in crossbar arrays scales as $\frac{1}{4{\ell }^2}$, where ℓ is set by the length scale of optical lithography (a few nanometers). The reported state of the art as we write this article is roughly 7 GB/mm${}^2$, and with writing currents of 0.1 nA [49].

Another challenge for storage based applications, besides volatility, is device variability, e.g., the variation of properties like the switching time $\tau$ for devices built under similar conditions. Current research in oxides focuses on these variability aspects, how to standardize the production of memristors, and the optimization of properties like the switching and retention time, and the durability of each singular device. The status of the technology for memory storage for different type of devices is presented in

Table 1. Characteristics of various storage devices (for details see [50]). The table compares to memristor technology to competitors like Phase-Change Materials (PCM) and more standard devices based on Spin-Transfer Torque (STT), dynamic random-access memory (DRam), Flash and hard disk (HD).

	Memristor	PCM	STT-RAM	DRAM	Flash	HD
Chip area per bit ($F^2$)	4	10	14–64	6–8	4–8	n/a
Energy per bit (pJ)	0.1–3	${10}^{1–2}$	0.1–1	${10}^0$	${10}^3$	${10}^4$
Read time (ns)	<10	20–70	10–30	10–50	${10}^{4–5}$	${10}^4$
Write time (ns)	20–30	10${}^{1–2}$	${10}^{1–2}$	${10}^1$	${10}^5$	${10}^6$
Retention (years)	10	10	${10}^{-1}$	${10}^{-5}$	10	10
Cycles endurance	${10}^{12}$	${10}^{7–8}$	${10}^{15}$	${10}^{17}$	${10}^{5–8}$	${10}^{15}$
3D capability	yes	no	no	no	yes	n/a

.

4.1. Crossbar Arrays

In this section, we briefly review the crossbar array architecture used in memristor based storage and its application in artificial neural networks.

Crossbar arrays are based on the architecture depicted in Figure 4. The figure shows an array composed of horizontal (e-lines) and vertical (b-lines) lines that are initially electrically isolated from each other. A 2-terminal component, e.g., a memristor, is connected across each pair of vertical and horizontal lines.

In order to operate crossbar arrays, a voltage ${\xi }_j$ is applied to the j-th e-line, and an another voltage ${\eta }_i$ to the i-th b-line. A memristance $M_{j,i}$, placed across j-th e-line and the i-th b-line, is controlled by the voltage ${\xi }_j-{\eta }_i$. This arrangement allows for simple indexing of the memristances, and is the mechanism behind a Content-Addressable-Memory (CAM), which is used in crossbar arrays. The idea of this construction dates back to Steinbuch’s “Die Lernmatrix” (the learning matrix) [51,53].

Crossbar arrays can be used for matrix-vector multiplication using the voltages $\left\{{\xi }_j\right\}$ as inputs and the voltages $\left\{{\eta }_i\right\}$ as outputs. For a resistance independent of the input voltage or current, the relation between them is given by $\overrightarrow{\eta }=A\overrightarrow{\xi }$, where the elements of the matrix A are:

$$A_{ij}=\frac{M_{ij}^{-1}}{{\tilde{R}}_i^{-1}+{\sum }_{s=1}{M}_{is}^{-1}}.$$

(17)

${\tilde{R}}_i$ are the resistances on the output b-lines $\overrightarrow{\eta }$, and $M_{ij}$ is the memristance of the memristor between the i-th b-line and j-th e-line. An algorithm for setting the (mem)resistances given a matrix can be found in [54]. To apply this method with memristors, the voltages differences need to be small/short, to avoid changing their memristance, or all memristors need to be in one of their limiting states $R_{\mathrm{on}}$ or $R_{\mathrm{off}}$.

The component density of crossbar arrays can be increased by stacking various layers of crossbar arrays on top of each other [55,56]. The stacking of L crossbar layers scales the density of components by a factor L, i.e., a theoretical scaling of $\frac{L}{4{\ell }^2}$. In multilayered arrays, memristors are controlled using the corresponding horizontal and vertical lines of each layer.

Memristive crossbar arrays can be used to encode the synaptic weights of feed-forward or recurrent artificial neural networks (ANNs) [57]. In ANNs, the input to neurons in a given layer (post-synaptic) is computed as the multiplication of the outputs of neurons in the previous layer (pre-synaptic) and the matrix of synaptic weights connecting the two layers (Figure 5). Then, the multiplication is carried over using the multiplication algorithm previously described. The output of pre-synaptic neurons is encoded in the voltages of e-lines of the crossbar array, and the input to post-synaptic neurons is decoded from the currents on the b-lines. Applications go beyond this direct implementation of the multiplication algorithm. For example, in [57], the synaptic weights are encoded as the difference of the conductance between two memristors. Similar ideas have been exploited to design other computational models based on stateful logic [58] and differential pair synapses (called “kT-RAM”) [59].

The potential benefits of utilizing memristor based ANNs are speed and energy efficiency. The computation and storage use the same location in the network, and analog inputs are directly fed to the neurons. This architecture minimizes the reading-writing costs incurred by the conventional von Neumann architecture, and potentially the energy losses of analog-to-digital conversion.

4.2. Synaptic Plasticity

Synaptic plasticity can be broadly defined as the modification of the synaptic conductance as a function of the activity of the neurons connected to it. This definition rules out autonomous plasticity, which is the slow synaptic conductance decay of inactive synapses (volatility). Autonomous plasticity is fundamental for data processing applications and will be considered in Section 5.

Non-autonomous synaptic plasticity can be classified in several types, e.g., spike-rate-dependent, spike-timing-dependent, short-term, long-term, etc. In the study of synaptic plasticity of the human cortex, Hebbian or Anti-Hebbian (related to the simultaneous firing of two neurons) [52,60,61] are often the underpinning learning mechanisms. Our aim here is not to describe the types of plasticity, their biological origin, or how difficult it is to isolate each class in biological and experimental systems. Our intention is but to illustrate how memristors, used as memories, can be used to implement the change in weights based on pre- and/or post-synaptic activity.

For an overview of the different types of synaptic plasticity and references to further reading, we refer the reader to the book chapter by La Barbera and Alibart [62].

Synaptic plasticity is modeled by choosing the plasticity inducing variables $x\in X$ (e.g., relative arrival times of spikes, relative neural activity, etc.) and a mapping from these to the change of the synaptic weight

$$f_X:X\to W_{\Delta }\subset \mathbb{R}.$$

(18)

This mapping is based on biological models, or simplified adaptation mechanisms. For example in spike-timing-dependent Plasticity (STDP), the inducing variable is the relative timing of two or more activity spikes in the connected neurons. Plasticity is then defined with a mapping $f_{\mathrm{STDP}}:T^{n-1}\to W_{\Delta }$, taking the relative timing of n spikes $\in T^{n-1}$ (typically 2 or 3) to a weight change $\in W_{\Delta }$ (usually represented as relative change). In spike-rate-dependent Plasticity, we replace the timing domain with a relative rate domain.

In order to implement plasticity in memristors as synaptic weights, we need a writing voltage that represents the synaptic change. That is, we need a further mapping

$$f_V:W_{\Delta }\to V,$$

(19)

where V is the set of valid writing voltages. The composed mapping $f_V\circ f_X:X\to V$ gives the final implementation of synaptic plasticity. The mapping $f_V$ depends on all the characteristics of the technology used, e.g., the physical mechanisms of memristance (see Appendix A), the neural network architecture (e.g., crossbar array), the controlling electronics, etc. Obtaining the function $f_V$ in the mapping above is the main challenge in synaptic plasticity applications, and thus requires considerable effort. A survey of complete and partial implementations of synaptic plasticity in nanoscale devices is summarized in [62] (Section 4.1). For example, Reference [63] uses STDP to implement unsupervised learning with resistive synapses.

STDP receives a lot of attention in the neuromorphic field as exemplified by the latter reference and the review of Serrano-Gotarredona et al., on which we base the following sentences. The reader interested in hardware implementations of STDP should consult that resource. STDP is among the most developed memristors implementation of in silico plasticity, it can be implemented in very large and very dense arrays of memristors without global synchronization, and learning occurs online in a single integrated phase (as opposed to offline learning). The impact of the dynamical model of the memristor has been studied in the implementation of STDP, and the learning rules can be adapted to the different behaviors. As in most application of memristors as non-volatile storage of (synaptic) weights, it suffers from the intrinsic variability of the units, which more general neuromorphic circuits are able to exploit [64].

5. Memristors for Data Processing

Device variability (Section 4) and volatility (Section 4.2) were mentioned as current challenges for most applications based on memristive memories. This is in contrast with biological systems, which are not built in clean rooms, and it is hard to believe that evolution exploits ideal systems in reference to its own architecture design. Biological systems perform despite noise, nonlinearity, variability, lack of robustness, and volatility. Whether these ingredients hinder the performance of biological systems or are actually a building block for it, it is still unknown. Sometimes they are avoided, not because it would have an undesired effect in practice, but simply because its effect cannot be easily modeled or studied (e.g., we have but a few tools to deal with nonlinear systems intrinsically, beyond the iteration of linear methods). Hence, there are no reasons to believe that eliminating naturally occurring properties is the path to success in achieving artificial systems that perform comparably to biological ones.

This section overviews some applications that embrace device volatility [65], nonlinear transients, and variability [66] (Section 7.5), to implement learning methods with memristors. To put these methods within an unifying framework, we first review the concept of analog computation, and discuss its relation to the physical substrate on which it is implemented.

5.1. Analog Computation

In order to pin down the concept of analog computation, we compare analog and digital computers, assuming that the latter is familiar to most readers. The principal distinction between analog and digital computers is that digital operates on discrete representations in discrete steps, while analog operates on continuous representations, i.e., discrete vs. continuous computation (refer to [67] for a complete discussion and historical overview) [68,69].

In all kinds of computation, the abstract mathematical structure of the problem and the algorithm are instantiated in the states and processes of the physical system used as computer [70]. For example, in the current digital computer, computation is carried out using strings of symbols that bare no physical relationship to the quantities of interest, while, in analog computers, the latter are proportional to the physical quantities used in the computation. Figure 6 illustrates the relation between representation of the problem in the designers mind and instantiation in the computers states.

We can decompose computation in three stages: (i) encoding, (ii) processing, (iii) decoding. Programming a computer implies first encoding our input to the processing unit and then decoding the output to obtain the result of the computation. These three stages require an advanced understanding of the behavior of the physical substrate, how it reacts to inputs and how it transforms its states. This is true if the computer is meant to implement an universal model of computation, or if it is specialized hardware optimized to solve a particular subclass of problems.

The encoding and decoding maps relate the states of the computer to our understanding of the problem, and their definition is a recurring challenge in the design of computers. We mentioned this difficulty when building memristive synaptic arrays with plasticity (Section 4.2); data representation in biological systems is still an open research field, and natural systems tend to smear out our pristine categorizations of encodings. There is yet another difficulty to attend to when defining encoding and decoding maps. To avoid confounding, the class of maps needs to be restricted because ill-defined maps (e.g., encryption) will complicate computation, while too sophisticated maps could render the contribution of the computing device negligible. The former will deteriorate the computing performance, while the latter is just bad design. In other words, the problem needs to be specified using the “language” of the computing device. Using the wrong language increases the difficulty of the problem, and consequently decreases performance. To understand the language of the device, we need the equivalent of Shannon’s analysis of the differential analyzer [71]. The encoding-decoding pair is also linked to the “natural basis of computation” [72] of a device, which refers to the description of the device behavior suited for the computation purposes.

The success of digital computers is in part given by the efficiency to instantiate and process a universal model of computation able to solve all kinds of computation problems, as conjectured by the Church–Turing(–Deutch) thesis [73]. This is achieved by a precise control of each step of the computation and the way the computer transforms (or evolves) its states. Another advantage of modern digital computers over analog prototypes is the very high precision they provide for the instantiation and solution of a problem’s quantities. This high precision, however, is unnecessary in many engineering applications, in which the input data are known to only a few digits, the equations may be approximate or derived from experiments, and the results are not sensitive to round-off errors (see [74] for an overview), Therefore, research into specialized hardware (analog of digital) is a worthy activity.

Since analog computers escape the frame of relevance of the Church–Turing thesis, it has been argued that they can be more powerful than digital computers [67] (Section “Analog Computation and the Turing Limit”). Besides this benefit, it is worth exploring the efficiency of analog computers to solve subclasses of problems, i.e., specialized analog hardware, and to understand their pervasiveness in natural systems, perhaps linked to the precision attained by systems built from many imprecise cheap modules. This is not a simple task, since many of these analog computers outsource some of the process control present in digital computers to the natural dynamics of the physical substrate, elevating the bar on the level of understanding required to build and program them. It is possible to achieve a significant speedup in computational time, usually at the expense of other quantities; for instance [75] reported an speedup for nondeterministic polynomial time (NP)-Complete problems at an exponential cost in energy.

As an example of an analog computer, let us consider a hydrostatic polynomial root finder proposed by Levi [76]. Consider the following polynomial equation:

$$\sum _{i=1}^n{x}^i{c}_i+c_0=0.$$

(20)

The aim is to find a real value x for which the equality holds, i.e., we want one root of the polynomial $p\left(x\right)={\sum }_{i=1}^n{x}^i{c}_i+c_0$. For the sake of this illustration, let us consider the case $n=2$. In Figure 7, we show the design of a hydrostatic root solver. On one side of a pivoting lever, a certain shape is set to represent each term in the derivative of the polynomial (e.g., flat for the derivative of $c_{1}x$, linear for the derivative of $c_2{x}^2$). On the other side of the lever, we set a weight for the constant term. When the device is submerged into the water holding the pivoting point, the depth at which the device equilibrates (i.e., becomes horizontal) is a solution of the polynomial equation.

The example above illustrates the major role played by the understanding of the physical device, and how it allows us to encode and solve the specific problem we are interested in. As mentioned before, this is common to all sort of computers; we proceed with mentioning a few. The electronic digital computer exploits the behavior of transistors to encode the symbols of the computational model (essentially Boolean logic) it uses for computation. Quantum annealers, considered for efficiently solving Mixed Integer quadratic Programming [77] (Section 2.7), use the spins of a physical system and the computation of a quantum Ising model to solve the optimization [78]. Deoxyribonucleic acid (DNA) computing exploits the self-assembly and preferential attachment of DNA strands to encode tiling systems with Wang tiles [79,80]. Slime mold [23] computation exploits the behavior of these protists to implement distributed optimization, e.g., shortest path, using the position of nutrients on the Petri dish to encode problems, and chemotatic behavior of the mold to solve them; the solution is read out from the distribution of mold cells in the Petri dish. Ant colony optimization [81] is inspired by the behavior of their natural counterpart and uses digital models of pheromone dynamics for computation (see also stigmergy [82]).

In the subsequent sections, we describe in some technical details the algorithms meant to realize computers using memristive systems.

5.2. Generalized Linear Regression, Extreme Learning Machines, and Reservoir Computing

Arguably, the most used method to relate a set of values X (inputs) to another set of values $y$ (outputs) via an algebraic relation is linear regression: $y=X^{\top }\mathit{\beta }$. However, in many realistic situations, we believe that the relation between these sets of values is unlikely to be linear. Hence, a nonlinear counterpart is needed. The simplest way to generalized a scalar linear regression model between inputs and outputs is to apply a nonlinear transformation to the inputs to obtain a new linear model $y=G{\left(X\right)}^{\top }\mathit{\gamma }$, where only the coefficient vector $\mathit{\gamma }$ (or matrix when the output is not scalar) is learned from the data. This nonlinear method is a linear regression in a space generated by nonlinear transformations of the input (see kernel methods [83,84]). In neuromorphic computation, the transformation is commonly given a particular structure by choosing a set of $N_g$ nonlinear functions and applying it to each input vector:

$$G{\left(X\right)}_{ij}^{\top }=g_j\left(x\left(i\right)\right),i=1,\dots ,Nj=1,\dots ,N_g.$$

(21)

This method is known as Extreme Learning Machines (ELM) [85] and has been implemented in hardware exploiting the variability of fabricated devices to generate the nonlinear transformations [86,87]. Briefly explained, the differences among the devices generate different outputs for the same input, which are then used as the dictionary $G{\left(X\right)}_{ij}^{\top }$.

This formulation resembles two other mathematical methods that are worth mentioning: generalized Fourier series and generalized linear methods. The relation with generalized Fourier series is made evident when the samples have a natural ordering (e.g., not i.i.d. time samples $i\leftrightarrow t_i$), then we can write the regression model as

$$y\left(t\right)=\sum _{k=1}^{N_g}{\gamma }_k{g}_k\left(x\left(t\right)\right),$$

(22)

which has the structure of a truncated generalized Fourier series (but not all the ingredients).

The resemblance with generalized linear models is made evident by considering the function $g\left(x\right)={\ell }^{-1}\left(x^{\top }\mathit{\eta }\right)$ (ℓ is the link function) and $N_g=1$; we obtain:

$$y_i={\gamma }_1{\ell }^{-1}\left(x{\left(i\right)}^{\top }\mathit{\eta }\right).$$

(23)

However, generalized linear models require that we learn the vector of coefficients $\mathit{\eta }$ from the data, rendering the problem nonlinear. This breaks the analogy with ELM in which the $\mathit{\eta }$ vector should be fixed a priori. The analogy is somehow rescued if we are given a distribution $p\left(\mathit{\eta }\right)$ for the $\mathit{\eta }$ vector encoding prior knowledge or beliefs about the solution to the generalized linear model. In this case, we can build an ELM with $N_g\gg 1$

$$y_i=\sum _{k=1}^{N_g}{\gamma }_k{\ell }^{-1}\left(x{\left(i\right)}^{\top }{\mathit{\eta }}_k\right)\approx {\int }_H{\ell }^{-1}\left(x{\left(i\right)}^{\top }\mathit{\eta }\right)p\left(\mathit{\eta }\right)\mathrm{d}\mathit{\eta },$$

(24)

where the set of $\left\{{\mathit{\eta }}_k\right\}$ coefficient vectors are sampled from the prior distribution, expecting that model averaging [88] will approximate the generalized linear model.

Summarizing, ELM uses a linear combination of a predefined dictionary of functions to approximate input-output relations. The next step of generalization is Reservoir Computing (RC) [89,90,91], in which the $N_g$ functions are the solution of a differential or difference equations using the data as inputs, e.g.,

$$\begin{array}{cc}\hfill \mathit{u}\left(t\right)& =\mathcal{E}\left(x\right)\left(t\right)\in {\mathbb{R}}^{dim\mathit{u}},\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill {\mathcal{D}}_{\mathit{\lambda }}\mathit{q}& =B\mathit{u}\left(t\right),B\in {\mathbb{R}}^{dim\mathit{q}\times dim\mathit{u}},\hfill \end{array}$$

(26)

$$\begin{array}{cc}\hfill \mathit{g}& =H\mathit{q},H\in {\mathbb{R}}^{N_g\times dim\mathit{q}},\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill y\left(t\right)& ={\mathbf{\gamma }}^{\top }\mathit{g}=\sum _{k=1}^{N_g}{\gamma }_k{g}_k(t,\mathit{u}\left(t\right),\mathit{\lambda }),\hfill \end{array}$$

(28)

where the differential or difference equations are denoted with ${\mathcal{D}}_{\mathit{\lambda }}$ (operator notation) with $\mathit{\lambda }$ a vector of parameters that includes physical properties and the boundary (initial) conditions, and $dim\mathit{q}\ge N_g$. The connectivity matrices B and H are typically random, the latter mixes the $dim\mathit{q}$ states to obtain $N_g$ signals (these could also be nonlinear mappings). As explained in Section 5.1, the input data is encoded by the transformation $\mathcal{E}$ (or $B\circ \mathcal{E}$) to properly drive the system. This encoding, the operator ${\mathcal{D}}_{\mathit{\lambda }}$, and the connectivity matrices are defined a priori, and only the coefficients $\mathit{\gamma }$ combining the $\mathit{g}$ functions are learned from the data, as in ELM. These coefficient (or ${\mathit{\gamma }}^{\top }H$) define the readout transformation $\mathcal{R}$ (see Figure 6).

The generalization proposed by RC is made obvious with the choice of arguments for the component functions $\left\{g_k\right\}$ in Equation (28): they can have (i) an intrinsic dependence on time, e.g., autonomous behavior of the dynamical system; (ii) they depend on the inputs, and (iii) they depend on the properties of the dynamical system. Property (ii) says that these functions are not fixed as in ELM, they are shaped by the data signal. Stated like this, the problem of implementing computation with reservoirs is strongly related to a nonlinear control problem.

Reservoir computing (RC) allows for machine learning applications using natural or random dynamical systems, as opposed to carefully engineered ones. Its capacity to exploit wildly different physical substrates for computation has been recently highlighted [92]. The only strong requirement is that we are able to stimulate states of the system independently with signals encoding the input data, a classical example is the perceptron in a water bucket [93]. Hence, RC implementations using memristive networks has received considerable attention (software [see [65] and references therein] and hardware [94]). In these applications, the memristive system (single device or network) plays the role of the dynamical system ${\mathcal{D}}_{\mathit{\lambda }}$ (cf. Figure 6), the decoding map is a linear readout of several system states (memories, currents, voltage drops, etc.), and the encoding is usually tailored for the given application at hand.

The case of memristor based RC using the HP-memristor, Equations (4) and (5), is fairly well understood: a differential equation to simulate the propagation of signals across the circuit and the interaction between memristors has been derived in [45] (see also Equation (36)). In those equations, the role of the circuit topology is extremely important in the collective dynamics of the circuit and in processing the input information. When working with RC and memristors, it is important to prevent the saturation of all devices, since a saturated memristor becomes a linear resistor that only scales the input. That is, RC heavily relies in the nonlinear (and volatile) behavior of the dynamical system.

5.3. Neural Engineering Framework

The Neural engineering framework (NEF) [66] exploits our current understanding of neural data processing to implement desired computations. It confines linear models to a particular class of basis functions, inspired by biologically plausible neuron models but not restricted to them. Here, we describe this framework in the case of function representation, the structure of the framework is analogous to other representation instances (scalar, vector, etc.). The reader is referred to the original work [66] for a comprehensive description.

The framework entails the characterization of an admissible set of functions that can be represented by a population of N neurons. In particular, the functions and their domain need to be bounded: $f:(x_{\min },x_{\max })\to (f_{\min },f_{\max })$. These functions are then encoded by a population of neurons with a predefined set of encoders of the form:

$$\begin{array}{cc}\hfill a_i\left(f\left(x\right)\right)& =G_i\left(I_i\left(f\left(x\right)\right),\right)\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill I_i\left(f\left(x\right)\right)& ={\alpha }_i\langle f\left(x\right){\tilde{\varphi }}_i\left(x\right)\rangle +I_i^{\mathrm{bias}},\hfill \end{array}$$

(30)

where $I_i$ represents the total input current to the i-th neuron soma. The functions $a_i$ and $G_i$ are the tuning curves observed by neurophysiologists and the response function of the i-th neuron, respectively. $G_i$ is a biologically inspired model of the firing rate, e.g., integrate-and-fire (LIF) neuron. The encoding generators $\left\{{\tilde{\varphi }}_i\right\}$ (analogous to preferred directions) are defined a priori, and $\langle f\left(x\right){\tilde{\varphi }}_i\left(x\right)\rangle$ is a functional defined on the space of functions to be represented, e.g., in the original description this is the mean over x. These encoders convert the function $f\left(x\right)$ into firing rates (or actual spike counts for the case of temporal encoding).

The encoding is matched with a corresponding decoding procedure that brings firing rates (spike counts) back to the function space. The decoder takes the generic form:

$$\widehat{f}\left(x\right)=\sum _i^N{a}_i\left(f\left(x\right)\right){\varphi }_i\left(x\right),$$

(31)

where $\varphi \left(x\right)$ are the unknowns of the framework. That is, given some input x and neural population’s firing rates (spike counts), we can build functions of the input using the decoders $\left\{{\varphi }_i\right\}$. In the original formulation, the decoders are obtained via minimization of least square errors (with regularization in the case of noisy encodings), but other methods could be used, e.g., optimal $L_2$ dictionaries [95].

The framework defined in Equations (29)–(31) can realize arithmetic on functions of the input as well as nonlinear transformations. It has also been used to represent linear time-independent systems with neuromorphic hardware [96].

Neural Engineering Framework (NEF), Reservoir Computing (RC), and Extreme Learning Machines (ELM) use the same form of decoding: the output is the scalar product of an input-independent vector with an input-dependent one. The input-dependent vector is given by intrinsic properties of the computing device, it is the result of internal mechanisms. The decoders are learned from data, and different decoders implement different computations (on the same input-dependent vectors). However, RC and ELM learn a finite dimensional vector $\mathit{\gamma }\in {\mathbb{R}}^{N_g}$ while NEF, in the functional form shown here, needs to learn N infinite dimensional decoders. The latter is mildly relaxed if the input domain is discretized with $N_x$ points, rendering the functional decoders $N_x$-dimensional vectors. In general, it is expected that $N_g\ll N_{x}N$, which are the degrees of freedom of NEF, is higher that the one of RC and ELM. Hence, NEF has the risk to transfer all the computation to the decoders making the contribution of the neural population marginal (or even an obstacle). Extra regularity assumption on the decoders $\left\{{\varphi }_i\left(x\right)\right\}$ (Equation (31)) are needed to match decoders effective capacity to the capacity of the dynamic responses $\left\{g_i\left(x\right)\right\}$ (Equation (28)).

Memristor networks can be used to implement NEF, by implementing the response functions $G_i$ of neural models. The $G_i$ functions corresponding to LIF neuron model is

$$G\left[I\left(z\right)\right]=\left\{\begin{array}{cc}\frac{1}{{\tau }_0-{\tau }_{\mathrm{RC}}log\left(1-\frac{I_F}{I\left(z\right)}\right)},\hfill & I\left(z\right)>I_F,\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}\right.$$

(32)

where $I\left(z\right)$ is given by Equation (30). A similar functional form can be achieved by a non-volatile memristor with parasitic capacitance, as shown in Equation (35) (see next section). However, other response functions, which might be easier to implement with memristors, can be used with NEF. Other aspects of neural networks, such as critical behavior [97,98,99] can be observed in networks of memristors [100], although a theoretical understanding of their collective behavior is still poor for both systems [101,102,103].

5.4. Volatility: Autonomous Plasticity

Volatility is a key feature when processing information with memristors (in contrast to memory applications). RC needs volatility to avoid trivial linear input-output mappings and NEF requires it to model the forgetting behavior of neurons. There are many physical processes that can lead to a memristive volatile device, hence the source of volatility should be discussed in the context of a given technology. In what follows, we show how volatility can be linked with a capacitance in parallel (parasitic) to a non-volatile device. Consider an ideal series memristor-capacitor circuit [40] feed with a controlled current. The memristor is modeled with Equations (4), with $\alpha =0$, Kirchhoff’s voltage law for this circuit gives:

$$\begin{array}{cc}\hfill R\left(w\right)\stackrel{I\left(t\right)}{\overbrace{\frac{\mathrm{d}q}{\mathrm{d}t}}}& =-\frac{1}{C}\stackrel{\int I\left(t\right)\mathrm{d}t}{\overbrace{q\left(t\right)}}\to \frac{\mathrm{d}q}{\mathrm{d}t}\left(t\right)=-\frac{q\left(t\right)}{R\left(q\right(t\left)\right)C},\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill R\left(q\right(t\left)\right)& =R_{\mathrm{on}}\left(1+\frac{q\left(t\right)}{\beta }\right)+R_{\mathrm{off}},\hfill \end{array}$$

(34)

from which we obtain a limiting solution for $t\gg 1$

$$q\left(t\right)=\frac{\beta }{\xi }W\left(\frac{\xi }{\beta }{e}^{-\frac{t}{R_{\mathrm{on}}C}}{e}^{-\frac{c_1}{\beta R_{\mathrm{on}}}}\right),$$

(35)

where W is the product-log (Lambert) function [104], $c_1$ is an integration constant, and as before $\xi =\frac{R_{\mathrm{off}}-R_{\mathrm{on}}}{R_{\mathrm{on}}}$. Equation (35) shows that, for large times, the system has a typical exponential $RC$ circuit decay, which is shown in Figure 8 (left). This behavior is observed in experiments [105] where after an external stimuli an exponential-like decay is observed (see Figure 8 (right)).

The limit of a normal RC circuit can be obtained from the above in the limit $\beta \to \infty$, in which ${lim}_{\beta \to \infty }\beta W\left({\beta }^{-1}G\left(t\right)\right)=G\left(t\right)$. The response of a memristor-capacitor and an RC circuit differs, with the first obtaining a longer retention than the first. In addition, the Lambert function W has several properties of the logarithm function, and thus the response is similar to the one suggested in Equation (35). In the case $\alpha \ne 0$ for the model we described here, one has that $w\left(t\right)=e^{t\alpha }\left(c_0+\frac{1}{\beta }{\int }_0^{t}d\tau e^{-t\tau }I\left(t\right)\right)$. Thus, the correspondence between the parameter w and the charge is not exact. In addition, the parameter $\alpha \equiv \mathrm{constant}$ is only an approximation. The conductance of memristive devices decays when there is no input, and the rate of decay depends on the state of the memristor. This is compatible with a state dependent parameter $\alpha$, rather than a constant (see Figure 1 of [34]). A survey of recent hardware designs for temporal memory is provided in [106].

5.5. Basis of Computation

As mentioned before, to design computers with memristors, we need to understand and harness their natural computational power. This is no simple task in general, but for networks of current-controlled memristors linear in an internal parameter, we can write down the differential equation describing the evolution of the memory states and obeying Kirchhoff voltage and current laws [45]:

$$\frac{\mathrm{d}\overrightarrow{w}}{\mathrm{d}t}\left(t\right)=\alpha \overrightarrow{w}\left(t\right)-\frac{1}{\beta }{(I+\frac{R_{\mathrm{off}}-R_{\mathrm{on}}}{R_{\mathrm{on}}}\Omega W)}^{-1}\Omega \overrightarrow{S}\left(t\right),$$

(36)

where $\alpha$ and $\beta$ are the parameters in Equations (4) and (5), $\Omega$ is a projector operator which depends on the circuit topology, $W_{ij}\left(t\right)={\delta }_{ij}{w}_i\left(t\right)$ and $\overrightarrow{S}\left(t\right)$ is vector of applied voltages. For arbitrary memristor components, the generalization of Equation (36) is not known. In the approximation $R_{\mathrm{off}}=p{R}_{\mathrm{on}}$, with p of order one, the equation above can be recast in the form of a (constrained) gradient descent [107], which is reminiscent of the fact that the dynamics of a purely memristive circuit has an approximate Lyapunov function [108,109]. In the simplified setting of purely memristive circuit without any other components, it can be shown that these circuits execute Quadratically Unconstrained Binary Optimization [110]. This idea is in general not recent, and it can be traced back to Hopfield [111,112,113] for continuous neurons. It is also not the only alternative approach to Hopfield networks using memristors [114,115,116,117], as we will also discuss it later.

6. Memristive Galore!

6.1. Memristive Computing

In this section, we review some works developing computation algorithms using networks of memristors and external control hardware based in crossbar arrays and FPGA.

In [118,119], it has been shown that memristive circuits can be used to solve mazes: connecting the entrance and exit of a maze, the memristive circuit as in Figure 9 will re-organize (when controlled in direct current (DC)) to allow the majority of the current to flow along the shortest path. Although this phenomenon already occurs with regular resistances (with a diode in parallel), it is enhanced with memristors. Memristors outside the shortest path go to their OFF state (high resistance) and the current difference (the contrast) to the active shortest path is augmented. This example shows that the wiring between memristors and the asymptotic resistance values are deeply connected; this fact is reminiscent of the ant-colony optimization algorithms [81], molecular computation [120], and other cellular automata models [121].

Ideas along these lines can be pushed further in order to explore memristors as complex adaptive systems [122] able to self-organize with the guidance of the circuit topology and the control of external voltages. Using Equation (36), it is possible to obtain approximate solutions, for instance, of the combinatorial Markowitz problem [109]. Hybrid CMOS/Memristive circuits can, in principle, tackle harder problems via a combination of external control and self-organization [123].

In the literature, several models of memristor-based architectures have been proposed. Many of these proposals are based on the attractor dynamics of volatile dissipative electronics and inspired by biological systems. For instance, a general theory of computing architecture based on memory components (memcapacitors, meminductors and memristors [124]) has been introduced recently in [125], called Universal Memory Machines (UMM), and shown to be Turing complete. Similarly, an architecture based on memristors which includes Anti-Hebbian and Hebbian learning (AHaH) has been proposed in [59,126] for the purpose of building logic gates and for machine learning. In both cases of UMM and AHaH, the solutions of the problems under scrutiny are theoretically embedded in the attractors structure of the proposed dynamical systems, and have not been tested experimentally.

One way to show that a memristive system is a universal computing architecture is to break the system modularly into logic gates based on memristors, and show that the set of obtained gates is universal (which includes NOT and at least one of an AND or OR gates, as in DeMorgan’s law [127]). Turing completeness follows from an infinite random access memory (the infinite tape). Experimentally, it has been shown that it is possible to build logic gates with memristors (we mention for instance [128,129]). An improvement upon this basic idea is to build input-output agnostic logic gates using memristors. Any port of an agnostic gate can be used as input or output, and the remaining states of the gate will converge to the states of a logic gate, regardless of whether the binary variable is at the output or at the input of the gate. For example, if the output of an agnostic AND gate is set to TRUE, the input variables will rearrange to be both TRUE, but, if the output is FALSE, the inputs will be re-arranged such as to contain at least one FALSE. These are called Self-Organizing Logic Gates (SOLG) and it is suggested to use these to solve the max-SAT problem [130,131] (see also [132] and references therein). Similar ideas were recently proposed using nanoscale magnetic materials rather than memristors [133].

The two examples above show that memristors can be used both for analog computation, as in the case of shortest path problems, or to reproduce and extend the properties of digital logic gates.

6.2. Natural Memristive Information Processing Systems: Squids, Plants, and Amoebae

In recent years, and with the participation of Chua, there have been several reports re-interpreting models of natural information processing systems (neural networks, chemical signaling, etc.) in terms of memristors units. Here, we mention three examples: giant axon of squids [134], electrical networks of some plants [135], and Amoeba adaptation [136].

Squids. Giant squids are model organisms big enough that they can be analyzed in detail at the singular cell level, and for which we posses a mechanistic model of the dynamics of their axons: the Hudgkin–Huxley (HH) model. The HH model describes the voltage at the interface between synapses and dendrites, which is regulated by the flow of calcium and potassium. The electrical circuit associated with this model is shown in Figure 10. It entails the introduction of a nonlinear variable resistor for the calcium channel, and a linear variable resistance for the potassium channel and a capacitance [137]. The equations of this model, when put in memristive form, are given by:

$$\begin{array}{cc}\hfill i_{\mathrm{K}}& =\stackrel{R_{\mathrm{K}}^{-1}}{\overbrace{g_{\mathrm{K}}{w}_1^4}}{V}_{\mathrm{K}},\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill i_{\mathrm{Na}}& =\stackrel{R_{\mathrm{Na}}^{-1}}{\overbrace{g_{Na}{w}_2^3{w}_3}}{V}_{\mathrm{Na}},\hfill \end{array}$$

(38)

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{w}_1}{\mathrm{d}t}& =\left({\mathrm{K}}_1{V}_{\mathrm{K}}+{\mathrm{K}}_2\right){\left[e^{{\mathrm{K}}_1{V}_{\mathrm{K}}+{\mathrm{K}}_2}-1\right]}^{-1}\left(1-w_1\right),\hfill \end{array}$$

(39)

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{w}_2}{\mathrm{d}t}& =\left({\mathrm{Na}}_1{V}_{\mathrm{Na}}+{\mathrm{Na}}_2\right){\left(e^{{\mathrm{Na}}_1{V}_{\mathrm{Na}}+{\mathrm{Na}}_2}-1\right)}^{-1}\left(1-w_2\right)+{\mathrm{Na}}_3{e}^{{\mathrm{Na}}_4{V}_{\mathrm{Na}}+{\mathrm{Na}}_5}{w}_2,\hfill \end{array}$$

(40)

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{w}_3}{\mathrm{d}t}& ={\mathrm{Na}}_6{e}^{{\mathrm{Na}}_7{V}_{\mathrm{Na}}+{\mathrm{Na}}_8}\left(1-w_3\right)-{\left(e^{{\mathrm{Na}}_1{V}_{\mathrm{Na}}+{\mathrm{Na}}_9}+1\right)}^{-1}{w}_3,\hfill \end{array}$$

(41)

where we see that a first order ($R_{\mathrm{K}}$) and second order ($R_{\mathrm{Na}}$) memristors are involved. The parameters $\left\{{\mathrm{K}}_i\right\}$ and $\left\{{\mathrm{Na}}_i\right\}$ characterize the dynamics of the channels (see [134] for details).

The model above is a fit of the observed voltage data for the giant axon, and is useful in the analytical study of brain cell dynamics. The proposed model allows the interpretation of synapses as circuits composed of rather nonlinear and non-ideal memristors. That is, that memristors can be central in providing an alternative interpretation of a established model and further the understanding of biological neural information processing.

Plants. In [135], three types of memristors models were developed and compared with the responses of some plants to periodic electrical stimulation. The authors observed that, in the studied plants, the pinched hysteresis loop did not collapse into a line for very high frequencies, as required for ideal memristors. To recover this non-ideal behavior, a parasitic resistor–capacitor pair was added in parallel to the ideal memristor model. The general solution for their models is:

$$\begin{array}{cc}\hfill i_m\left(t\right)& =\frac{e^{\beta t}V\left(t\right)}{\beta R_o{\int }_0^{t}h\left(V\left(x\right)\right)e^{\beta x}\mathrm{d}x+A},\hfill \end{array}$$

(42)

$$\begin{array}{cc}\hfill I& =i_m+i_{RC},\hfill \end{array}$$

(43)

where $\beta$ is a parameter related to the time constant of the memristor, $V\left(t\right)$ is the driving periodic voltage, and $R_{o}h\left(V\right)$ is the memristance of a voltage-controlled memristor. Depending on the model of the memristor considered, h and the constant A take different forms. The total current I is the observed magnitude, and $i_{RC}$ is the current through a series resistor–capacitor circuit in parallel with the memristor. The study hints that memristive behavior is intrinsic to plants electrical signaling and that plant physiology could be better understood if memristors are considered as “essential model building blocks”.

Amoeba. A memristive model of amoeba adaptation was introduced in the form of the simple circuit shown in Figure 11 [136,138], and the model is simple enough that we can report it here. Albeit the original article points to the concept of amoeba learning, we believe it is more appropriate to be addressed as a model of amoeba adaptation. The memristor considered is a voltage-controlled memristor introduced in [139]:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}M}{\mathrm{d}t}& =f\left(V_M\right)\left(\theta \left(V_M\right)\theta (M-R_1)+\theta (-V_M)\theta (R_2-M)\right),\hfill \end{array}$$

(44)

$$\begin{array}{cc}\hfill f\left(V\right)& =\frac{\beta -\alpha }{2}\left(|V+V_T|-|V-V_T|\right)-\beta V,\hfill \end{array}$$

(45)

where $\theta (·)$ is the Heaviside step function.

Because the inductance and the resistor in the circuit are in series, the same current I flows through them. The capacitor and the memristor are in parallel, hence their voltage drop are equal: $V_C=V_M$. The conservation of voltage on the mesh implies $V_C+V_L+V_R=V\left(t\right)$. We have $V_R=RI$ and $V_L=LI$. The memristance $M\left(t\right)$ affects the voltage drop on the capacitor,

$$C{V}_C+\frac{V_C}{M\left(t\right)}=I.$$

(46)

We thus obtain the three coupled differential equations:

$$\begin{array}{cc}\hfill \frac{\mathrm{d}I}{\mathrm{d}t}& =-\frac{R}{L}I+\frac{V-V_C}{L},\hfill \end{array}$$

(47)

$$\begin{array}{cc}\hfill \frac{\mathrm{d}{V}_C}{\mathrm{d}t}& =-\frac{1}{MC}{V}_C+\frac{I}{C},\hfill \end{array}$$

(48)

$$\begin{array}{cc}\hfill \frac{\mathrm{d}M}{\mathrm{d}t}& =f\left(V_M\right)\left(\theta \left(V_M\right)\theta (M-R_1)+\theta (-V_M)\theta (R_2-R)\right).\hfill \end{array}$$

(49)

The stationary state requires that all time derivatives are zero. In this state, the circuit is sensitive to new stimuli, i.e., it adapts. For instance, in Figure 12, we see the response of the system to new inputs, and new stationary states are obtained. Albeit amoeba’s adaptation are not as developed as in higher mammals, more general models entailing Pavloviav “conditioning" have also been proposed in the literature using memristors [140,141].

6.3. Self-Organized Critically in Networks of Memristors

We now consider the interaction between a high number of components with memory. A common feature of large systems of interacting units with thresholds or discontinuous dynamics is critical behavior. For example, self-organized criticality (SOC) is evinced when a dynamical system self-tunes into a configuration for which a qualitative change in the systems’ behavior is imminent (e.g., a bifurcation). These critical configurations are characterized by states with power law cross-correlation functions. One of the main motivations of SOC is the explanation of power spectra of the functional form $P\sim {\omega }^{\alpha }$, with $-1<\alpha <-3$, in physical systems and in nature [142]. The typical example is, for instance, the Gutenberg–Richter law of earthquakes, whose distribution of magnitude is Richter’s law [143,144]. The current characterization of the sufficient ingredients for SOC, however, is phenomenological: a system of interacting particles or agents in which thresholds are present and whose dynamics is dominated by their mutual interaction. SOC has been suggested to be the underlying mechanism in the observed critical behavior of the brain [98]. In this case, neurons can be interpreted as thresholding functions (logical gates) and it is thus tempting to interpret the criticality of the brain as a SOC phenomenon.

SOC can be produced using large networks of memristors: atomic switch networks are dominated by the interaction due to Kirchhoff laws [100], and thus the observed criticality seems intuitively (power law distribution of the power spectra, for instance) to be connected to a SOC-type [103] phenomenon because of the rather nonlinear and threshold-like behavior of memristors. It is however easy to observe that power law distributions in power spectra can be obtained in a rather simple way as follows [45]. Consider a system of interacting memristors, whose linearized dynamics close to a fixed point obtained with DC voltage stimulation (i.e., saturated memristors: resistors) is written as:

$$\frac{\mathrm{d}\overrightarrow{w}\left(t\right)}{\mathrm{d}t}=A\overrightarrow{w}\left(t\right).$$

(50)

The matrix A is a non-trivial combination of voltage sources and projectors on the subspace of the circuit’s graph [107]. We divide the spectrum of A in positive and negative eigenvalues, and the distribution ${\rho }_{+}\left(A\right)$ and ${\rho }_{-}\left(A\right)$. Since $0\le w_i\left(t\right)\le 1$, we look at the average relaxation $\langle w\left(t\right)\rangle ={\sum }_i\frac{w_i\left(t\right)}{N}$, which can be written as

$$\langle w\left(t\right)\rangle =\frac{1}{N}\sum _i\left(\sum _j{\left(e^{At}\right)}_{ij}{w}_j^0\right)=\frac{1}{N}trace\left(e^{At}{W}_0\right)=\frac{1}{N}trace\left(e^{{\lambda }^{+}t}{\tilde{W}}^0+e^{{\lambda }^{-}t}{\tilde{W}}^0\right).$$

(51)

The positive part of the spectrum will push memristors to the $w=1$ state, while the negative part to the $w=0$ state. We can write the trace on each positive and negative state as:

$$\frac{1}{N}trace\left(e^{-{\lambda }_i^{-}t}{\tilde{w}}_i\right)=\int d\lambda {\rho }^{-}\left(\lambda \right)e^{-\lambda t}\langle w_i^0\rangle =\frac{1}{2}\int d\lambda {\rho }^{-}\left(\lambda \right)e^{-\lambda t},$$

(52)

if the memristors are randomly initialized. As it turns out, if ${\rho }^{-}\left(\lambda \right)$ is power law distributed, then $\langle w\left(t\right)\rangle \approx t^{\gamma }$. From this, we observe that the power spectrum distribution is of the form $P\left(\omega \right)\approx {\omega }^{-(1-\gamma )}$, from which a “critical state" is obtained. It was shown in [45] that if the circuit is random enough, numerical simulations produce $\gamma \approx -1$. A similar argument can be obtained for ${\rho }^{+}\left(\gamma \right)$ with the transformation $w\to 1-w$. This result, when using networks of HP-memristors, is in line with what is experimentally observed in [100].

The phenomenon above is not classified as SOC, since only a certain matrix A is required to obtain it. This implies that unless thresholding is present, the criticality observed is not self-tuned, but it might be due to the complex interconnections. This is not the case if, however, the memristors themselves have voltage induced switching [103]. In this case, the criticality is due to the a SOC-like phenomenon which had been already observed in random fuse networks, which is described by a percolation transition [145].

6.4. Memristors and CMOS

The idea of using variable resistances in order to implement learning algorithms is not new. As we have seen, crossbar arrays were introduced already as early as 1961 [51]. The idea of using instead variables resistances precedes the paper of Steinbuch by one year, and was introduced by Widrow [146] in order to implement the Adaline algorithm explained below. The “memistor”, a name extremely similar to the one of “memristor” introduced ten years later by Chua, was a current-controlled variable resistance [147]. This limited the ability to package a huge number of memistor synapses. For (modern) machine learning applications, however, it is necessary to implement learning rules with a large number of neurons and synapses. As we have seen, memristors are the equivalent component for a synapse [148]. The neuron is instead the biological equivalent of a N-port logic gate (threshold function). For more general applications, a crossbar-array like packing is desirable. There are many ways of using memristors for applications in neural networks [149]. For instance, the circuit proposed in Figure 13 provides a simple circuit which has a linear output neuron controlled by a resistance R, without a threshold. The threshold can be however easily introduce via a Zener diode. In order to understand why memristors do not have necessarily an advantage over digital implementations of neural networks, we follow the argument of [150] to understand the energy efficiency. Using Landauer theory, the energy dissipated by a digital gate is $E_{gate}\approx -2log\left(p_{err}\right)kT$, where $p_{err}$ is the probability of an error. For the analog implementation above, the only dissipation is due to Johnson–Nyquist noise in the amplifier, which is of the order $4kTf$, with f the amplifier’s bandwidth and N the number of synapses. Keeping track of error and number of bit precision L, one reaches the conclusion that

$$\begin{array}{cc}\hfill E_{dig}& \approx 24log\left(\frac{1}{p_{err}}\right){log}_2^2\left(L\right)NkT,\hfill \end{array}$$

(53)

$$\begin{array}{cc}\hfill E_{memr}& \approx \frac{1}{24}log\left(\frac{1}{p_{err}}\right)L^2{N}^{2}kT,\hfill \end{array}$$

(54)

which scales in the number of artificial neurons in favor of digital implementations. This surprising result thus confirms that the devil is in the details, and that not necessarily all analog implementations of digital systems are better.

The architecture of Figure 13, however, does not take advantage of the scalability of crossbar arrays which we have mentioned earlier, in terms of number of neurons, and other implementations might be more energy efficient. For instance, in [151], first experimental results of memristive technology for pattern classification on a 3 × 3 image matrix was studied using crossbars and TiO${}_2$ memristors. In general, learning using crossbars follows a general weight update strategy. For regressions, current-controlled memristors can be used with a simple serial architecture [65] while unsupervised learning can be performed by implementing the K-means algorithm [152,153,154].

Next, we focus on applications of memristor technology for Machine Learning (ML). Algorithms like backpropagation on conventional general-purpose digital hardware (i.e., von Neumann architecture) are highly inefficient: one reason for this is the physical separation between the memory storage (RAM) of synaptic weights and the arithmetic module (CPU), which is used to compute the update rules. The bus between CPU and RAM acts as a bottleneck, commonly called von Neumann bottleneck. As mentioned before, the idea is thus to use memristive based technology to introduce computation and storage on the same platform. One way to introduce learning into crossbars is by introducing feedback into the system, as in Figure 14.

Let us consider a discretized dynamics, and call $W^k$ the weight matrix in the crossbar array at time step k. A general update is of the form

$$W^{k+1}=W^k+f(W^k,Q),$$

(55)

where $f\left(W^k\right)$ is a certain function of the weights and some ancillary variables Q (which could be training data, inputs, outputs, etc.). For instance, in the case of neural networks training (gradient descent type), $f\left(W^k\right)=-\eta {\nabla }_{W^k}{\parallel \overrightarrow{t}-\overrightarrow{o}\parallel }^2,$ where $\overrightarrow{t}$ is the output we aim to obtain (given the inputs), and $\overrightarrow{o}$ is the output. For resistive crossbars, we have seen that $\overrightarrow{o}=G\left(W^k\right)\overrightarrow{v}$ is linear in the inputs and G is the conductance, while $\eta$ is a time scale parameter. If $f{\left(W\right)}_{mn}=\eta x_m^k{x}_n^k$, where $x_n^k$ is the teacher inputs (patterns) indexed by the index $k=1,\cdots ,K$, then the Hebbian learning rule is called adaline algorithm [60,155]. From the point of view of circuit theory, the feedback can be introduced via CMOS-Memristor integration. For instance, in [156], one way to perform online learning with an adaline algorithm using metal-oxide-semiconductor field-effect transistors (MOSFETs) has been provided.

Models like the one just described can, for instance, be used for sparse coding [157]. Sparse coding can be mathematically formulated as a problem in linear algebra. Consider a vector which describes a certain quantity of interest $\overrightarrow{x}$ (for instance, an image), and a dictionary which is available as ${\overrightarrow{\varphi }}^i$, $i=1,\cdots ,M$. Furthermore, assume that ${\varphi }^i$ and $\overrightarrow{x}$ belong to ${\mathbb{R}}^N$. If $M>N$ and the vectors are independent, we can always find the coefficients $a_i$ such that ${\sum }_i{a}_i{\overrightarrow{\varphi }}^i=\overrightarrow{x}$, and there is an infinite number of ways to do this expansion. The goal of sparse coding is solving the following optimization problem:

$$\begin{array}{cc}\hfill \overrightarrow{x}& =\sum _{i=1}^M{a}_i{\overrightarrow{\varphi }}^i,\hfill \end{array}$$

(56)

$$\begin{array}{cc}\hfill {\min }_{\overrightarrow{a}}& \parallel \overrightarrow{a}{\parallel }_0,\parallel ·\parallel 0-\mathrm{norm},\hfill \end{array}$$

(57)

and which is notoriously NP-hard. The problem above can be relaxed by replacing the 0-norm with the 1-norm, and a system of differential equations for continuous neurons can be implemented in crossbar arrays. In general, these are equations of the type

$$\frac{\mathrm{d}\overrightarrow{u}\left(t\right)}{\mathrm{d}t}=F(\overrightarrow{u}\left(t\right),\overrightarrow{q}\left(t\right)),$$

(58)

where $u_i\left(t\right)$ and $q\left(t\right)$ are some control functions and $F(·)$ represents a generic continuous function. Some details are provided in Appendix B. The variables $u_i\left(t\right)$ are intended as the memory elements in a memristor: in a crossbar array system, the equations above have been implemented by combining a field programmable gate array (FPGA) with a crossbar in [158], where using a threshold function $T_{\lambda }\left(x\right)=x$ if $x>\lambda$ and zero otherwise. Another update rule used in experiments is Sanger’s update rule [159], defined as

$$W^{k+1}=W^k+2\eta {\overrightarrow{o}}^t(\overrightarrow{x}-(2W^k-I)\overrightarrow{o}),$$

(59)

where $\overrightarrow{o}$ is the output of the crossbar array, and $\overrightarrow{x}$ the input, which is used in [160] in order to perform PCA, again with the use of FPGA.

Recent advances in Deep Neural Network implementations using memristor technology (PCM) show remarkable efficiency in terms of energy consumption for a given constant accuracy, with gains up to two order of magnitudes when compared to graphics processing unit (GPU) implementations [116]. The architecture used therein is not dissimilar from the one described in this section using a combination of CMOS and crossbar arrays. This is another example of how crossbar arrays are amenable to efficient matrix multiplication, which renders them competitive to other GPU implementations [111,112,117].

We have already discussed reservoir computing (RC) [65,94] as another way of using memristors within the framework of machine learning whilst taking advantage of their temporal dynamics (see Section 5.2). In [94], RC was implemented with a memristor reservoir and one layer output with 32 × 32 crossbar of memristors and trained using an external FPGA. The model was then used successfully to classify the MNIST dataset (5 × 4 images) as a proof of principle application. The advantage of using RC is that it can be implemented both for online learning and for classification in a teacher–signal framework, and with relatively little computational effort.

In conclusions, CMOS provides an advantage for controlling memristive circuits, but it is possible to use just the inner dynamics of memristors to perform learning [107].

The main question that remains unanswered is whether analog system have an advantage over digital ones at all. A strong argument is provided by Vergis et al., where it is shown that analog devices can be simulated with polynomial resources on a digital machine with enough resources. We pointed out the importance of the collective properties of memristive circuits. The dynamics of a collection of memristors interacting on a circuit can, in principle, derived from the implementation of circuit voltage and current constraints, and strongly depends on the dynamics of a single unit. Understanding the interaction of memristors via Kirchhoff laws can in principle enable the application of memristors to a variety of computational tasks. Below, we make this more precise using a general mapping from digital to analog computation. We consider a differential equation of the form:

$$\frac{\mathrm{d}\overrightarrow{y}}{\mathrm{d}t}\left(t\right)=f(\overrightarrow{y}\left(t\right),t),y\left(t_0\right)=y_0,$$

(60)

which describes a physical system. In [161], a constructive proof based on a Euler integration method is provided, and it is shown that the amount of resources needed to simulate the system above on a digital machine is polynomial in the quantities R and $ϵ$:

$$R=\underset{t_0\le t\le t_f}{max}\left|\right|\frac{\mathrm{d}{}^2\overrightarrow{y}}{\mathrm{d}{t}^2}\left(t\right)\left|\right|,ϵ=\left|\right|\overrightarrow{y}\left(t_f\right)-{\overrightarrow{y}}^{*}\left|\right|,$$

(61)

where ${\overrightarrow{y}}^{*}$ is the simulated system, and thus $ϵ$ is our required precision. Since for quantum systems $R\propto 2^N{r}^{*}$, and $r^{*}$ is a constant, classical computers require an exponential amount of resources to simulate a quantum physical systems. This does not mean that classical systems can always be simulated: if the second derivative is large, our system requires a lot of computational power to be simulated on a digital machine. A similar argument applies also to quantum computers, on which there has been a huge effort in the past decades. In the case of a quantum system with N qubits, the vector $\overrightarrow{y}\left(t\right)$ is $2^N$-dimensional according to the Schrödinger equation. In a typical (analog) electronic computer, $\frac{\mathrm{d}{}^2\overrightarrow{y}}{\mathrm{d}{t}^2}$ is the derivative of the voltage. Thus, if our circuit presents instability, it is generically hard to simulate the system. This is specially striking in the case of chaotic behavior, which is known to emerges when a dynamical system is connected to a hard optimization problem [162]. These are also arguments against the simulation of memristive system on a digital computers, which do not apply to the actual physical system performing the analog computation.

7. Conclusions

In the present article, we have provided an introduction and overview of memristors, both the applications as memory, and the appealing features of memristors beyond the purpose of memory storage. We have discussed the history of memristors, with the purpose of helping the reader understand their role in modern electronic circuitry, and why memory is a normal feature at the nanoscale. The perspective which we have tried to provide is that, despite current applications focus on the implementation of standard machine learning algorithms on chips, memristors can be used to perform analog computation which goes beyond the standard framework of crossbars.

In magnetic materials, which are commonly used for memory purposes, the interaction between the magnetic spins (which represent the bits) is purposely screened to avoid the spins to flip, and the memory be lost. Via the interaction between the spins however, logic gates can be constructed [133]. Similarly, memristors can be used as memories, thus trying to avoid their interaction in the circuit via Kirchhoff laws, or one can try to harness their interaction to perform computation. At the most basic level, memristors can be interpreted as synapses, and the introduction of hybrid CMOS-memristor technology can allow the implementation of supervised and unsupervised machine learning algorithms that make use of these features. This resonates with the fact that memristors have been proposed to play an important role in biology, as for instance in the case of plant signaling, amoeba adaptation, and the Hudgkin–Huxley model of squids neurons.

Building on these ideas, it make sense to pursue the complex dynamical features of memristors, interacting via Kirchhoff laws, for self-organizing computational devices. Before the dynamical features of memristors can be fully harnessed, it is though imperative to be able to understand the single memristor physical principle in order to implement reliable memory units, in particular using the crossbar array framework we have described. This task requires a deep knowledge of the dynamics of a single memristor component, via models which accurately describe the relevant behavior of the device.

We have also tried to emphasize that there are several mechanisms which enable resistive switching, and different components will follow different mechanisms to change the internal state. Despite their differences, the dynamics of memristive components share a common feature, which is the competition between two internal phenomena. These phenomena can be cast as “forgetting” (decay to an off state when voltage is not applied) and “reinforcement” (tendency to state change which depends on the current). As discussed by Kohonen, this competition is an important feature among several analog computing paradigms. As examples, we take ant-colony optimization [81] and experimental results with memristive devices [34,163].

We have also pointed out the importance of the collective properties of memristive circuits for computation—in particular, how we can harness the intrinsic variability of memristors to different computational problems. However, analog machines are good at specific computational tasks, while digital ones excel for their generality. Thus, the integration of CMOS and analog systems in future computers is favorable in the long term. The memristor is one of many technologies which are able to encode computational tasks and simultaneously be used as memory, which can be easily integrated in modern computers. However, in certain instances, the von Neumann architecture is not the best architecture for performing calculations, and we mentioned the case of quadratic optimization and generic combinatorial problems.

Due to our focus in memristive devices for computation, and the models of computation that are compatible with their properties, we have omitted reviewing the role that memristive system can have in biological cognitive systems, and their relation to stochastic resonance [164,165,166,167,168,169,170,171]. This decision was taken to keep the presentation focused and consistent.

In conclusion, we have provided an overview of the current research questions and applications of memristive technology. We have provided also a rather long, but far from exhaustive bibliography on the subject which might help the interested reader in learning the subject.