Optimisation Challenge for a Superconducting Adiabatic Neural Network That Implements XOR and OR Boolean Functions

Abstract

In this article, we consider designs of simple analog artificial neural networks based on adiabatic Josephson cells with a sigmoid activation function. A new approach based on the gradient descent method is developed to adjust the circuit parameters, allowing efficient signal transmission between the network layers. The proposed solution is demonstrated on the example of a system that implements XOR and OR logical operations.

1. Introduction

A distinctive feature in the current era of information technology evolution is the widespread development and implementation of artificial intelligence (AI) [1,2,3,4,5,6]. In order to effectively solve a number of tasks, specialised hardware implementation of AI systems is required [7,8]. The most popular and exciting at the moment are the so-called neuromorphic chips or neuromorphic processors. In this field, world giants such as Intel (Loihi 1 and Loihi 2) and IBM (TrueNorth, NorthPole) have made their mark. In addition to neuromorphic processors, there are machine learning processors (Intel Movidius Myriad 2, Mobileye EyeQ) designed to accelerate data processing (video, machine vision, etc.) and tensor processors (Google TPU, Huawei Ascend, Intel Nervana NNP) that are designed to accelerate arithmetic operations. While the latter two types have been successfully implemented in modern hardware platforms (smartphones, cloud computing, etc.), neuromorphic processors, despite their potential, are unfortunately not yet widespread and remain mostly at the laboratory production and testing stage [9,10,11,12,13,14,15,16,17,18].

There are a number of post-Moore technology platforms that enable the realisation of AI technologies at the hardware level, promising advances in performance and/or energy efficiency. Optical neuromorphic networks are an excellent example [19,20] of energy-efficient systems with high performance. Photonic-superconducting interfaces [21,22,23,24] and other hybrid optical-superconducting neural networks [25,26,27,28] were once a major milestone in the development of this field of applied science. These systems use light pulses to transmit signals and superconducting circuits based on quantum interferometers to process and store information. Superconducting elements are known for their high energy efficiency [29,30,31,32,33,34,35,36,37]. In the context of modern data centres that require massive cooling, superconductor-based hybrid computers may become quite competitive players. It is also worth noting that quantum computers [38,39,40] are now being developed on the basis of superconductor technology. Therefore, the creation of superconducting neuromorphic chips, capable of hybridisation with quantum computers (QCs), seems very reasonable. Examples might be qubit spectrum detection of a QC’s output signal or a QC’s calculation of synaptic weights for an externally tunable artificial neural network. This study focuses on the optimisation of superconducting basic elements and their interconnections, specified for superconducting logic gates in neuromorphic systems (Figure 1).

It is also necessary to mention here imitations in the neural activity of living tissues with the help of superconducting electronics using Josephson contacts [28,29,30,31,32,34,37,41,42,43,44,45,46]. These works demonstrate the operation of bio-inspired neurons (capable of reproducing basic biological patterns of nervous activity, such as excitability, spiking, and bursting) and synapses, as well as simple neural networks. The possibility of using Josephson circuits for modelling and simulating the work of neurons and tissues, as well as for more applied tasks (e.g., recognition), will allow a new level of performance (computational and modelling speed, energy efficiency) in spiking neural networks.

Previously, we presented the concept of an adiabatic interferometer-based superconducting neuron [47,48,49], capable of operating in classical and quantum modes with ultra-low energy dissipation per operation (in the zJ range) [50,51,52,53,54,55,56,57]. The development of an adiabatic perceptron requires the realisation of a large number of connections between neurons via superconducting synapses [47]. Good synapses for perceptron-type networks should have the following important properties: a wide range of weights (both negative and positive, as well as zero), low noise, signal-type preservation (high linearity), and circuit simplicity (as few components as possible). Based on these requirements, we used the synapse scheme first presented in [58].

Combining these elements into an analog network implies the generally difficult task of studying the complex nonlinear dynamics of the system. We propose a solution to this problem and demonstrate the results on the example of a three-neuron network simulating XOR and OR logic gates.

2. The Model for Two Coupled Adiabatic Neurons

Before the simulation of the superconducting logic element, we have considered the system of two coupled $S_c$-neurons having sigmoid activation function. These basic elements are the superconducting interferometers connected by the inductive synapse—see Figure 2. The formation of the activation functions (flux-to-flux transformations) on individual $S_c$-neurons has been previously studied in detail in both classical [47,48,49] and quantum modes [55,56]. Here we consider the interaction between different parts of the system. We choose an inductive synapse instead of the Josephson one [59] because of the absolute linearity of its transfer characteristic and a wide dynamic range [58,60].

The $S_c$ neurons (areas outlined by the cyan and navy blue dashed lines in Figure 2) are designed according to the integrating-and-processing principle: the integrating part (IP) collects or $integrates$ all input signals, while the processing part (PP) processes this input signal and generates an output signal. Generally, an $S_c$-neuron consists of three branches (its processing part): two of them (the branches with the inductance $l_{out1,2}$ and the branch with the Josephson junction and the inductance $l_{1,2}$) form the circuit of the so-called quantron; the third branch, with a single inductance $l_{a1,2}$, shunts the quantron circuit. In Figure 2, the IP of the output neuron is highlighted by a light-yellow box and is formed by a so-called coupler—an inductive ring ($l_{t1,2,3,4}$) that collects the output flux from input neuron(s) (the IP of the input neuron is not shown in Figure 2). The signal, in the form of magnetic flux, flows from the neuron’s IP to the neuron’s PP through the inductances $l_{1,2}$ and $l_{a1,2}$. The inductance $l_{out1,2}$ is used to transmit the magnetic flux from the $S_c$-neuron to the subsequent element (in our case, the input neuron transmits its signal to the inductive synapse).

The inductive synapse (green box in Figure 2) in turn also has three branches: the input branch (containing the inductance $l_{in}$) is responsible for signal reception, and the branches containing the tunable kinetic inductances $l_{s1}$ and $l_{s2}$ [58,61,62,63] provide its further transmission. Synapse adjustment is realised by external magnetic or spin–current influence (not shown in Figure 2). By changing the values of the inductances $l_{s1}$ and $l_{s2}$, one can vary the weight of the synapse.

In the following, all inductances are normalised to the characteristic Josephson inductance of the output neuron Josephson junction, ${\Phi }_0/2\pi I_{C_2}$, where $I_{C_2}$ is the critical current of this junction. Magnetic fluxes are normalised to the magnetic flux quantum, $\phi =2\pi \Phi /{\Phi }_0$, ${\Phi }_0=h/2e$.

The input signal (${\phi }_{in}$) has been set in the form of a smoothed trapezoid, which makes it possible to take into account both the rising (rise time) and falling (fall time) phases of the signal. The duration of the plateau section can also be controlled:

$${\phi }_{in}(t)=A_{in}·\left\{\frac{1}{1+exp(-2D(t-t_1))}+\frac{1}{1+exp(2D(t-t_2))}\right\}-A_{in}.$$

(1)

The parameters $A_{in}$ and D set the level and the rise/fall rate of the input magnetic flux, respectively. As shown in [48], the input signal in the form of (1) allows one to obtain the sigmoid transfer function of the $S_c$-neuron for certain values of the inductances.

The circuit shown in Figure 2 is described by the following system of equations:

$$\left\{\begin{array}{c}{i}_{a1}+i_1+i_{out1}=0,\hfill \\ {\phi }_1-\frac{{\phi }_{in}}{2}+i_1{l}_1=i_{out1}{l}_{out1}+m_1{i}_{in},\hfill \\ {\phi }_1-\frac{{\phi }_{in}}{2}+i_1{l}_1=i_{a1}{l}_{a_1}+\frac{{\phi }_{in}}{2},\hfill \\ i_{in}+i_{s1}+i_{s2}=0,\hfill \\ i_{s1}(l_{s1}+l_p)+m_2{i}_{circ}=i_{s2}(l_{s2}+l_p)-m_2^{*}{i}_{circ},\hfill \\ i_{s1}(l_{s1}+l_p)+m_2{i}_{circ}=i_{in}(l_{in}+l_p)+m_1{i}_{out1},\hfill \\ i_{circ}\sum _{j=1}^4{l}_{tj}=i_2{m}_3-i_{a2}{m}_3^{*}-i_{s1}{m}_2+i_{s2}{m}_2^{*},\hfill \\ i_2+i_{a2}+i_{out2}=0,\hfill \\ {\phi }_2-m_3{i}_{circ}+i_2{l}_2=i_{out2}{l}_{out2},\hfill \\ {\phi }_2-m_3{i}_{circ}+i_2{l}_2=i_{a2}{l}_{a_2}+m_3^{*}{i}_{circ}.\hfill \end{array}\right.$$

(2)

Here, ${\phi }_{1,2}$ are the superconducting phase drops at the Josephson junctions of the input and output neurons, while $l_p$ is an additional non-adjustable (parasitic) inductance in this circuit, which is not explicitly shown in Figure 2 but is taken into account in our calculations. The currents $i_{a1}$, $i_1$, and $i_{out1}$ are the currents flowing through the corresponding inductances in the input neurons $l_{a1}$, $l_1$, and $l_{out1}$. The currents $i_{in}$, $i_{s1}$ and $i_{s2}$ are the currents in the synapse, flowing through $l_{in}$, $l_{s1}$ and $l_{s2}$. The currents $i_{s1}$ and $i_{s2}$ induce the circulating current $i_{circ}$ in the integrating part of the output neuron. The circulating current in turn induces currents in the processing part of the output neurons $i_2$, $i_{a2}$, and $i_{out2}$, which flow through the inductances $l_2$, $l_{a2}$, and $l_{out2}$. All currents in (2) are normalised by $I_{C_2}$. The parameters $m_k$ and $m_k^{*}$ are mutual inductance coefficients in transformer elements ($k=1,2,3$), which are considered equal to the average values of the inductances that constitute the corresponding transformers.

It can be shown that the currents in the proposed circuit (Figure 2) have a simple relationship with the phases of the Josephson junctions and the external flux:

$$i_{\gamma }=-{\kappa }_{\gamma }^{(1)}{\phi }_1-{\kappa }_{\gamma }^{(2)}{\phi }_2-{\kappa }_{\gamma }^{(in)}{\phi }_{in}.$$

(3)

Here, all coefficients ${\kappa }_{\gamma }^{(1)}$, ${\kappa }_{\gamma }^{(2)}$, and ${\kappa }_{\gamma }^{(in)}$ are obtained from the system in Equation (2) and represented in terms of inductances according to Figure 2. The subscripts $\gamma =1,2,in,s,a,out$ of the coefficients indicate the currents ($i_1$, $i_2$, $i_{in}$, and $\Delta i_s=i_{s1}-i_{s2}$, respectively) to which they belong. The superscript in Formula (3) takes the values of the corresponding phases of the junctions or the input magnetic flux. The analytical expressions for these coefficients are bulky, so they are given in Appendix A.

Note that, due to the complex dependence of ${\phi }_1$, ${\phi }_2$ on ${\phi }_{in}$, the currents $i_{\gamma }$ are not, in fact, linear in any of them. The non-linearity of the system comes from the Josephson junctions, whose currents, $I_n$ (where the subscript $n=1,2$ is the index of the junction), can be written as

$$I_n=\frac{\hslash }{2e}{C}_n\ddot{{\phi }_n}+\frac{\hslash }{2e{R}_n}\dot{{\phi }_n}+I_{C_n}sin{\phi }_n.$$

(4)

in the frame of the well-known resistively shunted junction model with capacitance (RSJC) [64].

Here, we consider an energy-efficient circuit consisting of tunnel superconducting–insulator–superconducting (SIS) Josephson junctions with a high normal state resistance $R_n$, so that the second term in Equation (4) becomes negligibly small and, as modelling shows, does not contribute significantly to the overall dynamics of the system and, therefore, can be safely omitted.

After normalisation of (4) by $I_{C_2}$, the equations take the following form:

$$c_n\ddot{{\phi }_n}+i_{C_n}sin{\phi }_n=i_n,$$

(5)

where $i_{C_n}=I_{C_n}/I_{C_2}$ is a dimensionless critical current; $t_C=\sqrt{\frac{\hslash C_2}{2e{I}_{C_2}}}$ is a characteristic time and $\tau =t/t_C$ is a dimensionless time; and $c_n=C_n/C_2$ is a dimensionless capacity. Note that such systems of interacting neurons can also be considered within the framework of the Hamiltonian formalism. As an example, in Appendix B, we present the derivation of the Hamiltonian of the system shown in Figure 2. This approach is quite simple and convenient in the case of scaling the circuit to a larger number of layers in a neural network, as well as for numerical modelling of nonlinear dynamics and further study of the quantum mode of operation of the circuit [55,56], including taking into account the influence of environments.

Solution of the system in Equation (5) gives the transfer characteristics of the input and output neurons as a response to the input magnetic flux in (1). Previous studies of single $S_c$-neurons [48] have shown that the sigmoid activation function can be realised under the following condition: $l_n<\sqrt{l_{outn}^2+1}-l_{outn}\equiv l_n^{*}$ and $l_{an}=l_n+1$. Hence, as in the single-neuron case, we consider values of inductances $l_n<l_n^{*}$ at which there are no plasmonic oscillations in the output characteristics of the first (input) and the second (output) neurons.

In the first step of the analysis, we assume that the coupler inductances should be equal: $l_{t1}=l_{t2}$ and $l_{t3}=l_{t4}$. Figure 3 illustrates the formation of sigmoid activation functions for the input and output neurons under this assumption. It is seen that the current at the output neuron (blue curve in Figure 3) drops by two orders of magnitude; this drawback reflects the difficulty in practical system implementation. It is also necessary to obtain the synapse weights that are at least in the range from −1 to +1, which turns out to be impossible in some situations (see Figure 4).

The above issues imply the need for parameter optimisation. In the next part of the paper, we propose this procedure, which can be generalised to the case of large computing systems.

3. Formulation and Solution of the Optimisation Problem

We consider the optimisation problem of a system of two coupled neurons from the point of view of solving the two problems of synapse weights and neuron response magnitudes mentioned above. However, a closer look at these problems reveals that they are closely related: achieving higher values of weights can potentially increase the response magnitude of the output. Therefore, further actions are aimed at finding a functional to describe the synapse weight as a function of the system parameters and finding its extrema using the gradient descent method. As such a functional, we consider the slope angle of the synapse characteristic $\alpha$, which can be expressed analytically in the following form:

$$tan\alpha =\frac{d\Delta i_s}{dt}/\frac{d{i}_{in}}{dt}=\frac{{\kappa }_s^{(1)}{\dot{\phi }}_1+{\kappa }_s^{(2)}{\dot{\phi }}_2+{\kappa }_s^{(in)}{\dot{\phi }}_{in}}{{\kappa }_{in}^{(1)}{\dot{\phi }}_1+{\kappa }_{in}^{(2)}{\dot{\phi }}_2+{\kappa }_{in}^{(in)}{\dot{\phi }}_{in}}.$$

(6)

When using the gradient descent method, it is necessary to solve a system of differential equations (Equation (5)) at each step, which is the main computational complexity due to the large number of varied system parameters. To overcome this difficulty, we propose several simplifications.

Since dynamic processes in the system are associated with changes in the input flux and, moreover, take place exactly at the rise/fall time intervals, and since the dependence $\Delta i_s(i_{in})$ is linear, it is sufficient to determine the value of the angle in (6) at the inflection point $t_1$ when ${\ddot{\phi }}_{in}(t_1)=0$. Additionally, ${\ddot{\phi }}_1(t_1)={\ddot{\phi }}_2(t_1)=0$ due to the sigmoid activation function. By using this approximation, we obtain the system of equations for ${\dot{\phi }}_1(t_1)$ and ${\dot{\phi }}_2(t_1)$:

$$\left\{\begin{array}{c}{\dot{\phi }}_1(t_1)={\dot{\phi }}_{in}{\eta }^{-1}(\kappa {\kappa }_2^{(in)}-{\kappa }_1^{(in)}{\kappa }_2^{(2)}-{\kappa }_1^{(in)}{i}_{C_2}cos({\phi }_2(t_1))),\hfill \\ {\dot{\phi }}_2(t_1)={\dot{\phi }}_{in}{\eta }^{-1}(\kappa {\kappa }_1^{(in)}-{\kappa }_2^{(in)}{\kappa }_1^{(1)}-{\kappa }_2^{(in)}{i}_{C_1}cos({\phi }_1(t_1))),\hfill \end{array}\right.$$

(7)

where $\eta \equiv -{\kappa }^2+(i_{C_1}cos({\phi }_1(t_1))+{\kappa }_1^{(1)})·(i_{C_2}cos({\phi }_2(t_1))+{\kappa }_2^{(2)})$, reminding one that $\kappa \equiv {\kappa }_1^{(2)}={\kappa }_2^{(1)}$, and the values of ${\phi }_1(t_1)$ and ${\phi }_2(t_1)$ can be found from

$$\left\{\begin{array}{c}{i}_{C_1}sin({\phi }_1(t_1))=-(\kappa {\phi }_2(t_1)+{\kappa }_1^{(1)}{\phi }_1(t_1)+{\kappa }_1^{(in)}{\phi }_{in}(t_1)),\hfill \\ i_{C_2}sin({\phi }_2(t_1))=-(\kappa {\phi }_1(t_1)+{\kappa }_2^{(2)}{\phi }_2(t_1)+{\kappa }_2^{(in)}{\phi }_{in}(t_1)).\hfill \end{array}\right.$$

(8)

By substituting the obtained values of ${\dot{\phi }}_1(t_1)$ and ${\dot{\phi }}_2(t_1)$ into the expression in (6), we obtain an explicit form for $\alpha$ that is dependent on all system parameters. This allows us to implement the gradient descent method to maximise the angle $\alpha$ without directly calculating the dynamics (5). A similar approach allows us to quickly optimise the parameters to maximise the current at the output neuron by using (3).

A visualisation of this method for different initial parameters is shown in Figure 5. We selected several initial sets of system inductances, for which $\alpha$ was calculated using (6) and maximised based on the gradient descent method. The angle $max\alpha$ is non-monotonic with respect to the system parameters and has several local maxima. In Figure 5, we show a section for several trajectories along which the angle $max\alpha$ is maximised in the subspace of inductances $l_{in}$ and $l_{t1}=l_{t2}$, where the arrow indicates the path from their initial values to the optimal ones. It is seen that all curves converge at $l_{t1,2}\to 2$ (which was chosen as an upper boundary value for the inductances $l_{t1\dots 4}$) and $l_{in}\to 0.3$, where a certain local maximum of optimisation is reached for $max\alpha$ and, therefore, for the achievable synapse weights in our system.

Figure 6a shows $\tan (\alpha )$ dependence on the inductance difference $\Delta l_s$ for optimal system parameters found by the gradient descent method. The good agreement between the results obtained from the exact calculation of Equation (5) (the red circles) and by using Equations (6)–(8) (the blue line) indicates the validity of the approximations used. Dependencies of the synapse output current $\Delta i_s$ on the input current $|i_{in}|$ for different values of $\Delta l_s$ are shown in Figure 6b.

The proposed method allows one to abandon the solution of the Hamiltonian system, which is a time-consuming computational task. We reduce the optimisation problem to solving a set of algebraic equations, which significantly reduces the computational time. This approach is promising from the point of view of scaling neural networks and calculating their optimal configuration parameters.

The obtained results demonstrate that the gradient descent method can be used to optimise the parameters of a synapse connecting two neurons. Extending the applicability of the method to more complex systems consisting of a larger number of neurons and synapses is also possible, but may require additional assumptions related to mutual influence of neurons on each other (localisation approximation). Hence, the challenge in the optimisation of the parameters of a large neural network is reduced to solving local problems of finding functionals that are similar to Equation (6) and then fine-tuning the found solutions by the gradient descent method in a multi-parameter space.

4. Circuit Structure Optimisation

The performed parameter optimisation does not eliminate the signal level drop at the output neuron in the considered circuit design (Figure 2). To overcome this problem, we are developing a modification of the circuit in which the magnetic connection between the input neuron and the synapse is replaced by a galvanic connection—see Figure 7.

Within the framework of the proposed approach, gradient descent was applied to the modified scheme to solve the optimisation problems. The analysis of the system showed that the main parameters responsible for the current at the output neuron are the coupling inductances $l_{tj}$ (where $j=1,2,3,4$). Figure 8a shows that we need to minimise the value of the inductance $l_{t4}$ connecting the coupler to the Josephson arm in the output neuron. The direction of the arrows shows the path of the trajectory (from the initial value to the optimal one) for maximising the angle $max\alpha$ during the gradient descent execution. Figure 8b shows the calculation for optimisation of the remaining coupler inductances. It can be seen that all trajectories tend to the values $l_{t3}\to 2$ and $l_{t1,2}\to 0.7$. We calculate the activation functions of the neurons shown in the inset of Figure 8b using these values. By application of the optimisation approach, we are able to significantly increase the current at the output neuron, which is important for the practical implementation of such systems.

After re-optimisation of the parameters, we re-examine the synaptic weights. We analyse the dependence of the current ratio $i_{out2}/i_{out1}$ on $\Delta l_S$, where the input flux reaches a plateau at $t=(t_1+t_2)/2$ see (Figure 9a). It can be seen that we can adjust the sum of the inductances such that the values of the out currents at the input and output neurons coincide (see Figure 9b). Note that the output current of the output neuron can even exceed the output current of the first neuron at small values of $l_{t4}$. Thus, depending on the technological limitations, it is possible to obtain the maximum response at the output layer of the neurons.

5. Analog Implementation of the XOR and OR Logic Elements

The classical XOR (logical inequality operator) element has two inputs and one output. If the input signals do not match, the output is “1”, and “0” otherwise. The basic neural network implementing XOR consists of three neurons (two input neurons and one output neuron). The inputs of the neural network are supplied with signal in the form of smoothed trapezoid “1” or no signal “0”—see Figure 10a. The optimisation problem is reduced to finding such parameters of the system at which the output layer neuron activates according to the XOR truth table. Similar considerations are valid for obtaining a neural network operating according to the OR gate principle.

The discussion of the neural-XOR/OR superconducting circuit based on three adiabatic neurons (shown in Figure 10b) begins with writing down the corresponding system of equations:

$$\left\{\begin{array}{c}{(i_1)}_{in1}+{(i_{a1})}_{in1}+{(i_{out1})}_{in1}=0,\hfill \\ {(i_1)}_{in2}+{(i_{a1})}_{in2}+{(i_{out1})}_{in2}=0,\hfill \\ {({\phi }_1)}_{in1}-\frac{{\phi }_{in1}}{2}+{(i_1{l}_1)}_{in1}={(i_{out1}{l}_{out1})}_{in1}+{(i_{s1}{l}_{s1})}_{in1}+{(m_2)}_{in1}{i}_{circ},\hfill \\ {({\phi }_1)}_{in2}-\frac{{\phi }_{in2}}{2}+{(i_1{l}_1)}_{in2}={(i_{out1}{l}_{out1})}_{in2}+{(i_{s1}{l}_{s1})}_{in2}+{(m_2)}_{in2}{i}_{circ},\hfill \\ {({\phi }_1)}_{in1}-\frac{\phi in1}{2}+{(i_1{l}_1)}_{in1}={(i_{out1}{l}_{out1})}_{in1}+{(i_{s2}{l}_{s2})}_{out1}-{(m_2^{*})}_{in1}{i}_{circ},\hfill \\ {({\phi }_1)}_{in2}-\frac{{\phi }_{in2}}{2}+{(i_1{l}_1)}_{in2}={(i_{out1}{l}_{out1})}_{in2}+{(i_{s2}{l}_{s2})}_{in2}-{(m_2^{*})}_{in2}{i}_{circ},\hfill \\ {({\phi }_1)}_{in1}-\frac{{\phi }_{in1}}{2}+{(i_1{l}_1)}_{in1}={(i_{a1}{l}_a)}_{in1}+\frac{{\phi }_{in1}}{2},\hfill \\ {({\phi }_1)}_{in2}-\frac{{\phi }_{in2}}{2}+{(i_1{l}_1)}_{in2}={(i_{a1}{l}_a)}_{in2}+\frac{{\phi }_{in2}}{2},\hfill \\ {(i_{out1})}_{in1}-{(i_{s1})}_{in1}-{(i_{s2})}_{in1}=0,\hfill \\ {(i_{out1})}_{in2}-{(i_{s1})}_{in2}-{(i_{s2})}_{in2}=0,\hfill \\ ({(l_{t1})}_{in1}+{(l_{t2})}_{in1}+{(l_{t1})}_{in2}+{(l_{t2})}_{in2}+{(l_{t3})}_{out1}+{(l_{t4})}_{out1})i_{circ}=\hfill \\ (-{(i_{s1}{m}_2)}_{in1}-{(i_{s1}{m}_2)}_{in2}+{(i_2{m}_3)}_{out1}+{(m_2^{*}{i}_{s2})}_{in1}+{(m_2^{*}{i}_{s2})}_{in2}-{(m_3^{*}{i}_{a2})}_{out1}),\hfill \\ {(i_2)}_{out1}+{(i_{a2})}_{out1}+{(i_{out2})}_{out1}=0,\hfill \\ {({\phi }_2)}_{out1}-{(m_3)}_{out1}{i}_{circ}+{(i_2{l}_2)}_{out1}={(i_{out2}{l}_{out2})}_{out1},\hfill \\ {({\phi }_2)}_{out1}-{(m_3)}_{out1}{i}_{circ}+{(i_2{l}_2)}_{out1}={(i_{a2}{l}_{a2})}_{out1}+{(m_3^{*})}_{out1}{i}_{circ},\hfill \end{array}\right.$$

(9)

where we preserve the notations according to Figure 7, but with the subscripts for the neurons in the input ($in1,2$) and output ($out1$) layers (a more detailed scheme with all designations can be found in Appendix C). The input signals defined by expression (1) are denoted accordingly as ${\phi }_{in1,2}$.

Solving the optimisation problem for the system in Equation (9) makes it possible to configure the neural network to be capable of operating both as an XOR or as an OR logic element, which is quite expected. An obvious choice for such a neural network configuration is the choice of weight coefficients: they should be asymmetric for XOR, and, on the contrary, they should be symmetric for OR implementation. By solving the system in Equation (9) that describes the circuit shown in Figure 10, the truth tables for XOR/OR network implementations were obtained and are presented in Figure 11. The case when there is no signal at the input of both input neurons is not shown: if there is no signal at both inputs of the circuit, there is no signal at the output as well.

One point is worth mentioning regarding the proposed implementations of the neural networks. Here, the XOR output can be of both positive and negative polarity (see Figure 11). The OR output with “1” in both inputs is twice as large as that with inputs “1” + “0” or “0” + “1” (see Figure 11). This is in contrast to the digital implementations, where the output can be “0” or “1” only.

6. Conclusions

In this paper, we demonstrate an optimisation algorithm for the parameters of adiabatic neural networks. The algorithm allowed us to find the optimal values for operation of the circuits with different combinations of synapses and neurons, including the ones mimicking logical XOR and OR elements. In addition, a generalisation of this algorithm to neural networks of higher dimensionality, consisting of superconducting $S_c$-neurons and synapses, was discussed.

It should be noted that, even in the development of such simple neural networks, we faced a significant signal decay problem. For larger neural networks, the solution may imply an addition of magnetic flux amplifiers (boosters), well-known in adiabatic superconducting logic [57]. The utilisation of an analogue–digital (and, apparently, optical-superconducting) approach for the network implementation is another option.

Regarding the experimental feasibility of the presented schemes, there are a number of experimental works [49,65,66,67] that use a similar technique for the fabrication of Josephson junctions and demonstrate their critical currents in the range of 50 to 150 μA, corresponding to characteristic values of inductance magnitudes at the level of 2.2–6.6 pH. This confirms the experimental feasibility of the design considerations presented.