A Method for Evaluating Chimeric Synchronization of Coupled Oscillators and Its Application for Creating a Neural Network Information Converter

Abstract

This paper presents a new method for evaluating the synchronization of quasi-periodic oscillations of two oscillators, termed “chimeric synchronization”. The family of metrics is proposed to create a neural network information converter based on a network of pulsed oscillators. In addition to transforming input information from digital to analogue, the converter can perform information processing after training the network by selecting control parameters. In the proposed neural network scheme, the data arrives at the input layer in the form of current levels of the oscillators and is converted into a set of non-repeating states of the chimeric synchronization of the output oscillator. By modelling a thermally coupled VO₂-oscillator circuit, the network setup is demonstrated through the selection of coupling strength, power supply levels, and the synchronization efficiency parameter. The distribution of solutions depending on the operating mode of the oscillators, sub-threshold mode, or generation mode are revealed. Technological approaches for the implementation of a neural network information converter are proposed, and examples of its application for image filtering are demonstrated. The proposed method helps to significantly expand the capabilities of neuromorphic and logical devices based on synchronization effects.

1. Introduction

Artificial neural networks are actively used in image and speech recognition applications [1,2], as well as in computer calculations [3] and data coding [4]. The functional importance of synchronization in information processing has stimulated the development of neural network models with oscillatory dynamics and neuromorphic algorithms based on synchronization effect.

In 2002, Kuramoto and Battogtokh reported that arrays of non-locally related oscillators could spontaneously divide into synchronized and desynchronized subpopulations [5]. This amazing discovery challenged the previous belief that the connected identical oscillators are either synchronized or will work incoherently and chaotically. Since the network had a hybrid nature uniting both coherent and non-coherent parts, it was proposed to call such states chimera, because of their resemblance to mythological Greek animals, assembled from incomparable parts [6]. Recent studies have demonstrated that chimera states are not limited to phase oscillators, but can be found in a wide variety of different systems and are observed in space-time dynamics. It is worth mentioning the studies on chimeric states in networks of Kuramoto phase oscillators [7], and leaky integrate-and-fire [8] FitzHugh-Nagumo [9] and Hindmarsh-Rose [10] models. In a previous study [11], chimera states are classified into stationary, turbulent, and breathing patterns, which, in turn, are divided into subclasses. In addition, a separation of amplitude and phase chimeras is identified [12,13]. Such a wide range of gradations is due to the diversity of the spatial and temporal network dynamics.

Consequently, the term “chimera states” has been well established in the literature. It characterizes an array of connected oscillators, where some of the oscillators operate in synchronous mode, and other oscillators function in asynchronous mode [14]. However, alternative terms of this phenomenon are found in the literature, for example, in a previous study [15], this behavior is called cluster synchronization.

In this paper, the term “chimeric synchronization” is used to refer to the synchronization effect and to describe the coexistence of synchronous and asynchronous states of coupled oscillators in the time domain. Chimeric synchronization is the phenomenon of synchronization of quasi-periodic oscillations [16,17,18,19,20,21]. Quasi-periodic oscillations can be observed when crossing the boundaries of the Arnold tongue, describing high order synchronization, during a transition from the synchronous to asynchronous state. A number of authors [16,17,18,19,20,21] have studied the synchronization of quasiperiodic oscillations; however, no specific methodology for evaluating this phenomenon for technical applications exists.

To determine the type of chimera state and for estimating the synchronization of quasi-periodic oscillations, the degree of synchronization is evaluated by Lyapunov spectra [12,22] or by calculating the complex order parameter [23]. However, these indicators do not contain information about the complex structure of synchronization, which, in the case of chimeric synchronization, consists of a set of different individual synchronous patterns.

Therefore, the development of methods for assessing chimeric synchronization and its application in practice is an important task. In addition, the author makes an assumption about the relationship of the formation of chimeric states and the phenomenon of chimeric synchronization, which is discussed in the paper results.

A class of oscillatory neural networks (ONN) can be identified, where the basic elements are relaxation oscillators that generate sequences of pulses (spikes), and ONN can encode information at pulses repetition rate. Such ONN are interesting due to simplicity of hardware implementation, as developed micro- and nano-electronic autogenerators ensure network compactness and energy efficiency. In addition, a pulsed-type ONN, where the periodic oscillation spectrum has a multi-frequency character, has a special mode of harmonics synchronization, or in other words, has a high order synchronization effect [24,25,26]. This effect has been demonstrated experimentally using the example of thermally coupled VO₂ oscillators [24]. In relaxation generators with elements based on vanadium dioxide film, oscillations are initiated by the electric switching effect caused by the metal-insulator phase transition [27].

Oscillators based on VO₂ structures are chosen as ONN elements due to the high speed of electrical switching (~10 ns) [28], high degree of nanoscale in manufacturing [29], and, most importantly, the presence of a significant thermal coupling effect that simplifies the ONN layout and circuitry of galvanically isolated oscillators [24]. The thermal coupling effect, having a local nature, makes the network resemble a cellular neural network (CNN) [3,30] and opens up 3D integration possibilities. For these reasons, VO₂-oscillators are actively used in prototyping of neuro-oscillators for the tasks of cognitive technologies.

The high-order synchronization of two oscillators can be characterized by a family of metrics—the ratio of subharmonics (SHR) and synchronization effectiveness η [26]. These metrics are used to create a neuromorphic device for pattern recognition. Synchronization is measured relative to the reference oscillator, which has a constant pulse generation frequency; this makes it possible to compare the synchronization of all oscillators in the network with each other. Although the system dynamics during synchronization can be described only by SHR and η value, there are a number of examples where oscillograms contain sections with different synchronization, termed chimeric synchronization in this paper. Chimeric synchronization appear quite often in the model, however, it does not affect operation of the device and belongs to an asynchronous type of oscillation. For example, Figure 1 shows the current pulses of two oscillators with numbers 0 and 1, with alternating synchronization patterns defined by two parameters SHR¹_0,1 = 4:3 and SHR²_0,1 = 9:7. The dotted lines mark the synchronization moments (phase locked) of the individual pulses.

Such an alternation of the synchronization pattern is a stable state in time and is characterized by a periodic jump of the phase of synchronous pulses.

In this paper, we describe the method for classification of chimeric synchronization and use it to analyze the dynamics of six thermally coupled VO₂ oscillators as applied to the implementation of a neural network converter.

2. Materials and Methods

We describe the classification method of chimeric synchronization at the beginning of this section, followed by a description of the single-oscillator circuit, ONN structure, formulation of a search problem and network training technique, and technical aspects of thermal coupling organization.

2.1. Chimeric Synchronization Classification

The method of determining the family of metrics of high-order synchronous states (SHR and µ) is described in detail in a previous study [26]. The method of classification of chimeric synchronization is based on its basis and is not significantly different.

At first, the analog oscillogram of oscillations is represented in the form of the corresponding array, called LE, which stores information on the position of the current pulse leading edges (see Figure 2).

Then, for two arbitrary oscillators i and j, between which it is necessary to determine synchronization, the arrays LE[i][t] and LE[j][t] are compared (see. Figure 3). The distance between the two nearest phase-locked pulses is denoted as T^z_s, the period of synchronization (where z is a conditional number of periods T_s). Pulses are considered phase-locked if the distance between them does not exceed 4Δt. The value of sub-harmonic ratio between the oscillator i and the oscillator j, called SHR_i,j, may be estimated using a phase-locking method:

$${\mathrm{SHR}}_{\mathrm{i},\mathrm{j}}=M_{\mathrm{j}}:M_{\mathrm{i}}$$

(1)

where M_i and M_j are numbers of signal periods falling into the synchronization periods T^z_s of two oscillators. Equation (1) defines the pattern of high order synchronization of two signals. As the notation “SHR” defines, the formula can determine the ratio of subharmonic numbers in the signal spectra at a common synchronization frequency [26].

In general, especially when a system behaves erratically, synchronization periods differ and spread in T^z_s ≠ T^z+1_s and the values of M_i and M_j may change within one oscillogram (see Figure 4).

Various values of synchronizations SHR_i,j may occur within one oscillogram. To determine the distribution of the SHR_i,j, it is necessary to find the occurrence probabilities P(M_j : M_i) for each pair (M_i : M_j) that are present in the whole oscillogram. To find the probabilities P(M_j : M_i), we can count how many times NP(M_j : M_i) the given pair appeared within the whole oscillogram of the oscillator i, multiply this by the number of periods in it (M_i), and divide this by the total number of all oscillation periods in the given signal (N_j). Thus, for P(M_j : M_i) we obtain:

$$P(M_j:M_i)=100\%\cdot NP(M_j:M_i)\cdot M_i/N_i$$

(2)

where N_i is the total number of periods in the oscillogram of oscillator i.

Therefore, each synchronization value SHR_i,j will correspond to the probability of its detection P (M_j : M_i), expressed as a percentage.

It is convenient to present the probabilities P(M_j : M_i) as a histogram, where the values are positioned in the descending order of the magnitude P. For example, for the oscillogram section in Figure 4, the following histogram can be performed:

The histogram in Figure 5 is calculated by Equation (2), when the pairs occur a number of times NP(2:7) = 2, NP(2:9) = 1, NP(2:5) = 1, and the total number of periods is N_i=28 (in real calculations, N_i was in the range of 1000–3000 for greater accuracy [26]).

In a model experiment, the shapes of the distribution oscillograms P(M_j : M_i) can differ significantly from each other. For example, Figure 6 presents the main histogram variants occurring during signal processing (oscillator i corresponds to the reference oscillator with a constant frequency).

The histogram in Figure 6a corresponds to the case of an absolutely synchronized signal with high order synchronization SHR_i,j = 11:8. The spectrum of oscillator j has a line character, and the phase diagram corresponds to a single high order synchronization limit cycle. The cases in Figure 6b,c have a set of different SHR_i,j and correspond to chimeric synchronization states.

A chimeric index, called CH, can be introduced, consisting of the first three values of the synchronization value SHR_i,j:

CH_i,j = (SHR¹_i,j, SHR²_i,j, SHR³_i,j)

(3)

Accordingly, in Figure 6, each histogram displays its values: (a) CH_i,j = (11:8), (b) CH_i,j = (3:2 7:5), and (c) CH_i,j = (11:7 24:17 8:5). For Figure 6a, the values of CH_i,j and SHR_i,j are the same.

For CH, the parameter of synchronization effectiveness η is defined as the sum of P (M_j, M_i) for the first three values in the histogram:

$$\eta ={\displaystyle \sum _{k=1}^3{P}^k(M_j:M_i)}$$

(4)

where k is the sequence number P(M_j, M_i) in the histogram.

Therefore, for each histogram (see Figure 6), we define the effectiveness: (a) η = 97.8%, (b) η = 99.7%, and (c) η = 74.8%.

The new family of metrics (CH_i,j, η) allows sufficient determination of the synchronization states of two oscillators, and classification of chimeric synchronization. Depending on the task, for example, the network training for data coding and pattern recognition, the problem of the presence or absence of synchronization can be solved by formally setting the synchronization effectiveness threshold η_th, so

$$\mathrm{signals}\mathrm{are}\{\begin{array}{l}\mathrm{synchronized},\mathrm{if}\eta \ge {\eta }_{th}\\ \mathrm{not}\mathrm{synchronized},\mathrm{if}\eta <{\eta }_{th}\end{array}$$

(5)

In the majority of cases, we set η_th = 90%, meaning the signals are synchronized if 90% of their durability has a certain synchronization pattern or a set of patterns of the chimeric synchronization. For the network training, this parameter can be selected within a selected range, and it is one of the important parameters of the network adjustment [25].

Let us discuss the reasons for the introduction of the concept of CH index chimera synchronization. For signals in Figure 6b, the synchronization is CH_i,j = (3:2 7:5) and η = 99.7%, thus, the signals are clearly synchronized (η > 90%) and have two patterns of synchronization. The spectrum of oscillator j has a linear character, and the phase diagram of voltages on oscillators has a complex, but not chaotic, attractor, most likely consisting of two limit cycles. If the technique [26] and only the concept of basic synchronization SHR_i,j, are applied, then the family of metrics would look like SHR_i,j = 3:2 and η = 54%. As a result, oscillators would be defined as not synchronized, since η < 90%. However, an accurate calculation of the chimeric synchronization value using the proposed metric allows more accurate and complete characterization of the synchronization state. In addition, such a metric can significantly expand the capabilities of neuromorphic and logical devices that operate on the synchronization effect.

For signals in Figure 6c, the synchronization is CH_i,j = (11:7 24:17 8:5) and η = 74.8%, so the signal is weakly synchronized, and at η_th = 90%,, it is not formally synchronized, as in Equation (5). This is confirmed by the type of phase trajectory that fills the entire phase space. In addition, the oscillator j spectrum is wide, contains many harmonics, and is close to the noise spectrum by its nature.

The main technical problem we faced was the problem of defining the synchronization between the reference oscillator No.0 and the oscillator of the output layer No.5 characterized by the values SHR_0,5 and CH_0,5:

$$\begin{array}{l}{\mathrm{SHR}}_{0,5}=M_0:M_5~M_0/M_5\\ {\mathrm{CH}}_{0,5}=({\mathrm{SHR}}^1{}_{0,5},{\mathrm{SHR}}^2{}_{0,5},{\mathrm{SHR}}^3{}_{0,5})\end{array}$$

(6)

Two parameters CH_0,5 and η are used as the main metrics for evaluation the degree of the two oscillators’ synchronization and are applied in the algorithm of ONN training.

Current oscillograms I_sw(t) of oscillators No.0–5 were calculated simultaneously and contained ~250,000 points with time interval Δt = 1 µs. Then, the oscillograms were automatically processed.

2.2. Method of Chimeric Synchronization Color Mapping

To represent the value of chimeric synchronization CH_i,j, we chose the RGB color display method, since the value of CH_i,j contains three components, as in Equation (6). Each color component is represented by the following algorithm:

$$\begin{array}{l}{\mathrm{CH}}_{\mathrm{i},\mathrm{j}}\to \mathrm{RGB}(\mathrm{red},\mathrm{green},\mathrm{blue})\\ {\mathrm{CH}}_{\mathrm{i},\mathrm{j}}({\mathrm{SHR}}^1,{\mathrm{SHR}}^2,{\mathrm{SHR}}^3)\to \mathrm{RGB}({\lambda }^1,{\lambda }^2,{\lambda }^3)\\ \\ {\lambda }^k=\{\begin{array}{l}{100+(\mathrm{SHR}}^k/{\mathrm{SHR}}_{\max })\cdot 100,{\mathrm{if}\mathrm{SHR}}^k\ge 1\\ 100-\left(\right(1/{\mathrm{SHR}}^k)/{\mathrm{SHR}}_{\max }))\cdot 100,{\mathrm{if}\mathrm{SHR}}^k<1\end{array}\\ k=1\dots 3\end{array}$$

(7)

where SHR_max is the maximum value of SHR on the graph, and k is an index denoting the sequence of SHR values in the formula for CH (6).

For example, CH (1:3 3:1 3:2) with SHR_max = 3 is converted to RGB (0, 200, 150).

2.3. Oscillator Circuit

A model diagram of a single oscillator consists of a current source I_p, a capacitance C connected in parallel with the VO₂ switch, and a noise source U_n (Figure 7a). The capacitance C remains constant C = 10nF, while I_p and U_n vary in the following ranges I_p (435–1220 μA), U_n (0–10 mV). The noise source simulates external or internal circuit noise, for example, switch current noise manifested in fluctuations of switch threshold voltages [31]. I_sw and U denote the current passing through the VO₂ switch and the voltage on it, respectively. The model current-voltage characteristic of the VO₂ switch is shown in Figure 7b. All model switches without coupling have the same I-V characteristic, with stationary natural parameters U_th = 5V, I_th = 550 μA (threshold voltage and current), U_h = 1.5 V, I_h = 1039 μA (holder voltage and current), U_cf = 0.82 V (cutoff voltage). Model curve I_sw = f(U) has high-resistance (OFF) and low-resistance (ON) segments with corresponding dynamic resistances R_off = 9100 Ω and R_on = 615 Ω. A more detailed description of the model circuit of a coupled oscillator-based neural network and methods for calculating oscillations in a circuit are given in a previous study [26]. The system of differential equations for the current in the circuit were numerically calculated with respect to time at regular intervals Δt =1 µs.

Tangibly, the operation of the circuit can be enacted as periodic charging and discharging of the capacitor, when the operating point of the circuit is kept in the negative differential resistance (NDR) section, at the expense of the current source I_p.

Examples of the calculated oscillogram sections for voltage U and current I_sw for the reference oscillator are shown in Figure 8a.

The dependence of the own oscillations frequency F₀ on the magnitude of the current I_p is shown in Figure 8b. Own oscillations occur when the circuit operation point lies in the region of negative differential resistance, in the current range of I_th ≤ I_p ≤ I_h (550 μA ≤ I_p ≤1105 μA), with limit frequencies of 1640 Hz and 10130 Hz. The maximum frequency of 12850 Hz corresponds to a current I_p = 1039 μA. In this way, an increase in the supply current I_p initially leads to an increase in frequency, due to a decrease in the charging time of the capacitor C to the threshold turn-on voltage U_th, then the frequency decreases due to an increase in the discharge time of the capacitor to the holder voltage U_h.

By setting the supply current below I_p < I_th or above I_p > I_h, the operating point of the circuit is set to the sub-threshold state, where own generation is absent and the switch is always either off or on, respectively (see Figure 8b). Further, we will call such an oscillator operation mode a sub-threshold oscillator operation mode.

2.4. ONN Structure

The studied ONN consists of a reference oscillator (No.0), whose frequency does not change, the input layer in the form of a one-dimensional matrix, each element of which is represented by one VO₂ oscillator (No.1–4), and the output oscillator (No.5) (Figure 9). Data in the form of binary four-digit numbers is transmitted to a layer of input oscillators in such a way that each oscillator of this layer is associated with two values of supply currents: I_p = I_OFF represents logical 0, or I_p = I_ON represents logical 1. The numbers can be associated with two-dimensional images, as shown in Figure 9. At the system output, the synchronization order of the output oscillator relative to the reference oscillator is expressed either by the high order synchronization SHR_0,5 or by the chimeric synchronization index CH_0,5 (see formula 6). Thus, four-digit numbers are converted to the synchronization state of the output oscillator.

The thermal effects of oscillators No.1–5 on each other in the circuit in Figure 9 are set to be mutually equivalent, that is, Δ_i,j = Δ_j,i, while the reverse effect on the reference oscillator (No.0) from other oscillators are not taken into account: Δ_1,i ≠ 0, Δ_i,1 = 0. Thus, the reference oscillator generates the thermal pulses and sets the rhythm of the entire ONN with a constant frequency.

The switch parameters are unchanged in numerical simulation, while current intensities I_{p_i} (I_ON, I_OFF, I_{p_0,} I_{p_5}), coupling strength constants Δ_j,i, noise amplitude U_n, and η_th vary.

2.5. Task Setting And Technique of ONN Training

One of the tasks that can be set for the network is to perform conversion functions. When 16 variants of input signals are applied, the output oscillator can take different synchronization values, including showing no synchronization if η < η_th, as in Equation (5). Denoting the total number of synchronous states at the output as x, x can take a value from 1 to 16. Among the x synchronous states, there can be n unique synchronization values. Then, the output can be codified as “n of x”. In the case where all of the output synchronization values are unique (n = x), this corresponds to the result “x of x”.

We have divided the network responses into 16 variants, corresponding to the condition when a certain number of synchronous states occur and none of them repeat. For example, the task “1 of 1” corresponds to one synchronous state at the output, all other states are not synchronous, and “2 of 2” corresponds to two different states (see explanatory

Table 1. Example of network responses for the “x of x” task. Empty cells correspond to the absence of synchronization by criterion of Equation (5). The responses are taken at high order SHR_0,5 and chimeric CH_0,5 synchronization.

	High Order Synchronization			Chimeric Synchronization
	“1 of 1”	“2 of 2”	“16 of 16”	“1 of 1”	“2 of 2”	“16 of 16”
Input data	SHR0,5	SHR0,5	SHR0,5	CH0,5	CH0,5	CH0,5
0000	-	-	1:1	(3:4)	-	(7:8 11:12 3:3)
0001	-	-	9:7	-	-	(27:29 19:21 9:10)
0010	-	-	9:8	-	-	(23:28 4:5 5:6)
0011	-	-	10:7	-	-	(5:6 5:7 12:17)
0100	-	-	14:11	-	-	(26:28 25:27 13:15)
0101	-	1:1	3:2	-	(13:11 6:5)	(46:48 25:27 6:7)
0110	-	-	5:3	-	-	(4:5 3:4 5:6)
0111	-	-	8:5	-	(19:17 29:26)	(8:10 5:6 9:11)
1000	-	-	20:17	-	-	(5:6 7:8 1:1)
1001	-	-	2:1	-	-	(42:43 39:40 17:19)
1010	-	-	11:7	-	-	(5:7 12:17 9:11)
1011	-	-	15:7	-	-	(5:7 12:17 10:12)
1100	-	-	15:8	-	-	(9:10 13:14 1:1)
1101	-	7:8	11:5	-	-	(9:10 6:7 3:3)
1110	-	-	19:9	-	-	(11:15 8:11 5:7)
1111	4:3	-	5:2	-	-	(8:11 5:7)

).

The most difficult task is the case of “16 of 16”, when each input state corresponds to its unique synchronous state, expressed by one of two indices, SHR_0,5 or CH_0,5 (see Equation (6)).

The possible number of representations of the task “x of x” is equal to the number of combinations x out of 16 (${\mathrm{C}}_{16}^x$). Therefore, the highest probability to find a solution is for x = 8, and the smallest probability is for x = 16 at ${\mathrm{C}}_{16}^{16}=1$ (see Figure 10). However, as it will be shown in the results, the type of distribution of the found solutions differs significantly from the distribution in Figure 10 and is determined by the network parameters. The total number of options for presenting tasks “x of x” is $\sum {\mathrm{C}}_{16}^x+1=2^{16}=65536$, where the absence of synchronization with any input data is taken into account. The total number of network response options is much higher if we take into account answers of the type “n of x”.

As ONN with the high order synchronization effect has not been studied before, there are no established methods for network training. One of the ways is to use the Simulated Annealing Algorithm (SAA) [1] for the network parameter selection: currents (I_ON, I_OFF, I_{p_0}, I_{p_5}), coupling strength Δ_j,i, noise amplitude U_n, and synchronization effectiveness threshold η_th. The algorithm’s key point is the random searching of problem solutions at some initial interval of parameters followed by narrowing of these intervals. In the majority of cases considered in the article, we used only a random search in a given range of parameters. In some cases, a better solution could be found by gradient descent near the found solution. The random search algorithm was tested in a previous study [26] and showed its effectiveness, since a feature of the neural network is the presence of a set of solutions (a set of system parameters) to satisfy the answer for the task “x of x”. Therefore, in the results, we present a distribution histogram of the number of solutions, called NS, from the value x.

The values of I_ON and I_OFF currents set the logic levels 1 and 0 by determining the supply current of the corresponding input oscillator. The range of current variation is (435–1220 µA). This range is wider than the range of existence of own oscillations (550–1105 µA) defined by Figure 8b in Section 2.1. If we represent a pair of currents (I_ON and I_OFF) in the diagram by the point, as in Figure 11, then the central area 1 corresponds to the mode where both logic levels make the oscillators oscillate. In other words, when applying either 1 or 0, the oscillator is in the generation zone, and regardless of the input signal, all oscillators oscillate. In area 3, currents I_ON and I_OFF lead the oscillator into sub-threshold mode, while the switch is either ON or OFF.

Area 2 corresponds to the case when one of the current levels I_ON or I_OFF leads to generation, while the other level sets the oscillator to the sub-threshold mode.

For example, point A in Figure 11 corresponds to currents I_ON = 894 µA and I_OFF = 958 µA, and both levels of current lead to oscillation. At point B (I_ON = 1153 μA and I_OFF = 1036 μA), a logical 1 (I_ON) leads the oscillator to enter the threshold mode, while the switch is in the on mode, its resistance is R_ON, and a large current passes through it. In turn, a logical 0 (I_OFF) leads to the generation of oscillations. Point C (I_ON = 1163 µA and I_OFF = 459 µA) corresponds to the mode when both 1 and 0 set the oscillator to the sub-threshold mode, while at 1 the switch is on, and at 0 the switch is off.

In the sub-threshold mode, the switch is turned on and heats the neighboring areas. Therefore, the switch affects the threshold voltages of the neighboring areas and plays the role of a constant displacement neuron. In the model, this is inherently taken into account, as the influence of oscillators is affected by the state of the switches (see Equations (A1)–(A3) in the previous study [26]).

The range of variable currents is chosen in a way that area 1 (pink space) is equal to the sum of areas 2 (blue) and 3 (brown). Therefore, with a random choice of a point in the space of the currents I_OFF and I_ON, the probabilities of falling into the region of oscillation generation and into the region of the sub-threshold state of the oscillators are equal, which allows us to compare histograms of the solutions NS.

The ONN was set up by brute force, searching the thermal coupling strength Δ_i,j and power supply levels (I_OFF and I_ON) of the input layer oscillators (No.1–4). The random search was performed in the ranges of 0–1 V for Δ_i,j (except Δ_0,j = 0.2 V) and 435–1220 µA for I_OFF and I_ON. The values of the supply currents of the reference I_{p_0} and output I_{p_5} oscillators, and the noise level U_n were fixed. The number of samples was 10⁵.

In the current study, the target was to find solutions “x of x” defining the operation of the neural network converter, as well as the use of this scheme for filtering images (see Section 3.6). Other tasks may require different network functions. For example, in the task of driving a vehicle, proximity sensors can input the signals of approaching an obstacle, and output synchronization would determine in which direction the vehicle should turn.

2.6. Method of Oscillators Coupling and Variants of Experimental Implementation

The main model elements of the oscillator network are VO₂ switches, which are two-electrode planar structures with a functional layer of vanadium dioxide and two metal contacts. The interaction between the oscillators is carried out by heat flows propagating through the substrate, resulting from the Joule heating when the switches are turned on. This method of coupling was experimentally demonstrated in a previous study [24].

The interaction of oscillators through thermal coupling assumes a reduction in the threshold switching voltage of each switch U_th by a coupling strength value Δ_i,j when the switches thermally affect each other at the moment of capacitance C discharge and the release of Joule heat. A detailed mathematical model of thermal coupling of VO₂ oscillators is given in a previous study [26].

An exemplary view of the location of the switches on the substrate is shown in Figure 12. In the study, we modeled the reference oscillator, which affected all other oscillators, and unidirectional thermal couplings. The question may arise of how to implement unidirectional thermal coupling in a real experiment. The task of the technical implementation of the proposed model objects is the subject of a separate study; nevertheless, possible variants of one-way thermal coupling between two oscillators are presented in Figure 13. In the diagram (Figure 13a), the connection is carried out through an additional current resistor included in the circuit and located on the substrate along with switches. Its resistance does not depend on temperature, however, when heated by the current, it transmits a thermal effect from oscillator 0 to oscillator 1. Therefore, the implementation of one-way thermal coupling is possible, in principle. In addition, the amplitude of the thermal interaction signal decays exponentially with the distance between the switches; see detailed descriptions of the physics of thermal coupling in previous studies [24,32]. Therefore, the one-way connection can be accomplished by making switches with different radii of thermal exposure R_TC, as in Figure 13b.

The thermal coupling is chosen due to simplicity of modelling in the case of a fully connected circuit of oscillators; oscillators are galvanically isolated, and this type of coupling has been confirmed experimentally.

In addition to thermal coupling, oscillators can be connected electrically via resistors or capacitors [33], optically [34,35], and via wireless channels [36], however, the method for determining chimeric synchronization is universal, regardless of the coupling type and the physics of the oscillators.

3. Results

3.1. Investigation of the Chimeric Synchronization of Two Coupled Oscillators

Let us consider how the distribution of synchronous states of two oscillators changes in the space of supply currents with the parameters η_th = 90%, U_n = 1 mV, Δ_0,5 =0.2 V, Δ_5,0 =0 V (Figure 14a). Oscillators are connected by one-way coupling; oscillator No.0 affects oscillator No.5, and oscillators No.1–4 in the diagram in Figure 14a are disabled. Figure 14b shows the synchronization distribution in the form of Arnold tongues, when the chimeric synchronization is not taken into account (the colors correspond to the procedure described in Section 2.2).

Arnold tongues have a linear extended form, however, they are transformed at high currents and shape regions with alternating synchronization (indicated by a dotted line). |This behavior is explained by the nonlinear dependence of the frequency on the supply current, as shown in Figure 8b. In addition, a well-known phenomenon is observed when synchronization occurs predominantly in areas where the frequency of oscillator No.0 is greater than the frequency of oscillator No.5. This is clearly seen for synchronization SHR_0,5 = 1:1, which occurs predominantly above the diagonal, as seen in Figure 14b. The unidirectional action of oscillator 0 is more effective in this case, as it leads and initiates oscillations in the adjacent circuit.

Application of the technique described in Section 2.1 makes areas with chimeric synchronization visible, indicated by the gray color in Figure 14c. For example, the state CH_0,5 = (21:20 20:19 1:1) occurs at the currents I_{p_0} = 948 µA, I_{p_5} =1004 µA, and the synchronization efficiency η = 98%. In general, the areas with detected synchronization with the efficiency η ≥ η_th are increased, due to the appearance of areas with chimeric synchronization. Now, the areas previously diagnosed as nonsynchronous operating modes of oscillators have a well-defined classification with chimeric synchronization, and can be used in creation of logical and neuromorphic devices.

3.2. The Study of the Neural Network Information Converter, without Accounting for Chimeric Synchronization

The ONN under investigation has the structure described in Section 2.5. The output oscillator’s state is determined by the conventional synchronization index SHR_0,5. The training was carried out according to the method described in Section 2.6, with the number of samples set as 10⁵.

The results of the distribution of the number of NS solutions for the “x of x” problem after training are shown in Figure 15a. Four curves are presented; the curve (“whole area”, black marker) corresponds to the entire space of the I_ON-I_OFF currents, and the remaining curves correspond to different areas of the I_ON-I_OFF space (see Figure 11). The calculation was performed at a noise level of U_n = 0 mV, η_th = 30% and I_{p_0} = 1039 µA, I_{p_5} = 750 µA.

It can be seen that the solution to the “16 of 16” problem is achieved only in regions 2 and 3 (see the section marked by the dotted line NS^2,3(16)~100), when one of the oscillators is in the sub-threshold mode. No solution NS¹(16) = 0 is found in area 1. The maximum number of solutions belongs to the “13 of 13” task, corresponding to area 2 (NS²(13)~400). For area 1, the maximum number of solutions falls within the task “9 of 9” (NS¹(9)~50). In area 2, there are more solutions than in all other areas, which means the circuit has the greatest potential when one of the currents (I_ON or I_OFF) sets the oscillator into the threshold state, and the other current creates the generation mode of the oscillator.

The type of distribution depends on a number of parameters, such as U_n, η_th, I_{p_0}, I_{p_5}. When adding noise to the system U_n = 1 mV (Figure 15b), the number of solutions to the “16 of 16” problem significantly decreases (NS^2,3(16)~10), but the number of solutions with a low x significantly increases. Therefore, noise can increase the number of solutions, which is associated with the decrease of solutions in one range and an increase in another. Apparently, there is an optimal noise value when the maximum number of solutions is observed (since with an unlimited increase in noise, the number of solutions will obviously decrease), and this effect is similar to the stochastic resonance effect observed in a previous study [26].

3.3. The Study of the Neural Network Information Converter, Accounting for Chimeric Synchronization

The distribution results of the number of NS solutions to the “x of x” problem after training are presented in Figure 16a. Inclusion of chimeric synchronization leads to an increase in the NS maximum by more than 10 times, and the number of solutions to the “16 of 16” problem increases by 30 times. The greatest number of solutions, as in the previous case, falls on area 2. Another interesting result is the appearance of solutions in area 1, where all generators are active at any input data.

Hence, accounting for chimeric synchronization significantly increases the probability of finding solutions.

Four curves are presented; the curve (black marker) corresponds to the entire space of the I_ON-I_OFF currents, and the other curves correspond to different areas of the I_ON-I_OFF space (see Figure 11). The distribution of solutions to the “16 of 16” problem in the I_ON-I_OFF space is presented in Figure 17a. The distribution is symmetrical to the diagonal and concentrated in area 2. No solutions exist when both currents are greater than 1105 mA or less than 550 mA. In this case, both the I_ON and I_OFF currents lead to either switching on or switching off the switch and set the oscillator to the subthreshold mode, and there is no dependence of current influence on the network from the input data.

The synchronization efficiency η_th and the noise level U_n affect the form of the distribution NS. Figure 16b presents the NS distribution for η_th = 90%. The distribution maximum is shifted to the left, and the maximum of the total number of solutions is in the “12 of 12” area. At the same time, the maximum value decreased by approximately 2 times, and the number of solutions to the “16 of 16” task fell by 10 times. Such a decrease happens due to the cut-off of solutions by the criterion η ≥ η_th. In Figure 17b (blue dots), the dependencies of the synchronization efficiency values on the input data are presented, which are elements of a single solution. So, at η_th = 30% (U_n = 0 mV) all points satisfy the condition η ≥ η_th, and the array of elements CH_0,5 satisfies the “16 of 16” problem. At the same time, at η_th = 90% only six points satisfy the η ≥ η_th condition and constitute a solution for the “6 of 6” task. Therefore, with the increase of η_th, the number of solutions to “x of x” problems with lower x grows and shifts the NS distribution to the left, while NS decreases for the “16 of 16” task.

Figure 16c,d represents the NS distributions for different noise levels U_n.

An increase in noise leads to a decrease in the number of solutions to the “16 of 16” problem, with the distribution shifted to the left. In contrast to the influence of η_th, an increase in noise leads to a significant increase in the number of solutions to “x of x” problems with lower x, and NS experiences the most significant increase for the “1 of 1” task. The noise has the least impact on solutions in area 3, as the input layer oscillators here are constantly in the sub-threshold stable state regardless of the noise voltage dynamics, and the noise affects only the reference oscillator No.0 and output oscillator No.5. With increasing noise (Figure 17b), the synchronization efficiency of almost all elements of the solution decreases, as reflected in the observed patterns of NS distribution.

3.4. Scheme of Converting with a Minimum Number of Couplings

A special case of a fully connected circuit is the circuit shown in Figure 18a, which contains only unidirectional links with the output oscillator, and the input oscillators do not interact.

The scheme is simple and contains only five couplings, however, its functionality allows solving the “x of x” tasks. The distribution of the number of NS solutions to the “x of x” problem after training is shown in Figure 18b. Solutions exist in all areas, however, the number of solutions is less by 2 times than for a fully connected network (see Figure 16a). The maximum of solutions at η_th = 30% and U_n = 0 mV falls on the “16 of 16” task.

Another important conclusion is that the input signal can be represented not by the levels of the I_ON-I_OFF currents, but by signals with a certain CH_{0, i}. In this way, the input and output of the network will be signals of the same nature—signals with a certain synchronization (see Figure 19c). This feature is important for designing large-scale networks, where the output of one network can become the input for another network, as in transistor–transistor logic (TTL) circuits. Next, we summarize the concept of devices that can be implemented on the studied scheme.

3.5. Technological Concepts of Neural Network Converters

This section provides options for the technological concepts of neural network converters. Figure 19a shows a diagram based on 6 oscillators. In this scheme, current levels are applied to the input, and the output is a signal with a certain chimeric synchronization. The circuit requires the generation mode of one of the input oscillators, therefore, the input currents must belong to areas 1 or 2 (see Figure 11).

The scheme in Figure 19b consists of only two oscillators, and the input signal is given by the currents of the thermistors. In fact, this is an imitation of the operation of the circuit on Figure 19a in area 3 of the current parameters. Thermistors set the offset of the threshold characteristics of oscillator No.5 and in this way change the output synchronization. The advantages of the scheme are a lower number of oscillators and increased resistance to noise.

The scheme in Figure 19c differs from the previous schemes, and the principle of its operation is analyzed in Section 3.4. At the input and output of the network there are signals of the same nature—signals with a certain synchronization. This is important for designing large-scale networks, where the output of one network becomes the input for another network, similar to TTL logic circuits.

The coupling forces Δ_i,j determine the input weights and largely determine the functionality of all devices.

3.6. Examples of Image Processing

The converters described in the previous paragraph can be used to filter images. As shown in Figure 19, the input data can be a 2 × 2 matrix of pixels. For example, fully filled pixels correspond to the input number “1111”, and the absence of any filled pixels constitutes the number “0000”. A defined pixel color can be associated with the output. By applying a 2 × 2 matrix transformation algorithm sequentially to all image pixels, it is possible to filter certain attributes.

Table 2. Examples of noise filter (Filter 1) and object boundaries filter (Filter 2).

	Filter 1	Filter 2		Filter 1	Filter 2
Input Data	Color	Color	Input Data	Color	Color
0000	White	White	1000	White	Black
0001	White	Black	1001	White	Black
0010	White	Black	1010	White	Black
0011	White	Black	1011	White	Black
0100	White	Black	1100	White	Black
0101	White	Black	1101	White	Black
0110	White	Black	1110	White	Black
0111	White	Black	1111	Black	White

demonstrates the variants of the simplest filters that allow removal of noise from an image (Filter 1) and a filter for searching the borders of objects (Filter 2). Filter 1 is a special case of the “1 of 1” task, and Filter 2 is the “2 of 2” task.

For example, the input image is a set of shapes with a noisy background (Figure 20a), then using filter 1, the clear image is shown in Figure 20b. After applying filter 2, the image Figure 20b is converted to the form of defined object boundaries (see Figure 20c).

In this way, logical neuromorphic devices can be successfully used for image and video processing.

4. Discussion

The random brute force search method used in this pape, is a part of Sample Average Approximation (SAA) method used to optimize neural networks. The method is a statistical algorithm to be applied when the network state is unknown during weighting factors and input parameters changes, or when the neuron activation function is too complex and does not have a specific derivative [1]. For the studied oscillatory neural network, the output parameters are synchronization metrics (CH, SHR, η), determined by approximately 1000 oscillations; therefore, the functional dependence of the output parameters is difficult to suggest and the application of the gradient approach is limited. The advantage of the SAA method over the Back Propagation Algorithm (BPA), which uses the gradient approach, is the ability of SAA to find the global minimum of the system (global optimization), while BPA tends to get stuck at local minima [37].

Using the brute force search method, the number of satisfying solutions NS for some problems exceeds 10³ from 10⁵ random variants (see Figure 16), which is equivalent to the probability of finding a solution of more than 1%. In addition, the use of chimeric synchronization increases the probability of finding solutions compared with the previously described method [26]. This confirms the effectiveness of the brute force search method of the parameters for optimization of the oscillatory neural network.

Further improvements in the network optimization algorithm can be associated with reduced time spent on calculating metrics (CH, SHR, η). This can be done by simplifying the oscillator model. In addition, the search can be accelerated by using high-performance computing systems with a parallel architecture, for example, GPU and FPGA. The development of optimal and energy-efficient network training algorithms is a task for future research, as the complexity of the oscillator network and the number of its elements will obviously increase in accordance with the complexity of the tasks solved by the network.

The paper demonstrates that the noise value U_n is a controlling parameter, and its value affects synchronization efficiency, which in turn affects the distribution of NS solutions. Figure 15 and Figure 16 illustrate the change in the distribution of NS solutions in the absence of noise in the system at U_n = 0 mV. In a real experiment, the noise value, determined by the fluctuation of the threshold turn-on voltage U_th, can reach tens of millivolts [31,32]. Nevertheless, the described neural network can be used not only as a circuit solution for oscillators based on real switch structures, but also as a model to be implemented in a numerical experiment on computing platforms (GPU, FPGA). In the latter case, the noise level can be model and its value can be varied, starting from U_n = 0 mV, depending on the tasks to be solved.

The problem of accounting for the interaction time delay in neural networks [38], and in general, in multi-agent systems [39], has been discussed in the literature. This paper does not take into account the effects of the oscillators’ time lag to simplify the understanding of the chimeric synchronization concept. The thermal coupling delay time for the synchronization process of two oscillators is discussed in a previous study [40]. Results have found a time shift between synchronous pulses, and the synchronization efficiency decreases with increasing time delay. The synchronization value SHR remains unchanged, which is consistent with the general findings for multi-agent systems [40] and networks with harmonic oscillators [41]. In addition, the time delay can be ignored if its value is much less than the oscillation period [40].

The stability of chimeric synchronization in the case of the individual oscillator’s failure or the presence of interference represents an interesting topic. Several studies investigated the problems of the stable functioning of multi-agent systems [42,43], and in particular, in the field of oscillatory neural networks [44,45], where resistance to variation of natural frequencies, coupling forces, and scaling [45] are in the focus.

The results of this study can be applied to widely known networks based on Kuramoto phase oscillators [46] by using the idea of universal high-order synchronization [46] and chimeric synchronization effects. Unlike pulsed oscillators, where a sharp pulse front due to the switching phenomenon of a nonlinear element can be observed, phase oscillators require an original method for calculating the parameters SHR, CH, and η. The sharp fronts are easily recorded and processed by technical systems. In the literature, the definition of the order parameter for Kuramoto networks can be found [47], which characterizes the synchronization level and can serve as an analogue of the synchronization efficiency value η (4). In addition, the concept of critical communication strength [47,48,49] correlates with the concept of the threshold synchronization efficiency η_th (5).

The value of SHR (Equation (1)) determines the pattern of high order synchronization of two signals. The CH equation (Equation (3)) defines a more complex pattern of synchronization and its generalized characteristics; however, CH does not include all the subtleties of the synchronization pattern. The histogram in Figure 6b demonstrates two components of the chimeric synchronization pattern SHR¹_{i, j} = 3:2 and SHR²_{i, j} = 7:5, cyclically repeating in time as {7:5, 3:2, 3:2}. In a general case, the order patterns in the time scan can be different, depending on the parameters of the system, for example {7:5, 3:2}, {7:5, 3:2, 3:2, 3:2}, or {7:5, 7:5, 7:5, 3:2}. In the latter case, СH can change from CH_i,j = (3:2 7:5) to CH_i,j = (7:5 3:2) and have a different synchronization efficiency. Therefore, the study of the fine structure patterns of high order chimeric synchronization may be the purpose of future studies.

Chimeric synchronization is the synchronization of quasi-periodic oscillations with a specific synchronization pattern, which determines the intermediate state between full synchronization and the asynchronous state. The transition from a synchronous to asynchronous state is observed when crossing the borders of the Arnold tongue, and a gradual broadening of the spectrum of signals and blurring of phase portraits [18,21] are usually witnessed, as demonstrated in Figure 6. This transition mechanism is called a period doubling bifurcation (or torus-doubling bifurcation) [18,21].

The described method allows the calculation of a family of synchronization metrics between two quasi-periodic sequences of impulses (impulse trains). In other words, the values of SHR and CH can be estimated relative to any two oscillators in the network. In the current study, one of the oscillators is a reference oscillator with a constant frequency to ensure correct functioning of the information converting device.

Chimeric synchronization can be estimated between any number of oscillators. In this case, the representation of the indices (SHR, CH) will be more complicated. For example, for three oscillators in a previous study [25], the SHR index in the form of the ternary fraction SHR_i,j,z = M_i:M_j:M_z was used. The study of multi-oscillatory chimeric synchronization could be the subject of future research.

The uniqueness of synchronization metrics (SHR, СH) is in its expression in terms of the ratio of integers (see formulas 1.3), and in the case of the restriction of the integers, the result is discrete. In a previous study [50], when the values of M_i, M_j in Equation (1) were restricted in a way that M_i ≤ 20 and M_j ≤ 20, the total number of variants SHR_i,j was 255. Discreteness of the output parameter is an advantage for technical applications and for interfacing neural networks with computer modules.

An assumption can be proposed that there is a connection between the physics of the formation of chimeric states and the phenomenon of chimeric synchronization. In a previous study [50], a chain of coupled oscillators and long-range synchronization were studied. Results indicate that high order synchronization between a pair of remote oscillators occurs even when intermediate oscillators in connecting links are not formally synchronized. Therefore, it can be assumed that synchronization in these links does exist, but it has a chimeric nature. If, in determining the chimeric states, the chimeric synchronization indices are calculated according to the presented method, the new patterns of the physics of chimeric state formation may emerge.

The method for estimating chimeric synchronization described in this paper gives an idea of the complex structure of synchronization of quasi-periodic oscillations, and can be used to design neuromorphic devices.

5. Conclusions

In this study, we have introduced the method for determining chimeric synchronization expressed by a family of metrics. This technique allows reliable determination of the synchronization state and provides an understanding of the complex structure of synchronization of quasi-periodic oscillations, compared to the previously proposed method, which uses a single high-order synchronization parameter [26]. In addition, such a metric can significantly expand the capabilities of neuromorphic and logical devices that operate on the synchronization effect, and increases the number of solutions to the problems posed. We have revealed the distribution of the number of solutions depending on the operating mode of the oscillators, sub-threshold mode, or generation mode. The implementation variants of unidirectional thermal coupling of oscillators are proposed. The thermal mode of oscillators interaction for 3D integration of VO₂ switches has an advantage, as thermal waves propagate radially from the switch in all directions along the substrate, and allow thermal coupling of oscillators located on different layers.

Several concepts of the neural network converter have been proposed, in which the input signals are either current or the signals have a certain chimeric synchronization. Examples of the converter application for filtering images have been elaborated.

Finally, the methods for classifying chimeric synchronization and neural network information conversion are universal and can be implemented in ONN with electrical coupling between VO₂ oscillators and in ONN based on any switching elements, such as thyristors, tunnel diodes, and resistive memory cells. In addition, these techniques are applicable for implementation on high-performance computing platforms with parallel and variable architecture (CPU, GPU, FPGA), and can be widely used in artificial intelligence systems.