An Oscillatory Neural Network Based Local Processing Unit for Pattern Recognition Applications

Abstract

Prolific growth of sensors and sensor technology has resulted various applications in sensing, monitoring, assessment and control operations. Owing to the large number of sensing units the the aggregate data volume creates a burden to the central data processing unit. This paper demonstrates an analog computational platform using weakly coupled oscillator neural network for pattern recognition applications. The oscillator neural network (ONN) has been studied over the last couple of decades for it’s increasing computational efficiency. The coupled ONN can realize the classification and pattern recognition functionalities based on its synchronization phenomenon. The convergence time and frequency of synchronization are considered as the indicator of recognition. For hierarchical sensing, the synchronization is detected in the first layer, and then the classification is accomplished in the second layer. In this work, a Kuramoto model based frequency synchronization approach is utilized, and simulation results indicate less than 160 ms convergence time and close frequency match for a simplified pattern recognition application. An array of 10 sensors is considered to affect the coupling weights of the oscillating nodes, and demonstrate network level computation. Based on MATLAB simulations, the proposed ONN architecture can successfully detect the close-in-match pattern through synchronization, and differentiate the far-out-match pattern through loss of synchronization in the oscillating nodes.

1. Introduction

Technological innovations of sensors and integrated circuit technology have resulted in wireless sensing for various applications [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16]. The massive scale of sensor data from an infrastructure creates a burden to the property owner to process and extract meaningful information. Conventional CMOS based boolean processors are reaching a plateau in norm of power per processing task. This has motivated increased research and innovation on non-Boolean function-based processing. Non-Boolean logic based recognition processes are very similar to processing in the human brain, where a large number of inferior oscillating nodes perform a network level computation with superior performance. Considering the extent of improvement in CMOS technology on device scaling, memory capacity, and power consumption in the near future, the CMOS based oscillator neural network for analog or non-Boolean computing applications has aroused interest among researchers for energy-efficient computational units. Oscillation is a common natural phenomenon [17,18,19,20,21,22,23]. Classification or pattern recognition tasks can be easily performed using synchronized coupled oscillator neural networks [24,25] compared to other approaches. Cloud computing has been used to access server and database through the internet [26] such as Amazon Web Services (AWS). The cloud services can easily connect to the hardware over a web application. However, could computing approaches in pattern recognition applications are based on online data processing. Considering the possibility of an offline mode that the server cannot respond to the local sensing machine with the request for pattern recognition or other required tasks, the hierarchical AM model for local processing unit has been proposed to solve the offline data processing problem.

To study the dynamics of coupled oscillator networks, different mathematical models have been reported. In recent years, several researchers have proposed the use of multiple coupled oscillators for pattern recognition. Fang et al. developed a new chemomechanical oscillating material for use in a BZ-PZ oscillator network for pattern recognition [21]. In [27,28], CMOS ring oscillators for pattern recognition were proposed. A non-Boolean ring oscillator coupled with a resistor network was proposed to function as a Hopfield network [27]. The computing efficiency in pattern recognition of a variety of coupled oscillator networks have been compared in references [28,29,30]. Each stored pattern can be built using an AM model using oscillator neural network (ONN). The recognition process initiates when the input stimulus patterns are applied to the stored pattern, and the degree of recognition is defined by the ease with which the stored pattern can be retrieved. The close-in-match pattern will be the winner pattern. A single AM with weakly coupled oscillators has been used for many years but requires excessive energy. In comparison, hierarchical structures have been shown to have higher efficiency than single oscillator neural networks without sacrificing the performance [31]. However, those works focus on evaluating the computational speed of the existing CMOS oscillators.

In contrast, the work presented here focuses on specific applications of synchronization and stable cluster to improve computational efficiency. By choosing an appropriate coupling function, a set of synchronization or stable cluster patterns can be obtained as stored pattern. When presenting a distorted or noisy pattern as input, the association is a nearest neighbor searching procedure. In this paper, networks of weakly coupled phase oscillators are used for pattern recognition. Specifically, the dynamics of coupled functions are analyzed for spatial, temporal rhythmic patterns including synchronization and stable cluster. In addition to improving computational efficiency, the use of such networks also facilitates energy-efficient hardware implementation and low-level signal processing using ONN compared to computation intensive software simulation. For hierarchical sensing process, each sensing data is read by the Kuramoto model as the natural frequency on the first layer. The Kuramoto model is a useful model for studying synchronization phenomenon. It provides a simple but solvable approach to synchronization in coupled oscillators [32]. In the Kuramoto model a sinusoidal coupling function among neighbor oscillators is used to characterize the local synchronization of the network. The synchronization frequency then acts as the natural phase of one of the oscillators in the second layer. Afterwards a classification process is performed using a stable cluster algorithm, where phase difference between the oscillators in each pair is classified into stable 2-cluster. Since the input pattern will always need to be compared to all the memorized patterns to search nearest neighbor, the hierarchical clustering greatly reduces the time complexity by increasing the space complexity through multi-layer processing.

In this paper, a non-Boolean or analog computing architecture based on CMOS oscillator neural networks is used to implement pattern recognition and classification tasks. The organization of the paper is as follows. Section 2 gives a short overview of the weakly coupled oscillatory neural network and pattern recognition task. Section 3 shows the proposed oscillatory neural network and pattern recognition using Kuramoto model. Section 4 describes hierarchical clustering of data from an impedance grid and the corresponding simulation results of stable cluster with specific coupling approach, which solve the complex behavior of coupled nonlinear oscillators. Section 5 is the validation of the hierarchical AM model’s performance. Finally, Section 6 provides a discussion and conclusion of the work.

2. Weakly Coupled Oscillatory Neural Network and Pattern Recognition

In this section, we describe a weakly phase-coupled oscillatory network with a dynamic coupling function. The behavior of the oscillator is governed by the following equation:

$$\frac{\mathrm{d}{\varphi }_i}{\mathrm{d}t}(t)={\omega }_i+\sum _{j=1}^N{K}_{ij}{H}_{ij}\left[{\varphi }_j(t)-{\varphi }_i(t)\right]$$

(1)

Here ${\varphi }_i$ is the phase of oscillator i, ${\omega }_i$ is its intrinsic frequency, $K_{ij}$ is the coupling weight, $H_{ij}$ is the coupling function between oscillator i and oscillator j. In general, by choosing appropriate coupling function $H_{ij}$, any arbitrarily complex behavior can achieve a stable cluster state. In particular, when $H_{ij}\left[{\varphi }_j\left(t\right)-{\varphi }_i\left(t\right)\right]$ is equal to $sin\left[{\varphi }_j\left(t\right)-{\varphi }_i\left(t\right)\right]$, then the system becomes Kuramoto model. The dynamics of weakly coupled oscillators can exhibit complex chaotic behavior; however, such a behavior cannot be obtained by the Kuramoto model using a simple sinusoidal coupling function.

Pattern recognition classifies the samples through the calculation according to the characteristics of the sample. A pattern recognition system is basically composed of three parts-data acquisition, data processing, and a classification decision or model matching. The objective is to compare the stored pattern with the detected pattern. There can be multiple stored patterns. We define the pattern recognition task with the convergence time and frequency to the stored pattern. The initialized detective pattern with the closest convergence time and frequency exhibit the best match between the detective pattern and stored pattern. If the time and frequency difference are asymptotic to zero, the detective pattern is synchronized with the stored pattern, which we consider as recognition. However, if the frequency does not synchronize, or the time and frequency differences are bigger than $\epsilon$ ($\epsilon$ depends on different recognition system), it is considered lack of recognition. The functional flow diagram for pattern recognition using ONN is shown in Figure 1.

3. Topology of the Proposed ONN System

In this section, we present the topology of the overall system used in the neural network design. The coupled oscillators are separated into two categories: stimulus oscillators and recognition oscillators. We assume that the topology between each oscillator is cross-connected with different weight $K_{ij}$ shown in Figure 2. The advantage of cross-connected topology is that if one oscillator fails, only the failed oscillator is unable to send or receive data [33].

In this work, we present an example of pattern recognition using the Kuramoto model. The structure of the ONN architecture is shown in Figure 2. The couplings between the ten oscillators comprised with two stimulus oscillators: $O_1$ and $O_2$, and eight recognition oscillators $O_3$, $O_4$, $O_5$, $O_6$, $O_7$, $O_8$, $O_9$, $O_{10}$. The coupling constant between oscillators is $k_{ij}(i\ne j)$, we set $k_{ij}=1$, except $k_{12}=k_{21}=0$. $A_1=4$, $A_2=4$, $A_3=A_4=\dots =A_{10}=1$. The weight between each oscillator is $K_{ij}$,

$$K_{ij}=A_i\ast A_j\ast k_{ij}$$

(2)

The weight between each recognition oscillator is $K_{ij}=1,i\ne j$. The weights between each stimulus are $K_{12}=K_{21}=0$. The weight from stimulus oscillator to recognition oscillator is $K_{ij}=4,i=1$ and $2,j=3\dots 10$, while the weight from recognition oscillator to stimulus oscillator is $K_{ij}=0,i=3\dots 10$ $j=1$ and 2. The weights between these ten oscillators are shown as,

$$K_{ij}=\left[\begin{array}{cccccccccc}0& 0& 4& 4& 4& 4& 4& 4& 4& 4\\ 0& 0& 4& 4& 4& 4& 4& 4& 4& 4\\ 0& 0& 0& 1& 1& 1& 1& 1& 1& 1\\ 0& 0& 1& 0& 1& 1& 1& 1& 1& 1\\ 0& 0& 1& 1& 0& 1& 1& 1& 1& 1\\ 0& 0& 1& 1& 1& 0& 1& 1& 1& 1\\ 0& 0& 1& 1& 1& 1& 0& 1& 1& 1\\ 0& 0& 1& 1& 1& 1& 1& 0& 1& 1\\ 0& 0& 1& 1& 1& 1& 1& 1& 0& 1\\ 0& 0& 1& 1& 1& 1& 1& 1& 1& 0\end{array}\right],$$

(3)

Frequency synchronization is a common phenomenon in nature, and this can be observed when the frequencies of the oscillators converge to a single value because of coupling. The Kuramoto model is a well-known model for studying the synchronization phenomenon of oscillators. This model has been widely used in the fields of biology, physics, chemistry, sociology, etc. to study the synchronization phenomena [34,35,36]. The equation of coupled oscillators using the Kuramoto model is [37],

$$\frac{\mathrm{d}{\varphi }_i}{\mathrm{d}t}(t)=2\pi \left\{f_i+\sum _{j=1}^N{A}_i{A}_j{k}_{ij}sin\left[{\varphi }_j(t)-{\varphi }_i(t)\right]\right\}$$

(4)

Here ${\varphi }_i$, $f_i$ and $A_i$ are the phase, intrinsic frequency, and amplitude of the $i^{th}$ oscillator, respectively. $k_{ij}$ is a coupling constant. Coupled oscillators are synchronized if they are at the same frequency and phase-locked [38].

3.1. Network Initialization

By choosing an appropriate value for $K_{ij}$, a binary pattern $\xi$ can be stored in the network, $\xi$ = 1 or 4. The coupling weight $K_{ij}$ can also be represented by ${\xi }_i$* ${\xi }_j$. In this example, we choose different intrinsic frequencies and weights for each of the ten oscillators. The frequency vector, f, and the weight vector, $K_{ij}$, are shown as below,

$$f=\left[\begin{array}{cccccccccc}21& 14& 10& 15& 20& 25& 12.5& 17.5& 17.5& 22.5\end{array}\right],$$

(5)

$$K_{ij}=\left[\begin{array}{cccccccccc}4& 4& 1& 1& 1& 1& 1& 1& 1& 1\end{array}\right],$$

(6)

Figure 3 shows the synchronization diagram of the ten coupled oscillators. The store pattern synchronized with the coupling approach of Kuramoto model. The convergence time of the stored pattern is 1.576800 × 10⁻¹ s, the convergence frequency is 17.5 Hz.

3.2. Pattern Recognition

In this work, the pattern recognition task is performed by monitoring the synchronizing time and frequency of the ONN. In this network the number of inputs is identical to the number of oscillating nodes, and each input is considered to disturb the intrinsic frequency of the corresponding oscillating node. In this scenario, when an input pattern is applied to the network, one or more oscillators experience different intrinsic frequencies from that of the stored pattern case. With the identical coupling approach and coupling weights of the stored pattern, the ONN exhibits a different synchronizing time and frequency for the applied input pattern. In this example, the intrinsic frequency of the oscillator is varied from 15 Hz to 10 Hz at coordinate (2, 2) and the corresponding the convergence time of the ONN is monitored as shown in Figure 4. The ten oscillators used in this simulation are eventually synchronized at 1.550100 × 10⁻¹ s and the convergence frequency appears to be 17.3 Hz. Figure 5 shows a detective pattern with lack of recognition. When changing the ninth oscillator’s frequency with coordinate (1, 5) to a more than twice the average value of those ten oscillators in the stored pattern, the coupled oscillators of the ONN are not able to synchronize.

Table 1. Summary of convergence time and frequency for recognition performance.

Synchronization Pattern	Convergence Time (s)	Convergence Frequency (Hz)
Stored	1.576800 × 10−1	17.5
Recognition	1.731000 × 10−1	17
Recognition upper threshold	2.742600 × 10−1	19.3
Recognition lower threshold	1.841300 × 10−1	16.5
Lack of Recognition	No Convergence	No Convergence

summarizes the synchronizing time and frequency for the stored pattern and the detective pattern. The differences in convergence time and synchronizing frequency between the input detective pattern and the stored pattern illustrate the pattern recognition status. Once the network frequency is synchronized, the convergence time and frequency are measured and compare to the stored pattern’s convergence time and frequency.

4. Hierarchical Clustering of Impedance Sensing Grid

4.1. Hierarchical Associative Memory Model

In this section, a hierarchical clustering approach is employed along with the ONN. Figure 6 illustrates a two-level hierarchical model for large array sensor signal processing. Each module represents a coupled ONN using the Kuramoto model in the first layer with 16 inputs and 1 output. The output of each 16 sensing nodes or unit cell in the first layer are read as the input of the second layer. For hierarchical sensing process, each sensing data is read by the Kuramoto model as natural frequency on the first layer, and the synchronization frequency of each unit cell acts as the natural phase of one of the oscillators in the second layer. A classification process is then done by using stable cluster algorithm to classify phase difference between each pair of oscillator in the second layer into stable 2-cluster.

In this paper, a hierarchical AM model is performed.

Table 2. Comparison of the proposed hierarchical AM model with the single AM model.

ONN Characteristics	Hierarchical AM Model	Single AM Model
No. of layers	2	1
No. of oscillators	$n^2$	$n^2$
No. of oscillators in each layer	n	$n^2$
Connections (Number of weights)	$C_{n+1}^2$	$C_{n^2}^2$

shows a comparison of the hierarchical AM model used in this work with a single AM model. The weights are corresponding to the connections of each two oscillators. When the number of oscillators is high, the hierarchical model uses much less connections compared with the single AM model, which is a big advantage for simplifying ONN. Our aim is to develop an algorithm such that reducing the computational complexity and minimizing the connections of stored pattern without sacrificing recognition accuracy. However, we still cannot guarantee the accuracy of the hierarchical AM model when using it to differentiate two very similar pattern. But with enough training data, we are able to improve coupling function for more accuracy recognition performance.

4.2. Sensing Device

Impedance sensing has been proved to be an efficient approach of detecting early pressure ulcers in vivo by utilizing the electrical ’signature’ of the cell membrane. The damaged cell performs more like a resistor and less like a capacitor [39]. The sensing device consists of impedance sensors and oscillators as shown in Figure 7. We assume that each oscillator is connected to a sensor.

As shown in Figure 8 and Figure 9, a white square corresponds to the phase difference equal to $2k\pi$, and a black square corresponds to the phase difference equal to $(k+\frac{1}{2})\pi$. All the white squares represent one cluster and all the black squares represent the other cluster.

The system of coupled nonlinear oscillators can perform complex behavior. The Kuramoto model applies simple sinusoidal coupling function; however, it is not suitable for coupled nonlinear oscillators. By choosing appropriate coupling function $H_{ij}$, the stable cluster state can be achieved. Any $H_{ij}$ can be represented as a Fourier series and for suitable coefficient values of $u_l$ and $v_l$, the Fourier series expansion of $H_{ij}$ up to the L-th harmonics can be shown as,

$$H_{ij}=\sum _{l=1}^L[u_{l}cos(l\varphi )+v_{l}sin(l\varphi )]$$

(7)

In this system, the network is considered to be in cross-connect topology with coupling weight $K_{ij}=1$. The Fourier series coefficients $u_1=\frac{1}{12},v_1=-\frac{49}{180},u_2=\frac{1}{4},v_2=\frac{1}{40}$ were calculated and designed as the first stored pattern as shown in Figure 8 for phase difference to achieve stable 2-cluster, $u_1=\frac{1}{12},v_1=-\frac{9}{120},u_2=\frac{1}{3},v_2=\frac{1}{40}$ was calculated and designed as the second store pattern shown in Figure 9. Appendix A shows how to design stable 2-cluster [40]. Based on these oscillators, we can design and build an AM circuit for each stored pattern. Since the input pattern will always need to be compared to all the memorized patterns to search nearest neighbor, the hierarchical clustering greatly reduces the time complexity by increasing the space complexity through multi-layer processing.

A binary pattern $\xi$,

$$\xi =\left[\begin{array}{ccccc}{\xi }_1& {\xi }_2& {\xi }_3& \dots & {\xi }_{16}\end{array}\right],{\xi }_i=\pm 1,i=1\dots 16,{\xi }_j=\pm 1,j=1\dots 16,$$

(8)

can be designed as stable 2-cluster and stored into the ONN while the second layer achieving stable 2-cluster. When the phase difference is $2k\pi$, ${\xi }_i$ is equal to 1 and corresponds to the white square as shown in Figure 8. On the other hand, while the phase difference is $(k+\frac{1}{2})\pi$, ${\xi }_j$ is equal to −1 and corresponds to the black square.

$${\xi }_i=1⟺{\varphi }_i={\varphi }_j+2k\pi ⟺{\varphi }_i-{\varphi }_j=2k\pi$$

(9)

$${\xi }_j=-1⟺{\varphi }_i={\varphi }_j+(k+{\displaystyle \frac{1}{2}})\pi ⟺{\varphi }_i-{\varphi }_j=(k+{\displaystyle \frac{1}{2}})\pi$$

(10)

We have simulated a phase difference array as a sensing device with sixteen oscillators. Apparently from the simulation results as shown in Figure 10 and Figure 11, the second stored pattern achieves a better matching with less convergence time. A part of the oscillators approach to one cluster with phase difference $2k\pi (k=0$ or $1)$ (for $k=0,{\varphi }_j-{\varphi }_1=0wherej=1$, and for $k=1,{\varphi }_j-{\varphi }_1=2\pi wherej\ne 1$) and the rest of the pattern approaches to the other cluster with phase difference $(k+\frac{1}{2})\pi$, which means this two groups of oscillators are converged to phase difference $(k+\frac{1}{2})\pi$ and $2\pi$, respectively. We call it stable 2-cluster.

When the state of the skin cell at Oscillator i changes, the signal from the sensor i will cause a change in the phase of oscillator i. The 16 oscillators will achieve a new state. Since our cluster is only related to the phase, the new state can find new nearest neighbor. The coupled oscillator array doesn’t have to be orthogonal for achieving stable cluster as shown in Figure 11. In our model, the stable 2-cluster can be achieved when the coefficient satisfied the condition as shown in Appendix A. If the new state approaches to the stored pattern with the shortest time we consider the stored pattern as the winner pattern.

4.3. Comparison with Other Algorithms

The comparison of our hierarchical AM model with other high dimensional algorithm is described in

Table 3. Comparison of the proposed work with other algorithms.

ONN Characteristics	This Work	ARENA Algorithm [31]	PCA [41]
Hardware Realization	Achievable	-	-
Dimension	High Dimension	High Dimension	High Dimension
Signal Processing	Kuramoto Model	Local Averaging	Discarding Eigenvectors
Robustness	High	Low	Low

. Our hierarchical AM model is a simple model can be achieved using hardware for local processing computation. The first layer of our model is using Kuramoto model with weight affection which is more robust to small image registration error than the other two.

5. Validation of the Proposed Approach

In this section, our results were obtained using images of human faces from the ATT Cambridge Database [42]. The ATT Cambridge Database contains images of 40 different people 40 with 10 images for each person. Each original face image is reduced to n × n using simple local averaging. For 92 × 112 full resolution images, there is no significant improvement beyond n = 64, so in this work, we present result with 64 × 64 oscillators for each image. In our ONN, the image pixel value is stored as the intrinsic frequency of oscillators. When a new photograph is coming, we convert it to the same pixel amount as our stored pattern. In this simulation, the image pixel values which are sensed by sensor i and sensor j are stored in oscillators and forming the ONN shown in Figure 7. We assume that each oscillator is connected to a pixel. However not all of the images are suitable for our algorithm. We select the images that satisfied the following constraints: near- frontal poses and near face to background proportion ratio. We select two individuals with two photographs for each of them as storage patterns and with one for each as recognition pattern. Figure 12 shows three images for each of two individuals from the datasets.

We have stored each individuals’ image with 64 × 64 oscillators. The first and second images are stored as storage pattern, the third image is acting as recognition pattern shown in Figure 12. For each image, in the lower layer, 8 × 8 oscillators are stored in each module. 8 × 8 modules are stored in the higher layer. The output of each module in the lower layer is read as the input of the higher layer. For hierarchical sensing process, each oscillator data is read by the Kuramoto model as intrinsic frequency on the lower layer, and the synchronization frequency of each module acts as the intrinsic phase of the oscillator in the higher layer. Apparently from the simulation result shown in Figure 13 and Figure 14, the first storage pattern achieves a better matching.

6. Discussion and Conclusions

We have learned that ONN are capable of sensing and computing on a local processing unit. Our aim is to develop an algorithm that reduces the computational complexity. In this paper, a weakly coupled phase oscillator theory is analyzed in the case of both linear and nonlinear couplings. For the Kuramoto model, a sinusoidal coupling function is able to exhibit synchronization for pattern recognition applications. The convergence time and frequency of synchronization are considered as the indicators of recognition. Moreover, a nonlinear coupling function is utilized and the nonlinear dynamics is represented as Fourier series. The dynamic coupling function provides an approach for analyzing the nonlinear coupled oscillator. By choosing the specific coefficient and coupling function, a stable 2-cluster is achieved for a sensing application. Since clustering can provide more than one stable state, it also provides another stable state of pattern compared to the single decision from Kuramoto model. In this work, a hierarchical AM model is performed. The hierarchical AM model performed in this work can be easily achieved in a local processing unit. A sinusoidal coupling function using the Kuramoto model is shown to exhibit the synchronization for the lower layer. By choosing the specific coefficient and coupling function, a stable 2-cluster is achieved for the higher layer. Since the input pattern will always need to be compared to all the memorized patterns to search nearest neighbor, the hierarchical clustering greatly reduces the time complexity by increasing the space complexity through multi-layer processing.