Construction of an Infinite Cyclic Group Formed by Artificial Differential Neurons

Abstract

Infinite cyclic groups created by various objects belong to the class to the class basic algebraic structures. In this paper, we construct the infinite cyclic group of differential neurons which are modifications of artificial neurons in analogy to linear ordinary differential operators of the n-th order. We also describe some of their basic properties.

1. Introduction

In our paper, we study artificial, or formal, neurons. Recall that these are the building blocks of mathematically modeled neural networks, e.g., [1]. The design and functionality of artificial neurons are derived from observations of biological neural networks. Our investigation belongs to the theory which is developed and applied in various directions contained in many publications, cf. [2,3,4,5,6]. The bodies of artificial neurons compute the sum of the weighted inputs and bias and “process” this sum with a transfer function, cf. [1,2,3,4,5,6,7,8,9,10].

In the next step, the information is passed via outputs (output functions). Thus, artificial neural networks have the structure similar to that of weighted directed graphs with artificial neurons being their nodes and connections between neuron inputs and outputs being directed edges with weights. Recall that in the framework of artificial neural networks there are networks of simple neurons called perceptrons. The basic concept (perceptron) was introduced by Rosenblatt in 1958. Perceptrons compute single outputs (the output function) from multiple real-valued inputs by forming a linear combination according to input weights, and then possibly putting the output through some nonlinear activation functions. Mathematically, this can be written as

$$y\left(t\right)=\phi \left(\sum _{i=1}^n{w}_i\left(t\right)x_i\left(t\right)+b\right)=\phi \left({\overrightarrow{w}}^T\left(t\right)\overrightarrow{x}\left(t\right)+b\right),$$

(1)

where $\overrightarrow{w}\left(t\right)=(w_1\left(t\right),\dots ,w_n\left(t\right))$ denotes the vector of time dependent weight functions, $\overrightarrow{x}\left(t\right)=(x_1\left(t\right),\dots ,x_n\left(t\right))$ is the vector of time dependent (or time varying) input functions, b is the bias and $\phi$ is the activation function. The use of time varying functions as weights and inputs is a certain generalization of the classical concept of artificial neurons from the work of Warren McCulloch and Walter Pitts (1943); see also [1,2,3,4,5,6,7,8,9,10] and references mentioned therein.

2. Differential Neurons and Their Output Functions

In accordance with our previous papers [1,7,8,9], we regard the above mentioned artificial neurons such that inputs $x_i$ and weights $w_i$ will be functions of argument t belonging into a linearly ordered (tempus) set T with the least element $0.$ As the index set we use the interval of real numbers $[1,\infty )=\{x\in \mathbb{R};1\le x\}$, where $\mathbb{R}$ denotes the set of all real numbers. So, denote by W the set of all non-negative functions $w:T\to \mathbb{R}$ forming a subsemiring of the ring of all real functions of one real variable $x:\mathbb{R}\to \mathbb{R}$. Denote by $Ne\left({\overrightarrow{w}}_r\left(t\right)\right)=Ne(w_{r1}\left(t\right),\dots ,w_{rn}\left(t\right))$ for $r\in [1,\infty )$, $n\in \mathbb{N}$ and the mapping

$$y_r\left(t\right)=\sum _{k=1}^n{w}_{r,k}\left(t\right)x_{r,k}\left(t\right)+b_r$$

(2)

which will be called the artificial neuron with the bias $b_r\in \mathbb{R},$ in fact the output function of the corresponding neuron. By $\mathbb{AN}\left(T\right)$ we denote the collection of all such artificial neurons.

Neurons are usually denoted by capital letters $X,Y$ or $X_i,Y_i$. However, we use also notation $Ne\left(\overrightarrow{w}\right)$, where $\overrightarrow{w}=(w_1,\dots ,w_n)$ is the vector of weights.

We suppose, for the sake of simplicity, that transfer functions (activation functions) $\phi$, $\sigma$ (or f) are the same for all neurons from the collection $\mathbb{AN}\left(T\right)$ or that this function is the identity function $f\left(y\right)=y.$

Now, similarly as in the case of the collection of linear differential operators, we will construct a cyclic group of artificial neurons, extending their monoid, cf. [1].

Denote by ${\delta }_{ij}$ the so called Kronecker delta, $i,j\in \mathbb{N},$, i.e., ${\delta }_{ii}={\delta }_{jj}=1$ and ${\delta }_{ij}=0$, whenever $i\ne j.$

Suppose $Ne\left({\overrightarrow{w}}_r\right),Ne\left({\overrightarrow{w}}_s\right)\in \mathbb{AN}\left(T\right)$, $r,s\in [1,\infty )$, ${\overrightarrow{w}}_r=(w_{r1},\dots ,w_{r,n})$, ${\overrightarrow{w}}_s=(w_{s1},\dots ,$$w_{s,n})$, $n\in \mathbb{N}$. Let $m\in \mathbb{N},1\le m\le n$ be a such an integer that $w_{r,m}>0.$ We define

$$Ne\left({\overrightarrow{w}}_v\left(t\right)\right)=Ne\left({\overrightarrow{w}}_r\left(t\right)\right){·}_{m}Ne\left({\overrightarrow{w}}_s\left(t\right)\right),$$

(3)

where

$${\overrightarrow{w}}_v\left(t\right)=\left(w_{v,1}\left(t\right),\dots ,w_{v,n}\left(t\right)\right),$$

(4)

$$w_{v,k}\left(t\right)=w_{r,m}\left(t\right)w_{s,k}\left(t\right)+(1-{\delta }_{m,k})w_{r,k}\left(t\right),t\in T$$

(5)

and, of course, the neuron $Ne\left({\overrightarrow{w}}_v\right)$ is defined as mapping $y_v\left(t\right)={\displaystyle \sum _{k=1}^n}{w}_k\left(t\right)x_k\left(t\right)+b_v,$ $t\in T,b_v=b_r{b}_s.$ Further, for a pair $Ne\left({\overrightarrow{w}}_r\left(t\right)\right),Ne\left({\overrightarrow{w}}_s\left(t\right)\right)$ of neurons from $\mathbb{AN}\left(T\right)$ we put

$$\begin{array}{c}Ne\left({\overrightarrow{w}}_r\left(t\right)\right){\le }_{m}Ne\left({\overrightarrow{w}}_s\left(t\right)\right),\\ {\overrightarrow{w}}_r\left(t\right)=\left(w_{r,1}\left(t\right),\dots ,w_{r,n}\left(t\right)\right),{\overrightarrow{w}}_s\left(t\right)=\left(w_{s,1}\left(t\right),\dots ,w_{s,n}\left(t\right)\right)\end{array}$$

(6)

if $w_{r,k}\left(t\right)\le w_{s,k}\left(t\right)$, $k\in \mathbb{N}$, $k\ne m$ and $w_{r,m}\left(t\right)=w_{s,m}\left(t\right)$, $t\in T$ and with the same bias.

Remark 1.

There exists a link between formal neurons and linear differential operators of the n-th order. This link is important for our future considerations. Recall the expression of formal neuron with inner potential $y_{-in}={\displaystyle \sum _{k=1}^n}{w}_k\left(t\right)x_k\left(t\right),$ where $\overrightarrow{x}\left(t\right)=\left(x_1\left(t\right),\dots ,x_n\left(t\right)\right)$ is the vector of inputs, $\overrightarrow{w}\left(t\right)=\left(w_1\left(t\right),\dots ,w_n\left(t\right)\right)$ is the vector of weights. Using the bias b of the considered neuron and the transfer function σ we can expressed the output as $y\left(t\right)=\sigma \left({\displaystyle \sum _{k=1}^n}{w}_k\left(t\right)x_k\left(t\right)+b\right).$

Now consider a fundamental function $u:J\to \mathbb{R},$ where $J\subseteq \mathbb{R}$ is an open interval; inputs are derived from the function $u\in {\mathbb{C}}^n\left(J\right)$ as follows:

$$x_1\left(t\right)=u\left(t\right),x_2=\frac{\mathrm{d}u\left(t\right)}{\mathrm{d}t},\dots ,x_n\left(t\right)=\frac{{\mathrm{d}}^{n-1}\left(t\right)}{\mathrm{d}{t}^{n-1}},n\in \mathbb{N}.$$

Further the bias $b=b_0\frac{{\mathrm{d}}^{n}u\left(t\right)}{\mathrm{d}{t}^n}.$ As weights we use continuous functions $w_k:J\to \mathbb{R}$, $k=1,\dots ,$$n-1.$

Then formula

$$y\left(t\right)=\sigma \left(\sum _{k=1}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}u\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_0\frac{{\mathrm{d}}^{n}u\left(t\right)}{\mathrm{d}{t}^n}\right)$$

(7)

is a description of the action of the neuron $Dn$ which will be called a formal (artificial) differential neuron. This approach allows to use solution spaces of corresponding linear differential equations.

3. Products and Powers of Differential Neurons

Suppose $\overrightarrow{w}\left(t\right)=\left(w_1\left(t\right),\dots ,w_n\left(t\right)\right)$ are fixed vectors of continuous functions $w_k:\mathbb{R}\to \mathbb{R}$ and $b_0$ be the bias for any polynomial $p\in {\mathbb{R}}_s\left[t\right],n\le s,s\in {\mathbb{N}}_0$. We consider a differential neuron $DN{e}_p\left(\overrightarrow{w}\right)$ by the action

$$y_1\left(t\right)=\sum _{k=1}^n{w}_{1,k}\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}$$

(8)

with the identity activation function $\phi \left(u\right)=u.$ According to the formula, we can calculate the output function of the differential neuron $D^{2}N{e}_p\left(\overrightarrow{w}\right)=DN{e}_p\left(\overrightarrow{w}\right)·DN{e}_p\left(\overrightarrow{w}\right).$

Firstly, we describe the product of neurons $Ne\left({\overrightarrow{w}}_r\right)·Ne\left({\overrightarrow{w}}_s\right)=Ne\left({\overrightarrow{w}}_u\right);$, i.e., outputs of neurons

$$y_r\left(t\right)=\sum _{k=1}^n{w}_{r,k}\left(t\right)x_k\left(t\right)+b_r,y_s\left(t\right)=\sum _{k=1}^n{w}_{s,k}\left(t\right)x_k\left(t\right)+b_s.$$

(9)

The vector of weights of the neuron $Ne\left({\overrightarrow{w}}_u\right)$ is of the form ${\overrightarrow{w}}_u\left(t\right)=(w_{u,1},\dots ,w_{u,n}),$ where

$$w_{u,k}\left(t\right)=w_{r,m}\left(t\right)w_{s,k}\left(t\right)+(1-{\delta }_{m,k})w_{r,k}\left(t\right),t\in T\mathrm{and}1\le m\le n.$$

(10)

Then the neuron $Ne\left({\overrightarrow{w}}_u\right)$ is defined using its output function $y_u\left(t\right)={\displaystyle \sum _{k=1}^n}{w}_{u,k}\left(t\right)x_k\left(t\right)+b_r{b}_s,t\in T.$

In a greater detail:

$$\begin{array}{cc}\hfill w_{u,1}\left(t\right)& =w_{r,m}\left(t\right)w_{s,1}\left(t\right)+w_{r,1}\left(t\right),\hfill \\ \hfill w_{u,2}\left(t\right)& =w_{r,m}\left(t\right)w_{s,2}\left(t\right)+w_{r,2}\left(t\right),\hfill \\ \hfill & ⋮\hfill \\ \hfill w_{u,m}\left(t\right)& =w_{r,m}\left(t\right)w_{s,m}\left(t\right),\hfill \\ \hfill & ⋮\hfill \\ \hfill w_{u,n}\left(t\right)& =w_{r,m}\left(t\right)w_{s,n}\left(t\right)+w_{r,n}\left(t\right).\hfill \end{array}$$

Application of the above product onto the case of differential neurons: Suppose $DN{e}_p\left({\overrightarrow{w}}_r\right)$, $DN{e}_p\left({\overrightarrow{w}}_s\right)$ are neurons with output functions

$$\begin{array}{c}{y}_r\left(t\right)={\displaystyle \sum _{k=1}^n}{w}_{r,k}\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_r\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n},\\ y_s\left(t\right)={\displaystyle \sum _{k=1}^n}{w}_{s,k}\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_s\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n},\end{array}$$

(11)

where $p\in {\mathbb{R}}_l\left[t\right]$, $n\le l.$ Denote

$$DN{e}_p\left({\overrightarrow{w}}_u\right)=DN{e}_p\left({\overrightarrow{w}}_r\right)·DN{e}_p\left({\overrightarrow{w}}_s\right).$$

(12)

Then the output function of the neuron $DN{e}_p\left({\overrightarrow{w}}_u\right)$ has the form

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill y_u\left(t\right)& =\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n\left(w_{r,m}\left(t\right)w_{s,k}\left(t\right)+w_{r,k}\left(t\right)\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\hfill \\ \hfill & +w_{r,m}\left(t\right)w_{s,m}\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+b_r{b}_s{\left(\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^2.\hfill \end{array}\end{array}$$

(13)

Now, using the above formula we can express output functions of powers $D^{2}N{e}_p\left({\overrightarrow{w}}_r\right)$, $D^{\alpha }N{e}_p\left({\overrightarrow{w}}_r\right)$ (for $\alpha \in \mathbb{N})$ and $D^{0}N{e}_p\left({\overrightarrow{w}}_r\right)$ (the neutral element-unit) of the infinite cyclic group $\{D^{\alpha }N{e}_p\left({\overrightarrow{w}}_r\right);\alpha \in \mathbb{Z}\}.$ The output function $y_u^{\left[2\right]}\left(t\right)$ of the differential neuron is of the form

$$\begin{array}{cc}\hfill y_u^{\left[2\right]}\left(t\right)& =\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n\left((w_{r,m}\left(t\right)+1)w_{r,k}\left(t\right)\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+w_{r,m}^2\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+b_r^2{\left(\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^2=\hfill \\ \hfill & =(w_{r,m}\left(t\right)+1)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_{r,k}\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+w_{r,m}^2\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+b_r^2{\left(\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^2.\hfill \end{array}$$

(14)

In the paper [1] the following theorem is proved:

Theorem 1.

Consider a differential neuron $DN{e}_p\left(\overrightarrow{w}\right)$ with the vector $\overrightarrow{w}\left(t\right)=\left(w_1\left(t\right),\dots ,w_n\left(t\right)\right)$ of time variable weights and the vector of inputs $\overrightarrow{x}\left(t\right)=\left(p\left(t\right),\frac{\mathrm{d}p\left(t\right)}{\mathrm{d}t},\dots ,\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)$ with polynomial $p\in {\mathbb{R}}_l\left[t\right]$, $n\le l$, $t\in T$ and $1\le m\le n$, $n\in \mathbb{N}=\{1,2,\dots \}.$ The output function $y\left(t\right)$ of the above mentioned neuron is of the form

$$y\left(t\right)=\sum _{k=1}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}$$

(15)

with the bias $b\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}.$ Suppose $\alpha \in \mathbb{N},2\le \alpha .$ Then the output function of the differential neuron $D^{\alpha }N{e}_p\left(\overrightarrow{w}\right)$ has the form

$$y^{\left[\alpha \right]}\left(t\right)=\sum _{k=0}^{\alpha -1}{w}_m^k\left(t\right)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+w_m^{\alpha }\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{\alpha }.$$

(16)

Now, we discuss a certain type of subgroup which appears in all groups. The following text up to Proposition 2 incl. contains well-known facts, which are overtaken from the monography [11] (Chapter 2, §2,4).

Take any group G and any element $a\in G.$ Consider all powers of $a:$ Define $a^0=e$ (the neutral element), $a^1=a,$ and for $k>1,$ define $a^k$ to be the product of k factors of $a.$ (A little more properly, $a^k$ is defined inductively by declaring $a^k=a{a}^{k-1}.)$ For $k>1$ define $a^{-k}={\left(a^{-1}\right)}^k.$

Recall briefly some well-known classical facts.

Definition 1.

Let a be an element of a group $G.$ The set of powers of a $⟨a⟩=\{a^k:k\in \mathbb{Z}\}$ is a subgroup of $G,$ called the cyclic subgroup generated by $a.$ If there is an element $a\in G$ such that $⟨a⟩=G,$ one says that G is a cyclic group. We say that a is a generator of the cyclic group.

There are two possibilities for $⟨a⟩,$ one possibility is that all the powers $a^k$ are distinct, in which case, of course, the subgroup $⟨a⟩$ is infinite; if this is so, we say that a has infinite order.

The other possibility is that two powers of a coincide, but this is not our case.

Definition 2.

The order of the cyclic subgroup generated by a is called the order of $a.$ If the order of a is finite, then it is the least positive integer n such that $a^n=e.$

Proposition 1.

Let a be an element of a group $G.$

(a) 

If a has infinite order then $⟨a⟩$ is isomorphic to $\mathbb{Z}.$

(b) 

If a has finite order $n,$ then $⟨a⟩$ is isomorphic to the group $C_n$ of n-th roots of $1.$

Proposition 2.

(a) 

Any non-trivial subgroup of $\mathbb{Z}$ is cyclic and isomorphic to $\mathbb{Z}.$

(b) 

Let $G=⟨a⟩$ be a finite cyclic group. Any subgroup of G is also cyclic.

For a construction of a cyclic group of artificial differential neurons we need to extend the cyclic monoid of differential neurons obtained in the paper [1] by negative powers of differential neurons, in particular to describe their output functions, so we need to construct negative powers $D^{-\alpha }Ne\left(\overrightarrow{w}\right)$ of differential neurons which belong to the basic contribution of this paper. We suppose the existence of such inverse elements, i.e., negative powers of the generated element of the considered group.

In general, for the construction of the negative power $D^{-\alpha }Ne\left(\overrightarrow{w}\right)$ with $\alpha \in \mathbb{N}$ it seems to be a suitable way of a using of this equality:

$$D^{\alpha +1}Ne\left(\overrightarrow{w}\right){·}_m{D}^{-\alpha }N{e}_p\left(\overrightarrow{w}\right)=DN{e}_p\left(\overrightarrow{w}\right),$$

(17)

where on the right hand side is given an arbitrary general differential neuron with the vector $\overrightarrow{w}\left(t\right)=\left(w_1\left(t\right),\dots ,w_m\left(t\right),\dots ,w_n\left(t\right)\right)$ of time variable weight functions, with the vector of inputs

$$\overrightarrow{x}\left(t\right)=\left(p\left(t\right),\frac{\mathrm{d}p\left(t\right)}{\mathrm{d}\left(t\right)},\dots ,\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right),$$

(18)

with a polynomial $p\in {\mathbb{R}}_l\left[t\right]$, $n\le l$, $t\in T$ and $1\le m\le n$, $n\in \mathbb{N}=\{1,2,\dots \}.$ The neuron $DN{e}_p\left(\overrightarrow{w}\right)$ has the output function

$$y\left(t\right)=\sum _{k=1}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n},$$

(19)

with the bias $b=b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}.$ However, we will construct the proof using mathematical induction—similarly as in [1]—the proof of the Theorem 1, which seems to be a more convenient way. So we are going to prove the following theorem.

Theorem 2.

Suppose the existence of an inverse elements (i.e., negative powers of the generated element of the considered group). Let $DN{e}_p\left(\overrightarrow{w}\right)$ be a differential neuron with the vector $\overrightarrow{w}\left(t\right)=(w_1\left(t\right),\dots ,w_m\left(t\right),\dots ,$$w_n\left(t\right))$ of time variable weights and with the vector of inputs $\overrightarrow{x}\left(t\right)=\left(p\left(t\right),\frac{\mathrm{d}p\left(t\right)}{\mathrm{d}t},\dots ,\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right),$ with a polynomial $p\in {\mathbb{R}}_l\left[t\right]$, $n\le l$, $t\in T$ and $1\le m\le n$, $n\in \mathbb{N}=\{1,2,\dots \}$, i.e., the output function $y\left(t\right)$ of the neuron $DN{e}_p\left(\overrightarrow{w}\right)$ is of the form

$$y\left(t\right)=\sum _{k=1}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n},$$

(20)

with the bias $b=b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}.$ Suppose $\alpha \in \mathbb{N}.$ Then the output function of the differential neuron $DN{e}_p^{-\alpha }\left(\overrightarrow{w}\right)$ has the form

$$y^{[-\alpha ]}\left(t\right)=-\frac{1}{w_m^{\alpha }\left(t\right)}\sum _{\xi =0}^{\alpha -1}{w}_m^{\xi }\left(t\right)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\frac{1}{w_m^{\alpha }\left(t\right)}·\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-\alpha }$$

or

$$y^{[-\alpha ]}\left(t\right)=-w_m^{-\alpha }\left(t\right)\sum _{k=0}^{\alpha -1}{w}_m^k\left(t\right)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+w_m^{-\alpha }\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-\alpha }.$$

Proof. 

Consider the equality

$$DN{e}_p\left(\overrightarrow{w}\right){·}_m{D}^{-1}N{e}_p\left(\overrightarrow{w}\right)=N1{\left(\overrightarrow{e}\right)}_m,$$

(21)

where the output function of the neuron $N1{\left(\overrightarrow{e}\right)}_m$ (the identity element of the monoid $(S^1,{·}_m)$ from [1]) is of the form $y_{N1}\left(t\right)=\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+1.$

Let $y\left(t\right)={\sum }_{k=1}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}$ be the output function of the neuron $DN{e}_p\left(\overrightarrow{w}\right)$ with the bias $b=b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}$ and

$$y^{[-1]}\left(t\right)=\sum _{k=1}^n{w}_{s,k}\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b_s$$

(22)

be the output function of the neuron $D^{-1}N{e}_p\left(\overrightarrow{w}\right).$ Since $0=w_{1,k}=w_m\left(t\right)·w_{s,k}\left(t\right)+w_k\left(t\right)$ and $w_m\left(t\right)·w_{s,m}\left(t\right)=1$ for any $k\in \{1,2,\dots ,n\}\setminus \left\{m\right\},$ we have

$$w_{s,m}\left(t\right)=\frac{1}{w_m\left(t\right)}\mathrm{and}{w}_{s,k}\left(t\right)=-\frac{w_k\left(t\right)}{w_m\left(t\right)}.$$

(23)

Moreover, $1=b·b_s=b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}·b_s$ which implies that the bias $b_s={\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-1}.$ Thus, the output function is of the form

$$\begin{array}{cc}\hfill y^{[-1]}\left(t\right)& =\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n\left(-\frac{w_k\left(t\right)}{w_m\left(t\right)}\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\frac{1}{w_m\left(t\right)}·\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+b_s=\hfill \\ \hfill & =\frac{-1}{w_m\left(t\right)}\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\frac{1}{w_m\left(t\right)}·\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-1}.\hfill \end{array}$$

(24)

Using of Equation (16) we obtain after some simple calculation the expression:

$$\begin{array}{c}{y}^{[-\alpha ]}\left(t\right)=-\frac{1}{w_m^{\alpha }\left(t\right)}\sum _{\xi =0}^{\alpha -1}{w}_m^{\xi }\left(t\right)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\hfill \\ \hfill +\frac{1}{w_m^{\alpha }\left(t\right)}·\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-\alpha }.\end{array}$$

(25)

This function is in a fact the output function of the neuron $D^{-\alpha }N{e}_p\left(\overrightarrow{w}\right)$.

Now, for $\alpha =1$ we obtain

$$y^{[-1]}\left(t\right)=-\frac{1}{w_m\left(t\right)}\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\frac{1}{w_m\left(t\right)}·\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-1},$$

which is in fact the Expression (24).

We have

$$\begin{array}{c}{y}^{[-\alpha -1]}\left(t\right)=-\frac{1}{w_m^{\alpha +1}\left(t\right)}\sum _{\xi =0}^{\alpha }{w}_m^{\xi }\left(t\right)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+\hfill \\ \hfill +\frac{1}{w_m^{\alpha +1}\left(t\right)}·\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-\alpha -1},\end{array}$$

which is the Equality (25) written for $-(\alpha +1)$ instead for $-\alpha .$ The other negative powers can be also obtained from example we have. □

Using output functions of corresponding differential neurons we verify a validity of equalities

$$D^{-\alpha }N{e}_p\left(\overrightarrow{w}\right){·}_{m}N1{\left(\overrightarrow{e}\right)}_m=D^{-\alpha }N{e}_p\left(\overrightarrow{w}\right)=N1{\left(\overrightarrow{e}\right)}_m{·}_m{D}^{-\alpha }N{e}_p\left(\overrightarrow{w}\right)$$

(26)

certifying that the neuron $N1{\left(\overrightarrow{e}\right)}_m$ is the neutral element also for negative powers of the neuron $DN{e}_p\left(\overrightarrow{w}\right).$

Denote by $y_u\left(t\right)$ the output function of the neuron

$$DN{e}_p\left({\overrightarrow{w}}_u\right)=D^{-\alpha }N{e}_p\left(\overrightarrow{w}\right){·}_{m}N1{\left(\overrightarrow{e}\right)}_m.$$

(27)

Since the output function of the neuron $N1\left(\overrightarrow{e}\right)$ (the unit element) has the form

$$y_1\left(t\right)=w_{N1,m}\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+1 \mathrm{with} w_{N1,m}\left(t\right)=1,$$

(28)

we have

$$y_u\left(t\right)=\frac{1}{w_m^{\alpha }\left(t\right)}\left(-\sum _{\xi =0}^{\alpha -1}{w}_m^{\xi }\left(t\right)\right)\sum _{\begin{array}{c}k=1\\ k\ne m\end{array}}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+1·w^{-\alpha }\left(t\right)\frac{{\mathrm{d}}^{m-1}p\left(t\right)}{\mathrm{d}{t}^{m-1}}+1·{\left(b_0\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)}^{-\alpha },$$

which is in fact the output function $y^{[-\alpha ]}\left(t\right)$ of the differential neuron $D^{-\alpha }N{e}_p\left(\overrightarrow{w}\right).$ In a similar way we can verify the second equality.

Remark 2.

In paper [12] there is defined a concept of a general n-hyperstructure as there follows:

Let $n\in \mathbb{N}$ be an arbitrary positive integer and $\{X_k;k=1,\dots ,n\}$ be a system of non-empty sets. By a general n-hyperstructure we mean the pair

$$\left(\{X_k;k=1,\dots ,n\},{\ast }_n\right),$$

where ${\ast }_n:{\displaystyle \prod _{k=1}^n}{X}_k\to P^{\ast }\left({\displaystyle \bigcup _{k=1}^n}{X}_k\right)$ is a mapping assigning to any n-tuple $[x_1,\dots ,x_n]\in {\displaystyle \prod _{k=1}^n}{X}_k$ a non-empty subset ${\ast }_n\left(x_1,\dots ,x_n\right)\subseteq {\displaystyle \bigcup _{k=1}^n}{X}_k.$ Here ${\mathcal{P}}^{\ast }\left(M\right)$ means the power set of M without the empty set $\varnothing .$

Similarly as above, with this hyperoperation there is associated a mapping of power sets

$${\otimes }_n:\prod _{k=1}^n{\mathcal{P}}^{\ast }\left(X_k\right)\to {\mathcal{P}}^{\ast }\left(\bigcup _{k=1}^n{X}_k\right)$$

(29)

defined by

$${\otimes }_n\left(A_1,\dots ,A_n\right)=\bigcup \left\{{\ast }_n(x_1,\dots ,x_n);[x_1,\dots ,x_n]\in \prod _{k=1}^n{A}_k\right\}.$$

(30)

This construction is also based on an idea of Nezhad and Hashemi for $N-2.$

At the end of this section we give this example:

Let $J\subseteq \mathbb{R}$ be an open interval, ${\mathbb{C}}^n\left(J\right)$ be the ring (with respect to the usual addition and multiplication of functions) of all real functions $f:J\to \mathbb{R}$ with continuous derivatives up to the order $n\ge 0$ including. Now, as in suppositions of Theorems 1 and 2, we consider a differential neuron $DN{e}_p\left(\overrightarrow{w}\right)$ with the vector $\overrightarrow{w}\left(t\right)=\left(w_1\left(t\right),\dots ,w_n\left(t\right)\right)$ of time variable weights and the vector of inputs $\overrightarrow{x}\left(t\right)=\left(p\left(t\right),\frac{\mathrm{d}p\left(t\right)}{\mathrm{d}t},\dots ,\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}\right)$ with the polynomial $p\in {\mathbb{R}}_l\left[t\right]$, $n\le l$, $t\in T$ and $1\le m\le n$, $n\in \mathbb{N}=\{1,2,\dots \}.$ The output function $y\left(t\right)$ of the mentioned neuron is of the form

$$y\left(t\right)=\sum _{k=1}^n{w}_k\left(t\right)\frac{{\mathrm{d}}^{k-1}p\left(t\right)}{\mathrm{d}{t}^{k-1}}+b\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}$$

(31)

with the bias $b\frac{{\mathrm{d}}^{n}p\left(t\right)}{\mathrm{d}{t}^n}$ and $w_k:T\to \mathbb{R},w_k\in {\mathbb{C}}^n\left(T\right).$ In accordance with [13], we put

$${\mathbb{DAN}}_k\left(T\right)=\left\{DN{e}_p\left({\overrightarrow{w}}_s\right);p\in {\mathbb{R}}_l\left[t\right],{\overrightarrow{w}}_s\in {\left[{\mathbb{C}}^n\left(T\right)\right]}^k\right\}.$$

As above, we put $DN{e}_p\left({\overrightarrow{w}}_s\right)\le DN{e}_p\left({\overrightarrow{w}}_r\right)$ whenever ${\overrightarrow{w}}_s\left(t\right)=\left(w_{s,1}\left(t\right),\dots ,w_{s,n}\left(t\right)\right),$ ${\overrightarrow{w}}_r\left(t\right)=\left(w_{r,1}\left(t\right),\dots ,w_{r,n}\left(t\right)\right)$ and $w_{s,k}\left(t\right)\le w_{r,k}\left(t\right),t\in T,k=1,2,\dots ,n.$ Defining

$$\begin{array}{cc}\hfill & {\ast }_{n,p}\left(DN{e}_p({\overrightarrow{w}}_1\left(t\right),DN{e}_p({\overrightarrow{w}}_2\left(t\right),\dots ,DN{e}_p({\overrightarrow{w}}_n\left(t\right)\right)=\hfill \\ \hfill & =\bigcup _{k=1}^n\left\{DN{e}_p(\overrightarrow{w}\left(t\right)\in {\mathbb{DAN}}_k{\left(T\right)}_p;N{e}_p\left({\overrightarrow{w}}_k\left(t\right)\right)\le N{e}_p\left(\overrightarrow{w}\left(t\right)\right)\right\}\hfill \end{array}$$

(32)

for any n-tuple $[N{e}_p\left({\overrightarrow{w}}_1\left(t\right)\right),N{e}_p\left({\overrightarrow{w}}_2\left(t\right)\right),\dots ,N{e}_p\left({\overrightarrow{w}}_n\left(t\right)\right)]\in {\displaystyle \prod _{k=1}^n}{\mathbb{DAN}}_k{\left(T\right)}_p,$ we obtain that

$${\mathbb{D}}_p\left(n\right)=(\{{\mathbb{DAN}}_k{\left(T\right)}_p;k=1,2,\dots ,n\},{\ast }_{n,p})$$

(33)

is a general n-hyperstructure for the polynomial $p\in {\mathbb{R}}_l\left[t\right].$

It is to be noted, that the used concept of investigated neurons is in a certain sense motivated by ordinary differential operators forming of left-hand sides of corresponding differential equations, see, e.g., [13,14].

Therefore, the construction of differential neurons consists of a certain modification of the concept of an artificial neuron which is investigated in a certain formal analogy to linear differential operators as mentioned above. Using the obtained cyclic group of differential neurons, we will construct a certain other hyperstructure of differential neurons. The mentioned relationship is in [8] described by the construction of a homomorphism.

It is to be noted that a hypergroup is a multistructure $(H,\ast )$, where H is a non-empty set and $\ast :H\times H\to \mathcal{P}\left(H\right)$ is a mapping which is associative, i.e.,

$$(a\ast b)\ast c=a\ast (b\ast c)$$

for any triad $a,b,c\in H,$ where $A\ast B={\displaystyle \bigcup _{(a,b)\in A\times B}}a\ast b$ for $A\ne \varnothing \ne B$, $A,B\subseteq H,$ and $b\ast A=\left\{b\right\}\ast A.$ Further, the reproduction axiom

$$a\ast H=H=H\ast a$$

for any element $a\in H$ is satisfied.

The above definition of a hypergroup is in the sense of F. Marty.

Let $J\subseteq \mathbb{R}$ be an open interval (bounded or unbounded) of real numbers, ${\mathbb{C}}^k\left(J\right)$ be the ring (with respect to usual addition and multiplication of functions) of all real functions with continuous derivatives up to the order $k\ge 0$ including. We write $\mathbb{C}\left(J\right)$ instead of ${\mathbb{C}}^0\left(J\right)$. For a positive integer $n\ge 2$ we denote by ${\mathbb{A}}_n$ the set of all linear homogeneous differential equations of the n-th order with continuous real coefficients on J,, i.e.,

$$y^{\left(n\right)}+p_{n-1}\left(x\right)y^{(n-1)}+\cdots +p_0\left(x\right)y=0,$$

(34)

(cf. [14,15,16]), where $p_k\in \mathbb{C}\left(J\right)$, $k=0,1,\dots ,n-1,p_0\left(x\right)>0$ for any $x\in J$ (this is not an essential restriction). Denote $L(p_0,\dots ,p_{n-1}):{\mathbb{C}}^n\left(J\right)\to {\mathbb{C}}^n\left(J\right)$ the above defined linear operator defined by

$$L(p_0,\dots ,p_{n-1})\left(y\right)=y^{\left(n\right)}+p_{n-1}\left(x\right)y^{(n-1)}+\cdots +p_0\left(x\right)y$$

(35)

and put

$${\mathbb{LA}}_n\left(J\right)=\{L(p_0,\dots ,p_{n-1});p_k\in \mathbb{C}\left(J\right),p_0>0\}.$$

(36)

Further ${\mathbb{N}}_0\left(n\right)=\{0,1,\dots ,n-1\}$ and ${\delta }_{ij}$ stands for the Kronecker $\delta ,\overline{{\delta }_{ij}}=1-{\delta }_{ij}$. For any $m\in {\mathbb{N}}_0\left(n\right)$ we denote by ${\mathbb{LA}}_n{\left(J\right)}_m$ the set of all linear differential operators of the n-th order $L_0(p_0,\dots ,p_{n-1}):{\mathbb{C}}^n\left(J\right)\to \mathbb{C}\left(J\right)$, where $p_k\in \mathbb{C}\left(J\right)$ for any $k\in {\mathbb{N}}_0\left(n\right)$, $p_m\in {\mathbb{C}}_1\left(J\right)$, (i.e., $p_m\left(x\right)>0$ for each $x\in J$). Using the vector notation $\overrightarrow{p}\left(x\right)=(p_0\left(x\right),\dots ,p_{n-1}\left(x\right)),x\in J$ we can write $L_n\left({\overrightarrow{p}}_0\right)y=y^{\left(n\right)}+(\overrightarrow{p}\left(x\right)·(y,y^{\prime },\dots ,y^{(n-1)})),$, i.e., a scalar product.

We define a binary operation ${\circ }_m$ and a binary relation ${\le }_m$ on the set ${\mathbb{LA}}_n{\left(J\right)}_m$ in this way:

For arbitrary pair $L\left(\overrightarrow{p}\right),L\left(\overrightarrow{q}\right)\in {\mathbb{LA}}_n{\left(J\right)}_m,\overrightarrow{p}=(p_0,\dots ,p_{n-1}),\overrightarrow{q}=(q_0,\dots ,q_{n-1})$ we put $L\left(\overrightarrow{p}\right){\circ }_{m}L\left(\overrightarrow{q}\right)=L\left(\overrightarrow{u}\right),\overrightarrow{u}=(u_0,\dots ,u_{n-1}),$ where

$$u_k\left(x\right)=p_m\left(x\right)q_k\left(x\right)+(1-{\delta }_{km})p_k\left(x\right),x\in J$$

(37)

and $L\left(\overrightarrow{p}\right)\le L\left(\overrightarrow{q}\right)$ whenewer $p_k\left(x\right)\le q_k\left(x\right),k\in {\mathbb{N}}_0\left(n\right),p_m\left(x\right)=q_m\left(x\right),x\in J.$ Evidently, $({\mathbb{LA}}_n{\left(J\right)}_m,{\le }_m)$ is an ordered set.

In paper [14] there is presented the sketch of the proof of the following lemma:

Lemma 1.

The triad $({\mathbb{LA}}_n{\left(J\right)}_m,{\circ }_m,{\le }_m)$ is an ordered (non-commutative) group.

4. Groups and Hypergroups of Artificial Neurons

As it is mentioned in the dissertation [2] neurons are the atoms of neural computation. Out of those simple computational units all neural networks are build up. For a pair $Ne\left({\overrightarrow{w}}_r\right),Ne\left({\overrightarrow{w}}_s\right)$ of neurons from $\mathbb{AN}\left(T\right)$ we put $Ne\left({\overrightarrow{w}}_r\right){\le }_{m}Ne\left({\overrightarrow{w}}_s\right),w_r=\left(w_{r,1}\left(t\right),\dots ,w_{r,n}\left(t\right)\right),w_s=\left(w_{s,1}\left(t\right),\dots ,w_{s,n}\left(t\right)\right)$ if $w_{r,k}\left(t\right)\le w_{s,k}\left(t\right),k\in \mathbb{N},k\ne m$ and $w_{r,m}\left(t\right)=w_{s,m}\left(t\right),t\in T$ and with the same bias. Evidently $(\mathbb{AN}\left(T\right),{\le }_m)$ is an ordered set. A relationship (compatibility) of the binary operation “·” and the ordering ${\le }_m$ on $\mathbb{AN}\left(T\right)$ is given by this assertion analogical to the above one. In paper [1] there is established that the structure $\left(\mathbb{AN}\left(T\right),{·}_m\right)$ is a non-commutative group.

Lemma 2.

The triad $\left(\mathbb{AN}\left(T\right),{·}_m,{\le }_m\right)$ (algebraic structure with an ordering) is a non-commutative ordered group.

Sketch of the proof is presented in [8]. Denoting

$${\mathbb{AN}}_1{\left(T\right)}_m=\{Ne\left(\overrightarrow{w}\right);\overrightarrow{w}=(w_1,\dots ,w_n),w_k\in \mathbb{C}\left(T\right),k=1,\dots ,n,w_m\left(t\right)\equiv 1\},$$

(38)

we get the following assertion, the proof of which with necessary concepts is contained in [1].

Proposition 3.

Let $T=⟨0,t_0)\subset \mathbb{R},t_0\in \mathbb{R}\cup \{\infty \}.$ Then for any positive integer $n\in \mathbb{N},n\ge 2$ and for any integer m such that $1\le m\le n$ the semigroup $({\mathbb{AN}}_1{\left(T\right)}_m,{·}_m)$ is an invariant subgroup of the group $(\mathbb{AN}{\left(T\right)}_m,{·}_m).$

If $m,n\in \mathbb{N},1\le m\le n-1,$ then a certain relationship between groups $({\mathbb{AN}}_n{\left(T\right)}_m,{·}_m),({\mathbb{LA}}_n{\left(T\right)}_{m+1},{\circ }_{m+1})$ is contained in the following proposition:

Proposition 4.

Let $t_0\in \mathbb{R},t_0>0,T=⟨0,t_0)\subset \mathbb{R}$ and $m,,n\in \mathbb{N}$ are integers such that $1\le m\le n-1.$ Define a mapping $F:{\mathbb{AN}}_n{\left(T\right)}_m\to {\mathbb{LA}}_n{\left(T\right)}_{m+1}$ by this rule: For an arbitrary neuron $Ne({\overrightarrow{w}}_r\in {\mathbb{AN}}_n{\left(T\right)}_m,$ where ${\overrightarrow{w}}_r=\left(w_{r,1}\left(t\right),\dots ,w_{r,n}\left(t\right)\right)\in {\left[\mathbb{C}\left(T\right)\right]}^n$ we put $F\left(Ne\left({\overrightarrow{w}}_r\right)\right)=L(w_{r,1},\dots ,w_{r,n})\in {\mathbb{LA}}_n{\left(T\right)}_{m+1}$ with the action:

$$L(w_{r,1},\dots ,w_{r,n})y\left(t\right)=\frac{{\mathrm{d}}^{n}y\left(t\right)}{\mathrm{d}{t}^n}+\sum _{k=1}^n{w}_{r,k}\left(t\right)\frac{{\mathrm{d}}^{k-1}\left(t\right)}{\mathrm{d}{t}^{k-1}},y\in {\mathbb{C}}^n\left(T\right).$$

(39)

Then the mapping $F:{\mathbb{AN}}_n{\left(T\right)}_m\to {\mathbb{LA}}_n{\left(T\right)}_{m+1}$ is a homomorphism of the group $({\mathbb{AN}}_n{\left(T\right)}_m,{·}_{m)}$ into the group $({\mathbb{LA}}_n{\left(T\right)}_{m+1},{\circ }_{m+1}).$

Consider $Ne\left({\overrightarrow{w}}_r\right),Ne\left({\overrightarrow{w}}_s\right)\in {\mathbb{AN}}_n{\left(T\right)}_m$ and denote $F\left(Ne\left({\overrightarrow{w}}_r\right)\right)=L(w_{r,1},\dots ,w_{r,n}),F\left(Ne\right({\overrightarrow{w}}_s=L(w_{s,1},\dots ,w_{s,n}).$ Denote $Ne\left({\overrightarrow{w}}_u\right)=Ne\left({\overrightarrow{w}}_r\right){·}_{m}Ne\left({\overrightarrow{w}}_s\right).$ There holds

$$F\left(Ne\left({\overrightarrow{w}}_r\right){·}_{m}Ne\left({\overrightarrow{w}}_s\right)\right)=F\left(Ne\left({\overrightarrow{w}}_u\right)\right)=L(w_{u,1},\dots ,w_{u,n}),$$

(40)

where

$$L(w_{u,1},\dots ,w_{u,n})y\left(t\right)=y^{\left(n\right)}\left(t\right)+\sum _{k=1}^n{w}_{u,k}\left(t\right)y^{(k-1)}\left(t\right).$$

(41)

Here $w_{u,k}\left(t\right)=w_{r,m+1}\left(t\right)w_{s,k}\left(t\right)+w_{r,k}\left(t\right),k\ne m,$ and $w_{u,m+1}\left(t\right)=w_{r,m+1}\left(t\right)w_{s,m+1}\left(t\right).$ Then $L(w_{u,1},\dots ,w_{u,n})=L(w_{r,1},\dots ,w_{r,n}){·}_{m}L(w_{s,1},\dots ,w_{s,n})=F\left(Ne\left({\overrightarrow{w}}_r\right)\right){·}_{m}F\left(Ne\left({\overrightarrow{w}}_s\right)\right).$ The neutral element $Ne\left(\overrightarrow{w}\right)\in {\mathbb{AN}}_n{\left(T\right)}_m$ is also mapped onto the neutral element of the group $({\mathbb{L}}_n\mathbb{A}{\left(T\right)}_{m+1},{·}_{m+1}),$ thus the mapping $F:({\mathbb{AN}}_n{\left(T\right)}_m,{·}_m)\to ({\mathbb{L}}_n\mathbb{A}{\left(T\right)}_{m+1},{\circ }_{m+1})$ is a group homomorphism.

Now, using the construction described in Lemma 2, we obtain the final transpozition hypergroup (called also non-commutative join space). Denote by $\mathbb{P}{\left(\mathbb{AN}{\left(T\right)}_m\right)}^{\ast }$ the power set of $\mathbb{AN}{\left(T\right)}_m$ consisting of all nonempty subsets of the last set and define a binary hyperoperation

$${\ast }_m:\mathbb{AN}{\left(T\right)}_m\times \mathbb{AN}{\left(T\right)}_m\to \mathbb{P}{\left(\mathbb{AN}{\left(T\right)}_m\right)}^{\ast }$$

(42)

by the rule

$$Ne\left({\overrightarrow{w}}_r\right){\ast }_{m}Ne\left({\overrightarrow{w}}_s\right)=\{Ne\left({\overrightarrow{w}}_u\right);Ne\left({\overrightarrow{w}}_r\right){·}_{m}Ne\left({\overrightarrow{w}}_s\right){\le }_{m}Ne\left({\overrightarrow{w}}_u\right)\}$$

for all pairs $Ne\left({\overrightarrow{w}}_r\right),Ne\left({\overrightarrow{w}}_s\right)\in \mathbb{AN}{\left(T\right)}_m.$ More in detail if $\overrightarrow{w}\left(u\right)=(w_{u,1},\dots ,w_{u,n}),\overrightarrow{w}\left(r\right)=(w_{r,1},\dots ,w_{r,n}),\overrightarrow{w}\left(s\right)=(w_{s,1},\dots ,w_{s,n})$, then $w_{r,m}\left(t\right)w_{s,m}\left(t\right)=w_{u,m}\left(t\right),w_{r,m}\left(t\right)w_{s,k}\left(t\right)+w_{r,k}\left(t\right)\le w_{u,k}\left(t\right),$ if $k\ne m,t\in T.$ Then we have that $(\mathbb{AN}{\left(T\right)}_m,{\ast }_m)$ is a non-commutative hypergroup. The above defined invariant (termed also normal) subgroup $({\mathbb{AN}}_1{\left(T\right)}_m,{·}_m)$ of the group $(\mathbb{AN}{\left(T\right)}_m,{·}_m)$ is the carried set of a subhypergroup of the hypergroup $(\mathbb{AN}{\left(T\right)}_m,{\ast }_m)$ and it has certain significant properties.

Using certain generalization of methods from [8] we obtain after investigation of constructed structures this result:

Let $T=[0,t_0)\subset \mathbb{R},t_0\in \mathbb{R}\cup \{\infty \}.$ Then for any positive integer $n\in \mathbb{N},n\ge 2$ and for any integer m such that $1\le m\le n$ the hypergroup $(\mathbb{AN}{\left(T\right)}_m,{\ast }_m)$, where

$$\mathbb{AN}{\left(T\right)}_m=\{Ne\left({\overrightarrow{w}}_r\right);{\overrightarrow{w}}_r=(w_{r,1}\left(t\right),\dots ,w_{r,n}\left(t\right))\in {\left[\mathbb{C}\left(T\right)\right]}^n,w_{r,m}\left(t\right)>0,t\in T\},$$

is a transpozition hypergroup (i.e., a non-commutative join space) such that $(\mathbb{AN}{\left(T\right)}_m,{\ast }_m)$ is its subhypergroup, which is

-

Invertible (i.e., $Ne\left({\overrightarrow{w}}_r\right)/Ne\left({\overrightarrow{w}}_s\right)\cap {\mathbb{AN}}_1{\left(T\right)}_m\ne \varnothing$ implies $Ne\left({\overrightarrow{w}}_s\right)/Ne\left({\overrightarrow{w}}_r\right)\cap {\mathbb{AN}}_1{\left(T\right)}_m\ne \varnothing$ and $Ne\left({\overrightarrow{w}}_r\right)Ne\left({\overrightarrow{w}}_s\right)\cap {\mathbb{AN}}_1{\left(T\right)}_m\ne \varnothing$ implies $Ne\left({\overrightarrow{w}}_s\right)Ne\left({\overrightarrow{w}}_r\right)\cap {\mathbb{AN}}_1{\left(T\right)}_m\ne \varnothing$ for all pairs of neurons $Ne\left({\overrightarrow{w}}_r\right),Ne\left({\overrightarrow{w}}_s\right)\in {\mathbb{AN}}_1{\left(T\right)}_m,$

-

Closed (i.e., $Ne\left({\overrightarrow{w}}_r\right)/Ne\left({\overrightarrow{w}}_s\right)\subset {\mathbb{AN}}_1{\left(T\right)}_m,Ne\left({\overrightarrow{w}}_r\right)\setminus Ne\left({\overrightarrow{w}}_s\right)\subset {\mathbb{AN}}_1{\left(T\right)}_m$ for all pairs $Ne\left({\overrightarrow{w}}_r\right),/,Ne\left({\overrightarrow{w}}_s\right)\in {\mathbb{AN}}_1{\left(T\right)}_m,$

-

Reflexive (i.e., $Ne\left({\overrightarrow{w}}_r\right){\mathbb{AN}}_1{\left(T\right)}_m={\mathbb{AN}}_1{\left(T\right)}_m/Ne\left({\overrightarrow{w}}_r\right)$ for any neuron $Ne\left({\overrightarrow{w}}_r\right)\in \mathbb{AN}{\left(T\right)}_m$ and

-

Normal (i.e., $Ne\left({\overrightarrow{w}}_r\right)\ast {\mathbb{AN}}_1{\left(T\right)}_m={\mathbb{AN}}_1{\left(T\right)}_m\ast Ne\left({\overrightarrow{w}}_r\right)$ for any neuron $Ne\left({\overrightarrow{w}}_r\right)\in \mathbb{AN}{\left(T\right)}_m.$

Remark 3.

We can define a certain transformation function which mappes the output function $y^{\left[\alpha \right]}\left(t\right)$ into the output function $y^{[\alpha +1]}\left(t\right)$. This function denoting by ${\rho }^{\left[\alpha \right]}$ also determines the transformation $S^{\left[\alpha \right]}$ of powers of corresponding differential neurons: $D^{\alpha }\stackrel{S^{\left[\alpha \right]}}{⟶}{D}^{\alpha +1}.$ In more detail, let us describe output functions $y^{\left[\alpha \right]}\left(t\right),y^{[\alpha +1]}\left(t\right)$ and mentioned transformation function ${\rho }^{\left[\alpha \right]}.$

$$y^{\left[\alpha \right]}\left(t\right)=(1+w_m\left(t\right)+\cdots +w_m^{\alpha -1}\left(t\right))(w_1\left(t\right)p\left(t\right)+w_2\left(t\right)\frac{dp\left(t\right)}{dt}+\cdots +w_{m-1}\left(t\right)\frac{d^{m-2}p\left(t\right)}{d{t}^{m-2}}+$$

$$w_m^{\alpha }\frac{d^{m-1}p\left(t\right)}{d{t}^{m-1}}+w_{m+1}\frac{d^{m}p\left(t\right)}{d{t}^m}+\cdots +{\left(b\frac{d^{n}p\left(t\right)}{d{t}^n}\right)}^{\alpha }),$$

$$y^{[\alpha +1]}\left(t\right)=(1+w_m\left(t\right)+\cdots +w_m^{\alpha -1}\left(t\right)+w_m^{\alpha }\left(t\right))(w_1\left(t\right)p\left(t\right)+w_2\left(t\right)\frac{dp\left(t\right)}{dt}+\cdots$$

$$w_{m-1}\left(t\right)\frac{d^{m-2}p\left(t\right)}{d{t}^{m-2}}+w_m^{\alpha +1}\frac{d^{m-1}p\left(t\right)}{d{t}^{m-1}}+w_{m+1}\frac{d^{m}p\left(t\right)}{d{t}^m}+\cdots +{\left(b\frac{d^{n}p\left(t\right)}{d{t}^n}\right)}^{\alpha +1}).$$

Transformation function ${\rho }^{\left[\alpha \right]}$ of the output function $y^{\left[\alpha \right]}\left(t\right)$ into the output function $y^{[\alpha +1]}\left(t\right)$ which determines the transformation $D^{\alpha }\stackrel{S^{\left[\alpha \right]}}{⟶}{D}^{\alpha +1}$ of powers of corresponding differential neurons.

So,

$${\rho }^{\left[\alpha \right]}\left[\left(\sum _{r=0}^{\alpha -1}{w}_m^r\left(t\right)\right)·\left(\sum _{\begin{array}{c}k-1\\ k\ne m\end{array}}^{m+1}{w}_k\left(t\right)\frac{d^{k-1}p\left(t\right)}{d{t}^{k-1}}+w_m^{\alpha }\left(t\right)\frac{d^{m-1}p\left(t\right)}{d{t}^{m-1}}+{\left(b\frac{d^{n}p\left(t\right)}{d{t}^n}\right)}^{\alpha }\right)\right]=$$

$$=\left[\left(\sum _{r=0}^{\alpha }{w}_m^r\left(t\right)\right)·\left(\sum _{\begin{array}{c}k-1\\ k\ne m\end{array}}^{m+1}{w}_k\left(t\right)\frac{d^{k-1}p\left(t\right)}{d{t}^{k-1}}+w_m^{\alpha +1}\left(t\right)\frac{d^{m-1}p\left(t\right)}{d{t}^{m-1}}+{\left(b\frac{d^{n}p\left(t\right)}{d{t}^n}\right)}^{\alpha +1}\right)\right].$$

Denoting

$$w_m^{[\alpha -1]}=\sum _{r=0}^{\alpha -1}{w}_m^r\left(t\right) \mathrm{and} v^{\left[\alpha \right]}=\sum _{\begin{array}{c}k-1\\ k\ne m\end{array}}^{m+1}{w}_k\left(t\right)\frac{d^{k-1}p\left(t\right)}{d{t}^{k-1}}+w_m^{\alpha }\left(t\right)\frac{d^{m-1}p\left(t\right)}{d{t}^{m-1}}+{\left(b\frac{d^{n}p\left(t\right)}{d{t}^n}\right)}^{\alpha },$$

we can write

$${\rho }^{\left[\alpha \right]}\left[\left(w_m^{[\alpha -1]}\right)·w_m^{\left[\alpha \right]}\right]=w_m^{\left[\alpha \right]}·v^{[\alpha +1]}.$$

5. Conclusions

We have constructed the infinite cyclic group $(\mathbb{G}Dn,{·}_m)$ of differential neurons which is isomorphic to the cyclic group $(\mathbb{Z},+),$ possessing the neuron $N1{\left(\overrightarrow{e}\right)}_m$ as the identity element of $(\mathbb{G}Dn,{·}_m).$ Thus,

$$\left(\left\{N1{\left(\overrightarrow{e}\right)}_m\right\}\cup \{D^{\alpha }N{e}_p(\overrightarrow{w};\alpha \in \mathbb{Z},\alpha \ne 0\},{·}_m\right)=(\mathbb{G}Dn,{·}_m)\cong (\mathbb{Z},+).$$

(43)

It is to be noted that the above constructed cyclic (infinite) group of artificial differential neurons can be also used for the construction of certain hyperstructures formed by such neurons [17,18,19,20]. So the above presented approach enables an additional elaboration of the hyperstructure theory ([8,9,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32]) in connection with time varying weights and with vectors of differentiable input functions.

The construction of the considered infinite cyclic group of differential neurons can be onto other its isomorphic images under the using other weights and inputs. After those constructions there is possible to create abelian finitely or infinitely generated groups of artificial differential neurons and to investigate their direct products or sums. Using a suitable ordering these considerations involve to obtain neural networks with prescribed structures.