Next Article in Journal
Nonlinear Rayleigh Quotients and Nonlinear Spectral Theory
Next Article in Special Issue
Results on Functions on Dedekind Multisets
Previous Article in Journal
Dynamic Partitioning Supporting Load Balancing for Distributed RDF Graph Stores
Previous Article in Special Issue
Elements of Hyperstructure Theory in UWSN Design and Data Aggregation

Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

# Series of Semihypergroups of Time-Varying Artificial Neurons and Related Hyperstructures

1
Department of Mathematics, Faculty of Electrical Engineeering and Comunication, Brno University of Technology, Technická 8, 616 00 Brno, Czech Republic
2
Department of Quantitative Methods, University of Defence in Brno, Kounicova 65, 662 10 Brno, Czech Republic
*
Author to whom correspondence should be addressed.
Symmetry 2019, 11(7), 927; https://doi.org/10.3390/sym11070927
Received: 30 May 2019 / Revised: 28 June 2019 / Accepted: 2 July 2019 / Published: 16 July 2019

## Abstract

:
Detailed analysis of the function of multilayer perceptron (MLP) and its neurons together with the use of time-varying neurons allowed the authors to find an analogy with the use of structures of linear differential operators. This procedure allowed the construction of a group and a hypergroup of artificial neurons. In this article, focusing on semihyperstructures and using the above described procedure, the authors bring new insights into structures and hyperstructures of artificial neurons and their possible symmetric relations.

## 1. Introduction

As mentioned in the PhD thesis , neurons are the atoms of neural computation. Out of those simple computational units all neural networks are build up. The output computed by a neuron can be expressed using two functions $y = g ( f ( w , x ) )$. The details of computation consist in several steps: In a first step the input to the neuron, $x : = { x i }$, is associated with the weights of the neuron, $w : = { w i }$, by involving the so-called propagation function f. This can be thought as computing the activation potential from the pre-synaptic activities. Then from that result the so-called activation function g computes the output of the neuron. The weights, which mimic synaptic strength, constitute the adjustable internal parameters of the neuron. The process of adapting the weights is called learning [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18].
From the biological point of view it is appropriate to use an integrative propagation function. Therefore, a convenient choice would be to use the weighted sum of the input $f ( w , x ) = ∑ i w i x i$, that is the activation potential equal to the scalar product of input and weights. This is, in fact, the most popular propagation function since the dawn of neural computation. However, it is often used in a slightly different form:
$f ( w , x ) = ∑ i w i x i + Θ .$
The special weight $Θ$ is called bias. Applying $Θ ( x ) = 1$ for $x > 0$ and $Θ ( x ) = 0$ for $x < 0$ as the above activation function yields the famous perceptron of Rosenblatt. In that case the function $Θ$ works as a threshold.
Let $F : R → R$ be a general non-linear (or piece-wise linear) transfer function. Then the action of a neuron can be expressed by
where $x i ( k )$ is input value in discrete time k where $i = 0 , ⋯ , m , w i ( k )$ is weight value in discrete time where $i = 0 , ⋯ , m , b$ is bias, $y i ( k )$ is output value in discrete time $k .$
Notice that in some very special cases the transfer function F can be also linear. Transfer function defines the properties of artificial neuron and this can be any mathematical function. Usually it is chosen on the basis of the problem that the artificial neuron (artificial neural network) needs to solve and in most cases it is taken (as mentioned above) from the following set of functions: step function, linear function and non-linear (sigmoid) function [1,2,5,7,9,12,16,19].
In what follows we will consider a certain generalization of classical artificial neurons mentioned above such that inputs $x i$ and weight $w i$ will be functions of an argument t belonging into a linearly ordered (tempus) set T with the least element $0$. As the index set we use the set $C ( J )$ of all continuous functions defined on an open interval $J ⊂ R$. So, denote by W the set of all non-negative functions $w : T → R$ forming a subsemiring of the ring of all real functions of one real variable $x : R → R$. Denote by $N e ( w → r ) = N e ( w r 1 , ⋯ , w r n )$ for $r ∈ C ( J )$, $n ∈ N$ the mapping
$y r ( t ) = ∑ k = 1 n w r , k ( t ) x r , k ( t ) + b r$
which will be called the artificial neuron with the bias $b r ∈ R .$ By $AN ( T )$ we denote the collection of all such artificial neurons.
Neurons are usually denoted by capital letters $X , Y$ or $X i , Y i$, nevertheless we use also notation $N e ( w → )$, where $w → = ( w 1 , ⋯ , w n )$ is the vector of weights [20,21,22].
We suppose - for the sake of simplicity - that transfer functions (activation functions) $φ , σ ( o r f )$ are the same for all neurons from the collection $AN ( T )$ and the role of this function plays the identity function $f ( y ) = y$.
Feedforward multilayer networks are architectures, where the neurons are assembled into layers, and the links between the layers go only into one direction, from the input layer to the output layer. There are no links between the neurons in the same layer. Also, there may be one or several hidden layers between the input and the output layer [5,9,16].

## 2. Preliminaries on Hyperstructures

From an algebraic point of view, it is useful to describe the terms and concepts used in the field of algebraic structures. A hypergroupoid is a pair $( H , · )$, where H is a (nonempty) set and
$· : H × H → P * ( H ) ( = P ( H ) − { ∅ } )$
is a binary hyperoperation on the set $H .$ If $a · ( b · c ) = ( a · b ) · c$ for all $a , b , c ∈ H$ (the associativity axiom), the the hypergroupoid $( H , · )$ is called a semihypergroup. A semihypergroup is said to be a hypergroup if the following axiom:
$a · H = H = H · a$
for all $a ∈ H$ (the reproduction axiom), is satisfied. Here, for sets $A , B ⊆ H , A ≠ ∅ ≠ B$ we define as usually
$A · B = ⋃ { a · b ; a ∈ A , b ∈ B } .$
Thus, hypergroups considered in this paper are hypergroups in the sense of F. Marty [23,24]. In some constructions it is useful to apply the following lemma (called also the Ends-lemma having many applications—cf. [25,26,27,28,29]). Recall, first that by a (quasi-)ordered semigroup we mean a triad $( S , · , ≤ ) ,$ where $( S , · )$ is a semigroup, $( S , ≤ )$ is a (quasi-)ordered set, i.e., a set S endowed with a reflexive and transitive binary relation “≤” and for all triads of elements $a , b , c ∈ S$ the implication $a ≤ b ⇒ a · c ≤ b · c , c · a ≤ c · b$ holds.
Lemma 1 (Ends-Lemma).
Let $( S , · , ≤ )$ be a (quasi-)ordered semigroup. Define a binary hyperoperation
$∗ : S × S → P * ( S ) b y a ∗ b = { x ∈ S ; a · b ≤ x } .$
Then $( S , ∗ )$ is a semihypergroup. Moreover, if the semigroup $( S , · )$ is commutative, then the semihypergroup $( S , ∗ )$ is also commutative and if $( S , · , ≤ )$ is a (quasi-)ordered group then the semihypergroup $( S , ∗ )$ is a hypergroup.
Notice, that if $( G , · ) , ( H , · )$ are (semi-)hypergroups, then a mapping $h : G → H$ is said to be the homomorphism of $( G , · )$ into $( H , · )$ if for any pair $a , b ∈ G$ we have
$h ( a · b ) ⊆ h ( a ) · h ( b ) .$
If for any pair $a , b ∈ G$ the equality $h ( a · b ) = h ( a ) · h ( b )$ holds, the homomorphism h is called the good (or strong) homomorphism—cf. [30,31]. By $E n d G$ we denote the endomorphism monoid of a semigroup (group) $G .$
Linear differential operators described in the article and used e.g., in [29,42] are of the following form:
Definition 1.
Let $J ⊆ R$ be an open interval, $C ( J )$ be the ring of all continuous functions $φ : J → R .$ For $p k ∈ C ( J ) , k = 0 , ⋯ , n − 1 , p 0 ≠ 0$ we define
$L ( p n − 1 , ⋯ , p 0 ) y ( x ) = y ( n ) ( x ) + ∑ k = 0 n − 1 p k ( x ) y ( k ) ( x ) , y ∈ C n ( J )$
(the ring of all smooth functions up to order n, i.e., having derivatives up to order n defined on the interval $J ⊆ R ) .$
Definition 2
([41,49]). Let $( G , · )$ be a semigroup and $P ⊂ G , P ≠ ∅ .$ A hyperoperation $∗ P : G × G → P ( G )$ defined by $[ x , y ] → x P y , i . e . , x ∗ y = x P y$ for any pair $[ x , y ] ∈ P × P$ is said to be the P-hyperoperation in G. If
$x ∗ P ( y ∗ P z ) = x P y P z = ( x ∗ P y ) ∗ P z$
holds for any triad $x , y , z ∈ G ,$ the P-hyperoperation is associative. If also the axiom of reproduction is satisfied, the hypergrupoid $( G , ∗ P )$ is said to be a P-hypergroup.
Evidently, if $( G , · )$ is a group, then also $( G , ∗ P )$ is a P-hypergroup. If the set P is a singleton, then the P-operation $∗ P$ is a usual single—valued operation.
Definition 3.
A subset $H ⊂ G$ is said to be a sub-P-hypergroup of $( G , ∗ P )$ if $P ⊂ H ⊂ G$ and $( H , * P )$ is a hypergroup.
Now, similarly as in the case of the collection of linear differential operators , we will construct a group and hypergroup of artificial neurons, cf. [29,32,42,43,44].
Denote by $δ i j$ Kronecker delta, $i , j ∈ N ,$ i.e., $δ i i = δ j j = 1$ and $δ i j = 0$, whenever $i ≠ j .$
Suppose $N e ( w → r ) , N e ( w → s ) ∈ AN ( T ) , r , s ∈ C ( J ) , w → r = ( w r 1 , ⋯ , w r , n ) , w → s = ( w s 1 , ⋯ , w s , n ) , n ∈ N .$ Let $m ∈ N , 1 ≤ m ≤ n$ be a such an integer that $w r , m > 0$. We define
$N e ( w → r ) · m N e ( w → s ) = N e ( w → u ) ,$
where
$w → u = ( w u , 1 , ⋯ , w u , n ) = ( w u , 1 ( t ) , ⋯ , w u , n ( t ) ) ,$
$w → u , k ( t ) = w r , m ( t ) w s , k ( t ) + ( 1 − δ m , k ) w r , k ( t ) , t ∈ T$
and, of course, the neuron $N e ( w → u )$ is defined as the mapping $y u ( t ) = ∑ k = 1 n w k ( t ) x k ( t ) + b u , t ∈ T ,$$b u = b r b s .$ Further, for a pair $N e ( w → r ) , N e ( w → s )$ of neurons from $AN ( T )$ we put $N e ( w → r ) ≤ m N e ( w → s ) ,$$w r = ( w r , 1 ( t ) , ⋯ , w r , n ( t ) ) ,$$w s = ( w s , 1 ( t ) , ⋯ , w s , n ( t ) )$ if $w r , k ( t ) ≤ w s , k ( t ) , k ∈ N , k ≠ m$ and $w r , m ( t ) = w s , m ( t ) ,$$t ∈ T$ and with the same bias. Evidently $( AN ( T ) , ≤ m )$ is an ordered set. A relationship (compatibility) of the binary operation "$· m$" and the ordering $≤ m$ on $AN ( T )$ is given by this assertion analogical to Lemma 2 in .
Lemma 2.
The triad $( AN ( T ) , · m , ≤ m )$ (algebraic structure with an ordering) is a non-commutative ordered group.
Sketch of the proof was published in .
Denoting
$AN 1 ( T ) m = { N e ( w → ) ; w → = ( w 1 , ⋯ , w n ) , w k ∈ C ( T ) , k = 1 , ⋯ , n , w m ( t ) ≡ 1 } , 1 ≤ m ≤ n ,$
we get the following assertion:
Proposition 1
(Prop. 1. , p. 239). Let $T = 〈 0 , t 0 ) ⊂ R , t 0 ∈ R ∪ { ∞ }$. Then for any positive integer $n ∈ N , n ≥ 2$ and for any integer m such that $1 ≤ m ≤ n$ the semigroup $( AN 1 ( T ) m , · m )$ is an invariant subgroup of the group $( AN ( T ) m , · m ) .$
Proposition 2
(Prop. 2. , p. 240). Let $t 0 ∈ R , t 0 > 0 , T = 〈 0 , t 0 ) ⊂ R$ and $m , , n ∈ N$ are integers such that $1 ≤ m ≤ n − 1$. Define a mapping $F : AN n ( T ) m → LA n ( T ) m + 1$ by this rule: For an arbitrary neuron $N e ( w → r ) ∈ AN n ( T ) m ,$ where $w → r = ( w r , 1 ( t ) , ⋯ , w r , n ( t ) ) ∈ [ C ( T ) ] n$ we put $F ( N e ( w → r ) ) = L ( w r , 1 , ⋯ , w r , n ) ∈ LA n ( T ) m + 1$ with the action:
$L ( w r , 1 , ⋯ , w r , n ) y ( t ) = d n y ( t ) d t n + ∑ k = 1 n w r , k ( t ) d k − 1 y ( t ) d t k − 1 , y ∈ C n ( T ) .$
Then the mapping $F : AN n ( T ) m → LA n ( T ) m + 1$ is a homomorphism of the group $( AN n ( T ) m , · m )$ into the group $( LA n ( T ) m + 1 , ∘ m + 1 ) .$
Now, using the construction described in the Lemma 1, we obtain the final transposition hypergroup (called also non-commutative join space). Denote by $P ( AN ( T ) m ) *$ the power set of $AN ( T ) m$ consisting of all nonempty subsets of the last set and define a binary hyperoperation
$* m : AN ( T ) m × AN ( T ) m → P ( AN ( T ) m ) *$
by the rule
$N e ( w → r ) * m N e ( w → s ) = { N e ( w → u ) ; N e ( w → r ) · m N e ( w → s ) ≤ m N e ( w → u ) }$
for all pairs $N e ( w → r ) , N e ( w → s ) ∈ AN ( T ) m$. More in detail if $w → ( u ) = ( w u , 1 , ⋯ , w u , n ) , w → ( r ) = ( w r , 1 , ⋯ , w r , n ) , w → ( s ) = ( w s , 1 , ⋯ , w s , n )$, then $w r , m ( t ) w s , m ( t ) = w u , m ( t ) , w r , m ( t ) w s , k ( t ) + w r , k ( t ) ≤ w u , k ( t ) ,$ if $k ≠ m , t ∈ T .$ Then we have that $( AN ( T ) m , * m )$ is a non-commutative hypergroup. We say that this hypergroup is constructed by using the Ends Lemma (cf. e.g., [8,25,29]. These hypergroups can be called as EL-hypergroups. The above defined invariant (called also normal) subgroup $( AN 1 ( T ) m , · m )$ of the group $( AN ( T ) m , · m )$ is the carrier set of a subhypergroup of the hypergroup $( AN ( T ) m , * m )$ and it has certain significant properties.
Using certain generalization of methods from  (p. 283), we obtain, after we investigate the constructed structures, the following result:
Theorem 1.
Let $T = 〈 0 , t 0 ) ⊂ R , t 0 ∈ R ∪ { ∞ } .$ Then for any positive integer $n ∈ N , n ≥ 2$ and for any integer m such that $1 ≤ m ≤ n$ the hypergroup $( AN ( T ) m , * m )$,where
$AN ( T ) m = { N e ( w → r ) ; w → r = ( w r , 1 ( t ) , ⋯ , w r , n ( t ) ) ∈ [ C ( T ) ] n , w r , m ( t ) > 0 , t ∈ T } ,$
is a transposition hypergroup (i.e., a non-commutative join space) such that $( AN ( T ) m , * m )$ is its subhypergroup, which is
-
invertible (i.e., $N e ( w → r ) / N e ( w → s ) ∩ AN 1 ( T ) m ≠ ∅$ implies $N e ( w → s ) / N e ( w → r ) ∩ AN 1 ( T ) m ≠ ∅$ and $N e ( w → r ) N e ( w → s ) ∩ AN 1 ( T ) m ≠ ∅$ implies $N e ( w → s ) N e ( w → r ) ∩ AN 1 ( T ) m ≠ ∅$ for all pairs of neurons $N e ( w → r ) , N e ( w → s ) ∈ AN 1 ( T ) m ,$
-
closed (i.e., $N e ( w → r ) / N e ( w → s ) ⊂ AN 1 ( T ) m , N e ( w → r ) ∖ N e ( w → s ) ⊂ AN 1 ( T ) m$ for all pairs $N e ( w → r ) , / , N e ( w → s ) ∈ AN 1 ( T ) m ,$
-
reflexive (i.e., $N e ( w → r ) AN 1 ( T ) m = AN 1 ( T ) m / N e ( w → r )$ for any neuron $N e ( w → r ) ∈ AN ( T ) m$ and
-
normal (i.e.$N e ( w → r ) ∗ AN 1 ( T ) m = AN 1 ( T ) m ∗ N e ( w → r )$ for any neuron $N e ( w → r ) ∈ AN ( T ) m .$
Remark 1.
A certain generalization of the formal (artificial) neuron can be obtained from expression of a linear differential operator of the n-th order. Recall the expression of formal neuron with inner potential $y − i n = ∑ k = 1 n w k ( t ) x k ( t ) ,$ where $x → ( t ) = ( x 1 ( t ) , ⋯ , x n ( t ) )$ is the vector of inputs, $w → ( t ) = ( w 1 ( t ) , ⋯ , w n ( t ) )$ is the vector of weights. Using the bias b of the considered neuron and the transfer function σ we can expressed the output as
Now consider a tribal function $u : J → R ,$ where $J ⊆ R$ is an open interval; inputs are derived from $u ∈ C n ( J )$ as follows: Inputs $x 1 ( t ) = u ( t ) , x 2 = d u ( t ) d t , ⋯ , x n ( t ) = d n − 1 ( t ) d t n − 1 , n ∈ N .$ Further the bias $b = b 0 d n u ( t ) d t n .$ As weights we use the continuous functions $w k : J → R , k = 1 , ⋯ , n − 1 .$
Then formula
is a description of the action of the neuron $D n$ which will be called a formal(artificial) differential neuron. This approach allows to use solution spaces of corresponding linear differential equations.
Proposition 3
(, p. 16). Let $( G 1 , · ) , ( G 2 , · )$ be two groups $f ∈ H o m ( G 1 , G 2 )$ and $P ⊂ G 1 .$ Then the homomorphism f is a good homomorphism between P-hypergroups $( G 1 , ∗ P )$ and $( G 2 , ∗ P ) .$
Concerning the discussed theme see [26,27,28,30,32,36,39,45]. Now denote by $S ⊆ C ( T )$ an arbitrary non/empty subset and let
$P = { N e ( w → u ( t ) ) ; u ∈ S } ⊆ AN ( T ) .$
Then defining
$N e ( w → p ( t ) ) ∗ N e ( w → q ( t ) ) = N e ( w → p ( t ) ) · m P · m N e ( w → q ( t ) ) =$
${ N e ( w → p ( t ) ) · m N e ( w → u ( t ) ) · m N e ( w → q ( t ) ) ; u ∈ S }$
for any pair of neurons $N e ( w → p ( t ) ) , N e ( w → q ( t ) ) ∈ AN ( T )$, we obtain a P-hypergroup of artificial time varying neurons. If S is a singleton, i.e., P is a one-element subset of $AN ( T )$, the obtained structure is a variant of $AN ( T )$. Notice, that any $f ∈ E n d G$ for a group $( G , · )$ induces a good homomorphism of the P-hypergroups $( G , ∗ P ) , ( G , ∗ f ( P ) )$ and any automorphism creates an isomorphism beween the above P-hypergroups.
Let $( Z , + )$ be the additive group of all integers. Let $N e ( w → s ( t ) ) ∈ AN ( T )$ be arbitrary but fixed chosen artificial neuron with the output function $y s ( t ) = ∑ k = 1 n w s , k ( t ) x s , k ( t ) + b s$. Denote by $λ s : AN ( T ) → AN ( T )$ the left translation within the group of time varying neurons determined by $N e ( w → s ( t )$, i.e.,
$λ s ( N e ( w → p ( t ) ) = N e ( w → s ( t ) · m N e ( w → p ( t ) )$
for any neuron $N e ( w → p ( t ) ) ∈ AN ( T )$. Further, denote by $λ s r$ the r-th iteration of $λ s$ for $r ∈ Z$. Define the projection $π s : AN ( T ) × Z → AN ( T )$ by
$π s ( N e ( w → p ( t ) ) , r ) = λ s r ( N e ( w → p ( t ) ) .$
It is easy to see that we get a usual (discrete) transformation group, i.e., the action of $( Z , + )$ (as the phase group) on the group $AN ( T )$. Thus the following two requirements are satisfied:
• $π s ( N e ( w → p ( t ) ) , 0 ) = N e ( w → p ( t ) )$ for any neuron $N e ( w → p ( t ) ) ∈ AN ( T )$,
• $π s ( N e ( w → p ( t ) ) , r + u ) = π s ( π s ( N e ( w → p ( t ) ) , r ) , u )$ for any integers $r , u ∈ Z$ and any artificial neuron $N e ( w → p ( t ) )$. Notice that, in the dynamical system theory this structure is called a cascade.
On the phase set we will define a binary hyperoperation. For any pair of neurons $N e ( w → p ( t ) ) ,$$N e ( w → q ( t ) )$ define
$N e ( w → p ( t ) ) ∗ N e ( w → q ( t ) ) = π s ( N e ( w → p ( t ) ) , Z ) ∪ π s ( N e ( w → q ( t ) ) , Z ) =$
${ λ s a ( N e ( w → p ( t ) ) ; a ∈ Z } ∪ { λ s b ( N e ( w → q ( t ) ) ; b ∈ Z } .$
Then we have that $∗ : AN ( T ) × AN ( T ) → P ( AN ( T ) )$ is a commutative binary hyperoperation and since $N e ( w → p ( t ) ) , N e ( w → q ( t ) ) ∈ N e ( w → p ( t ) ) ∗ N e ( w → q ( t ) )$, we obtain that the hypergroupoid $( AN ( T ) , ∗ )$ is a commutative, extensive hypergroup [20,27,29,30,31,34,35,38,43,46,47]. Using its properties we can characterize certain properties of the cascade $( AN ( T ) , Z , π s ) .$ The hypergroup $( AN ( T ) , ∗ )$ can be called phase hypergroup of the given cascade.
Recall now the concept of invariant subsets of the phase set of a cascade $( X , Z , π s )$ and the concept of a critical point. A subset M of of a phase set X of the cascade $( X , Z , π s )$ is called invariant whenever $π ( x , r ) ∈ M$, for all $x ∈ M$ and all $r ∈ Z$. A critical point of a cascade is an invariant singleton. It is evident that a subset M of neurons, i.e., $M ⊆ AN ( T )$ is invariant in the cascade $( AN ( T ) , Z , π s )$ whenever it is a carrier set of a subhypergroup of the hypergroup $( AN ( T ) , ∗ )$, i.e., M is closed with respect to the hyperoperation ∗, which means $M ∗ M = ⋃ a , b ∈ M a ∗ b ⊆ M$. Moreover, union or intersection of an arbitrary non-empty system $M ⊆ AN ( T )$ is also invariant.

## 3. Main Results

Now, we will construct series of groups and hypergroups of artificial neurons using certain analogy with series of groups of differential operators described in .
We denote by $LA n ( J )$ (for an open interval $J ⊆ R$) the set of all linear differential operators $L ( p n − 1 , ⋯ , p 0 ) , p 0 ≠ 0 , p k ∈ C n ( J ) ,$ i.e., the ring of all continuous functions defined on the interval J, acting as
$L ( p n − 1 , ⋯ , p 0 ) y ( x ) = y n ( x ) + ∑ k = 0 n − 1 p k ( x ) y k ( x ) , y ∈ C n ( J )$
and endowed the binary operation
$L ( q n − 1 , ⋯ , q 0 ) ∘ L ( p n − 1 , ⋯ , p 0 ) = L ( q 0 p n − 1 + q n − 1 , ⋯ , q 0 p 1 + q 1 , q 0 p 0 ) .$
Now denote by $LA ¯ n ( J )$ the set of all operators $L ¯ ( q n , ⋯ , q 0 ) , q 0 ≠ 0 , q k ∈ C ( J )$ acting as
$L ¯ ( q n , ⋯ , q 0 ) y ( x ) = ∑ k = 0 n q k ( x ) y ( k ) ( x ) , q 0 ≠ 0 , q k ∈ C ( J )$
with similarly defined binary operations such that $LA n ( J ) , LA ¯ n ( J )$ are noncommutative groups. Define mappings $F n : LA n ( J ) → LA n − 1 ( J )$ by
$F n ( L ( p n − 1 , ⋯ , p 0 ) ) = L ( p n − 2 , ⋯ , p 0 )$
and $ϕ n : LA ( J ) → LA ¯ n − 1 ( J )$ by
$ϕ n ( L ( p n − 1 , ⋯ , p 0 ) ) = L ¯ ( p n − 2 , ⋯ , p 0 ) .$
It can be easily verified that both $F n$ and $ϕ n ,$ for an arbitrary $n ∈ N$, are group homomorphisms.
Evidently, $LA n ( J ) ⊂ LA ¯ n ( J ) , LA ¯ n − 1 ( J ) ⊂ LA ¯ n ( J )$ for all $n ∈ N$. Thus we obtain complete sequences of ordinary linear differential operators with linking homomorphisms $F n , ϕ n :$
Now consider the groups of time-varying neurons $( AN ( T ) m , · m )$ from Proposition 3 and above defined homomorphism of the group $( AN n ( T ) m , · m )$ into the group $( LA n ( T ) m + 1 , ∘ m + 1 )$. Then we can change the diagram in the following way:
Using the Ends lemma and results the theory of linear operators we can describe also mapping morphisms in sequences groups of linear differential operators:
$LA n ( J ) ← F n + 1 LA n + 1 ( J ) ← F n + 2 LA n + 2 ( J ) ← F n + 3 LA n + 3 ( J ) ← F n + 4 ⋯ as so analogy in sequences groups of time-varying neurons : AN n ( T ) m ← F n + 1 * AN n + 1 ( T ) m ← F n + 2 * AN n + 2 ( T ) m ← F n + 3 * AN n + 3 ( T ) m ← F n + 4 * ⋯$
Theorem 2.
Let $T = 〈 0 , t 0 ) ⊂ R , t 0 ∈ R ∪ { ∞ }$, $n ∈ N$ such that $n ≥ 2 , m ∈ N$ such that $m ≤ n$. Let $( HAN n ( T ) m , * m )$ be the hypergroup obtained from the group $( AN n ( T ) m , ∘ m )$ by Proposition 2. Suppose that $F n : ( AN n ( T ) m , ∘ m ) → ( AN n − 1 ( T ) m , ∘ m )$ are the above defined surjective group homomorphisms. Then $F n : ( HAN n ( T ) m , * m ) → ( HAN n − 1 ( J ) m , * m )$ are surjective homomorphisms of hypergroups.
Remark 2.
The second sequence of (2) can thus be bijectively mapped onto sequence of hypergroups
$HAN n ( T ) m ← F n + 1 HAN n + 1 ( T ) m ← F n + 2 HAN n + 2 ( T ) m ← F n + 3 HAN n + 3 ( T ) m ← F n + 1 ⋯$
wit the linking surjective homomorphisms $F n$. Therefore, the bijective mapping of the above mentioned sequences is functorial.
Now, shift to the concept of an automaton. This was developed as a mathematical interpretation of real-life systems that work on a discrete time-scale. Using the binary operation of concatenation of chains of input symbols we obtain automata with input alphabets endowed with the structure of a semigroup or a group. Considering mainly the structure given by transition function and neglecting output functions with output sets we reach a very useful generalization of the concept of automaton called quasi—automaton [29,31,48,49]. Let us introduce the concept of automata as an action of time varying neurons. Moreover, let system $( A , S , δ )$, consists of nonempty time-varying neuron set of states $A ⊆ AN ( T ) m$, arbitrary semigroup of their inputs S and let mapping $δ : A × S → A$ fulfill the following condition:
$δ ( δ ( a , r ) , s ) = δ ( a , r s ˙ )$
for arbitrary $a ∈ A$ and $r , s ∈ S$ can be understood as a analogy of concept of quasi-automaton, as a generalization of the Mealy-type automaton. The above condition is some times called Mixed Associativity Condition (MAC).
Definition 4.
Let A be a nonempty set, $( H , · )$ a semihypergroup and $δ : A × H → A$ a mapping satisfying the condition:
$δ ( δ ( s , a ) , b ) ∈ δ ( s , a b ˙ ) ( 3 )$
for any triad $( s , a , b ) ∈ A × H × H ,$ where $δ ( s , a b ˙ ) = { δ ( s , x ) ; x ∈ a · b }$. The triad $( A , H , δ )$ is called a quasi-multiautomaton with the state set A and the input semihypergroup $( H , · )$. The mapping $δ : A × H → A$ is called transition function (or next-state function) of the quasi-multiautomaton $( A , H , δ )$. Condition $( 3 )$ is called Generalized Mixed Associativity Condition (or GMAC).
The just defined structures are also called as actions of semihypergroups $( H , · )$ on sets A (called state sets).
Neuron $N e ( w → )$ acts as described above:
$y ( t ) = ∑ i = 1 n w i ( t ) x i ( t ) + b ,$
where i goes from 0 to $n ,$$w i ( t )$ is the weight value in continuous time, b is a bias and $y ( t )$ is the output value in continuous time $t$. Here the transition function F is the identity function.
Now suppose that the input functions $x i$ are differentiable up to arbitrary order $n$.
We consider linear differential operators
$L ( m , w n , ⋯ , w 0 ) : C n ( T ) × ⋯ × C n ( T ) → C n ( T ) , i . e . C n ( T ) × ⋯ × C n ( T ) = [ C n ( T ) ] n + 1 ,$
defined
$L ( m , w n , ⋯ , w 0 ) x ( t ) =$
$= m b + ∑ k = 1 n w k ( t ) d k x k ( t ) d t k , x ( t ) = ( x 0 ( t ) , x 1 ( t ) , ⋯ , x n ( t ) ) ∈ C n ( T ) × ⋯ × C n ( T ) = [ C n ( T ) ] n + 1 .$
Then we denote by $L N e n ( T )$ the additive Abelian group of linear differential operators $L ( m , w n , ⋯ , w 0 )$, where for $L ( m , w n , ⋯ , w 0 ) , L ( k , w n * , ⋯ , w 0 * ) ∈ L N e n ( T )$ with the bias b we define
$L ( m , w n , ⋯ , w 0 ) + L ( s , w n * ⋯ , w 0 * ) = L ( m + s , w n + w n * , ⋯ , w 0 + w 0 * ) ,$
where
$L ( m + s , w n + w n * , ⋯ , w 0 + w 0 * ) x ( t ) =$
$= ( m + s ) b + ∑ k = 0 n ( w k ( t ) + w k * ( t ) ) d k x k ( t ) d t k , t ∈ T a n d x ( t ) = ( x 0 ( t ) , x 1 ( t ) , ⋯ , x n ( t ) ) ∈ [ C n ( T ) ] n + 1 .$
Suppose that $w k ( t ) ∈ C n ( T )$ and define
$δ n : C n ( T ) × L N e n ( T ) → C n ( T )$
by
$δ n ( x ( t ) , L ( m , w n , ⋯ , w 0 ) ) = m b + x ( t ) + m + ∑ k = 0 n w k ( t ) d k x ( t ) d t k , x ( t ) ∈ C n ( T ) , w h e r e$
$w n , ⋯ , w 0$ are weights corresponding with inputs and b is the bias of a neuron corresponding to the operator $L ( m , w n , ⋯ , w 0 ) ∈ L N e n ( T )$.
Theorem 3.
Let $L N e n ( T ) , C n ( T )$ be the above defined structures and $δ n : C n ( T ) × L N e n ( T ) → C n ( T )$ be the above defined mapping. Then the triad $( C n ( T ) , L N e n ( T ) , δ n )$ is an action of the group $L N e n ( T )$ on the group $C n ( T )$, i.e., a quasi-automaton with the state space $C n ( T )$ and with the alphabet $L N e n ( T )$ with the group structure of artificial neurons.
Proof.
We are going to verify the mixed associativity condition (MAC). Suppose $x ∈ C n ( T )$ and $L ( m , w n , ⋯ , w 0 ) , L ( k , u n , ⋯ , u 0 ) ∈ L N e n ( T )$. Then we have
$δ n ( δ n ( x ( t ) , L ( m , w n , ⋯ , w 0 ) , L ( k , u n , ⋯ , u 0 ) ) =$
$= δ n ( m b + x ( t ) + m + ∑ k = 0 n w k ( t ) d k x ( t ) d t k , L ( k , u n , ⋯ , u 0 ) ) =$
$= k b + m b + x ( t ) + m + k + ∑ k = 0 n w k ( t ) d k x ( t ) d t k + ∑ k = 0 n u k ( t ) d k x ( t ) d t k =$
$= ( m + k ) b + x ( t ) + m + k + ∑ k = 0 n ( w k ( t ) + u k ( t ) ) d k x ( t ) d t k =$
$δ n ( x ( t ) , L ( m + k , w n ( t ) + u n ( t ) , ⋯ , w 0 ) ( t ) + u 0 ) ( t ) ) =$
$δ n ( x ( t ) , L ( m , w n , ⋯ , w 0 ) + L ( k , u n , ⋯ , u 0 ) ) ,$
thus MAC is satisfied. □
Consider an interval $T ⊆ R$ and the ring $C ( T )$ of all continuous functions defined on the interval. Let ${ φ k ; k ∈ N }$ be a sequence of ring-endomorphism of $C ( T )$. Denote $A n + k N ( T ) m$ the EL-hypergroup of artificial neurons constructed above, with vectors of weights of the dimension $n + k ∈ N ( = { 1 , 2 , 3 ⋯ } )$. Let $[ C ( T ) ] n + k = C ( T ) × C ( T ) × ⋯ × C ( T ) ( n + k − t i m e s ) i . e . , C ( T ) ] n + k$ is the $n + k$-dimensional cartesian cube. Denote by $φ ¯ k : [ C ( T ) ] n + k → [ C ( T ) ] n + k − 1$ the extension of $φ k$ such that $φ ¯ k ( w → ) = φ ¯ k ( ( w 1 , ⋯ , w n + k − 1 , w n + k ) ) = ( w 1 , ⋯ , w n + k − 1 )$. Let us denote the mapping $F k : A n + k N ( T ) m → A n + k − 1 N ( T ) m$ defined by $F k ( N e ( w → ) = N e ( w → 1 )$ with $w → 1 = ( w 1 , ⋯ , w n + k )$. Consider underlying sets of hypergroups $A n + k N ( T ) m$ endowed with the above defined ordering relation:
$f o r w → = ( w 1 , ⋯ , w n + k ) , u → = ( u 1 , ⋯ , u n + k ) ∈ [ C ( T ) ] n + k$
we hanve $w → ≤ u →$ if $w r ≤ u r , r = 1 , 2 , ⋯ , n + k a n d w m ≤ u m$. Now, for $N e ( w → ) , N e ( u → ) ∈ A n + k N ( T ) m$ such that $w → = ( w 1 , ⋯ , w n + k ) , u → = ( u 1 , ⋯ , u n + k ) , N e ( w → ) ≤ N e ( u → ) ,$ which means $w → ≤ u →$ ($w m = u m$ and biases of corresponding neurons are the same) we have $φ ¯ k ( w → ) = ( w 1 , ⋯ , w n + k − 1 ) ≤ ( u 1 , ⋯ , u n + k − 1 ) = φ ¯ k ( u → ) ,$ which implies $F k ( w → ) ≤ F k ( u → )$.
Consequently the mapping $F k : ( A n + k N ( T ) m , ≤ ) → ( A n + k − 1 N ( T ) m , ≤ )$ is order-preserving, i.e., this is an order-homomorphism of hypergroups. The final result of our considerations is the following sequence of hypergroups of artificial neurons and linking homomorphisms:
$A n N ( T ) m ← F 1 A n + 1 N ( T ) m ← F 2 ⋯ ← F K A n + k N ( T ) m ← F K + 1 A n + k + 1 N ( T ) m ⋯$

## 4. Conclusions

Artificial neural networks and structured systems of artificial neurons have been discussed by a great number of researchers. They are an important part of artificial intelligence with many useful applications in various branches of science and technical constructions. Our considerations are based on algebraic and analytic approach using certain formal similarity with classical structures and new hyperstructures of differential operators. We discussed a certain generalizations of classical artificial time-varying neurons and studied them using recently derived methods. The presented investigations allow further development.

## Author Contributions

Investigation, J.C. and B.S.; Methodology, J.C. and B.S.; Supervision, J.C. and B.S.; Writing original draft, J.C. and B.S.; Writing review and editing, J.C. and B.S.

## Funding

The first author was supported by the FEKT-S-17-4225 grant of Brno University of Technology.

## Conflicts of Interest

The authors declare no conflict of interest.

## References

1. Koskela, T. Neural Network Methods in Analysing and Modelling Time Varying Processes. Ph.D. Thesis, Helsinki University of Technology, Helsinki, Finland, 2003; pp. 1–113. [Google Scholar]
2. Behnke, S. Hierarchical Neural Networks for Image Interpretation. In Lecture Notes in Computer Science; Springer: Heidelberg, Germany, 2003; Volume 2766, p. 245. [Google Scholar]
3. Bishop, C.M. Neural Networks for Pattern Recognition; Oxford University Press: Oxford, UK, 1995; pp. 1–482. [Google Scholar]
4. Buchholz, S. A Theory of Neural Computation With Clifford-Algebras; Technical Report Number 0504; Christian-Albrechts-Universität zu Kiel, Institut für Informatik und Praktische Mathematik: Kiel, Germany, 2005; pp. 1–135. [Google Scholar]
5. Gardner, M.W.; Dorling, S.R. Artificial Neural Networks (the Multilayerperceptron)–a Review of Applications in Theatmospheric Sciences. Atmos. Environ. 1998, 32, 2627–2636. [Google Scholar] [CrossRef]
6. Koudelka, V.; Raida, Z.; Tobola, P. Simple electromagnetic modeling of small airplanes: Neural network approach. Radioengineering 2009, 18, 38–41. [Google Scholar]
7. Krenker, A.; Bešter, J.; Kos, A. Introduction to the artificial neural networks. In Artificial Neural Networks—Methodological Advances and Biomedical Applications; Suzuki, K., Ed.; Tech: Rijeka, Croatia, 2011; pp. 3–18. [Google Scholar]
8. Raida, Z.; Lukeš, Z.; Otevřel, V. Modeling broadband microwave structures by artificial neural networks. Radioengineering 2004, 13, 3–11. [Google Scholar]
9. Rosenblatt, F. The Perceptron: A probabilistic model for information storage and organiyation in the brain. Psychol. Rev. 1958, 65, 386–408. [Google Scholar]
10. Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
11. Tučková, J. Comparison of two Approaches in the Fundamental Frequency Control by Neural Nets. In Proceedings of the 6th Czech-German Workshop in Speech Processing Praha, Praha, Czech Republic, 2–4 September 1996; p. 37. [Google Scholar]
12. Tučková, J.; Boreš, P. The Neural Network Approach in Fundamental Frequency Control. In Speech Processing: Forum Phoneticum; Wodarz, H.-W., Ed.; Hector Verlag: Frankfurt am Main, Germany, 1997; pp. 143–154. ISSN 0341–3144. ISBN 3-930220-10-5. [Google Scholar]
13. Tučková, J.; Šebesta, V. Data Mining Approach for Prosody Modelling by ANN in Text-to-Speech Synthesis. In Proceedings of the IASTED Inernational Conference on “Artificial Intelligence and Applications (AIA 2001)”, Marbella, Spain, 4–7 September 2001; Hamza, M.H., Ed.; ACTA Press Anaheim-Calgary-Zurich: Marbella, Spain, 2001; pp. 164–166. [Google Scholar]
14. Volná, E. Neuronové sítě 1, 2nd ed.; Ostravská Univerzita: Ostrava, Czech Republic, 2008; p. 86. [Google Scholar]
15. Waldron, M.B. Time varying neural networks. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, New Orleans, LA, USA, 4–7 November 1988. [Google Scholar]
16. Widrow, B.; Lehr, A. 30years of adaptive networks: Perceptron, Madaline, and Backpropagation. Proc. IEEE 1990, 78, 1415–1442. [Google Scholar] [CrossRef]
17. Kremer, S. Spatiotemporal Connectionist Networks: A Taxonomy and Review. Neural Comput. 2001, 13, 249–306. [Google Scholar] [CrossRef]
18. Narendra, K.; Parthasarathy, K. Identification and control of dynamical systems using neural networks. IEEE Trans. Neural Netw. 1990, 1, 4–27. [Google Scholar] [CrossRef] [PubMed]
19. Hagan, M.; Demuth, H.; Beale, M. Neural Network Design; PWS Publishing: Boston, MA, USA, 1996. [Google Scholar]
20. Chvalina, J.; Smetana, B. Models of Iterated Artificial Neurons. In Proceedings of the 18th Conference on Aplied Mathematics Aplimat, Bratislava, Slovakia, 5–7 February 2019; pp. 203–212, ISBN 978-80-227-4884-1. [Google Scholar]
21. Chvalina, J.; Smetana, B. Groups and Hypergroups of Artificial Neurons. In Proceedings of the 17th Conference on Aplied Mathematics Aplimat, Bratislava, Slovakia, 6–8 February 2018; University of Technology in Bratislava, Faculty of Mechanical Engineering: Bratislava, Slovakia, 2018; pp. 232–243. [Google Scholar]
22. Chvalina, J.; Smetana, B. Solvability of certain groups of time varying artificial neurons. Ital. J. Pure Appl. Math. 2018. submitted. [Google Scholar]
23. Chvalina, J. Commutative hypergroups in the sense of Marty and ordered sets. In Proceedings of the Summer School an General Algebra and Ordered Sets, Olomouc, Czech Republic, 4–12 September 1994; pp. 19–30. [Google Scholar]
24. Marty, F. Sur une généralisation de la notion de groupe. In Proceedings of the IV Congrès des Mathématiciens Scandinaves, Stockholm, Sweden, 14–18 August 1934; pp. 45–49. [Google Scholar]
25. Novák, M.; Cristea, I. Composition in EL–hyperstructures. Hacet. J. Math. Stat. 2019, 48, 45–58. [Google Scholar] [CrossRef]
26. Novák, M. n-ary hyperstructures constructed from binary quasi-ordered semigroups. An. Stiintifice Ale Univ. Ovidius Constanta Ser. Mat. 2014, 22, 147–168. [Google Scholar] [CrossRef]
27. Novák, M. On EL—Semihypergroups. Eur. J. Comb. 2015, 44, 274–286. [Google Scholar] [CrossRef]
28. Novák, M. Some basic properties of EL-hyperstructures. Eur. J. Combin. 2013, 34, 446–459. [Google Scholar] [CrossRef]
29. Chvalina, J.; Novák, M.; Staněk, D. Sequences of groups and hypergroups of linear ordinary differential operators. Ital. J. Pure Appl. Math. 2019. accepted for publication. [Google Scholar]
30. Corsini, P. Prolegomena of Hypergroup Theory; Aviani Editore Tricesimo: Udine, Italy, 1993. [Google Scholar]
31. Corsini, P.; Leoreanu, V. Applications of Hyperstructure Theory; Kluwer Academic Publishers: Dordrecht, The Netherlands; Boston, MA, USA; London, UK, 2003. [Google Scholar]
32. Chvalina, J.; Hošková-Mayerová, Š.; Dehghan Nezhad, A. General actions of hypergroups and some applications. An. Stiintifice Ale Univ. Ovidius Constanta 2013, 21, 59–82. [Google Scholar]
33. Cristea, I. Several aspects on the hypergroups associated with n-ary relations. An. Stiintifice Ale Univ. Ovidius Constanta 2009, 17, 99–110. [Google Scholar]
34. Cristea, I.; Ştefănescu, M. Binary relations and reduced hypergroups. Discret. Math. 2008, 308, 3537–3544. [Google Scholar] [CrossRef][Green Version]
35. Cristea, I.; Ştefănescu, M. Hypergroups and n-ary relations. Eur. J. Combin. 2010, 31, 780–789. [Google Scholar] [CrossRef][Green Version]
36. Leoreanu-Fotea, V.; Ciurea, C.D. On a P-hypergroup. J. Basic Sci. 2008, 4, 75–79. [Google Scholar]
37. Pollock, D.; Waldron, M.B. Phase dependent output in a time varying neural net. Proc. Ann Conf. EMBS 1989, 11, 2054–2055. [Google Scholar]
38. Račková, P. Hypergroups of symmetric matrices. In Proceedings of the 10th International Congress of Algebraic Hyperstructures and Applications (AHA 2008), Brno, Czech Republic, 3–9 September 2008; pp. 267–272. [Google Scholar]
39. Vougiouklis, T. Generalization of P-hypergroups. In Rendiconti del Circolo Matematico di Palermo; Springer: Berlin, Germany, 1987; pp. 114–121. [Google Scholar]
40. Vougiouklis, T. Hyperstructures and their Representations. In Monographs in Mathematics; Hadronic Press: Palm Harbor, FL, USA, 1994. [Google Scholar]
41. Vougiouklis, T.; Konguetsof, L. P-hypergoups. Acta Univ. C. Math. Phys. 1987, 28, 15–20. [Google Scholar]
42. Chvalina, J.; Chvalinová, L. Modelling of join spaces by n-th order linear ordinary differential operators. In Proceedings of the Fourth International Conference APLIMAT 2005, Bratislava, Slovakia, 1–4 February 2005; pp. 279–284, ISBN 80-969264-2- X. [Google Scholar]
43. Chvalina, J.; Chvalinová, L. Action of centralizer hypergroups of n-th order linear differential operators on rings on smooth functions. J. Appl. Math. 2008, 1, 45–53. [Google Scholar]
44. Neuman, F. Global Properties of Linear Ordinary Differential Equations; Academia Praha—Kluwer Academic Publishers: Dordrecht, The Netherlands; Boston, MA, USA; London, UK, 1991; p. 320. [Google Scholar]
45. Vougiouklis, T. Cyclicity in a special class of hypergroups. Acta Univ. C Math. Phys. 1981, 22, 3–6. [Google Scholar]
46. Chvalina, J.; Svoboda, Z. Sandwich semigroups of solutions of certain functional equations and hyperstuctures determined by sandwiches of functions. J. Appl. Math. 2009, 2, 35–43. [Google Scholar]
47. Cristea, I.; Novák, M.; Křehlík, Š. A class of hyperlattices induced by quasi-ordered semigroups. In Proceedings of the 16th Conference on Aplied Mathematics Aplimat 2017, Bratislava, Slovakia, 31 January–2 February 2017; pp. 1124–1135. [Google Scholar]
48. Bavel, Z. The source as a tool in automata. Inf. Control 1971, 18, 140–155. [Google Scholar] [CrossRef][Green Version]
49. Borzooei, R.A.; Varasteh, H.R.; Hasankhani, A. $ℱ$-Multiautomata on Join Spaces Induced by Differential Operators. Appl. Math. 2014, 5, 1386–1391. [Google Scholar] [CrossRef]

## Share and Cite

MDPI and ACS Style

Chvalina, J.; Smetana, B. Series of Semihypergroups of Time-Varying Artificial Neurons and Related Hyperstructures. Symmetry 2019, 11, 927. https://doi.org/10.3390/sym11070927

AMA Style

Chvalina J, Smetana B. Series of Semihypergroups of Time-Varying Artificial Neurons and Related Hyperstructures. Symmetry. 2019; 11(7):927. https://doi.org/10.3390/sym11070927

Chicago/Turabian Style

Chvalina, Jan, and Bedřich Smetana. 2019. "Series of Semihypergroups of Time-Varying Artificial Neurons and Related Hyperstructures" Symmetry 11, no. 7: 927. https://doi.org/10.3390/sym11070927

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.