Quantum Finite Automata and Quiver Algebras ^†

Abstract

We find an application in quantum finite automata for the ideas and results of [JL21] and [JL22]. We reformulate quantum finite automata with multiple-time measurements using the algebraic notion of a near-ring. This gives a unified understanding towards quantum computing and deep learning. When the near-ring comes from a quiver, we have a nice moduli space of computing machines with a metric that can be optimized by gradient descent.

1. Motivation: QFA and Near-Ring

In quantum theory, the evolution of states is unitary. An observable is modeled by a self-adjoint operator whose eigenvalues are the possible output values, and whose eigenvectors form an orthonormal basis of the state space. As a result, a typical quantum model simply consists of linear operators that form an algebra.

However, when passing from the quantum world to the real world, an actual probabilistic projection to an eigenstate is necessary. Such a probabilistic operation destroys the linear structure, so we need a non-linear (meaning non-distributive) algebraic structure to accommodate such operators. In the 20th century, there were several attempts to solve this problem. See for instance [1,2], and [3] (Chapter 3) for a nice survey. In particular, Pascual Jordan attempted to use near-ring for quantum mechanics.

Let us consider the scenario of quantum computing.

Definition 1

([4]). A quantum finite automata (QFA) is a tuple $\mathcal{Q}=(V,q_0,F,\Sigma ,{\left(U_{\sigma }\right)}_{\sigma \in \Sigma })$ where:

1 

V is a finite set of states which generate the Hilbert space ${\mathcal{H}}_V$;

2 

$F\subset V$ is a set of final or accept states;

3 

$q_0$ the initial state which is a unit vector in ${\mathcal{H}}_V$;

4 

Σ is a finite set called the alphabet;

5 

For each $\sigma \in \Sigma$, $U_{\sigma }$ is a unitary operator on ${\mathcal{H}}_V$.

An input to a QFA consists of a word w in the alphabet $\Sigma$ of the form $w=w_1{w}_2\cdots w_n$ where $w_i\in \Sigma$ for all i. w acts on the initial state of the QFA by $\langle q_0|U_w$, where $U_w$ is the matrix $U_w:=U_{w_1}{U}_{w_2}\cdots U_{w_n}$, and $\langle q_0|$ is the row vector presentation of $q_0$. The probability that word w will end in an accept state is

$$Pr\left(w\right)=\parallel {\langle q_0|U_{w}P\parallel }^2$$

where $P:{\mathcal{H}}_V\to {\mathcal{H}}_F$ is the projection from ${\mathcal{H}}_V$ to subspace ${\mathcal{H}}_F$ spanned by F.

Please note that the above definition has not taken the probabilistic projection into account. We make the following reformulation.

Definition 2.

A quantum computing machine is a tuple $(({\mathcal{H}}_V,h),{\mathcal{H}}_F,e,{\rho }_G,{\sigma }^F)$, where

1 

$({\mathcal{H}}_V,h)$ is a Hermitian vector space;

2 

${\mathcal{H}}_F={\mathbb{C}}^n$ equipped with the standard metric, which is called the framing space;

3 

$e:{\mathcal{H}}_F\to {\mathcal{H}}_V$ is an isometric embedding;

4 

${\rho }_G:G\to U({\mathcal{H}}_V,h)$ is a unitary representation of a group G.

5 

${\sigma }^F:{\mathcal{H}}_F\to {\mathcal{H}}_F$ is a probabilistic projection.

Ignoring the last item (5) for the moment, this coincides with Definition 1 by setting $G=\langle \Sigma \rangle$, the free group generated by a set $\Sigma$, and fixing an initial vector $q_0\in {\mathcal{H}}_V$.

Here, we treat ${\mathcal{H}}_F$ as a vector space of its own and take an isometric embedding $e:{\mathcal{H}}_F\to {\mathcal{H}}_V$, rather than directly identifying ${\mathcal{H}}_F$ as a subspace in ${\mathcal{H}}_V$. The state space ${\mathcal{H}}_V$ is treated as an abstract vector space without a preferred basis, while ${\mathcal{H}}_F$ is equipped with a fixed basis that has a real physical meaning (like up/down spinning of an electron). The framing map $e:{\mathcal{H}}_F\to {\mathcal{H}}_V$ is interpreted as a bridge between the classical and the quantum world; the image of the fixed basis under e determines a subset of pure state vectors of a certain observable. The adjoint $e^{*}:{\mathcal{H}}_V\to {\mathcal{H}}_F$ is an orthogonal projection. In the next section, e is no longer required to be an embedding when we consider non-unitary generalizations for machine learning.

For the last item (5), the probabilistic projection ${\sigma }^F:{\mathcal{H}}_F\to {\mathcal{H}}_F$ can be modeled by a probability space. Specifically, consider a ${\mathcal{H}}_F$-family of random variables

$$k:{\mathcal{H}}_F\times \Omega \to \{1,\dots ,|F\left|\right\}$$

where $\Omega$ is a probability space (that has a probability measure), with the assumption that $\mathrm{Pr}(k\left(v\right)=j)=\langle \frac{v}{\parallel v\parallel },{ϵ}_j\rangle$ for every $v\in {\mathcal{H}}_F$, where ${ϵ}_j\in {\mathcal{H}}_F$ denotes the j-th basic vector. Then ${\sigma }^F\left(v\right):={ϵ}_{k\left(v\right)}$.

The major additional ingredients in Definition 2, compared to Definition 1, are $e,e^{*}$ and ${\sigma }^F$. Please note that they are not yet included in the machine language, which is currently the group G. Since e and ${\sigma }^F$ are not invertible, we cannot enlarge G to include e nor ${\sigma }^F$ as a group.

To remedy this, first note that (4) can be replaced with an algebra rather than a group, which exhibits linearity and allows not being invertible. Specifically, we require instead:

(4’)

${\rho }_A:A\to \mathrm{End}\left({\mathcal{H}}_V\right)$ is an algebra homomorphism for an algebra A (with unit $1_A$).

For instance, A can be the free algebra generated by a set $\Sigma$.

With such a modification, we can easily include the framing e and $e^{*}$ into our language by taking the augmented algebra

$$\mathcal{A}=A\langle 1_F,e,e^{*}\rangle /R$$

(1)

where R is generated by the relations $1_F·1_F=1_F,1_F·e^{*}=e^{*},e·1_F=e,1_A·e=e,e·e=0,e^{*}·e^{*}=0,1_F·e=0,1_F·a=0,e·1_A=0,e^{*}·1_F=0,1_A·e^{*}=0$ for any $a\in A$. The unit of $\mathcal{A}$ is $1_A+1_F$.

However, we cannot further enlarge $\mathcal{A}$ to include ${\sigma }^F$ as an algebra. The reason for this is that ${\sigma }^F$ always maps to unit vectors and cannot be linear:

$${\sigma }^F(v+w)\ne {\sigma }^F\left(v\right)+{\sigma }^F\left(w\right).$$

To extend $\mathcal{A}$ by ${\sigma }^F$ which models actual quantum measurement, we need the notion of a near-ring. It is a set A with two binary operations +, ∘ such that A is a group under `+’, `∘’ is associative, and right multiplication is distributive over addition: $(x+y)\circ z=x\circ z+y\circ z$ for all $x,y,z\in A$ (but left multiplication is not required distributive: $z\circ (x+y)\ne z\circ x+z\circ y$).

Define $\tilde{\mathcal{A}}$ to be the near-ring

$$\tilde{\mathcal{A}}:=(1_F+e^{*}·\mathcal{A}·e)\left\{{\sigma }^F\right\}.$$

This near-ring can be understood as the language that controls quantum computing machines. Elements of $\tilde{\mathcal{A}}$ can be recorded as rooted trees. An example is $a_1{\sigma }^F\circ (a_{11}+a_{12})$ where $a_1,a_{11},a_{12}\in (1_F+e^{*}·\mathcal{A}·e)$. See also the tree on the left-hand side of Figure 1.

The advantage of putting all the algebraic structures into a single near-ring is that we can consider all the quantum computing machines (mathematically $\tilde{\mathcal{A}}$-modules) controlled by a single near-ring at the same time. An element of $\tilde{\mathcal{A}}$ is a quantum algorithm, which can run in all quantum computers controlled by $\tilde{\mathcal{A}}$.

2. Near-Ring and Differential Forms

In the setting of Definition 1 and Example 1, it is natural to relax the representations from unitary groups to matrix algebras $\mathfrak{gl}(n,\mathbb{C})$. Moreover, the quantum measurement can also be simulated by a non-linear function (called activation function). Such a modification will produce a computational model of deep learning.

Definition 3.

An activation module consists of:

1 

A noncommutative algebra A and vector space V, F;

2 

A family of metrics $h_{(\rho ,e)}$ on V over the space of framed A-modules

$$R={\mathit{Hom}}_{\mathit{alg}}(A,\mathit{End}\left(V\right))\times \mathit{Hom}(F,V)$$

which is G-equivariant where $G=\mathrm{GL}\left(V\right)$;

(3)

A collection of possibly non-linear functions

$${\sigma }_j^F:F\to F.$$

As in (1), we take the augmented algebra $\mathcal{A}=A\langle 1_F,e,e^{*}\rangle /R$ which produces linear computations in all framed A-modules simultaneously. With item (3), elements in the near-ring

$$\tilde{\mathcal{A}}:=(1_F+e^{*}·\mathcal{A}·e)\{{\sigma }_1^F,\dots ,{\sigma }_N^F\}$$

induce non-linear functions on F, and so they are called non-linear algorithms. An example of how $\tilde{\mathcal{A}}$ induces non-linear functions on F upon fixing a point in R is given in Example 1.

$R={\mathrm{Hom}}_{\mathrm{alg}}(A,\mathrm{End}\left(V\right))\times \mathrm{Hom}(F,V)$ is understood as a family of computing machines: a point $(w,e)$ in R fixes how $\mathcal{A}$ acts on V and the framing map $e\in \mathrm{Hom}(F,V)$, and hence entirely determines how an algorithm runs in the machine corresponding to $(w,e)$.

Let us emphasize that the state space V is basis-free. The family of metrics is $\mathrm{GL}\left(V\right)$-equivariant: $h_{(\rho ,e)}(v,w)=h_{g·(\rho ,e)}(g·v,g·w)$ for any $g\in \mathrm{GL}\left(V\right)$. Thus, given $a\in \tilde{\mathcal{A}}$, the non-linear functions that a induces for the two machines $r\in R$ and $g·r\in R$ equal to each other. In other words, an algorithm $a\in \tilde{\mathcal{A}}$ drives all machines parametrized by the moduli stack$[R/\mathrm{GL}(V\left)\right]$ to produce functions on F:

$$\tilde{\mathcal{A}}\times [R/\mathrm{GL}\left(V\right)]\to \mathrm{Map}(F,F).$$

(2)

As mentioned above, the advantage is that the single near-ring $\tilde{A}$ controls all machines in $[R/\mathrm{GL}(V\left)\right]$ and for all V simultaneously (independent of $dimV$).

In [5], we formulated noncommutative differential forms on a near-ring $\tilde{\mathcal{A}}$, which induce $\mathrm{Map}(F,F)$-valued differential forms on the moduli $[R/\mathrm{GL}(V\left)\right]$. It is extended from the Karoubi-de Rham complex [6,7,8,9] for algebras to near-rings. (2) above is the special case for 0-forms, which are simply elements in $\tilde{\mathcal{A}}$. The cases of 0-forms and 1-forms are particularly important for gradient descent: recall that gradient of a function is the metric dual of the differential of that function.

Theorem 1

([5]). There exists a degree-preserving map

$$D{R}^{•}\left(\tilde{\mathcal{A}}\right)\to {\left({\Omega }^{•}(R,\mathrm{Map}(F,F))\right)}^{\mathrm{GL}\left(V\right)}$$

which commutes with d on the two sides.

A differential form on $\tilde{\mathcal{A}}$ can be recorded as a form-valued tree, see the right-hand side of Figure 1. They are rooted trees whose edges are labeled by $\varphi \in D{R}^{•}\left({\mathrm{Mat}}_F\left(\widehat{\mathcal{A}}\right)\right)$; leaves are labeled by $\alpha \in \tilde{\mathcal{A}}$; the root is labeled by 1 (if not a leaf); nodes which are neither leaves nor the root are labeled by the symbols $D_{{\sigma }_{\ell }}^{\left(p\right)}{|}_{\alpha }$ that correspond to the p-th order symmetric differentials of ${\sigma }_{\ell }$.

In application to machine learning, an algorithm $\tilde{\gamma }\in \tilde{\mathcal{A}}$ induces a 0-form of $\tilde{\mathcal{A}}$, for instance

$${\int }_K{\left|\tilde{\gamma }\left(x\right)-f\left(x\right)\right|}^{2}dx$$

(3)

for a given dataset encoded as a function $f:K\to \mathbb{R}$. This 0-form and its differential induces the cost function and its differential on $[R/G]$, respectively, which are the central objects in machine learning.

The differential forms are G-equivariant by construction. There have been a lot of recent works in learning for input data set that has Lie group symmetry [10,11,12,13,14,15,16]. On the other hand, our work has focused on the internal symmetry of the computing machine.

In general, the existence of fine moduli is a big problem in mathematics: the moduli stack $[R/G]$ may be singular and pose difficulties in applying gradient descent. Fortunately, if A is a quiver algebra, its moduli space of framed quiver representations $[R/G]$ is a smooth manifold $\mathcal{M}$ (with respect to a chosen stability condition) [17]. This leads us to deep learning explained in the next section.

3. Deep Learning over the Moduli Space of Quiver Representations

An artificial neural network (see Figure 2 for a simple example) consists of:

• a graph $Q=(Q_0,Q_1)$, where $Q_0$ is a (finite) set of vertices (neurons) and $Q_1$ is a (finite) set of arrows starting and ending in vertices in $Q_0$ (transmission between neurons);
• a quiver representation of Q, which associates a vector space $V_i$ to each vertex i and a linear map $w_a$ (called weights) to each arrow a. We denote by $t\left(a\right)$ and $h\left(a\right)$ the tail and head of an arrow a, respectively.
• a non-linear function $V_i\to V_i$ for each vertex i (called an activation function for the neuron).

Activation functions are an important ingredient for neural network; on the other hand, it rarely appears in quiver theory. Its presence allows the neural network to produce non-linear functions.

Remark 1.

In the recent past there has been rising interest in the relations between machine learning and quiver representations [5,18,19,20]. Here, we simply put quiver representation as a part of the formulation of an artificial neural network.

In many applications, the dimension vector $\overrightarrow{d}\in {\mathbb{Z}}_{\ge 0}^{Q_0}$ is set to be $(1,\dots ,1)$, i.e., all vector spaces $V_i$ associated with the vertices are one-dimensional. For us, it is an unnecessary constraint, and we allow $\overrightarrow{d}$ to be any fixed integer vector.

Any non-trivial non-linear function $V_i\to V_i$ cannot be $\mathrm{GL}\left(V_i\right)$-equivariant. However, in quiver theory, $V_i$ is understood as a basis-free vector space and requires $\mathrm{GL}\left(V_i\right)$-equivariance. We resolve this conflict between neural network and quiver theory in [19] using framed quiver representations. The key idea is to put the non-linear function on the framing rather than on the basis-free vector spaces $V_i$.

Combining with the setting of the last section (Definition 2), we take:

• $A=\mathbb{C}Q$, the quiver algebra. Elements are formal linear combinations of paths in Q (including the trivial paths at vertices), and product is given by concatenation of paths.
• $V={⨁}_i{V}_i$, the direct sum of all vector spaces over vertices.
• Each vertex is associated with a framing vector space $F_i$. Then $F={⨁}_i{F}_i$.
• Each point $(w,e)\in R={\mathrm{Hom}}_{\mathrm{alg}}(\mathbb{C}Q,\mathrm{End}\left(V\right))\times {⨁}_i\mathrm{Hom}(F_i,V_i)$ is a framed quiver representation. Specifically, $w\in {\mathrm{Hom}}_{\mathrm{alg}}(\mathbb{C}Q,\mathrm{End}\left(V\right))$ associates a matrix $w_a$ to each arrow a of Q; $e^{\left(i\right)}\in \mathrm{Hom}(F_i,V_i)$ are the framing linear maps.
• The group G is taken to be ${\prod }_i\mathrm{GL}\left(V_i\right)$. An element $g={\left(g_i\right)}_{i\in Q_0}$ acts on R by

  $$g·\left({\left(w_a\right)}_{a\in Q_1},{\left(e^{\left(i\right)}\right)}_{i\in Q_0}\right)=\left({(g_{h\left(a\right)}·w_a·g_{t\left(a\right)}^{-1})}_{a\in Q_1},{(g_i·e^{\left(i\right)})}_{i\in Q_0}\right).$$
• We have (possibly non-linear) maps ${\sigma }_i:F_i\to F_i$ for each vertex. To match the notation of Definition 2, ${\sigma }_i$ can be taken as maps $F\to F$ by extension by zero.

By the celebrated result of [17], we have a fine moduli space of framed quiver representations $\mathcal{M}={\mathcal{M}}_{n,d}=R⫽G$, where $n,d$ are the dimension vectors for the framing ${\left\{F_i\right\}}_{i\in Q_0}$ and representation ${\left\{V_i\right\}}_{i\in Q_0}$, respectively. In particular, we have the universal vector bundles ${\mathcal{V}}_i$ over $\mathcal{M}$, whose fiber over each framed representation $[w,e]\in \mathcal{M}$ is the representing vector space $V_i$ over the vertex i.

${\mathcal{V}}_i$ plays an important role in our computational model, namely a vector $v\in {\mathcal{V}}_i$ over a point $[w,e]\in \mathcal{M}$ is the state of the i-th neuron in the machine parametrized by $[w,e]$.

Remark 2.

The topology of $\mathcal{M}$ is well understood by [21] as iterated Grassmannian bundles. Framed quiver representations and their doubled counterparts play an important role in geometric representation theory [22,23].

To fulfill Definition 2 (see Item (2)), we need to equip each ${\mathcal{V}}_i$ with a bundle metric $h_i$, so that the adjoint $e^{*}$ makes sense. In [19], we have found a bundle metric that is merely written in terms of the algebra $\mathcal{A}$. It means the formula works for (infinitely many) quiver moduli for all dimension vectors of representations simultaneously.

Theorem 2

([19]). For a fixed vertex $\left(i\right)$, let ${\rho }_i$ be the row vector whose entries are all the elements of the form $w_{\gamma }{e}^{\left(t\right(\gamma \left)\right)}:R_{n,d}\to \mathrm{Hom}({\mathbb{C}}^{n_{t\left(\gamma \right)}},{\mathbb{C}}^{d_i})$ such that $h\left(\gamma \right)=i$. Consider

$$H_i:={\rho }_i{\rho }_i^{*}=\sum _{h\left(\gamma \right)=i}\left(w_{\gamma }{e}^{\left(t\right(\gamma \left)\right)}\right){\left(w_{\gamma }{e}^{\left(t\right(\gamma \left)\right)}\right)}^{*}$$

(4)

as a map ${\rho }_i{\rho }_i^{*}:R_{n,d}\to \mathit{End}\left({\mathbb{C}}^{d_i}\right)$. Then ${\left({\rho }_i{\rho }_i^{*}\right)}^{-1}$ is $\mathrm{GL}\left(d\right)$-equivariant and descends to a Hermitian metric on ${\mathcal{V}}_i$ over $\mathcal{M}$.

Example 1.

Consider the network in Figure 2. The quiver has the arrows $a_{1,k}^{\left(1\right)},a_{2,k}^{\left(1\right)}$ for $k=1,\dots ,s$ (between the input and hidden layers) and $a_k^{\left(2\right)}$ (between the hidden and output layers). In application, we consider the algorithm

$$\tilde{\gamma }:=\sum _{k=1}^s{\widehat{a}}_k^{\left(2\right)}{\sigma }_k\circ \sum _{j=1}^2{\widehat{a}}_{jk}^{\left(1\right)}\in \tilde{\mathcal{A}}$$

where ${\widehat{a}}_{jk}^{\left(1\right)}:={\left(e^{\left(j\right)}\right)}^{*}{a}_{jk}^{\left(1\right)}{e}^{{\mathit{in}}_j}$ and ${\widehat{a}}_k^{\left(2\right)}:={\left(e^{\mathit{out}}\right)}^{*}{a}_k^{\left(2\right)}{e}^{\left(k\right)}$. Please note that the adjoints ${\left(e^{\left(j\right)}\right)}^{*}$ and ${\left(e^{\mathit{out}}\right)}^{*}$ are with respect to the metric $H_i$ and $H_{\mathit{out}}$, respectively.

$\tilde{\gamma }$ is recorded by the activation tree on the left-hand side of Figure 1, for the case $s=2$. $\tilde{\gamma }$ drives any activation module (with this given quiver algebra) to produce a function $F_{{\mathrm{in}}_1}\times F_{{\mathrm{in}}_2}\to F_{\mathrm{out}}$. For instance, setting the representing dimension to be 1 and taking ${\sigma }_i$ to be the ReLu function $max\{x,0\}$ on $\mathbb{R}$ is a popular choice. Data passes from the leaves to the root, which is called forward propagation.

Figure 1 shows the differential of $\tilde{\gamma }\in \tilde{\mathcal{A}}$. This 1-form is given by

$$d(a_1{\sigma }_1\circ (a_{11}+a_{12})+a_2{\sigma }_2\circ (a_{21}+a_{22}))$$

$$=d{a}_1({\alpha }_1+D_{{\sigma }_1}^{\left(1\right)}{|}_{{\alpha }_1}(d{a}_{11}+d{a}_{12}))+d{a}_2({\alpha }_2+D_{{\sigma }_2}^{\left(1\right)}{|}_{{\alpha }_2}(d{a}_{21}+d{a}_{22}))$$

where ${\alpha }_j={\sigma }_j\circ (a_{j1}+a_{j2})$. The terms are obtained by starting at the output node and moving backwards through the activation tree, which is well known as the backpropagation algorithm. Please note that this works on the algebraic level and is not specific to any representation.

$d\tilde{\gamma }$ induces a $\mathrm{Map}(F_{{\mathrm{in}}_1}\times F_{{\mathrm{in}}_2},F_{\mathrm{out}})$-valued 1-form on ${\mathcal{M}}_{(n,d)}$. We can also easily produce $\mathbb{R}$-valued 1-forms, for instance by (3).

For stochastic gradient descent over the moduli ${\mathcal{M}}_{n,d}$, to find the optimal machine, we still need one more ingredient: a metric on ${\mathcal{M}}_{n,d}$, to turn a one-form to a vector field. Very nicely, the Ricci curvature of the metric (4) given above gives a well-defined metric on ${\mathcal{M}}_{n,d}$. So far, all the ingredients involved (namely the algorithm $\tilde{\gamma }$, its differential, the bundle metric $H_i$ and the metric on moduli) are purely written in algebraic symbols and work for moduli spaces in all dimensions $(n,d)$ simultaneously.

Theorem 3

([19]). Suppose Q has no oriented cycles. Then

$$H_T:=\sum _i\partial \overline{\partial }log\mathit{det}{H}_i=\sum _i\left(tr{(\partial {\rho }_i)}^{*}{H}_i\partial {\rho }_i-tr\left(H_i{\rho }_i{(\partial {\rho }_i)}^{*}{H}_i(\partial {\rho }_i){\rho }_i^{*}\right)\right)$$

(5)

defines a Kähler metric on ${\mathcal{M}}_{n,d}$ for any $(n,d)$.

Example 2.

Let us consider the network of Figure 2 again, with $s=2$ for simplicity. Let $n=d=(1,1,1,1)$. Over the chart where $e^{\left(i\right)}\ne 0$ for all $i={\mathit{in}}_1,{\mathit{in}}_2,1,2,\mathit{out}$, the $\mathrm{GL}\left(d\right)$-equivariance allows us to assume that $e^{\left(i\right)}{\left(e^{\left(i\right)}\right)}^{*}=1$. Then $H_{{\mathit{in}}_j}$ are trivial for $j=1,2$, and so $\partial \overline{\partial }log{H}_{{\mathrm{in}}_j}=0$. Let $x_1=(w_{11}^{\left(1\right)},w_{21}^{\left(1\right)})$ and $x_2=(w_{12}^{\left(1\right)},w_{22}^{\left(1\right)})$. We have

$$\partial \overline{\partial }log{H}_j=\partial \overline{\partial }log{\left(1+|x_j{|}^2\right)}^{-1}=\frac{(1+|x_j{|}^2)d{x}_j\wedge d{\overline{x}}_j^t+\overline{x_j}d{x}_j^{t}d\overline{x_j}{x}_j^t}{(1+|x_j{{|}^2)}^2};$$

$$\partial \overline{\partial }log{H}_{out}=\partial \overline{\partial }log{\left(1+|w_1^{\left(2\right)}{|}^2|x_1{|}^2+|w_2^{\left(2\right)}{|}^2{\left|x_2\right|}^2\right)}^{-1}=\frac{d{x}_j\wedge d\overline{x_j}+d{w}_j^{\left(2\right)}\wedge d{\overline{w_j}}^{\left(2\right)}}{(1+|w_1^{\left(2\right)}{|}^2|x_1{|}^2+|w_2^{\left(2\right)}{|}^2\left|x_2{|}^2\right)}$$

$$+\frac{\left(\right|w_j^{\left(2\right)}{|}^2\overline{x_j}d{x}_j^{t}d\overline{x_j}{x}_j^t)+(|x_j{|}^{2}d{w}_j\wedge d\overline{w_j})+(\overline{w_j}{x}_{j}d{w}_j\wedge d\overline{x_j})+(w_j\overline{x_j}d{x}_j\wedge d\overline{w_j})}{(1+|w_1^{\left(2\right)}{|}^2|x_1{|}^2+|w_2^{\left(2\right)}{|}^2|x_2{{|}^2)}^2}.$$

4. Uniformization of Metrics over the Moduli

The original formulation of deep learning is over the flat vector space of representations ${\mathrm{Hom}}_{\mathrm{alg}}(\mathbb{C}Q,\mathrm{End}\left(V\right))$, rather than the moduli space $\mathcal{M}=R⫽G$ of framed representations which has a semi-positive metric $H_T$. In [5], we found the following way of connecting our new approach with the original approach by varying the bundle metric $H_i$ in (4).

We shall assume $n_i\ge d_i$$\forall i$. Let us write the framing map (which is a rectangular matrix) as $e^{\left(i\right)}=({ϵ}_i$$b_i)$ where ${ϵ}_i$ is the largest square matrix and $b_i$ is the remaining part. (In applications $b_i$ usually consists of `bias vectors’.) This allows us to rewrite Equation (4) in the following way:

$$H_i\left(\alpha \right)={\left({ϵ}_i{ϵ}_i^{*}+{\alpha }_{e^{\left(i\right)}}{b}_i{b}_i^{*}+\sum _{\gamma :h\left(\gamma \right)=i,\gamma \ne e^{\left(i\right)}}{\alpha }_{\gamma }{w}_{\gamma }{e}^{t\left(\gamma \right)}{\left(w_{\gamma }{e}^{t\left(\gamma \right)}\right)}^{*}\right)}^{-1}.$$

with $\alpha ={\left({\alpha }_{\gamma }\right)}_{\gamma :h\left(\gamma \right)=i}=(1,\dots ,1)$.

If instead, we set ${\alpha }_{\gamma }$ to different values, then $H_i\left(\alpha \right)$ will still be G-equivariant, but it may no longer positive definite on the whole space $\mathcal{M}$. Motivated by the brilliant construction of dual Hermitian symmetric spaces, we define

$$\mathcal{M}\left(\alpha \right):=\{[w,e]\in \mathcal{M}:H_i\left(\alpha \right)\mathrm{is}\mathrm{positive}\mathrm{definite}\mathrm{at}[w,e]\}.$$

Elements $[w,e]\in \mathcal{M}\left(\alpha \right)$ are called space-like representations with respect to $H_i\left(\alpha \right)$. If we set $\alpha =\overrightarrow{0}$, it becomes

$$H_i^0:={\left({ϵ}_i{ϵ}_i^{*}\right)}^{-1}$$

and ${\mathcal{M}}^0:=\mathcal{M}(\alpha =\overrightarrow{0})$ is exactly the flat vector space ${\mathrm{Hom}}_{\mathrm{alg}}(\mathbb{C}Q,\mathrm{End}\left(V\right))$. This recovers the original Euclidean learning.

On the other hand, setting $\alpha =(-1,\dots ,-1)$, we obtain a semi-negative moduli space $({\mathcal{M}}^{-},H_T)$ [5], which is a generalization of the hyperbolic spaces or non-compact dual of Grassmannians. This is a very useful setting as there have been several fascinating works done on machine learning performed over hyperbolic spaces, for example [24,25,26,27].

Remark 3.

The above construction gives a family of metrics $H_i$ parametrized by α. ${\alpha }_{\gamma }$ can be interpreted as filtering parameters that encode the importance of the paths γ. It is interesting to compare this with the celebrated attention mechanism. We can build models that use these parameters to filter away noise signals during the learning process. We are investigating the applications in transfer learning, especially in situations where we only have a small sample of representative data.