Natural Methods of Unsupervised Topological Alignment

Abstract

In the paper, we present a comparison analysis of the methods of the topological alignment and extract the main mathematical principles forming the base of the concept. The main narrative is devoted to the so-called coupled methods dealing with the data sets of various natures. As a main theoretical result, we obtain harmonious generalizations of the graph Laplacian and kernel-based methods with the central idea to find a natural structure coupling data sets of various natures. Finally, we discuss prospective applications and consider far-reaching generalizations related to the hypercomplex numbers.

1. Introduction

In many papers the method of coupling images of heterogenous data sets is based on the invention of a special technique involving artificial constructions that at least do not reflect, at most contradict, the mathematical nature of the mapping, i.e., by saying “mathematical nature” we mean abstract classical mathematical notions such as the space, the algebraic structure, etc. It is clear that dealing with the classical mathematical notions, we have an opportunity to develop the theory harmoniously using well-known methods and putting them as a fundamental base inventing new ones that can be in their own turn classified in accordance with the accepted terminology. Quite the contrary is the case when we involve artificial technical formulas allowing us to achieve a concrete goal but likely preventing the development of the theory in general, for in this case we know nothing of the consequent influence of the used construction on the scheme of reasonings, and it is rather reasonable, due to the technical obstacles, to expect the end of the theory.

Manifold alignment is a class of algorithms that create a mapping of heterogeneous data sets into a common, lower-dimensional latent space. The central point is to find a mapping that reveals the entire structure of the initial manifold. It is clear that a metric reflecting the distance plays a great role in the issue as well as the mathematical nature of the mapping; however, the latter should be harmoniously connected with applications. Diagonal integration represents a joint analysis of multi-omics biological data—genomic, transcriptomic, proteomic, and metabolomic. A critical view of existing methods and tools for diagonal integration must be supported by experience in analyzing individual data types. The scientific team has analyzed genomic data to identify damaging mutations [1], scRNA-seq transcriptomic data of cell lines [2], proteomic data [3], small non-coding RNA data [4], as well as metabolomic profiles. The accumulated experience of working with various types of omics data creates a foundation for developing methods for their integration.

Experimental methods on individual cells reveal transcriptomic and epigenetic heterogeneity between cells, but the question on their relationship remains open. Note that various types of topological alignment include the principle concept of the relationship between images or sometimes between preimages. The relationship is provided by an artificial structure generally represented by an operator with the idea to spread a unified structure on the data sets of the different nature. The literary survey represented below produces arguments creating prerequisites for the comprehensive investigation.

In the paper [5], an approach for integrating several types of measurements of observed data on a single cell is considered. However, the stated results are not based on a detailed theory; moreover, the concept of connecting heterogeneous data is not represented. In the paper [6], the authors present the manifold alignment method that is used to integrate several types of measurements of observed data performed on various aliquots of a given cell population. The MMD-MA (Maximum Mean Discrepancy Manifold Alignment) method minimizes the discrepancy between images of different modalities in a common latent space using the maximum mean discrepancy as a measure of the dissimilarity that provides a theoretically sound approach. However, minimizing the discrepancy can lead to loss of the information specific to each modality since a smooth representation does not reflect the real heterogeneity of the biological data. Moreover, the connection between heterogenous data sets is based on the mathematical construction obtained by virtue of the optimization, where the artificially constructed penalty functions are involved, what does not fully preserve the inherent structure of the data sets.

In the paper [7], the authors present a modification of the Gromov–Wasserstein distance-based manifold alignment method [8], which combines heterogeneous multiomic data sets of a single cell in order to describe and represent common and data set-specific cellular structures in different modalities. The Pamona (Partial Gromov–Wasserstein distance-based manifold alignment) method applies partial optimal alignment of manifolds, allowing not all elements to be compared between modalities, but only those for which reliable correspondences can be found. However, the method requires additional criteria to determine the threshold for the reliability of correspondences and may lead to the loss of potentially informative connections due to the specific approach to establishing correspondences.

In the paper [9], the authors combine two clustering processes so that in clustering cells corresponding to the scRNA-seq sample, the information from the scATAC-seq sample could also be used, and vice versa. The authors formulate this problem of coupled clustering as an optimization problem and present a method for solving it called coupled non-negative matrix factorization.

In the paper [10], a new algorithm for unsupervised topological alignment of multiomic integration for a single cell is presented. The method does not require any information on correspondence between cells or between measurements. At the first stage, the method embeds the internal low-dimensional structure of each data set of a single cell into a matrix, the elements of which reflect the distances corresponding to cells within the same data set, and then aligns the cell images by comparison of distance matrices using the matrix optimization method. Finally, the method projects separate, non-comparable measurements corresponding to single cell data sets into a latent embedding space to provide comparability of measurements by virtue of aligned cell images.

In the paper [11], the topological alignment method is formulated for two modalities, where a modality is understood as a set of features for a sample of cells, the information on the features is reflected in a matrix, the columns of which represent cells, and the rows represent various features. Since the objects in the two data sets are different, the additional technical tool is required for the method, the so-called transition matrix connecting the heterogenous data sets. The transition matrix is understood as a projection of one data set onto the space corresponding to another data set. The transition matrix is obtained from the profiles of scATAC-seq due to the information given in the bodies of genes [12,13,14,15].

Resuming the descriptions given above, we come to the conclusion that in each case there is a technically involved unnatural structure providing a relationship between the images of the data sets. Generally, a rather abstract term “relationship” means the property of preservation of the proportion between the distances in the initial and latent spaces. However, from the mathematical point of view, the constructed mapping should inherit the abstract properties of the involved structure. Thus, classical mathematical structures, such as the structures defined on the Hilbert space or generated by the properties of well-known operators acting in the Hilbert space (here we use a notion including Euclidean spaces), generate mappings having classical properties. The corresponding methods of the topological alignment are called by nature. On the other hand, the practical relevance of the methods considered above appears just due to mathematical tools applied in some complicated order that may not be harmonious from the fundamental theory point of view. The corresponding methods of the topological alignment are called by unnatural. In this paper, we study natural methods such as Laplacian graph and kernel-based methods of the topological alignment. Generally, we consider a natural algebraic structure such as the finite-dimensional unital algebra of the hypercomplex numbers and produce concrete reasonings corresponding to the field of the complex numbers, showing the relevance of the approach. The real and imaginary parts of the complex number correspond to the coupled data sets, respectively; thus, the construction of a mapping defined on the complex Hilbert space includes a naturally coupled structure. The main achievement is an attempt to construct an abstract qualitative theory, creating an opportunity to consider a number of heterogenous data sets endowed with the coupled natural structure, let alone far-reaching modifications and generalizations.

The paper is organized as follows: in Section 2, we consider notations and some well-known facts; in Section 3, we introduce the idea of the coupled unsupervised manifold alignment; in Paragraph 3.1, we consider a coupled Laplacian mapping and prove auxiliary propositions; in Paragraph 3.2, we consider the unsupervised manifold alignment via the reproducing kernel; in Paragraph 3.3, we consider prospective applications to the modern methods; and in Section 4, we finalize the theoretical results and discuss prospective generalizations.

2. Preliminaries

Throughout the paper, we consider finite-dimensional matrices over the field of the complex numbers ${\mathbb{C}}^{n\times m}$, i.e., of the dimension $(n\times m),n,m\in \mathbb{N},$ and use the following notation $A=\left\{a_{sj}\right\}\in {\mathbb{C}}^{n\times m},s=1,2,\dots ,n,j=1,2,\dots ,m$ for a matrix. We use the following standard notation for the transpose operation: $A^T=\left\{a_{js}\right\}.$ Consider a matrix $A=\left\{a_{sj}\right\}\in {\mathbb{C}}^{n\times m},a_{sj}\in \mathbb{C},$ denote

$${\mathbf{a}}_s:={(a_{s1},a_{s2},\dots ,a_{sm})}^T,{\mathbf{a}}_j^{.}={(a_{1j},a_{2j},\dots ,a_{nj})}^T.$$

We consider a complex linear vector space ${\mathbb{C}}^n$ consisting of the set of column matrices whose elements are complex numbers $\mathbf{x}={(x_1,x_2,\dots ,x_n)}^T,x_j\in \mathbb{C},j=1,2\dots ,n.$ Analogously, we define the real linear vector space ${\mathbb{R}}^n.$ A complex linear vector space supplied with the given bellow structure of the inner (scalar ) product operation is called by the complex Euclidean space

$${(\mathbf{x},\mathbf{y})}_{{\mathbb{C}}^n}:=\sum _{j=1}^n{x}_j{\overline{y}}_j,\mathbf{x},\mathbf{y}\in {\mathbb{C}}^n.$$

Note that the complex Euclidean space represents a particular case of the unitary space, i.e., a linear space over the field of the complex numbers. Define the norm in the general sense in a space with the inner product as follows: $\parallel \mathbf{x}\parallel =\sqrt{(\mathbf{x},\mathbf{x})}.$ Consider a matrix $A=\left\{a_{sj}\right\}\in {\mathbb{C}}^{n\times m},$ and denote $\overline{A}=\left\{{\overline{a}}_{sj}\right\}.$ In accordance with the theorem on the orthogonal decomposition theorem, we have

$${\mathbb{C}}^n={\mathbb{C}}^m\stackrel{.}{+}{\mathbb{C}}^{n-m},n>m,$$

i.e., for an arbitrary element $\mathbf{x}\in {\mathbb{C}}^n$ there exists a unique pair ${\mathbf{x}}_1\in {\mathbb{C}}^m,{\mathbf{x}}_2\in {\mathbb{C}}^{n-m}$ of elements so that the following decomposition holds $\mathbf{x}={\mathbf{x}}_1+{\mathbf{x}}_2.$ In this way we can put the element ${\mathbf{x}}_1$ into correspondence with the element $\mathbf{x},$ which is called the orthogonal projection of the space ${\mathbb{C}}^n$ onto the subspace ${\mathbb{C}}^m.$ Given an arbitrary set $X,$ a totally ordered set $Y,$ and a function $f:X\to Y,$ the $\mathrm{argmin}$ over some subset $\mathsf{\Omega }\subseteq X$ is defined by the following expression

$$\underset{x\in \mathsf{\Omega }}{\mathrm{argmin}}f\left(x\right):=\{y\in \mathsf{\Omega }:f\left(y\right)\le f\left(x\right),x\in \mathsf{\Omega }\}.$$

Consider a set of the elements $\{{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n\}$ belonging to a normed space. Let us construct a weighted graph G having n vertices, where one vertex corresponds to one element, and a set of edges connecting the adjacent vertices of the graph. We numerate the vertexes according to the given order, i.e., j vertex corresponds to ${\mathbf{x}}_j.$ We suppose that the vertexes s and j are adjacent to each other if the elements ${\mathbf{x}}_s$ and ${\mathbf{x}}_j$ are related, for instance close in some sense. The matter is how to define the rather vague notion of closeness in a more concrete way [16].

Here, we consider two variants: The first one is the so-called $\epsilon$-neighborhood method, which postulates the following sense of closeness. Suppose that the vertexes s and j are adjacent to each other if

$$\parallel {\mathbf{x}}_s-{\mathbf{x}}_j\parallel <\epsilon ,$$

where the norm is understood in the abstract sense. However, the Euclidean norm is mostly used in applications. Note that this type of closeness is geometrically motivated, the relationship is naturally symmetric. However, it often leads to a graph with several connected components, and it is difficult to choose the convenient value of $\epsilon$ to avoid the unconnected graph.

The second variant is the so-called n-nearest neighbors method, which postulates the following sense of closeness. Suppose that the vertexes s and j are adjacent to each other if ${\mathbf{x}}_s$ belongs to the set of n nearest, in the sense of the norm neighbors, of the element ${\mathbf{x}}_j$ or ${\mathbf{x}}_j$ belongs to the set of n-nearest neighbors of the element ${\mathbf{x}}_s.$

Having constructed the graph in accordance with one of the above methods, we can choose weights to form the weight matrix. The first one relates to the heat kernel; we assume that if the vertexes s and j are connected, in symbol $s\sim j,$ then the weight matrix is formed from the elements

$$W_{sj}=e^{-t^{-1}{\parallel {\mathbf{x}}_s-{\mathbf{x}}_j\parallel }^2},t\in \mathbb{R}\setminus \left\{0\right\},$$

in the contrary case, we assume that $W_{sj}=0.$ Thus, we have

$$W_{sj}=\left\{\begin{array}{c}\hfill e^{-t^{-1}{\parallel {\mathbf{x}}_s-{\mathbf{x}}_j\parallel }^2},s\sim j\\ \hfill 0,s\nsim j\end{array}\right..$$

In accordance with the made definition, we obtain the adjacency matrix of the weighted graph $W=\left\{W_{sj}\right\}\in {\mathbb{C}}^{n\times n}.$ The substantiation for choosing this weight is represented in the papers [17,18]. The second one is the so-called simple weight; we assume that $W_{sj}=1$ if the vertices s and j are connected and $W_{sj}=0$ if s and j are disconnected, i.e.,

$$W_{sj}=\left\{\begin{array}{c}\hfill 1,s\sim j\\ \hfill 0,s\nsim j\end{array}\right..$$

This kind of simplicity gives the opportunity to avoid choosing the value of the parameter $t.$ Note that a matrix $A\in {\mathbb{C}}^{n\times m}$ generates the finite-dimensional operator $A:{\mathbb{C}}^m\to {\mathbb{C}}^n.$ We use the following notations for the Hermitian components of the operator

$$\mathfrak{Re}A={\displaystyle \frac{A+A^{\ast }}{2}},\mathfrak{Im}A={\displaystyle \frac{A-A^{\ast }}{2i}},\overline{A}:=\left\{{\overline{a}}_{sj}\right\},\mathrm{Re}\left(A\right):=\left\{\mathrm{Re}{a}_{sj}\right\},\mathrm{Im}\left(A\right):=\left\{\mathrm{Im}{a}_{sj}\right\},$$

the latter matrices are called by the real and imaginary parts of the matrix A, respectively. The detailed information on the Hermitian components’ properties is given in the paper [19]. Denote by $\mathrm{D}\left(A\right),\mathrm{R}\left(A\right),\mathrm{N}\left(A\right)$ the domain of definition, the range, and the kernel or null space of the operator A, respectively.

3. Coupled Unsupervised Manifold Alignment

The manifold alignment represents a solution to the problem of alignment and at the same time forms the basis for finding a unified representation of different data sets. The principal concept of manifold alignment is to use the relationships between objects within each data set to obtain the information on the relationships between data sets and eventually to create a mapping of initially different data sets into a common latent space. The approaches described in this paragraph deal with different data sets having the same underlying structure. The basic low-dimensional representation is extracted by modeling the local geometry in the complex vector space using the generalized graph Laplacian operator associated with the data sets naturally coupled on the complex plane. Apparently, the alignment of manifolds can be considered as a method of reducing the dimension of the latent space [20], where the goal is to find a low-dimensional embedding of different data sets that preserves any known correspondences between them. Although we consider unsupervised methods, they can be easily modified to semi-supervised or supervised methods due to the corresponding term reflecting the distance from the chosen initial point in the sense of the given norm. In the reduced simplified form, we can formulate the coupled problem of the dimension decreasing as follows: Given n elements

$${\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n\in {\mathbb{C}}^l,{\mathbf{x}}_j={(x_{1j},x_{2j},\dots ,x_{lj})}^T,x_{sj}\in \mathbb{C},l\in \mathbb{N},$$

we are challenged to fined a set of the elements

$${\mathbf{y}}_1,{\mathbf{y}}_2,\dots ,{\mathbf{y}}_n\in {\mathbb{C}}^m,1\le m\le l,$$

so that the element ${\mathbf{y}}_j$ is the image of the element ${\mathbf{x}}_j$ under the searched mapping, i.e., ${\mathbf{x}}_j\to {\mathbf{y}}_j.$ Moreover, we are interested in finding a mapping for which the images would be located close to each other in some sense. In the case corresponding to the Euclidean space the desired mapping can be created due to the subspace generated by the generalized eigenvectors of the graph Laplacian operator. Below, we present the detailed analysis of the algorithm solving this problem in the case corresponding to the complex Euclidean space, which simultaneously solves the problem of coupling different data sets.

3.1. Coupled Laplacian Mapping

Throughout the paper, we consider a symmetric matrix $W\in {\mathbb{C}}^{n\times n},W=\left\{W_{sj}\right\}$ having non-negative real and imaginary parts, and define a diagonal matrix

$$D=\left\{D_{sj}\right\},D_{sj}=\left\{\begin{array}{c}\hfill \sum _{s=1}^n{W}_{sj},s=j\\ \hfill 0,s\ne j\end{array}\right..$$

Further, we assume that $D_{jj}\ne 0,j=1,2,\dots ,n$, which does not restrict the appropriate class of matrices due to the given sense of the elements. Consider an operator $L:=D-W.$ Note that L is not self-adjoint; the verification is left to the reader. However, it has the following property: $L^{\ast }=\overline{L}.$ Since D is a diagonal matrix, then it commutes with an arbitrary matrix of the corresponding dimension. In accordance with the symmetry property, we have

$$\mathrm{Re}{L}_{ss}=\sum _{j\ne s}\mathrm{Re}{W}_{sj},s=1,2,\dots n,$$

therefore, the matrix $\mathrm{Re}\left(L\right)$ is a symmetric diagonally dominant. Hence, it is positive semidefinite, i.e., $\mathrm{Re}\left(L\right)\ge 0$, from what follows that $\mathfrak{Re}L\ge 0.$ The analogous reasoning leads to the fact that $\mathfrak{Im}L\ge 0.$ The following lemma establishes a technical tool relevant in the further reasonings.

Lemma 1. 

Assume that $W\in {\mathbb{C}}^{n\times n}$, then, the following relation holds

$$\mathrm{tr}\left\{A^{\ast }LA\right\}=\sum _{j=1}^m{(L{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}={\displaystyle \frac{1}{2}}\sum _{s,j=1}^n{\parallel {\mathbf{a}}_s-{\mathbf{a}}_j\parallel }_{{\mathbb{E}}^m}^2{W}_{sj}.$$

Proof. 

By direct calculation, we have

$$\mathrm{tr}\left\{A^{\ast }LA\right\}=\sum _{q=1}^m\sum _{s,j=1}^n{L}_{sj}{\overline{a}}_{sq}{a}_{jq}.$$

It is clear that

$$\sum _{s,j=1}^n{L}_{sj}{\overline{a}}_{sq}{a}_{jq}=\sum _{s=1}^n{L}_{ss}|a_{sq}{|}^2+\sum _{\underset{s,j=1}{s\ne j}}^n{L}_{sj}{\overline{a}}_{sq}{a}_{jq}=\sum _{s=1}^n{D}_{ss}|a_{sq}{|}^2-\sum _{s=1}^n{W}_{ss}{|a_{sq}|}^2+\sum _{\underset{s,j=1}{s\ne j}}^n{L}_{sj}{\overline{a}}_{sq}{a}_{jq}=$$

$$=\sum _{s=1}^n{D}_{ss}|a_{sq}{|}^2-\sum _{s=1}^n{W}_{ss}|a_{sq}{|}^2-\sum _{\underset{s,j=1}{s\ne j}}^n{W}_{sj}{\overline{a}}_{sq}{a}_{jq}=\sum _{s=1}^n{D}_{ss}{|a_{sq}|}^2-\sum _{s,j=1}^n{\overline{a}}_{sq}{W}_{sj}{a}_{jq}={(L{\mathbf{a}}_q^{.},{\mathbf{a}}_q^{.})}_{{\mathbb{C}}^n}.$$

(1)

On the other hand,

$$\mathrm{Re}\sum _{s,j=1}^n{|a_{sq}-a_{jq}|}^2{W}_{sj}=\mathrm{Re}\left\{\sum _{s,j=1}^n\left(|a_{sq}{|}^2+{|a_{jq}|}^2-2\mathrm{Re}\left\{a_{sq}{\overline{a}}_{jq}\right\}\right)W_{sj}\right\}=$$

$$=2\mathrm{Re}\left\{\sum _{s=1}^n\mathrm{Re}{D}_{ss}{\left|a_s\right|}^2-\sum _{s,j=1}^n{a}_{sq}{\overline{a}}_{jq}\mathrm{Re}{W}_{sj}\right\}=2{({\mathbf{a}}_q^{.},\mathfrak{Re}L{\mathbf{a}}_q^{.})}_{{\mathbb{C}}^n}=2\mathrm{Re}{(L{\mathbf{a}}_q^{.},{\mathbf{a}}_q^{.})}_{{\mathbb{C}}^n}.$$

Analogously, we get

$$\mathrm{Im}\sum _{s,j=1}^n{|a_{sq}-a_{jq}|}^2{W}_{sj}=\mathrm{Im}\left\{\sum _{s,j=1}^n\left(|a_{sq}{|}^2+{|a_{jq}|}^2-2\mathrm{Re}\left\{a_{sq}{\overline{a}}_{jq}\right\}\right)W_{sj}\right\}=$$

$$=\mathrm{Re}\left\{\sum _{s=1}^n\mathrm{Im}{D}_{ss}{\left|a_s\right|}^2-\sum _{s,j=1}^n{a}_{sq}{\overline{a}}_{jq}\mathrm{Im}{W}_{sj}\right\}=2\mathrm{Re}{({\mathbf{a}}_q^{.},\mathfrak{Im}L{\mathbf{a}}_q^{.})}_{{\mathbb{C}}^n}=2\mathrm{Im}{(L{\mathbf{a}}_q^{.},{\mathbf{a}}_q^{.})}_{{\mathbb{C}}^n}.$$

Combining the above relations, we get

$$\sum _{q=1}^m{(L{\mathbf{a}}_q^{.},{\mathbf{a}}_q^{.})}_{{\mathbb{C}}^n}={\displaystyle \frac{1}{2}}\sum _{q=1}^m\sum _{s,j=1}^n{W}_{sj}|a_{sq}-a_{jq}{|}^2={\displaystyle \frac{1}{2}}\sum _{s,j=1}^n{W}_{sj}\sum _{q=1}^m|a_{sq}-a_{jq}{|}^2={\displaystyle \frac{1}{2}}\sum _{s,j=1}^n{\parallel {\mathbf{a}}_s-{\mathbf{a}}_j\parallel }_{{\mathbb{C}}^m}^2{W}_{ij}.$$

Using Formula (1), we obtain the desired result.   □

Theorem 1. 

Assume that $W\in {\mathbb{C}}^{n\times n},A\in {\mathbb{C}}^{n\times n},A^{\ast }DA=e^{i\theta }·I,$ then, the following relation holds

$$\mathrm{tr}\left\{A^{\ast }LA\right\}=2e^{i\theta }·\mathrm{tr}\left\{D^{-1}L\right\}.$$

Proof. 

Consider a decomposition

$$D=\mathfrak{Re}D+i\mathfrak{Im}D.$$

Note that $D_{jj}\ne 0,j=1,2,\dots ,n,$ therefore the matrix D is invertible and commutes, as well as its inverse, since they are diagonal, with an arbitrary matrix of the corresponding dimension. Thus, applying Lemma 1, we get

$${\displaystyle \frac{1}{2}}\mathrm{tr}\left\{A^{\ast }LA\right\}=\sum _{j=1}^n{(L{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\sum _{j=1}^n{(D^{-1}LD{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=$$

$$=\sum _{j=1}^n{(D^{-1}L\mathfrak{Re}D{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}+i\sum _{j=1}^n{(D^{-1}L\mathfrak{Im}D{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}.$$

Using the condition $\mathrm{Re}W,\mathrm{Im}W\ge 0$ and the symmetry property, we can define the nonnegative self-adjoint square roots and rewrite the previous relation in the following form

$${\displaystyle \frac{1}{2}}\mathrm{tr}\left\{A^{\ast }LA\right\}=\sum _{j=1}^n{(D^{-1}L\sqrt{\mathfrak{Re}D}{\mathbf{a}}_j^{.},\sqrt{\mathfrak{Re}D}{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}+i\sum _{j=1}^n{(D^{-1}L\sqrt{\mathfrak{Im}D}{\mathbf{a}}_j^{.},\sqrt{\mathfrak{Im}D}{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}.$$

(2)

Note that the condition $A^{\ast }DA=e^{i\theta }·I$ can be rewritten in the form

$${\psi }_{sq}:=\sum _{j=1}^n{D}_{jj}{\overline{a}}_{js}{a}_{jq}={\delta }_{sq}{e}^{i\theta },s,q=1,2,\dots ,n,$$

where $A^{\ast }DA=\left\{{\psi }_{sq}\right\}.$ It is clear that $A^{\ast }{D}^{\ast }A=e^{-i\theta }·I,$ therefore,

$$A^{\ast }\mathfrak{Re}DA={\displaystyle \frac{1}{2}}\left\{A^{\ast }DA+A^{\ast }{D}^{\ast }A\right\}=\cos \theta ·I,A^{\ast }\mathfrak{Im}DA=\sin \theta ·I.$$

(3)

The latter can be rewritten in the form

$${(\sqrt{\mathfrak{Re}D}{\mathbf{a}}_s^{.},\sqrt{\mathfrak{Re}D}{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\cos \theta ·{\delta }_{sj},{(\sqrt{\mathfrak{Im}D}{\mathbf{a}}_s^{.},\sqrt{\mathfrak{Im}D}{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\sin \theta ·{\delta }_{sj},s,j=1,2,\dots ,n.$$

It follows from the conditions that the following sets

$$\left\{\sqrt{\mathfrak{Re}D}{\mathbf{a}}_j^{.}\right\},\left\{\sqrt{\mathfrak{Im}D}{\mathbf{a}}_j^{.}\right\},j=1,2,\dots ,n$$

are orthogonal in ${\mathbb{C}}^n$; moreover, in accordance with Equation (3), they become orthonormal if we consider the corresponding multipliers. Applying the well-known theorem on the finite-dimensional operator trace, taking into account Equation (2), we obtain the desired result.   □

3.1.1. Generalized Eigenvectors

Assume additionally that the matrix $W\in {\mathbb{C}}^{n\times n}$ is normal; let us show that the corresponding operator $L=D-W$ is normal. We have

$$L^{\ast }L=(H_1-i{H}_2)(H_1+i{H}_2)=H_1^2+H_2^2+i{H}_1{H}_2-i{H}_2{H}_1,$$

where $H_1:=\mathfrak{Re}L=\mathfrak{Re}D-\mathfrak{Re}W,H_2:=\mathfrak{Im}L=\mathfrak{Im}D-\mathfrak{Im}W.$ Note that $\mathfrak{Re}D$ and $\mathfrak{Im}D$ are diagonal; therefore, they commute with an arbitrary matrix of the corresponding dimension. Hence, using the commutative property of the Hermitian components of the normal operator, i.e., $H_1{H}_2=H_2{H}_1,$ we obtain the desired result.

Having generalized the corresponding self-adjoint operator [21], we will call the operator L by the generalized graph Laplacian operator. Consider the following generalized eigenvalue problem

$$L\mathbf{e}=\lambda D\mathbf{e},\mathbf{e}\in {\mathbb{C}}^n,\lambda \in \mathbb{C},$$

(4)

where L is the generalized graph Laplacian operator. Assume that the following condition is imposed upon the solutions of the problem (4), i.e.,

$${(D\mathbf{e},\mathbf{e})}_{{\mathbb{C}}^n}=e^{i\theta },\theta \in (0,\pi /2),$$

then the solutions are called by the generalized eigenvectors of the operator $L.$ Denote by $\mathfrak{N}\left(L\right)$ the subspace generated by the generalized eigenvectors of the operator $L.$ We have the following decomposition

$${\mathbb{C}}^n=\mathfrak{N}\left(L\right)\stackrel{.}{+}\mathrm{N}\left(L\right).$$

(5)

To prove this fact, we should use the decomposition formula for an arbitrary bounded operator acting in ${\mathbb{C}}^n,$ i.e.,

$${\mathbb{C}}^n=\mathrm{R}\left(L^{\ast }\right)\stackrel{.}{+}\mathrm{N}\left(L\right).$$

Since the operator $D^{-1}L$ is normal, then $\mathfrak{N}\left(L\right)=\mathfrak{N}\left(L^{\ast }\right)=\mathrm{R}\left(L^{\ast }\right),$ the latter leads to Equation (5). Denote

$$\xi :=\dim \left\{\mathfrak{N}\right(L\left)\right\},\mu :=\dim \left\{\mathrm{N}\right(L\left)\right\};$$

then, in accordance with Equation (5), we have $\xi +\mu =n.$ Consider a set the generalized eigenvectors

$$L{\mathbf{e}}_1={\lambda }_{1}D{\mathbf{e}}_1,L{\mathbf{e}}_2={\lambda }_{2}D{\mathbf{e}}_2,\dots ,L{\mathbf{e}}_n={\lambda }_{n}D{\mathbf{e}}_n,$$

numbered in accordance with the order of the corresponding eigenvalues

$$|{\lambda }_1|<|{\lambda }_2|<\dots <|{\lambda }_{\xi }|,{\lambda }_{n-\mu +1}={\lambda }_{n-\mu +2}=\dots ={\lambda }_n=0.$$

We have the following implication

$$L{\mathbf{e}}_j={\lambda }_{j}D{\mathbf{e}}_j,\Rightarrow L^{\ast }{\mathbf{e}}_j={\overline{\lambda }}_j{D}^{\ast }{\mathbf{e}}_j,j=1,2,\dots ,\xi .$$

This fact can be proved easily if we notice that $D^{-1}L$ is a normal operator. Let us show that the generalized eigenvectors satisfy the following condition

$${(D{\mathbf{e}}_s,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}=e^{i\theta }·{\delta }_{sj}.$$

(6)

For this purpose, note that the case $s=j$ is given due to the definition, assume that $s\ne j$ and consider the following reasonings

$${\lambda }_s{(D{\mathbf{e}}_s,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}={(L{\mathbf{e}}_s,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}={({\mathbf{e}}_s,L^{\ast }{\mathbf{e}}_j)}_{{\mathbb{C}}^n}={({\mathbf{e}}_s,{\overline{\lambda }}_j{D}^{\ast }{\mathbf{e}}_j)}_{{\mathbb{C}}^n}={\lambda }_j{(D{\mathbf{e}}_s,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}.$$

Taking into account the fact that ${\lambda }_s\ne {\lambda }_j$, we obtain the desired result. Note that the multiplier $e^{-i\theta }$ defines the unitary operator, and it is clear that the operator $L_1:=e^{-i\theta }L$ is normal. Denote by ${\left\{{\beta }_j\right\}}_1^{\xi },{\left\{{\gamma }_j\right\}}_1^{\xi }$ the set of the eigenvalues ${\left\{{\lambda }_j\right\}}_1^{\xi }$ numerated as follows

$$\forall \{{\beta }_j,{\gamma }_j\},\exists \{{\lambda }_p,{\lambda }_q\}:\mathrm{Re}{\beta }_j=\mathrm{Re}{\lambda }_p,\mathrm{Im}{\gamma }_j=\mathrm{Im}{\lambda }_q,j,p,q\in \{1,2,\dots ,\xi \},$$

$$\mathrm{Re}{\beta }_j\le \mathrm{Re}{\beta }_{j+1},\mathrm{Im}{\gamma }_j\le \mathrm{Im}{\gamma }_{j+1},j=1,2,\dots ,\xi -1.$$

It is clear that we can enumerate the sets of the generalized eigenvectors in accordance with the given order so that the sets

$${\mathbf{g}}_1,{\mathbf{g}}_2,\dots ,{\mathbf{g}}_{\xi },{\mathbf{s}}_1,{\mathbf{s}}_2,\dots ,{\mathbf{s}}_{\xi }$$

correspond to the real and imaginary parts of eigenvalues, respectively. Denote by

$${\mathfrak{N}}_{\xi }:=\mathfrak{N}\left(L\right),{\mathfrak{N}}_m,m=1,2,\dots ,\xi$$

the subspace generated by the generalized eigenvectors ${\mathbf{e}}_1,{\mathbf{e}}_2,\dots ,{\mathbf{e}}_m.$ The proof of the following theorem represents the modified concept of the gradient descent method

Theorem 2. 

Assume that the matrix $W\in {\mathbb{C}}^{n\times n}$ is normal, then the following relation holds

$$\underset{A^{\ast }DA=e^{i\theta }I}{\min }\sum _{j=1}^m\mathrm{Re}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\sum _{j=1}^m\mathrm{Re}{\beta }_j,\underset{A^{\ast }DA=e^{i\theta }I}{\min }\sum _{j=1}^m\mathrm{Im}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\sum _{j=1}^m\mathrm{Im}{\gamma }_j.$$

(7)

Proof. 

Consider the following reasonings

$$e^{-i\theta }\sum _{j=1}^m{(L{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=e^{-i\theta }\sum _{j=1}^m{\left(\sum _{s=1}^{\xi }{c}_{sj}{\beta }_{s}D{\mathbf{g}}_s,\sum _{s=1}^{\xi }{c}_{sj}{\mathbf{g}}_s\right)}_{{\mathbb{C}}^n}=\sum _{j=1}^m\sum _{s=1}^{\xi }|c_{sj}{|}^2{\beta }_s=\sum _{s=1}^{\xi }{\beta }_s\sum _{j=1}^m{\left|c_{sj}\right|}^2,$$

where we used the following decomposition

$${\mathbf{a}}_j^{.}=\sum _{s=1}^{\xi }{c}_{sj}{\mathbf{g}}_s.$$

Therefore, observing the real parts on the both sides of the equality, we get

$$\mathrm{Re}\sum _{j=1}^m{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\sum _{s=1}^{\xi }\mathrm{Re}{\beta }_s\sum _{j=1}^m{\left|c_{sj}\right|}^2.$$

Note that

$$e^{i\theta }·{\delta }_{jk}={(D{\mathbf{a}}_j^{.},{\mathbf{a}}_k^{.})}_{{\mathbb{C}}^n}={\left(\sum _{s=1}^{\xi }{c}_{sj}D{\mathbf{g}}_s,\sum _{s=1}^{\xi }{c}_{sk}{\mathbf{g}}_s\right)}_{{\mathbb{C}}^n}=e^{i\theta }\sum _{s=1}^{\xi }{c}_{sj}{\overline{c}}_{sk};\sum _{s=1}^{\xi }{c}_{sj}{\overline{c}}_{sk}={\delta }_{jk}.$$

Thus, the first Equation (7) can be rewritten in the form

$$\underset{{\mathcal{C}}^{\ast }\mathcal{C}=I}{\min }\sum _{s=1}^{\xi }\mathrm{Re}{\beta }_s\sum _{j=1}^m|c_{sj}{|}^2,\sum _{j=1}^m\sum _{s=1}^{\xi }{|c_{sj}|}^2=m,$$

where $\mathcal{C}:=\left\{c_{sj}\right\}\in {\mathbb{C}}^{\xi \times m}.$ Denote

$$x_s:=\sum _{j=1}^m{\left|c_{sj}\right|}^2,$$

then the expression corresponding to the main problem can be rewritten in the form

$$f_{\xi }(x_1,x_2,\dots ,x_{\xi }):=\sum _{s=1}^{\xi }{x}_s\mathrm{Re}{\beta }_s,\sum _{s=1}^{\xi }{x}_s=m,$$

where the function $f_{\xi }$ is the function of several real variables defined on ${\mathbb{R}}^{\xi }.$ Substituting, we get

$$g_{\xi -1}(x_1,x_2,\dots ,x_{\xi -1}):=\sum _{s=1}^{\xi -1}{x}_s\mathrm{Re}{\beta }_s+\mathrm{Re}{\beta }_{\xi }\left(m-\sum _{s=1}^{\xi -1}{x}_s\right)=\sum _{s=1}^{\xi -1}{x}_s\mathrm{Re}\left({\beta }_s-{\beta }_{\xi }\right)+m\mathrm{Re}{\beta }_{\xi },$$

$$\sum _{s=1}^{\xi -1}{x}_s\le m.$$

Therefore,

$$\nabla g_{\xi -1}=\left(\begin{array}{c}{c}_{1\xi }\\ c_{2\xi }\\ ⋮\\ c_{\xi -1\xi }\end{array}\right),c_{s\xi }:=\mathrm{Re}\left({\beta }_s-{\beta }_{\xi }\right)<0.$$

Combining this fact with the properties of the hyperplane, we get

$$\underset{\mathbf{x}\in {\mathbb{R}}_m^{\xi -1}}{\min }{g}_{\xi -1}\left(x\right)=\underset{\mathbf{x}\in {\mathbb{T}}_m^{\xi -1}}{\min }\sum _{s=1}^{\xi -1}{x}_s\mathrm{Re}{\beta }_s=\underset{\mathbf{x}\in {\mathbb{T}}_m^{\xi -1}}{\min }{f}_{\xi -1}\left(x\right),$$

where

$${\mathbb{R}}_m^n:=\left\{\mathbf{x}\in {\mathbb{R}}^n:\sum _{s=1}^n{x}_s\le m\right\},{\mathbb{T}}_m^n:=\left\{\mathbf{x}\in {\mathbb{R}}^n:\sum _{s=1}^n{x}_s=m\right\},n,m\in \mathbb{N}.$$

Implementing the same reasonings, we get

$$g_{\xi -2}(x_1,x_2,\dots ,x_{\xi -2}):=\sum _{s=1}^{\xi -2}{x}_s\mathrm{Re}{\beta }_s+\mathrm{Re}{\beta }_{\xi -1}\left(m-\sum _{s=1}^{\xi -2}{x}_s\right)=$$

$$=\sum _{s=1}^{\xi -2}{x}_s\mathrm{Re}\left({\beta }_s-{\beta }_{\xi }\right)+m\mathrm{Re}{\beta }_{\xi -1},\sum _{s=1}^{\xi -2}{x}_s\le m.$$

Therefore, the constant vector $\nabla g_{\xi -2}$ has negative coordinates. Analogously to the above, we get

$$\underset{\mathbf{x}\in {\mathbb{R}}_m^{\xi -2}}{\min }{g}_{\xi -2}\left(x\right)=\underset{\mathbf{x}\in {\mathbb{T}}_m^{\xi -2}}{\min }{f}_{\xi -2}\left(x\right).$$

Implementing the same reasonings, we come to the problem

$$\underset{\mathbf{x}\in {\mathbb{T}}_m^m}{\min }{f}_m\left(x\right).$$

On the other hand, we have

$$f_m\left(x\right)=\sum _{s=1}^m\mathrm{Re}{\beta }_s\sum _{j=1}^m{\left|c_{sj}\right|}^2,\sum _{s=1}^m{\overline{c}}_{sk}{c}_{sj}={\delta }_{jk},j,k=1,2\dots ,m.$$

(8)

Note that the last condition can be rewritten in the form

$${\mathcal{C}}^{\ast }\mathcal{C}=I,\mathcal{C}\in {\mathbb{C}}^{m\times m},\Rightarrow \mathcal{C}{\mathcal{C}}^{\ast }=I,$$

i.e., $\mathcal{C}$ is the unitary matrix. Thus, in accordance with the well-known property of the unitary matrix, the latter equality can be rewritten in the form

$$\sum _{j=1}^m{c}_{sj}{\overline{c}}_{kj}={\delta }_{sk},s,k=1,2\dots ,m.$$

Substituting this relation into Equation (8), we obtain

$$\underset{A^{\ast }DA=e^{i\theta }I}{\min }\sum _{j=1}^m\mathrm{Re}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}=\underset{\mathbf{x}\in {\mathbb{T}}_m^m}{\min }{f}_m\left(x\right)=\sum _{s=1}^m\mathrm{Re}{\beta }_s.$$

The proof corresponding to the imaginary part is absolutely analogous.   □

The following lemma represents a technical tool to construct mappings having the required properties,

Lemma 2. 

Assume that the matrix $W\in {\mathbb{C}}^{n\times n}$ is normal, the following relation holds

$$\mathrm{Re}{\beta }_j=\mathrm{Re}{\lambda }_p,\mathrm{Im}{\gamma }_j=\mathrm{Im}{\lambda }_q,j,p,q\in \{1,2,\dots ,m\},$$

then, for a matrix $Y\in {\mathbb{C}}^{n\times m}$ satisfying the conditions

$${\left\{{\mathbf{y}}_j^{.}\right\}}_1^m\subset {\mathfrak{N}}_m,{(D{\mathbf{y}}_s^{.},{\mathbf{y}}_j^{.})}_{{\mathbb{C}}^n}=e^{i\theta }·{\delta }_{sj},s,j=1,2,\dots ,m,$$

the following relation holds

$$\underset{A^{\ast }DA=e^{i\theta }I}{\mathrm{argmin}}\left|\mathrm{tr}\left\{A^{\ast }LA\right\}\right|=Y,A\in {\mathbb{C}}^{n\times m}.$$

Proof. 

Applying Lemma 1 and Theorem 2, we obtain

$$\underset{A^{\ast }DA=e^{i\theta }I}{\min }\mathrm{tr}\left\{A^{\ast }LA\right\}=\underset{A^{\ast }DA=e^{i\theta }I}{\min }{\left|\sum _{j=1}^m{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}\right|}^2=$$

$$=\underset{A^{\ast }DA=e^{i\theta }I}{\min }\left\{{\left(\sum _{j=1}^m\mathrm{Re}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}\right)}^2+{\left(\sum _{j=1}^m\mathrm{Im}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}\right)}^2\right\}\ge$$

$$\ge \underset{A^{\ast }DA=e^{i\theta }I}{\min }{\left(\sum _{j=1}^m\mathrm{Re}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}\right)}^2+\underset{A^{\ast }DA=e^{i\theta }I}{\min }{\left(\sum _{j=1}^m\mathrm{Im}{(L_1{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n}\right)}^2=$$

$$={\left(\sum _{j=1}^m\mathrm{Re}{\beta }_j\right)}^2+{\left(\sum _{j=1}^m\mathrm{Im}{\gamma }_j\right)}^2={\left(\sum _{j=1}^m\mathrm{Re}{\lambda }_j\right)}^2+{\left(\sum _{j=1}^m\mathrm{Im}{\lambda }_j\right)}^2.$$

The latter relation holds due to the one-to-one correspondence between the eigenvalues and their real or imaginary parts, numerated in order of their increasing values. In accordance with the well-known theorem on the trace of the finite-dimensional operator, analogously to the proof of Theorem 1, taking into account Equations (2) and (3), we get

$${\left|\sum _{j=1}^m{(L_1{\mathbf{y}}_j^{.},{\mathbf{y}}_j^{.})}_{{\mathbb{C}}^n}\right|}^2=\sum _{j=1}^m{(D^{-1}{L}_1\sqrt{\mathfrak{Re}D}{\mathbf{y}}_j^{.},\sqrt{\mathfrak{Re}D}{\mathbf{y}}_j^{.})}_{{\mathbb{C}}^n}+i\sum _{j=1}^m{(D^{-1}{L}_1\sqrt{\mathfrak{Im}D}{\mathbf{y}}_j^{.},\sqrt{\mathfrak{Im}D}{\mathbf{y}}_j^{.})}_{{\mathbb{C}}^n}=$$

$$=\sum _{j=1}^m{(D^{-1}{L}_1\sqrt{\mathfrak{Re}D}{\mathbf{e}}_j,\sqrt{\mathfrak{Re}D}{\mathbf{e}}_j)}_{{\mathbb{C}}^n}+i\sum _{j=1}^m{(D^{-1}{L}_1\sqrt{\mathfrak{Im}D}{\mathbf{e}}_j,\sqrt{\mathfrak{Im}D}{\mathbf{e}}_j)}_{{\mathbb{C}}^n}=$$

$$={\left|\sum _{j=1}^m{\left(L_1{\mathbf{e}}_j,{\mathbf{e}}_j\right)}_{{\mathbb{C}}^n}\right|}^2={\left(\sum _{j=1}^m\mathrm{Re}{(L_1{\mathbf{e}}_j,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}\right)}^2+{\left(\sum _{j=1}^m\mathrm{Im}{(L_1{\mathbf{e}}_j,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}\right)}^2=$$

$$={\left(\sum _{j=1}^m\mathrm{Re}{\lambda }_j\right)}^2+{\left(\sum _{j=1}^m\mathrm{Im}{\lambda }_j\right)}^2.$$

The latter relation leads to the desired result.   □

3.1.2. Restriction of the Dimension

Consider the sets of elements

$${\mathbf{u}}_1,{\mathbf{u}}_2,\dots ,{\mathbf{u}}_n\in {\mathbb{R}}^l,{\mathbf{v}}_1,{\mathbf{v}}_2,\dots ,{\mathbf{v}}_n\in {\mathbb{R}}^p,l,p\in \mathbb{N},$$

(9)

and assume that the graph G is constructed in accordance with one of the methods described in the introduction section, i.e., $\epsilon$-neighborhood or n-nearest neighbors methods. Thus, we can put weighted matrices $W^{\left(1\right)},W^{\left(2\right)}$ into correspondence with the sets in accordance with the given order, where

$$W_{sj}^{\left(1\right)}=\left\{\begin{array}{c}\hfill e^{-t^{-1}{\parallel {\mathbf{u}}_s-{\mathbf{u}}_j\parallel }_{{\mathbb{E}}^l}^2},s\sim j\\ \hfill 0,s\nsim j\end{array}\right.,W_{sj}^{\left(2\right)}=\left\{\begin{array}{c}\hfill e^{-t^{-1}{\parallel {\mathbf{v}}_s-{\mathbf{v}}_j\parallel }_{{\mathbb{E}}^p}^2},s\sim j\\ \hfill 0,s\nsim j\end{array}\right..$$

The following matrix is called by the matrix of the coupled mapping

$$W=\alpha H+i\beta H,\alpha ,\beta >0,\alpha +\beta =1,$$

where

$$H=\left\{H_{sj}\right\}\in {\mathbb{C}}^{n\times n},H_{sj}=\eta W_{sj}^{\left(1\right)}+\mu W_{sj}^{\left(2\right)},\eta ,\mu >0,\eta +\mu =1.$$

(10)

Since the Hermitian components of the operator W commute, we conclude that the operator W is normal. However, this fact can be established by direct calculation. Thus, we can apply the propositions proved in the previous paragraphs. Note that the values of the parameters $\alpha ,\beta ,\eta ,\mu$ reflect the influence of the corresponding components on the image. Let us construct the corresponding generalized graph Laplacian operator $L=D-W.$ It is not hard to prove that relation Equation (6) holds, where $\theta =\arctan \beta /\alpha ,$ we should notice that in accordance with Equation (10), we have

$$\theta =\arg {(D\mathbf{z},\mathbf{z})}_{{\mathbb{C}}^n}=\arctan \left\{{\displaystyle \frac{\mathrm{Im}{(D\mathbf{z},\mathbf{z})}_{{\mathbb{C}}^n}}{\mathrm{Re}{(D\mathbf{z},\mathbf{z})}_{{\mathbb{C}}^n}}}\right\}=\arctan {\displaystyle \frac{\beta }{\alpha }},\mathbf{z}\in {\mathbb{C}}^n.$$

Analogously, we get

$$\arg {(L\mathbf{z},\mathbf{z})}_{{\mathbb{C}}^n}=\arctan {\displaystyle \frac{\beta }{\alpha }},\mathbf{z}\in {\mathbb{C}}^n.$$

Note that the Hermitian components of the operator L are diagonally dominant matrices, therefore, they are positive semidefinite matrices. We have

$$\mathrm{Re}{(L{\mathbf{e}}_j,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}>0,\mathrm{Im}{(L{\mathbf{e}}_j,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}>0,j=1,2,\dots ,\xi .$$

Using the relation ${(L{\mathbf{e}}_j,{\mathbf{e}}_j)}_{{\mathbb{C}}^n}={\lambda }_j{e}^{i\theta },j=1,2,\dots ,\xi ,$ we can obtain easily $\arg {(L\mathbf{z},\mathbf{z})}_{{\mathbb{C}}^n}=\theta$, therefore, $\arg {\lambda }_j=0,j=1,2,\dots ,\xi .$ Without loss of generality, assume that $l\ge p,$ Then, we can put the subset of ${\mathbb{C}}^l$ into correspondence with the subsets of Equation (9) in the following way: ${\mathbf{x}}_j={\mathbf{u}}_j+i{\mathbf{v}}_j,j=1,2,\dots ,n,$ where we assume that the element ${\mathbf{v}}_j$ has the zero coordinates with the indexes more than the value $p<l.$

Thus, if we find the mapping from ${\mathbb{C}}^l$ into ${\mathbb{C}}^m,1\le m\le p,$ i.e., $F:{\mathbb{C}}^l\to {\mathbb{C}}^m,$ then due to the correspondence between the real and imaginary parts of the complex numbers we obtain naturally coupled mappings

$$F_1:{\mathbb{R}}^l\to {\mathbb{R}}^m,F_2:{\mathbb{R}}^p\to {\mathbb{R}}^m.$$

Assume that the matrix W satisfies Equation (10); consider the following problem

$$\underset{A^{\ast }DA=e^{i\theta }I}{\mathrm{argmin}}\left|\sum _{s,j=1}^n{\parallel {\mathbf{a}}_s-{\mathbf{a}}_j\parallel }_{{\mathbb{C}}^m}^2{W}_{sj}\right|,A\in {\mathbb{C}}^{n\times m}.$$

(11)

Eventually, we can resume the following theorem

Theorem 3. 

The solution of Equation (11) is represented by a set ${\left\{{\mathbf{y}}_j\right\}}_1^n\subset {\mathbb{C}}^m$ satisfying the following conditions

$${\left\{{\mathbf{y}}_j^{.}\right\}}_1^m\subset {\mathfrak{N}}_m,{(D{\mathbf{y}}_s^{.},{\mathbf{y}}_j^{.})}_{{\mathbb{C}}^n}=e^{i\theta }·{\delta }_{sj},s,j=1,2,\dots ,m.$$

Moreover, the following coupled mapping is defined

$${\mathbf{u}}_j\to \mathrm{Re}{\mathbf{y}}_j,{\mathbf{v}}_s\to \mathrm{Im}{\mathbf{y}}_j,j=1,2,\dots ,n.$$

Proof. 

The proof follows immediately from Lemma 1, Lemma 2.   □

3.1.3. Probabilistic Approach

It is remarkable that Theorem 3 creates a prerequisite for further optimization problems. In this regard, we can observe the Kullback–Leibler divergence as the most suitable tool practically showing the preservation between the elements’ characteristics. The generalized graph of the Laplacian method projects the distinct unmatched features across single-cell multi-omics data sets into a common embedding space. The aim is to preserve the intrinsic low-dimensional structures as well as the aligned cells simultaneously. However, the next reasonable step is to apply an analog of the t-distribution stochastic neighbor embedding method [22]. Stochastic Neighbor Embedding (SNE) [23] starts by converting the high-dimensional Euclidean distances between data elements into conditional probabilities that represent similarities. The similarity between the data element ${\mathbf{x}}_j$ and the data element ${\mathbf{x}}_s,(s,j=1,2,\dots ,n)$ is the conditional probability, $p_{j|s},$ that ${\mathbf{x}}_s$ would correspond to ${\mathbf{x}}_j$ as its neighbor if neighbors are corresponded to each other in proportion to their probability density under the Gaussian function centered at ${\mathbf{x}}_s.$ For nearby data elements, $p_{j|s}$ is relatively high; at the same time, for widely separated data elements, $p_{j|s}$ is almost infinitesimal for reasonable values of the variance of the Gaussian function ${\sigma }_s.$ Theoretically, the conditional probability $p_{j|s}$ is defined as follows

$$p_{j|s}:={\displaystyle \frac{e^{-\parallel {\mathbf{x}}_s-{\mathbf{x}}_j\parallel /2{\sigma }_s^2}}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel {\mathbf{x}}_s-{\mathbf{x}}_m\parallel /2{\sigma }_s^2}}},$$

where ${\sigma }_s$ is the variance of the Gaussian function [22] centered on the data element ${\mathbf{x}}_s.$ Since we are only interested in modeling pairwise similarities, we set the value of $p_{s|s}$ to zero. For the low-dimensional counterparts ${\mathbf{y}}_s$ and ${\mathbf{y}}_j$ of the high-dimensional data elements ${\mathbf{x}}_s$ and ${\mathbf{x}}_j,$ it is easy to compute a similar conditional probability, which we denote by $q_{j|s}.$ We set the variance of the Gaussian function that is employed in the calculation of the conditional probabilities $q_{j|s}$ to the value $2^{-1/2}.$ Therefore, we model the similarity of the map element ${\mathbf{y}}_j$ to the map element ${\mathbf{y}}_s$ by

$$q_{j|s}:=\frac{e^{-\parallel {\mathbf{y}}_s-{\mathbf{y}}_j\parallel }}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel {\mathbf{y}}_s-{\mathbf{y}}_m\parallel }}.$$

Analogously to the previous case we set $q_{s|s}=0.$ In accordance with the description given in [22], we assume that if the map elements ${\mathbf{y}}_s$ and ${\mathbf{y}}_j$ correctly model the similarity between the high-dimensional data elements ${\mathbf{x}}_s$ and ${\mathbf{x}}_j,$ the conditional probabilities $p_{j|s}$ and $q_{j|s}$ will be equal. Having taken into account this description, the SNE method aims to find a low-dimensional data representation that minimizes the divergence between $p_{j|s}$ and $q_{j|s}.$ Note that a natural measure of the faithfulness with which $q_{j|s}$ models $p_{j|s}$ is the Kullback–Leibler divergence

$$\mathrm{KL}(P_s\left|\right|Q_s)=\sum _{j=1}^n{p}_{j|s}\log {\displaystyle \frac{p_{j|s}}{q_{j|s}}},$$

where $P_s$ represents the conditional probability distribution corresponding to the data element ${\mathbf{x}}_s$ over all other data elements. The symbol $Q_s$ represents the conditional probability distribution corresponding to the map element ${\mathbf{y}}_s$ over all other map elements. The indefinite value of the function for elements with the same indexes is defined as follows

$$p_{s|s}\log {\displaystyle \frac{p_{s|s}}{q_{s|s}}}=0,s=1,2,\dots ,n.$$

The SNE method minimizes the sum of Kullback–Leibler divergences over all data elements using a gradient descent method. The objective function is represented by the following expression

$$\sum _{s=1}^n\mathrm{KL}(P_s\left|\right|Q_s).$$

Since the Kullback–Leibler divergence includes the logarithmic function, various types of effects can appear. For instance, there is a large cost for using widely separated map elements to represent nearby data elements, i.e., for using a small $q_{j|s}$ to model a large $p_{j|s},$ but there is only a small cost for using nearby map elements to represent widely separated data elements. This observation leads to the conclusion that the cost function focuses on preserving the local structure of the data in the mapping. In accordance with [22], in most cases there does not exist a single value of ${\sigma }_s$ that is appropriate for all data elements in the data set since the density of the data is variable. The method represented in [22] produces a scheme of reasonings allowing one to choose a suitable value of the parameter ${\sigma }_s.$ Since we consider the symmetric case, which corresponds to the constant value of ${\sigma }_s,$ we represent the general idea of the selection strategy of the variance ${\sigma }_s.$

In accordance with [22], in the general case, it is not likely that there is a single value of ${\sigma }_s$ that is optimal for all data elements in the data set because the density of the data is likely to vary. In dense regions, a smaller value of ${\sigma }_s$ is usually more appropriate than in sparser regions. Any particular value of ${\sigma }_s$ induces a probability distribution, $P_s$ over all of the other data elements. This distribution has an entropy that increases as ${\sigma }_s$ increases. In accordance with [23] SNE performs a binary search for the value of ${\sigma }_s$ that produces a $P_s$ with a fixed perplexity [22] that is specified by the user. The perplexity is defined as follows

$$Perp\left(P_s\right)=2^{H\left(P_s\right)},$$

where $H\left(P_s\right)$ is the Shannon entropy defined as follows

$$H\left(P_s\right):=-\sum _{j=1}^n{p}_{j|s}{\log }_2{p}_{j|s},p_{j|s}={\displaystyle \frac{e^{-\parallel {\mathbf{x}}_s-{\mathbf{x}}_j\parallel /2{\sigma }_s^2}}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel {\mathbf{x}}_s-{\mathbf{x}}_m\parallel /2{\sigma }_s^2}}}.$$

As it was mentioned above, the perplexity is specified by the user [22,23], what gives a key to obtain the corresponding set ${\left\{{\sigma }_s\right\}}_1^n.$

The fact that the solution of Equation (11) is not unique allows to extend the theoretical investigation and apply probabilistic approach represented above. Thus, the idea is to choose the concrete solution from the set of solutions belonging to the generalized eigenvector subspace having the desirable dimension.

This solution should satisfy the penalty term given by Kullback–Leibler divergence. Here, we use the following notations

$$X={\left({\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_n\right)}^T,Y={\left({\mathbf{y}}_1,{\mathbf{y}}_2,\dots ,{\mathbf{y}}_n\right)}^T.$$

In accordance with Theorem 3, we have

$$X_1\to Y_1,X_2\to Y_2.$$

where

$$X_1:=\mathrm{Re}X,Y_1:=\mathrm{Re}Y,X_2:=\mathrm{Im}X,Y_2:=\mathrm{Im}Y.$$

Consider the constructions related to the symmetric SNE [22], in this case, we have $p_{j|s}=p_{s|j},q_{j|s}=q_{s|j},$ thus we can use the unified form of notations $p_{sj},q_{sj}.$ The pairwise similarities are given by

$$P_s^{\left(1\right)}:=\left\{p_{sj}^{\left(1\right)}\right\},p_{sj}^{\left(1\right)}:={\displaystyle \frac{e^{-\parallel \mathrm{Re}({\mathbf{x}}_s-{\mathbf{x}}_j)\parallel /2{\sigma }^2}}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel \mathrm{Re}({\mathbf{x}}_s-{\mathbf{x}}_m)\parallel /2{\sigma }^2}}},Q_s^{\left(1\right)}:=\left\{q_{sj}^{\left(1\right)}\right\},q_{sj}^{\left(1\right)}:={\displaystyle \frac{e^{-\parallel \mathrm{Re}({\mathbf{y}}_s-{\mathbf{y}}_j)\parallel }}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel \mathrm{Re}({\mathbf{y}}_s-{\mathbf{y}}_m)\parallel }}}.$$

Analogously, we define $P_s^{\left(2\right)},Q_s^{\left(2\right)},$ where

$$p_{sj}^{\left(2\right)}:={\displaystyle \frac{e^{-\parallel \mathrm{Im}({\mathbf{x}}_s-{\mathbf{x}}_j)\parallel /2{\sigma }^2}}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel \mathrm{Im}({\mathbf{x}}_s-{\mathbf{x}}_m)\parallel /2{\sigma }^2}}},q_{sj}^{\left(2\right)}:={\displaystyle \frac{e^{-\parallel \mathrm{Im}({\mathbf{y}}_s-{\mathbf{y}}_j)\parallel }}{{\displaystyle \sum _{m\ne s}}{e}^{-\parallel \mathrm{Im}({\mathbf{y}}_s-{\mathbf{y}}_m)\parallel }}}.$$

Using the assumptions related to the equal indexes and the indefinite value of the function given above, consider the following optimization problem

$$\underset{Y_1,Y_2}{\min }\mathcal{F}(Y_1,Y_2),$$

where the functional is defined as follows

$$\mathcal{F}(Y_1,Y_2)=\sum _{s=1}^n\mathrm{KL}(P_s^{\left(1\right)}\left|\right|Q_s^{\left(1\right)})+\sum _{s=1}^n\mathrm{KL}(P_s^{\left(2\right)}\left|\right|Q_s^{\left(2\right)})+\zeta \parallel Y_1-Y_2{\parallel }_F^2,\zeta \ge 0.$$

The penalty term reflects the distance between images, however it can be regulated by the parameters $\alpha ,\beta$ given in Paragraph 3.1.2. Moreover, the idea of the problem corresponding to the value $\zeta =0$ represents the interest itself due to the exclusive natural structure of the complex plane.

3.1.4. Characteristic of the Mapping

Consider an abstract approach to a function creating a weight matrix, assume that

$$W_{sj}=\psi ({\mathbf{x}}_s,{\mathbf{x}}_j),s,j=1,2,\dots ,n,$$

where $\psi$ is a complex-valued function. Thus, having imposed on the function $\psi$ corresponding conditions formulated in the previous paragraphs, we can create a mapping satisfying the required properties. It is remarkable that by virtue of the function $\psi$ we can give sense, dictated by the relations between the elements ${\mathbf{x}}_s$ and ${\mathbf{x}}_j,$ to the values $W_{sj},$ i.e., the value $W_{sj}$ reflects the degree of a relationship between the elements ${\mathbf{x}}_s$ and ${\mathbf{x}}_j.$ Apparently, we can state that Theorem 1 provides an indicator of the function $\psi$ in some sense. Assume that the conditions of Theorem 1 holds; applying Lemma 1 observe the following relation

$$\mathrm{tr}\left\{A^{\ast }LA\right\}={\displaystyle \frac{1}{2}}\sum _{s,j=1}^n{\parallel {\mathbf{a}}_s-{\mathbf{a}}_j\parallel }_{{\mathbb{C}}^n}^2{W}_{sj}=\sum _{j=1}^n{(L{\mathbf{a}}_j^{.},{\mathbf{a}}_j^{.})}_{{\mathbb{C}}^n},A^{\ast }DA=e^{i\theta }.$$

On the other hand, in accordance with Theorem 1, we have

$$\mathrm{tr}\left\{A^{\ast }LA\right\}=2e^{i\theta }\mathrm{tr}\left\{D^{-1}L\right\}=2e^{i\theta }\sum _{j=1}^n\left(1-{\displaystyle \frac{W_{jj}}{D_{jj}}}\right),$$

here we should notice the obvious fact $|W_{jj}/D_{jj}|\le 1.$ Therefore,

$$\sum _{s,j=1}^n{\parallel {\mathbf{a}}_s-{\mathbf{a}}_j\parallel }_{{\mathbb{C}}^n}^2{W}_{sj}=4e^{i\theta }\sum _{j=1}^n\left(1-{\displaystyle \frac{W_{jj}}{D_{jj}}}\right),A^{\ast }DA=e^{i\theta }.$$

Motivated by this phenomena, we involve the functional $f\left(\psi \right)$ as the indicator of the function $\psi$ influence on the constructed mapping, where

$$f\left(\psi \right):=e^{i\theta }\sum _{j=1}^n\left(1-{\displaystyle \frac{W_{jj}}{D_{jj}}}\right).$$

The simple heuristic reasoning leads us to the conclusion that the smaller the values $W_{sj},s\ne j,$ indicating a relationship between the elements, the more efficient the corresponding mapping from the point of view of the elements’ closeness. It is rather clear that Theorem 1 can be used to reveal fully the properties of a function that are relevant within the context. In order to observe the particular case given by the weight matrix corresponding to distributions, assume that $\psi ({\mathbf{x}}_s,{\mathbf{x}}_j)=p_{sj},s,j=1,2,\dots ,n,$ then, in accordance with the assumptions accepted above $D_{ss}=1,W_{ss}=0;$ therefore, $f\left(\psi \right)=n{e}^{i\theta }.$ The latter relation reflects some intrinsic sense appealing to the connection of the weight matrix and the dimension of the complex Euclidean space. This rather simple observation fully reveals the dependence of the mapping on the function $\psi$ since the last equality can be used as the standard.

3.2. Unsupervised Manifold Alignment via the Reproducing Kernel

Nowadays, many methods of sequencing individual cells are available, but difficulties arise when applying several different sequencing methods to the same cell. In the paper [6], the authors represent unsupervised manifold alignment algorithm MMD-MA for integrating multiple measurements of observed data performed on various aliquots of a given cell population.

The idea of the method is to map the cells measurement data produced by various methods into a latent space.

According to the algorithm, the data of a single cell corresponding to several types of measurements are aligned by optimizing an objective function consisting of three components: (1) maximum average discrepancy (MMD), which sets constrictions on the mapping into the latent space; the “distances” between images should be minimal in the sense of some function determined by MMD; (2) a distortion term, a construction that preserves the structure of the initial data under the mapping; and (3) a penalty function that eliminates a trivial solution. It should be noted that the MMD-MA method does not require any information on the correspondence between the cell data processing methods. The requirements for the considered type of mapping are rather weak. They allow the algorithm to integrate data from measurements made in a single cell of heterogeneous features, such as gene expression, DNA availability, chromatin structure, methylation, and visualization data. In the paper [6] the authors demonstrate the relevance of the MMD-MA method in simulated experiments and using a set of real data, including data on gene expression in a single cell and data on methylation.

In this paragraph, we study the construction of the mapping based on the concept of the reproducing kernel. In order to implement the classical approach, the additional information from the theory of Hilbert spaces is provided. The issues related to the concept of the reproducing kernel in the Hilbert space are considered and a theoretical justification of the opportunity to construct a suitable mapping is given. Further, we study the objective function consisting of three terms considered in the paper [6]. In the conclusion, some ideas on the possibility of developing the mathematical concept of the method are represented.

3.2.1. Reproducing Kernel Hilbert Space

Let us define a complex linear space of elements of an arbitrary nature as a set of elements with a given structure of the linear operation acting into the initial set, i.e., the linear combinations of the elements belong to the initial set. A linear space with a given inner (scalar) product operation is called a unitary space. Thus, the inner product naturally defines the metric on the elements of a linear space. However, it may happen that some limits of sequences do not belong to the initial set of the linear space. Recall that a unitary space containing all its limiting elements is called by the Hilbert space. There are some discrepancies in the definitions of the Hilbert space. In some sources there is an additional condition related to the infinite dimension, i.e., an infinite-dimensional unitary space is called by the Hilbert space. As can be seen from the definition given above, the Euclidean space is a finite-dimensional unitary space over the field of real numbers. Further, we will denote the abstract Hilbert space by $\mathfrak{H}$ and the corresponding inner product operation by ${(f,g)}_{\mathfrak{H}},f,g\in \mathfrak{H}.$

Let $\mathfrak{A}$ be a set of elements of an arbitrary nature and let $\mathfrak{H}$ be a system of complex functionals defined on the set $\mathfrak{A}$ and forming a Hilbert space $\mathfrak{H}.$ A complex functional of two variables $K(x,y),x,y\in \mathfrak{A}$ is called by the reproducing kernel of the space $\mathfrak{H}$ if the following condition is satisfied. For an arbitrary fixed value x, the functional $K(x,y)$ is an element of the space $\mathfrak{H},$ in symbols we have $K_x\left(y\right):=K(x,y)\in \mathfrak{H},$ moreover, for an arbitrary functional $f\in \mathfrak{H}$, we have

$$f\left(x\right)={(f,K_x)}_{\mathfrak{H}}.$$

It is remarkable that there are such representatives of the Hilbert space of functionals defined on a set that a corresponding reproducing kernel does not exist. The Hilbert space containing the reproducing kernel is called by the reproducing kernel Hilbert space (RKHS). The most known representatives of RKHS are the Bergman space and the Dirichlet space, i.e., the spaces of holomorphic functions on the unit disk.

The following theorem establishes a criterion of the reproducing kernel existence.

Theorem 4. 

A Hilbert space $\mathfrak{H}$ is RKSH if and only if for an arbitrary element $y\in \mathfrak{A}$ there exists a constant $C_y$ such that

$$\left|f\left(y\right)\right|\le C_y{\parallel f\parallel }_{\mathfrak{H}},f\in \mathfrak{H}.$$

We need the following theorem.

Theorem 5. 

(Moore-Aronszajn) Suppose that $K(x,y)$ is a Hermitian symmetric, positive definite kernel on the set $\mathfrak{A}\times \mathfrak{A}.$ Then there exists a unique Hilbert space $\mathfrak{H}$ of functions defined on $\mathfrak{A},$ for which K is the reproducing kernel. The space is defined as follows, consider the linear span of the functionals $K_x,x\in \mathfrak{A},$ completing this set according to the norm generated by the following inner product, we obtain the desired RKHS

$${\left(\sum _{j=1}^n{\alpha }_j{K}_{x_j},\sum _{j=1}^m{\beta }_j{K}_{y_j}\right)}_{\mathfrak{H}}:=\sum _{j=1}^n\sum _{k=1}^m{\alpha }_j\overline{{\beta }_k}K(x_j,y_k),\alpha ,\beta \in \mathbb{C}.$$

Note that the axioms of the inner product can be verified directly taking into account the positive definiteness of the kernel $K.$ According to this definition, we have

$${(K_x,K_y)}_{\mathfrak{H}}=K(x,y),x,y\in \mathfrak{A},f=\sum _{j=1}^{\infty }{\alpha }_j{K}_{x_j},x_j\in \mathfrak{A},f\in \mathfrak{H},$$

where convergence is understood in the sense of the norm of the Hilbert space $\mathfrak{H}.$

3.2.2. Kernel-Based Mapping

Consider the sets of elements ${\mathfrak{X}}_1,{\mathfrak{X}}_2,$ in general, these can be sets of elements of an arbitrary nature. The elements of sets correspond to objects (cells, samples) having a set of features (various measurements reflecting one or more properties of the object). Thus, a correspondence naturally arises between a set of objects and a set of vectors, the coordinates of which are quantitative characteristics of features. We will consider the subsets

$$X_s=\left(x_1^{\left(s\right)},x_2^{\left(s\right)},\dots ,x_{n_s}^{\left(s\right)}\right)\subset {\mathfrak{X}}_s,s=1,2.$$

The numbers $n_1,n_2$ reflect the number of objects under consideration. Note that we do not assume the existence of any similarity between these subsets. At the same time, the idea of further reasonings is to construct a mapping of these subsets into a certain space in which their images would be comparable.

As the main tool for further study, we use the following positively defined Hermitian symmetric kernels ${\gamma }_s(x,y),x,y\in {\mathfrak{X}}_s$ having the range of values belonging to $\mathbb{C}.$ The biological meaning of these mathematical constructions may be to quantify the similarity between objects. Let us introduce the following notation

$$K^{\left(s\right)}=\left\{K_{qj}^{\left(s\right)}\right\}:={\gamma }_s\left(x_q^{\left(s\right)},x_j^{\left(s\right)}\right),q,j=1,2,\dots ,n_s,s=1,2,$$

i.e., $K^{\left(s\right)}\in {\mathbb{C}}^{n_s\times n_s}.$ Since, in accordance with the made assumptions, both kernels are Hermitian symmetric, positive definite, then using Theorem 2, we can construct the Hilbert spaces ${\mathfrak{H}}_s$ (RKHS) of functionals defined on ${\mathfrak{X}}_s$ for which ${\gamma }_s$ are reproducing kernels.

Following the idea of finding mappings into some space in which the images of ${\mathfrak{X}}_s$ would be comparable, consider the following functionals

$$z_l^{\left(s\right)}\left(x\right)=\sum _{j=1}^{n_s}{\alpha }_{lj}^{\left(s\right)}{\gamma }_s\left(x_j^{\left(s\right)},x\right),x\in {\mathfrak{X}}_s,l=1,2,\dots ,p,p\in \mathbb{N}.$$

Note that in accordance with Theorem 2 these functionals are elements of the space ${\mathfrak{H}}_s.$ Thus, the following operator is defined

$$F^{\left(s\right)}:{\mathfrak{X}}_s\to {\mathbb{C}}^p,F^{\left(s\right)}x={\mathbf{z}}^{\left(s\right)}\left(x\right),x\in {\mathfrak{X}}_s,$$

(12)

where

$${\mathbf{z}}^{\left(s\right)}\left(x\right)={\left(z_1^{\left(s\right)}\left(x\right),z_2^{\left(s\right)}\left(x\right),\dots ,z_p^{\left(s\right)}\left(x\right)\right)}^T.$$

It is clear that it is possible to implement the correspondence in this way, regardless of the nature of the set ${\mathfrak{X}}_s,$ this universality provides the idea of the manifold alignment. Below, for the convenient form of writing we omit the index $s,$ however the further reasonings are correct in both $(s=1,2)$ cases. Denote

$$A\in {\mathbb{C}}^{p\times n},A:=\left\{{\alpha }_{qj}\right\}.$$

Note that

$$AK=\left\{z_q\left(x_j\right)\right\}\in {\mathbb{C}}^{p\times n}.$$

(13)

Using Equation (13), applying Theorem 5, we get

$$AK{A}^{\ast }=\left\{v_{qj}\right\},v_{qj}=\sum _{k=1}^n{z}_q\left(x_k\right){\overline{\alpha }}_{jk}=\sum _{k=1}^n{\overline{\alpha }}_{jk}\sum _{m=1}^n{\alpha }_{qm}\gamma \left(x_m,x_k\right)=$$

$$={\left(\sum _{m=1}^n{\alpha }_{jm}{K}_{x_m},\sum _{k=1}^n{\alpha }_{qk}{K}_{x_k}\right)}_{\mathfrak{H}}={\left(z_j,z_q\right)}_{\mathfrak{H}}.$$

(14)

Applying these reasonings in the both cases corresponding to the value of the index $s,$ we can rewrite the latter relations in the form

$$A^{\left(s\right)}{K}^{\left(s\right)}=\left\{z_q^{\left(s\right)}\left(x_j^{\left(s\right)}\right)\right\}\in {\mathbb{C}}^{p\times n_s},A^{\left(s\right)}{K}^{\left(s\right)}{A}^{\left(s\right)\ast }=\left\{{\left(z_j^{\left(s\right)},z_q^{\left(s\right)}\right)}_{\mathfrak{H}}\right\}\in {\mathbb{C}}^{p\times p}.$$

(15)

The next challenge is to select the mapping parameters in order to minimize the distances, understood in the sense of a function of the Euclidian metric, between the images. Using the Gaussian radial basic function (RBF), we will regulate matrices $A^{\left(s\right)}$ under a certain condition connecting them and determined by the relation between the elements belonging to $X_s.$ Thus, we can determine the similarity between the elements of the sets ${\mathfrak{X}}_s,s=1,2$ in the sense of the distance between them given by the following formula

$$G_{qj}^{(s,k)}:=G\left\{{\mathbf{z}}^{\left(s\right)}\left(x_q^{\left(s\right)}\right),{\mathbf{z}}^{\left(k\right)}\left(x_j^{\left(k\right)}\right)\right\},s,k=1,2,q=1,2,\dots ,n_s,j=1,2,\dots ,n_k,$$

where the Gaussian RBF is defined as follows

$$G(\mathbf{u},\mathbf{v}):=e^{-t^{-1}{\parallel \mathbf{u}-\mathbf{v}\parallel }_{{\mathbb{C}}^p}^2},\mathbf{u},\mathbf{v}\in {\mathbb{C}}^p,t\in \mathbb{R}\setminus \left\{0\right\}.$$

The choice of the parameter t is determined by a concrete application. Note that the term distance is used in the heuristic sense since the construction does not satisfy the axioms of metric space, the verification is left to the reader. It is clear that using the Gaussian RBF, we can establish similarity in the sense of the distance between the elements $X_s,$ for identical and different values of $s.$ This approach leads to the following function, reflecting both the relationship between elements within the sets and the relationship between elements of different sets

$$\mathfrak{G}\left\{A^{\left(1\right)},A^{\left(2\right)}\right\}={\displaystyle \frac{1}{n_1^2}}\sum _{i,j=1}^{n_1}{G}_{ij}^{(1,1)}-{\displaystyle \frac{2}{n_1{n}_2}}\sum _{i=1}^{n_1}\sum _{j=1}^{n_2}{G}_{ij}^{(1,2)}+{\displaystyle \frac{1}{n_2^2}}\sum _{i,j=1}^{n_2}{G}_{ij}^{(2,2)}.$$

In accordance with [6], the last formula has been related to MMD construction [24]. However, we use the exclusive notation since the similarity of definitions is rather vague.

Taking into account the specifics of the applications, the authors of the paper [6] consider that it is not sufficient to find the minimum of the function $\mathfrak{G}$ relative to the matrices $A^{\left(s\right)}.$ Thus, an additional term characterizing distortion is introduced in order to ensure the preservation of relations between the images of elements, i.e.

$$\mathrm{dis}\left\{A^{\left(s\right)}\right\}={∥K^{\left(s\right)}-K^{\left(s\right)}{A}^{\left(s\right)\ast }{A}^{\left(s\right)}{K}^{\left(s\right)}∥}_F,$$

where the Frobenius norm is used. This expression quantifies how much the matrix $K^{\left(s\right)}$ differs from the matrix of inner products after mapping. The restriction imposed on $\mathrm{dis}\left\{A^{\left(s\right)}\right\}$ inherently guarantees that the distortion between the data in the initial space and the corresponding data after mapping should be small. Consider the following penalty function

$$\mathrm{pen}\left\{A^{\left(s\right)}\right\}={∥I_p-A^{\left(s\right)}{K}^{\left(s\right)}{A}^{\left(s\right)\ast }∥}_F.$$

Note that in accordance with Equation (14), we have

$$I_p-A^{\left(s\right)}{K}^{\left(s\right)}{A}^{\left(s\right)\ast }=\left\{{\delta }_{qj}-{\left(z_j^{\left(s\right)},z_q^{\left(s\right)}\right)}_{{\mathfrak{H}}_s}\right\},q,j=1,2,\dots ,p.$$

The latter relation characterizes the influence of the penalty function; thus, it finds the mapping having coordinates with the property close to the orthonormal one. In order to ensure that the mapping to ${\mathbb{C}}^p$ satisfies this property, we can add the penalty function as a summand.

Taking into account the above, we arrive at the problem of finding the minimum of the objective function

$$\underset{A^{\left(1\right)},A^{\left(2\right)}}{\mathrm{argmin}}\left(\mathfrak{G}\left\{A^{\left(1\right)},A^{\left(2\right)}\right\}+\sum _{s=1}^2{\lambda }_1\mathrm{dis}\left\{A^{\left(s\right)}\right\}+{\lambda }_2\mathrm{pen}\left\{A^{\left(s\right)}\right\}\right).$$

The solution can be found due to the Gradient Descent method [6]. In practice, it is necessary to specify several parameters to solve the optimization problem. The parameters include the dimension p of the space ${\mathbb{C}}^p,$ the parameter of Gaussian RBF, as well as the parameters ${\lambda }_1,{\lambda }_2.$ In the paper [6], the authors assume that the parameters p and ${\lambda }_1,{\lambda }_2$ are set by the user.

Now assume that the set of elements ${\mathfrak{X}}_s$ is endowed with the complex structure. In this case, we have a mapping naturally induced by mapping Equation (12), i.e.

$$F^{\left(s\right)}:\mathrm{Re}{\mathfrak{X}}_s\to {\mathbb{R}}^p,F^{\left(s\right)}:\mathrm{Im}{\mathfrak{X}}_s\to {\mathbb{R}}^p.$$

Apparently, we obtain a mapping implementing the manifold alignment where a union of four data sets coupled in the natural way is considered. The advantage of the kernel-based approach is in the following. Firstly, we should remark that the set ${\mathfrak{X}}_s$ has or does not have an arbitrary topological structure. It can be an infinite-dimensional space endowed with the Hilbert space structure or a finite set without any structure. This assumption makes a great freedom in modifications of the method. In particular the interest arises in the case when ${\mathfrak{X}}_s$ is endowed with the quaternion structure $\mathbb{H}.$ In this case the problem is to modify the reproducing kernel construction to construct the desirable mapping. Further generalizations in this direction lead us to the hypercomplex numbers. The arbitrary structure of the preimages is fully adopted to the implementation of the idea to unite several data sets of various natures. However, the concept of the mapping requires a concrete technique that should be invented in the general case corresponding to the hypercomplex numbers.

3.3. Prospective Application to the Modern Methods

The mixOmics method [25] represents a specialized package for the R language, developed for multivariate analysis of biological data with emphasis on data exploration, dimensionality reduction, and results visualization. The input data structure in mixOmics assumes use of matrix X of size N samples by P predictors with continuous values and a categorical outcome vector $y,$ which is automatically converted to an indicator matrix Y of size N by K classes. Output data represents a comprehensive set of results, including latent components for projecting samples into lower-dimensional space, loading vectors demonstrating each feature’s contribution, molecular signatures as selected features, and classification and prediction results for new samples, as well as various visualization types, including sample plots, variable plots, correlation networks, and heatmaps. All methods in the package are implemented as projection techniques where samples are summarized through H latent components defined as linear combinations of original predictors. This approach ensures not only effective data dimensionality reduction but also preserves result interpretability, which is critically important for understanding biological mechanisms and identifying significant biomarkers in omics studies. However, the general concept of mapping requires the usage of the unified structure represented in this paper. The kernel-based method can be involved as the universal method related to the abstract data sets.

The MultiVI method [26] represents a deep generative model for probabilistic analysis of multimodal single-cell data, designed to integrate different molecular modalities into a unified representation. This method is developed for joint analysis of transcriptome, chromatin accessibility, and surface protein expression data from individual cells, even when information for some cells is available only for one or several modalities. MultiVI finds application in cellular diversity studies and characterization of cell types and states, as well as in data integration tasks from different laboratories and sequencing technologies. The method can combine single-cell RNA sequencing data (scRNA-seq), chromatin accessibility analysis (scATAC-seq), and surface protein measurements using antibodies (CITE-seq). MultiVI effectively works with both fully paired data, where all modalities are measured in the same cells, and partially paired or unpaired data, where different cells are characterized by different sets of modalities. The data processing workflow in MultiVI consists of several sequential stages. First, the method uses modality-specific encoders based on deep neural networks to create latent representations of each modality, including accounting for batch effects and technical data features. Then these representations are combined into a joint latent space through averaging with the application of a distance penalty between modalities. At the final stage, modality-specific decoders reconstruct observations from the latent representation using appropriate distributions for each data type. Input data is provided in count matrix format for each modality, where rows correspond to cells and columns to features (genes, genomic regions, or proteins). MultiVI generates low-dimensional joint cell representations, normalized and batch-corrected values for all modalities, as well as uncertainty estimates for imputed values. Here, we should remark that it is possible to involve the unified natural structure to create a low-dimensional representation. Using the coupled generalized Laplacian mapping, we can form an intermediate subspace of images and then apply the invented algorithm in order to implement the coupling much more accurate.

The MUON method [27] represents a multimodal analytical platform for omics data, developed for organizing, analyzing, visualizing, and sharing multimodal biological data. This tool is designed to solve computational challenges arising when working with multi-omics experiments, including efficient storage, indexing, and seamless access to large data volumes, tracking biological and technical metadata, as well as processing dependencies between different omics layers. MUON can combine various types of omics data, including single-cell RNA sequencing (scRNA-seq), chromatin accessibility data (scATAC-seq), epitope profiling (CITE-seq), and DNA methylation, as well as spatial omics data. The platform supports trimodal analyses such as scNMT-seq or TEA-seq and can process arbitrary numbers of data modalities. The processing workflow in MUON consists of several sequential stages. First, preprocessing of individual modalities occurs, including quality control, sample filtering, data normalization, and feature selection for analysis. Then dimensionality reduction methods are applied, which can work with individual modalities (e.g., principal component analysis) or jointly process multiple modalities through approaches such as multi-omics factor analysis (MOFA) or weighted nearest neighbors (WNN). At the next stage, cell neighborhood graphs are constructed based on obtained representations, which can use information from individual modalities or combined multimodal representations. The final stage includes creating nonlinear embeddings through UMAP-type methods and clustering for cell type identification. Input data is provided in count matrix format for each modality along with corresponding metadata. MUON uses the MuData container, which represents a hierarchical data structure where each omics modality is stored as an AnnData object. Output data includes processed count matrices, low-dimensional representations, cell embeddings, neighborhood graphs, cluster labels, and differential analysis results, which can be visualized and used for biological interpretation of results. Apparently, the concept of embedding used in the method admits application of the methods elaborated in the paper.

The PolarBear method [28] represents a semi-supervised machine learning model for predicting missing modalities and aligning single-cell data between different types of omic measurements. PolarBear’s main task lies in solving the problem of integrating multimodal single-cell data when most cells have only one type of measurement available (e.g., only transcriptome or only chromatin accessibility), while comprehensive co-assay data measuring multiple modalities in a single cell are available in limited quantities. The method can be applied in cellular regulation studies, cellular heterogeneity analysis, differential gene expression studies between cell types, identification of cell-specific regulatory elements in tumor samples, and generating hypotheses about biological processes in modalities inaccessible for direct measurement. PolarBear can combine scRNA-seq and scATAC-seq data, working with various types of co-assay technologies, including CAR-seq, SNARE-seq, Paired-seq, and SHARE-seq. Principally, the method can be adapted to work with other types of omic data since it does not require feature correspondence between different measurement modalities. Data processing in PolarBear occurs in two main stages. At the first stage, the method trains two separate beta-variational autoencoders (beta-VAE) for each data modality, using both paired co-assay data and much more numerous unimodal data from public databases. The autoencoders learn to create stable latent cell representations independent of sequencing depth and batch effects. For scRNA-seq, the autoencoder assumes that gene read counts follow a zero-inflated negative binomial distribution, while for scATAC-seq the Bernoulli distribution is used for binary chromatin accessibility peaks. At the second stage, a fully connected translator layer is added between the trained autoencoders, which is trained in supervised mode exclusively on co-assay data to translate between latent representations of two modalities in both directions. Input data is provided as raw gene count matrices for scRNA-seq and binarized peak count matrices for scATAC-seq, with additional information about batches and sequencing depth for each cell. Output data includes predicted profiles in the missing modality, sequencing depth-normalized expression estimates, latent cell representations for alignment between modalities, differential expression analysis results, and cell-specific marker genes, providing possibilities for biological interpretation and subsequent analysis of integrated multimodal data. The alignment between modalities creates the prerequisite for the paper results application. However, the nature of the semi-supervised model and the only type of measurements may create some obstacles to the direct application of the methods discussed in the paper.

The sciCAN method [29] represents a method for integrating single-cell chromatin accessibility and gene expression data using cycle-consistent adversarial networks. The method is designed to combine scATAC-seq and scRNA-seq data into a unified representation without requiring prior information about cell correspondence between modalities. The sciCAN method can be applied for hematopoietic hierarchy analysis, studying cellular responses to CRISPR perturbations, constructing joint developmental cell trajectories, transferring cell type labels between modalities, and other integrative single-cell data analyses. The method can combine single-cell RNA sequencing and single-cell ATAC sequencing data, transforming chromatin accessibility peak matrices into gene activity matrices to ensure compatibility with gene expression data. Input data undergoes logarithmic transformation normalization with pseudocount addition, after which the top 3000 highly variable genes are identified for each modality and used as features for integration. Data processing in sciCAN consists of two main components: representation learning and modality alignment. Encoder E projects high-dimensional data from both modalities into a joint low-dimensional space using a noise contrastive estimation loss function to learn discriminative representation. For modality alignment, two separate discriminator networks are used: Drna distinguishes modality source in latent space, while Datac works with generator G to create connections between modalities through adversarial training with the addition of cycle-consistent losses. The architecture includes fully connected layers with batch normalization and ReLU activation for the encoder, three-layer multilayer perceptrons for discriminators with sigmoid activation, and a two-layer decoder for the generator. Additionally, linear transformation of 128-dimensional latent representation to 32-dimensional output and 25-dimensional SoftMax-activated output for NCE loss computation is applied. Input data is provided in gene expression and gene activity matrix format after preprocessing and normalization, while output data represents 128-dimensional joint latent cell representation used for subsequent integrative analysis, including trajectory construction, label transfer, and clustering of cells from both modalities in unified feature space. In this case, we should italicize that the combination of scATAC-seq and scRNA-seq data can be considered in the framework of the paper results. In particular, heterogenous data can be successfully coupled by the kernel-based method.

The SCIM method (Single-Cell data Integration via Matching) represents a scalable approach for integrating single-cell data obtained using different profiling technologies without requiring feature correspondence between modalities. The method is designed to recover correspondences between cells measured by different technologies when paired correspondences between data sets are lost due to cell consumption during profiling processes. The SCIM method can be applied to integrate any single-cell omics technologies including scRNA-seq, CyTOF, proteomics, and genomics, provided there is a common latent structure in the data. The method can combine data from two or more technologies that measure non-overlapping feature sets, for example expression of different gene sets, gene expression with image characteristics, or any other single-cell measurements originating from the same cell suspension. The SCIM method consists of two main processing stages: first, an integrated latent space invariant to technology is created using an autoencoder/decoder framework with an adversarial objective function, where separate encoder and decoder networks are trained for each technology, and a single discriminator acts in latent space to ensure indistinguishability of representations from different technologies. Then a bipartite matching scheme is applied for pairwise connection of cells between technologies, using their low-dimensional latent representations through an efficient bipartite matching algorithm based on the minimum cost maximum flow problem. Input data is provided in cell-by-feature matrix format for each technology, where features are specific to the profiling technology but can represent gene expression, protein levels, or other measurements. Output data represents eight-dimensional latent cell representations in common feature space and paired correspondences between cells from different technologies obtained through the bipartite matching algorithm, allowing use of true observed signals per cell pair for any subsequent analysis while maintaining technology-invariant data integration. It should be noted that the specifics of the conditions, such as the use of various profiling technologies and the need for paired correspondences between data sets, determine the relevance of the paper results application and subsequent comparison analysis.

From the point of view of applications, the aim of this paper is to develop methods for the diagonal integration of multimodal biological data for a comprehensive study of the biological processes, in particular the aging process. The prospective project based on the paper’s fundamental results plans to develop new mathematical methods of diagonal integration that take into account the identified limitations of existing approaches. We propose to combine and jointly analyze seven different types of data: genomic, transcriptomic, proteomic, metabolomic, and lipidomic data. The aging process acts as a suprasystem factor for diagonal integration as a fundamental biological phenomenon that manifests itself at all levels of the molecular organization. A comprehensive analysis of the integrated data will make it possible to identify new pharmaceutical targets for therapeutic interventions in the aging process, identify multidimensional biomarkers of aging with increased predictive power, and establish causal relationships between molecular changes at various levels of biological organization in the context of age-associated processes.

4. Conclusions

Having analyzed the main mathematical principles forming the concept of the unsupervised topological alignment, we represent a natural algebraic structure coupling the heterogenous data sets as well as their images. In this regard, the harmonious generalization of the graph Laplacian method was obtained over the field of the complex numbers. The kernel-based method with the central idea to find an appropriate structure coupling images of data sets of various natures was extended to the complex vector space. The prospective theoretical results appeal to more complicated algebraic structures, allowing to couple naturally an arbitrary number of data sets and their images. Thus, the quaternion structure can be considered as further generalization that leads to the hypercomplex numbers and more abstract mathematical objects. It was shown that the kernel-based methods are completely efficient for the implementation of the idea to unite data sets of various natures. The main obstacle in further development is that a mapping requires a concrete technique, i.e., generalizations of the well-known results of the operator theory for the module over the hypercomplex structure. Finally, we presented the detailed analysis of the modern biological methods supplied with the comments on possible applications of the invented approach. The authors believe that the represented theoretical approach is principally novel, while the obtained conclusions admit relevant biological applications.