Tensor Train Completion from Fiberwise Observations Along a Single Mode ^†

Abstract

Tensor completion is an extension of matrix completion aimed at recovering a multiway data tensor by leveraging a given subset of its entries (observations) and the pattern of observation. The low-rank assumption is key in establishing a relationship between the observed and unobserved entries of the tensor. The low-rank tensor completion problem is typically solved using numerical optimization techniques, where the rank information is used either implicitly (in the rank minimization approach) or explicitly (in the error minimization approach). Current theories concerning these techniques often study probabilistic recovery guarantees under conditions such as random uniform observations and incoherence requirements. However, if an observation pattern exhibits some low-rank structure that can be exploited, more efficient algorithms with deterministic recovery guarantees can be designed by leveraging this structure. This work shows how to use only standard linear algebra operations to compute the tensor train decomposition of a specific type of “fiber-wise” observed tensor, where some of the fibers of a tensor (along a single specific mode) are either fully observed or entirely missing, unlike the usual entry-wise observations. From an application viewpoint, this setting is relevant when it is easier to sample or collect a multiway data tensor along a specific mode (e.g., temporal). The proposed completion method is fast and is guaranteed to work under reasonable deterministic conditions on the observation pattern. Through numerical experiments, we showcase interesting applications and use cases that illustrate the effectiveness of the proposed approach.

1. Introduction

We live in a data-driven world, where massive amounts of data are generated and collected daily. In addition to the huge volume and high velocity, the structural complexity of the data is becoming so high that it renders standard techniques inadequate. Multidimensional arrays (tensors) are data structures that offer a better way to organize and analyze the data with a higher-order structure. As the number of dimensions of the data increases, the memory required to store it (and the computational effort required for its analysis) increases exponentially—an obstacle known as the “curse of dimensionality.” Tensor decompositions offer efficient approaches to analyzing higher-order datasets, allowing for the retention of intrinsic information within the data while taming the curse of dimensionality [1,2,3,4,5,6,7,8,9,10,11].

Real-world datasets are often noisy and incomplete for various reasons, including sensor malfunctions, recording errors, constraints imposed by privacy regulations, delays in obtaining access permissions, and deliberate incomplete sampling to meet memory and time requirements. Analyzing incomplete datasets is a big challenge, as the missing information may affect the accuracy and reliability of the findings and, therefore, limit subsequent applications [12]. Completion of a partially observed dataset may be thought of as a particular specification of its unobserved entries. Completion is more crucial for higher-order datasets as they are larger, increasing the chance of missing or unreliable entries. Many problems can be framed as instances of tensor completion, e.g., image and video inpainting, gene expression imputation, and weather and traffic data imputation (see, e.g., [13,14,15,16,17,18]). Without any constraints on the completed tensor, there are infinitely many ways to specify the missing entries. Therefore, to make the estimation meaningful, it is necessary to assume that the completed tensor satisfies specific properties (e.g., low rank, minimum volume) that lower the degrees of freedom and enable a unique solution. These properties constrain the unobserved entries and help establish their relationships with the observations. The principle of parsimony (Occam’s razor) serves as a (heuristic) guideline for model selection, stating that if multiple competing models explain the same data, the model with lower complexity is the best. The rank can be considered a kind of natural measure of the complexity; therefore, low rankness is a reasonable perspective for promoting parsimonious models with only a few parameters explaining the data.

The literature outlines two approaches to employing low-rank constraints in tensor completion: the rank minimization method (implicit method) and the error minimization method (explicit method). In the former approach, rank is used implicitly as an optimization objective to minimize, with observed entries as constraints to be satisfied. As the rank function is non-convex and NP-hard [19,20], convex surrogates of the rank function are used to relax this problem, allowing for an approximate but efficient solution. In the latter approach, a hypothesis (tensor decomposition) model is explicitly imposed on a partially observed tensor with a specific, fixed low rank. The loss function—typically continuous and differentiable—is minimized to fit the model parameters (see, e.g., [13,14,15,21,22,23,24]). In this case, the flexibility in choosing a differentiable objective function enables the use of gradient-based optimization approaches (e.g., first- and/or second-order methods). Moreover, this approach explicitly uses rank information, which can be meaningful in applications where the rank has a physical significance; therefore, tweaking it may not be allowed. See [10] for an overview of matrix and tensor completion approaches.

Tensor completion problems are typically solved by numerical optimization algorithms (see, e.g., [13,14,15,17,23,24,25]). An essential aspect of a reliable completion algorithm is its recovery guarantees—specifically, the conditions under which it can uniquely recover unobserved entries from partial observations. Existing theories generally study probabilistic guarantees for recovery based on conditions such as entries being observed uniformly at random and satisfying incoherence requirements [21,22,26]. However, if an observation pattern has some structure, better and faster algorithms with deterministic guarantees can be designed by exploiting the structure. Note that, depending on the application, the observation pattern may be structured rather than random; it may even be fixed, for instance, when an incomplete dataset is given “as such”, without the possibility of acquiring more entries. There are many interesting problems where conditions like entries being observed uniformly at random may not be explicitly satisfied. In this work, we discuss one such interesting observation pattern where the fibers of a tensor (along a single specific mode) are either fully observed or entirely missing, unlike the usual entry-wise observations. This observation pattern is interesting because: (i) it occurs in many real-life applications; (ii) it makes an intriguing distinction in the uniqueness of the completion between matrix and tensor settings. Specifically, if some fibers (rows or columns) are entirely missing in a matrix, the completion problem becomes underdetermined, whereas completion is still possible in a higher-order tensor, even if some fibers are completely missing along a specific mode (see, e.g., [27,28,29,30]). In fact, there are many applications where it is easier to collect data (or sample a multivariate function) along one mode (variable) than along others. For example, consider weather time series (e.g., temperature and humidity) collected across various locations [31]. Think of collecting the data (temperature, latitude, latitude) at specific combinations of geospatial locations. Another example of a fiber-wise observation pattern arises when recording traffic speed data (road segment, day, time window) for specific combinations of road segments and days (see, e.g., [27], where this particular dataset is studied). Other examples include chemical reaction data with “time” and “concentration” modes. Obtaining samples along the temporal mode may be easier than varying the concentrations (which may require conducting new experiments).

Algebraic algorithms exploit the low-rank structure in a specific algebraic manner to design completion algorithms that rely solely on standard numerical linear algebra (NLA) techniques. These algorithms are fast and are guaranteed to work under reasonable deterministic conditions on an observation pattern. In this line, a novel algebraic method for fitting a low-rank matrix to a matrix with missing entries was proposed [30,32,33]. More detailed results on low-rank matrix completion, with extensive discussions on recovery guarantees, have also been presented (see, e.g., [30,34,35]). Previous studies have shown that the canonical polyadic decomposition (CPD) and multilinear singular value decomposition (MLSVD) of an incomplete tensor, observed along a single mode, can be computed using only standard NLA by exploiting the fiber-wise observation pattern [28,29,30]. As big data becomes more prevalent, the need for stable and scalable algorithms has become more pressing. The tensor train (TT) decomposition is stable, like the MLSVD and breaks the curse of dimensionality, like the CPD, since it has, asymptotically, the same number of parameters [1,36]. In light of these advantages, we present some of the results, including an extension of the algebraic algorithm to the TT format [37]. Note that there is an important difference compared with the popular technique of (TT) cross-approximation, in the sense that the latter obtains TT decomposition from a subset of fibers sampled across all modes [38]. In our method, fibers are sampled along a single mode.

1.1. Contributions

In this paper, we present more detailed results regarding our approach, with the following main additions:

• We provide more insights into piecewise subspace learning, specifically the conditions for determining the column space of a low-rank matrix, where only some pieces (submatrices) are observed.
• We include the subspace intersection approach, in addition to the subspace constraint approach, for computing the column space of a low-rank matrix from an informationally complete set of observed submatrices.
• We utilize the piecewise subspace learning approaches to compute TT approximation using only standard NLA operations.
• Convincing numerical experiments have been included to show that the proposed method is practically fast and reliable.
• In line with [39], we show that the TT decomposition obtained through algebraic completion can serve as a “proxy” for efficient subsequent computations (e.g., a constrained CPD is fitted to the TT approximation rather than to the actual tensor).

1.2. Preliminaries and Notation

We use lower-case, bold lower-case, bold capital, and calligraphic letters to denote scalars, vectors, matrices, and tensors, i.e., x, $\mathbf{x}$, $\mathbf{X}$, and $\mathcal{X}$, respectively. The order of a tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times I_2\times \cdots \times I_N}$ is the number of modes or ways it has. MATLAB-like indexing is used to specify a particular part of a tensor, employing commas, colons, and semicolons. For instance, the specific entry of the Nth-order tensor is denoted by $x_{i_1{i}_2\dots i_N}=\mathcal{X}(i_1,i_2,\dots ,i_N)$ with $i_n\in \{1,2,3,\dots ,I_n\}$ for $n\in \{1,2,3,\dots ,N\}$. A mode-n fiber of the Nth-order tensor is obtained by fixing every index except the nth. For a third-order tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times I_2\times I_3}$, the mode-1 fibers ${\mathbf{x}}_{:i_2{i}_3}$, mode-2 fibers ${\mathbf{x}}_{i_1:i_3}$, and mode-3 fibers ${\mathbf{x}}_{i_1{i}_2:}$ are also known as column, row, and tube fibers, respectively. Similarly, mode-1 slices ${\mathbf{X}}_{i_1::}$, mode-2 slices ${\mathbf{X}}_{:i_2:}$, and mode-3 slices ${\mathbf{X}}_{::i_3}$ are also known as horizontal, lateral, and frontal slices, respectively. The permutation and reshaping of a tensor are reflected in the ordering of its indices and the position of the semicolon (;), where the semicolon indicates a new mode, and the ordering of indices indicates the order in which the entries are stacked. In the nth matrix unfolding that is denoted by ${\mathbf{X}}_{[1,\dots ,n;n+1,\dots ,N]}\in {\mathbb{R}}^{I_1\cdots I_n\times I_{n+1}\cdots I_N}$, the first n indices enumerate rows and the remaining indices enumerate columns. The rank, column space (also called the range) and kernel of a matrix $\mathbf{X}$ are denoted by $\mathrm{rank}\left(\mathbf{X}\right)$, $\mathrm{col}\left(\mathbf{X}\right)$, and $ker\left(\mathbf{X}\right)$, respectively. The dimension of subspace S is denoted by $dim\left(S\right)$. Let ${\mathbf{e}}_i^{\left(I\right)}\in {\{0,1\}}^I$ denote a vector with a unit entry at index i and zeros elsewhere. We denote a set with a lower-case Greek letter. The cardinality of a set $\alpha$ is denoted by $\left|\alpha \right|$. The union and intersection of a sequence of the sets are denoted by ${\cup }_{l=1}^L{\alpha }_l$ and ${\cap }_{l=1}^L{\alpha }_l$, respectively.

Furthermore, we introduce the following notation to represent a submatrix of a matrix $\mathbf{X}\in {\mathbb{R}}^{J\times K}$. Let ${\alpha }_l=\{j_1,\dots ,j_{J_l}\}\subseteq \{1,\dots ,J\}$ and ${\beta }_l=\{k_1,\dots ,k_{K_l}\}\subseteq \{1,\dots ,K\}$ be the indices of the $J_l$ selected rows and $K_l$ selected columns, respectively. We define the corresponding row and column selection matrices as ${\mathbf{S}}_r^{\left(l\right)}={\left[{\mathbf{e}}_{j_1}^{\left(J\right)}\dots {\mathbf{e}}_{j_{J_l}}^{\left(J\right)}\right]}^{\top }\in {\{0,1\}}^{J_l\times J}$ and ${\mathbf{S}}_c^{\left(l\right)}=\left[{\mathbf{e}}_{k_1}^{\left(K\right)}\dots {\mathbf{e}}_{k_{K_l}}^{\left(K\right)}\right]\in {\{0,1\}}^{K\times K_l}$, respectively. Matrix ${\mathbf{S}}_r^{\left(l\right)}$ selects the $J_l$ rows indexed by ${\alpha }_l$ upon left multiplication, and ${\mathbf{S}}_c^{\left(l\right)}$ selects the $K_l$ columns indexed by ${\beta }_l$ upon right multiplication. For example, the submatrix ${\mathbf{X}}_{obs}^{\left(l\right)}={\mathbf{S}}_r^{\left(l\right)}\mathbf{X}{\mathbf{S}}_c^{\left(l\right)}\in {\mathbb{R}}^{J_l\times K_l}$ contains elements of $\mathbf{X}$ observed at $(j,k)\in {\alpha }_l\times {\beta }_l$.

Definition 1

(Isorank submatrix). A submatrix ${\mathbf{X}}_{obs}^{\left(l\right)}$ of a matrix $\mathbf{X}$ is called an isorank submatrix or rank-preserving submatrix if

$$\mathit{rank}\left({\mathbf{X}}_{obs}^{\left(l\right)}\right)=\mathit{rank}\left(\mathbf{X}\right),$$

where ${\mathbf{X}}_{obs}^{\left(l\right)}={\mathbf{S}}_r^{\left(l\right)}\mathbf{X}{\mathbf{S}}_c^{\left(l\right)}$. Here, ${\mathbf{S}}_r^{\left(l\right)}$ and ${\mathbf{S}}_c^{\left(l\right)}$ are row and column selection matrices defined by sets ${\alpha }_l$ and ${\beta }_l$, respectively, such that $min\left(\right|{\alpha }_l|,|{\beta }_l\left|\right)\ge \mathit{rank}\left(\mathbf{X}\right)$.

Definition 2

(Row overlap). Two submatrices ${\mathbf{X}}_{obs}^{\left(1\right)}$ and ${\mathbf{X}}_{obs}^{\left(2\right)}$ of a matrix $\mathbf{X}$, with the corresponding row selection index sets ${\alpha }_1$ and ${\alpha }_2$, are said to have a row overlap of size k if there exist k indices that are common to both ${\alpha }_1$ and ${\alpha }_2$, i.e., $|{\alpha }_1\cap {\alpha }_2|=k$.

Definition 3

(Contraction). The product between the last mode of tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times I_2\times \dots \times I_N}$ and the first mode of tensor $\mathcal{Y}\in {\mathbb{R}}^{J_1\times J_2\times \dots \times J_M}$, where $I_N=J_1=K$, yields an $(N+M-2)$-order tensor $\mathcal{Z}=\mathcal{X}•\mathcal{Y}$, the elements of which are given by

$$z_{i_1\dots i_{N-1}{j}_2\dots j_M}=\sum _{k=1}^K{x}_{i_1\dots i_{N-1}k}{y}_{k{j}_2\dots j_M}.$$

1.3. Organization

In Section 2, we provide a brief overview of the TT decomposition of a fully observed tensor. Following that, in Section 3, we outline our method for obtaining the TT decomposition of a tensor observed fiber-wise along a single mode. In Section 3.2 and Section 3.3, we discuss the algorithm and uniqueness conditions, respectively. Finally, we present convincing numerical experiments in Section 4 and a brief conclusion in Section 5.

2. TT Decomposition of Fully Observed Tensor

A TT decomposition of a tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times I_2\times \cdots \times I_N}$ corresponds to a contraction of a sequence of third-order core tensors (TT cores) ${\mathcal{G}}^{\left(n\right)}\in {\mathbb{R}}^{R_{n-1}\times I_n\times R_n}(1\le n\le N)$ with $R_0=R_N=1$, such that each entry of $\mathcal{X}$ can be expressed as the sequence of matrix products [36,40]:

$$\mathcal{X}={\mathcal{G}}^{\left(1\right)}•{\mathcal{G}}^{\left(2\right)}•\cdots •{\mathcal{G}}^{(N-1)}•{\mathcal{G}}^{\left(N\right)},$$

(1)

or entry-wise, we can write:

$$x_{i_1{i}_2\cdots i_{N-1}{i}_N}={\mathbf{G}}_{:i_1:}^{\left(1\right)}{\mathbf{G}}_{:i_2:}^{\left(2\right)}\cdots {\mathbf{G}}_{:i_{N-1}:}^{(N-1)}{\mathbf{G}}_{:i_N:}^{\left(N\right)},$$

(2)

where the matrix ${\mathbf{G}}_{:i:}^{\left(n\right)}\in {\mathbb{R}}^{R_{n-1}\times R_n}$ is the ith mode-2 slice of the core tensor ${\mathcal{G}}^{\left(n\right)}$. The tuple of minimal integers $(R_0,\dots ,R_N)$ for which the equality in Equations (1) and (2) holds is the TT rank of $\mathcal{X}$, denoted by ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)$. The dense tensor $\mathcal{X}$ has a total storage complexity of ${\prod }_{n=1}^N{I}_n$, whereas in the TT format, the storage complexity is ${\sum }_{n=1}^N{R}_{n-1}{I}_n{R}_n$. Hence, a low TT rank can greatly reduce storage complexity. A visualization of the TT decomposition of a 5th-order tensor is shown in Figure 1.

Additionally, we introduce a shorthand notation for TT decomposition using partial products. A tensor $\mathcal{X}$ can be represented as

$$\mathcal{X}={\mathcal{G}}^{(<n)}•{\mathcal{G}}^{(n:m)}•{\mathcal{G}}^{(>m)},1\le n\le m\le N,$$

(3)

where the left partial product ${\mathcal{G}}^{(<n)}={\mathcal{G}}^{\left(1\right)}•\cdots •{\mathcal{G}}^{(n-1)}$, the right partial product ${\mathcal{G}}^{(>m)}={\mathcal{G}}^{(m+1)}•\cdots •{\mathcal{G}}^{\left(N\right)}$ and ${\mathcal{G}}^{(n:m)}={\mathcal{G}}^{\left(n\right)}•\cdots •{\mathcal{G}}^{\left(m\right)}$.

TT decomposition of a given tensor $\mathcal{X}$ is computed by a sequence of truncated SVDs (TT-SVD) [36]. The TT-SVD algorithm sequentially computes TT cores and essentially alternates between the SVD computation step and projection step, which makes it difficult to parallelize. Inspired by the natural parallelizability of MLSVD [41], a similar method was proposed that computes the orthonormal basis for the column spaces of all matrix unfoldings independently and then utilizes them to compute the TT cores (Parallel-TTSVD) [42]. It should be noted that independently computing the orthonormal basis for the column spaces of each matrix unfolding is more expensive than the basis computations in TT-SVD, where the projection step reduces the complexity of subsequent computations. One can reduce this computational burden by replacing SVDs with randomized SVDs [43,44]. The algorithm for computing the TT decomposition with parallel SVDs (Parallel-TTSVD) is shown in Algorithm 1 (for more in-depth details, refer to [42], Algorithm 3.1).

Algorithm 1: Parallel-TTSVD

3. TT Completion from Fiberwise Observations Along a Single Mode

Roadmap: This section presents our main contribution. Our goal is to design an algebraic algorithm for computing the TT decomposition—analogous to Algorithm 1, but able to handle an incomplete tensor observed through mode-N fibers. We begin by examining the structured observation pattern that appears in the matrix unfoldings of such tensors. In Section 3.1, we show how to compute the column space of a low-rank matrix formed by stacking two slices of a fiber-wise observed tensor, and then extend this approach to the partially observed matrix unfolding that we are actually dealing with (and which consists of multiple slices). This leads to two subspace-estimation methods presented in Section 3.1.1 and Section 3.1.2. Finally, in Section 3.2, we use this core subspace-estimation step to build the full TT completion algorithm and, in Section 3.3, summarize the conditions under which it is guaranteed to succeed.

Assume a given tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times \cdots \times I_N}$ with ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)=(R_0,\dots ,R_N)$ of which only some of the mode-N (last mode) fibers are observed. One essential step to compute the TT cores is to find orthonormal bases for the ranges of the matrix unfoldings. However, since the matrix unfoldings have missing entries, it is not directly possible to utilize, e.g., SVD or QR factorization to obtain the ranges. Note that under fiber-wise observations, two types of observation patterns arise in matrix unfoldings. In the $(N-1)$th matrix unfolding, i.e., ${\mathbf{X}}_{[1,\dots ,N-1;N]}$, the rows are either fully observed or entirely missing. If the number of observed rows is greater than or equal to $R_{N-1}$, we can generically (i.e., with probability 1 when the matrix entries are drawn from a continuous distribution) obtain the last TT core $({\mathcal{G}}^{\left(N\right)}\in {\mathbb{R}}^{R_{N-1}\times I_N\times 1})$ by computing the top $R_{N-1}$ right-singular vectors of the matrix formed by the observed rows. The other matrix unfoldings can be seen as the horizontal stack of mode-2 slices ${\tilde{\mathbf{X}}}_{:i:}$ of a third-order reshaping $\tilde{\mathcal{X}}={\mathcal{X}}_{[1,\dots ,n;n+1,\dots ,N-1;N]}\in {\mathbb{R}}^{{\prod }_{i=1}^n{I}_i\times {\prod }_{i=n+1}^{N-1}{I}_i\times I_N}$ of the tensor $\mathcal{X}$. That is,

$${\mathbf{X}}_{[1,\dots ,n;n+1,\dots ,N]}=\left[{\tilde{\mathbf{X}}}_{:1:}\cdots {\tilde{\mathbf{X}}}_{:{\prod }_{i=n+1}^{N-1}{I}_i:}\right].$$

(4)

In this case, the rows of the lateral slices (submatrices of the unfolding) of $\tilde{\mathcal{X}}$ are either fully observed or entirely missing. However, different slices may have observed rows at different row indices. A visual representation of such a matrix unfolding is shown in Figure 2.

The following section will discuss how to determine the orthonormal basis for the column space of such a partially observed matrix unfolding.

3.1. Piecewise Subspace Learning

In this section, we will discuss how to determine the overall column space of a low-rank matrix, of which only some pieces (submatrices) are observed. Moreover, we will characterize the sampling of pieces such that the overall subspace is guaranteed to be unique (i.e., the subspace is identifiable). Subspace identifiability is closely related to the matrix completion problem. The conditions required to identify the subspace (e.g., column space) are necessary but not sufficient for matrix completion [30,45]. We will first explore the conditions for unique matrix completion, a problem that is widely studied in the literature, and then discuss how these conditions are relaxed in the context of subspace identification.

For simplicity, we use $\mathbf{M}\in {\mathbb{R}}^{J\times K}$ to denote a rank-R matrix unfolding of which we need to determine the R-dimensional column space. In the literature, unique minimal rank completions for various types of observation patterns have been studied (see, e.g., [46,47,48]). This work will focus on an observation pattern shown in Figure 2. Before proceeding to the general case, we will begin with a simple matrix formed by stacking two slices of a tensor, as explained next. Denote the observed submatrices of $\mathbf{M}$ by ${\mathbf{M}}_{obs}^{\left(1\right)}\in R^{(J_1+J_2)\times K_1}$ and ${\mathbf{M}}_{obs}^{\left(2\right)}\in R^{(J_2+J_3)\times K_2}$ that have $J_2$ overlapping rows, and $J=J_1+J_2+J_3$ and $K=K_1+K_2$. (Observed submatrices of $\mathbf{M}$ are shown shaded.) While the overlap can occur anywhere among the rows, let us assume, without loss of generality, that the row overlap occurs in the middle block, as shown below:

$$\mathbf{M}=\left[\begin{array}{cc}{\displaystyle {\mathbf{M}}_{11}}& {\mathbf{M}}_{12}\\ {\displaystyle {\mathbf{M}}_{21}}& {\displaystyle {\mathbf{M}}_{22}}\\ {\mathbf{M}}_{33}& {\displaystyle {\mathbf{M}}_{32}}\end{array}\right],{\mathbf{M}}_{obs}^{\left(1\right)}=\left[\begin{array}{c}{\mathbf{M}}_{11}\\ {\mathbf{M}}_{21}\end{array}\right]\mathrm{and}{\mathbf{M}}_{obs}^{\left(2\right)}=\left[\begin{array}{c}{\mathbf{M}}_{22}\\ {\mathbf{M}}_{32}\end{array}\right].$$

(5)

It has been proven that there is a unique rank-R completion for $\mathbf{M}$ if and only if the following condition holds (see, e.g., Corollary 2.3 in [46]):

$$\mathrm{rank}\left({\mathbf{M}}_{obs}^{\left(1\right)}\right)=\mathrm{rank}\left({\mathbf{M}}_{21}\right)=\mathrm{rank}\left(\left[{\mathbf{M}}_{21}{\mathbf{M}}_{22}\right]\right)=\mathrm{rank}\left({\mathbf{M}}_{22}\right)=\mathrm{rank}\left({\mathbf{M}}_{obs}^{\left(2\right)}\right)=R.$$

(6)

As noted above, subspace identifiability is less restrictive than matrix completion. To illustrate this, consider an incomplete matrix formed by concatenating a matrix $\mathbf{M}$ that satisfies condition (6), and a vector $\mathbf{z}$. Let this concatenated matrix be denoted as $\widehat{\mathbf{M}}=\left[\mathbf{M}\mid \mathbf{z}\right]$, and assume that $\mathrm{rank}\left(\widehat{\mathbf{M}}\right)=\mathrm{rank}\left(\mathbf{M}\right)=R$. The subspace identifiability of $\widehat{\mathbf{M}}$ is ensured by $\mathbf{M}$ since the latter satisfies the row overlapping condition. As a result, $\mathrm{col}\left(\widehat{\mathbf{M}}\right)$ can be determined solely from $\mathbf{M}$ [30,35,45]. However, ensuring the unique completion of $\widehat{\mathbf{M}}$ requires an additional condition: the vector $\mathbf{z}$ should also have at least R observed entries. Without this, a unique recovery is not possible.

The study in [30] investigates the subspace identifiability of a partially observed low-rank matrix formed by stacking multiple slices (submatrices) of an incomplete tensor as shown in Figure 2. The authors show that subspace identifiability also requires a row-overlapping condition as stated in (6). However, this is not very restrictive as only some, rather than all, pairs of observed submatrices need to satisfy it. As noted earlier, for the full completion to be unique, we also need to ensure that every column contains at least R observed entries. For further details, see [30,35,45].

In the following example of a partially observed rank-1 matrix, we provide a geometrical interpretation of the conditions required to identify the column space and set out the basis for the algebraic algorithm for computing the desired column space more generally.

Example 1.

A partially observed rank-1 matrix is given: $\mathbf{M}=\left[\begin{array}{cc}{\displaystyle x_1}& x_2\\ {\displaystyle y_1}& {\displaystyle y_2}\\ z_1& {\displaystyle z_2}\end{array}\right]$. Suppose we are required to determine its 1-dimensional range in ${\mathbb{R}}^3$.

We discuss the subspace identifiability of the rank-1 matrix $\mathbf{M}$ in terms of subspaces associated with partially observed columns. In the first column, $z_1$ is missing and may lay anywhere on the line parallel to the z-axis passing through $(x_1,y_1)$, as shown in Figure 3a. The subspace $S_1$ (shaded region) accounts for all possible completions of the first column. Similarly, in the second column, entry $x_2$ is missing and may lay anywhere on the line parallel to the x-axis passing through $(y_2,z_2)$, and the corresponding subspace that accounts for all possible completions of the second column is $S_2$ (shaded region), as shown in Figure 3b. Given that the unknown 1-dimensional subspace $\mathit{col}\left(\mathbf{M}\right)$ spans both columns, it must lie in the intersection of these affine subspaces, i.e., $\mathit{col}\left(\mathbf{M}\right)\subseteq S_1\cap S_2$. Note that there is a subset relationship between $\mathit{col}\left(\mathbf{M}\right)$ and $S_1\cap S_2$, not an equality. However, if $dim(S_1\cap S_2)=1$, i.e., if $S_1\cap S_2$ forms a line, then the desired $\mathit{col}\left(\mathbf{M}\right)$ is necessarily equal to $S_1\cap S_2$. Therefore, any nonzero vector $\mathbf{v}\in S_1\cap S_2$ spans $\mathit{col}\left(\mathbf{M}\right)$, as shown in Figure 3c. In this case, any technique that computes the intersection of subspaces can be employed to find the desired subspace $\mathit{col}\left(\mathbf{M}\right).$

One approach to determining the intersecting subspace is via the null spaces. Specifically, find ${\mathbf{n}}_{\mathbf{1}}$ and ${\mathbf{n}}_{\mathbf{2}}$ such that $\langle {\mathbf{n}}_{\mathbf{1}},{\mathbf{m}}_{:1}\rangle =0$, and $\langle {\mathbf{n}}_{\mathbf{2}},{\mathbf{m}}_{:2}\rangle =0$. Define $\mathbf{N}=\left[{\mathbf{n}}_{\mathbf{1}}{\mathbf{n}}_{\mathbf{2}}\right]\in {\mathbb{R}}^{3\times 2}$. If $dim(ker\left({\mathbf{N}}^{\top }\right))=1$, then the subspace S is given by $ker\left({\mathbf{N}}^{\top }\right)$. The condition $dim(ker\left({\mathbf{N}}^{\top }\right))=1$ represents the dual of the condition $dim(S_1\cap S_2)=1$. Several other approaches have been proposed to find the intersection of subspaces (see, e.g., [49]).

Before proceeding to the rank-R case, for ease of notation, we use $\mathbf{M}\in {\mathbb{R}}^{J\times K}$ to denote a low-rank matrix ${\mathbf{X}}_{[1,\dots ,n;n+1,\dots ,N]}$ for which we need to determine the R-dimensional column space. Let ${\mathbf{M}}_{obs}^{\left(l\right)}={\mathbf{S}}_r^{\left(l\right)}\mathbf{M}{\mathbf{S}}_c^{\left(l\right)}\in {\mathbb{R}}^{J_l\times K_l}$ represent the lth fully observed isorank submatrix of $\mathbf{M}$. We denote a matrix that holds an orthonormal basis for $\mathrm{col}\left(\mathbf{M}\right)$ by $\mathbf{A}\in {\mathbb{R}}^{J\times R}$. Below, we discuss two approaches to determine $\mathbf{A}$: (1) the subspace constraint approach, which computes it via null spaces of observed submatrices, and (2) the subspace intersection method, which computes it via column spaces of observed submatrices.

3.1.1. Subspace Constraint Approach

The main idea of the subspace constraint method is to determine the overall range of a matrix from a set of constraints derived from the submatrices [32]. This method involves identifying an informationally complete set of fully observed isorank submatrices ${\left\{{\mathbf{M}}_{obs}^{\left(l\right)}\right\}}_{l=1}^L$ of $\mathbf{M}$ [29]. By informationally complete, we mean that the set of constraints derived from the submatrices should fully characterize the desired subspace $\mathrm{col}\left(\mathbf{A}\right)$. As we assume ${\mathbf{M}}_{obs}^{\left(l\right)}$ is an isorank submatrix of $\mathbf{M}$, we can write ${\mathbf{M}}_{obs}^{\left(l\right)}={\mathbf{A}}_{obs}^{\left(l\right)}{\mathbf{B}}_{obs}^{\left(l\right)\top }$, where matrices ${\mathbf{A}}_{obs}^{\left(l\right)}\in {\mathbb{R}}^{J_l\times R}$ and ${\mathbf{B}}_{obs}^{\left(l\right)}\in {\mathbb{R}}^{K_l\times R}$ are full column rank. Stacking a basis for the orthogonal complement of $\mathrm{col}\left({\mathbf{M}}_{obs}^{\left(l\right)}\right)$ in a matrix ${\mathbf{N}}_{obs}^{\left(l\right)}\in {\mathbb{R}}^{J_l\times (J_l-R)}$, each column of the resulting matrix after zero padding, i.e., ${\mathbf{S}}_r^{\left(l\right)\top }{\mathbf{N}}_{obs}^{\left(l\right)}\in {\mathbb{R}}^{J\times (J_l-R)}$, represents a vector that is orthogonal to the desired subspace $\mathrm{col}\left(\mathbf{A}\right)$, thus imposing constraints on what $\mathrm{col}\left(\mathbf{A}\right)$ may be. To obtain the desired $\mathrm{col}\left(\mathbf{A}\right)$, we find orthogonal complements for L such submatrices ${\mathbf{M}}_{obs}^{\left(l\right)}$ and concatenate them in a matrix $\mathbf{N}=\left[{\mathbf{S}}_r^{\left(1\right)\top }{\mathbf{N}}_{obs}^{\left(1\right)},\cdots ,{\mathbf{S}}_r^{\left(L\right)\top }{\mathbf{N}}_{obs}^{\left(L\right)}\right]\in {\mathbb{R}}^{J\times \left({\sum }_{l=1}^L(J_l-R)\right)}$. If the L orthogonal complements are enough to ensure the $ker\left({\mathbf{N}}^{\top }\right)$ is of minimal dimension, i.e., $dim(ker\left({\mathbf{N}}^{\top }\right))=R$, where the very existence of $\mathrm{col}\left(\mathbf{A}\right)$ implies that the dimension is at least $R$, then the desired $\mathrm{col}\left(\mathbf{A}\right)$ is necessarily given by $ker\left({\mathbf{N}}^{\top }\right)$. In other words, the number of independent constraints must be at least $J-R$, which leads to the inequality ${\sum }_{l=1}^L(J_l-R)\ge J-R$. For example, if we assume that each observed submatrix imposes one (independent) constraint and we have $J_l=R+1$, then the minimum number of the observed submatrices required is $L=J-R$. This can be seen as the generalization of the rank-1 case mentioned in Example 1. In what follows, we assume that our working assumptions hold, i.e., ${\left\{{\mathbf{M}}_{obs}^{\left(l\right)}\right\}}_{l=1}^L$ is a set of isorank submatrices of $\mathbf{M}$ and $dim(ker\left({\mathbf{N}}^{\top }\right))=R$. Then, this set of submatrices is informationally complete. In the noisy case, we estimate ${\mathbf{N}}_{obs}^{\left(l\right)}$ from the left-singular vectors corresponding to the smallest singular values of ${\mathbf{M}}_{obs}^{\left(l\right)}$. Let the SVD of ${\mathbf{M}}_{obs}^{\left(l\right)}$ be ${\mathbf{M}}_{obs}^{\left(l\right)}={\mathbf{U}}^{\left(l\right)}{\mathbf{\sum }}^{\left(l\right)}{\mathbf{V}}^{\left(l\right)\top }$. Then, ${\mathbf{N}}_{obs}^{\left(l\right)}=\left[{\mathbf{u}}_{:R+1}^{\left(l\right)}\cdots {\mathbf{u}}_{:J_l}^{\left(l\right)}\right]$. In the noisy case, we also estimate the desired $\mathrm{col}\left(\mathbf{A}\right)$ to be the subspace that is most orthogonal to the space spanned by the columns of $\mathbf{N}$, i.e., we estimate $\mathrm{col}\left(\mathbf{A}\right)$ as the (approximate) kernel of ${\mathbf{N}}^{\top }$. Let the SVD of $\mathbf{N}$ be $\mathbf{U}\mathbf{\sum }{\mathbf{V}}^{\top }$. Then $\mathbf{A}=\left[{\mathbf{u}}_{:J-R+1}\dots {\mathbf{u}}_{:J}\right]$.

3.1.2. Subspace Intersection Approach

Let us first define a binary matrix ${\tilde{\mathbf{S}}}_r^{\left(l\right)}\in {\{0,1\}}^{(J-J_l)\times J}$, in which the rows correspond to standard unit vectors that indicate which rows of $\mathbf{M}$ are missing in ${\mathbf{M}}_{obs}^{\left(l\right)}$. (The matrix ${\tilde{\mathbf{S}}}_r^{\left(l\right)}$ is such that ${\mathbf{I}}_J=\left[{\mathbf{S}}_r^{\left(l\right)\top }{\tilde{\mathbf{S}}}_r^{\left(l\right)\top }\right]\mathbf{\prod }\in {\mathbb{R}}^{J\times J}$ for some permutation matrix $\mathbf{\prod }\in {\mathbb{R}}^{J\times J}$.) Additionally, let $S_l$ denote a subspace that accounts for all possible completions of rows that are missing in ${\mathbf{M}}_{obs}^{\left(l\right)}$. Specifically, let the SVD of ${\mathbf{M}}_{obs}^{\left(l\right)}$ be ${\mathbf{M}}_{obs}^{\left(l\right)}={\mathbf{U}}^{\left(l\right)}{\mathbf{\sum }}^{\left(l\right)}{\mathbf{V}}^{\left(l\right)\top }$. Then, the subspace $S_l$ is given by $S_l=span\left({\mathbf{Q}}_{S_l}\right)$, where ${\mathbf{Q}}_{S_l}:=\left[{\mathbf{S}}_r^{\left(l\right)\top }[{\mathbf{u}}_{:1}^{\left(l\right)}\cdots {\mathbf{u}}_{:R}^{\left(l\right)}]{\tilde{\mathbf{S}}}_r^{\left(l\right)\top }\right]\in {\mathbb{R}}^{J\times (R+J-J_l)}$; note that the columns of ${\mathbf{Q}}_{S_l}$ form an orthonormal basis for the subspace $S_l$.

As illustrated in Example 1, the desired subspace $\mathrm{col}\left(\mathbf{M}\right)$ satisfies $\mathrm{col}\left(\mathbf{M}\right)\subseteq \left({\cap }_{l=1}^L{S}_l\right)$. However, since we assume the subspaces are associated with an informationally complete set of fully observed isorank submatrices ${\left\{{\mathbf{M}}_{obs}^{\left(l\right)}\right\}}_{l=1}^L$, i.e., $dim\left({\cap }_{l=1}^L{S}_l\right)=R\iff dim(ker\left({\mathbf{N}}^{\top }\right))=R$, it follows that the R-dimensional subspace $\mathrm{col}\left(\mathbf{M}\right)$ is necessarily equal to ${\cap }_{l=1}^L{S}_l$. Hence, we simply need to find the intersection of the subspaces.

A closed-form solution for computing the intersection of $L\ge 2$ subspaces in finite-dimensional spaces is discussed in [49], and a low-complexity implementation of these formulas based on SVD is presented in [30,50]. The procedure used to compute the intersection is as follows. Given a set of matrices ${\left\{{\mathbf{Q}}_{S_l}\right\}}_{l=1}^L$, the intersection of the corresponding subspaces can be computed as ${\cap }_{l=1}^L{S}_l=ker\left(L{\mathbf{I}}_J-{\sum }_{l=1}^L{\mathbf{Q}}_{S_l}{\mathbf{Q}}_{S_l}^{\top }\right)$. It has been shown that if $dim\left({\cap }_{l=1}^L{S}_l\right)=R$, then the solution can be computed more efficiently via the SVD of a concatenated matrix $\mathbf{Q}=\left[{\mathbf{Q}}_{S_1},\cdots ,{\mathbf{Q}}_{S_L}\right]\in {\mathbb{R}}^{J\times \left(LR+{\sum }_{l=1}^L(J-J_l)\right)}$, without first calculating the orthogonal projectors $\left\{{\mathbf{Q}}_{S_l}{\mathbf{Q}}_{S_l}^{\top }\right\}$. Specifically, let the SVD of $\mathbf{Q}$ be $\mathbf{Q}=\mathbf{U}\mathbf{\sum }{\mathbf{V}}^{\top }$. Then, $\mathbf{A}=\left[{\mathbf{u}}_{:1},\cdots ,{\mathbf{u}}_{:R}\right]$ forms an orthonormal basis for the desired subspace $\mathrm{col}\left(\mathbf{M}\right)$. See [30,49,51] for an in-depth discussion. In case the matrix $\mathbf{Q}$ is too large to fit in memory, one can use an incremental SVD to find the dominant left singular subspace [52,53].

3.2. Algorithm

This section presents our algorithm for computing the TT decomposition of a tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times I_2\times \cdots \times I_N}$ observed along the Nth mode. Our algorithm is similar to Algorithm 1, with the key subspace computation step in line 2 adapted to handle a tensor observed through mode-N fibers. The key steps of the algorithm are as follows. The piecewise subspace learning approach is employed to compute orthonormal bases for the column spaces of the partially observed matrix unfoldings ${\mathbf{X}}_{[1,\dots ,n;n+1,\dots ,N]}$ for $1\le n\le N-2$, assuming that the corresponding observed submatrices satisfy the informational-completeness condition of Section 3.3. These orthonormal bases are used to compute the TT cores ${\mathcal{G}}^{\left(1\right)},\dots ,{\mathcal{G}}^{(N-2)}$. The last core ${\mathcal{G}}^{\left(N\right)}$ is obtained from an orthonormal basis of the observed rows of the $(N-1)$th matrix unfolding, and the penultimate core ${\mathcal{G}}^{(N-1)}$ is computed in a least-squares sense, as explained in Section 3.2.1. The pseudocode for the proposed method is outlined in Algorithm 2.

Algorithm 2: TT mode-N fiber-wise

3.2.1. Computing the Next-to-Last TT Core ${\mathcal{G}}^{(N-1)}$

In the fully observed case, the last core can be computed using the SVD of the $(N-1)$th unfolding matrix ${\mathbf{X}}_{[1,\dots ,N-1;N]}$. Let its SVD be ${\mathbf{X}}_{[1,\dots ,N-1;N]}={\mathbf{U}}^{(N-1)}{\mathbf{\sum }}^{(N-1)}{\mathbf{V}}^{(N-1)\top }$. Then, the last TT core is obtained as ${\mathcal{G}}^{\left(N\right)}={\mathbf{\sum }}^{(N-1)}{\mathbf{V}}^{(N-1)\top }$. However, for fiber-wise observed tensors, this is not directly possible. Indeed, since $\mathcal{X}$ is observed through mode-N fibers, some rows of ${\mathbf{X}}_{[1,\dots ,N-1;N]}$ are now completely observed while other rows are entirely missing. Consequently, under the informational-completeness condition, ${\mathbf{V}}^{(N-1)\top }$ can be estimated but ${\mathbf{U}}^{(N-1)}$ and ${\mathbf{\sum }}^{(N-1)}$ cannot. To address this issue, we set the last core ${\mathcal{G}}^{\left(N\right)}={\mathbf{V}}^{(N-1)\top }$, i.e., the orthonormal basis for the observed rows $\left({\mathbf{S}}_r{\mathbf{X}}_{[1,\dots ,N-1;N]}\right)$. Here, ${\mathbf{S}}_r\in {\{0,1\}}^{\left|\alpha \right|\times {\prod }_{i=1}^{N-1}{I}_i}$ represents the row selection matrix for the $(N-1)$th matrix unfolding, where $\left|\alpha \right|$ denotes the number of observed mode-N fibers (rows) indexed by $\alpha$. We then fix the scaling indeterminacies by computing the next-to-last TT core in a least-squares sense. For the third-order tensor $\tilde{\mathcal{X}}={\mathcal{X}}_{[1,\dots ,N-2;N-1;N]}$, the slice-wise representation, using (3), can be written as

$${\tilde{\mathbf{X}}}_{:i:}={\mathbf{G}}^{(<N-1)}{\mathbf{G}}_{:i:}^{(N-1)}{\mathbf{G}}^{(>N-1)},i\in \{1,\dots ,I_{N-1}\},$$

(7)

where the matrices ${\mathbf{G}}^{(<N-1)}\in {\mathbb{R}}^{{\prod }_{i=1}^{N-2}{I}_i\times R_{N-2}}$ and ${\mathbf{G}}^{(>N-1)}={\mathbf{G}}^{\left(N\right)}\in {\mathbb{R}}^{R_{N-1}\times I_N}$ have orthonormal columns and orthonormal rows, respectively. Let ${\left\{{\mathbf{S}}_r^{\left(i\right)}\right\}}_{i=1}^{I_{N-1}}$ be the row selection matrices for the mode-2 slices of $\tilde{\mathcal{X}}$. The mode-2 slices ${\mathbf{G}}_{:i:}^{(N-1)}$ of ${\mathcal{G}}^{(N-1)}$ can now be computed by solving the following linear systems:

$${\mathbf{S}}_r^{\left(i\right)}\left({\tilde{\mathbf{X}}}_{:i:}{\mathbf{G}}^{\left(N\right)\top }\right)=\left({\mathbf{S}}_r^{\left(i\right)}{\mathbf{G}}^{(<N-1)}\right){\mathbf{G}}_{:i:}^{(N-1)},i\in \{1,\dots ,I_{N-1}\}.$$

(8)

3.2.2. Computational Complexity

The computational cost of the algorithm is dominated by the subspace computation step in line 2 of Algorithm 2. This step can be performed in parallel since the unfolding matrices are independent; hence, we report the per-processor cost. For the nth unfolding matrix ${\mathbf{X}}_{[1,\dots ,n;n+1,\dots ,N]}$, we compute an orthonormal basis ${\mathbf{A}}^{\left(n\right)}$ for its column space via a rank-R SVD of the concatenated sparse matrix $\mathbf{Q}\in {\mathbb{R}}^{I^n\times \left(LR+{\sum }_{l=1}^L(I^n-J_l)\right)}$, as discussed in Section 3.1.2 (alternatively, $\mathbf{N}$ can be used; see Section 3.1.1), assuming $I_n=I$ and $R_n=R$. Here, L is the number of slices used—at most $I^{N-n-1}$ (as shown in (4))—that satisfy the informational-completeness condition of Section 3.3, and $J_l$ is the number of observed rows in the lth slice.

In practice, one uses sparse (or randomized) SVD methods on $\mathbf{Q}$, yielding a cost of $\mathcal{O}\left(Tnnz\left(\mathbf{Q}\right)R\right)$, where T is the number of iterations and $nnz\left(\mathbf{Q}\right)$ denotes the number of nonzeros in $\mathbf{Q}$ given by, ${\sum }_{l=1}^L{J}_{l}R+{\sum }_{l=1}^L(I^n-J_l)$. Under minimal oversampling, where each slice contributes as few as $J_l\sim R+c$ observed rows for some small constant $c>0$, and all $L=I^{N-n-1}$ slices are required (worst-case), we have $nnz\left(\mathbf{Q}\right)=L{I}^n+L(R^2+cR-R-c)\sim L{I}^n\mathrm{for}{I}^n\gg R$. Consequently, the resulting computational cost scales as $\mathcal{O}\left(T{I}^{N-1}R\right)$.

3.3. Uniqueness Conditions

This section summarizes the deterministic conditions under which a tensor $\mathcal{X}\in {\mathbb{R}}^{I_1\times I_2\times \cdots \times I_N}$, observed through mode-N fibers, admits a unique TT decomposition $\mathcal{X}={\mathcal{G}}^{\left(1\right)}•\cdots •{\mathcal{G}}^{\left(N\right)}$ (up to basis transformations (i.e., inserting ${\mathbf{RR}}^{-\mathbf{1}}$ between consecutive TT cores, with invertible $\mathbf{R}$, leaves the tensor unchanged)), in the noiseless setting. Intuitively, deterministic recovery follows from the fact that the fiber-wise observation pattern allows for an informational overlap to be created that algebraically locks in the solution and, consequently, reduces the completion problem to computing simple SVDs and solving a few linear systems.

To compute the TT cores ${\mathcal{G}}^{\left(1\right)},\dots ,{\mathcal{G}}^{(N-2)}$, Algorithm 2 utilizes orthonormal bases for the column spaces of the unfoldings ${\mathbf{X}}_{[1,\dots ,n;n+1,\dots ,N]}$, $n=1,\dots ,N-2$, which exhibit a structured observation pattern, as shown in Figure 2. For each n, the basis ${\mathbf{A}}^{\left(n\right)}$ is obtained via the piecewise subspace learning approach of Section 3.1, with the unfolding playing the role of $\mathbf{M}$ and the observed slice rows ${\mathbf{S}}_r^{\left(l\right)}{\tilde{\mathbf{X}}}_{:l:}$ playing the role of ${\mathbf{M}}_{obs}^{\left(l\right)}$. Accordingly, ${\mathbf{A}}^{\left(n\right)}$ is recovered from $\mathbf{N}$ (Section 3.1.1) or $\mathbf{Q}$ (Section 3.1.2). The last core ${\mathcal{G}}^{\left(N\right)}$ is obtained by computing the top $R_{N-1}$ right-singular vectors of the matrix formed by the observed rows of the $(N-1)$th unfolding, and ${\mathcal{G}}^{(N-1)}$ is obtained by solving (8).

Theorem 1

(Algebraic conditions). The TT cores ${\{{\mathcal{G}}^{\left(n\right)}\in {\mathbb{R}}^{R_{n-1}\times I_n\times R_n}\}}_{n=1}^N$ are uniquely determined (up to basis transformations) if the following hold:

I. 

Cores ${\mathcal{G}}^{\left(\mathbf{1}\right)}$–${\mathcal{G}}^{(\mathbf{N}-\mathbf{2})}$: For each $n=1,\dots ,N-2$, the nth unfolding contains sufficiently many fully observed rank-$R_n$ (isorank) submatrices $\left\{{\mathbf{S}}_r^{\left(l\right)}{\tilde{\mathbf{X}}}_{:l:}\right\}$ such that $dim(ker\left({\mathbf{N}}^{\top }\right))=R_n$ (equivalently $dim\left({\cap }_{l=1}^L{S}_l\right)=R_n$); this is the informational-completeness condition (see Section 3.1.1 and Section 3.1.2).

II. 

Core ${\mathcal{G}}^{(\mathbf{N}-\mathbf{1})}$: The systems in (8) admit a unique solution, which is the case if ${\mathbf{S}}_r^{\left(i\right)}{\mathbf{G}}^{(<N-1)}$ has full column rank for all i.

III. 

Core ${\mathcal{G}}^{\left(\mathbf{N}\right)}$: The matrix ${\mathbf{S}}_r{\mathbf{X}}_{[1,\dots ,N-1;N]}$ containing the observed mode-N fibers has rank $R_{N-1}$.

Remark 1.

The conditions in Theorem 1 are not only sufficient for computation but also necessary for uniqueness. If a core fails to satisfy its condition, it is not uniquely identifiable.

Corollary 1

(Generic conditions). Theorem 1 holds generically if the following hold:

I’. 

For each $n=1,\dots ,N-2:$

(i) 

Among the observed submatrices $\left\{{\mathbf{S}}_r^{\left(l\right)}{\tilde{\mathbf{X}}}_{:l:}\right\}$ of the nth unfolding, some pairs of submatrices overlap in at least $R_n$ observed rows [30], Corollary 3.5.

(ii) 

Every row of the unfolding is (partially) observed in at least one submatrix.

These generically imply Condition I; see [30,35] for details.

II’. 

Each mode-2 slice of ${\mathcal{X}}_{[1,\dots ,N-2;N-1;N]}$ contains at least $R_{N-2}$ observed rows.

III’. 

At least $R_{N-1}$ mode-N fibers are observed.

In the noiseless case, the proposed method computes the exact TT decomposition of the fiber-wise observed tensor if the uniqueness conditions are satisfied. In the presence of noise, each step can be computed in a least-squares sense, and thus the estimated TT decomposition is expected to be close to the true TT decomposition.

4. Numerical Experiments

This section evaluates the performance of the proposed method across three setups. First, we experiment with synthetic data to show that our algebraic method is accurate and computationally efficient, comparing it with the TT weighted optimization (TT-WOPT) [15], parallel matrix factorization for low-rank TT completion (TMac-TT) [25,54], and simple low-rank TT completion (SiLRTC-TT) [25,55]. Second, we showcase two real-world applications, namely multidimensional harmonic retrieval (MHR) and spatiotemporal weather data completion. Finally, we show the use of the proposed algebraic approach as a “proxy” for efficiently performing subsequent computations. In line with [39], we fit a non-negative CPD to the TT approximation instead of the actual tensor.

Baselines: TT-WOPT is an error minimization method that fits the parameters of a fixed rank TT model by minimizing a weighted least-squares loss using the gradient descent approach; see [15] for details. TMac-TT is an error minimization method that enforces low TT rank by fitting matrix factorizations to each mode unfolding of the tensor and updating the factors using alternating least squares; see [25,54] for more details. SiLRTC-TT is a rank minimization method that minimizes a nuclear norm relaxation of the TT rank of the tensor. The resulting convex problem is solved by a singular value thresholding algorithm; see [25,55] for more details.

The experiments were performed on an HP EliteBook 845 G8 Notebook PC with an AMD Ryzen 7 PRO 5850U CPU and 32 GB RAM.

4.1. Synthetic Data

4.1.1. Completion from Noisy Observations

A 4th-order tensor in TT format $\mathcal{X}\in {\mathbb{R}}^{15\times 15\times 15\times 15}$ is generated with ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)=(1,3,3,4,1)$ by sampling entries of the TT cores from a standard normal distribution. Random Gaussian i.i.d. noise $\mathcal{N}$ is added to the generated tensor, resulting in a noisy tensor ${\mathcal{X}}_{noisy}=\mathcal{X}+\mathcal{N}$ with a fixed signal-to-noise ratio (SNR), defined as $\mathrm{SNR}=20{log}_{10}{\displaystyle \frac{\parallel \mathcal{X}\parallel }{\parallel \mathcal{N}\parallel }}$. Forty percent of the mode-4 fibers are randomly removed from the noisy tensor. Algorithm 2 is run along with the reference algorithms to compute the low-rank TT approximations with SNR varying from $-10$ dB to 50 dB. The reference algorithms are run with default parameters (see [15,25]), except for parameter $\alpha$ in SiLRTC-TT, which is determined through an extensive grid search. The median of the relative error, defined as $\mathrm{Relative}\mathrm{Error}={\displaystyle \frac{\parallel \mathcal{X}-\widehat{\mathcal{X}}{\parallel }_F}{{\parallel \mathcal{X}\parallel }_F}}$, is plotted across 30 trials in Figure 4 (left). The relative error of the proposed algorithm is observed to be lower than that of the SiLRTC-TT algorithm, while the lowest error is noted for TT-WOPT. TMac-TT shows marginally lower accuracy than our approach in high-noise regimes, but marginally higher accuracy in low-noise regimes. TT-WOPT and TMac-TT perform (marginally) better because they explicitly aim to minimize the error. In comparison, the accuracy of the proposed algorithm is very good despite the fact that it only relies on standard NLA operations. The time required to compute the approximation is shown in Figure 4 (right), which shows that the proposed method is more than a magnitude faster, and the effect of the SNR on computation time is not significant.

4.1.2. Scalability

This experiment compares the scalability of the proposed approach with the reference algorithms. The reference algorithms are based on optimization and are terminated when the relative change in the function value falls below ${10}^{-7}$. If this level of precision is not achieved, the algorithms terminate upon reaching the maximum number of iterations (maximum iterations = $7\times {10}^2$). A 4th-order tensor in TT format $\mathcal{X}\in {\mathbb{R}}^{I\times I\times I\times I}$ is generated with a fixed ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)=(1,4,4,4,1)$, where I is varied from 10 to 50. Random Gaussian i.i.d. noise is added to the generated tensor to achieve an SNR of 25 dB, and 35% of the mode-4 fibers are randomly removed. Figure 5 (left) shows the median relative error computed over 25 trials and plotted as a function of I. The TT-WOPT algorithm is shown to achieve the highest accuracy, while a good but not optimal accuracy is achieved by our method. Notably, for the proposed approach, accuracy consistently improves as the problem size increases. This behaviour is expected because, for a fixed missing rate and TT rank, an increase in I results in relatively more observed data being used to estimate the model parameters (TT cores). A similar reasoning applies to the reference optimization methods; see [25,55] for more details. In Figure 5 (right), it is shown that the computation time for the proposed method is the lowest, in line with expectations. Specifically, in the current experimental settings, as I increases from 30 to 50, the time increases by a factor of $4.7$ in our approach, while the time increases by factors of $14.9,$$16.4$ and $57.3$ for TT-WOPT, TMac-TT and SiLRTC-TT algorithms, respectively.

4.2. Real-Life Applications

4.2.1. Multidimensional Harmonic Retrieval

Harmonic retrieval is a classical problem in signal processing. The goal is to estimate the parameters of a signal that is modelled as a sum of complex exponentials. MHR is the natural multidimensional extension. An MHR data tensor sampled over the $(D+1)$th-order tensor grid can be modelled as

$$x_{i_1{i}_2\cdots i_{d}k}=\sum _{r=1}^R{s}_r\left(k\right)\prod _{d=1}^D{\mathrm{e}}^{j\left(i_d-1\right){\mu }_r^{\left(d\right)}}+n_{i_1{i}_2\cdots i_{D}k},$$

(9)

where $j^2=-1$ and $s_r\left(k\right)$ is the kth complex symbol carried by the rth multidimensional harmonic. The noise $n_{i_1{i}_2\cdots i_{D}k}$ is modelled as zero-mean i.i.d additive Gaussian noise.

A 5th-order data tensor $\mathcal{X}\in {\mathbb{C}}^{10\times 10\times 10\times 10\times 25}$ is generated by a CPD model of rank $R=4$, using (9). The parameter D is set to 4, and binary phase shift keying sources $s_r\left(k\right)\in \{-1,1\}$ of length $K=25$ are used. The parameter vectors are set as follows: ${\mathit{\mu }}^{\left(1\right)}=[1,-0.5,0.1,-0.8]$, ${\mathit{\mu }}^{\left(2\right)}=[-0.5,1,-0.9,0.2]$, ${\mathit{\mu }}^{\left(3\right)}=[-0.2,-0.6,1.0,0.4]$, and ${\mathit{\mu }}^{\left(4\right)}=[-0.8,0.4,0.3,-0.1]$. The parameter settings and data generation are performed as described in [56]. In the first experiment, Gaussian noise is added to the generated tensor with SNR varying from −10 dB to 40 dB. Forty percent of fibers in mode-5 are randomly removed. The estimated TT approximations are subsequently used to compute the parameter vectors ${\widehat{\mathit{\mu }}}^{\left(d\right)}$ using the classical ESPRIT algorithm [57]. Root mean square error (RMSE), which is defined as $\mathrm{RMSE}=\sqrt{{\displaystyle \frac{1}{RD}}{\sum }_{r=1}^R{\sum }_{d=1}^D{\left({\mu }_r^{\left(d\right)}-{\widehat{\mu }}_r^{\left(d\right)}\right)}^2}$, is used to assess the accuracy. The results are summarized in the left column of Figure 6. Next, the SNR is fixed to 25 dB, and the effect of the missing rate on the accuracy is studied. The RMSE and computation time are plotted as a function of the missing fiber rate in the right column of Figure 6.

4.2.2. Spatiotemporal Weather Data Imputation

Spatiotemporal weather data are typically collected at fixed spatial coordinates, with measurements at each location varying over time. In practice, however, it is often not feasible to gather (or store) data at every location—especially when the spatial resolution is high. In such cases, time series data are recorded only for a subset of spatial coordinates across selected time windows. These data may be organized in a tensor of size—for example, “longitudes × latitudes × year × day of year”—with observations in a fiber-wise pattern. In this experiment, such a dataset, consisting of the maximum temperature time series (TMAX in °C) from the NASA POWER database (these data were obtained from the NASA Langley Research Center (LaRC) POWER Project funded through the NASA Earth Science/Applied Science Program: https://power.larc.nasa.gov) is used. The dataset comprises $5478$ daily observations from 1 January 2005 through 31 December 2019, and spans the region bounded by 4.0° $\mathrm{E}$ to 50.5° $\mathrm{E}$ longitude and 30.0° $\mathrm{N}$ to 54.5° $\mathrm{N}$ latitude, on a regular 0.5° × 0.5° grid. The code to download the dataset is provided in [31]. The data are reshaped into a 4th-order tensor, organized yearly, with each year comprising 366 days (day-of-year alignment is performed, with missing entries imputed via nearest neighbor averaging), resulting in a shape of $94\times 50\times 15\times 366$. We simulate a scenario in which time series data are observed only for a subset of spatial coordinates during specific years. This results in a spatiotemporal tensor whose fibers along the last mode are either fully observed or entirely missing.

In the first experiment, the mode-4 fibers of the data tensor are randomly removed, with the rate of missing fibers varying from $40\%$ to $65\%$. The approximation is computed using our method for different TT ranks, and the median relative error between the ground truth and the estimated tensor (i.e., the overall error, which includes both prediction error and reconstruction error) is recorded over 30 trials. In

Table 1. For a fixed missing rate, the completion accuracy improves with increasing TT rank. For a fixed TT rank, the error increases only slightly, showing that the method can achieve reasonably good approximations even at higher missing rates if the working conditions are met. If not (as in the highlighted case), the approximation becomes inaccurate.

		Rate of Missing Fibers (%)
${\mathbf{rank}}_{\mathit{TT}}\mathbf{\left(}\widehat{\mathcal{X}}\mathbf{\right)}$	40	45	50	55	60	65
(1, 10, 10, 14, 1)	0.1393	0.1395	0.1396	0.1396	0.1396	0.1397
(1, 18, 18, 22, 1)	0.1174	0.1176	0.1177	0.1180	0.1181	0.1180
(1, 26, 26, 30, 1)	0.1044	0.1044	0.1045	0.1046	0.1049	0.1050
(1, 34, 34, 38, 1)	0.0943	0.0944	0.0946	0.0947	0.0956	0.0956
(1, 42, 42, 46, 1)	0.0869	0.0869	0.0874	0.0873	0.0881	0.6304

, it is observed that the approximation is improved by increasing the TT rank. A reasonably good approximation is obtained even when up to 65% of the mode-4 fibers are completely missing. However, the error is observed to rise once the missing fibers exceed this rate. A sharp rise in relative error is noted when the approximation rank is increased under a high missing rate (see, e.g., the value highlighted in Table 1). With such a high TT rank (i.e.,${rank}_{TT}\left(\widehat{\mathcal{X}}\right)=(1,42,42,46,1)$), the overlapping conditions are no longer satisfied, resulting in the estimated approximation being rendered inaccurate; see Section 3.3. Nevertheless, a valid approximation can still be obtained if the TT rank is low enough to satisfy the recovery conditions.

Next, the rate of missing fibers is set to 50%, and mode-4 fibers are randomly removed. The approximation is computed using ${rank}_{TT}\left(\widehat{\mathcal{X}}\right)=(1,15,15,50,1)$, which is determined by analyzing the singular values of the matrix unfoldings. The median relative error across 10 trials is found to be below 9.7%. Figure 7 visualizes the observed, estimated, and residual values (i.e., the absolute differences between the ground truth and the estimates) for segments of the time series between 1 January 2013 and 12 December 2018 at four locations. It is observed that the maximum temperature (TMAX) exhibits a sinusoidal pattern, with peaks corresponding to summer temperatures and valleys corresponding to winter temperatures.

4.3. TT Approximation as a Prior for Efficient Computation

One can use our algebraic method as an initialization for optimization-based methods, improving computational speed and potentially reducing the risk of convergence to local minima, since the algebraic method—which works under deterministic conditions—yields a solution already close to the true one. Moreover, the solution obtained from the proposed method can also be directly used for downstream tasks, particularly in low-noise settings. In line with [39], our experiments show that the estimated TT approximation can also serve as a proxy for efficiently computing other tensor decompositions.

4.3.1. Initialization of Optimization Methods

In a noiseless setting, the proposed algorithm computes the exact TT decomposition when the uniqueness conditions are satisfied. However, in noisy settings, the approximation is good but not optimal; further refinement can be performed using optimization methods. This experiment compares TT-WOPT—run with different random initializations—to a hybrid approach in which the algebraic method serves as an initialization for TT-WOPT.

A 4th-order random tensor in TT format $\mathcal{X}\in {\mathbb{R}}^{40\times 40\times 40\times 40}$ is generated with a fixed ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)=(1,5,5,5,1)$. Random Gaussian i.i.d. noise is added to the generated tensor at SNR levels of 100 dB, 75 dB, 50 dB, and 25 dB to simulate low and moderate noise conditions. Sixty percent of the mode-4 fibers are randomly removed. Completion is then performed with two initialization strategies: one using 10 different random initializations, and a hybrid approach where the TT approximation computed by the algebraic algorithm is used as the initialization for (one run of) TT-WOPT. Figure 8 compares the number of iterations required to reach the same convergence criterion for the algebraic and random initialization strategies. Each dot indicates the number of iterations required to reach the convergence criterion in a single experiment. For the random strategy, each dot corresponds to the median number of iterations across 10 different initializations. A total of 100 experiments is conducted. The (median) accuracies achieved at SNR levels of 100 dB, 75 dB, 50 dB, and 25 dB are $4.51\times {10}^{-6}$, $3.27\times {10}^{-5}$, $2.54\times {10}^{-4}$, and $2.76\times {10}^{-3}$, respectively, for the hybrid strategy, and $1.28\times {10}^{-6}$, $3.01\times {10}^{-5}$, $2.52\times {10}^{-4}$, and $2.76\times {10}^{-3}$, respectively, for the random initialization strategy; thus, both methods achieve comparable accuracies for the considered SNR levels. Meanwhile, with algebraic initialization, the TT-WOPT method reaches this accuracy in significantly fewer iterations, as the algebraic initialization is already close to the optimal solution; see Figure 8 (left).

Next, we investigate the success of the two approaches in accurately estimating the underlying decomposition when the rate of missing fibers is high. A 4th-order random tensor in TT format $\mathcal{X}\in {\mathbb{R}}^{20\times 20\times 20\times 20}$ is generated with a fixed ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)=(1,4,4,4,1)$. Random Gaussian i.i.d. noise is added to the generated tensor at SNR levels of 75 dB, 50 dB, and 25 dB. Approximately 20% of the mode-4 fibers are sampled in such a way that the working conditions, as discussed in Section 3.3, on the observation pattern are satisfied. Completion of the tensor is achieved similarly to the method described in the previous experiment. Figure 9 shows the distribution of relative error across three different noise levels. Each dot represents one of 100 experiments, with medians calculated over 10 initializations for the random strategy. It is observed that the TT-WOPT becomes highly successful when initialized with the algebraic method. To assess the accuracy, we define thresholds for SNR values of 25 dB, 50 dB, and 75 dB as 0.015, 0.00125, and 0.000175, respectively. A completion is considered successful if the relative error is below the threshold associated with the specified noise level. Figure 10 shows the success rate at each noise level, defined as the proportion of 100 experiments that satisfy the accuracy criterion. The success rate of the algebraic strategy is defined as the fraction of successful trials, whereas the success rate of the random strategy is computed by first calculating the fraction of successful trials for each of the 10 initializations and then averaging these fractions to obtain the final success rate. Note that each noise level has its corresponding threshold, meaning comparisons are made within the same SNR level rather than across different levels. At an SNR of 75 dB, the algebraic strategy successfully completed 96 out of 100 experiments. In comparison, the random strategy has an average success rate of 76.2 out of 100 over 10 initializations. Overall, the algebraic strategy consistently demonstrated a higher success rate than the random strategy under different noise conditions.

4.3.2. Proxy for Non-Negative CPD

In this experiment, we focus on computing a non-negative CPD of a non-negative data tensor that is observed through mode-N fibers. The constrained CPD is computed using two strategies: the full strategy, where partial data in a dense format is directly used to compute the non-negative CPD, and the compressed strategy, where we first compute the TT approximation using our method and then use the compressed representation to compute the non-negative CPD. We use projected Gauss–Newton algorithm to compute the CPD (cpd_nls with nlsb_gndl solver), as discussed in [39].

A 4th-order tensor in CPD format $\mathcal{X}\in {\mathbb{R}}^{I\times I\times I\times I}$ is generated with a fixed ${\mathrm{rank}}_{CP}\left(\mathcal{X}\right)=4$ by sampling the factor matrices from a uniform distribution $\mathcal{U}(0,1)$. Random Gaussian i.i.d. noise is added to the generated tensor to achieve an SNR of 40 dB. Fifty percent of the mode-4 fibers are randomly removed from the noisy tensor. The constrained rank-4 CPD is computed using both the full and compressed strategies for I varying from 30 to 50. The median of the computation time is plotted across 50 trials in Figure 11 (left). It is observed that the use of the TT approximation as a prior significantly improves speed. The overall computation time in the compressed strategy, which includes the proxy step and the computation of the constrained CPD from the compressed tensor, remains lower than that of the full strategy. The small accuracy gap that arises in noisy settings can be eliminated by refining the proxy over a few iterations. Moreover, the computational cost of the proxy step can be further reduced by using parallel implementations or randomized SVD algorithms.

4.4. Note on Accuracy Gap Between the Algebraic and Optimization Methods

The observed accuracy gap between the algebraic and optimization methods stems not only from the algebraic method’s reliance solely on standard NLA operations—which makes it slightly less accurate in noisy cases—but also from the difference in the proportion of data points that each method uses. When 70% of the fibers along a specific mode are randomly removed, the optimization method uses all of the remaining 30% of the observations to compute the completion. In contrast, the algebraic method only uses the observed submatrices that satisfy the working conditions for recovering the column spaces of the matrix unfoldings, discarding the rest. Consequently, the algebraic method uses fewer than 30% of the observed fibers. Under deterministic sampling, where all 30% of the observed fibers satisfy the working conditions, the accuracy gap is expected to be small.

Another key point is that only the observed rows of the (individual) mode-2 slices, as in Equation (4), are used in the above experiments. The observed submatrices spanning multiple slices, satisfying the working conditions, can also be utilized. Let us consider an example of an observed submatrix spanning two mode-2 slices, ${\mathbf{S}}_r^{\left(l_1\right)}{\tilde{\mathbf{X}}}_{:l_1:}\in {\mathbb{R}}^{|{\alpha }_{l_1}|\times I_N}$ and ${\mathbf{S}}_r^{\left(l_2\right)}{\tilde{\mathbf{X}}}_{:l_2:}\in {\mathbb{R}}^{|{\alpha }_{l_2}|\times I_N}$, which have overlapping rows indexed by ${\alpha }_{l_1\cap l_2}={\alpha }_{l_1}\cap {\alpha }_{l_2}$, where $|{\alpha }_{l_1\cap l_2}|=K\ge R$. Then, the concatenated matrix ${\mathbf{S}}_r^{(l_1\cap l_2)}\left[{\tilde{\mathbf{X}}}_{:l_1:}{\tilde{\mathbf{X}}}_{:l_2:}\right]\in {\mathbb{R}}^{K\times 2I_N}$, where the row selection matrix ${\mathbf{S}}_r^{(l_1\cap l_2)}$ selects rows indexed by ${\alpha }_{l_1\cap l_2}$, can also serve as an additional observed submatrix in the subspace computation approaches discussed in Section 3.1.1 and Section 3.1.2. As a result, the column subspaces can be estimated more accurately by incorporating these additional submatrices. However, this approach comes with additional computational cost as it requires identifying the overlapping slices. Computing the SVDs of these larger matrices can also be relatively expensive, but the results are expected to be close to those obtained by optimization methods. In the remainder of this section, we demonstrate this through an experiment. We compute the (TT) approximation of an incomplete tensor observed fiber-wise using the algebraic algorithm with two different approaches: one that utilizes only individual slices and another that incorporates both individual mode-2 slices and the largest observed submatrix from each pair of slices.

A 4th-order tensor in TT format $\mathcal{X}\in {\mathbb{R}}^{16\times 16\times 16\times 16}$ is generated with a fixed ${\mathrm{rank}}_{TT}\left(\mathcal{X}\right)=(1,3,3,4,1)$. Random Gaussian i.i.d. noise is then added to the generated tensor with SNR varying from 0 dB to 45 dB. Sixty percent of the mode-4 fibers are randomly removed from the noisy tensor. Completion is then performed using the algebraic algorithm with the two strategies. Figure 12 shows that using the additional observed submatrices significantly improves accuracy, bringing it close to that of TT-WOPT. However, this approach is slightly slower due to the extra computational overhead, as mentioned previously. Overall, the computational cost remains low compared to TT-WOPT, while the accuracy gap is very small. It is worth noting that, in this experiment, we only combined pairs of slices; however, it may be possible to achieve an accuracy closer to TT-WOPT by selecting triplets or more slices.

5. Conclusions

We introduced an algebraic framework for computing the TT decomposition of an incomplete tensor observed fiber-wise (along a single specific mode). This framework builds on established algebraic techniques known for CPD and MLSVD [28,30]. The TT decomposition combines the features of MLSVD and CPD, making the extension of such algebraic methods in TT format highly consequential. The proposed approach relies solely on standard NLA operations and is fast, being guaranteed to work under reasonable deterministic conditions on the observation pattern.

We provided theoretical insights into piecewise subspace learning, an essential ingredient of our method, and discussed both the algebraic and generic uniqueness conditions for retrieving the TT cores (up to basis transformation).

Convincing numerical experiments demonstrate that our proposed approach is practical and valuable for real-life applications. When compared with state-of-the-art methods, our method has been observed to be fast, in line with expectations, while achieving competitive accuracy in recovering partially (fiber-wise) observed tensors. Moreover, our experiments show that the solution obtained from the proposed method can also serve as a proxy for efficient subsequent computations, including the initialization of optimization-based methods, which are typically more computationally expensive.