Tensor Train-Based Higher-Order Dynamic Mode Decomposition for Dynamical Systems

Abstract

Higher-order dynamic mode decomposition (HODMD) has proved to be an efficient tool for the analysis and prediction of complex dynamical systems described by data-driven models. In the present paper, we propose a realization of HODMD that is based on the low-rank tensor decomposition of potentially high-dimensional datasets. It is used to compute the HODMD modes and eigenvalues to effectively reduce the computational complexity of the problem. The proposed extension also provides a more efficient realization of the ordinary dynamic mode decomposition with the use of the tensor-train decomposition. The high efficiency of the tensor-train-based HODMD (TT-HODMD) is illustrated by a few examples, including forecasting the load of a power system, which provides comparisons between TT-HODMD and HODMD with respect to the computing time and accuracy. The developed algorithm can be effectively used for the prediction of high-dimensional dynamical systems.

1. Introduction

Dynamical systems appear in many applications in physics. The description of a complex dynamical system can be unfeasible because the physical model is too complicated. In this way, data-driven models based on the dynamic mode decomposition (DMD) [1] can be effectively used. Such models are entirely based on measured data (snapshots) obtained over time. With the use of DMD, key dynamic modes can be identified that drastically reduce the computational complexity of the problem and lead to a reduced-order model.

Higher-order dynamic mode decomposition (HODMD), which was first introduced by Le Clainche and Vega [2], is an extension of the dynamic mode decomposition. HODMD extends the application of the standard DMD to the data with the spatial complexity less than the spectral complexity. This occurs when the spatial dimension is less than the number of dynamical modes, for example, in the Lorenz system [3]. The efficiency of HODMD has been confirmed numerous times in the literature for different applications such as the study of flow dynamics from numerical and experimental data [4,5,6,7], filtering experimental noise from a limited number of snapshots [8,9] and prediction of load for power systems [10].

However, it is problematic to use HODMD for high-dimensional systems because of the curse of dimensionality when the computational complexity growths exponentially with the increase of dimension. As a result, only quite a limited amount of data can be analyzed. Some tensor formats have been proposed to mitigate this problem such as the canonical tensor format, the Tucker format, or the tensor-train format (TT-format). While the canonical format is structurally the simplest, it is well known that it is numerically unstable, as the approximation with a fixed canonical rank in the Frobenius norm can be ill-posed [11]. At the same time, computing the canonical rank is an NP-hard problem (non-deterministic polynomial–time problem) [12]. Furthermore, despite the Tucker format being numerically stable, it suffers from the so-called curse of dimensionality, as the size of the core tensor grows exponentially with the increase of the number of dimensions [13]. To overcome this problem, the tensor-train decomposition (TTD) [12] can be used. TTD can be interpreted as a multi-dimensional generalization of the singular value decomposition (SVD). As a result, TTD incorporates the advantages from both the canonical formats and Tucker formats and can be efficiently applied to high-dimensional problems. A tensor-train-based DMD has recently been proposed in [14] where it is demonstrated that the realization of DMD in the TT-format significantly increases the efficiency of the entire algorithm.

The main objective of this paper is to make the use of HODMD more efficient and feasible for high-dimensional dynamical systems. For this purpose, we realize HODMD in TT-format for the first time. The proposed algorithm can be applied, in particular, to DMD that leads to a modified TT-DMD algorithm.

This paper is organized as follows. Section 2 briefly describes the standard HODMD. Then, Section 2.2 introduces the TT-format first. Then, it is extended to HODMD with the TT-HODMD algorithm. Effectively, the proposed algorithm can be applied to DMD, which leads to an alternative realization of TT-DMD. In Section 3, numerical results for comparison between TT-HODMD and HODMD are presented. They are first applied to an analytical example for validation purposes. Then, a practical application is considered. In this example, HODMD is used for a long-term prediction of the electrical power load in a power grid. It is demonstrated that the developed technique is capable of tremendously reducing the computing time that makes the algorithm potentially valuable for a real-time control. Finally, the proposed realization of TT-DMD is compared against the algorithm presented in [14] with respect to the computing time. Finally, the conclusion is provided in Section 4.

2. Methods

In this section, we first introduce HODMD. Then, we use SVD to reduce data and effectively calculate HODMD modes and eigenvalues. Finally, we rewrite the algorithm with the use of the low-rank decomposition of potentially high-dimensional data.

The algorithm of HODMD presented in this section is explained in more detail in [2]. In contrast to [2], we introduce it as a temporary convolution.

2.1. High-Order Dynamic Mode Decomposition

Consider a dataset of K equispaced snapshots ${\mathit{v}}_k\in {\mathbb{R}}^J$ collected at time instant $t_k$ ($k=1,\dots ,K$). For further consideration, we suppose that in the general case, snapshots ${\mathit{v}}_k$ represent tensors of dimension m: ${\mathbf{T}}_k\in {\mathbb{R}}^{n_1\times \dots \times n_m},k=1,\dots ,K$, $J=n_1\times \dots \times n_m$. As such, vector ${\mathit{v}}_k$ is obtained via unfolding tensor ${\mathbf{T}}_k$.

The key DMD assumption is that there is a linear operator which allows us to predict each snapshot from the previous one. As noted by Mezić [15], this operator can be interpreted as a finite-dimensional approximation of the Koopman operator. As a rule, the DMD assumption is only valid to some extend, and to approximate the DMD operator, a large set of data is needed.

In contrast to DMD, HODMD is based on more series of snapshots in such a way that each snapshot is supposed to be related with d previous consequent snapshots via some linear operators:

$$\begin{array}{c}\hfill {\mathit{v}}_k\cong \sum _{l=1}^d{\mathit{R}}_l{v}_{k-l}={\mathit{R}}_1{v}_{k-1}+\sum _{l=2}^d{\mathit{R}}_l{v}_{k-l},\end{array}$$

(1)

where $1\le d\le K-1,k=d+1,\dots ,K$. It is worth noting that such an extension is especially important when the temporal complexity exceeds the spatial complexity.

It is clear that in case $d=1$, we obtain the standard DMD approach. It is interesting to note that Equation (1) represents a discrete convolution over time. It provides a relationship between the HODMD and Mori–Zwanzig formalism [16]. Thus, the first term represents the Markovian term and approximates the Koopman operator. At the same time, the sum in Equation (1) can be interpreted as the memory term with a kernel having a bounded support. The bounded support is justified since, as noted in [16], the kernel vanishes for large enough times. In the framework of the Koopman analysis, the kernel contains the contribution of unresolved observables over time. The omitted noise term on the right-hand side represents the contribution of unresolved dynamics. The Koopman operator is adjoint to the Perron–Frobenius operator [17] used for the analysis of dynamical systems. Thus, approximation (1) of the Koopman operator can be considered as an alternative approach to the approximation of the Perron–Frobenius operator [18].

Next, we split all snapshots onto $d+1$ overlapping matrices with $K-d$ elements each:

$$\begin{array}{c}\hfill {\mathit{V}}_d^{K-l}=[{\mathit{v}}_{d+1-l},\dots ,{\mathit{v}}_{K-l}],\end{array}$$

(2)

where $l=0,\dots ,d$. Then, we can rewrite Equation (1) as follows:

$$\begin{array}{c}\hfill {\mathit{V}}_d^K\cong \sum _{l=1}^d{\mathit{R}}_l{\mathit{V}}_d^{K-l}.\end{array}$$

(3)

To reduce the dimension of the data considered, we apply the singular value decomposition (SVD) to the snapshot matrix ${\mathit{V}}_d^K$, written in the SVD format as

$$\begin{array}{c}\hfill {\mathit{V}}_d^{K-1}\cong {\mathit{U}}_d{\Sigma }_d{\mathit{T}}_d^T={\mathit{U}}_d{\widehat{\mathit{V}}}_d^{K-1},\end{array}$$

(4)

where

$${\widehat{\mathit{V}}}_d^{K-1}={\Sigma }_d{\mathit{T}}_d^T(={\mathit{U}}_d^T{\mathit{V}}_d^{K-1}).$$

The diagonal of matrix ${\Sigma }_d$ contains the singular values ${\sigma }_1,{\sigma }_2,\dots ,{\sigma }_N$, ${\mathit{U}}_d\in {\mathbb{R}}^{J\times N}$. Here, we presume that matrices ${\mathit{U}}_d$ and ${\mathit{T}}_d$ are unitary and all singular values are numerated according to their diminishing: ${\sigma }_i\ge {\sigma }_{i+1}(i=1,2,\dots ,K-d-1)$. In addition, suppose that starting from some element $r_k<N$, the singular values can be truncated: $\sqrt{{\sigma }_{r_k+1}^2+{\sigma }_{r_k+2}^2+\dots +{\sigma }_N^2}\le {\epsilon }_0$, where ${\epsilon }_0>0$ is small enough. Thus, we introduce the threshold parameter ${\epsilon }_0$ to determine the error of approximation in the Frobenius norm in the case of truncation.

The orthonormal columns of $J\times N$ unitary matrix ${\mathit{U}}_d$ and $K\times N$ unitary matrix ${\mathit{T}}_d$ are the spatial and temporal SVD-modes, respectively. The rows of the $N\times (K-d)$ matrix ${\widehat{\mathit{V}}}_d^K$ are proportional to the SVD temporal modes and can be seen as rescaled temporal modes. Equation (3) implies that the snapshots ${\mathit{V}}_d^K$ can be projected onto the subspace of SVD spatial modes which can have a much lower dimension depending on N. In that space, we then consider snapshots ${\widehat{\mathit{V}}}_d^K$ from ${\mathbb{R}}^{N\times (K-d)}$.

Now, we can formulate the DMD-d algorithm for the reduced snapshot matrices as follows:

$$\begin{array}{c}\hfill {\widehat{\mathit{V}}}_d^K\cong \sum _{l=1}^d{\widehat{\mathit{R}}}_l{\widehat{\mathit{V}}}_d^{K-l},\end{array}$$

(5)

where ${\widehat{\mathit{V}}}_d^{K-l}\in {\mathbb{R}}^{N\times (K-d)}$, $\widehat{{\mathit{R}}^k}\in {\mathbb{R}}^{N\times N}$, ${\widehat{\mathit{V}}}_d^{K-l}={\mathit{U}}_d^T{\mathit{V}}_d^{K-l}$ and ${\widehat{\mathit{R}}}_k={\mathit{U}}_d^T{\mathit{R}}_k{\mathit{U}}_d$ for $k=1+d,\cdots ,K$.

Recurrently, from Equation (5), we obtain the following relationship

$$\begin{array}{c}\hfill {\tilde{\mathit{V}}}^K\cong \tilde{\mathit{R}}{\tilde{\mathit{V}}}^{K-1}\end{array}$$

(6)

between ${\tilde{\mathit{V}}}^K$ and ${\tilde{\mathit{V}}}^{K-1}$, where the reduced modified snapshot matrix ${\tilde{\mathit{V}}}^K$ is given by

$$\begin{array}{c}\hfill {\tilde{\mathit{V}}}^K\equiv \left[\begin{array}{c}{\widehat{\mathit{V}}}_d^{K-d+1}\\ {\widehat{\mathit{V}}}_d^{K-d+1}\\ \cdots \\ {\widehat{\mathit{V}}}_d^{K-2}\\ {\widehat{\mathit{V}}}_d^K\end{array}\right]\in {\mathbb{R}}^{\left(N·d\right)\times (K-d)}.\end{array}$$

(7)

In turn

$$\begin{array}{c}\hfill \tilde{\mathit{R}}\equiv \left[\begin{array}{cccccc}\mathbf{0}& \mathit{I}& \mathbf{0}& \cdots & \mathbf{0}& \mathbf{0}\\ \mathbf{0}& \mathbf{0}& \mathit{I}& \cdots & \mathbf{0}& \mathbf{0}\\ \cdots & \cdots & \cdots & \cdots & \cdots & \cdots \\ \mathbf{0}& \mathbf{0}& \mathbf{0}& \cdots & \mathit{I}& \mathbf{0}\\ {\widehat{\mathit{R}}}_d& {\widehat{\mathit{R}}}_{d-1}& {\widehat{\mathit{R}}}_{d-2}& \cdots & {\widehat{\mathit{R}}}_2& {\widehat{\mathit{R}}}_1\end{array}\right]\in {\mathbb{R}}^{N·d\times N·d}\end{array}$$

(8)

Here, matrices ${\tilde{\mathit{V}}}^K$ and ${\tilde{\mathit{V}}}^{K-1}$ are supposed to be known from measurements.

It is well known that the solution to the overdetermined system (6) in the Frobenius norm is obtained via the pseudo-inverse matrix of ${\tilde{\mathit{V}}}^{K-1}$:

$$\begin{array}{c}\hfill \tilde{\mathit{R}}{=\tilde{\mathit{V}}}^K\times {(\tilde{\mathit{V}}}^{K-1}{)}^{+},\end{array}$$

(9)

where

$${\left({\tilde{\mathit{V}}}^{K-1}\right)}^{+}=\mathit{V}{\Sigma }^{-1}{\mathit{U}}^T.$$

Then, its eigenvectors, called HODMD modes ${\mathit{q}}_m$ and eigenvalues ${\mu }_m$, can be immediately calculated. These eigenvalues coincide with the eigenvalues of the transition operator for the original snapshots:

$$\begin{array}{c}\hfill {\mathit{V}}^K\equiv \left[\begin{array}{c}{\mathit{V}}_d^{K-d+1}\\ {\mathit{V}}_d^{K-d+1}\\ \cdots \\ {\mathit{V}}_d^{K-2}\\ {\mathit{V}}_d^K\end{array}\right]\in {\mathbb{R}}^{\left(J·d\right)\times (K-d)}.\end{array}$$

(10)

Indeed, then

$$\begin{array}{c}\hfill {\mathit{V}}^K=\mathit{R}{\mathit{V}}^{K-1}\end{array}$$

(11)

and

$$\begin{array}{c}\hfill {\tilde{\mathit{V}}}^K=\left({\mathit{I}}_d⨂{\mathit{U}}_d^T\right){\mathit{V}}^K,\end{array}$$

(12)

where $\mathit{R}\in {\mathbb{R}}^{J·d\times J·d}$, ${\mathit{I}}_d\in {\mathbb{R}}^{d\times d}$ is a unit matrix. Hence

$$\begin{array}{c}\hfill \tilde{\mathit{R}}=\left({\mathit{I}}_d⨂{\mathit{U}}_d^T\right)\mathit{R}\left({\mathit{I}}_d⨂{\mathit{U}}_d\right).\end{array}$$

(13)

Thus, if $\lambda$ and $\tilde{\mathit{g}}$ are the eigenvalue and eigenvector of matrix $\tilde{\mathit{R}}$, respectively, then $\lambda$ is an eigenvalue of matrix $\mathit{R}$ and the appropriate eigenvector is given by

$$\begin{array}{c}\hfill \mathit{g}=\left({\mathit{I}}_d⨂{\mathit{U}}_d\right)\tilde{\mathit{g}}.\end{array}$$

(14)

More precisely, this is a DMD mode projected onto the space of snapshots by the previous time step while the exact mode can be calculated as [19]

$$\begin{array}{c}\hfill \mathit{g}=\frac{1}{\lambda }{\mathit{V}}^K\mathit{V}{\Sigma }^{-1}\tilde{\mathit{g}}.\end{array}$$

(15)

Then, the snapshots can be expanded via DMD modes:

$$\begin{array}{c}\hfill {\mathit{v}}_k\cong \sum _{m=1}^M{a}_m{\mathit{g}}_m{e}^{({\delta }_m+i{\omega }_m)(k-1)\Delta t},k=1,2,\dots ,K,\end{array}$$

(16)

where ${\lambda }_m=e^{({\delta }_m+i{\omega }_m)}\Delta t$, ${\delta }_m$ is a growth mode, ${\omega }_m$ is a frequency, and M is the number of retained DMD modes that represents the spectral complexity.

The amplitudes $a_m(m=1,\dots ,M)$ can be determined from fitting the snapshots ${\mathit{v}}_k(k=1,\dots ,K)$ in the Frobenius norm. In this way, it is more efficient to consider the reduced snapshots ${\widehat{\mathit{v}}}_k={\mathit{U}}_d^T{\mathit{v}}^k(k=1,\dots ,K)$ [2].

Then, we arrive at equation

$$\mathit{L}\mathit{a}=\mathit{b},$$

(17)

where

$$\mathit{L}=\left[\begin{array}{c}\mathit{Q}\\ \mathit{Q}\mathit{M}\\ \cdots \\ \mathit{Q}{\mathit{M}}^{K-1}\end{array}\right],\mathit{a}=\left[\begin{array}{c}{a}_1\\ a_2\\ \cdots \\ a_M\end{array}\right],\mathit{b}=\left[\begin{array}{c}{\mathit{b}}_1\\ {\mathit{b}}_2\\ \cdots \\ {\mathit{b}}_M\end{array}\right].$$

Equation (17) can be immediately solved via the pseudo-inverse matrix to $\mathit{L}$ calculated via SVD.

2.2. Proposed Algorithm

In this subsection, we first describe the tensor-train format (TT-format) and then show how HODMD can be realized with TTD. The aim is to make the application of HODMD more efficient to analyze dynamical systems of high dimension.

2.2.1. TT-Format

Over the last years, different tensor formats such as the canonical format, the Tucker format, and the TT-format have been developed, see e.g., [20,21,22,23,24]. The TT-format is not affected by the curse of dimensionality; in addition, it provides a very efficient way to use basic linear algebra and rounding operations when the data are formatted [12].

The TTD is introduced in detail in [12]. Here, we repeat the main algorithm. Consider a data array presented as tensor $\mathit{X}\in {\mathbb{R}}^{n_1\times \dots \times n_{m+1}}$. First, we can split the dimension $n_1\times \dots \times n_{m+1}$ into two parts: $x_1=n_1$ and $x_2=n_2\times \dots \times n_{m+1}$. Then, we are able to reshape $\mathit{X}$ into a matrix based on $x_1$ and $x_2$, which is marked as $\mathit{B}={\left.\mathit{X}\right|}_{x_1}^{x_2}$. This process is called the matricization. After the matricization, it is possible to compute SVD for this reshaped matrix as follows:

$$\begin{array}{c}\hfill {\mathit{M}}_1={\left.\mathit{X}\right|}_{r_0,n_1}^{n_2,\dots ,n_{m+1}}\cong {\mathit{U}}_1{\Sigma }_1{\mathit{V}}_1^T,\end{array}$$

(18)

where ${\mathit{U}}_1{\in \mathbb{R}}^{(r_0\times n_1)\times r_1}$ and ${\Sigma }_1{\mathit{V}}_1^T{\in \mathbb{R}}^{r_1\times (n_2,\dots ,n_{m+1})}$, $r_0=1$. Here, $r_0$ is formally introduced to retain a uniform approach.

In the second approximate equality, we presume that all singular values exceed some threshold ${\epsilon }_1$ and their number is equal to $r_1$. Thus, ${\epsilon }_1$ determines the tensor rank. Next, matrix ${\Sigma }_1{\mathit{V}}_1^T$ is reshaped into matrix

$$\begin{array}{c}\hfill {\mathit{M}}_2={\Sigma }_1{\mathit{V}}_1^T{|}_{r_1,n_2}^{n_3,\dots ,n_{m+1}}\cong {\mathit{U}}_2{\Sigma }_2{\mathit{V}}_2^T,\end{array}$$

(19)

for which threshold ${\epsilon }_2$ is introduced that determines rank $r_2$ of the SVD decomposition for ${\mathit{M}}_2$. Thus, ${\mathit{U}}_2{\in \mathbb{R}}^{(r_1\times n_2)\times r_2}$ and ${\Sigma }_2{\mathit{V}}_2^T{\in \mathbb{R}}^{r_2\times (n_3,\dots ,n_{m+1})}$. This procedure is repeated until the final index $n_{m+1}$.

In this way, the first TT-core ${\mathit{G}}^{\left(1\right)}$ is obtained by reshaping matrix ${\mathit{U}}_1$ into a tensor with dimension $r_0\times n_1\times r_1$. In turn, the second TT-core ${\mathit{G}}^{\left(2\right)}$ is determined by reshaping matrix ${\mathit{U}}_2$ into a tensor with dimension $r_1\times n_2\times r_2$. Finally, the TT-decomposition of $\mathit{X}$ can be presented in the following form

$$\begin{array}{c}\hfill \mathit{X}(i_1,i_2,\dots ,i_{m+1})={\mathit{G}}^{\left(1\right)}\left(i_1\right){\mathit{G}}^{\left(2\right)}\left(i_2\right)\dots {\mathit{G}}^{(m+1)}\left(i_{m+1}\right).\end{array}$$

(20)

Thus, the original tensor can be approximated and reconstructed via a series of multilinear products between the cores. Formally, each core represents a tensor with the order of three. At the same time, the order of the first and last cores is actually reduced to two.

Figure 1 depicts the TT-decomposition of a 4-dimensional tensor (inspired by [25]). A tensor $\mathit{X}$ of size $i_1\times i_2\times i_3\times i_4$ can be decomposed into four parts. Four rows in the figure represent four types of expressions: (from top to bottom) regular, kernel-linked, fibres, and slices. All four types of expressions represent a schematic description of the TT-decomposition. It looks like a train with carriages and links between them, and that justifies the name of the algorithm [12].

2.2.2. Tensor-Train-Based HODMD

In this section, we introduce the TT-based HODMD algorithm. It is novel not only in application to HODMD but also if applied to DMD when $d=1$.

Next, we fold vectors ${\mathit{v}}_k(k=1,\dots ,K)$ as tensors $\mathit{T}\in {\mathbb{R}}^{n_1\times n_2\times \dots \times n_m\times K}$.

Then, we decompose each tensor with TT-SVD:

$$\begin{array}{cc}\hfill & \mathit{T}(:,\dots ,:,k)=\hfill \\ \hfill & \sum _{l_0=1}^{r_0}\cdots \sum _{l_m=1}^{r_m}{\mathbf{x}}_{l_0,:,l_1}^{\left(1\right)}\otimes {\mathbf{x}}_{l_1,:,l_2}^{\left(2\right)}\otimes \dots \otimes {\mathbf{x}}_{l_{m-1},:,l_m}^{\left(m\right)}{\mathbf{x}}_{l_m}^{(m+1)}\left(k\right)\hfill \end{array}$$

(21)

We presume tensor $\underline{\mathit{T}}\in {\mathbb{R}}^{n_1\times n_2\times \dots \times n_m\times l_m}$:

$$\begin{array}{c}\hfill \underline{\mathit{T}}={\mathbf{x}}^{\left(1\right)}\otimes \dots \otimes {\mathbf{x}}^{\left(m\right)}\end{array}$$

(22)

is left–orthonormal: ${\underline{\mathit{T}}}^T\underline{\mathit{T}}=\mathit{I}$. Any tensor train can be orthogonalized [12]. Therefore, our assumption does not violate the generality.

Thus,

$$\begin{array}{c}\hfill {\mathit{v}}_k=\sum _{l_m=1}^{r_m}\mathit{B}(:,l_m)\mathit{R}(l_m,k)=\sum _{l_m=1}^{r_m}\mathit{B}(:,l_m){\mathit{t}}_k\left(l_m\right),\end{array}$$

(23)

where

$$\mathit{B}=\underline{\mathit{T}}{|}_{n_1{n}_2\dots n_m=J}^{l_m}$$

is a unitary matrix: ${\mathit{B}}^T\mathit{B}=\mathit{I}$.

Multiplying Equation (1) by ${\mathit{B}}^T$, we arrive at equation:

$$\begin{array}{c}\hfill {\mathit{t}}_k\cong \sum _{l=1}^d{\widehat{\mathit{R}}}_l{\mathit{t}}_{k-l},k=d+1,\dots ,K,\end{array}$$

(24)

where ${\widehat{\mathit{R}}}_l={\mathit{B}}^T{\mathit{R}}_l\mathit{B}$.

Thus, we effectively reduce the analysis to the evolution of the temporary modes ${\mathit{t}}_k\in {\mathbb{R}}^{l_m}$. As soon as the temporary modes are obtained, the full vectors of snapshots $v_k$ can be immediately restored via Equation (23):

$$\begin{array}{c}\hfill {\mathit{v}}_k=\sum _{l_m=1}^{r_m}\mathit{B}(:,l_m){\mathit{t}}_k\left(l_m\right).\end{array}$$

Next, we can apply the HODMD analysis described in Section 2 immediately.

3. Results and Discussion

Accurate power load forecasting has important practical value and significance for ensuring the economic and safe operation of the power service sector [26]. In the energy interconnection system, with the development of communication technology and the application of measurement equipment, relevant departments obtain large amounts of information data, which can be exploited for improving the performance of load forecasting models.

For the power sector, to better formulate grid dispatch plans and optimize the daily power generation plan of conventional power sources, short-term load forecasting (STLF) is often used to obtain electrical loads for 1 to 3 days (1 to 72 h). According to the technical characteristics of STLF, it can be roughly divided into statistical models, neural network models and combined forecasting models. The forecasting model should be reasonably selected according to the specific forecasting environment and the characteristics of the input data [27,28]. Among them, HODMD-based STLF techniques have gradually attracted attention due to their good properties, which do not depend on any given dynamic system expression [10]. At the same time, the data obtained in the power system have obvious big data “4V” characteristics [29] (i.e., high Veracity, high Variety, high Velocity and high Value), and it is attractive to use TT-HODMD to achieve more efficient short-term forecasting.

In the case of this section, we realize the load forecasting of the power system by generating a short-term forecasting model based on HODMD and TT-HODMD. First, we verify the proposed TT-HODMD algorithm. Subsequently, we apply both HODMD and TT-HODMD for short-term forecasting through a simple power system example. Finally, we present the analysis of the obtained results and comparison of alternative algorithms.

In the numerical experiments, the codes are developed in Spyder (Python 3.7) and the calculations are implemented on a MacOS machine with Intel Core i5 CPU (2.4 GHz), 8 GB RAM and 4 cores.

3.1. Validation of TT-HODMD

We use a test dataset to validate the TT-HODMD code developed. This dataset is obtained from function f which is created by the following function:

$$\begin{array}{cc}\hfill & f(x,t)={\mathrm{e}}^{-x^2/5}\cos \left(2x\right)\sqrt{1+\frac{1}{2}\sin t},\hfill \\ \hfill & x\in [-3,3],t\in [0,4\pi ].\hfill \end{array}$$

(25)

The temporal part of the test function is taken from a test case presented in [2]. In this test case, the spatial complexity is always supposed to be less than the spectral complexity such that DMD is not immediately applicable.

Next, we introduce uniform meshing for x with 400 nodes and consider 1000 equispaced snapshots. The function $f(x,t)$ is shown in Figure 2a, where the horizontal coordinate is x, the vertical coordinate corresponds to t and the shade of color represents the value of f. In Figure 2b, the horizontal coordinate corresponds to the snapshots whilst the vertical coordinate is $f(x_i,t)(i=1,\cdots ,400)$. For the test case, the data generated via the function $f(x,t)$ are formally folded in a $20\times 20\times 1000$ array.

As an example, we compare the results against all snapshots at the $10\mathrm{th}$ row: $f(x_{10},t_k)k=1,\cdots ,K$. The intercepted part is presented by the solid line in Figure 3.

Next, we assume $d=300$, which brings us to a modified dataset, further called $f^{{}^{\prime }}$, with the size of $[400\ast 300]\times \left[\right(1000-300\left)\right]$ (see Equation (10) for details).

From Section 2, ${\epsilon }_0$ is considered as a threshold parameter for HODMD and TT-HODMD. The thresholds ${\epsilon }_i(i=1,2,\dots )$ are supposed to be equal to ${\epsilon }_0={10}^{-7}$.

Figure 3 shows the comparison of reconstructions with TT-HODMD $(d=1)$ and TT-HODMD $(d=300)$. It is clear that the algorithm with $d=1$ fails to reconstruct the dataset. In this case, TT-DMD $(d=1)$ does not reproduce the number of modes sufficient for reconstruction. By contrast, the TT-HODMD with $d=300$ is capable of reconstructing the function f. A relatively high value of d provides a high degree of delay-embedding without excessive loss of influence of the later snapshots.

Next, with $d=300$, we select the $100\mathrm{th}$ row of the dataset: $f(x_{100},t_k)k=1,\dots ,K$. Figure 4 demonstrates the sensitivity of the reconstruction to the threshold ${\epsilon }_0$. As expected with the decrease of ${\epsilon }_0$, TT-HODMD provides the reconstruction closer to the result with HODMD thanks to the decrease of TT thresholds ${\epsilon }_i(i=1,2,\cdots )$.

The results for prediction obtained for different d for ${\epsilon }_0={10}^{-7}$ are presented in Figure 5.

As a quantitative measure, we use RMSE (root mean square error) to estimate the deviation between the predicted and real values:

$$\begin{array}{c}\hfill RMSE(\mathit{y},\widehat{\mathit{y}})=\sqrt{\frac{{\sum }_{i=0}^{N-1}{(y_i-{\widehat{y}}_i)}^2}{N}}\end{array}$$

(26)

where $\mathit{y}$ is the vector of predicted values and $\widehat{y}$ represents the real values. The comparison of RMSE with different d values is shown as

Table 1. RMSE of different d.

	RMSE
$d=1$	$2977\times {10}^{-5}$
$d=20$	$334.3\times {10}^{-5}$
$d=100$	$107.5\times {10}^{-5}$
$d=300$	$1.394\times {10}^{-5}$

. As can be seen, the prediction with $d=300$ is very close to the exact result. In turn, the standard DMD with $d=1$ fails. As noted in [2], an optimal d essentially depends on K.

3.2. Realization of STLF in Power System Based on TT-HODMD

A real STLF simulation analysis in a power system is presented in this section. The data are taken from the UK National Grid Electricity System Operator [30]. In this test case, we select three years $(2017,2018,2019)$ of historic demand data for simulation. We choose 12 months of data per year, 20 days of data per month, 24 h of data per day, and then the data are measured every 15 minutes (resulting in 96 data per day). In this way, we obtain three-dimensional data of $3\left(\mathrm{years}\right)\times 12\left(\mathrm{months}\right)\times 1920\left(\mathrm{data}\right)$, where the last figure represents the number of units per month. On the basis of three-year load data, we predict the power load for the next six months starting from January 2020. In this way, we compare TT-HODMD and HODMD with respect to the accuracy, robustness and computing time.

The tensor-based data model is more useful for analyzing and processing data than the traditional time series. The differences between the tensor-based data and traditional time-series-based data are presented in

Table 2. The difference between tensor-based data and time-series-based data.

	Tensor-Based Data	Time-Series-Based Data
Data structure	Effective structure protection for high-dimensional data	Only the structure of one-dimensional data is well protected, and for high-dimensional data only the unfolding process is possible
Data meaning	Each dimension of high-dimensional data has a corresponding meaning and does not facilitate unfolding	It is mainly the temporal arrangement of the data in advance that is of relatively simple significance
Data processing	Can handle high-dimensional data immediately	High-dimensional data must be unfolded before they can be processed, significantly reducing efficiency
Data efficiency	Provides a compact form	Unfolding can lead to matrices of a very high dimension

in terms of data structure, data meaning and data processing, respectively.

As discussed in Section 2.2.2, we only consider the temporary modes $T_k$. The results obtained by TT-HODMD with $d=1,d=400\mathrm{and}{\epsilon }_0={10}^{-5}$ are shown in Figure 6. As can be seen, TT-HODMD with $d=400$, which is around 20% of the number of snapshots, provides a much better reconstruction and prediction.

It is worth noting that HODMD allows us to effectively analyze the stability of a power system. Figure 7 demonstrates the DMD eigenvalues ${\lambda }_j$ from Equation (16) obtained with $d=400$. These eigenvalues correspond to the Koopman transition operator [31] and determine the stability of the power system in study. As can be seen from Figure 7, the considered power system is stable since there are no modes with the eigenvalues outside the unit circle. The modes with the eigenvalues situated inside the unit circle vanish over time, whilst the others retain stable.

The key comparison of TT-HODMD and HODMD is related to the runtime. It is shown in Figure 8 for different thresholds ${\epsilon }_0$. As can be seen, the threshold ${\epsilon }_0$ has much greater impact on HODMD, whilst it is very well mitigated with TT-HODMD. Thus, with the increase of the accuracy of prediction, TT-HODMD has a tremendous advantage.

Finally, in Figure 9, we compare the proposed algorithm for TT-DMD against the approach from [14] for different thresholds. A principal difference between the two approaches is that in our algorithm, we effectively consider only temporary modes that significantly improve the efficiency. It is clear that this effect becomes more significant with the increase of the number of modes. In turn, the number of modes involved is determined by the threshold. As a result, with the decrease of the threshold, the proposed approach essentially overcomes the algorithm from [14].

4. Conclusions

In this paper, a new approach to the tensor-train realization of dynamic mode decomposition has been proposed. It is effectively reduced to the computation of temporary modes in an orthogonal basis. The algorithm has been implemented to realize the High-Order DMD in the TT-format for the first time and applied to the forecasting of load in a power system. It is demonstrated that in comparison to DMD, HODMD has a significant advantage in its capability of reconstructing and predicting load data. In contrast to the original HODMD, the developed TT-HODMD algorithm has very little sensitivity to the value of the threshold with respect to the runtime. With the increase of the accuracy of prediction, TT-HODMD operates much faster than HODMD. In addition, it is shown that the proposed approach to TT-DMD is essentially faster than the existing algorithm already known in the literature. The developed algorithm can be effectively applied to real-life problems such as the prediction of the load in a power system.