On Numerical Approximations of the Koopman Operator

Abstract

We study numerical approaches to computation of spectral properties of composition operators. We provide a characterization of Koopman Modes in Banach spaces using Generalized Laplace Analysis. We cast the Dynamic Mode Decomposition-type methods in the context of Finite Section theory of infinite dimensional operators, and provide an example of a mixing map for which the finite section method fails. Under assumptions on the underlying dynamics, we provide the first result on the convergence rate under sample size increase in the finite-section approximation. We study the error in the Krylov subspace version of the finite section method and prove convergence in pseudospectral sense for operators with pure point spectrum. Since Krylov sequence-based approximations can mitigate the curse of dimensionality, this result indicates that they may also have low spectral error without an exponential-in-dimension increase in the number of functions needed.

1. Introduction

Spectral theory of dynamical systems shifts the focus of investigation of dynamical systems behavior away from trajectories in the state space and towards spectral features of an associated infinite-dimensional linear operator. Of particular interest is the composition operator—in a measure-preserving setting called the Koopman operator [1,2,3,4,5]. Its spectral triple—eigenvalues, eigenfunctions and eigenmodes—can be used in a variety of contexts, from model reduction [5] to stability and control [6]. In practice, we only have access to finite-dimensional data from observations or outputs of numerical simulations. Thus, it is important to study approximation properties of finite-dimensional numerical algorithms devised to compute spectral objects [7]. Compactness is the property that imbues infinite-dimensional operators with quasi-finite-dimensional properties. Self-adjointness also helps in proving the approximation results. However, the composition operators under study here are rarely compact or self-adjoint. In addition, in the classical, measure-preserving case, the setting is that of unitary operators (and essentially self-adjoint generators for the continuous-time setting [8]), but in the general, dissipative case, composition operators are neither.

There are three main approaches to finding spectral objects of the Koopman operator:

1.

The first, suggested already in [9] is based on long time weighted averages over trajectories, rooted in ergodic theory of measure-preserving dynamical systems. An extension of that work that captures properties of continuous spectrum was presented in [10]. This approach was named Generalized Laplace Analysis (GLA) in [11], where concepts pertaining to dissipative systems were discussed also in terms of weighted averages along trajectories. In that sense, the ideas in this context provide an extension of ergodic theory for capturing transient (off-attractor) properties of systems. For on-attractor evolution, the properties of the method acting on $L^2$ functions were studied in [4]. The off-attractor case was pursued in [4,12] where Fourier averages (which are Laplace averages for the case when the eigenvalue considered is on the imaginary axis) were used to compute the eigenfunction whose level sets are isochrons, and [13] in which analysis for general eigenvalue distributions was pursued in Hardy-type spaces. This study was continued in [14] to construct dynamics-adapted Hilbert spaces. The advantage of the method is that it does not require the approximation of the operator itself, as it constructs eigenfunctions and eigenmodes directly from the data. In this sense, it is close (and in fact related) to the power method of approximating the spectrum of a matrix from data on iteration of a vector. In fact, the methodology extends the power method to the case when eigenvalues can be of magnitude 1. It requires separate computation to first determine the spectrum of the operator, which is also done without constructing it. This can potentially be hard to do because of issues such as spectral pollution—see remarks at the end of Section 3; also note the general long-standing problems of spectral pollution and computing the full spectrum of Schrödinger operators on a lattice were recently solved in [15]. The recent work [16] enables computation of full spectral measures using the combination of resolvent operator techniques (used for the first time in the Koopman operator context in [17]) and ResDMD—an extension of Dynamic Mode Decomposition (introduced next) technique that incorporates computation of residues from data snapshots (computation of residues was considered earlier in [18]).

2.

The second approach requires construction of an approximate operator acting on a finite-dimensional function subspace i.e., a finite section—the problem that is also of concern in a more general context of approximating infinite dimensional operators [7,19,20]. The best known such method is the Dynamic Mode Decomposition (DMD), invented in [21] and connected to the Koopman operator in [22]. It has a number of extensions (many of which are summarized in [23]), for example, Exact DMD [24]; Bayesian/subspace DMD [25]; Optimized DMD [26,27]; Recursive DMD [28]; Variational DMD [29]; DMD with control [30,31]; sparsity promoting DMD [32]; DMD for noisy systems [33,34,35]. The original DMD algorithm featured state observables. The Extended Dynamic Mode Decomposition [36] recognizes that nonlinear functions of state might be necessary to describe a finite-dimensional invariant subset of the Koopman operator and provides an algorithm for finite-section approximation of the Koopman operator. A study of convergence of such approximations is provided in [37], but the convergence was established only along subsequences, and the rate of convergence was not addressed. Here, we provide the first result on the rate of convergence of the finite section approximation under assumptions on the nature of the underlying dynamics. In addition, spectral convergence along subsequences is proven in [37] under the assumption of the weak limit of eigenfunction approximations not being zero. This condition is hard to verify in practice. Instead, in Section 5.2, we prove a result that obviates the weak convergence assumption using some additional information on the underlying dynamics. It was observed already in [9] that, instead of an arbitrary set of observables forming a basis, one can use observables generated by the dynamics—namely time delays of a single observable filling a Krylov subspace—to study spectral properties of the Koopman operator. In the DMD context, the methods developed in this direction are known under the name Hankel-DMD [38,39]. It is worth noticing that the Hankel matrix approach of [38] is in fact based on the Prony approximation and requests a different sample structure than the Dynamic Mode Decomposition. Computation of residues was considered in [18] to address the problem of spectral pollution, where discretization introduces spurious eigenvalues. As mentioned before, the recent work [16] provides another method to resolve the spectral pollution problem, introducing ResDMD—an extension of Dynamic Mode Decomposition that incorporates computation of residues from data snapshots. The relationship between GLA and finite section methods was studied in [40].

3.

The third approach is based on the kernel integral operator combined with the Krylov subspace methodology [41], enabling approximation of continuous spectrum. While GLA and EDMD techniques have been extended to dissipative systems, the kernel integral operator technique is currently available only for measure-preserving (on-attractor) systems.

In this paper, we continue with the development of ergodic theory-rooted ideas to understanding and numerically computing the spectral triple—eigenvalues, eigenfunctions and modes—for the Koopman operator. After some preliminaries, we start in Section 3 with discussing properties of algorithms of Generalized Laplace Analysis type in Banach spaces. Such results have previously been obtained in Hardy-type spaces [13], and here, we introduce a Gel’fand-formula-based technique that allows us to expand to general Banach spaces. We continue in Section 4 with setting the finite-section approximation of the Koopman operator in the ergodic theory context. An explicit relationship of finite section coefficients to dual basis is established. Under assumptions on the underlying dynamics, we provide the first result on the convergence rate under sample size increase in the finite-section approximation. The error in the finite section approximation is analyzed. In Section 5, we study finite section approximations of the Koopman operator based on Krylov sequences of time-delays of observables, and prove that under certain conditions, the approximation error decreases as the number of samples is increased, without dependence on the dimension of the problem. Namely, the Krylov subspace (Hankel-DMD) methodology has the advantage of convergence in the number of iterates and does not require a basis exponentially large in the number of dimensions. This solves the problem of the choice of observables, since the dynamics selects the basis by itself. In Section 6, we discuss an alternative point of view on the DMD approximations which is not related to finite sections, but samples of continuous functions on finite subsets of the state-space. The concept of weak eigenfunctions is discussed, continuing the analysis in [37]. We conclude in Section 7.

2. Preliminaries

For a Lipshitz-continuous (ensuring global existence and uniqueness of solutions) dynamical system

$$\dot{\mathbf{x}}=\mathbf{F}\left(\mathbf{x}\right),$$

(1)

defined on a manifold $M\in {\mathbb{R}}^m$ (i.e., $\mathbf{x}\in M$—where we by slight abuse of notation identify a point in a manifold M with its vector representation $\mathbf{x}$ in ${\mathbb{R}}^m$), where $\mathbf{x}$ is a vector and $\mathbf{F}$ is a possibly nonlinear vector-valued smooth function, of the same dimension as its argument $\mathbf{x}$, denote by ${\mathbf{S}}^t\left({\mathbf{x}}_0\right)$ the position at time t of trajectory of (1) that starts at time 0 at point ${\mathbf{x}}_0$. We call the family of functions ${\mathbf{S}}^t$ the flow.

Denote by $\mathbf{f}$ an arbitrary, vector-valued observable $\mathbf{f}:M\to {\mathbb{R}}^k$. The value $\mathbf{f}(t,{\mathbf{x}}_0)$ of the observable $\mathbf{f}$ that the system trajectory starting from ${\mathbf{x}}_0$ at time 0 sees at time t is

$$\mathbf{f}(t,{\mathbf{x}}_0)=\mathbf{f}\left({\mathbf{S}}^t\left({\mathbf{x}}_0\right)\right).$$

(2)

Note that the space of all observables $\mathbf{f}$ is a linear vector space. The family of operators $U^t,$ acting on the space of observables parameterized by time t is defined by

$$U^t\mathbf{f}\left({\mathbf{x}}_0\right)=\mathbf{f}\left({\mathbf{S}}^t\left({\mathbf{x}}_0\right)\right).$$

(3)

Thus, for a fixed time $\tau$, $U^{\tau }$ maps the vector-valued observable $\mathbf{f}\left({\mathbf{x}}_0\right)$ to $\mathbf{f}(\tau ,{\mathbf{x}}_0)$. We will call the family of operators $U^t$ indexed by time t the Koopman operator of the continuous-time system (1). This family was defined for the first time in [1], for Hamiltonian systems. In operator theory, such operators, when defined for general dynamical systems, are often called composition operators, since $U^t$ acts on observables by composing them with the mapping ${\mathbf{S}}^t$ [3]. Discretization of ${\mathbf{S}}^t$ for times $\tau ,2\tau ,\dots ,n\tau ,\dots$ leads to the $\tau$-mapping $T={\mathbf{S}}^{\tau }:M\to M$ with the discrete dynamics

$${\mathbf{x}}^{\prime }=T\mathbf{x},$$

(4)

and the associated Koopman operator U defined by

$$Uf=f\circ T.$$

(5)

Let $\mathcal{F}$ be a space of observables and $U:\mathcal{F}\to \mathcal{F}$ the Koopman operator associated with a map T (note this means that $f\circ T\in \mathcal{F}$ if $f\in \mathcal{F}$). Appropriate (dynamics-adapted) spaces are discussed in [14]. A function ${\varphi }_{\lambda }\in \mathcal{F}$ is an eigenfunction of U associated with eigenvalue $\lambda$ provided

$$U{\varphi }_{\lambda }=\lambda {\varphi }_{\lambda }.$$

(6)

Let $\sigma \left(U\right)\subset \mathbb{C}$ be the spectrum of U. The operator U is called scalar [42] on $\mathcal{F}$ provided

$$U={\int }_{\sigma \left(U\right)}\beta dE\left(\beta \right),$$

(7)

where E is a family of spectral projections forming resolution of the identity, and the integral is over $\beta \in \sigma \left(U\right)\subset \mathbb{C}$. Further, the operator U is called spectral provided

$$U=S+N,$$

(8)

where S is scalar and N quasi-nilpotent. Examples of functional spaces in which Koopman operators are scalar and spectral are given in [14]. Let $\mathbf{f}\in \mathcal{F}$ be a vector of observables. For a scalar operator U, the Koopman mode ${\mathbf{s}}_{\lambda }$ of $\mathbf{f}$ associated with an eigenvalue $\lambda$ of algebraic multiplicity 1 is given by

$${\mathbf{s}}_{\lambda }{\varphi }_{\lambda }={\mathbf{f}}_{\lambda },$$

(9)

where ${\varphi }_{\lambda }$ is the unit norm eigenfunction associated with $\lambda$, and

$${\mathbf{f}}_{\lambda }=\mathbf{f}-{\int }_{\sigma \left(U\right)/\left\{\lambda \right\}}\beta dE\left(\beta \right)\mathbf{f}={\int }_{\left\{\lambda \right\}}\beta dE\left(\beta \right)\mathbf{f}$$

(10)

Note that, denoting by $E_{\lambda }$ the projection on the eigenspace associated with the eigenvalue $\lambda$, we have

$$E_{\lambda }{\mathbf{f}}_{\lambda }=E_{\lambda }\mathbf{f},$$

(11)

since $E_{\lambda }{f}_{\lambda }=f_{\lambda }$ and $E_{\lambda }\circ {\int }_{\sigma \left(U\right)/\left\{\lambda \right\}}\beta dE\left(\beta \right)\left(\mathbf{f}\right)=0,$ by one of the key properties of the spectral resolution [42]. Now,

$$E_{\lambda }\mathbf{f}=\mathbf{c}{\varphi }_{\lambda },$$

(12)

for some constant $\mathbf{c}$, proving that ${\mathbf{s}}_{\lambda }$ is well-defined and independent of $\mathbf{x}$.

Remark 1. 

Note that in the more general case with algebraic multiplicities of eigenvalues larger than 1, an analogous definition of the Koopman mode can be obtained. For example, if algebraic multiplicity and geometric multiplicity are 2 and there are two linearly independent eigenfunctions ${\varphi }_{\lambda }^1$ and ${\varphi }_{\lambda }^2$ associated with the eigenvalue λ of multiplicity 2, and we are computing ${\mathbf{s}}_{\lambda }^1$, then (10) contains an additional term on the RHS, $-{\mathbf{s}}_{\lambda }^2{\varphi }_{\lambda }^2$, and similarly for ${\mathbf{s}}_{\lambda }^2$, forming 2 equations. In the case of spectral operators, one works similarly, but the added complexity is in the use of generalized eigenfunctions [14].

We assume that the dynamical system T has a Milnor attractor $\mathcal{A}$ such that for every continuous function g, for almost every $\mathbf{x}\in M$ with respect to an a priori measure $\nu$ on M (without loss of generality as we can replace M with the basin of attraction of $\mathcal{A}$) the limit

$$g^{*}\left(\mathbf{x}\right)=\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{U}^{i}g\left(\mathbf{x}\right),$$

(13)

exists. This is the case, e.g., for smooth systems on subsets of ${\mathbb{R}}^n$ with Sinai–Bowen–Ruelle measures, where $\nu$ is the Lebesgue measure [43]. For such systems, Hilbert spaces on which the Koopman operator is spectral have been constructed in [14].

3. Generalized Laplace Analysis

An example of what we call Generalized Laplace Analysis (GLA) is the computation of eigenspace at 0 (namely, invariants) of dynamical systems using time averages: recall

$$h^{*}\left(\mathbf{x}\right)=\underset{t\to \infty }{lim}\frac{1}{t}{\int }_0^{t}h\left(S^{\tau }\left(\mathbf{x}\right)\right)d\tau$$

(14)

is the time average at initial condition $\mathbf{x}$ of the function h under the dynamics of $S^t$. For fixed point attractors

$$U^t{h}^{*}\left(\mathbf{x}\right)=1·h^{*}\left(\mathbf{x}\right)$$

(15)

As shown previously, this is valid in a much larger context: limit cycle attractors, toroidal attractors, Milnor attractors, and measure-preserving systems.

We generalize the idea that averages along trajectories produce eigenfunctions, by introducing weights:

$$h^{*}\left(\mathbf{x}\right)=\underset{t\to \infty }{lim}\frac{1}{t}{\int }_0^{t}a\left(\tau \right)h\left(S^{\tau }\left(\mathbf{x}\right)\right)d\tau \approx \frac{1}{n}\sum _{j=0}^{n_j}a(j\Delta \tau )h\left(S^{j\Delta \tau }\left(\mathbf{x}\right)\right)$$

(16)

where $a\left(t\right)$ is a function of time—typically a (possibly complex) exponential, and $\Delta \tau$ is a sampling time interval. If we have a vectorized set of initial conditions ${\mathbf{x}}_k,k=1,\dots ,n_k$, then we can generate a data matrix

$$H_{jk}=h\left(S^{j\Delta \tau }\left({\mathbf{x}}_k\right)\right)$$

(17)

Vectorizing ${\left(\mathbf{a}\right)}_j=a(j\Delta \tau )$, we get

$${\mathbf{h}}_a^{*}=H\mathbf{a}.$$

(18)

H is the data matrix. For $a\left(t\right)=1=e^{0·t}$, we get ${\mathbf{h}}_a^{*}=H\mathbb{1}$, where $\mathbb{1}$ is a vector of 1’s with $n_j$ components. To obtain eigenfunctions using Fourier averages, as developed in [12], we set $a\left(t\right)=1=e^{-i\omega t}$, to obtain

$$h_{e^{-i\omega t}}^{*}\left(\mathbf{x}\right)=\underset{t\to \infty }{lim}\frac{1}{t}{\int }_0^t{e}^{-i\omega \tau }h\left(S^{\tau }\left(\mathbf{x}\right)\right)d\tau \approx \frac{1}{n}\sum _{j=0}^{n_j}{e}^{i\omega j\Delta \tau }h\left(S^{j\Delta \tau }\left(\mathbf{x}\right)\right)$$

(19)

Both of the above examples were for the case when $\left|a\right(t\left)\right|=1$ corresponding to eigenvalues $0,i\omega$, both on the imaginary axis. In the next subsection, we provide a general theorem that deals with eigenvalues distributed arbitrarily in the complex plane.

GLA for Fields of Observables

Many of the problems of interest in applications feature a distributed field of observables. For example, time evolution of temperature in a linear rod described by the coordinate $z\in [0,1]$, is $T(t,T_0,z)$, where $T_0\left(z\right)$ is the initial condition that belongs to the state space of all possible temperature distributions satisfying the boundary conditions, and t is time. We will set our analysis up having this example in mind—namely, we consider a field of observables $f(\mathbf{x},\mathbf{z})$, where $\mathbf{x}$ is in state space, and $\mathbf{z}$ is an indexing set—and consider the time evolution of such observables starting from an initial condition $\mathbf{x}$.

Let $f(\mathbf{x},\mathbf{z})$ be a bounded field of observables $f(\mathbf{x},\mathbf{z}):M\times A\to {\mathbb{R}}^m$, continuous in $\mathbf{x}$, where the observables are indexed over elements $\mathbf{z}$ of a set A, and M is a compact metric space. We will occasionally drop the dependence on the state-space variable $\mathbf{x}$ and denote $f(\mathbf{x},\mathbf{z})=f\left(\mathbf{z}\right)$ and the iterates of f by $f(T^i\mathbf{x},\mathbf{z})=f^i\left(\mathbf{z}\right)$. Let U be the Koopman operator associated with a map $T:M\to M$. We assume that U is bounded, and acting in a closed manner on a Banach space of continuous functions C (this does not have to be the space of all continuous functions on M, see the remark after the theorem).

Theorem 1. 

(Generalized Laplace Analysis). Let ${\lambda }_0,\dots ,{\lambda }_K$ be simple eigenvalues of U such that $|{\lambda }_0|\ge |{\lambda }_1|\ge \dots \ge |{\lambda }_K|>0,$ and there are no other points λ in the spectrum of U with $\left|\lambda \right|\ge |{\lambda }_K|$. Let ${\varphi }_k$ be the eigenfunction of U associated with ${\lambda }_k,k\in \{0,\dots ,K\}$. Then, the Koopman mode associated with ${\lambda }_k$ is obtained by computing

$$\begin{array}{ccc}\hfill f_k& =& \underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{\lambda }_k^{-i}\left(f(T^i\mathbf{x},\mathbf{z})-\sum _{j=0}^{k-1}{\lambda }_j^i{\varphi }_j\left(\mathbf{x}\right)s_j\left(\mathbf{z}\right)\right)\hfill \\ & =& \underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{\lambda }_k^{-i}\left(f^i\left(\mathbf{z}\right)-\sum _{j=0}^{k-1}{\lambda }_j^i{f}_j\right)\hfill \end{array}$$

(20)

where $f_k={\varphi }_k\left(\mathbf{x}\right)s_k\left(\mathbf{z}\right)$, ${\varphi }_k$ is an eigenfunction of U with $|{\varphi }_k|=1$ and $s_k$ is the k-th Koopman mode.

Proof. 

We introduce the operator

$$U_{{\lambda }_0}={\lambda }_0^{-1}U.$$

(21)

Then, for some function $g\left(\mathbf{x}\right),$ consider

$$\begin{array}{ccc}\hfill U\left(\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{U}_{{\lambda }_0}^{i}g\left(\mathbf{x}\right)\right)& =& \underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{\lambda }_0^{-i}{U}^{i}g\left(T\mathbf{x}\right)\hfill \\ & =& \underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{\lambda }_0^{-i}{U}^{i+1}g\left(\mathbf{x}\right)\hfill \\ & =& {\lambda }_0\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{\lambda }_0^{-(i+1)}{U}^{i+1}g\left(\mathbf{x}\right)\hfill \\ & =& {\lambda }_0\left[\underset{n\to \infty }{lim}\frac{1}{n}\left({\lambda }_0^{-n}g\left(T^n\mathbf{x}\right)-g\left(\mathbf{x}\right)+\sum _{i=0}^{n-1}{U}_{{\lambda }_0}^{i}g\left(\mathbf{x}\right)\right)\right]\hfill \\ & =& {\lambda }_0\left[\underset{n\to \infty }{lim}\frac{1}{n}\left({\lambda }_0^{-n}g\left(T^n\mathbf{x}\right)+\sum _{i=0}^{n-1}{U}_{{\lambda }_0}^{i}g\left(\mathbf{x}\right)\right)\right],\hfill \end{array}$$

(22)

where the last line is obtained by boundedness of g. Due to the boundedness of U and continuity of g, we have

$$\underset{n\to \infty }{lim}|{\lambda }_0^{-n}{U}^{n}g|\le |g|.$$

(23)

This is obtained as the consequence of the so-called Gel’fand formula that states that for a bounded operator V on a Banach space X, ${lim}_{n\to \infty }{\left|V^n\right|}^{1/n}=\rho$ where $\rho$ is the spectral radius of V [44] (note that in our case $\rho =|{\lambda }_0|$). Thus, the first term in (22) vanishes in the limit. Denoting

$$g_{{\lambda }_0}^{*}\left(\mathbf{x}\right)=\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{U}_{{\lambda }_0}^{i}g\left(\mathbf{x}\right),$$

(24)

where the convergence is again obtained from the Gel’fand formula, utilizing the assumption on convergence of time averages and (23). Thus, we obtain

$$U{g}_{{\lambda }_0}^{*}\left(\mathbf{x}\right)={\lambda }_0{g}_{{\lambda }_0}^{*}\left(\mathbf{x}\right)$$

(25)

and, thus, $g_{{\lambda }_0}^{*}\left(\mathbf{x}\right)$ is an eigenfunction of U at eigenvalue ${\lambda }_0$ (note that $g_{{\lambda }_0}^{*}\left(\mathbf{x}\right)\in \mathbb{C}$ by the fact that partial sums form a Cauchy sequence). If we have a field of observables $f(\mathbf{x},\mathbf{z})$, parameterized by $\mathbf{z}$, we get

$$f_{{\lambda }_0}^{*}(\mathbf{x},\mathbf{z})={\varphi }_k\left(\mathbf{x}\right)s_j\left(\mathbf{z}\right),$$

(26)

since $f_{{\lambda }_0}^{*}(\mathbf{x},\mathbf{z})$ is an eigenfunction of U at eigenvalue ${\lambda }_0$, so for every $\mathbf{z}$, it is just a constant (depending on $\mathbf{z}$) multiple of the eigenfunction ${\varphi }_k\left(\mathbf{x}\right)$ of norm 1. If we denote

$$P_{{\lambda }_0}=\underset{n\to \infty }{lim}\frac{1}{n}\sum _{i=0}^{n-1}{U}_{{\lambda }_0}^i,$$

(27)

(note $P_{{\lambda }_0}$ is a bounded projection operator), we can split the space of functions C into the direct sum $P_{{\lambda }_0}C⨁(I-P_{{\lambda }_0})C$.

Now, let $0<k<K$. Consider the space of observables

$$(I-P_{{\lambda }_0,{\lambda }_1,\dots ,{\lambda }_{k-1}})C=(I-\sum _{j=0}^{k-1}{P}_{{\lambda }_j})C,$$

(28)

complementary to the subspace $\mathsf{\Phi }$ spanned by ${\varphi }_j,0\le j<k$. The operator ${U|}_{\mathsf{\Phi }}$, the restriction of U to $\mathsf{\Phi }$ has eigenvalues ${\lambda }_0,\dots ,{\lambda }_{k-1}$. Since

$$g_k=g-P_{{\lambda }_0,{\lambda }_1,\dots ,{\lambda }_{k-1}}g$$

(29)

does not have a component in $\mathsf{\Phi }$, we can reduce the space of observables to $(I-P_{{\lambda }_0,{\lambda }_1,\dots ,{\lambda }_{k-1}})C$, on which $U_{{\lambda }_k}$ satisfies the assumptions of the theorem, and obtain

$$U{\left(g_k\right)}_{{\lambda }_k}^{*}\left(\mathbf{x}\right)={\lambda }_k{\left(g_k\right)}_{{\lambda }_k}^{*}\left(\mathbf{x}\right).$$

(30)

If we have a field of observables $f(\mathbf{x},\mathbf{z})$, then

$$f_k(\mathbf{x},\mathbf{z})=f(\mathbf{x},\mathbf{z})-P_{{\lambda }_0,{\lambda }_1,\dots ,{\lambda }_{k-1}}f,$$

(31)

and, thus,

$$f_k(\mathbf{x},\mathbf{z})={\varphi }_k\left(\mathbf{x}\right)s_k\left(\mathbf{z}\right).$$

(32)

□

In other words, $f_k$ is the skew-projection of the field of observables $f(\mathbf{x},\mathbf{z})$ on the eigenspace of the Koopman operator associated with the eigenvalue ${\lambda }_k$.

Remark 2.

The assumptions on eigenvalues in the above theorem would not be satisfied for dynamical systems whose eigenvalues are dense on the unit circle (e.g., a map that, as $n\to \infty$ approaches a unit circle in the complex plane on which the dynamics is given by $z^{\prime }=e^{i\omega }z$, where ω is irrational w.r.t. π). However, in such a case, the space of functions can be restricted to the span of functions $e^{ik\theta },k=1,\dots ,N,\theta \in [0,2\pi )$, and the requirements of the theorem would be satisfied. This amounts to restricting the observables to a set with finite resolution, which is standard in data analysis.

Remark 3.

Function spaces in which Koopman operators are spectral are typically special tensor products of on-attractor Hilbert spaces—for example, $L^2\left(\mu \right)$ where μ is the physical invariant measure—and off-attractor spaces of functions that are continuous or possess additional smoothness [14]. Provided we do not restrict the on-attractor part to a finite-dimensional subset like we did in the previous remark, the above theorem would apply to the off-attractor subset (which is an ideal set of functions that vanish a.e. on the attractor). However, the on-attractor Koopman modes can be obtained a.e. using the same procedure as above, and results relying on the Birkhoff’s Ergodic Theorem, valid in $L^2\left(\mu \right)$, as in [4,5,45,46].

In principle, one can find the full spectrum of the Koopman operator by performing Generalized Laplace Analysis, where Theorem 1 is used on some function $g\left(\mathbf{x}\right)$ starting from the unit circle, successively subtracting parts of the signal corresponding to eigenvalues with decreasing $\left|\lambda \right|$. In practice, such computation can be unstable, since at large t, it involves a multiplication of a very large with a very small number. In addition, the eigenvalues are typically not known a priori. A large class of dynamical systems have eigenvalues on and inside the unit circle (or left half of the complex plane inclusive of the imaginary axis in the continuous time case) [14]. The eigenvalues on the unit circle can be found using the Fast Fourier Transform (FFT). Once the contributions to the dynamics from those eigenvalues are subtracted, the next largest set of eigenvalues have magnitude less than 1. Thus, the power method would enable finding the magnitude $|{\lambda }_1|$ of the resulting eigenvalue. Scaling the operator (restricted to the space of functions not containing components from eigenspaces corresponding to eigenvalues of magnitude 1) with that magnitude, FFT can be performed again to identify the arguments of the eigenvalues of magnitude $|{\lambda }_1|$. Alternatively, as shown in the next section, we describe the finite section method, in which the operator is represented in a basis, and a finite-dimensional truncation of the resulting infinite matrix—a finite section—is used to approximate its spectral properties. Under some conditions [37], increasing the dimension of the finite section and the number of sample points, eigenvalues of the operator can be obtained.

4. The Finite Section Method

The GLA method for approximating eigenfunctions (and thus modes) of the Koopman operator, analyzed in the previous section, was proposed initially in [4,5,9] in the context of on-attractor (measure-preserving) dynamics, and extended to off-attractor dynamics in [11,12,13,39,47]. It is predicated on the knowledge of (approximate) eigenvalues—since the eigenvalues need to be known a priori to be able to perform weighted trajectory sums in (20). There is always the eigenvalue 1 that is known, and the trajectory sums in that case lead to invariants of the dynamics [45,46]. Other eigenvalues with modulus 1 can be approximated using signal processing methods (see e.g., [39]). Importantly, the GLA does not require the knowledge of an approximation to the Koopman operator and is in effect a sampling method which avoids the curse of dimensionality. In contrast, DMD-type methods, invented initially in [21] without the Koopman operator background, and connected to the Koopman operator setting in [22] produce a matrix approximation to the Koopman operator. There are many forms of the DMD methodology, but all of them require a choice of a finite set of observables that span a subspace. In this section, we analyze such methods in the context of finite section of the operator and explore connections to the dual basis.

4.1. Finite Section and the Dual Basis

Consider the Koopman operator acting on an observable space $\mathcal{F}$ of functions on the state space M, equipped with the complex inner product $〈·,·〉$ (note that we are using the complex inner product linear in the first argument here; the physics literature typically employs the so-called Dirac notation, where the inner product is linear in its second argument) and let $\left\{f_j\right\},j\in \mathbb{N}$ be an orthonormal basis on $\mathcal{F}$, such that, for any function $f\in \mathcal{F}$, we have

$$f=\sum _{j\in \mathbb{N}}{c}_j{f}_j.$$

(33)

Let

$$u_{kj}=〈U{f}_j,f_k〉.$$

(34)

Then,

$${\left(Uf\right)}_k=〈Uf,f_k〉=\sum _{j\in \mathbb{N}}{c}_j〈U{f}_j,f_k〉=\sum _{j\in \mathbb{N}}{u}_{kj}{c}_j.$$

(35)

Consider the (not necessarily orthogonal) unconditional basis $\left\{f_j\right\}$. The action of U on an individual basis function $f_j$ is given by

$$U{f}_j=\sum _{k\in \mathbb{N}}{u}_{kj}{f}_k,$$

(36)

where $u_{kj}$ are now just coefficients of $U{f}_j$ in the basis. We obtain

$$Uf=\sum _{j\in \mathbb{N}}{c}_{j}U{f}_j=\sum _{j\in \mathbb{N}}{c}_j\sum _{k\in \mathbb{N}}{u}_{kj}{f}_k=\sum _{k\in \mathbb{N}}\left(\sum _{j\in \mathbb{N}}{u}_{kj}{c}_j\right)f_k,$$

(37)

and we again have

$${\left(Uf\right)}_k=\sum _{j\in \mathbb{N}}{u}_{kj}{c}_j.$$

(38)

As in the previous section, associated with any closed linear subspace $\mathcal{G}$ of $\mathcal{F}$, there is a projection onto it, denoted $P=P^2$, that we can think of as projection “along” the space $(I-P)\mathcal{F}$, since, for any $f\in \mathcal{F}$, we have

$$P(I-P)f=(P-P^2)f=0,$$

(39)

and, thus, any element of $(I-P)\mathcal{F}$ has projection 0. We denote by $\tilde{U}$ the infinite-dimensional matrix with elements $u_{kj},k,j\in \mathbb{N}$. Thus, the finite-dimensional section of the matrix

$${\tilde{U}}_n=\left[\begin{array}{ccccc}{u}_{11}& u_{12}& \cdots & & u_{1n}\\ u_{21}& u_{22}& & & u_{2n}\\ & & & & \\ ⋮& & \ddots & & ⋮\\ u_{n1}& u_{n2}& \cdots & & u_{nn}\end{array}\right],$$

(40)

is the so-called compression of $\tilde{U}$ that satisfies

$${\tilde{U}}_n=P_n\tilde{U}{P}_n,$$

(41)

where $P_n$ is the projection “along” $(I-P_n)\mathcal{F}$ to the span of the first n basis functions, $span\left(f_1,\dots ,f_n\right)$.

The key question now is: how are the eigenvalues of ${\tilde{U}}_n$ related to the spectrum of the infinite-dimensional operator U? This was first addressed in [37].

Example 1.

Consider the translation T on the circle $S^1$ given by

$$z^{\prime }=e^{i\omega }z,z\in S^1,$$

(42)

Let $f_j=e^{ij\theta },\theta \in [0,2\pi ).$ Then,

$$U{f}_j=f_j\circ T=e^{ij\omega }{e}^{ij\theta }.$$

(43)

Thus, from (34) $u_{kj}={\delta }_{kj}{e}^{ij\omega }$, where ${\delta }_{kj}=1$ for $k=j$ and zero otherwise (the Kronecker delta), and $\tilde{U}$ is a diagonal matrix. In this case, the finite section method provides us with the subset of the exact eigenvalues of the Koopman operator.

The following example shows how careful we need to be with the finite-section method when the underlying dynamical system has chaotic behavior:

Example 2.

Consider the map T on the circle $S^1$ given by

$$z^{\prime }=z^2,z\in S^1,$$

(44)

This is a mixing map that does not have any eigenvalues of the Koopman operator on $L^2\left(S^1\right)$ except for the (trivial) 1, while its spectrum is the whole unit circle [48]. Let $f_j=e^{ij\theta },\theta \in [0,2\pi ).$ Then,

$$U{f}_j=f_j\circ T=e^{ij2\theta }.$$

(45)

Let

$$f\left(\theta \right)=\sum _{j\in \mathbb{Z}}{c}_j{e}^{ij\theta }.$$

(46)

Then,

$$Uf\left(\theta \right)=\sum _{j\in \mathbb{Z}}{c}_j{e}^{i2j\theta }.$$

(47)

Thus, ${\tilde{U}}_n$ is given by

$${\tilde{U}}_n=\left[\begin{array}{ccccc}0& 0& 0& 0& \cdots \\ 1& 0& 0& 0& \cdots \\ 0& 0& 0& 0& \cdots \\ 0& 1& 0& 0& \cdots \\ & & & & \\ ⋮& & \ddots & & ⋮\\ 0& 0& \cdots & & 0\end{array}\right],$$

(48)

provided $n\ne 2k,k\in \mathbb{N}$. In this case, the finite section method fails, as $U_n$ has eigenvalue 0 of multiplicity n. This example illustrates how the condition in [37] that the weak convergence of a subsequence of eigenfunctions of ${\tilde{U}}_N$ to a function ϕ must be accompanied by the requirement $\left|\right|\varphi \left|\right|\ne 0$ in order that the limit of the associated subsequence of eigenvalues converges to a true eigenvalue of the Koopman operator. In particular, no subsequence of eigenvalues in this case converges to the true eigenvalue of the Koopman operator, since the map is measure preserving, and thus, its eigenvalues are on the unit circle. The example shows the peril of applying the finite section method to find eigenvalues of the Koopman operator when the underlying dynamical system has a continuous spectral part [5] (in this case, Lebesgue [48]) spectrum. Continuous spectrum is effectively dealt with in [10,49] using harmonic analysis and periodic approximation methods, respectively.

To apply the finite-section methodology of approximation of the Koopman operator, we need to estimate the coefficients $u_{kj}$ from data. If we have access to measurements of N orthogonal functions $f_1,\dots ,f_N$ on m points on state space, as indicated in [37], assuming ergodicity, this becomes possible:

Theorem 2.

Let $\{f_1,\dots ,f_N\}$ be an orthogonal set of functions in $L^2(M,\mu )$ and let T be ergodic on M with respect to an invariant measure μ. Let ${\mathbf{x}}_l,l\in \mathbb{N}$ be a trajectory on M. Then, for almost any ${\mathbf{x}}_1\in M$

$$u_{kj}=\underset{m\to \infty }{lim}\frac{1}{m}\sum _{l=1}^m{f}_k^c\left({\mathbf{x}}_l\right)f_j\circ T\left({\mathbf{x}}_l\right)=\underset{m\to \infty }{lim}\frac{1}{m}\sum _{l=1}^m{f}_k^c\left({\mathbf{x}}_l\right)f_j\left({\mathbf{x}}_{l+1}\right)$$

(49)

Proof. 

This is a simple consequence of the Birkhoff ergodic theorem ([50]). Recall that

$$u_{kj}=〈U{f}_j,f_k〉={\int }_{M}U{f}_j{f}_k^{c}d\mu ,$$

(50)

and the last expression is equal to

$$\underset{m\to \infty }{lim}\frac{1}{m}\sum _{l=1}^m{f}_k^c\left({\mathbf{x}}_l\right)f_j\circ T\left({\mathbf{x}}_l\right),$$

(51)

by the Birkhoff Ergodic Theorem applied to the function $U{f}_j{f}_k^c$. □

In the case of non-orthonormal Riesz basis, denote by ${\widehat{f}}_k$ the dual basis vectors, such that

$$〈f_j,{\widehat{f}}_k〉={\delta }_{jk},$$

(52)

where ${\delta }_{jj}=1$ for any j, and ${\delta }_{jk}=0$ if $j\ne k$. For the infinite-dimensional Koopman matrix coefficients, we get

$$u_{kj}=〈U{f}_j,{\widehat{f}}_k〉.$$

(53)

Let us consider the finite set of independent functions $\tilde{\mathbf{f}}=\{f_1,\dots ,f_N\}$ and the associated dual set $\{{\widehat{g}}_1,\dots ,{\widehat{g}}_N\}$ in the span $\tilde{\mathcal{F}}$ of $\tilde{\mathbf{f}}$, that satisfy

$$〈f_j,{\widehat{g}}_k〉={\delta }_{jk}.$$

(54)

Note that the functions ${\widehat{g}}_k$ are unique, since they are each orthonormal to $n-1$ vectors in $\tilde{\mathcal{F}}$. Let

$$\mathcal{F}=\tilde{\mathcal{F}}+{\tilde{\mathcal{F}}}^T,$$

(55)

and $P_{\tilde{\mathcal{F}}}$ the orthogonal projection on $\tilde{\mathcal{F}}$ (this in effect assumes all the remaining basis functions are orthogonal to $\tilde{\mathcal{F}}$). Then,

$${\widehat{g}}_k=P_{\tilde{\mathcal{F}}}{\widehat{f}}_k,$$

(56)

since, by self-adjointness of orthogonal projections, and $P_{\tilde{\mathcal{F}}}{\widehat{f}}_k\in \tilde{\mathcal{F}}$

$$〈f_j,P_{\tilde{\mathcal{F}}}{\widehat{f}}_k〉=〈P_{\tilde{\mathcal{F}}}{f}_j,{\widehat{f}}_k〉=〈f_j,{\widehat{f}}_k〉={\delta }_{jk}$$

(57)

Now, we have

$${\tilde{u}}_{kj}=〈U{f}_j,{\widehat{g}}_k〉=〈U{f}_j,P_{\tilde{\mathcal{F}}}{\widehat{f}}_k〉=〈P_{\tilde{\mathcal{F}}}U{f}_j,{\widehat{f}}_k〉$$

(58)

and thus, since $f_j\in \mathcal{F}$, the coefficients ${\tilde{u}}_{kj}$ are the elements of the finite section $P_{\tilde{\mathcal{F}}}U{P}_{\tilde{\mathcal{F}}}$ in the basis $\tilde{\mathbf{f}}$. It is again possible to obtain ${\tilde{u}}_{kj}$ from data:

Theorem 3. 

Let $\{f_1,\dots ,f_N\}$ be a non-orthogonal set of functions in $L^2(M,\mu )$ and let T be ergodic on M with respect to an invariant measure μ. Let ${\mathbf{x}}_l,l\in \mathbb{N}$ be a trajectory on M. Then, for almost any ${\mathbf{x}}_1\in M$

$${\tilde{u}}_{kj}=\underset{m\to \infty }{lim}\frac{1}{m}\sum _{l=1}^m{f}_j\circ T\left({\mathbf{x}}_l\right){\widehat{g}}_k^c\left({\mathbf{x}}_l\right)=\underset{m\to \infty }{lim}\frac{1}{m}\sum _{l=1}^m{f}_j\left({\mathbf{x}}_{l+1}\right){\widehat{g}}_k^c\left({\mathbf{x}}_l\right),$$

(59)

where, for any finite m, ${\widehat{g}}_k^c\left({\mathbf{x}}_l\right)/m,l=1,\dots ,m$ are obtained as rows of the matrix ${\left(F^{†}F\right)}^{-1}{F}^{†}$, where

$$F=\left[f_1\left(\mathbf{X}\right)f_2\left(\mathbf{X}\right)\dots f_N\left(\mathbf{X}\right)\right],$$

(60)

$F^{†}={\left(F^c\right)}^T$ is the conjugate (Hermitian) transpose of F , and $f_j\left(\mathbf{X}\right)$ is the column vector ${(f_j\left({\mathbf{x}}_1\right)\dots f_j\left({\mathbf{x}}_m\right))}^T$.

Proof. 

The fact that ${\widehat{g}}_k^c\left({\mathbf{x}}_l\right)/m,l=1,\dots ,m$ are obtained as rows of the matrix ${\left(F^{†}F\right)}^{-1}{F}^{†}$ follows from

$${\left(F^{†}F\right)}^{-1}{F}^{†}F=I_N$$

(61)

where $I_N$ is the $N\times N$ identity matrix. The rest of the proof is analogous to the proof of Theorem 2. □

Remark 4.

The key idea in the above results—Theorems 2 and 3—is that we sample the functions $f_i,i=1,\dots ,N$ and the dual basis $g_k,k=1,\dots ,N$ on m points in the state space, and then take the limit $m\to \infty$. Thus, besides approximating the action of U using the finite section ${\tilde{U}}_N$, we also approximate individual functions $f_j,g_k$ by their sample on m points. The corollary of the theorems is that the finite sample approximations ${\tilde{U}}_{N,m}$, obtained by setting the coefficients

$${\tilde{u}}_{kj,m}=\frac{1}{m}\sum _{l=1}^m{f}_j\circ T\left({\mathbf{x}}_l\right){\widehat{g}}_k^c\left({\mathbf{x}}_l\right)$$

(62)

converges to ${\tilde{U}}_N$ as $m\to \infty$. This result has been obtained in [51], without the use of the dual basis, relying on the Moore–Penrose pseudoinverse, the connection which we discuss next.

We call F the data matrix. Note that the matrix $F^{+}={\left(F^{†}F\right)}^{-1}{F}^{†}$ is the so-called Moore–Penrose pseudoinverse of F. Using matrix notation, from (59), the approximation of the finite section can be written as

$${\tilde{U}}_N^a=F^{+}F\left(T\left(\mathbf{X}\right)\right)=F^{+}{F}^{\prime },$$

(63)

where $\mathbf{X}={({\mathbf{x}}_1,\dots ,{\mathbf{x}}_m)}^T$,

$$F^{\prime }=F\left(T\left(\mathbf{X}\right)\right)=\left[\begin{array}{c}{f}_1\left(T\mathbf{X}\right)f_2\left(T\mathbf{X}\right)\dots f_N\left(T\mathbf{X}\right)\end{array}\right].$$

(64)

and $f_k\left(T\mathbf{X}\right)$ is the column vector

$$\left[\begin{array}{c}{f}_k\left(T{\mathbf{x}}_1\right)\\ ⋮\\ f_k\left(T{\mathbf{x}}_m\right)\end{array}\right].$$

(65)

If we now assume that there is an eigenfunction-eigenvalue pair $\lambda ,\varphi$ of U such that $\varphi \in span\tilde{\mathcal{F}}$, then

$$P_{\tilde{\mathcal{F}}}U{P}_{\tilde{\mathcal{F}}}\varphi =P_{\tilde{\mathcal{F}}}U\varphi =U\varphi =\lambda \varphi .$$

(66)

Thus, the eigenvalue $\lambda$ will be in the spectrum of ${\tilde{U}}_N$. More generally, it is known that an operator U and a projection $P_{\tilde{\mathcal{F}}}$ commute if and only if $\tilde{\mathcal{F}}$ is an invariant subspace of U. Thus, the spectrum of the finite-section operator ${\tilde{U}}_N$ is a subset of the spectrum of U for the case when $\tilde{\mathcal{F}}$ is an invariant subspace.

If an eigenfunction $\varphi$ of U is in $\tilde{\mathcal{F}},$ it can be obtained from an eigenvector $\mathbf{a}$ of the finite section ${\tilde{U}}_N$ as

$$\varphi =\mathbf{a}·\tilde{\mathbf{f}}=\sum _{k=1}^N{a}_k{f}_k,$$

(67)

where $\mathbf{a}=(a_1,\dots ,a_N)$ satisfies ${\tilde{U}}_N\mathbf{a}=\lambda \mathbf{a}$, since, for such $\varphi$,

$$U\varphi =\mathbf{a}·U\tilde{\mathbf{f}}=\lambda \varphi =\lambda \mathbf{a}·\tilde{\mathbf{f}}=U_N\mathbf{a}·\tilde{\mathbf{f}}.$$

(68)

We have introduced above the dot notation, that produces a function in $\mathcal{F}$ from an N-vector $\mathbf{a}$ and a set of functions $\tilde{\mathbf{f}}$.

Remark 5. 

The Theorems 2 and 3 are convenient in their use of sampling along trajectory and an invariant measure, thus enabling construction of finite section representations of the Koopman operator from a single trajectory. However, the associated space of functions $L^2\left(\mu \right)$ is restricted since the resulting spectrum is on the unit circle. Choosing a more general measure ν that has support in the basin of attraction is possible. Namely, when we construct the finite section, we then use a sequence ${\mathbf{x}}_l,l=1,\dots ,m$ of points that weakly converges to the measure ν, and their images under T, ${\mathbf{y}}_l=T\left({\mathbf{x}}_l\right)$. This is the approach in [51]. The potential issue with this approach is the choice of space—typically, $L^2\left(\nu \right)$ will have a very large spectrum, for example, filling the entire unit disk of the complex plane [52]. In contrast, Hilbert spaces adapted to the dynamics of a dissipative systems can be constructed [14], starting from the ideal set of continuous functions that vanish on the attractor, enabling a natural setting for computation of spectral objects for dissipative systems.

Koopman mode is the projection of a field of observables on an eigenfunction of U. Approximations of Koopman modes can also be obtained using a finite section. Let ${\tilde{U}}_N$ be a finite section of $\tilde{U}$. Let $\mathbf{h}:M\to {\mathbb{C}}^K$ be a vector observable (thus, a field of observables indexed over a discrete set). Then, the Koopman mode $s_{\lambda }\left(\mathbf{h}\right)$ associated with the eigenvalue $\lambda$ of U is obtained as

$$s_{\lambda }\left(\mathbf{h}\right)=〈\mathbf{h},\widehat{\varphi }〉\varphi ,$$

(69)

where $\varphi ,\widehat{\varphi }$ are the eigenfunction and the dual eigenfunction associated with the eigenvalue $\lambda$. Let ${\mathbf{a}}_j,j=1,\dots ,N$ be eigenvectors of ${\tilde{U}}_N$, and thus, the associated eigenfunctions of the finite section are

$${\varphi }_j={\mathbf{a}}_j·\tilde{\mathbf{f}},j=1,\dots ,N$$

(70)

where ${\mathbf{a}}_j=(a_{j1},\dots ,a_{jN})$. Then, we get the dual basis

$${\widehat{\varphi }}_j=〈{\widehat{\mathbf{a}}}_j,\widehat{\mathbf{g}}〉,j=1,\dots ,N$$

(71)

where

$$〈{\widehat{\mathbf{a}}}_j,{\mathbf{a}}_k〉={\delta }_{jk}.$$

(72)

This is easily checked by expanding:

$$\begin{array}{ccc}\hfill 〈{\varphi }_j,{\widehat{\varphi }}_j〉& =& 〈\sum _{k=1}^N{a}_{jk}{f}_k,\sum _{l=1}^N{\widehat{a}}_{il}{\widehat{g}}_l〉\hfill \\ & =& \sum _{k=1}^N\sum _{l=1}^N{a}_{jk}{\widehat{a}}_{il}^c〈f_k,{\widehat{g}}_l〉\hfill \\ & =& \sum _{k=1}^N{a}_{jk}{\widehat{a}}_{ik}^c={\delta }_{ji}.\hfill \end{array}$$

(73)

Thus, the approximation ${\tilde{s}}_j\left(\mathbf{h}\right)$ to the Koopman mode $s_j\left(\mathbf{h}\right)$ associated with the eigenvalue ${\lambda }_j$ of the finite section reads

$${\tilde{s}}_j\left(\mathbf{h}\right)=〈\mathbf{h},{\widehat{\varphi }}_j〉{\varphi }_j=\sum _{k=1}^N{\widehat{a}}_{jk}〈\mathbf{h},{\widehat{g}}_k〉{\varphi }_j.$$

(74)

Now, assume that $\mathbf{h}=\tilde{\mathbf{f}}$,

$$〈\tilde{\mathbf{f}},{\widehat{\varphi }}_j〉{\varphi }_j=\sum _{k=1}^N{\left({\widehat{\mathbf{a}}}_j\right)}_k〈\tilde{\mathbf{f}},{\widehat{g}}_k〉{\varphi }_j={\widehat{\mathbf{a}}}_j{\varphi }_j.$$

(75)

Thus, the Koopman modes associated with the data vector of observables $\tilde{\mathbf{f}}$ are obtained as the left eigenvector ${\widehat{\mathbf{a}}}_j{\varphi }_j$ of the finite section of the Koopman operator ${\tilde{U}}_N$.

Assuming that the approximation of the finite section, the $N\times N$ matrix ${\tilde{U}}_N^a$ has distinct eigenvalues ${\lambda }_1^a,\dots ,{\lambda }_N^a$, we write the spectral decomposition

$${\tilde{U}}_N^a=A\mathsf{\Lambda }{A}^{-1},$$

(76)

where $\mathsf{\Lambda }$ is the diagonal eigenvalue matrix and

$$A=\left[{\mathbf{a}}_1{\mathbf{a}}_2\dots {\mathbf{a}}_N\right]$$

(77)

is the column eigenvector matrix. From

$${\tilde{U}}_N^a={\left(F^{†}F\right)}^{-1}{F}^{†}{F}^{\prime }=A\mathsf{\Lambda }{A}^{-1},$$

(78)

we get that the data can be reconstructed by first observing

$$F^{†}{F}^{\prime }=F^{†}FA\mathsf{\Lambda }{A}^{-1}.$$

(79)

This represents N equations with m unknowns for each column of $F^{\prime }$. Assuming $m>N$, it is an underdetermined set of equations that can have many solutions for columns of $F^{\prime }$. Then,

$$F_p^{\prime }={\left(F{F}^{†}\right)}^{-1}F{F}^{†}FA\mathsf{\Lambda }{A}^{-1}=FA\mathsf{\Lambda }{A}^{-1}$$

(80)

is the projection of all these solutions on the subspace spanned by the columns of F. If $m<N$, (79) is overdetermined, and the solution $F_p^{\prime }$ is the closest—in least squares sense—to $F^{\prime }$ in the span of the columns of F.

Note that $A^{-1}$ is the matrix in which rows are the Koopman modes ${\widehat{\mathbf{a}}}_k$:

$$A^{-1}=\left[\begin{array}{c}{\widehat{\mathbf{a}}}_1\\ ⋮\\ {\widehat{\mathbf{a}}}_N\end{array}\right],$$

(81)

and, thus,

$$\mathsf{\Lambda }{A}^{-1}=\left[\begin{array}{c}{\lambda }_1{\widehat{\mathbf{a}}}_1\\ ⋮\\ {\lambda }_N{\widehat{\mathbf{a}}}_N\end{array}\right].$$

(82)

Using (68), we get

$$FA=\left[\begin{array}{ccc}\tilde{\mathbf{f}}\left({\mathbf{x}}_1\right)·{\mathbf{a}}_1& \cdots & \tilde{\mathbf{f}}\left({\mathbf{x}}_1\right)·{\mathbf{a}}_N\\ \tilde{\mathbf{f}}\left({\mathbf{x}}_2\right)·{\mathbf{a}}_1& \cdots & \tilde{\mathbf{f}}\left({\mathbf{x}}_2\right)·{\mathbf{a}}_N\\ ⋮& \cdots & ⋮\\ \tilde{\mathbf{f}}\left({\mathbf{x}}_m\right)·{\mathbf{a}}_1& \cdots & \tilde{\mathbf{f}}\left({\mathbf{x}}_m\right)·{\mathbf{a}}_N\end{array}\right]=\left[\begin{array}{ccc}{\tilde{\varphi }}_1\left({\mathbf{x}}_1\right)& \cdots & {\tilde{\varphi }}_N\left({\mathbf{x}}_1\right)\\ {\tilde{\varphi }}_1\left({\mathbf{x}}_2\right)& \cdots & {\tilde{\varphi }}_N\left({\mathbf{x}}_2\right)\\ ⋮& \cdots & ⋮\\ {\tilde{\varphi }}_1\left({\mathbf{x}}_m\right)& \cdots & {\tilde{\varphi }}_N\left({\mathbf{x}}_m\right)\end{array}\right]$$

(83)

where ${\tilde{\varphi }}_j$ is an eigenfunction of the finite section, and ${\mathbf{a}}_j$’s are the columns of A. Note that ${\tilde{\varphi }}_k\left({\mathbf{x}}_l\right)={\tilde{\lambda }}_k^{l-1}{\tilde{\varphi }}_k\left({\mathbf{x}}_1\right)$. Using (80), we get

$$F_p^{\prime }=FA\mathsf{\Lambda }{A}^{-1}=\left[\begin{array}{c}\sum _{k=1}^N{\tilde{\lambda }}_k{\tilde{\varphi }}_k\left({\mathbf{x}}_1\right){\widehat{\mathbf{a}}}_k\\ \sum _{k=1}^N{\lambda }_k^2{\tilde{\varphi }}_k\left({\mathbf{x}}_1\right){\widehat{\mathbf{a}}}_k\\ ⋮\\ \sum _{k=1}^N{\lambda }_k^m{\tilde{\varphi }}_k\left({\mathbf{x}}_1\right){\widehat{\mathbf{a}}}_k\end{array}\right].$$

(84)

Remark 6.

The novelty in this section is the explicit treatment of the finite section approximation in terms of the dual basis that enables error estimates in the next subsection. The finite section is also known under the name Galerkin projection [36]. The relationship between GLA and finite section methods was studied in [40].

4.2. Convergence of the Finite Sample Approximation to the Finite Section

The time averages in (59) converge due to the Birkhoff’s Ergodic Theorem [50]. In the case when a dynamical system is globally stable to an attractor with a physical invariant measure, the rates of convergence depend on the type of asymptotic dynamics that the system is exhibiting. Namely, the Koopman operator U, when restricted to measure-preserving, on-attractor dynamics, is unitary. Its spectrum can in that case be written as ${\sigma }_p\left(U\right)\cup {\sigma }_c\left(U\right)$, where ${\sigma }_p$ denotes the point spectrum corresponding to eigenvalues of U and ${\sigma }_c$ the continous spectrum [53]. The next theorem describes convergence of the finite sample approximation to ${\tilde{U}}_N$ when the asymptotic dynamics has only the point spectrum—e.g., when the attractor dynamics is that of a fixed point, limit cycle or ergodic rotation on a higher dimensional torus:

Theorem 4.

Let $T:M\to M$ be a $C^{\infty }$ dynamical system with an attractor A and an invariant measure supported on the attractor. Let U be the Koopman operator on $L^2\left(\mu \right)$, with a pure point spectrum that is either a non-dense set on the unit circle, or generated by a set of eigenvalues whose imaginary parts $\mathbf{\omega }$$=({\omega }_1,\dots ,{\omega }_m)$ satisfy the Diophantine conditions $|\mathbf{k}·$$\mathbf{\omega }$$-k_0|\ge 4c_0{\left|\mathbf{k}\right|}^{-\mu },\mu >m+1,\mathbf{k}\in {\mathbb{Z}}^m,k_0\in \mathbb{Z}$. Let $f_j,g_k$ be $C^{\infty }$ for all $j,k$. Note that the coefficients in the finite section matrices depend on the initial condition $\mathbf{x}$ of the trajectory that was used to generate the finite section, with the notation $U_{N,m}\left(\mathbf{x}\right),{\tilde{U}}_N\left(\mathbf{x}\right)$. Then, for almost all initial conditions $\mathbf{x}\in M$

$$\left|\right|{\tilde{U}}_{N,m}\left(\mathbf{x}\right)-{\tilde{U}}_N\left(\mathbf{x}\right){\left|\right|}_2\le \frac{c\left(N\right)}{m}$$

(85)

where $\left|\right|·{\left|\right|}_2$ is the Frobenius norm.

Proof. 

We suppress the dependence on $\mathbf{x}$ in the notation. The entries ${\tilde{u}}_{kj,m}=\frac{1}{m}{\sum }_{l=1}^m{f}_j\circ T\left({\mathbf{x}}_l\right){\widehat{g}}_k^c\left({\mathbf{x}}_l\right)$ of $U_{N,m}$ (see (62)) converge a.e. w.r.t. $\mu$. Since T is conjugate to a rotation on an Abelian group [54], which is either discrete or the dynamics is uniformly ergodic (in which case, by assumption, the Diophantine condition is satisfied), for sufficiently smooth T and $f_j,{\widehat{g}}_k$ [55,56,57], we have

$$\left|\right|{\tilde{u}}_{kj,m}-{\tilde{u}}_{kj}{\left|\right|}_2\le \frac{c(f_j,g_k)}{m}$$

(86)

and the statement follows by setting $c\left(N\right)=N^2{max}_{j,k}c(f_j,g_k)$. □

Remark 7.

The smoothness of $T,f_j,{\widehat{g}}_k$ and the Diophantine condition are required in order for the solution of the homological equation to exist [55]. Only finite smoothness is required [55], but we have assumed $C^{\infty }$ for simplicity here.

The above means that ${\tilde{U}}_{N,m}\left(\mathbf{x}\right)$ converges to ${\tilde{U}}_N\left(\mathbf{x}\right)$ spectrally:

Corollary 1.

Let ${\lambda }_m$ be an eigenvalue of ${\tilde{U}}_{N,m}\left(\mathbf{x}\right)$ with multiplicity h. Then, for arbitrary $ϵ>0$, for sufficiently large $m>M$, there is a set of eigenvalues λ of ${\tilde{U}}_N\left(\mathbf{x}\right)$ whose multiplicity sums to h such that $|{\lambda }_m-\lambda |\le ϵ.$

Proof. 

This follows from continuity of eigenvalues [58] to continuous perturbations (established by theorem 4). □

Remark 8.

If $f\circ T^n$ are independent, the convergence estimate above deteriorates to $\mathcal{O}\left(1/\sqrt{m}\right)$. Presence of continuous spectrum without the strong mixing property can lead to convergence estimates $\mathcal{O}(1/m^{\alpha })$ with $0<\alpha <1/2$ [56].

Remark 9.

Spectral convergence in the infinite-dimensional setting is a more difficult question (see [37] in which only convergence along subsequences was established under certain assumptions). Even if the result could be obtained, the practical question is the convergence in m and N. To address it further, we start with the formula for error in the finite section.

4.3. The Error in the Finite Section

It is of interest to find out how big is the error we are making in the finite section approximations discussed above. We have the following result.

Proposition 1.

Let $\tilde{\varphi }=\tilde{\mathbf{e}}·\tilde{\mathbf{f}}$ be an eigenfunction of the finite section associated with the eigenvalue $\tilde{\lambda }$ and eigenvector $\tilde{\mathbf{e}}$. Then,

$$U\tilde{\varphi }-\tilde{\lambda }\tilde{\varphi }=\tilde{\mathbf{e}}·(U\tilde{\mathbf{f}}-P_{\tilde{\mathcal{F}}}U\tilde{\mathbf{f}}).$$

(87)

Proof. 

The first term on the right side of (87) follows from the definition of $\tilde{\varphi }$. We then need to show

$$\tilde{\lambda }\tilde{\varphi }=\tilde{\mathbf{e}}·P_{\tilde{\mathcal{F}}}U\tilde{\mathbf{f}}.$$

(88)

However, the left side is just $U_N\tilde{\varphi }$, and since $\tilde{\mathbf{f}}\in \tilde{\mathcal{F}}$, $U_N\tilde{\mathbf{f}}=P_{\tilde{\mathcal{F}}}U{P}_{\tilde{\mathcal{F}}}\tilde{\mathbf{f}}=P_{\tilde{\mathcal{F}}}U\tilde{\mathbf{f}}$, which proves the claim. □

5. Krylov Subspace Methods

A particularly useful feature of dynamical systems theory based on Koopman operator methods is that properties of the system can be surmised from data. Indeed, in the previous section, we found how a finite section of the matrix representation of the Koopman operator can be found from data. However, the discussion was based on existence of a basis, that typically might come from taking products on basis elements on 1-dimensional subspaces—for example, Fourier basis on an interval subset of $\mathbb{R}$. Such constructions lead to an exponential growth in the number of basis elements, and the so-called curse of dimensionality. In this section, we study finite section numerical methods that are based on the dynamical evolution of a single or many observables—functions on state space—that span the so-called Krylov subspace. The idea is that one might start with a single observable, and due to its evolution, span an invariant subspace of the Koopman operator (note the connection of such methods with the Takens embedding theorem ideas [4,39]). Since the number of basis elements is in this case equal to the number of dynamical evolution steps, in any dimension, Krylov subspace-based methods do not suffer from the curse of dimensionality.

5.1. Single Observable Krylov Subspace Methods

Let T be a discrete-time dynamical system on a compact metric space M equipped with a measure $\mu$ on the Borel $\sigma$-algebra. Let $\mathcal{F}$ be a Hilbert space of functions on M (for suitable spaces, see [14]). For a finite-time evolution of an initial function $f\left(\mathbf{x}\right)\in \mathcal{F}$ under T, we get a (Krylov) sequence

$$(f\left(\mathbf{x}\right),f\circ T\left(\mathbf{x}\right),\dots ,f\circ T^N\left(\mathbf{x}\right))=(f\left(\mathbf{x}\right),Uf\left(\mathbf{x}\right),\dots ,U^{N}f\left(\mathbf{x}\right)),$$

(89)

where U is the Koopman operator associated with T. Let $f_i=f\circ T^{i-1}\left(\mathbf{x}\right)$. Then, clearly $f_{i+1}=U{f}_i$, for $i=1,\dots ,N$. If $f_{N+1}$ was in the space spanned by $f_1,\dots ,f_N$, and these were linearly independent functions, we would have

$$f_{N+1}=\sum _{i=1}^N{c}_i{f}_i,$$

(90)

for some constants $c_i,i=1,\dots N$. In that case, the operator U would have a finite-dimensional approximation ${\tilde{U}}_N$ on the $span(f_1,\dots f_N)$, given by the companion matrix

$$\tilde{U}=C=\left(\begin{array}{ccccc}0& 0& \dots & 0& c_1\\ 1& 0& \dots & 0& c_2\\ 0& 1& \dots & 0& c_3\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& \dots & 1& c_N\end{array}\right)$$

(91)

The above is, in the terminology of the previous section, the finite section representation of U.

Example 3.

Let V be a subspace of $L^2\left(M\right)$ spanned by eigenfunctions $e_1,\dots ,e_N$ that satisfy

$$U{e}_j=e^{i2\pi {\omega }_j}{e}_j$$

(92)

Let

$$g=\sum _{j=1}^N{a}_j{e}_j,$$

(93)

Then,

$$\begin{array}{ccc}\hfill U^{k}g& =& \sum _{j=1}^N{a}_j{U}^k{e}_j\hfill \\ & =& \sum _{j=1}^N{a}_j{e}^{i2\pi k{\omega }_j}{e}_j,\hfill \end{array}$$

(94)

and

$$\begin{array}{ccc}\hfill U^{N}g& =& \sum _{l=1}^N{d}_j{e}_j\hfill \\ & =& \sum _{j=1}^N{a}_j{e}^{i2\pi N{\omega }_j}{e}_j\hfill \\ & =& \sum _{k=1}^N{c}_k\sum _{j=1}^N{a}_j{e}^{i2\pi k{\omega }_j}{e}_j\hfill \\ & =& \sum _{j=1}^N\left(\sum _{k=1}^N{c}_k{e}^{i2\pi k{\omega }_j}\right)a_j{e}_j.\hfill \end{array}$$

(95)

Thus, the numbers $c_k,k=1,\dots ,N$ in the companion matrix are determined by N equations with N unknowns

$$\left(\sum _{k=1}^N{c}_k{e}^{i2\pi k{\omega }_j}\right)a_j=d_j,j=1,\dots ,N.$$

(96)

Now, let $a_1=1$, $a_j=0,j=2,\dots ,N$. We get $U^{N}g=d_1{e}_1=e^{i2\pi N{\omega }_1}{e}_1$ and, thus,

$$c_1=e^{i2\pi N{\omega }_1}.$$

(97)

It is clear that $c_j=0,j=2,\dots ,N$. Note that, if ${\omega }_1=j/N$ for some integer j, we get $c_1=1$ and, thus, the companion matrix becomes the circulant shift matrix

$$\tilde{U}=\left(\begin{array}{ccccc}0& 0& \dots & 0& 1\\ 1& 0& \dots & 0& 0\\ 0& 1& \dots & 0& 0\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& \dots & 1& 0\end{array}\right).$$

(98)

Consider now the case when $U^{N}f$ is not in the span of $f_1,\dots ,f_N$. We have the projection formula (58)

$${\tilde{u}}_{kj}=〈U{f}_j,{\widehat{g}}_k〉.$$

(99)

Since

$$〈f_j,{\widehat{g}}_k〉={\delta }_{k,j},$$

(100)

for $j=1,\dots ,N-1,k=1,\dots ,N$ we have

$${\tilde{u}}_{kj}=〈U{f}_j,{\widehat{g}}_k〉=〈f_{j+1},{\widehat{g}}_k〉={\delta }_{k,j+1}$$

(101)

which produces zeros in all columns of row k except in the column $j-1$, where we have a 1. There is no column $j-1$ for row 1, so we get all zeros up to the last column. Now, for the last column, we have

$${\tilde{u}}_{kN}=〈U{f}_N,{\widehat{g}}_k〉=〈P_{\tilde{\mathcal{F}}}U{f}_N,{\widehat{g}}_k〉,$$

(102)

and, thus, $c_k$ in the matrix (91) is the k-th coefficient of the orthogonal projection of $U{f}_N$ on $\tilde{\mathcal{F}}$ in the basis $\tilde{\mathbf{f}}$ that here consists of the Krylov sequence of independent observables

$$(f,Uf,\dots ,U^{N-1}f)\equiv (f_1,\dots ,f_N),$$

(103)

where we defined $f_1,\dots ,f_N$ by the last relationship.

5.2. Error in the Companion Matrix Representation

Let $\tilde{\mathbf{e}}={(e_1,\dots ,e_N)}^T$ be an eigenvector of $\tilde{U}$ satisfying

$$\tilde{U}\tilde{\mathbf{e}}=\tilde{\lambda }\tilde{\mathbf{e}},$$

(104)

and $\tilde{\mathbf{f}}=(f_1,f_2,f_3,\dots f_N)$. The action of U on $\tilde{\mathbf{e}}·\tilde{\mathbf{f}}$ is given as

$$\begin{array}{ccc}\hfill U\tilde{\mathbf{e}}·\tilde{\mathbf{f}}& =& \tilde{\mathbf{e}}·\tilde{\mathbf{f}}\circ T=\sum _{i=1}^N{e}_i{f}_i\circ T\hfill \\ & =& \sum _{i=1}^N{e}_i{f}_{i+1}.\hfill \end{array}$$

(105)

Now, we also have

$$\begin{array}{ccc}\hfill \tilde{U}\tilde{\mathbf{e}}& =& \left(\begin{array}{ccccc}0& 0& \dots & 0& c_1\\ 1& 0& \dots & 0& c_2\\ 0& 1& \dots & 0& c_3\\ ⋮& ⋮& \ddots & ⋮& ⋮\\ 0& 0& \dots & 1& c_N\end{array}\right)\left(\begin{array}{c}{e}_1\\ e_2\\ e_3\\ ⋮\\ e_N\end{array}\right)=\left(\begin{array}{c}{c}_1{e}_N\\ e_1+c_2{e}_N\\ e_1+c_3{e}_N\\ ⋮\\ e_{N-1}+c_N{e}_N\end{array}\right)\hfill \\ & =& \left(\begin{array}{c}0\\ e_1\\ e_2\\ ⋮\\ e_{N-1}\end{array}\right)+e_N\left(\begin{array}{c}{c}_1\\ c_2\\ c_3\\ ⋮\\ c_N\end{array}\right)=\tilde{\lambda }\tilde{\mathbf{e}}\hfill \end{array}$$

(106)

Using this in (105), and denoting $\tilde{\mathbf{c}}=(c_1,\dots ,c_N)$, we obtain

$$\begin{array}{ccc}\hfill U\tilde{\mathbf{e}}·\tilde{\mathbf{f}}& =& \lambda \tilde{\mathbf{e}}·\tilde{\mathbf{f}}-e_N\tilde{\mathbf{c}}·\tilde{\mathbf{f}}+e_{N}f\circ T^N\hfill \\ & =& \tilde{\lambda }\tilde{\mathbf{e}}·\tilde{\mathbf{f}}+e_N(f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}).\hfill \end{array}$$

(107)

This formula also follows directly from (87) by observing that $U\tilde{\mathbf{f}}=(f_2,\dots ,f_N,f_{N+1})$, where $f_{N+1}=f\circ T^N$, the fact that $P(f_2,\dots ,f_N)=(f_2,\dots ,f_N)$, and $P_{\tilde{\mathcal{F}}}f\circ T^N=\tilde{\mathbf{c}}·\tilde{\mathbf{f}}$. Thus,

$$\tilde{\mathbf{e}}·(U\tilde{\mathbf{f}}-P_{\tilde{\mathcal{F}}}U\tilde{\mathbf{f}})=e_N(f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}).$$

(108)

We have the following simple consequence:

Lemma 1.

If $f\circ T^N$ is in the $span(f_1,\dots f_N)$, then $\tilde{\varphi }=\tilde{\mathbf{e}}·\tilde{\mathbf{f}}$ is an eigenfunction of U associated with the eigenvalue $\tilde{\lambda }$.

If the assumption that $f\circ T^N$ is in $span(f_1,\dots f_N)$ is relaxed, and $\tilde{\mathbf{c}}·\tilde{\mathbf{f}}$ is the orthogonal projection of $\mathbf{f}\circ T^N$ to $\tilde{\mathcal{F}}$, then $\tilde{\mathbf{e}}·\tilde{\mathbf{f}}$ is an approximation to the eigenvector of U with an approximate eigenvalue $\tilde{\lambda }$, with the error

$$e_N(f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}})=e_{N}r,$$

(109)

where $r=f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}$ is called the residual. Note that Equation (107) could be written as

$$|U\tilde{\mathbf{e}}·\tilde{\mathbf{f}}-\tilde{\lambda }\tilde{\mathbf{e}}·\tilde{\mathbf{f}}|=|e_{N}r|,$$

(110)

which means that $\tilde{\mathbf{e}}·\tilde{\mathbf{f}}$ is in the $(\tilde{\lambda },ϵ)$-pseudospectrum of U for $ϵ=|e_{N}r|$ (see [59]).

The calculations above, first presented in [39], allow us to show how the finite section spectrum approximates the spectrum of the Koopman operator when the number of functions N in the Krylov sequence goes to infinity. The specific sense of approximation here is pseudospectral, and for a class of systems that satisfy a convergence requirement on the Krylov sequence, convergence in the pseudospectral sense can be proven:

Lemma 2.

Let the Krylov sequence satisfy

$$\underset{N\to \infty }{lim}\left|\right|f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}\left|\right|=0$$

(111)

Then, for and $ϵ>0$, for large enough n, an eigenfunction of the finite section $\tilde{\varphi }$ is in the ϵ-pseudospectrum of U.

Proof. 

Without loss of generality, we assume $|\tilde{\mathbf{e}}|=1$ and, thus, $|e_N|\le 1$. From (109), taking N large enough, we get

$$|U\tilde{\mathbf{e}}·\tilde{\mathbf{f}}-\tilde{\lambda }\tilde{\mathbf{e}}·\tilde{\mathbf{f}}|=|e_N(f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}})|\le |f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}|<ϵ,$$

(112)

which proves the claim. □

Theorem 5.  

Assume that for any function g in the space of observables $\mathcal{F}$ equipped with a norm $\left|\right|·\left|\right|$, we have

$$g=\sum _{j=1}^{\infty }{c}_k{\varphi }_k$$

(113)

where ${\varphi }_k$ are normalized ($\left|\right|{\varphi }_k\left|\right|=1$) eigenfunctions of the Koopman operator associated with eigenvalues $|{\lambda }_k|\le 1$, i.e., U has a pure point spectrum in $\mathcal{F}$. Let $\tilde{\mathbf{f}}$ be the Krylov sequence generated by f and ${\mathcal{F}}_f$ the cyclic invariant subspace of U generated by f. Let $P_{\tilde{\mathcal{F}}}^n$ be the orthogonal projection on the subspace of $\mathcal{F}$ generated by the first n elements of the Krylov sequence. Let $\tilde{\lambda },\tilde{\varphi }$ be an eigenvalue-eigenfunction pair for the finite section. Then, for any $ϵ>0$, there is an N such that $n\ge N$ implies

$$\left|\right|U\tilde{\varphi }-\tilde{\lambda }\tilde{\varphi }\left|\right|<ϵ.$$

(114)

Proof. 

Due to Lemma 2, we only need to prove that, under the assumption on the spectrum,

$$\underset{N\to \infty }{lim}\left|\right|f\circ T^N-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}\left|\right|=0.$$

(115)

Due to the assumption in Equation (113), we have

$$f=\sum _{k=1}^{\infty }{c}_k{\varphi }_k,$$

(116)

and, thus,

$$f^N=f\circ T^N=\sum _{j=1}^{\infty }{c}_k{\lambda }_k^N{\varphi }_k.$$

(117)

We split the spectrum of U in $\mathcal{F}$ as ${\sigma \left(U\right)=\sigma \left(U\right)|}_{S^1}{+\sigma \left(U\right)|}_D$, where D is the interior of the unit disk in the complex plane. Then,

$$f\circ T^N=f_{S_1}^N+f_D^N=\sum _{{\lambda }_k{\in \sigma \left(U\right)|}_{S^1}}{c}_k{\lambda }_k^N{\varphi }_k+\sum _{{\lambda }_j{\in \sigma \left(U\right)|}_D}{c}_j{\lambda }_j^N{\varphi }_j.$$

(118)

For sufficiently large N, for any $ϵ/2$

$$|\sum _{{\lambda }_j{\in \sigma \left(U\right)|}_D}{c}_j{\lambda }_j^N{\varphi }_j|\le ϵ/2.$$

(119)

In addition,

$$\sum _{{\lambda }_k{\in \sigma \left(U\right)|}_{S^1}}{c}_k{\lambda }_k^N{\varphi }_k$$

(120)

is an almost periodic function and, thus, for sufficiently large $M>N$, we have

$$|f_{S_1}^M-f_{S_1}^N|\le ϵ/2.$$

(121)

Combining (119) and (121) proves the claim, since $f^M$ is $ϵ$-away from an element $f^j$ of the $span\left(f,\dots ,f^{M-1}\right)$, and $\left|\right|f\circ T^M-\tilde{\mathbf{c}}·\tilde{\mathbf{f}}\left|\right|$ is the minimal distance of $f^M$ to the subspace $span\left(f,\dots ,f^{M-1}\right)$ that contains $f^j$. □

Remark 10.

The above construction only requires the Krylov sequence, and shows that the finite section approximation reveals the pseudospectrum of the Koopman operator. Thus, methods relying on Krylov sequences are “sampling” the high-dimensional space and can approximate the part of the spectrum contained in their invariant subspace irrespective of the dimension of the problem.

The use of Krylov sequences is also of interest because they span the smallest invariant subset that the observable f belongs to:

Theorem 6. 

Let f be an observable. Then, $\mathsf{span}\left(f,f\circ T,\dots ,f\circ T^n,\dots \right)$ is the smallest forward invariant subspace of U that contains f.

Proof. 

Assume not. Then, there is $A\subset \mathsf{span}\left(f,f\circ T,\dots ,f\circ T^n,\dots \right)$, where A is a proper subset, that contains f, meaning that there is $f\circ T^j$ for some integer j, that is not in A. However, then, A is not invariant since it contains f and $U^{j}f$ is not in A. □

Remark 11. 

The assumptions in Theorem 5 are satisfied by any dynamical system with a quasi-periodic attractor with the space of observables being an appropriately constructed Hilbert space [14]. However, they exclude systems with mixed or purely continuous spectrum, as evidenced by the Example 2.

5.3. Krylov Sequences from Data

If M is not a finite discrete set, numerically, we do not have $\tilde{\mathbf{f}}$ on the whole state space. Instead, we might be able to sample the function f on a discrete subset of points $\mathbf{X}={\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_m\}}^T\subset M$. We can think of f as a column vector, and form again the $m\times N$data matrixF

$$F=\left[\begin{array}{c}{f}_1\left(\mathbf{X}\right)f_2\left(\mathbf{X}\right)\dots f_N\left(\mathbf{X}\right)\end{array}\right]$$

(122)

and its first iterate

$$\begin{array}{ccc}\hfill F^{\prime }& =& \left[\begin{array}{c}{f}_2\left(\mathbf{X}\right)f_3\left(\mathbf{X}\right)\dots f_{N+1}\left(\mathbf{X}\right)\end{array}\right]\hfill \\ & =& \left[\begin{array}{c}{f}_1\left(T\mathbf{X}\right)f_2\left(T\mathbf{X}\right)\dots f_N\left(T\mathbf{X}\right)\end{array}\right]\hfill \\ & =& \left[\begin{array}{c}{f}_1\left(\mathbf{Y}\right)f_2\left(\mathbf{Y}\right)\dots f_N\left(\mathbf{Y}\right),\end{array}\right].\hfill \end{array}$$

(123)

where $\mathbf{Y}=T\mathbf{X}$. We have

$$F^{\prime }=FC,$$

(124)

or

$$C=F^{+}{F}^{\prime }$$

(125)

as could be surmised from (63), and the following corollary of Lemma 1 holds:

Corollary 2.

Let $f\circ T^N$ be in the $span(f_1,\dots f_N)$, and $rankF=N$. Let $\tilde{\lambda },\tilde{\mathbf{e}}$ be an eigenvalue and the eigenvector of the companion matrix $\tilde{U}$. Then, an eigenvalue $\tilde{\lambda }$ of $\tilde{U}$ is an eigenvalue of U, and $\tilde{\mathbf{f}}\left(\mathbf{X}\right)·\tilde{\mathbf{e}}$ is a sample of the corresponding eigenfunction of U on $\mathbf{X}$.

Proof. 

As soon as we know N samples of the function f, the vector $\tilde{\mathbf{c}}$ in the companion matrix is fixed, and thus the residual is zero. □

When ${\mathbf{x}}_k=T{\mathbf{x}}_{k-1},k=1,\dots ,m+n-1$, i.e., the sampling points are on a single trajectory, the matrix F becomes the Hankel–Takens matrix

$$H=\left[\begin{array}{cccc}f\left(\mathbf{x}\right)& f\left(T\mathbf{x}\right)& \dots & f\left(T^{n-1}\mathbf{x}\right)\\ f\left(T\mathbf{x}\right)& f\left(T^2\mathbf{x}\right)& \dots & f\left(T^n\mathbf{x}\right)\\ ⋮& \cdots & \cdots & ⋮\\ f\left(T^m\mathbf{x}\right)& f\left(T^{m+1}\mathbf{x}\right)& \dots & f\left(T^{m+n-1}\mathbf{x}\right)\end{array}\right].$$

(126)

The reason for calling H the Hankel–Takens matrix is that, besides the usual property of Hankel matrices that have constant skew-diagonal terms—in this case $H_{i,j}=f\left(T^k\mathbf{x}\right)$, where $k=\left|i\right|+\left|j\right|-2$—it also satisfies $H_{i,j+1}=H_{i,j}\circ T=H_{i+1,j}$, a property which is related to the Takens embedding [4,60].

Let C have distinct eigenvalues. We diagonalize it using

$$C=A\mathsf{\Lambda }{A}^{-1}.$$

(127)

The companion matrix is diagonalized by the so-called Vandermonde matrix

$$A^{-1}=\left[\begin{array}{ccccc}1& {\lambda }_1& {\lambda }_1^2& \cdots & {\lambda }_1^{N-1}\\ 1& {\lambda }_2& {\lambda }_2^2& \cdots & {\lambda }_2^{N-1}\\ 1& {\lambda }_3& {\lambda }_3^2& \cdots & {\lambda }_3^{N-1}\\ ⋮& ⋮& ⋮& \ddots & ⋮\\ 1& {\lambda }_N& {\lambda }_N^2& \cdots & {\lambda }_N^{N-1}\end{array}\right].$$

(128)

Thus, the Koopman modes of the vector of observables $\tilde{\mathbf{f}}$ composed of time delays are precisely the columns of the Vandermonde matrix, while the right eigenvectors are the columns of the inverse of the Vandermonde matrix.

5.4. Schmid’s Dynamical Mode Decomposition as a Finite Section Method

The key numerical issue with the Krylov subspace-based algorithms is the fact that the procedure requires inversion of the Vandermonde matrix (128). Since the condition number $\left|\right|A\left|\right|\left|\right|A^{-1}\left|\right|$ (where $\left|\right|·\left|\right|$ is the induced matrix norm) of the Vandermonde matrix scales exponentially in its size provided ${\lambda }_k\ne e^{i\omega }$, for some k, even if $|{\lambda }_k|\approx 1$ [61]. There are a variety of ways to resolve this issue, and the first one that appeared [21] is the following version of the Koopman operator approximation, based on singular value decomposition.

Let

$$F=G\Sigma V^{†}$$

(129)

be the “thin” singular value decomposition of the $m\times N$ “data matrix” F, whose columns are samples of functions $f_1,\dots ,f_N$. The $m\times N$ matrix G and $N\times N$ matrix V are unitary matrices, $V^{†}$ is the conjugate transpose of $V,$ and $\Sigma$ is an $N\times N$ diagonal matrix. Note that

$$FV=G\Sigma ,$$

(130)

and, thus,

$$F{\mathbf{v}}_j={\sigma }_j{\mathbf{u}}_j,j=1,\dots ,n$$

(131)

where ${\mathbf{v}}_j$ is the j-th column of V and ${\mathbf{u}}_j$ is the j-the column of $\mathbf{u}$. Clearly then, ${\mathbf{u}}_j$ are linear combinations of vectors $f_1\left(\mathbf{X}\right),f_2\left(\mathbf{X}\right),\dots ,f_N\left(\mathbf{X}\right)$, and for $m\ge N$, there are N such linear combinations. We could consider each of these combinations as a sample of a function,

$${\tilde{u}}_j={\mathbf{v}}_j·\tilde{\mathbf{f}},$$

(132)

where $\tilde{\mathbf{f}}=(f_1,\dots ,f_N)$ is the vector of independent functions. In other words,

$$\tilde{\mathbf{u}}=(u_1,\dots ,u_N)$$

(133)

spans $\mathcal{F}$ and is an orthogonal basis for it. Now, G is in fact the data matrix whose columns are ${\mathbf{u}}_j$’s:

$$G=[{\mathbf{u}}_1{\mathbf{u}}_2\dots {\mathbf{u}}_N]=[{\mathbf{u}}_1\left(\mathbf{X}\right){\mathbf{u}}_2\left(\mathbf{X}\right)\dots {\mathbf{u}}_N\left(\mathbf{X}\right)]$$

(134)

Then, the finite section is

$$\begin{array}{ccc}\hfill {\tilde{U}}_N^S& =& G^{+}{G}^{\prime }={\left(G^{†}G\right)}^{-1}{G}^{†}{G}^{\prime }\hfill \\ & =& G^{†}{F}^{\prime }V{\Sigma }^{-1}.\hfill \end{array}$$

(135)

Now, since

$$G^{†}={\left(FV{\Sigma }^{-1}\right)}^{†}={\Sigma }^{-1}{V}^{†}{F}^{†},$$

(136)

$$G^{†}G={\Sigma }^{-1}{V}^{†}{F}^{†}FV{\Sigma }^{-1},$$

(137)

and, thus,

$${\left(G^{†}G\right)}^{-1}{G}^{†}=\Sigma V^{†}{\left(F^{†}F\right)}^{-1}V\Sigma {\Sigma }^{-1}{V}^{†}{F}^{†}=\Sigma V^{†}{\left(F^{†}F\right)}^{-1}{F}^{†},$$

(138)

we have

$${\tilde{U}}_N^S=\Sigma V^{†}{\left(F^{†}F\right)}^{-1}{F}^{†}{F}^{\prime }V{\Sigma }^{-1}=\Sigma V^{†}{F}^{+}{F}^{\prime }V{\Sigma }^{-1}=\Sigma V^{†}{\tilde{U}}_N^{a}V{\Sigma }^{-1}.$$

(139)

Therefore, ${\tilde{U}}_N^S$ and ${\tilde{U}}_N^a$ are similar matrices that thus have the same spectrum. If ${\mathbf{a}}_j$ is an eigenvector of ${\tilde{U}}_N^S$, then $V{\Sigma }^{-1}{\mathbf{a}}_j$ is an eigenvector of ${\tilde{U}}_N^a$, and, according to (67)

$${\tilde{\varphi }}_j^N=G{\mathbf{a}}_j$$

(140)

is a finite section approximation to an eigenfunction of the Koopman operator.

6. Weak Eigenfunctions from Data

In the sections above we presented finite section approximations of the Koopman operator, starting from the idea that bounded infinite-dimensional operators are, given a basis, represented by infinite matrices, and then truncated those. In this section, we will present an alternative point of view that provides additional insights into the relationship between the finite-dimensional approximation and the operator. As a consequence of this approach, we show how the concept of a weak eigenfunction, first discussed in [37], arises.

We start again with a vector of observables, $\tilde{\mathbf{f}}=(f_1,\dots ,f_N)$. Except when we can consider this problem analytically, we know the values of observables only on a finite set of points in state space, $\mathbf{X}=\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_m\}$. Assume also that we know the value of $\tilde{\mathbf{f}}$ at $\mathbf{Y}=\left\{{\mathbf{y}}_k\right\}=\left\{T\left({\mathbf{x}}_k\right)\right\}$. We can think of $f_j\left(\mathbf{X}\right)=(f_j\left({\mathbf{x}}_1\right),\dots ,f_j\left({\mathbf{x}}_m\right)),j\in \{1,\dots ,N\}$ as a sample of the observable $f_j$ on $\mathbf{X}\subset M$.

Consider the case ${\mathbf{x}}_{k+1}=T{\mathbf{x}}_k,k=1,\dots ,m-1$. There are many $m\times m$ matrices A such that

$$f_j{\left(\mathbf{Y}\right)}^T=A{f}_j{\left(\mathbf{X}\right)}^T$$

(141)

One of them is the transpose of the companion matrix (91)

$${\tilde{U}}^T=\left(\begin{array}{ccccc}0& 1& 0& \dots & 0\\ 0& 0& 1& \dots & 0\\ 0& 0& 0& \dots & 0\\ ⋮& ⋮& & \ddots & ⋮\\ c_{j1}& c_{j2}& c_{j3}& \dots & c_{jm}\end{array}\right),$$

(142)

but there are many values that $c_{jk},k=1,\dots m$ can assume, since the only requirement on them is

$$\sum _{k=1}^m{c}_{jk}{f}_j\left({\mathbf{x}}_k\right)=f_j\left({\mathbf{y}}_m\right)$$

(143)

and there are m unknowns and 1 equation that determines them. However, the $c^{\prime }s$ need not depend on j, since the operator that maps the vectors $f_j{\left(\mathbf{X}\right)}^T$ to $f_j{\left(\mathbf{Y}\right)}^T$ is not dependent on j. Clearly, if there are m observables, then we get

$$\sum _{k=1}^m{c}_k{f}_j\left({\mathbf{x}}_k\right)=f_j\left({\mathbf{y}}_m\right),j\in \{1,\dots ,m\},$$

(144)

and, thus, we can determine $\mathbf{c}=(c_1,\dots ,c_m)$ uniquely.

If the number of observables N is larger than m, then $f_{jk}=f_j\left({\mathbf{x}}_k\right)$ are elements of an $N\times m$ matrix F (note that this data matrix is precisely the transpose of the one we have used before, in (122)) and, thus, there are not enough components in $\mathbf{c}$ to solve

$$F\mathbf{c}=\tilde{\mathbf{f}}{\left({\mathbf{y}}_m\right)}^T.$$

(145)

This system is overdetermined, so in general does not have a solution. The Dynamic Mode Decomposition method then solves for $\mathbf{c}$ using the following procedure: let P be the orthogonal projection onto span of columns of F. Then,

$$F{\mathbf{c}}_{MP}=P\tilde{\mathbf{f}}{\left({\mathbf{y}}_m\right)}^T,$$

(146)

has a solution, provided F has rank m: $P\tilde{\mathbf{f}}{\left({\mathbf{y}}_m\right)}^T$ is an N-dimensional vector in the span of the columns of F and thus can be written as a linear combination of those vectors. In fact, we can write

$${\mathbf{c}}_{MP}=F^{+}\tilde{\mathbf{f}}{\left({\mathbf{y}}_m\right)}^T.$$

(147)

We now discuss the nature of the approximation of the Koopman operator U by the companion matrix (91)

$${\tilde{U}}^T=C^T=\left(\begin{array}{ccccc}0& 1& 0& \dots & 0\\ 0& 0& 1& \dots & 0\\ 0& 0& 0& \dots & 0\\ ⋮& ⋮& & \ddots & ⋮\\ c_1& c_2& c_3& \dots & c_m\end{array}\right),$$

(148)

where $\mathbf{c}=(c_1,\dots ,c_m)={\mathbf{c}}_{MP}$ obtained from Equation (147).

Let $\mathcal{S}=\{{\mathbf{x}}_1,\dots ,{\mathbf{x}}_m\}$ be an invariant set for $T:M\to M$, where M is a measure space, with measure $\mu$. Consider the space ${\mathcal{C}|}_{\mathcal{S}}$, of continuous functions in $L^2\left(\mu \right)$ restricted to $\mathcal{S}$. This is an m-dimensional vector space. The restriction ${U|}_{\mathcal{S}}$ of the Koopman operator to ${\mathcal{C}|}_{\mathcal{S}}$, is then a finite-dimensional linear operator that can be represented in a basis by an $m\times m$ matrix. An explicit example is given when ${\mathbf{x}}_j,j=1,\dots ,m$ represent successive points on a periodic trajectory, and the resulting matrix representation in the standard basis is the $m\times m$ cyclic permutation matrix

$$\mathsf{\Pi }=\left(\begin{array}{ccccc}0& 1& 0& \dots & 0\\ 0& 0& 1& \dots & 0\\ 0& 0& 0& \dots & 0\\ ⋮& ⋮& & \ddots & ⋮\\ 1& 0& 0& \dots & 0\end{array}\right),$$

(149)

If $\mathcal{S}$ is not an invariant set, an $m\times m$ approximation of the reduced Koopman operator can still be provided. Namely, if we know m independent functions’ restrictions $\left(f_j\right){|}_{\mathcal{S}}$, $j=1,\dots ,m$ in ${\mathcal{C}|}_{\mathcal{S}},$ and we also know $f_j\left(T{\mathbf{x}}_k\right)$, $j,k\in \{1,\dots ,m\}$, we can provide a matrix representation of ${U|}_{\mathcal{S}}$. However, while in the case where $\mathcal{S}$ is an invariant set, the iterate of any function in ${\mathcal{C}|}_{\mathcal{S}}$ can be obtained in terms of the iterate of m independent functions, for the case when $\mathcal{S}$ is not invariant, this is not necessarily so. Namely, the fact that $\mathcal{S}$ is not invariant means that functions in ${\mathcal{C}|}_{\mathcal{S}}$ do not necessarily experience linear dynamics under ${U|}_{\mathcal{S}}$. However, one can take N observables $f_j$, $j=1,\dots ,N$, where $N>m$, and approximate the nonlinear dynamics using linear regression on $\tilde{\mathbf{f}}\left(\mathbf{X}\right)\equiv (\mathbf{f}\left({\mathbf{x}}_1\right),\dots ,\mathbf{f}\left({\mathbf{x}}_m\right)),$ where $\mathbf{f}(·)={(f_1(·),\dots ,f_N(·))}^{\mathsf{T}}$—i.e., by finding an $m\times m$ matrix C that gives the best approximation of the data in the Frobenius norm,

$$C^T=\underset{B\in {\mathbb{C}}^{m\times m}}{argmin}\left|\right|\mathbf{f}\left(T\mathbf{x}\right)-{\mathbf{f}\left(\mathbf{x}\right)B\left|\right|}_F\equiv \underset{B\in {\mathbb{C}}^{m\times m}}{argmin}{\parallel {\left(f_j\left(T{\mathbf{x}}_k\right)\right)}_{j,k=1,1}^{n,m}-{\left(f_j\left({\mathbf{x}}_k\right)\right)}_{j,k=1,1}^{n,m}B\parallel }_F.$$

(150)

We have the following:

Theorem 7.

Let $T:M\to M$ be a measure μ-preserving transformation on a metric space M, and let ${\mathcal{S}}_m=\left\{{\mathbf{x}}_j\right\},j=1,\dots ,m$ be a trajectory such that, when $m\to \infty$, ${\mathcal{S}}_m$ becomes dense in a compact invariant set $A\in M$. Then, for any N-vector of observables ${\mathbf{f}\in \mathcal{C}|}_{{\mathcal{S}}_m},N\ge m$, we have

$$\underset{m\to \infty }{lim}{\left|U\right|}_{{\mathcal{S}}_m}\mathbf{f}-C\mathbf{f}|=0$$

(151)

Proof. 

By density of ${\mathcal{S}}_{\infty }$, for sufficiently large M, $m\ge M$ implies $|{\mathbf{x}}_m-{\mathbf{x}}_j|<{ϵ}_M$ for some ${\mathbf{x}}_j\in {\mathbf{x}}_1,\dots ,{\mathbf{x}}_{m-1}$. By continuity of observables,

$${\left|U\right|}_{{\mathcal{S}}_m}\mathbf{f}-C\mathbf{f}|\le D{ϵ}_M$$

(152)

for some constant D. Taking M sufficiently large makes ${ϵ}_M\to 0$. □

Consider an m-dimensional eigenvector $\tilde{\mathbf{e}}=(e_1,\dots ,e_m)$ of ${\tilde{U}}^T$, associated with the eigenvalue $\lambda$. Since the eigenvector satisfies

$${\tilde{U}}^T\tilde{\mathbf{e}}=\lambda \tilde{\mathbf{e}}$$

(153)

we have

$${\tilde{\mathbf{e}}}_{k+1}=\lambda {\tilde{\mathbf{e}}}_k,k=1,\dots ,m-2.$$

(154)

Thus, $\tilde{\mathbf{e}}$ can be considered as an eigenfunction on the finite set ${\mathbf{x}}_1,\dots ,{\mathbf{x}}_{m-1}$. On the last point of the sample, ${\mathbf{x}}_m$, we have

$$\sum _{j=1}^m{c}_j{\tilde{\mathbf{e}}}_j=\lambda {\tilde{\mathbf{e}}}_m$$

(155)

Let us now consider the concept of the weak eigenfunction, or eigendistribution. Let $\nu$ be some prior measure of interest on M. Let $\varphi$ be a bounded function that satisfies $\varphi \circ T=\lambda \varphi$. We construct the functional L on $C\left(M\right)$ by defining

$$L\left(h\right)={\int }_{M}h\varphi d\nu .$$

(156)

Set $UL\left(h\right)={\int }_{M}h\varphi \left(T\mathbf{x}\right)d\nu$ and we get

$$UL\left(h\right)={\int }_{M}h\varphi \left(T\mathbf{x}\right)d\nu =\lambda {\int }_{M}h\varphi d\nu =\lambda L\left(h\right).$$

(157)

Clearly, this is satisfied if $\varphi$ is a continuous eigenfunction of U at eigenvalue $\lambda$. However, Equation (157) is applicable for cases with much less regularity. Namely, if $\mu$ is a measure and

$$L(f)=\int f(\mathbf{x})d\mu (\mathbf{x})$$

(158)

the associated linear functional, then we can define the action of U on L by

$$UL(f)=\int f(\mathbf{x})d\mu (T\mathbf{x}).$$

(159)

Consider, for example, a set of points ${\mathbf{x}}_k,k\in {\mathbb{N}}^{+}$ and assume that for every continuous h there exists the limit

$$L\left(h\right)=\underset{K\to \infty }{lim}\frac{1}{K}\sum _{k=1}^{K}h\left({\mathbf{x}}_k\right)\tilde{\mathbf{e}}\left({\mathbf{x}}_k\right).$$

(160)

Then, by the Riesz representation theorem, there is a measure $\mu$ such that

$$L\left(h\right)={\int }_{M}hd\mu .$$

(161)

Definition 1.

Let a measure μ be such that the associated linear functional L satisfies

$$UL=\lambda L,$$

(162)

for some $\lambda \in \mathbb{C}$. Then, μ is called a weak eigenfunction of U.

Now, we have

$$UL\left(h\right)=\underset{K\to \infty }{lim}\frac{1}{K}\sum _{k=1}^{K}h\left({\mathbf{x}}_k\right)\tilde{\mathbf{e}}\left({\mathbf{x}}_{k+1}\right)=\lambda \underset{K\to \infty }{lim}\frac{1}{K}\sum _{k=1}^{K}h\left({\mathbf{x}}_k\right)\tilde{\mathbf{e}}\left({\mathbf{x}}_k\right)=\lambda L\left(h\right),$$

(163)

proving the following theorem:

Theorem 8.

Consider a set of points ${\mathbf{x}}_k,k\in {\mathbb{N}}^{+}$, on a trajectory of T, and assume that for every continuous h, there exists the limit

$$L\left(h\right)=\underset{K\to \infty }{lim}\frac{1}{K}\sum _{k=1}^{K}h\left({\mathbf{x}}_k\right)\tilde{\mathbf{e}}\left({\mathbf{x}}_k\right),$$

(164)

where

$$\tilde{\mathbf{e}}\left({\mathbf{x}}_k\right)=\lambda \tilde{\mathbf{e}}\left({\mathbf{x}}_{k-1}\right).$$

(165)

Then, the μ associated with $L\left(h\right)$ by

$$L\left(h\right)={\int }_{M}hd\mu ,$$

(166)

is a weak eigenfunction of U associated with the eigenvalue λ.

From the above, it follows that the left eigenvectors of ${\tilde{U}}^T$ are approximations of the associated (possibly weak) Koopman modes, as it is assumed that ł is such an eigenvector,

$${\mathrm{ł}}_j\tilde{U}={\lambda }_j{\mathrm{ł}}_j.$$

(167)

Then,

$$〈\mathrm{ł},f_j\left(\mathbf{X}\right)〉$$

(168)

is the projection of $f_j\left(\mathbf{X}\right)$ on the eigenspace spanned by the eigenvector ${\mathbf{e}}_j$. Moreover, since

$${\mathrm{ł}}_j=\lambda {\mathrm{ł}}_{j+1}-c_m{\mathrm{ł}}_m$$

(169)

the statement can be obtained in the limit $K\to \infty$ by the so-called Generalized Laplace Analysis (GLA) that we described in Section 3.

Remark 12.

The standard interpretation of the Dynamic Mode Decomposition (e.g., on Wikipedia) was in some way a transpose of the one presented here: the observables $f_1^T,\dots f_m^T$ (interpreted as column vectors) were assumed to be related by a matrix $A:f_{j+1}=A{f}_j$. Instead, in the nonlinear, Koopman operator interpretation, each row is mapped into its image, and this allows interpretation on the space of observables. This is particularly important in the context of evolution equations, for example, fluid flows, where the evolution of the observables’ field—the field of velocity vectors at different spatial points—is not evolving linearly.

7. Conclusions

In this paper, we pursued analysis of two of the major approaches to computation of the Koopman operator spectrum: the Generalized Laplace Analysis and the finite section method. We derived approximation results and reinterpreted finite section as a method acting on samples of continuous functions on the state space. The example of a chaotic system with continuous spectrum shows how a failure of the finite section method can occur for that class of systems. The question of choice of observables is often raised in the context of finite-section approximations such as the EDMD. Specifically, the number of basis functions—e.g., Fourier basis on a box in a d-dimensional space—selected as observables can increase exponentially with the dimension d. The pseudospectral result proven here shows that choosing time-delayed observables avoids this issue, making time-delayed observations a natural choice. However, it is clear from the example we gave that the finite section method can fail to converge spectrally for systems with continuous spectrum.

One can understand the Krylov subspace approach as sampling by dynamics in the observables space. The weak eigenfunction approach is based on sampling in the state space. Thus, both techniques avoid the curse of dimensionality that methods such as EDMD potentially introduce.

There are a number of directions for future research based on the work presented here. Generalized Laplace Analysis methods could use results in numerical approximations of Laplace transfroms [62] to remedy some of the difficulties arising in computation. It could be coupled with the power methods in numerical linear algebra for computation of eigenvalues, eigenfunctions and modes. There is a vast literature on Krylov subspace methods that can be used to refine the computations using finite section methodologies. Finally, recent results in computation of Koopman operator approximations might provide a direction for obtaining pseudospectrum convergence results for the cases of dynamical systems with (partially) continuous spectrum.