A Convex Data-Driven Approach for Nonlinear Control Synthesis

Abstract

We consider a class of nonlinear control synthesis problems where the underlying mathematical models are not explicitly known. We propose a data-driven approach to stabilize the systems when only sample trajectories of the dynamics are accessible. Our method is built on the density-function-based stability certificate that is the dual to the Lyapunov function for dynamic systems. Unlike Lyapunov-based methods, density functions lead to a convex formulation for a joint search of the control strategy and the stability certificate. This type of convex problem can be solved efficiently using the machinery of the sum of squares (SOS). For the data-driven part, we exploit the fact that the duality results in the stability theory can be understood through the lens of Perron–Frobenius and Koopman operators. This allows us to use data-driven methods to approximate these operators and combine them with the SOS techniques to establish a convex formulation of control synthesis. The efficacy of the proposed approach is demonstrated through several examples.

1. Introduction

The celebrated Lyapunov theory lays the foundation of stability analysis for general nonlinear dynamical systems. Lyapunov functions provide stability certificates for a nonlinear system. For a given system, searching for a proper Lyapunov function can often be formulated as a convex optimization problem, and thus is easy to address. For instance, for polynomial dynamics, this is achieved through the sum of squares (SOS). Regardless of its similarity to stability analysis, the problem of nonlinear controller synthesis is much more challenging. Other than a few special cases such as linear quadratic control problems, the joint search for Lyapunov stability certificate and control strategy can no longer be cast as convex optimization problems. This is exacerbated by the fact that in many applications, the underlying mathematical models are not available. Our objective in this paper is to establish a principled approach for nonlinear control synthesis when the mathematical models of the underlying dynamics are not explicitly given.

We provide a systematic approach for data-driven control synthesis for a class of control-affine nonlinear systems of the form

$$\dot{\mathbf{x}}=\mathbf{F}\left(\mathbf{x}\right)+\mathbf{G}\left(\mathbf{x}\right)\mathbf{u},$$

(1)

where the state $\mathbf{x}\in {\mathbb{R}}^n$ and control inputs $\mathbf{u}\in {\mathbb{R}}^m$, $\mathbf{F}$ represent the open-loop dynamics, and $\mathbf{G}\left(\mathbf{x}\right)=[{\mathbf{G}}_1\left(\mathbf{x}\right),\dots ,{\mathbf{G}}_m\left(\mathbf{x}\right)]$ constitutes the feedback control loop corresponding to control inputs $\mathbf{u}={[u_1,\dots ,u_m]}^{\top }$. The objective is to design a state feedback controller $\mathbf{u}=\mathbf{u}\left(\mathbf{x}\right)$ such that the closed-loop system is asymptotically stable. To achieve this objective, we use density function-based dual stability formulation introduced by Rantzer for almost everywhere stability analysis and synthesis for nonlinear control systems [1]. Unlike the Lyapunov function-based approach for control design, the co-design problem of simultaneously finding the density function and almost everywhere stabilizing controller can be written as a convex optimization problem. We exploit this convexity property for data-driven control synthesis. In [2,3], it was shown that the duality between density and Lyapunov function in the stability theory could be understood using the linear operator theoretic framework. In particular, the duality between Koopman and Perron–Frobenius operators is at the heart of the duality in the stability theory. This linear operator theoretic framework is also exploited for the data-driven control design [4,5].

The recent advances in data-driven approximation of the Koopman operator are used to discover a data-driven approach for the nonlinear control synthesis. In Koopman theory, a nonlinear system is lifted to an, albeit infinite-dimensional, linear system. This lifting can be approximated using data generated from the underlying nonlinear dynamics by the well-known Extended Dynamic Mode Decomposition (EDMD) algorithm [6]. These tools have been successfully applied in many domains such as fluid dynamics [7] and power systems [8,9], to understand the principle components/modes of given nonlinear dynamics [10]. Recently, Koopman theory has been introduced to the control synthesis tasks, in the hopes that a controller designed in the lifted space could be easier than that in the original state space. It turns out to be a challenging problem, since the lifting argument in the presence of control is no longer valid. Regardless of the progress that has been made in this direction during the last few years [11,12,13,14], a principled data-driven framework for nonlinear control synthesis is not yet available.

We use the EDMD algorithm combined with the duality results for the data-driven approximation of the Perron–Frobenius (P-F) operator corresponding to the control system. This linear P-F operator for the control system is then used to formulate a convex optimization problem for control synthesis. This optimization is over polynomials and can be solved using the SOS solvers. The complexity of the resulting optimization problem depends on the polynomial basis used to approximate the linear operators. Since control often does not require high-fidelity models, we expect to construct a reliable controller using a relatively small number of basis functions. We envision that this method can be applied to low-dimensional and medium-dimensional dynamical systems (e.g., robotics, distributedpower-electronics control applications).

Recently, several methods have been developed for control synthesis [15,16]. One major difference is that [15,16] focus on polynomial dynamics, while our methods applies to more general systems. Another important difference is that we use a stronger notion of stability compared with [16]. Another line of research that is related to this work is optimal control synthesis based on generalized moment problems. One major difference between [17] and our method is that we use a rational parametrization for the control strategy.

The rest of the paper is organized as follows. In Section 2, we provide a review on density function methods, SOS, and Koopman theory; these are the components of our approach. A stronger notion of stability is discussed in Section 3. Problem formulation and the details of our method are presented in Section 4. This is followed by several numerical examples in Section 5, and a short concluding remark in Section 6.

2. Background

Our proposed method for control synthesis utilizes the density function method for controller design, SOS for polynomial optimization and Koopman theory for data-driven approximations. Necessary background information on these components is discussed in this section.

2.1. Density Function Approach for Control Synthesis

Consider a control-affine system (1) with a feedback control strategy $\mathbf{u}\left(\mathbf{x}\right)$. It is well known that the closed-loop system is asymptotically stable with respect to the origin $\mathbf{x}=0$ if a Lyapunov function V exists, such that

$${\frac{\partial V}{\partial \mathbf{x}}}^{\top }(\mathbf{F}\left(\mathbf{x}\right)+\mathbf{G}\left(\mathbf{x}\right)\mathbf{u}\left(\mathbf{x}\right))<0,\forall \mathbf{x}\ne 0.$$

(2)

Thus, for the purpose of control synthesis, one seeks a pair $(V,\mathbf{u})$ so that (2) holds. Note that this inequality is bilinear with respect to $V,\mathbf{u}$ and thus the problem is non-convex. This is a major obstacle preventing the Lyapunov theory from being widely used in control synthesis. In [1], a dual to Lyapunov’s stability theorem was established.

Theorem 1

([1]). Given the system $\dot{\mathbf{x}}=\mathbf{F}\left(\mathbf{x}\right)$, where $\mathbf{F}$ is continuously differentiable and $\mathbf{F}\left(0\right)=0$, suppose there exists a nonnegative ρ that is continuously differentiable for $\mathbf{x}\ne 0$ such that $\rho \left(\mathbf{x}\right)\mathbf{F}\left(\mathbf{x}\right)/\left|\mathbf{x}\right|$ is integrable on $\{\mathbf{x}\in {\mathbb{R}}^n:|\mathbf{x}|\ge 1\}$ and

$$[\nabla ·(\rho \mathbf{F}\left)\right]\left(\mathbf{x}\right)>0for almost all\mathbf{x}.$$

(3)

Then, for almost all initial states $\mathbf{x}\left(0\right)$, the trajectory $\mathbf{x}\left(t\right)$ tends to zero as $t\to \infty$. Moreover, if the equilibrium $\mathbf{x}=0$ is stable, then the conclusion remains valid even if ρ takes negative values.

The density $\rho$ serves as a stability certificate and can be viewed as a dual to the Lyapunov function [1]. Applying Theorem 1 to the closed-loop system, we arrive at

$$\nabla ·\left(\rho \right(\mathbf{F}+\mathbf{G}\mathbf{u}\left)\right)>0\mathrm{for} \mathrm{almost} \mathrm{all}\mathbf{x}.$$

(4)

The control synthesis problem becomes that of searching for a pair $(\rho ,\mathbf{u})$ of functions so that (4) holds. Even though (4) is again bilinear, it becomes linear in terms of $(\rho ,\rho \mathbf{u})$. Thus, the density-function-based method for control synthesis is a convex problem.

2.2. Sum of Squares

SOS [18,19,20,21] is a powerful technique to solve polynomial optimization with positive polynomial constraints. Briefly, SOS polynomials are a set of polynomials which can be described as a non-negative linear combination of a square of polynomials, that is, a polynomial of the form $p={\sum }_{i=1}^{\ell }{d}_i{p}_i^2$, where $p_i$ are polynomials and $d_i$ are non-negative coefficients. Clearly, SOS is a sufficient condition for the non-negativity of a polynomial. Hence, SOS relaxation provides a lower bound on the minimization problems of polynomial optimizations. Using the SOS relaxation, a large class of polynomial optimization problems with positive constraints can be formulated as SOS optimization:

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \underset{\mathbf{d}}{min}{\mathbf{w}}^{\top }\mathbf{d}\mathrm{s}.\mathrm{t}.p_s(\mathbf{x};\mathbf{d})\in \mathsf{\Sigma }\left[\mathbf{x}\right],p_e(\mathbf{x};\mathbf{d})=0,\end{array}\end{array}$$

(5)

where $\mathsf{\Sigma }\left[\mathbf{x}\right]$ denotes an SOS set, $\mathbf{w}$ consists of weighting coefficients, and $p_s, p_e$ are polynomials with coefficients $\mathbf{d}$. The problem in (5) can be converted into Semidefinite Programming (SDP) [19,22]. There are readily available SOS optimization packages, such as SOSTOOLS [23] and SOSOPT [24] that are designed to solve (5).

2.3. Linear Koopman and Perron–Frobenius Operators

For a dynamical system, $\dot{\mathbf{x}}=\mathbf{F}\left(\mathbf{x}\right)$, there are two different ways of linearly lifting the finite dimensional nonlinear dynamics from state space to some infinite dimension space of functions $\mathcal{F}$. They are the Koopman operator and Perron–Frobenius operator. The solution of system (1) with zero control is denoted by ${\mathbf{s}}_t\left(\mathbf{x}\right)$. The definitions of these operators, along with the corresponding infinitesimal generators, are as follows.

Definition 1

(Koopman Operator). ${\mathbb{K}}_t:\mathcal{F}\to \mathcal{F}$ for dynamical system (1) is defined as

$$\left[{\mathbb{K}}_t\phi \right]\left(\mathbf{x}\right)=\phi \left({\mathbf{s}}_t\left(\mathbf{x}\right)\right),\phi \in \mathcal{F},t\ge 0.$$

The infinitesimal generator for the Koopman operator is

$$\begin{array}{c}\hfill \underset{t\to 0}{lim}\frac{{\mathbb{K}}_t\phi -\phi }{t}=\mathbf{F}\left(\mathbf{x}\right)·\nabla \phi \left(\mathbf{x}\right)=:{\mathcal{K}}_{\mathbf{F}}\phi .\end{array}$$

(6)

Definition 2

(Perron–Frobenius Operator). ${\mathbb{P}}_t:\mathcal{F}\to \mathcal{F}$ for dynamical system (1) is defined as

$$\left[{\mathbb{P}}_t\psi \right]\left(\mathbf{x}\right)=\psi \left({\mathbf{s}}_{-t}\left(\mathbf{x}\right)\right)\left|\frac{\partial {\mathbf{s}}_{-t}\left(\mathbf{x}\right)}{\partial \mathbf{x}}\right|,\psi \in \mathcal{F},t\ge 0$$

where $\left|·\right|$ stands for the determinant. The infinitesimal generator for the P-F operator is given by

$$\begin{array}{c}\hfill \underset{t\to 0}{lim}\frac{{\mathbb{P}}_t\psi -\psi }{t}=-\nabla ·\left(\mathbf{F}\left(\mathbf{x}\right)\psi \left(\mathbf{x}\right)\right)=:{\mathcal{P}}_{\mathbf{F}}\psi .\end{array}$$

(7)

These two operators are dual to each other, where the duality is expressed as

$$\begin{array}{c}\hfill {\int }_{{\mathbb{R}}^n}\left[{\mathbb{K}}_t\phi \right]\left(\mathbf{x}\right)\psi \left(\mathbf{x}\right)d\mathbf{x}={\int }_{{\mathbb{R}}^n}\left[{\mathbb{P}}_t\psi \right]\left(\mathbf{x}\right)\phi \left(\mathbf{x}\right)d\mathbf{x}.\end{array}$$

(8)

3. Stabilization with Stronger Notion of Stability

In this section, we present a stronger notion of stability than that in Theorem 1. Consider the following dynamical system without control:

$$\begin{array}{c}\hfill \dot{\mathbf{x}}=\mathbf{F}\left(\mathbf{x}\right).\end{array}$$

(9)

Assumption 1.

Assume that $\mathbf{x}=0$ is a locally stable equilibrium point for the system (9) with a local domain of attraction denoted by $\mathcal{N}$. Let $B_{\delta }$ be the neighborhood of the origin for any given fixed $\delta >0$ such that $0\in B_{\delta }\subset \mathcal{N}$. Denote ${\mathbf{X}}_1:={\mathbb{R}}^n\setminus B_{\delta }$.

Definition 3

(Almost everywhere uniform stability). The equilibrium point $\mathbf{x}=0$ satisfying Assumption 1 is said to be almost everywhere uniform stable with respect to finite measure μ if for every $ϵ>0$ there exists a time $T\left(ϵ\right)$ such that

$$\begin{array}{c}\hfill {\int }_{T\left(ϵ\right)}^{\infty }\mu \left(A_t\right)dt<ϵ\end{array}$$

(10)

where

$$A_t:=\{\mathbf{x}\in {\mathbb{R}}^n:{\mathbf{s}}_t\left(\mathbf{x}\right)\in A\}$$

for any set $A\subset {\mathbf{X}}_1$.

The following theorem from Theorem 13 [3] provides a sufficient condition for almost everywhere uniform stability.

Theorem 2.

The equilibrium point satisfying Assumption 1 is almost everywhere uniform stable with respect to measure μ with density h if a density function $\rho \in {\mathcal{C}}^1({\mathbb{R}}^n\setminus \left\{0\right\},{\mathbb{R}}^{+})$ exists that is integrable over ${\mathbf{X}}_1$ and satisfies

$$\begin{array}{c}\hfill \nabla ·\left(\mathbf{F}\right(\mathbf{x}\left)\rho \right(\mathbf{x}\left)\right)=h\left(\mathbf{x}\right).\end{array}$$

(11)

Definition 4

(Almost everywhere geometric stability). The equilibrium point is said to be almost everywhere uniformly exponential stable if a positive constant $\beta >0$ exists and for every $ϵ>0$, a time $T\left(ϵ\right)$ exists, such that

$$\begin{array}{c}\hfill {\int }_{T\left(ϵ\right)}^{\infty }{e}^{\beta t}\mu \left(A_t\right)<ϵ,\end{array}$$

(12)

where $A_t:=\{\mathbf{x}\in {\mathbb{R}}^n:{\mathbf{s}}_t\left(\mathbf{x}\right)\in A\}$ for any set $A\subset {\mathbf{X}}_1$.

Next, we establish a sufficient condition that resembles (11) for geometric stability.

Theorem 3.

The equilibrium point $\mathbf{x}=0$ for system (9) satisfying Assumption 1 is almost everywhere stable with geometric decay with respect to measure μ with density h if a density function $\rho \left(\mathbf{x}\right)\in {\mathcal{C}}^1({\mathbb{R}}^n\setminus \left\{0\right\},{\mathbb{R}}^{+})$ exists which is integrable on ${\mathbf{X}}_1$ and satisfies

$$\begin{array}{c}\hfill \nabla ·\left(\mathbf{F}\rho \right)=\beta \rho \left(\mathbf{x}\right)+h\end{array}$$

(13)

for some positive constant $\beta >0$.

Proof. 

Equation (13) can be rewritten as

$$\begin{array}{c}\hfill \sum _{i=1}^n{\mathbf{F}}_i\left(\mathbf{x}\right){\rho }_{{\mathbf{x}}_i}+\nabla ·(\mathbf{F}\left(\mathbf{x}\right)-\frac{\beta }{n}\mathbf{x})\rho \left(\mathbf{x}\right)=h\left(\mathbf{x}\right).\end{array}$$

(14)

Since (14) is a first-order PDE, we can use the method of characteristics to obtain a solution. The characteristic curves are given by the solution of the following ODE:

$$\dot{\mathbf{x}}\left(t\right)=\mathbf{F}\left(\mathbf{x}\right).$$

(15)

Let $\tilde{\mathbf{F}}\left(\mathbf{x}\right)=\mathbf{F}\left(\mathbf{x}\right)-\frac{\beta }{n}\mathbf{x}$, then Equation (14) can be rewritten as

$${\displaystyle \frac{d}{dt}}\rho \left({\mathbf{s}}_t\left(\mathbf{x}\right)\right)+\rho \left({\mathbf{s}}_t\left(x\right)\right)\nabla ·\tilde{\mathbf{F}}=h\left({\mathbf{s}}_t\left(\mathbf{x}\right)\right),$$

(16)

which is a first-order ODE in the t variable.

The solution to (16) is obtained by multiplying (16) by the integrating factor $exp\{{\int }_0^t\nabla ·\tilde{\mathbf{F}}\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau \}$, which points to

$${\displaystyle \frac{d}{dt}}(\rho \left({\mathbf{s}}_t\left(\mathbf{x}\right)\right)exp\{{\int }_0^t▽·\tilde{\mathbf{F}}\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau \})=exp\{{\int }_0^t\nabla ·\tilde{\mathbf{F}}\left({\mathbf{s}}_{\tau }\left(x\right)\right)d\tau \}h\left({\mathbf{s}}_t\left(\mathbf{x}\right)\right).$$

(17)

It follows

$$exp\{{\int }_0^t\nabla ·\tilde{\mathbf{F}}\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau \}\rho \left({\mathbf{s}}_t\left(\mathbf{x}\right)\right)=\rho \left({\mathbf{s}}_0\left(\mathbf{x}\right)\right)+{\int }_0^{t}exp\{{\int }_0^s\nabla ·\tilde{\mathbf{F}}\left(\mathbf{x}\left(\tau \right)\right)d\tau \}h\left({\mathbf{s}}_s\left(\mathbf{x}\right)\right)ds.$$

(18)

In view of $|\frac{d\left({\mathbf{s}}_t\left(\mathbf{x}\right)\right)}{d\mathbf{x}}|=exp\{{\int }_0^t\nabla ·\mathbf{F}\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau \}$ and

$$exp\{{\int }_0^t\nabla ·\tilde{\mathbf{F}}\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau \}=exp\{-\beta t\}exp\{{\int }_0^t\nabla ·\mathbf{F}\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau \}$$

we obtain

$$exp\{-\beta t\}|\frac{d\left({\mathbf{s}}_t\left(\mathbf{x}\right)\right)}{d\mathbf{x}}|\rho \left({\mathbf{s}}_t\left(\mathbf{x}\right)\right)=\rho \left({\mathbf{s}}_0\left(\mathbf{x}\right)\right)+{\int }_0^{t}exp\{-\beta \tau \}\left|\frac{d\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)}{d\mathbf{x}}\right|h\left({\mathbf{s}}_{\tau }\left(\mathbf{x}\right)\right)d\tau .$$

(19)

Now, using the fact that ${\mathbf{s}}_0\left(\mathbf{x}\right)=\mathbf{x}$, performing change of variable $\mathbf{y}={\mathbf{s}}_t\left(\mathbf{x}\right)\Rightarrow {\mathbf{s}}_{-t}\left(\mathbf{y}\right)=\mathbf{x}$, by the definition of P-F operator, we establish

$$\begin{array}{c}\hfill \rho \left(\mathbf{x}\right)=exp\left\{\beta t\right\}\left[{\mathbb{P}}_t\rho \right]\left(\mathbf{x}\right)+{\int }_0^{t}exp\left\{\beta (t-\tau )\right\}\left[{\mathbb{P}}_{t-\tau }h\right]\left(\mathbf{x}\right)d\tau .\end{array}$$

(20)

Integrating Equation (20) over set $A\subset {\mathbf{X}}_1$ yields

$${\int }_A\rho \left(\mathbf{x}\right)d\mathbf{x}={\int }_{A}exp\left\{\beta t\right\}\left[{\mathbb{P}}_t\rho \right]\left(\mathbf{x}\right)d\mathbf{x}+{\int }_0^t{\int }_{A}exp\left\{\beta (t-\tau )\right\}\left[{\mathbb{P}}_{t-\tau }h\right]\left(\mathbf{x}\right)d\mathbf{x}d\tau .$$

It follows that

$${\int }_A\rho \left(\mathbf{x}\right)d\mathbf{x}={\int }_{A}exp\left\{\beta t\right\}\left[{\mathbb{P}}_t\rho \right]\left(\mathbf{x}\right)d\mathbf{x}+{\int }_0^t{\int }_{A}exp\left\{\beta \tau \right\}\left[{\mathbb{P}}_{\tau }h\right]\left(\mathbf{x}\right)d\tau .$$

Since the P-F operator preserves positivity and $\rho ,h$ are both positive, we have

$${\int }_0^t{\int }_{A}exp\left\{\beta \tau \right\}\left[{\mathbb{P}}_{\tau }h\right]\left(\mathbf{x}\right)d\mathbf{x}d\tau <{\int }_A\rho \left(\mathbf{x}\right)d\mathbf{x}<\infty$$

(21)

for any $t>0$. We thus conclude

$${\int }_0^{\infty }exp\left\{\beta \tau \right\}\mu \left(A_{\tau }\right)d\tau <\infty .$$

The geometric stability then follows. This completes the proof. □

Apparently, condition (13) is stronger than (11). The latter is a special case of the former when $\beta =0$. Moreover, both of them imply (3) if $h>0$ in the region of interest. Thus, in this work, we seek an efficient algorithm to design a controller for (1) so that the closed-loop dynamics has geometric stability.

4. Data-Driven Numerical Algorithm for Control Synthesis

In this section, we propose a data-driven framework to solve the stability certificate in (13) without knowing the models $\mathbf{F}$ and $\mathbf{G}$ in (1) explicitly. Instead, we assume that we have access to time-series sample data from (1). The solution provides state feedback $\mathbf{u}$ that globally exponentially stabilizes (1). The core of the framework is two-fold: (i) we leverage the definition of the infinitesimal P-F generator shown in (7) to approximate the divergence terms $\nabla ·(\mathbf{F}·)$ and $\nabla ·({\mathbf{G}}_i·)$ in the stability certificate; (ii) we transform the almost everywhere geometric stability certificate described in Section 3 as an SOS optimization problem using P-F generators and rational parameterization [25].

4.1. Density Function Approach Reformulation

By (13), to find a geometrically stabilizing controller for (1), it suffices to find a pair $\left(\rho \right(\mathbf{x}),\mathbf{u}(\mathbf{x}\left)\right)$ that solves

$$\begin{array}{c}\hfill \nabla ·\left(\rho \right(\mathbf{F}+\mathbf{G}\mathbf{u}\left)\right)=\beta \rho \left(\mathbf{x}\right)+h.\end{array}$$

(22)

This is not a convex problem in terms of variable $\left(\rho \right(\mathbf{x}),\mathbf{u}(\mathbf{x}\left)\right)$, but it is convex in terms of $\left(\rho \right(\mathbf{x}),\rho \mathbf{u}(\mathbf{x}\left)\right)$.

The above is an infinite dimension problem. To establish an implementable algorithm, we first construct rational parameterization of density functions described in [25] as

$$\rho \left(\mathbf{x}\right)=\frac{a\left(\mathbf{x}\right)}{b{\left(\mathbf{x}\right)}^{\alpha }},\rho \left(\mathbf{x}\right)\mathbf{u}\left(\mathbf{x}\right)=\frac{\mathbf{c}\left(\mathbf{x}\right)}{b{\left(\mathbf{x}\right)}^{\alpha }},$$

(23)

where a and $\mathbf{c}={[c_1,\dots ,c_m]}^{\top }$ are polynomials, b is a positive polynomial (positive for any $\mathbf{x}\ne 0$), and $\alpha$ is a sufficiently large number such that $\rho \left(\mathbf{x}\right)$ is integrable over ${\mathbf{X}}_1$. One choice of b is the quadratic control Lyapunov function corresponding to the linearized dynamics at the origin [25]. Note that the optimization variables include a and $\mathbf{c}$.

With the parametrization (23), (22) becomes

$$\nabla ·\left[\frac{1}{b^{\alpha }}(\mathbf{F}a+\mathbf{G}\mathbf{c})\right]=\frac{\beta a}{b^{\alpha }}+h.$$

(24)

Rearranging the terms and using the fact that $h>0$, we establish the SOS condition:

$$(1+\alpha )b\nabla ·(\mathbf{F}a+\mathbf{G}\mathbf{c})-\alpha \nabla ·(b\mathbf{F}a+b\mathbf{G}\mathbf{c})-\beta ab>0.$$

(25)

4.2. Data-Driven Approximation of Linear Operators

For the data-driven approximation of Koopman operators and subsequent P-F operators, we adopt the algorithmic techniques in [12,26,27]. Specifically, we leverage the numerical algorithm in [27] to directly approximate Koopman generators. For this, we first collect time-series data from the dynamical system in (1) by feeding different control inputs: (i) zero inputs, $\mathbf{u}=0$, and (ii) unit step inputs, $\mathbf{u}={\mathbf{e}}_i$ (${\mathbf{e}}_i\in {\mathbb{R}}^m$ denotes unit vectors, i.e., ith entry of ${\mathbf{e}}_i$ is 1, otherwise 0.) for $i=1,\dots ,m$ for a finite time horizon with sampling stepsize $\delta t$ in the matrices

$$\begin{array}{c}\hfill {\mathbf{X}}_i=\left[{\mathbf{x}}_1,\dots ,{\mathbf{x}}_{T_i}\right],{\dot{\mathbf{X}}}_i=\left[{\dot{\mathbf{x}}}_1,\dots ,{\dot{\mathbf{x}}}_{T_i}\right],\end{array}$$

(26)

with $i=0,1,\dots ,m$ for zero and step control inputs, where $T_i$ are the number of data points for the ith input case. Time derivatives of the states $\dot{\mathbf{x}}$ can be accurately estimated using numerical algorithms, as shown in [28,29]. Additionally, the pair $\{\mathbf{x},\dot{\mathbf{x}}\}$ in (26) do not have to be from a single trajectory; it can be a concatenation of multiple experiment/simulation trajectories.

Next, we construct a polynomial basis vector

$$\begin{array}{c}\hfill \mathsf{\Psi }\left(\mathbf{x}\right)={[{\psi }_1\left(\mathbf{x}\right),\dots ,{\psi }_Q\left(\mathbf{x}\right)]}^{\top },\end{array}$$

(27)

which can be monomials or Legendre/Hermite polynomials. The time derivative of $\mathsf{\Psi }\left(\mathbf{x}\right)$ is

$$\begin{array}{c}\hfill \dot{\mathsf{\Psi }}(\mathbf{x},\dot{\mathbf{x}})={[{\dot{\psi }}_1(\mathbf{x},\dot{\mathbf{x}}),\dots ,{\dot{\psi }}_Q(\mathbf{x},\dot{\mathbf{x}})]}^{\top },\end{array}$$

(28)

where ${\dot{\psi }}_k(\mathbf{x},\dot{\mathbf{x}})={(\nabla {\psi }_k)}^{\top }\dot{\mathbf{x}}={\sum }_{j=1}^n\frac{\partial {\psi }_k}{\partial x_j}\frac{d{x}_j}{dt}$. Then, the Koopman generator for each input case denoted by ${\mathbf{L}}_i$ can be approximated as

$${\mathbf{L}}_i=\underset{{\mathbf{L}}_i}{\mathrm{argmin}}\left|\right|{\mathbf{B}}_i-{\mathbf{A}}_i{\mathbf{L}}_i{\left|\right|}_F,$$

(29)

where

$$\begin{array}{ccc}\hfill {\mathbf{A}}_i& =& \frac{1}{T_i}{\sum }_{\ell =1}^{T_i}\mathsf{\Psi }\left({\mathbf{X}}_{i,\ell }\right)\mathsf{\Psi }{\left({\mathbf{X}}_{i,\ell }\right)}^{\top },\hfill \\ \hfill {\mathbf{B}}_i& =& \frac{1}{T_i}{\sum }_{\ell =1}^{T_i}\mathsf{\Psi }\left({\mathbf{X}}_{i,\ell }\right)\dot{\mathsf{\Psi }}{({\mathbf{X}}_{i,\ell },{\dot{\mathbf{X}}}_{i,\ell })}^{\top },\hfill \end{array}$$

and ${\mathbf{X}}_{i,\ell }$ and ${\dot{\mathbf{X}}}_{i,\ell }$ denote the ℓth snapshot of the time-series data in ${\mathbf{X}}_i$ and ${\dot{\mathbf{X}}}_i$, respectively. The solution of (29) is explicitly known, ${\mathbf{K}}_i={\mathbf{A}}_i^{†}{\mathbf{B}}_i$, where † stands for pseudo-inverse.

With the approximations ${\mathbf{L}}_i$, the Koopman generator for zero input $(i=0)$ is given by

$$\begin{array}{c}\hfill {\mathcal{K}}_{\mathbf{F}}={\mathbf{L}}_0.\end{array}$$

(30)

In addition, using the linearity of Koopman operator, Koopman generators for each step inputs $(i=1,\dots ,m)$ are given by

$$\begin{array}{c}\hfill {\mathcal{K}}_{{\mathbf{G}}_i}={\mathbf{L}}_i-{\mathbf{L}}_0.\end{array}$$

(31)

The above is one method to estimate ${\mathcal{K}}_{\mathbf{F}}$ and ${\mathcal{K}}_{{\mathbf{G}}_i}$. They can also be approximated jointly by using trajectories subject to arbitrary inputs and solving a single least square optimization problem.

The P-F generator for vector field $\mathbf{F}$ can be written as

$$\begin{array}{c}\hfill -{\mathcal{P}}_{\mathbf{F}}\psi =\nabla ·\left(\mathbf{F}\psi \right)=\mathbf{F}·\nabla \psi +\nabla ·\mathbf{F}\psi ={\mathcal{K}}_{\mathbf{F}}\psi +\nabla ·\mathbf{F}\psi .\end{array}$$

(32)

The divergence of $\mathbf{F}$ in (32) can be approximated as

$$\begin{array}{c}\hfill \nabla ·\mathbf{F}=\nabla ·{[{\mathcal{K}}_{\mathbf{F}}{x}_1,\dots ,{\mathcal{K}}_{\mathbf{F}}{x}_n]}^{\top }\approx \nabla ·\left({\mathbf{𝓒}}_x^{\top }{\mathbf{L}}_0\mathsf{\Psi }\right),\end{array}$$

(33)

where ${\mathbf{𝓒}}_x$ is a vector of coefficients such that $\mathbf{x}={\mathbf{𝓒}}_x^{\top }\mathsf{\Psi }$. This can be found easily if $\mathsf{\Psi }$ includes all the first order monomials. Similarly, the divergence of ${\mathbf{G}}_i$ are approximated as

$$\begin{array}{c}\hfill \nabla ·\left({\mathbf{G}}_j\right)\approx \nabla ·\left({\mathbf{𝓒}}_x^{\top }{\mathbf{L}}_i\mathsf{\Psi }\right),i=1,\dots ,m.\end{array}$$

(34)

Using (30)–(34), P-F generators are approximated by

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathbf{P}}_i& ={\mathbf{L}}_i+\nabla ·\left({\mathbf{𝓒}}_x^{\top }{\mathbf{L}}_i\mathsf{\Psi }\right)\mathbf{I}\hfill \end{array}\end{array}$$

(35)

with $\mathbf{I}$ denoting the identify matrix.

4.3. Convex Control Synthesis: Combining SOS with Koopman

Using approximated infinitesimal P-F generators in (35), the condition (25) reads

$$\begin{array}{cc}\hfill & (1+\alpha )b\left(\mathbf{x}\right)\left({\mathbf{𝓒}}_a^{\top }{\mathbf{P}}_0\mathsf{\Psi }\left(\mathbf{x}\right)+{\sum }_{j=1}^m{\mathbf{𝓒}}_{c_j}^{\top }{\mathbf{P}}_j\mathsf{\Psi }\left(\mathbf{x}\right)\right)\hfill \\ \hfill & -\alpha \left({\mathbf{𝓒}}_{ab}{\mathbf{P}}_0\mathsf{\Psi }\left(\mathbf{x}\right)+{\sum }_{j=1}^m{\mathbf{𝓒}}_{b{c}_j}^{\top }{\mathbf{P}}_j\mathsf{\Psi }\left(\mathbf{x}\right)\right)-\beta a\left(\mathbf{x}\right)b\left(\mathbf{x}\right)>0.\hfill \end{array}$$

(36)

Here, ${\mathbf{𝓒}}_a,{\mathbf{𝓒}}_{c_j},{\mathbf{𝓒}}_{ab},{\mathbf{𝓒}}_{b{c}_j}$ denote the coefficients of $a\left(\mathbf{x}\right),c_j\left(\mathbf{x}\right),a\left(\mathbf{x}\right)b\left(\mathbf{x}\right),c_j\left(\mathbf{x}\right)b\left(\mathbf{x}\right)$, respectively, with respect to the basis $\mathsf{\Psi }$. Thus, our control synthesis problem can be formulated as a SOS feasibility problem

$$\begin{array}{c}\hfill \begin{array}{c}\hfill \mathrm{Find}\mathbf{d}\mathrm{subject}\mathrm{to}\left(36\right)\in \mathsf{\Sigma }\left[\mathbf{x}\right],{\mathbf{𝓒}}_a^{\top }\mathsf{\Psi }\left(\mathbf{x}\right)\in \mathsf{\Sigma }\left[\mathbf{x}\right],\end{array}\end{array}$$

(37)

where $\mathbf{d}$ collects all coefficients of the polynomials $a\left(\mathbf{x}\right)$ and $\mathbf{c}\left(\mathbf{x}\right)$. The last term in (37) reflects the constraint, $\rho >0$.

Subsequent to solving (37), we can construct the controller by $u_j\left(\mathbf{x}\right)=c_j\left(\mathbf{x}\right)/a\left(\mathbf{x}\right)$, $j=1,\dots ,m$ to stabilize the dynamical system (1).

5. Numerical Case Studies

In this section, we provide several numerical examples to illustrate the proposed method. In particular, the second example is for a non-polynomial dynamical system and the last example is for a rigid body dynamical system with state dimension 6.

5.1. Van der Pol Oscillator

The dynamics of a Van der Pol Oscillator is [30]

$$\begin{array}{c}\hfill {\dot{x}}_1=x_2,{\dot{x}}_2=(1-x_1^2)x_2-x_1+u.\end{array}$$

We collect time-series data points for each input case by performing repeated simulations with time spans from 0 to $0.01$ s and time step $\delta t=0.01$ s, starting from $2\times {10}^4$ random initial points in $[x_1,x_2]={[-5,5]}^2$. The total data points for each input response case are $T_0=\mathrm{19,952}$, $T_1=\mathrm{19,958}$. The polynomial $b\left(\mathbf{x}\right)$ is chosen as an LQR solution associated with the linearized system at the origin. The value of $\alpha$ is set to be $\alpha =4$. The optimization variables $a\left(\mathbf{x}\right)$ and $\mathbf{c}\left(\mathbf{x}\right)$ are polynomials with degrees ranging from 0 to 2 and from 0 to 4, respectively. The basis $\mathsf{\Psi }\left(\mathbf{x}\right)$ is chosen to be Legendre polynomials up to 15th order. Figure 1 shows the results of the control synthesized by following the proposed method described in Section 4 for $\beta =0$ and $\beta =1$. Clearly, the case with geometric stable term ($\beta =1$) converges more aggressively to the origin.

5.2. Non-Polynomial System Example: Inverted Pendulum

The dynamics of a simple two-dimensional inverted pendulum is

$$\begin{array}{c}\hfill {\dot{x}}_1=x_2,{\dot{x}}_2=\sin x_1-0.5x_2+u,\end{array}$$

which is non-polynomial due to a sinusoidal function. We collect time-series data points by performing repeated simulations, from 0 to $0.01$ s with time step $\delta t=0.01$ s, starting from ${10}^4$ random initial points from $[x_1,x_2]={[-\pi ,\pi ]}^2$. The number of data points for both input response cases is $T_0=T_1=9989$. The value of $\alpha$ is set to be $\alpha =4$. The polynomial $b\left(\mathbf{x}\right)$ is an LQR solution associated with the linearized system at the origin. The optimization variables $a\left(\mathbf{x}\right)$ and $\mathbf{c}\left(\mathbf{x}\right)$ are polynomials with degrees from 0 to 2 and from 0 to 6, respectively. The basis $\mathsf{\Psi }\left(\mathbf{x}\right)$ consists of monomials up to the 10th order. Figure 2 shows the results of the synthesized control for $\beta =0$ and $\beta =7$, demonstrating that the control solutions from the proposed method can effectively stabilize non-polynomial dynamical systems, and this is also the case with geometric stable term ($\beta =7$), which can stabilize the system more aggressively.

5.3. Lorenz System Dynamics

The dynamics of Lorenz attractor is given by [26]

$$\begin{array}{cc}\hfill {\dot{x}}_1& ={\sigma }_1(x_2-x_1),\hfill \\ \hfill {\dot{x}}_2& =x_1({\sigma }_2-x_3)-x_2+u,\hfill \\ \hfill {\dot{x}}_3& =x_1{x}_2-{\sigma }_3{x}_3,\hfill \end{array}$$

where the parameters are set to be ${\sigma }_1=10$, ${\sigma }_2=28$, and ${\sigma }_3=\frac{8}{3}$. We collect time-series data points from repeated simulations, from 0 to $0.001$ s, with time step $\delta t=0.001$ s, starting from random initial points in $[x_1,x_2,x_3]={[-5\times 5]}^3$. The data points collected for all input cases have $T_0=T_1=9949$ snapshots. For the parameters of stability conditions, we choose $\alpha =4$ and $b\left(\mathbf{x}\right)$ to be the LQR solution for the linearized system. The optimization variables $a\left(\mathbf{x}\right)$ and $\mathbf{c}\left(\mathbf{x}\right)$ are polynomials with degrees ranging from 0 to 2 and from 0 to 6, respectively. The basis $\mathsf{\Psi }\left(\mathbf{x}\right)$ consists of Legendre polynomials up to the 10th order. Figure 3 shows the results of the synthesized controls for $\beta =0$ and $\beta =3$. We can observe that the chaotic dynamics of the Lorenz attractor is stabilized to the origin by the control synthesized by our proposed method, and furthermore, the geometric stable term ($\beta =3$) stabilizes the system more aggressively.

5.4. Rigid Body Control

Consider a rigid body system [25]

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \dot{\mathbf{\omega }}& ={\mathbf{J}}^{-1}S\left(\mathbf{\omega }\right)\mathbf{J}\mathbf{\omega }+{\mathbf{J}}^{-1}\mathbf{u},\hfill \\ \hfill \dot{\mathbf{\psi }}& =\mathbf{H}\left(\mathbf{\psi }\right)\mathbf{\omega },\hfill \end{array}\end{array}$$

(38)

where the angular velocity vector $\mathbf{\omega }\in {\mathbb{R}}^3$, Rodrigues parameter vector $\mathbf{\psi }\in {\mathbb{R}}^3$, and control torque $\mathbf{u}\in {\mathbb{R}}^3$. The explicit form of the parameters can be found in [25]. The dimension of the state space is 6. Time-series data points are sampled from repeated simulations with a time span from 0 to $0.001$ s with time step $\delta t=0.001$ s, starting from uniformly distributed random initial points, $[{\mathbf{\omega }}^{\top },{\mathbf{\psi }}^{\top }]={[-3\times 3]}^6$. Each data matrix, ${\mathbf{X}}_{1\sim 4}$, ${\dot{\mathbf{X}}}_{1\sim 4}$ has 9990 snapshots. The value of $\alpha$ is set to be $\alpha =4$. The polynomial b is chosen to be $b\left(\mathbf{x}\right)={|\mathbf{\omega }+\mathbf{\psi }|}^2+{\left|\mathbf{\psi }\right|}^2$, which is known to be a CLF of the linearized dynamics of (38) from [25]. Degrees of $a\left(\mathbf{x}\right)=1$ and $c_j\left(\mathbf{x}\right)$ are chosen to be from 0 to 1 and from 0 to 4, respectively. Figure 4 shows the trajectories of the states ${\mathbf{\omega }}_{1\sim 3}$ and ${\mathbf{\psi }}_{1\sim 3}$ starting from some random initial points, stabilized by the proposed method for $\beta =0$ (left) and $\beta =10$ (right). Clearly, the case with $\beta =10$ has a faster convergence property.

6. Concluding Remark

A systematic convex optimization-based framework is provided for the data-driven stabilization of control-affine nonlinear systems. The proposed approach relies on a combination of SOS optimization methods and recent advances in the data-driven computation of the Koopman operator. Future research efforts will focus on data-driven optimal control of the nonlinear system and the robust counterpart of this work by exploiting the sample complexity of Koopman and P-F operators [31].