Data Driven Economic Model Predictive Control

Abstract

This manuscript addresses the problem of data driven model based economic model predictive control (MPC) design. To this end, first, a data-driven Lyapunov-based MPC is designed, and shown to be capable of stabilizing a system at an unstable equilibrium point. The data driven Lyapunov-based MPC utilizes a linear time invariant (LTI) model cognizant of the fact that the training data, owing to the unstable nature of the equilibrium point, has to be obtained from closed-loop operation or experiments. Simulation results are first presented demonstrating closed-loop stability under the proposed data-driven Lyapunov-based MPC. The underlying data-driven model is then utilized as the basis to design an economic MPC. The economic improvements yielded by the proposed method are illustrated through simulations on a nonlinear chemical process system example.

1. Introduction

Control systems designed to manage chemical process operations often face numerous challenges such as inherent nonlinearity, process constraints and uncertainty. Model predictive control (MPC) is a well-established control method that can handle these challenges. In MPC, the control action is computed by solving an open-loop optimal control problem at each sampling instance over a time horizon, subject to the model that captures the dynamic response of the plant, and constraints [1]. In early MPC designs, the objective function was often utilized as a parameter to ensure closed-loop stability. In subsequent contributions, Lyapunov-based MPC was proposed where feasibility and stability from a well characterized region was built into the MPC [2,3].

With increasing recognition (and ability) of MPC designs to focus on economic objectives, the notion of Economic MPC (EMPC) was developed for linear and nonlinear systems [4,5,6], and several important issues (such as input rate-of-change constraint and uncertainty) addressed. The key idea with the EMPC designs is the fact that the controller is directly given the economic objective to work with, and the controller internally determines the process operation (including, if needed, a set point) [7].

Most of the existing MPC formulations, economic or otherwise, have been illustrated using first principles models. With growing availability of data, there exists the possibility of enhancing MPC implementation for situations where a first principles model may not be available, and simple ‘step-test’, transfer-function based model identification approaches may not suffice. One of the widely utilized approaches in the general direction of model identification are latent variable methods, where the correlation between subsequent measurements is used to model and predict the process evolution [8,9]. In one direction, Dynamic Mode Decomposition with control (DMDc) has been utilized to extract low-order models from high-dimensional, complex systems [10,11]. In another direction, subspace-based system identification methods have been adapted for the purpose of model identification, where state-space model from measured data are identified using projection methods [12,13,14]. To handle the resultant plant model mismatch with data-driven model based approaches, monitoring of the model validity becomes especially important.

One approach to monitor the process is to focus on control performance [15], where the control performance is monitored and compared against a benchmark control design. To focus more explicitly on the model behavior, in a recent result [16], an adaptive data-driven MPC was proposed to evaluate model prediction performance and trigger model identification in case of poor model prediction. In another direction, an EMPC using empirical model was proposed [17]. The approach relies on a linearization approach, resulting in closed-loop stability guarantees for regions where the plant-model mismatch is sufficiently small, and illustrate results on stabilization around nominally stable equilibrium points. In summary, data driven MPC or EMPC approaches, which utilize appropriate modeling techniques to identify data from closed-loop tests to handle operation around nominally unstable equilibrium points, remain to be addressed.

Motivated by the above considerations, in this work, we address the problem of data driven model based predictive control at an unstable equilibrium point. In order to identify a model around an unstable equilibrium point, the system is perturbed under closed-loop operation. Having identified a model, a Lyapunov-based MPC is designed to achieve local and practical stability. The Lyapunov-based design is then used as the basis for a data driven Lyapunov-based EMPC design to achieve economical goals while ensuring boundedness. The rest of the manuscript is organized as follows: first, the general mathematical description for the systems considered in this work and a representative formulation for Lyapunov-based model predictive control are presented. Then, the proposed approach for closed-loop model identification is explained. Subsequently, a Lyapunov-based MPC is designed and illustrated through a simulation example. Finally, an economic MPC is designed to consider economical objectives. The efficacy of the proposed method is illustrated through implementation on a nonlinear continuous stirred-tank reactor (CSTR) with input rate of change constraints. Finally, concluding remarks are presented.

2. Preliminaries

This section presents a brief description of the general class of processes that are considered in this manuscript, followed by closed-loop subspace identification and Lyapunov based MPC formulation.

2.1. System Description

We consider a multi-input multi-output (MIMO) controllable systems where $u\in {\mathbb{R}}^{n_u}$ denotes the vector of constrained manipulated variables, taking values in a nonempty convex subset $\mathcal{U}\subset {\mathbb{R}}^{n_u}$, where $\mathcal{U}=\left\{u\in {\mathbb{R}}^{n_u}\mid u_{\min }\le u\le u_{\max }\right\}$, $u_{\min }\in {\mathbb{R}}^{n_u}$ and $u_{\max }\in {\mathbb{R}}^{n_u}$ denote the lower and upper bounds of the input variables, and $y\in {\mathbb{R}}^{n_y}$ denotes the vector of measured output variables. In keeping with the discrete implementation of MPC, u is piecewise constant and defined over an arbitrary sampling instance k as:

$$u(t)=u(k),k\Delta t\le t<(k+1)\Delta t,$$

where $\Delta t$ is the sampling time and $x_k$ and $y_k$ denote state and output at the kth sample time. The central problem that the present manuscript addresses is that of designing a data driven modeling and control design for economic MPC.

2.2. System Identification

In this section, a brief review of a conventional subspace-based state space system identification methods is presented [16,18,19]. These methods are used to identify the system matrices for a discrete-time linear time invariant (LTI) system of the following form:

$$\begin{array}{cc}\hfill x_{k+1}& =A{x}_k+B{u}_k+w_k,\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill y_k& =C{x}_k+D{u}_k+v_k,\hfill \end{array}$$

(2)

where $x\in {\mathbb{R}}^{n_x}$ and $y\in {\mathbb{R}}^{n_y}$ denote the vectors of state variables and measured outputs, and $w\in {\mathbb{R}}^{n_x}$ and $v\in {\mathbb{R}}^{n_y}$ are zero mean, white vectors of process noise and measurement noise with the following covariance matrices:

$$\begin{array}{cc}\hfill E[\left(\begin{array}{c}{w}_i\\ v_j\end{array}\right)\left(\begin{array}{cc}{w}_i^T& v_j^T\end{array}\right)]& =\left(\begin{array}{cc}Q& S\\ S^T& R\end{array}\right){\delta }_{ij},\hfill \end{array}$$

(3)

where $Q\in {\mathbb{R}}^{n_x\times n_x}$, $S\in {\mathbb{R}}^{n_x\times n_y}$ and $R\in {\mathbb{R}}^{n_y\times n_y}$ are covariance matrices, and ${\delta }_{ij}$ is the Kronecker delta function. The subspace-based system identification techniques utilize Hankel matrices constructed by stacking the output measurements and manipulated variables as follows:

$$\begin{array}{c}\hfill U_{1|i}=\left[\begin{array}{cccc}{u}_1& u_2& \cdots & u_j\\ u_2& u_3& \cdots & u_{j+1}\\ \cdots & \cdots & \cdots & \cdots \\ u_i& u_{i+1}& \cdots & u_{i+j-1}\end{array}\right],\end{array}$$

(4)

where i is a user-specified parameter that limits the maximum order of the system (n), and, j is determined by the number of sample times of data. By using Equation (4), the past and future Hankel matrices for input and output are defined:

$$\begin{array}{c}\hfill U_p=U_{1|i},U_f=U_{1|i},Y_p=Y_{1|i},Y_f=Y_{1|i}.\end{array}$$

(5)

Similar block-Hankel matrices are made for process and measurement noises $V_p,V_f\in {\mathbb{R}}^{i{n}_y\times j}$ and $W_p,W_f\in {\mathbb{R}}^{i{n}_x\times j}$ are defined in the similar way. The state sequences are defined as follows:

$$\begin{array}{cc}\hfill X_p& =\left[\begin{array}{cccc}{x}_1& x_2& \cdots & x_j\end{array}\right],\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill X_f& =\left[\begin{array}{cccc}{x}_{i+1}& x_{i+2}& \cdots & x_{i+j}\end{array}\right].\hfill \end{array}$$

(7)

Furthermore, these matrices are used in the algorithm:

$$\begin{array}{c}\hfill {\Psi }_p=\left[\begin{array}{c}{Y}_p\\ U_p\end{array}\right],{\Psi }_f=\left[\begin{array}{c}{Y}_f\\ U_f\end{array}\right],{\Psi }_{pr}=\left[\begin{array}{c}{R}_f\\ {\Psi }_p\end{array}\right].\end{array}$$

(8)

By recursive substitution into the state space model equations Equations (1) and (2), it is straightforward to show:

$$\begin{array}{cc}\hfill Y_f& ={\Gamma }_i{X}_f+{\Phi }_i^d{U}_f+{\Phi }_i^s{W}_f+V_f,\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill Y_p& ={\Gamma }_i{X}_p+{\Phi }_i^d{U}_p+{\Phi }_i^s{W}_p+V_p,\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill X_f& =A^i{X}_p+{\Delta }_i^d{U}_p+{\Delta }_i^s{W}_p,\hfill \end{array}$$

(11)

where:

$$\begin{array}{cc}\hfill {\Gamma }_i& =\left[\begin{array}{c}C\\ CA\\ C{A}^2\\ ⋮\\ C{A}^{i-1}\end{array}\right],{\Phi }_i^d=\left[\begin{array}{ccccc}D& 0& 0& \cdots & 0\\ CB& D& 0& \cdots & 0\\ CAB& CB& D& \cdots & 0\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ C{A}^{i-2}B& C{A}^{i-3}B& C{A}^{i-4}B& \cdots & D\end{array}\right],\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\Phi }_i^s& =\left[\begin{array}{cccccc}0& 0& 0& \cdots & 0& 0\\ C& 0& 0& \cdots & 0& 0\\ CA& C& 0& \cdots & 0& 0\\ \cdots & \cdots & \cdots & \cdots & 0& 0\\ C{A}^{i-2}& C{A}^{i-3}& C{A}^{i-4}& \cdots & C& 0\end{array}\right],\hfill \end{array}$$

(13)

$$\begin{array}{cc}\hfill {\Delta }_i^d& =\left[\begin{array}{ccccc}{A}^{i-1}B& A^{i-2}B& \cdots & AB& B\end{array}\right],{\Delta }_i^s=\left[\begin{array}{ccccc}{A}^{i-1}& A^{i-2}& \cdots & A& I\end{array}\right].\hfill \end{array}$$

(14)

Equation (9) can be rewritten in the following form to have the input and output data at the left hand side of the equation [20]:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}I& -{\Phi }_i^d\end{array}\right]\left[\begin{array}{c}{Y}_f\\ U_f\end{array}\right]={\Gamma }_i{X}_f+{\Phi }_i^s{W}_f+V_f.\end{array}$$

(15)

In open loop identification methods, in the next step, by orthogonal projecting of Equation (15) onto ${\Psi }_p$:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}I& -{\Phi }_i^d\end{array}\right]{\Psi }_f/{\Psi }_p={\Gamma }_i{X}_f/{\Psi }_p.\end{array}$$

(16)

Note that, the last two terms in RHS of Equation (15) are eliminated since the noise terms are independent, or othogonal to the future inputs. Equation (16) indicates that:

$$\begin{array}{c}\hfill Column\_Space(W_f/W_p)=Column\_Space({({{{\Gamma }_i}^{\perp }}^T\left[\begin{array}{cc}I& -H_i^d\end{array}\right])}^T).\end{array}$$

(17)

Therefore, ${\Gamma }_i$ and $H_i^d$ can be calculated using Equation (17) by decomposition methods. These can in turn be utilized to determine the system matrices (some of these details are deferred to Section 3.1). For further discussion on system matrix extraction, the readers are referred to references [18,19].

2.3. Lyapunov-Based MPC

The Lyapunov-based MPC (LMPC) for linear system has the following form:

$$\begin{array}{cc}\hfill \underset{{\tilde{u}}_k,\dots ,{\tilde{u}}_{k+P}}{min}& {\displaystyle \sum _{j=1}^{N_y}}||{\tilde{y}}_{k+j}-y_{k+j}^{\mathrm{SP}}{||}_{Q_y}^2+{\displaystyle \sum _{j=1}^{N_u}}||{\tilde{u}}_{k+j}-{\tilde{u}}_{k+j-1}{||}_{R_{du}}^2,\hfill \end{array}$$

(18)

$$\begin{array}{c}\hfill \mathrm{subject}\mathrm{to}:\end{array}$$

(19)

$$\begin{array}{c}{\tilde{x}}_{k+1}=A{\tilde{x}}_k+B{\tilde{u}}_k,\hfill \end{array}$$

(20)

$$\begin{array}{c}{\tilde{y}}_k=C{\tilde{x}}_k+D{\tilde{u}}_k,\hfill \end{array}$$

(21)

$$\begin{array}{c}\tilde{u}\in \mathcal{U},\Delta \tilde{u}\in {\mathcal{U}}_{\circ },\tilde{x}(k)={\widehat{x}}_l,\hfill \end{array}$$

(22)

$$\begin{array}{c}V({\tilde{x}}_{k+1})\le \alpha V({\tilde{x}}_k)\forall V({\tilde{x}}_k)>{ϵ}^{\ast },\hfill \end{array}$$

(23)

$$\begin{array}{c}V({\tilde{x}}_{k+1})\le {ϵ}^{\ast }\forall V({\tilde{x}}_k)\le {ϵ}^{\ast },\hfill \end{array}$$

(24)

where ${\tilde{x}}_{k+j}$, ${\tilde{y}}_{k+j}$, $y_{k+j}^{SP}$ and ${\tilde{u}}_{k+j}$ denote predicted state and output, output set-point and calculated manipulated input variables j time steps ahead computed at time step k, and ${\widehat{x}}_l$ is the current estimation of state, and $0<\alpha <1$ is a user defined parameter. The operator ${||.||}_Q^2$ denotes the weighted Euclidean norm defined for an arbitrary vector x and weighting matrix W as ${||x||}_W^2=x^{T}Wx$. Furthermore, $Q_y>0$ and $R_{du}\ge 0$ denote the positive definite and positive semi-definite weighting matrices for penalizing deviations in the output predictions and for the rate of change of the manipulated inputs, respectively. Moreover, $N_y$ and $N_u$ denote the prediction and control horizons, respectively, and the input rate of change, given by $\Delta {\tilde{u}}_{k+j}={\tilde{u}}_{k+j}-{\tilde{u}}_{k+j-1}$, takes values in a nonempty convex subset ${\mathcal{U}}_{\circ }\subset {\mathbb{R}}^m$, where ${\mathcal{U}}_{\circ }=\left\{\Delta u\in {\mathbb{R}}^{n_u}\mid \Delta u_{\min }\le \Delta u\le \Delta u_{\max }\right\}$. Note finally that, while the system dynamics are described in continuous time, the objective function and constraints are defined in discrete time to be consistent with the discrete implementation of the control action.

Equations (23) and (24) are representatives of Lyapunov-based stability constraint [21,22], where $V(x_k)$ is a suitable control Lyapunov function, and $\alpha ,{ϵ}^{\ast }>0$ are user-specified parameters. In the presented formulation, ${ϵ}^{\ast }>0$ enables practical stabilization to account for the discrete nature of the control implementation.

Remark 1.

Existing Lyapunov-based MPC approaches exploit the fact that the feasibility (and stability) region can be pre-determined. The feasibility region, among other things, depends on the choice of the parameter α, the requested decay factor in the value of the Lyapunov function at each time step. If (reasonably) good first principles models are available, then these features of the MPC formulation provide excellent confidence over the operating region under closed-loop. In contrast, in the presence of significant plant-model mismatch (as is possibly the case with data driven models), the imposition of such decay constraints could result in unnecessary infeasibility issues. In designing the LMPC formulation with a data driven model, this possible lack of feasibility must be accounted for (as is done in Section 3.2).

3. Integrating Lyapunov-Based MPC with Data Driven Models

In this section, we first utilize an identification approach necessary to identify good models for operation around an unstable equilibrium point. The data driven Lyapunov-based MPC design is presented next.

3.1. Closed-Loop Model Identification

Note that, when interested in identifying the system around an unstable equilibrium point, open-loop data would not suffice. To begin with, nominal open-loop operation around an unstable equilibrium point is not possible. If the nominal operation is under closed-loop, but the loop is opened to perform step tests, the system would move to the stable equilibrium point corresponding to the new input value, thereby not providing dynamic information around the desired operating point. The training data, therefore, has to be obtained using closed-loop step tests, and an appropriate closed-loop model identification method employed. Such a method is described next.

In employing closed-loop data, note that the assumption of future inputs being independent of future disturbances no longer holds, and, if not recognized, can cause biased results in system identification [18]. In order to handle this issue, the closed-loop identification approach in the projection utilizes a different variable ${\Psi }_{pr}$ instead of ${\Psi }_p$. The new instrument variable, which satisfies the independence requirement, is used to project both sides of Equation (15) and the result is used to determine LTI model matrices. For further details, refer to [16,18,23].

By projecting Equation (15) onto ${\Psi }_{pr}$ we get:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}I& -{\Phi }_i^d\end{array}\right]{\Psi }_f/{\Psi }_{pr}={\Gamma }_i{X}_f/{\Psi }_{pr}+{\Phi }_i^s{W}_f/{\Psi }_{pr}+V_f/{\Psi }_{pr}.\end{array}$$

(25)

Since the future process and measurement noises are independent of the past input/output and future setpoint in Equation (25), the noise terms cancel, resulting in:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}I& -{\Phi }_i^d\end{array}\right]{\Psi }_f/{\Psi }_{pr}={\Gamma }_i{X}_f/{\Psi }_{pr}.\end{array}$$

(26)

By multiplying Equation (26) by the extended orthogonal observability ${\Gamma }_i^{\perp }$, the state term is eliminated:

$$\begin{array}{c}\hfill {({\Gamma }_i^{\perp })}^T\left[\begin{array}{cc}I& -{\Phi }_i^d\end{array}\right]{\Psi }_f/{\Psi }_{pr}=0.\end{array}$$

(27)

Therefore, the column space of ${\Psi }_f/{\Psi }_{pr}$ is orthogonal to the row space of $\left[\begin{array}{cc}{({\Gamma }_i^{\perp })}^T& -{({\Gamma }_i^{\perp })}^T{\Phi }_i^d\end{array}\right]$. By performing singular value decomposition (SVD) of ${\Psi }_f/{\Psi }_{pr}$:

$$\begin{array}{c}\hfill {\Psi }_f/{\Psi }_{pr}=U\Sigma V=\left[\begin{array}{cc}{U}_1& U_2\end{array}\right]\left[\begin{array}{cc}{\Sigma }_1& 0\\ 0& 0\end{array}\right]\left[\begin{array}{c}{V_1}^T\\ {V_2}^T\end{array}\right],\end{array}$$

(28)

where ${\Sigma }_1$ contains dominant singular values of ${\Psi }_f/{\Psi }_{pr}$ and, theoretically, it has the order $n_{u}i+n$ [18,23].

Therefore, the order of the system can be determined by the number of the dominant singular values of the ${\Psi }_f/{\Psi }_{pr}$ [20]. The orthogonal column space of ${\Psi }_f/{\Psi }_{pr}$ is $U_{2}M$, where $M\in {\mathbb{R}}^{(n_y-n)i\times (n_y-n)i}$ is any constant nonsingular matrix and is typically chosen as an identity matrix [18,23]. One approach to determine the LTI model is as follows [18]:

$$\begin{array}{c}\hfill {(\left[\begin{array}{cc}{\Gamma }_i^{\perp }& -{\Gamma }_i^{\perp }{\Phi }_i^d\end{array}\right])}^T=U_{2}M.\end{array}$$

(29)

From Equation (29), ${\Gamma }_i$ and ${\Phi }_i^d$ can be estimated:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{{\Gamma }_i}^{\perp }\\ -{({\Phi }_i^d)}^T{{\Gamma }_i}^{\perp }\end{array}\right]=U_2,\end{array}$$

(30)

which results in (using MATLAB (2017a, MathWorks, Natick, MA, USA) matrix index notation):

$$\begin{array}{c}\hfill \left\{\begin{array}{c}{\widehat{\Gamma }}_i=U_2{(1:n_{y}i,:)}^{\perp },\hfill \\ {\widehat{\Phi }}_i^d=-{({U_2(1:n_{y}i,:)}^T)}^{†}{U_2(n_{y}i+1:end,:)}^T.\hfill \end{array}\right.\end{array}$$

(31)

The past state sequence can be calculated as follows:

$$\begin{array}{c}\hfill {\widehat{X}}_i={\widehat{\Gamma }}_i^{†}\left[\begin{array}{cc}I& -{\widehat{\Phi }}_i^d\end{array}\right]{\Psi }_f/{\Psi }_{pr}.\end{array}$$

(32)

The future state sequence can be calculated by changing data Hankel matrices as follows [18]:

$$\begin{array}{cc}\hfill R_f& =R_{i+2|2i},\hfill \end{array}$$

(33)

$$\begin{array}{cc}\hfill U_p& =U_{1|i+1},\hfill \end{array}$$

(34)

$$\begin{array}{cc}\hfill Y_p& =Y_{1|i+1},\hfill \end{array}$$

(35)

$$\begin{array}{cc}\hfill U_f& =U_{i+2|2i},\hfill \end{array}$$

(36)

$$\begin{array}{cc}\hfill Y_f& =Y_{i+2|2i},\hfill \end{array}$$

(37)

$$\begin{array}{cc}\hfill \Rightarrow {\widehat{X}}_{i+1}& ={\underset{¯}{\widehat{\Gamma }}}_i^{†}\left[\begin{array}{cc}I& -{\underset{¯}{\widehat{H}}}_i^d\end{array}\right]{\Psi }_f/{\Psi }_{pr},\hfill \end{array}$$

(38)

where ${\underset{¯}{\widehat{\Gamma }}}_i$ is obtained by eliminating the last $n_y$ rows of ${\Gamma }_i$, and ${\underset{¯}{H}}_i^d$ is obtained by eliminating the last $n_y$ rows and the last $n_u$ columns of $H_i^d$. Then, the model matrices can be estimated using least squares:

$$\begin{array}{c}\hfill \left[\begin{array}{c}{X}_{i+1}\\ Y_{i|i}\end{array}\right]=\left[\begin{array}{cc}A& B\\ C& D\end{array}\right]\left[\begin{array}{c}{X}_i\\ U_{i|i}\end{array}\right]+\left[\begin{array}{c}{W}_{i|i}\\ V_{i|i}\end{array}\right].\end{array}$$

(39)

Note that the difference between the proposed method in [18] and described method is that, in order to ensure that the observer is stable (eigenvalues of $A-KC$ are inside unit circle), instead of innovation form of LTI model, Equations (1) and (2) are used [16] to derive extended state space equations. The system matrices can be calculated as follows:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}\widehat{A}& \widehat{B}\\ \widehat{C}& \widehat{D}\end{array}\right]=\left[\begin{array}{c}{X}_{i+1}\\ Y_{i|i}\end{array}\right]{\left[\begin{array}{c}{X}_i\\ U_{i|i}\end{array}\right]}^{†}.\end{array}$$

(40)

With the proposed approach, process and measurement noise Hankel matrices can be calculated as the residual of the least square solution of Equation (39):

$$\begin{array}{c}\hfill \left[\begin{array}{c}{\widehat{W}}_{i|i}\\ {\widehat{V}}_{i|i}\end{array}\right]=\left[\begin{array}{c}{X}_{i+1}\\ Y_{i|i}\end{array}\right]-\left[\begin{array}{cc}\widehat{A}& \widehat{B}\\ \widehat{C}& \widehat{D}\end{array}\right]\left[\begin{array}{c}{X}_i\\ U_{i|i}\end{array}\right].\end{array}$$

(41)

Then, the covariances of plant noises can be estimated as follows:

$$\begin{array}{c}\hfill \left[\begin{array}{cc}\widehat{Q}& \widehat{S}\\ {\widehat{S}}^T& \widehat{R}\end{array}\right]=E(\left[\begin{array}{c}{\widehat{W}}_{i|i}\\ {\widehat{V}}_{i|i}\end{array}\right]\left[\begin{array}{cc}{\widehat{W}}_{i|i}^T& {\widehat{V}}_{i|i}^T\end{array}\right]).\end{array}$$

(42)

Model identification using closed-loop data has a positive impact on the predictive capability of the model (see the simulation section for a comparison with a model identified using open-loop data).

3.2. Control Design and Implementation

Having identified an LTI model for the system (with its associated states), the MPC implementation first requires a determination of the state estimates. To this end, an appropriate state estimator needs to be utilized. In the present manuscript, a Luenberger observer is utilized for the purpose of illustration. Thus, at the time of control implementation, state estimates ${\widehat{x}}_k$ are generated as follows:

$$\begin{array}{c}\hfill {\widehat{x}}_{k+1}=A{\widehat{x}}_k+B{u}_k+L(y_k-C{\widehat{x}}_k),\end{array}$$

(43)

where L is the observer gain and is computed using pole placement method, and $y_k$ is the vector of measured variables (in deviation form, from the set point).

In order to stabilize the system at an unstable equilibrium point, a Lyapunov-based MPC is designed. The control calculation is achieved using a two-tier approach (to decouple the problem of stability enforcement and objective function tuning). The first layer calculates the minimum value of Lyapunov function that can be reached subject to the constraints. This tier is formulated as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill V_{min}=\underset{{\tilde{u}}_k^1}{min}& (V({\tilde{x}}_{k+1})),\hfill \\ \hfill \mathrm{subject}\mathrm{to}:& \\ & {\tilde{x}}_{k+1}=A{\tilde{x}}_k+B{{\tilde{u}}^1}_k,\hfill \\ & {\tilde{y}}_k=C{\tilde{x}}_k+D{{\tilde{u}}^1}_k,\hfill \\ & {\tilde{u}}^1\in \mathcal{U},\Delta {\tilde{u}}^1\in {\mathcal{U}}_{\circ },\tilde{x}(k)={\widehat{x}}_l-x^{SP},\hfill \end{array}\end{array}$$

(44)

where $\tilde{x}$, $\tilde{y}$ are predicted state and output and ${\tilde{u}}^1$ is the candidate input computed in the first tier. $x^{SP}$ is underlying state setpoint (in deviation form from the nominal equilibrium point), which here is the desired unstable equilibrium point (and therefore zero in terms of deviation variables). For setpoint tracking, this value can be calculated using the target calculation method; readers are referred to [24] for further details.

Note that the first tier has a prediction horizon of 1 because the objective is to only compute the immediate control action that would minimize the value of the Lyapunov function at the next time step. V is chosen as a quadratic Lyapunov function with the following form:

$$\begin{array}{c}\hfill V(\tilde{x})={\tilde{x}}^{T}P\tilde{x},\end{array}$$

(45)

where P is a positive definite matrix computed by solving the Riccati equation with the LTI model matrices as follows:

$$\begin{array}{c}\hfill A^{T}PA-P-A^{T}PB{(B^{T}PB+R)}^{-1}+Q=0,\end{array}$$

(46)

where $Q\in {\mathbb{R}}^{n_x\times n_x}$ and $R\in {\mathbb{R}}^{n_u\times n_u}$ are positive definite matrices. Then, in the second tier, this minimum value is used as a constraint (upper bound for Lyapunov function value at the next time step). The second tier is formulated as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{{\tilde{u}}_k^2,\dots ,{\tilde{u}}_{k+N_p}^2}{min}& {\displaystyle \sum _{j=1}^{N_y}}||{\tilde{y}}_{k+j}-{\tilde{y}}_{k+j}^{\mathrm{SP}}{||}_{Q_y}^2+||{\tilde{u}}_{k+j}^2-{\tilde{u}}_{k+j-1}^2{||}_{R_{du}}^2,\hfill \\ \hfill \mathrm{subject}\mathrm{to}:& \\ & {\tilde{x}}_{k+1}=A{\tilde{x}}_k+B{\tilde{u}}_k,\hfill \\ & {\tilde{y}}_k=C{\tilde{x}}_k+D{\tilde{u}}_k,\hfill \\ & {\tilde{u}}^2\in \mathcal{U},\Delta {\tilde{u}}^2\in {\mathcal{U}}_{\circ },\tilde{x}(k)={\widehat{x}}_l,\hfill \\ & V({\tilde{x}}_{k+1})\le V_{min}\forall V({\tilde{x}}_k)>{ϵ}^{\ast },\hfill \\ & V({\tilde{x}}_{k+1})\le {ϵ}^{\ast }\forall V({\tilde{x}}_k)\le {ϵ}^{\ast }\hfill \end{array}\end{array}$$

(47)

where $N_p$ is the prediction horizon and ${\tilde{u}}^2$ denotes the control action computed by the second tier. In essence, in the second tier, the controller calculates a control action sequence that can take the process to the setpoint in an optimal fashion optimally while ensuring that the system reaches the minimum achievable Lyapunov function value at the next time step. Note that, in both of the tiers, the input sequence is a decision variable in the optimization problem, but only the first value of the input sequence of the second tier is implemented on the process. The solution of the first tier, however, is used to ensure and generate a feasible initial guess for the second tier. The two-tiered control structure is schematically presented in Figure 1.

Remark 2.

Note that Tiers 1 and 2 are executed in series and at the same time, and the implementation does not require a time scale separation. The overall optimization is split into two tiers to guarantee feasibility of the optimization problem. In particular, the first tier computes an input move with the objective function only focusing on minimizing the Lyapunov function value at the next time step. Notice that the constraints in the first tier are such that the optimization problem is guaranteed to be feasible. With this feasible solution, the second tier is used to determine the input trajectory that achieves the best performance, while requiring the Lyapunov function to decay. Again, since the second tier optimization problem uses the solution from Tier 1 to impose the stability constraint, feasibility of the second tier optimization problem, and, hence, of the MPC optimization problem, is guaranteed. In contrast, if one were to require the Lyapunov function to decay by an arbitrary chosen factor, determination of that factor in a way that guarantees feasibility of the optimization problem would be a non-trivial task.

Remark 3.

It is important to recognize that, in the present formulation, feasibility of the optimization problem does not guarantee closed-loop stability. A superfluous (and incorrect) reason is as follows: the first tier computes the control action that minimizes the value of the Lyapunov function at the next step, but does not require that it be smaller than the previous time step, leading to potential destabilizing control action. The key point to realize here, however, is that if such a control action were to exist (that would lower the value of the Lyapunov function at the next time step), the optimization problem would determine that value by virtue of the Lyapunov function being the objective function, and lead to closed-loop stability. The reasons closed-loop stability may not be achieved are two: (1) the current state might be such that closed-loop stability is not achievable for the system dynamics and constraints; and (2) due to plant model mismatch, where the control action that causes the Lyapunov function to decay for the identified model does not do so for the system in question. The first reason points to a fundamental limitation due to the presence of input constraints, while the second is due to the lack of availability of the ‘correct’ system dynamics, and as such will be true in general for data driven MPC formulations. Note that inclusion of a noise/plant model mismatch term in the model may help with the predictive capability of the model, however, unless a bound on the uncertainty can be assumed, closed-loop stability can not be guaranteed.

Remark 4.

Along similar lines, consider the scenario where, based on the model, and constraints, an input value exists for which $V(x(k))<=V(x(k-1))$ is achievable. It can be readily shown that any solution computed by the first tier of the optimization problem would also result in $V(x(k))<=V(x(k-1))$ by virtue of the objective function being the Lyapunov function at the next time step. Thus, in such a case, the explicit incorporation of the constraint $V(x(k))<=V(x(k-1))$ (as is traditionally done in Lyapunov based MPC) does not help, and is not required. On the other hand, for the scenario where such an input does not exist, the inclusion of the constraint will cause the optimization problem to be infeasible. In contrast, in the proposed formulation, the MPC will compute a control action where the value of the Lyapunov function might be greater than the previous value, but greater by the smallest margin possible. The real impact of this phenomenon is in making the MPC formulation more pliable, especially when dealing with plant-model mismatch. In such scenarios, the proposed MPC continues to compute feasible (best possible, in terms of stabilizing behavior) solutions, and, should the process move into a region from where stabilization is possible, smoothly transits to computing stabilizing control action.

Remark 5.

In the current manuscript, we focus on the cases where a first principal model is not available. If a good first principles model was available, it could be utilized directly in a nonlinear MPC design, or linearized if one were to implement a linear MPC. In the case of linearization, the applicability would be limited by the region over which the linearization holds. In contrast, note that the model utilized in the present manuscript does not result from a linearization of a nonlinear model. Instead, it is a linear model, possibly with a higher number of states than the original nonlinear model, albeit identified, and applicable, over a ‘larger’ region of operation, compared to a linearized model.

Remark 6.

To account for possible plant-model mismatch, model validity can be monitored with model monitoring methods [16], resulting in appropriately triggering re-identification in case of poor model prediction. In another direction, in line with control performance monitoring approaches, the Lyapunov function value could be utilized. Thus, unacceptable increases in Lyapunov function value could be utilized as a means of triggering re-identification.

Remark 7.

As mentioned previously, in order to create rich training data around unstable operating points, closed-loop data must be generated. In turn, since open-loop methods result in biased estimation [25,26] in model identification, a suitable closed-loop identification method is utilized, and adapted to ensure that the model accurately captures the key dynamics.

4. Simulation Results

We next illustrate the proposed approach using a nonlinear CSTR example [27]. To this end, consider a CSTR where a first-order, exothermic and irreversible reaction of the form $A\stackrel{k}{\to }B$ takes place. The mass and energy conservation laws results in the following mathematical model:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\dot{C}}_A& =\frac{F}{V}(C_{A0}-C_A)-k_0{e}^{\frac{-E}{R{T}_R}}{C}_A,\hfill \\ \hfill {\dot{T}}_R& =\frac{F}{V}(T_{A0}-T_R)+\frac{(-\Delta H)}{\rho c_p}{k}_0{e}^{\frac{-E}{R{T}_R}}{C}_A+\frac{Q}{\rho c_{p}V}.\hfill \end{array}\end{array}$$

(48)

The description of the process variables and the values of the system parameters are presented in

Table 1. Variable and parameter description and values for the continuous stirred-tank reactor (CSTR) example.

Variable	Description	Unit	Value
$C_{A,S}$	Nominal Value of Concentration	$\frac{kmol}{m^3}$	$0.573$
$T_{R,S}$	Nominal Value of Reactor Temperature	K	395
F	Flow Rate	$\frac{m^3}{min}$	$0.2$
V	Volume of the Reactor	$\frac{m^3}{min}$	$0.2$
$C_{A0,S}$	Nominal Inlet Concentration	$\frac{kmol}{m^3}$	$0.787$
$k_0$	Pre-Exponential Constant	−	$72\times {10}^9$
E	Activation Energy	$\frac{kJ}{mol}$	$8.314\times {10}^4$
R	Ideal Gas Constant	$\frac{kJ}{KmolK}$	$8.314$
$T_{A0}$	Inlet Temperature	K	$352.6$
$\Delta H$	Enthalpy of the Reaction	$\frac{kJ}{Kmol}$	$4.78\times {10}^4$
$\rho$	Fluid Density	$\frac{kg}{m^3}$	${10}^3$
$c_p$	Heat Capacity	$\frac{kj}{kg.K}$	$0.239$

. The control objective is to stabilize the system at an unstable equilibrium point using inlet concentration, $C_{A_0}$, and the rate of heat input, Q, while the manipulated inputs are constrained to be within the limits $|C_{A_0}|\le 1$ kmol/m³ and $|Q|\le 9\times {10}^3$ KJ/min, and the input rate is constrained as $|\Delta C_{A_0}|\le 0.1$ Kmol/m³ and $|\Delta Q|\le 9\times 200$ KJ/min. We assume that both of the states are measured. The system has an unstable equilibrium point at $C_A=0.573$ Kmol/m³ and $T=395.3$ K. The goal is to stabilize the system at this equilibrium point. To this end, first an LTI model is identified using closed-loop data; then, an MPC is designed to stabilize the system at the unstable equilibrium point.

For system identification of the CSTR model, proportional–integral (PI) controllers (pairing $C_A$ with $C_{A,in}$ and T with Q) are implemented in the process. In particular, pseudo-random binary signals are used as set-points for PI controllers. The identified LTI model order is selected as $n=4$ and $i=12$, in order to achieve the best fit in model prediction (using cross-validation). Note that these four states are the states of the identified LTI model. When dealing with setpoint tracking, these states can be augmented with additional states and utilized as part of an offset-free MPC design. Model validation results under a different set of set-point changes from training data are presented in Figure 2 and Figure 3. The identified system is unstable with absolute eigenvalues $\left[\begin{array}{cccc}0.9311& 0.9311& 0.9998& 1.0002\end{array}\right],$ which has an eigenvalue outside unit circle. The unstable nature of the identified model is consistent with the operation of the system around the unstable equilibrium point.

For the model validation, initially, a steady state Kalman filter (gain calculated by the identification method) is utilized to update state estimate until $t=0.8$ min and after convergence of the states (gaged via convergence of the outputs), the model and the input trajectory (without the state estimator) are used to predict the future output. Figure 2 illustrates the results of the model validation, and compares against a model obtained from open-loop step pseudo-random binary sequence (PRBS) on the input. As expected, the model identified using closed-loop data predicts better.

Next, closed-loop simulation results for proposed controller and conventional MPC (i.e., MPC without Lyapunov constraint) with horizons 1 and 10 are presented in Figure 4, Figure 5, Figure 6 and Figure 7. The controllers parameters are presented in

Table 2. List of controllers parameters for the CSTR reactor.

Variable	Value
$\Delta t$	$0.2$ min
$Q_x$	$\left[\begin{array}{cc}1& 0\\ 0& 1\end{array}\right]$
$Q_{x,MPC}$	$10\times diag([1/C_{A,s},1/T_{R,S}])$
$R_{\Delta u,MPC}$	$diag([1/C_{A0,max},1/Q_{max}])$
$Q_K$	$diag([{10}^3,{10}^3])$
$R_K$	$diag([{10}^{-3},{10}^{-3}])$
${\tau }_{min}$	0
${\tau }_{max}$	5
${\epsilon }_i$	${10}^{-3}\times x_{i,Sp}$
$\Delta u_{min}$	$\left[\begin{array}{cc}-0.1& -200\end{array}\right]$
$\Delta u_{max}$	$\left[\begin{array}{cc}0.1& 200\end{array}\right]$
${ϵ}^{\ast }$	1
$V(x)$	${(x-x_{sp})}^T(x-x_{sp})$
$c_y$	${\left[\begin{array}{cc}{10}^8& 0\end{array}\right]}^T$
$c_u$	${\left[\begin{array}{cc}0& 0.1\end{array}\right]}^T$
$\rho$	$7.83\times {10}^5$

. As can be seen, the LMPC has the best performance in stabilizing the system at the unstable equilibrium point. The MPC with a horizon of 1 is not capable of stabilizing the system, and the controller with a horizon of 10 reaches the set-point later compared to the LMPC. In addition, the evolution of the subspace states indicates better performance under the proposed LMPC.

5. Data-Driven EMPC Design and Illustration

Having illustrated the ability of the LMPC to achieve stabilization, it is next utilized to achieve economical objectives while ensuring stability. The Lyapunov based EMPC formulation is as follows:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \underset{{\tilde{u}}_k,\dots ,{\tilde{u}}_{k+P}}{max}& {\displaystyle \sum _{j=1}^{N_y}}{c}_y^T{\tilde{y}}_{k+j}-c_u^T{\tilde{u}}_{k+j},\hfill \\ \hfill \mathrm{subject}\mathrm{to}:& \\ & {\tilde{x}}_{k+1}=A{\tilde{x}}_k+B{\tilde{u}}_k,\hfill \\ & {\tilde{y}}_k=C{\tilde{x}}_k+D{\tilde{u}}_k,\hfill \\ & \tilde{u}\in \mathcal{U},\Delta \tilde{u}\in {\mathcal{U}}_{\circ },\tilde{x}(k)={\widehat{x}}_l,\hfill \\ & V({\tilde{x}}_{k+j})\le \rho \mathrm{for}j=1,\cdots ,P,\hfill \end{array}\end{array}$$

(49)

where the value of $\rho$ dictates the neighborhood that the process states are allowed to evolve within. $c_y$ and $c_u$ indicate output and input cost vectors. Other variables have the same definition as Equation (47).

Remark 8.

In recent contributions [17,28], a Lyapunov-Based EMPC is proposed that utilizes data-driven methods to identify an empirical model for the system where the number of empirical model states is equal to the order of the plant model. In contrast, in the present work, the order of the model is selected based on the ability of the model to fit and predict dynamic behavior over a suitable range of operation, in turn allowing for an EMPC design that can reliably operate over a larger region.

Remark 9.

The EMPC formulation in the present manuscript utilizes a linear form of the cost function for the purpose of illustration. The proposed approach is not limited by this particular choice. Any other form of the cost function, including those where the costs could be time dependent, could be readily utilized within the proposed formulation. In such scenarios, the presence of the stability constraints provide the safeguards that allow the EMPC to move the process as needed to achieve economical goals.

Remark 10.

The use of linear models in the control design opens up the possibility of utilizing MPC formulations [3,29] that enable stabilization from the entire null controllable region (the region from which stabilization is achievable subject to input constraints). The use of the NCR can, in turn, be utilized to maximize the region over which the EMPC can be implemented, thereby maximizing the potential economic benefit. Such an implementation, however, needs to account for potential plant model mismatch owing to the use of the linear model, and remains the subject of future work.

Next, the proposed Lyapunov-based EMPC (LEMPC) is implemented on the CSTR simulation example and compared to the LMPC implementation. The closed-loop results are presented in Figure 8, Figure 9, Figure 10 and Figure 11. Exploiting the flexibility of operation within a neighborhood of the origin, the LEMPC drives the system to a point on the border of that neighborhood, which happens to be the optimal operating point, instead of the nominal operating point. Figure 12 shows the comparison of the LEMPC and LMPC. As expected, the LEMPC achieves improved economic returns compared to the conventional MPC.

6. Conclusions

In this study, a novel data-driven MPC is developed that enables stabilization at nominally unstable equilibrium points. This LMPC is then utilized within an economic MPC formulation to yield a data driven EMPC formulation. The proposed approach is described and compared against a representative MPC, and shown to be able to provide improved closed-loop performance.