Robust Multi-Stage Nonlinear Model Predictive Control Using Sigma Points

Abstract

We address the question of how to reduce the inevitable loss of performance that is incurred by robust multi-stage NMPC due to the lack of knowledge compared to the case where the exact plant model (no uncertainty) is available. Multi-stage NMPC in the usual setting over-approximates a continuous parametric uncertainty set by a box and includes the corners of the box and the center point into the scenario tree. If the uncertainty set is not a box, this augments the uncertainty set and results in a performance loss. In this paper, we propose to mitigate this problem by two different approaches where the scenario tree of the multi-stage NMPC is built using sigma points. The chosen sigma points help to capture the true mean and covariance of the uncertainty set more precisely. The first method computes a box over-approximation of the reachable set of the system states whereas the second method computes a box over-approximation of the reachable set of the constraint function using the unscented transformation. The advantages of the proposed schemes over the traditional multi-stage NMPC are demonstrated using simulation studies of a simple semi-batch reactor and a more complex industrial semi-batch polymerization reactor benchmark example.

1. Introduction

The process industries strive to cut down their operational costs while adhering to strict quality, safety, and environmental specifications. This results in an increased interest in using optimization-based control strategies to control the plants. Among the different optimization-based controllers, model predictive control (MPC) is most widely used because of its ability to handle multivariate systems with constraints efficiently [1,2].

MPC uses a plant model to predict the future evolution of the plant over a certain period, known as the prediction horizon, and optimizes the future control moves with respect to a performance criterion (tracking or economic). Only the control input that was obtained for the first time step by solving the optimization problem is applied to the plant. At the next time step, the MPC optimization problem is reinitialized with the plant measurements and resolved. This is commonly known as the moving horizon strategy [3,4].

The performance of any model-based controller depends strongly on the accuracy of the model used. Often there exists a discrepancy between the true plant dynamics and the model predictions due to external disturbances, inaccurate model parameters or structural plant–model mismatch. This may lead to constraint violations or performance degradation when the plant is controlled using the MPC controller with a nominal model. Robust MPC strategies take the uncertainties into account when computing the control moves [5,6,7,8,9,10]. The most prominent ones are min-max MPC, tube-based MPC, and multi-stage MPC.

Min-max MPC extends the idea behind min-max optimal control [11] to the MPC framework. Min-max MPC solves the MPC optimization problem for the worst-case realization of uncertainty [6]. The initial work on min-max MPC does not take into account the presence of future feedback information, hence it is rather conservative and may easily result in infeasible optimization problems [7]. A closed-loop min-max MPC formulation was presented in [12,13] where the future availability of measurement information and that the future control moves can be adapted depending on the realization of the uncertainty is taken into account. The closed-loop formulation requires solving an optimization over all possible feedback policies and results in a computationally very hard problem. It can be simplified by assuming a fixed control structure along the prediction horizon [12] at the expense of reduced performance.

Tube-based MPC [8,14] computes robust control actions using set-theoretic methods. It consists of two controllers, the nominal controller and the ancillary controller. The nominal controller considers tightened constraints and uses the nominal model of the plant to predict the trajectory of the system in the absence of uncertainty. The ancillary controller keeps the true plant dynamics in the neighborhood of the nominal trajectory in the presence of uncertainty such that the original constraints are satisfied. Several variants of the tube-based MPC are proposed in the literature with various levels of computational complexity and conservatism [15,16,17,18,19]. The main differences among these approaches lie in the computation of the uncertainty region around the nominal trajectory and the chosen ancillary controller. A variant of tube-based nonlinear MPC without the ancillary controller has been presented in [20]. The controller constructs a robust forward reachability tube that encloses the true plant trajectories in the presence of feedback actions using min-max differential inequalities.

Multi-stage MPC models the uncertainty by a tree of discrete scenarios [10]. It leads to an open-loop formulation of the optimal closed-loop control problem for the uncertainties that are included in the scenario tree and is less conservative when compared to other robust NMPC approaches [21]. For a nonlinear system, multi-stage NMPC rigorously guarantees constraint satisfaction for the uncertainties that are explicitly considered in the scenario tree. In the presence of continuous-valued uncertainty, representing all possible values of the uncertainty is infeasible. Usually, a good trade-off between robustness and computational cost is achieved by generating the scenario tree for all combinations of the minimum, nominal and maximum values of the uncertain parameters or disturbances [21,22]. This results in a box over-approximation of the true uncertainty region.

The application of any robust control mechanism inevitably results in conservatism and a loss of performance compared to the case when perfect information about the system is available. The performance loss of the robust schemes is related to the amount of uncertainty that is considered. If the parameters are estimated from data, the uncertainty can often be represented by ellipsoids that define the confidence region of the uncertain parameters [23,24]. Over-approximating an ellipsoidal confidence region by a box when generating the scenario tree of multi-stage NMPC then leads to an additional loss of performance.

The scenario tree of multi-stage NMPC can be generated using sigma points such that it more tightly approximates the uncertainty set, resulting in better performance. The sigma points capture the mean and the covariance of the uncertainty set [25,26]. In the unscented Kalman Filter [25,27,28], the sigma points in the state space are propagated to get an approximation of the distribution of the a-priori state estimate. Here we propagate the uncertain parameters which usually are less than the dimension of the state space. Then a box over-approximation of the reachable set can be computed from the propagated sigma points.

Several other papers also develop robust nonlinear model predictive control schemes using the unscented transformation principle [29,30,31,32,33]. Robust NMPC using the unscented transformation computes the statistical moments of the process trajectory in the presence of uncertain initial conditions and parameters using the unscented transformation [30]. This approach was extended to stochastic model predictive control with chance constraints in [31] and its advantages were shown for an automotive emergency braking system with collision avoidance in [32]. These approaches are open-loop approaches because they do not take into account the presence of future feedback information. A closed-loop formulation of robust MPC using the unscented transformation can be obtained by embedding a state estimation scheme based on the unscented transformation (e.g., the unscented Kalman filter) into the model predictive control optimization problem in order to achieve robustness against the estimation error [29,34]. Similar work was carried out in this direction by [33,35,36]. These papers mainly differ in the way in which the unscented Kalman filter equations are considered in the NMPC optimization problem. Since the complete unscented Kalman filter equations are embedded into the NMPC optimization problem, these approaches are in general computationally demanding.

In this paper, we propose two novel computationally efficient closed-loop robust multi-stage NMPC strategies that are based upon using the sigma points and unscented transformation principles. The scenario tree of the proposed schemes is generated using sigma points. The sigma points tightly approximate the uncertainty set in contrast to standard multi-stage NMPC where the uncertainty set is over-approximated by a box [21] and result in a better performance.

In the first approach, we propagate the sigma points along the prediction horizon to compute the state mean and the state covariance matrix using the unscented transformation. A box over-approximation of the reachable set of states is computed along the prediction horizon using the state mean and the scaled covariance matrix. This takes into account that the dimension of the uncertain parameters is usually smaller than the dimension of the state space as mentioned before. The objective and constraint function of the NMPC are evaluated for the mean and for the vertices of the box over-approximation of the reachable set of states. The initial work done in this direction showed promising results [37,38]. In the second approach, we compute the mean and the covariance matrix for the vector of constraint functions evaluated at the state predictions, and a box over-approximation of the reachable set of the constraint functions is obtained similar to the first approach. This approximates the reachable set of the constraint functions tighter when compared to the first approach and results in a better performance.

The proposed schemes take into account the effect of future control moves on the over-approximated reachable sets of the model and provide a closed-loop formulation as recourse (dependency of future control inputs on future information) is included. This results in a better performance when compared to open-loop approaches [30,31]. The proposed approaches require propagation of sigma points and only a part of the unscented Kalman filter equations have to be embedded into the NMPC optimization problem when compared to some of the existing robust NMPC schemes using the unscented transformation [29,30,33], hence the proposed approaches are computationally less demanding. Full state measurements are assumed, but the schemes can be combined with existing output feedback schemes [39,40,41] in a straightforward manner to obtain robustness against estimation errors.

The closed-loop performance of the controller can further be improved using the measurements obtained from the controlled system. We use the measurements to enhance the knowledge about the system via parameter estimation, e.g., in the least-squares sense, and thus to reduce the range of uncertainty. This is usually referred to as adaptive robust control [42,43,44,45]. An adaptive variant of the proposed robust multi-stage NMPC scheme using sigma points is presented also in this paper, extending the initial work in this direction in [38]. We compute an optimal ellipsoidal over-approximation of the intersection of the initial confidence region and the confidence regions obtained from the measurements and update the scenario tree of the adaptive multi-stage NMPC whenever a new measurement information from the plant becomes available. This results in a better performance if the observed measurements provide more information about the uncertain parameters when compared to the initial confidence region. The performance of the different robust NMPC controllers that are proposed in this paper is compared for a simple semi-batch reactor benchmark example and for a more complex industrial semi-batch polymerization reactor case study. Extensive simulation studies on the simple case study are performed to analyze the influence of the tuning parameters on the performance of the different robust multi-stage NMPC schemes.

The remainder of the paper is organized as follows. The problem statement and the unscented transformation are explained in Section 2. Standard multi-stage NMPC [10,21], and the proposed multi-stage NMPC using sigma points are described in Section 3. The adaptive variants of the NMPC schemes are presented in Section 4. The case studies along with the results are discussed in Section 5. Finally, the paper is concluded in Section 6.

2. Preliminaries

The model of the plant is assumed to be known, yet some of the model parameters are unknown. The true values of the uncertain parameters are assumed to be contained in a set. The model equations are given by Equations (1) and (2):

$$\begin{array}{c}\hfill {\mathit{x}}_{k+1}=\mathit{f}({\mathit{x}}_k,{\mathit{u}}_k,\mathit{d}),\end{array}$$

(1)

where $\mathit{x}\in {\mathbb{R}}^{n_x}$ and $\mathit{u}\in {\mathbb{R}}^{n_u}$ represent the state and control variables. The vector $\mathit{d}\in \mathbb{D}({\mathit{d}}_0,{\mathit{P}}_0)\subset {\mathbb{R}}^{n_d}$ represents the uncertain model parameters that are contained in an ellipsoidal set $\mathbb{D}({\mathit{d}}_0,{\mathit{P}}_0)$. The ellipsoidal set could represent the confidence region of an estimation of the uncertain parameters and is described by

$$\begin{array}{c}\hfill {\displaystyle \mathbb{D}}({\mathit{d}}_0,{\mathit{P}}_0):=\{\mathit{d}\in {{\displaystyle \mathbb{R}}}^{n_d}|{(\mathit{d}-{\mathit{d}}_0)}^T{\mathit{P}}_0^{-1}(\mathit{d}-{\mathit{d}}_0)\le 1\},\end{array}$$

(2)

where ${\mathit{d}}_0\in {\mathbb{R}}^{n_d}$ represents the nominal values of the uncertain parameters and acts as the center of the confidence region, and ${\mathit{P}}_0\in {\mathbb{R}}^{n_d\times n_d}$ represents the parameter covariance matrix and describes the shape of the confidence region. The set is assumed to form a finite support of the probability distribution of the model parameters. All the states are considered to be measured directly from the plant. ${\mathit{x}}_s^m\in {\mathbb{R}}^{n_x}$ gives the plant measurement obtained at $s^{\mathrm{th}}$ sample, where $s=\frac{t}{t_s}$, t represents the current time and $t_s$ represents the sampling time of the plant. The state measurements are assumed to be corrupted by uncorrelated white Gaussian noise with a covariance matrix $\mathsf{\Sigma }={\mathrm{diag}}^2\left(\mathit{\sigma }\right)$ with $\mathit{\sigma }\in {\mathbb{R}}^{n_x}$.

The unscented transformation is used to compute the statistics of variables which undergo a nonlinear transformation [26,46,47]. The principle behind the unscented transformation is illustrated in Figure 1, where $n_d=2$ and the uncertain parameters undergo a nonlinear transformation by an arbitrary function $\mathit{n}:{\mathbb{R}}^{2\times 2}\to {\mathbb{R}}^2$. The pink shaded region in Figure 1a represents the confidence region of the uncertain parameters and in Figure 1b it represents the image set of the nonlinear function $\mathit{n}$ (i.e., $\mathit{n}\left(\mathit{d}\right),\forall \mathit{d}\in \mathbb{D}({\mathit{d}}_0,{\mathit{P}}_0)$). $2n_d+1$ points known as the sigma points (marked with red squares in Figure 1a) are chosen such that they capture the true mean and the covariance of the uncertain parameter set $\mathbb{D}(·)$ and are given as Equations (3) and (4):

$$\begin{array}{c}\hfill \mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0)={\mathit{d}}_0\cup \left(\bigcup _{i=1}^{n_d}{\mathit{d}}_0-{\mathit{P}}_{0,[i,\star ]}^{\frac{1}{2},T}\right)\cup \left(\bigcup _{i=1}^{n_d}{\mathit{d}}_0+{\mathit{P}}_{0,[i,\star ]}^{\frac{1}{2},T}\right),\end{array}$$

(3)

where ${\mathit{P}}_{0,[i,\star ]}^{\frac{1}{2},T}$ gives the transpose of the $i^{\mathrm{th}}$ row vector of the matrix square root of ${\mathit{P}}_0\in {\mathbb{R}}^{n_d\times n_d}$ obtained using Cholesky decomposition. The sigma points are propagated through the nonlinear function $\mathit{n}(·)$ to compute the mean and covariance matrix of the transformed points that result from the nonlinear transformation.

$${\mathit{x}}^i=\mathit{n}\left({\mathit{d}}^i\right),\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\forall i\in I_{sp},{\mathit{d}}^i\in \mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0),$$

(4a)

$${\mathit{x}}_m=\sum _{i=1}^{2n_d+1}{\mathit{v}}_{\left[i\right]}{\mathit{x}}^i,\hspace{1em}\hspace{1em}\sum _{i=1}^{2n_d+1}{\mathit{v}}_{\left[i\right]}=1,$$

(4b)

$${\mathit{X}}_c={\kappa }^2\sum _{i=1}^{2n_d+1} {\mathit{v}}_{\left[i\right]}({\mathit{x}}^i-{\mathit{x}}_m){({\mathit{x}}^i-{\mathit{x}}_m)}^T,$$

(4c)

where $I_{sp}:=\{1,\dots ,2n_d+1\}$, ${\mathit{d}}^i$ represents the $i^{\mathrm{th}}$ element of the set $\mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0)$ and $\mathit{v}\in {\mathbb{R}}^{2n_d+1}$ are the sigma point weights, so ${\mathit{v}}_{\left[i\right]}$ represents the weight associated with the sigma point ${\mathit{d}}^i$. The sigma points are propagated using the nonlinear function $\mathit{n}(·)$ using Equation (4a) as shown using red dotted lines in Figure 1. The mean $\left({\mathit{x}}_m\right)$ and the covariance matrix $\left({\mathit{X}}_c\right)$ are computed using Equations (4b) and (4c). $\kappa \in \mathbb{R}$ is the scaling factor of the covariance matrix. The ellipsoids obtained using the mean and the covariance matrix for different values of the scaling factor are shown in blue in Figure 1. The function that computes the mean and covariance matrix of a nonlinear transformation using unscented transformation can be compactly represented as $\left({\mathit{x}}_m,{\mathit{X}}_c\right)=\mathbb{U}(\kappa ,{\mathit{x}}^{[1:2n_d+1]})$, where ${\mathit{x}}^{[1:2n_d+1]}$ represents values corresponding to $2n_d+1$ evaluations of the nonlinear function $\mathit{n}(·)$ (see Equation (4a)). e.g., in Figure 1, the values ${\mathit{x}}^1,{\mathit{x}}^2,{\mathit{x}}^3,{\mathit{x}}^4,$ and ${\mathit{x}}^5$ are represented by ${\mathit{x}}^{[1:2n_d+1]}$.

3. Robust Multi-Stage NMPC

3.1. Standard Multi-Stage NMPC

Multi-stage NMPC [21] is a robust NMPC strategy that models the effect of the uncertainties by a tree of discrete scenarios as shown in Figure 2. Each branch of the scenario tree represents the trajectory of the system states for a given control input and a particular realization of the uncertainty that can vary at each point in the prediction horizon. Multi-stage NNPC computes control inputs while taking into account that measurement information will be available in the future and that the future control moves can be adapted accordingly, i.e., the control inputs beyond the next time step become scenario-dependent recourse variables. This allows us to solve a feedback problem as an open-loop optimization problem. The closed-loop formulation results in a significant improvement in the performance of the robust controller in comparison to open-loop schemes [7,48].

3.1.1. Multi-Stage NMPC (MS NMPC)

The standard multi-stage NMPC proposed in [10,21] represents uncertainty region by a box. The scenario tree of multi-stage NMPC is usually built for all possible combinations of the minimal, maximal, and nominal values of the uncertain parameters. The formulation of standard multi-stage NMPC solved at time t ($s^{\mathrm{th}}$ sampling time) for the case where the parameter uncertainty is given by an ellipsoidal set $\mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0)$ reads as Equation (5):

$$\underset{{\mathit{x}}_k^j,{\mathit{u}}_k^j,{\mathit{d}}^j}{min} \sum _{k=s}^{s+N_r-1}\sum _{j=1}^{N_b^{k-s+1}} {\omega }_{k+1}^{j}l({\mathit{x}}_{k+1}^j,\Delta {\mathit{u}}_k^j)+\sum _{k=s+N_r}^{s+N_p-1}\sum _{j=1}^{N_b^{N_r}}{\omega }_{k+1}^{j}l({\mathit{x}}_{k+1}^j,\Delta {\mathit{u}}_k^j),\forall (j,k+1)\in I_{st},$$

(5a)

subject to

$${\mathit{x}}_{k+1}^j=\mathit{f}({\mathit{x}}_k^{p\left(j\right)},{\mathit{u}}_k^j,{\mathit{d}}^{r\left(j\right)}),\hspace{1em}\hspace{1em}\forall (j,k+1)\in I_{st},{\mathit{d}}^{r\left(j\right)}\in {\displaystyle \mathcal{D}},$$

(5b)

$$\mathit{g}({\mathit{x}}_{k+1}^j,{\mathit{u}}_k^j)\le 0,\hspace{1em}\hspace{1em}\forall (j,k+1)\in I_{st},$$

(5c)

$${\mathit{u}}_k^j={\mathit{u}}_k^l\mathrm{if}{\mathit{x}}_k^{p\left(j\right)}={\mathit{x}}_k^{p\left(l\right)},\hspace{1em}\hspace{1em}\forall (j,k),(l,k)\in I_{st},$$

(5d)

$$\underline{\mathit{u}}\le {\mathit{u}}_k^j\le \overline{\mathit{u}},\hspace{1em}\hspace{1em}\forall (j,k)\in I_{st},$$

(5e)

$${\mathit{x}}_s^1={\mathit{x}}_s^m$$

(5f)

$$\underline{\mathit{d}}={\mathit{d}}_0-{\mathrm{diag}}^{\frac{1}{2}}\left({\mathit{P}}_0\right),\overline{\mathit{d}}={\mathit{d}}_0+{\mathrm{diag}}^{\frac{1}{2}}\left({\mathit{P}}_0\right),$$

(5g)

$$\mathcal{D}={\mathbb{C}}_a(\underline{\mathit{d}},{\mathit{d}}_0,\overline{\mathit{d}}).$$

(5h)

The state trajectory along the branches of the scenario tree for different realizations of the uncertainty is given by Equation (5b). The state prediction ${\mathit{x}}_{k+1}^j$ at stage $k+1$ and position j in the scenario tree is obtained using the parent state ${\mathit{x}}_k^{p\left(j\right)}$, the control input ${\mathit{u}}_k^j$, and the uncertainty realization ${\mathit{d}}^{r\left(j\right)}\in \mathcal{D}$. The notation is explained briefly using Figure 2 for the optimization problem solved at time $t=0$. The root node is represented by ${\mathit{x}}_0^1$ and is initialized at current plant measurement ${\mathit{x}}_0^m$. The root node branches into the child node ${\mathit{x}}_1^1$, ${\mathit{x}}_1^2$ and ${\mathit{x}}_1^3$ depending on the applied control inputs and realization of the uncertainty. For e.g., ${\mathit{x}}_1^1$ is obtained if the uncertainty ${\mathit{d}}^1$ is realized and control move ${\mathit{u}}_0^1$ is applied at the root node ${\mathit{x}}_0^1$. The root node ${\mathit{x}}_0^1$ acts as the parent node for the child nodes ${\mathit{x}}_1^1$, ${\mathit{x}}_1^2$, and ${\mathit{x}}_1^3$. The nodes ${\mathit{x}}_0^1$, ${\mathit{x}}_1^1$, ${\mathit{x}}_2^1$, ${\mathit{x}}_3^1$, and ${\mathit{x}}_4^1$ together form a scenario of the scenario tree. $I_{st}$ denotes the set of indices $(j,k)$ that occur in a given scenario tree. The set $\mathcal{D}$ represents different realizations of the uncertain parameters considered by the standard multi-stage NMPC. The additional constraints that must be satisfied at each node in the scenario tree are given by Equation (5c). The controller cannot anticipate the future realization of the uncertainty i.e., the control decision originating from the same parent node must be the same (for e.g., in Figure 2 ${\mathit{u}}_0^1={\mathit{u}}_0^2={\mathit{u}}_0^3;{\mathit{u}}_1^1={\mathit{u}}_1^2={\mathit{u}}_1^3;\dots$). This is enforced using the non-anticipativity constraints Equation (5d). The inputs are bounded using Equation (5e), where $\underline{\mathit{u}}$ and $\overline{\mathit{u}}$ gives the lower and upper bounds on the control inputs. The robust NMPC is reinitialized at the plant measurement obtained at the $s^{\mathrm{th}}$ sampling time using Equation (5f). The minimum and maximum values of the uncertain parameters can be obtained using their nominal values and the parameter covariance matrix from Equation (5g), where ${\mathrm{diag}}^{\frac{1}{2}}:{\mathbb{R}}^{n_d\times n_d}\to {\mathbb{R}}^{n_d}$ gives a vector that contains the square roots of the diagonal elements of the matrix. The uncertain parameter values considered by the standard multi-stage NMPC are obtained using Equation (5h), where the operator ${\mathbb{C}}_a(·)$ generates a set of all possible combinations of the minimum, nominal and maximal values of the uncertainty. The scenario tree grows exponentially along the prediction horizon $\left(N_p\right)$ which can be prevented by stopping the branching after a certain point in time along the prediction horizon known as the robust horizon $\left(N_r\right)$ under the assumption that the uncertainty realization remains constant after the robust horizon. The objective function of the NMPC optimization problem is given in Equation (5a), where $\Delta {\mathit{u}}_k={\mathit{u}}_k-{\mathit{u}}_{k-1}$ gives the deviation between two consecutive control moves. The first part of the objective function represents the contribution of all the nodes until the robust horizon and the second part of the objective function gives the contribution of all the nodes after the robust horizon on the objective function. $N_b$ is the number of branches at each node in the scenario tree, where $N_b=3^{n_d}$ for standard multi-stage NMPC. $l(·)$ represents the cost at each node in the scenario tree and ${\omega }_{k+1}^j$ is the weight associated with each node in the scenario tree and is given by Equation (6):

$$\begin{array}{c}\hfill \sum _{j=1}^{{N_b}^{k-s+1}}{\omega }_{k+1}^j=1,\forall k\in I_{br},\sum _{j=1}^{{N_b}^{N_r}}{\omega }_{k+1}^j=1,\forall k\in I_{ar},\end{array}$$

(6)

where $I_{br}:=\{s,\dots ,s+N_r-1\}$ and $I_{ar}:=\{s+N_r,\dots ,s+N_p-1\}$.

Remark 1.

The stability properties of the robust MPC have been previously analyzed in the literature using standard tracking objective functions. The first analysis was presented in [7] for linear systems where a min-max cost function is considered. Several papers have been published for linear stochastic systems and exponential stability in the mean square sense has been established [49,50]. The setting considered in this paper is more complex, as it considers nonlinear systems and a deterministic representation of the uncertainty. Initial work on the stability analysis of standard multi-stage NMPC was carried out in [51]. The consideration of different weights for the cost function at different nodes in the scenario tree (as opposed to a worst-case or probabilistic formulation) is a challenge for the stability analysis. The stability property is satisfied in [51] via continuity assumptions on the model equations. As with other robust approaches, only convergence to a neighborhood of the equilibrium point can be achieved because the different branches of the uncertainty are considered at each sampling time, even if the disturbance vanishes [52]. Convergence can be recovered by using a dual-mode approach as used in other MPC schemes [53]. A rigorous analysis of the stability properties of the multi-stage approach in the context of economic model predictive control [54,55] is still an open issue.

3.1.2. Multi-Stage NMPC Based on the Vertex Over-Approximation (MS-VA NMPC)

In standard multi-stage NMPC all combinations of the minimum, nominal and maximum values of the uncertain parameters are chosen to approximate the uncertainty set, this results in $3^{n_d}$ branches to be considered at each node in the scenario tree. Similar performance with reduced computational effort can be achieved if we choose only the nominal parameter value along with the vertices of the box over-approximation of the uncertainty set. This results in only $2^{n_d}+1$ branches to be considered at each node in the scenario tree. We call this multi-stage NMPC based on the vertex approximation (MS-VA). The formulation of the MS-VA NMPC optimization problem (The formulation of the entire MS-VA, MS-SB, and MS-CB NMPC optimization problems are provided in S-1 as Supplementary Material. The formulation of entire adaptive robust multi-stage NMPC optimization problem is provided in S-2 of the Supplementary Material) that is solved at time t ($s^{\mathrm{th}}$ sampling time) is similar to the optimization problem Equation (5), where Equation (5h) is replaced with Equation (7):

$$\mathcal{D}={\mathit{d}}_0\cup {\mathbb{C}}_{vp}(\underline{\mathit{d}},\overline{\mathit{d}}),$$

(7)

${\mathbb{C}}_{vp}(·)$ generates the set of all possible combinations of the lower and upper bound of the uncertainty set along with its nominal value. The objective function of the multi-stage NMPC based on the vertex over-approximation approach is similar to the standard multi-stage NMPC Equation (5a) approach except that $N_b=2^{n_d}+1$.

Remark 2.

The uncertain parameter values chosen to build the scenario tree of the MS NMPC and MS-VA NMPC can guarantee robust constraint satisfaction in the presence of continuous-valued uncertainty only if the parametric monotonicity property of the problem constraints is satisfied (i.e., the sensitivity of the constraints with respect to the uncertain model parameters has the same sign for all parameter values within the uncertainty set) [56]. If the parametric monotonic property is not satisfied, robust constraint satisfaction is guaranteed only for the scenarios that are explicitly considered in the scenario tree of the multi-stage NMPC. In such cases, the multi-stage approach can be combined with reachability analysis as shown in [57] to obtain robust constraint satisfaction but this increases the computational complexity of the resulting robust control problem.

3.2. Multi-Stage NMPC Using Sigma Points

In standard multi-stage NMPC (see Section 3.1), the uncertainty set is over-approximated even if more precise information about the uncertain parameters is available and this results in a performance loss. This will be avoided using sigma points. The sigma points ($2n_d+1$ sample points from the boundary of the ellipsoidal uncertainty set) and their propagation can be used to approximate the box of the predicted uncertain states more tightly. In this paper, we propose two methods to make use of the propagation of the sigma points, in first method the box over-approximation of the reachable set of states is computed and in the second method a box over-approximation of the reachable set of the constraint function is employed.

3.2.1. Multi-Stage NMPC Based on the Box Over-Approximation of the Reachable Set of States (MS-SB NMPC)

The key difference between MS-VA NMPC (Section 3.1.2) and MS-SB NMPC is illustrated in Figure 3 for a system with two states, two uncertain parameters, and a given control input at time $k=1$. Figure 3a represents the parametric space. The pink shaded region represents the ellipsoidal confidence region of the uncertain parameters given by Equation (2). Black dots (vertices of the box over-approximation of the uncertainty set) and red squares (sigma points obtained using Equation (3)) represent the parameter samples that are chosen to build the scenario tree of MS-VA NMPC and MS-SB NMPC, respectively. The nominal value of the uncertain parameter $({\mathit{d}}^1={\mathit{d}}_0)$ is considered in the scenario tree of both MS-VA NMPC and MS-SB NMPC.

Figure 3b,c represent the reachable set in the state space (${\mathbb{X}}_1^1$) by a pink shaded region. Black circles in Figure 3b represent the state predictions obtained on the branches of the scenario tree of MS-VA NMPC (e.g., ${\mathit{x}}_1^1$ is obtained for the uncertainty realization ${\mathit{d}}^1$ when the control input ${\mathit{u}}_0^1$ is applied). Red squares in Figure 3c represent the state predictions obtained on the branches of the scenario tree of MS-SB NMPC. The inner (blue) ellipsoid represents the state covariance ellipsoid described by the state mean and the state covariance matrix (i.e., $\kappa =1$) computed using the state predictions of MS-SB NMPC. The state covariance matrix is then enlarged such that the box over-approximation (represented by the dotted blue line) of the extended ellipsoid (represented by the outer (blue) ellipsoid) over-approximates the reachable set of states. Blue squares represent the state mean and the vertices of the box over-approximation of the reachable set for which the objective and constraint functions of MS-SB NMPC are evaluated.

Tuning of the Scaling Factor $\kappa$

The scaling factor $\kappa$ for the enlargement of the ellipsoidal set that results from the propagation of the sigma points can be computed such that the box over-approximation of the scaled state ellipsoids tightly over-approximates the entire reachable set of model (1) by solving $n_x$ optimization problems given below (Equation (8)), where $j\in I_x:=\{1,\dots ,n_x\}$,

$${\widehat{\mathit{\kappa }}}_{s,\left[j\right]}=\underset{\kappa ,{\mathit{x}}_0^{\mathrm{wc}},{\mathit{u}}_0^{\mathrm{wc}},{\mathit{d}}^{\mathrm{wc}}\in \mathbb{D},{\mathit{d}}^i\in \mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0)}{max}\frac{\left|{\mathit{x}}_{1,\left[j\right]}^{\mathrm{wc}}-{\mathit{x}}_{m,\left[j\right]}\right|}{\sqrt{{\mathit{X}}_{c,1,[j,j]}^p}},$$

(8a)

subject to

$${\mathit{x}}_1^i=\mathit{f}({\mathit{x}}_0^{\mathrm{wc}},{\mathit{u}}_0^{\mathrm{wc}},{\mathit{d}}^i),\hspace{1em}\hspace{1em}\forall i\in I_{sp},{\mathit{d}}^i\in \mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0),$$

(8b)

$$\left({\mathit{x}}_{m,1},{\mathit{X}}_{c,1}^p\right)=\mathbb{U}(1,{\mathit{x}}_1^{[1:2n_d+1]}),$$

(8c)

$${\mathit{x}}_1^{\mathrm{wc}}=\mathit{f}({\mathit{x}}_0^{\mathrm{wc}},{\mathit{u}}_0^{\mathrm{wc}},{\mathit{d}}^{\mathrm{wc}}),$$

(8d)

$$\mathit{g}({\mathit{x}}_0^{\mathrm{wc}},{\mathit{u}}_0^{\mathrm{wc}})\le 0,$$

(8e)

$${\mathit{X}}_{c,1,[j,j]}^p>0,$$

(8f)

where $\mathit{v}$ is the weight associated with the sigma points and is chosen such that ${\sum }_{i=1}^{2n_d+1}{\mathit{v}}_{\left[i\right]}=1$. ${\mathit{d}}^i$ represents the $i^{\mathrm{th}}$ uncertain parameter combination in the set of uncertain parameters obtained using $\mathbb{S}(·)$. The objective function Equation (8a) gives the maximum value by which the state ellipsoid should be scaled, for the box over-approximation of the scaled state ellipsoid to enclose the reachable set with respect to state j. The state mean $\left({\mathit{x}}_{m,1}\right)$ and the unscaled state covariance matrix $\left({\mathit{X}}_{c,1}^p\right)$ are obtained using Equation (8c), where ${\mathit{x}}_1^{[1:2n_d+1]}$ represents all the model predictions obtained using the sigma points given by Equation (8b). The vector ${\mathit{x}}_1^{\mathrm{wc}}$ denotes the prediction obtained in the whole operating region of the plant that results in the maximum value of the objective function. Vectors ${\mathit{x}}_0^{\mathrm{wc}}$ and ${\mathit{u}}_0^{\mathrm{wc}}$ are bounded by operating region of the plant using Equation (8e). The constraint Equation (8f) makes sure that the optimization problem becomes infeasible if the sensitivity of the states for the uncertain parameters is 0 (i.e., the uncertain parameter does not influence the state predictions). The scaling factor corresponding to the different states is given by Equation (9):

$$\begin{array}{ccc}\hfill {\widehat{\mathit{\kappa }}}_{x,\left[j\right]}^{\star }=\left\{\begin{array}{cc}0,\hfill & \mathrm{if}\mathrm{Equation}\left(8\right)\mathrm{is}\mathrm{infeasible}\hfill \\ {\widehat{\mathit{\kappa }}}_{x,\left[j\right]},\hfill & \mathrm{otherwise}\hfill \end{array}\right.,& \hfill & \hfill \forall j\in I_x\end{array}$$

(9)

where ${\widehat{\mathit{\kappa }}}_x^{\star }\in {\mathbb{R}}^{n_x}$. The scaling factor is set to 0 if the uncertainty considered to build the scenario tree of the MS-SB NMPC does not influence the states. The scaling factor ${\kappa }_x^{\star }$ is given by $\left|\right|{\widehat{\mathit{\kappa }}}_x^{\star }{\left|\right|}_{\infty }$. This makes sure that the box over-approximation of the scaled state ellipsoids encloses the reachable set of model (1) provided the optimization problem Equation (8) is solved to its global optimum. It was shown to be feasible in [58], where an optimization problem similar to Equation (8) was solved to its global optimum.

The formulation of the MS-SB NMPC problem that is solved at time t ($s^{\mathrm{th}}$ iteration), where $J_{so}$ represents the value of the objective function of MS-SB NMPC until the robust horizon reads as Equation (10):

$$\underset{{\mathit{x}}_k^j,{\mathit{u}}_k^j,{\mathit{d}}^j,{\mathit{x}}_{m,k}^j,{\mathit{X}}_{c,k}^j}{min}{J}_{so}({{\displaystyle \mathcal{X}}}_{k+1}^q,\Delta {\mathit{u}}_k^j)+\sum _{k=s+N_r}^{s+N_p-1}\sum _{j=1}^{N_b^{N_r}}{\omega }_{k+1}^{j}l({\mathit{x}}_{k+1}^j,\Delta {\mathit{u}}_k^j),\forall (j,k+1)\in I_{st},$$

(10a)

subject to:

$$\begin{array}{cccc}\hfill & \mathrm{Equations}\left(5\mathrm{b}\right)–\left(\mathrm{f}\right),\hfill & \hfill & \hfill \end{array}$$

$$\begin{array}{cccc}\hfill & {\displaystyle \mathcal{D}}=\mathbb{S}({\mathit{d}}_0,{\mathit{P}}_0),\hfill & \hfill & \hfill \end{array}$$

(10b)

$$\begin{array}{cccc}\hfill & \left({\mathit{x}}_{m,k+1}^q,{\mathit{X}}_{c,k+1}^q\right)=\mathbb{U}({\kappa }_{x,k},{\mathit{x}}_{k+1}^{[(q-1)N_b+1:q{N}_b]}),\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\hfill \end{array}$$

(10c)

$$\begin{array}{cccc}\hfill & {\underline{\mathit{x}}}_{m,k+1}^q={\mathit{x}}_{m,k+1}^q-{\mathrm{diag}}^{\frac{1}{2}}\left({\mathit{X}}_{c,k+1}^q\right),\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\hfill \end{array}$$

(10d)

$$\begin{array}{cccc}\hfill & {\overline{\mathit{x}}}_{m,k+1}^q={\mathit{x}}_{m,k+1}^q+{\mathrm{diag}}^{\frac{1}{2}}\left({\mathit{X}}_{c,k+1}^q\right),\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\hfill \end{array}$$

(10e)

$$\begin{array}{cccc}\hfill & {{\displaystyle \mathcal{X}}}_{k+1}^q={\mathit{x}}_{m,k+1}^q\cup {\mathbb{C}}_{vp}({\underline{\mathit{x}}}_{m,k+1}^q,{\overline{\mathit{x}}}_{m,k+1}^q),\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\hfill \end{array}$$

(10f)

$$\begin{array}{cccc}\hfill & \mathit{g}(\mathit{x},{\mathit{u}}_k^{(q-1)N_b+1})\le 0,\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\mathit{x}\in {{\displaystyle \mathcal{X}}}_{k+1}^q.\hfill \end{array}$$

(10g)

where $I_b\left(k\right):=\{1,\dots ,{N_b}^{k-s}\}$. The scenario tree of MS-SB NMPC considers $N_b=2n_d+1$ branches at each node. The sigma points of the ellipsoidal uncertainty set are obtained from Equation (10b). ${\underline{\mathit{x}}}_{m,k+1}^q$ and ${\mathit{X}}_{c,k+1}^q$ represent the state mean and the state covariance matrix obtained while applying control input ${\mathit{u}}_k^{(q-1)N_b+1}$ at the parent state ${\mathit{x}}_k^{p((q-1)N_b+1)}$. Referring to Figure 2, ${\mathit{X}}_{c,2}^2$ is computed using the state predictions ${\mathit{x}}_2^4$, ${\mathit{x}}_2^5$ and ${\mathit{x}}_2^6$ that were obtained from the parent state ${\mathit{x}}_1^2$ when the control input ${\mathit{u}}_1^4$ (${\mathit{u}}_1^4={\mathit{u}}_1^5={\mathit{u}}_1^6$ due to non-anticipativity constraints) is applied. The state covariance matrix is scaled using a scaling factor ${\kappa }_{x,k}$ which is a tuning parameter. The scaling factor ${\kappa }_{x,k}$ must be chosen in a way that the box over-approximation of the state ellipsoid described by the state mean and the state covariance matrix encloses the reachable states sets of model (1) along the robust horizon. The scaling factor ${\kappa }_{x,s}$ for the first stage can be obtained by solving the optimization problem Equation (8) and is given as ${\kappa }_x^{\star }$. At the next stage, the true plant state can be present anywhere in the predicted box over-approximation of the reachable states set but the scenario tree of MS-SB NMPC branches from the predictions obtained using the sigma points. This introduces additional uncertainty in the initial condition of the states and can be overcome by increasing the scaling factor ${\kappa }_{x,k}$ by a constant factor $\beta$ (i.e., ${\kappa }_{x,k}=\beta {\kappa }_{x,k-1}$ s.t.$\beta \ge 1$). The lower and upper bounds on the state predictions can be obtained using Equations (10d) and (10e). The vertices of the box over-approximation of state ellipsoid along with its center are obtained in Equation (10f). The constraint function $\mathit{g}(·)$ is satisfied for the vertices of the box over-approximation of the predicted state ellipsoids using Equation (10g). The objective function of the MS-SB NMPC scheme is given in Equation (10a), and reads as Equation (11):

$$\begin{array}{c}\hfill J_{so}({{\displaystyle \mathcal{X}}}_{k+1}^q,\Delta {\mathit{u}}_k^j)=\sum _{k=s}^{s+N_r-1}\sum _{q=1}^{N_b^{k-s+1}}\sum _{i=1}^{2^{n_x}+1}{\omega }_{k+1}^{i+q-1}L({\mathit{x}}^i,\Delta {\mathit{u}}_k^{(q-1)N_b+1}),\end{array}$$

(11)

where ${\mathit{x}}^i$ represents the $i^{\mathrm{th}}$ element of the set ${\mathcal{X}}_{k+1}^i$. ${\omega }_k^{i+q-1}$ is the weight associated with vertices of the box over-approximation of the state ellipsoids along with its mean and is chosen such that Equation (12):

$$\begin{array}{c}\hfill \sum _{q=1}^{N_b^{k-s+1}}\sum _{i=1}^{2^{n_x}+1}{\omega }_k^{i+q-1}=1,\forall k\in I_{br}.\end{array}$$

(12)

Remark 3.

The tuning of scaling factor ${\kappa }_{x,k}$ plays a major role in the performance of MS-SB NMPC. If a very large value for ${\kappa }_{x,k}$ is chosen, the reachable states of model (1) are loosely over-approximated and this results in performance loss. On the contrary, if a very small value is chosen, the reachable states are under approximated and this may result in a constraint violation. The scaling factor ${\kappa }_{x,s}$ can be obtained by solving optimization problem 8 so that the reachable states set is tightly over-approximated by a box. The robust constraint satisfaction of MS-SB NMPC is guaranteed only if the MS-SB NMPC optimization problem Equation (10) is recursively feasible and the following condition is satisfied:

$$\begin{array}{c}\hfill \underset{\mathit{x}\in {\mathcal{X}}_{k+1}^q}{max}\mathit{g}(\mathit{x},{\mathit{u}}_k^{(q-1)N_b+1})\ge \underset{\mathit{x}\in {\mathbb{X}}_{k+1}^q}{max}\mathit{g}(\mathit{x},{\mathit{u}}_k^{(q-1)N_b+1}),\end{array}$$

(13)

where ${\mathbb{X}}_{k+1}^q$ represents the reachable states set of model (1). The condition Equation (13) is satisfied if the predicted box over-approximation of the state ellipsoid encloses the set of reachable states and the sensitivity of the constraints with respect to the states does not change its sign. Otherwise, ${\kappa }_{x,0}$ should be tuned based on posterior analysis using simulation studies such that the condition Equation (13) is satisfied. In this work, the recursive feasibility of multi-stage NMPC using sigma points is verified using simulation studies. The scaling factor ${\kappa }_{x,k}$ is proportional to the value of the parameter β. The parameter β is chosen based on posterior analysis using simulation studies in a way that the predicted state ellipsoids enclose the reachable states sets along the robust horizon. Theoretical guarantees on the stability and recursive feasibility of the multi-stage NMPC using sigma points are not considered in this work but can be achieved by choosing a full robust horizon (i.e., $N_r=N_p$) and tuning β systematically [17,59]. This will be part of our future work.

Remark 4.

The performance of multi-stage NMPC using sigma points can be improved using an ellipsoidal over-approximation of the reachable states set and satisfying the constraints for all states contained in the ellipsoidal set [60]. One of the major drawbacks of this approach is that if $n_x>2n_d+1$, the number of sigma points chosen to build the scenario tree may not be sufficient to capture the covariance matrix of the nonlinear transformation and may lead to degenerate ellipsoids. In this case, it is not feasible to compute an ellipsoidal over-approximation of the reachable states set using $2n_d+1$ sigma points but a box over-approximation can still be computed using the scaled covariance matrix as explained in Section 3.2. An ellipsoidal over-approximation of the reachable set of states can be computed by choosing $n_x+1$ sigma points from the uncertainty set instead of $2n_d+1$ sigma points.

3.2.2. Multi-Stage NMPC Based on the Box Over-Approximation of the Reachable Set of the Constraint Function (MS-CB NMPC)

MS-CB NMPC computes a box over-approximation of the reachable set of the constraint function $\mathit{g}(·)$ using the predictions obtained from the scenario tree of the multi-stage NMPC. The difference between MS-VA NMPC (Section 3.1.2), MS-SB NMPC (Section 3.2.1) and MS-CB NMPC (Section 3.2.2) is shown in Figure 4 for a system with two states, two uncertain parameters, a given control input and two constraints at time $k=1$ in the prediction horizon. Figure 4a is same as the Figure 3a. Figure 4b–d represent the reachable sets in the constraint space. The pink shaded region represents the image of the constraint functions for the reachable states of the model (1). The control inputs ${\mathit{u}}_1^1$, ${\mathit{u}}_1^2$, ${\mathit{u}}_1^3$, ${\mathit{u}}_1^4$ and ${\mathit{u}}_1^5$ are equal due to the non-anticipativity constraint.

Black dots in Figure 4b represent the value of the constraint function for the state predictions obtained on the branches of the scenario tree of MS-VA NMPC. It is assumed that $\mathit{g}(\mathit{x},\mathit{u})\le 0$. The optimal control input computed using MS-VA NMPC satisfies the constraints only for the parameter combinations considered in its scenario tree. The box over-approximation of the parametric confidence region introduces additional uncertainty in the parameter values which may result in a large back-off from the bounds (shown for ${\mathit{g}}_{\left[2\right]}(\mathit{x},\mathit{u})$ in Figure 4b) and may result in a loss of performance.

The blue squares in Figure 4c represent the values of the constraint functions evaluated using the state mean and vertices of the box over-approximation of the set if reachable states obtained using the MS-SB NMPC approach. The optimal control input satisfies the constraints only for the state values that are obtained using the vertices of the box over-approximation of the state ellipsoids and its mean. This may lead to constraint violation and can be avoided as mentioned in Remark 3. In addition, the box over-approximation of the set of reachable states may add an additional region and may lead to an additional loss of performance (can be seen for ${\mathit{g}}_{\left[2\right]}(\mathit{x},\mathit{u})$ in Figure 4b).

The red squares in Figure 4d represent the values of the constraint function evaluated using the state predictions obtained on the branches of the scenario tree of MS-CB NMPC. The inner (brown) ellipsoid represents the ellipsoid computed using the unscented transformation with scaling factor $\kappa =1$. The constraint covariance matrix is then scaled using the scaling factor $\kappa$ so that the box over-approximation (brown dotted lines) of the extended ellipsoidal set (outer brown ellipsoid) over-approximates the reachable set of the constraint function. MS-CB NMPC computes optimal control inputs such that all the elements within the reachable set of the constraint function satisfy $\mathit{g}(\mathit{x},\mathit{u})\le 0$ as shown in Figure 4d even in the presence of a nonlinear model and nonlinear constraints, in contrast to the MS-VA and MS-SB NMPC approaches.

Tuning of the Scaling Factor $\kappa$

The scaling factor $\kappa$ can be computed in a way that the box over-approximation of the constraint function ellipsoid represented by scaled constraint covariance matrix and mean tightly over-approximates the reachable set of the constraint function $\mathit{g}(·)$ by solving $n_c$ optimization problem similar to Equation (8), where $n_c$ represents the number of constraints. The vector ${\widehat{\kappa }}_x$ in Equation (8a) is replaced with ${\widehat{\kappa }}_c$ and the model predictions obtained in Equations (8b) and (8d) are propagated through the constraint function along with the control input ${\mathit{u}}_0^{\mathrm{wc}}$ to compute the box over-approximation of the reachable set of the constraint function. The final scaling factor ${\kappa }_c^{\star }$ is then given by $\left|\right|{\widehat{\kappa }}_c^{\star }{\left|\right|}_{\infty }$. The full formulation of the optimization problem solved to obtain the scaling factor ${\kappa }_c^{\star }$ is provided in the Supplementary Material in Section S3. The optimization problems must be solved to their global optimum [61,62] which of course is computationally costly but are computed once before solving the NMPC optimization problem.

The formulation of MS-CB NMPC optimization problem solved at time t ($s^{\mathrm{th}}$ iteration) reads as Equation (14):

$$\begin{array}{cc}\hfill & \underset{{\mathit{x}}_k^j,{\mathit{u}}_k^j,{\mathit{d}}^j,{\mathit{g}}_{m,k}^q,{\mathit{G}}_{c,k}^q}{min}\sum _{k=s}^{s+N_r-1}\sum _{j=1}^{N_b^{k-s+1}}{\omega }_{k+1}^{j}l({\mathit{x}}_{k+1}^j,\Delta {\mathit{u}}_k^j)+\sum _{k=s+N_r}^{s+N_p-1}\sum _{j=1}^{N_b^{N_r}}{\omega }_{k+1}^{j}l({\mathit{x}}_{k+1}^j,\Delta {\mathit{u}}_k^j),\forall (j,k+1)\in I_{st},\hfill \end{array}$$

(14a)

subject to:

$$\begin{array}{cccc}\hfill & \mathrm{Equations}\left(5\mathrm{b}\right)–\left(5\mathrm{f}\right),\left(10\mathrm{b}\right),\hfill & \hfill & \hfill \end{array}$$

$$\begin{array}{cccc}\hfill & {\mathit{g}}_{n,k+1}^j=\mathit{g}({\mathit{x}}_{k+1}^j,{\mathit{u}}_k^j),\hfill & \hfill & \forall (j,k+1)\in I_{st},\hfill \end{array}$$

(14b)

$$\begin{array}{cccc}\hfill & \left({\mathit{g}}_{m,k+1}^q,{\mathit{G}}_{c,k+1}^q\right)=\mathbb{U}({\kappa }_{c,k},{\mathit{g}}_{n,k+1}^{[(q-1)N_b+1:q{N}_b]}),\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\hfill \end{array}$$

(14c)

$$\begin{array}{cccc}\hfill & {\mathit{g}}_{m,k+1}^q+{\mathrm{diag}}^{\frac{1}{2}}\left({\mathit{G}}_{c,k+1}^q\right)\le 0,\hfill & \hfill & \forall k\in I_{br},q\in I_b\left(k\right),\hfill \end{array}$$

(14d)

The scenario tree of MS-CB NMPC considers $N_b=2n_d+1$ branches at each node. The value of the constraint function evaluated at each node is given by Equation (14b). The mean $\left({\mathit{g}}_m\right)$ and the covariance of the constraint function $\left({\mathit{G}}_c\right)$ are obtained using Equation (14c). The constraint covariance matrix is scaled using a scaling factor ${\kappa }_{c,k}$. The ${\kappa }_{c,s}$ (scaling factors for the first stage) is chosen as ${\kappa }_c^{\star }$ and is obtained by solving $n_c$ offline optimization problems as explained before. The scaling factor after the first stage ${\kappa }_{c,k}$ is chosen as $\beta {\kappa }_{c,k-1}$, where $\beta \ge 1$. The constraint Equation (14d) ensures that all the elements inside the box over-approximation of the constraint function reachable set satisfy the constraints. The objective function Equation (14a) of MS-CB NMPC is similar to the standard multi-stage NMPC Equation (5a). MS-CB NMPC guarantees robust constraint satisfaction if the scaling factor ${\kappa }_c^{\star }$ is obtained by solving an offline global optimization problem and the optimization problem Equation (14) is recursively feasible. The recursive feasibility of the MS-CB NMPC optimization problem must be verified using simulation studies.

A short comparison between the complexity of the different robust multi-stage NMPC optimization problems is given in

Table 1. A complexity comparison of different robust multi-stage NMPC optimization problem. $N_{br}$ and $N_{ar}$ represent the number of nodes considered in the scenario tree for $k<N_r$ and for $N_r\le k\le N_p$ and are given by $\frac{N_b^{N_r}-1}{N_b-1}$ and $N_b^{N_r}(N_p-N_r+1)$. $N_n=N_{br}+N_{ar}$. OV—Optimization variables.

Number of		MS	MS-VA	MS-SB	MS-CB
Samples $\left(N_b\right)$ in Set $\mathcal{D}$		$3^{n_d}$	$2^{n_d}+1$	$2n_d+1$	$2n_d+1$
OV	model equations	$N_n{n}_x+$	$N_n{n}_x+$	$N_n{n}_x+$	$N_n{n}_x+$
	control inputs	$(N_n-1)n_u$	$(N_n-1)n_u$	$(N_n-1)n_u$	$(N_n-1)n_u$
	unscented transformation	−	−	$(n_x^2+n_x)N_{br}$	$(n_c^2+n_c)N_{br}$
constraints	model equations	$N_n{n}_x+$	$N_n{n}_x+$	$N_n{n}_x$	$N_n{n}_x+$
	control bounds	$2(N_n-1)n_u+$	$2(N_n-1)n_u+$	$2(N_n-1)n_u+$	$2(N_n-1)n_u+$
	non-anticipativity constraints	$\frac{{(N_b-1)}^{N_r+1}-n_u}{N_b-2}$	$\frac{n_u{(N_b-1)}^{N_r+1}-n_u}{N_b-2}$	$\frac{n_u{(N_b-1)}^{N_r+1}-n_u}{N_b-2}$	$\frac{n_u{(N_b-1)}^{N_r+1}-n_u}{N_b-2}$
	additional constraint $\mathit{g}(·)$	$(N_n-1)n_c$	$(N_n-1)n_c$	$(N_n-1)n_c$	$(N_n-1)n_c$
	unscented transformation	−	−	$(n_x^2+n_x)N_{br}$	$(n_c^2+n_c)N_{br}$
	enforcing $\mathit{g}(·)$ for reachable set	−	−	$(2^{n_x}+1)N_{br}{n}_c$	$N_{br}{n}_c$

. The optimization variables required for the computation of the model predictions, the control inputs, and the unscented transformation (mean and covariance matrix) and the constraints, and the unscented transformation and constraints for enforcing control bounds, non-anticipativity constraints, additional constraints on all nodes in the scenario tree and the reachable sets (obtained using the mean and the covariance matrix) are reported in Table 1. The scenario tree of MS NMPC and MS-VA NMPC considers $3^{n_d}$ and $2^{n_d}+1$ branches at each node whereas MS-SB and MS-CB NMPC schemes consider $2n_d+1$ branches at each node in their scenario tree. The number of nodes considered in the scenario tree of all formulations is given by $\left(N_b^{N_r}-1\right)/\left(N_b-1\right)+N_b^{N_r}(N_p-N_r+1)$. The computational complexity of MS NMPC and MS-VA NMPC grows exponentially with respect to the number of uncertain parameters whereas it grows linearly for MS-SB and MS-CB NMPC approaches. MS-VA NMPC considers more scenarios in its scenario tree than MS-SB and MS-CB NMPC approaches for $n_d>2$ and may require more computational effort than MS-SB and MS-CB NMPC to solve them. The MS-SB and MS-CB NMPC optimization problems require additional optimization variables for the computation of the mean and of the covariance matrix of the nonlinear transformation and are reported as OV of the unscented transformation in Table 1. The computational complexity of the MS-CB NMPC increases with the number of constraints $\left(n_c\right)$ due to the computation of the constraint covariance matrix, whereas the computational complexity of MS-SB NMPC increases with the number of states $\left(n_x\right)$ due to computation of the state covariance matrix. MS-SB NMPC may result in a performance loss when compared to MS-CB NMPC due to the box over-approximation of the set of reachable states. The scaling factor of MS-SB NMPC must be chosen properly for robust constraint satisfaction (see Remark 3).

4. Adaptive Robust Multi-Stage NMPC

The principle of adaptive control suggests using all the information available from the system during operation to improve the performance of the controller. The measurement information can be used to improve the knowledge about the plant thereby reducing the range of the uncertainty associated with the uncertain parameters (i.e., in our case the size of the ellipsoidal set $\mathbb{D}({\mathit{d}}_0,{\mathit{P}}_0)$). An estimate of the uncertain parameters can be obtained by solving a least-squares estimation problem using all the measurements collected until time t (i.e., s measurements from the plant). The formulation of the estimation problem reads as Equation (15):

$$\begin{array}{c}\hfill {\mathit{d}}_s=arg\underset{\mathit{d}}{min}\sum _{k=0}^s{({\mathit{x}}_{k+1}^m-{\mathit{x}}_{k+1})}^T\mathit{Q}({\mathit{x}}_{k+1}^m-{\mathit{x}}_{k+1})\end{array}$$

(15a)

subject to

$$\begin{array}{ccc}\hfill {\mathit{x}}_{k+1}=\mathit{f}({\mathit{x}}_k,{\mathit{u}}_k,\mathit{d}),& \hfill & \hfill \forall k\in \{0,\dots ,s-1\},\end{array}$$

(15b)

where ${\mathit{d}}_s$ represents the nominal value of the uncertain parameters and $\mathit{Q}$ denotes the inverse of the variance-covariance matrix of the measurement noise. ${\mathit{x}}_0$ represents the initial condition of the system and $u_k$ denotes the sequence of control inputs that have been applied to the plant.

If we assume that the model is structurally identifiable, and white Gaussian noise is superposed on the measurements, a joint-confidence region of parameter estimates depending upon the information content of the data can be obtained according to the Cramer-Rao inequality [24]. The parametric variance-covariance matrix can be over-approximated using the inverse of the Fisher information matrix. The Fisher information matrix is given by Equation (16):

$$\begin{array}{c}\hfill {\mathit{F}}_s=\sum _{k=0}^s{\mathit{s}}_k^T\mathit{Q}{\mathit{s}}_k,\end{array}$$

(16)

where ${\mathit{F}}_s$ denotes the Fisher information matrix obtained using measurements observed from time 0 to t, and ${\mathit{s}}_k=\frac{\partial {\mathit{x}}_k}{\partial \mathit{d}}$ represents the sensitivities of the states with respect to the uncertain parameters in the presence of full state measurements. A joint-confidence ellipsoid centered at the value of the least-squares parameter estimate that bounds the values of the least-squares parameter estimates (under different realizations of the measurement noise) with a prescribed confidence level $\alpha \left(z\right)=\frac{2}{\sqrt{\pi }}{\int }_0^{\frac{z}{\sqrt{2}}}{e}^{-y^2}dy$ ( $z=1$ corresponds to the $1\sigma$ confidence level of a Gaussian distribution) is given by Equation (17):

$${(\mathit{d}-{\mathit{d}}_s)}^T{\mathit{F}}_s(\mathit{d}-{\mathit{d}}_s)\le n_d{F}_{dist}(n_d,s{n}_x-n_d,\alpha \left(z\right)),$$

(17)

where $F_{dist}$ represents the upper $\alpha \left(z\right)$ quantile of the Fisher distribution with $n_d$ and $s{n}_x-n_d$ degrees of freedom in the numerator and denominator (see Equation (18)). ${\mathit{P}}_s$ represents the upper bound on the parameter covariance matrix with the confidence level $\alpha \left(z\right)$

$$\begin{array}{c}\hfill {\mathit{P}}_s=\frac{1}{n_d{F}_{dist}(n_d,s{n}_x-n_d,\alpha \left(z\right))}{\mathit{F}}_s^{-1}.\end{array}$$

(18)

We assume that the true realization of the uncertain parameter always lies within the confidence region of the uncertain parameter obtained with the chosen confidence level $\alpha \left(z\right)$ (i.e., we assume that $\mathbb{D}({\mathit{d}}_s,{\mathit{P}}_s)$ has a confidence level of $100\%$ instead of $\alpha \left(z\right)$ which is approximately satisfied if z = 3 sigma).

Remark 5.

The computational complexity of the optimization problem (15) increases as $t\to \infty$. This can be overcome by solving the parameter estimation problem (15) in a moving horizon fashion [63], where instead of using all the available measurements from the plant only the recent measurements within a window of chosen size are considered.

4.1. Adaptive Standard Multi-Stage NMPC

The adaptive standard multi-stage NMPC (A-MS) and adaptive multi-stage NMPC based on the vertex over-approximation (A-MS-VA) update their scenario trees based on the observed plant measurements. A tight box over-approximation of the intersection between the initial ellipsoidal confidence region $\mathbb{D}({\mathit{d}}_0,{\mathit{P}}_0)$, and the ellipsoidal confidence region can be obtained using the observed measurements, by solving the following optimization problem (Equation (19)).

$$\begin{array}{c}\hfill {\underline{\mathit{d}}}_s,{\overline{\mathit{d}}}_s=arg\underset{{\underline{\mathit{d}}}_o,{\overline{\mathit{d}}}_o\forall o\in I_d}{min}-\sum _{o=1}^{n_d}{\underline{\mathit{d}}}_{o,\left[o\right]}+\sum _{o=1}^{n_d}{\overline{\mathit{d}}}_{o,\left[o\right]}\end{array}$$

(19a)

subject to

$$\begin{array}{cccc}\hfill & {({\underline{\mathit{d}}}_o-{\mathit{d}}_0)}^T{\mathit{P}}_0^{-1}({\underline{\mathit{d}}}_o-{\mathit{d}}_0)\le 1,\hfill & \hfill & \forall o\in I_d,\hfill \end{array}$$

(19b)

$$\begin{array}{cccc}\hfill & {({\underline{\mathit{d}}}_o-{\mathit{d}}_s)}^T{\mathit{P}}_s^{-1}({\underline{\mathit{d}}}_o-{\mathit{d}}_s)\le 1,\hfill & \hfill & \forall o\in I_d,\hfill \end{array}$$

(19c)

$$\begin{array}{cccc}\hfill & {({\overline{\mathit{d}}}_o-{\mathit{d}}_0)}^T{\mathit{P}}_0^{-1}({\overline{\mathit{d}}}_o-{\mathit{d}}_0)\le 1,\hfill & \hfill & \forall o\in I_d,\hfill \end{array}$$

(19d)

$$\begin{array}{cccc}\hfill & {({\overline{\mathit{d}}}_o-{\mathit{d}}_s)}^T{\mathit{P}}_s^{-1}({\overline{\mathit{d}}}_o-{\mathit{d}}_s)\le 1,\hfill & \hfill & \forall o\in I_d.\hfill \end{array}$$

(19e)

where $I_d:=\{1,\dots ,n_d\}$. $n_d$ parameter combinations (${\underline{\mathit{d}}}_o$ and ${\overline{\mathit{d}}}_o$) are chosen to compute the lower and upper bounds on the ellipsoidal intersection region [64].

The formulation of the A-MS NMPC optimization problem solved at time t ($s^{\mathrm{th}}$ iteration) is similar to the MS NMPC optimization problem Equation (5), where Equations (5g) and (5h) are replaced by Equation (20):

$$\begin{array}{cccc}\hfill & {\underline{\mathit{d}}}_{\left[o\right]}={\underline{\mathit{d}}}_{s,\left[o\right]},{\overline{\mathit{d}}}_{\left[o\right]}={\overline{\mathit{d}}}_{s,\left[o\right]},\hfill & \hfill & \forall o\in I_d,\hfill \end{array}$$

(20a)

$$\begin{array}{cc}\hfill & {\displaystyle \mathcal{D}}={\mathbb{C}}_a(\underline{\mathit{d}},{\mathit{d}}_s,\overline{\mathit{d}}).\hfill \end{array}$$

(20b)

The formulation of the A-MS-VA NMPC optimization problem solved at time t ($s^{\mathrm{th}}$ iteration) is similar to the A-MS NMPC problem, where instead of Equation (20) the following equations (Equation (21)) are used

$$\begin{array}{cccc}\hfill & {\underline{\mathit{d}}}_{\left[o\right]}={\underline{\mathit{d}}}_{s,\left[o\right]},{\overline{\mathit{d}}}_{\left[o\right]}={\overline{\mathit{d}}}_{s,\left[o\right]},\hfill & \hfill & \forall o\in I_d,\hfill \end{array}$$

(21a)

$$\begin{array}{cc}\hfill & {\displaystyle \mathcal{D}}={\mathit{d}}_s\cup {\mathbb{C}}_{vp}(\underline{\mathit{d}},\overline{\mathit{d}}).\hfill \end{array}$$

(21b)

$\underline{\mathit{d}}$ and $\overline{\mathit{d}}$ represent the lower and upper bounds on the uncertain parameters which bound the intersection of the ellipsoidal confidence regions $\mathbb{D}({\mathit{d}}_0,{\mathit{P}}_0)$ and $\mathbb{D}({\mathit{d}}_s,{\mathit{P}}_s)$. Whenever a measurement from the plant becomes available, the new parameter estimates are obtained by solving the least-squares estimation problem Equation (15), and the corresponding parameter covariance matrix is computed using Equation (18). The optimization problem Equation (19) is then solved to compute the new bounds on the uncertain parameters. The scenario trees of the A-MS/A-MS-VA NMPC optimization problem are updated according to Equation (20) or Equation (21) and the adapted optimization problem is solved to compute the optimal control input which is applied to the plant.

4.2. Adaptive Multi-Stage NMPC Using Sigma Points

The adaptive multi-stage NMPC formulation based on the box over-approximation of the reachable states set (A-MS-SB) and the adaptive multi-stage NMPC formulation based on the box over-approximation of the reachable constraint function set (A-MS-CB) also use updated information about the uncertain parameters to improve the performance of multi-stage NMPC using sigma points. The true value of the uncertain parameters will be contained in the intersection region between the ellipsoidal confidence region used in the previous A-MS-SB and A-MS-CB NMPC optimization problem, and the confidence region of the uncertain parameters obtained in addition using the current plant measurements $\left(\mathbb{D}({\mathit{d}}_s,{\mathit{P}}_s)\right)$. We can compute an ellipsoidal over-approximation of the intersection region between two ellipsoids as shown in [65] and then update the scenario tree of A-MS-SB and A-MS-CB NMPC using the sigma points that result from the over-approximating ellipsoid.

The formulations of A-MS-SB NMPC and A-MS-CB NMPC are similar to MS-SB NMPC and MS-CB NMPC Equations (10) and (14) where Equation (10b) are replaced by the following Equation (22):

$$\begin{array}{cc}\hfill & {\mathit{P}}_s^{\star }=(1-\varphi (1-\varphi ){({\mathit{d}}_s-{\mathit{d}}_{s-1}^{\star })}^T{\mathit{P}}_s^{-1}{\widehat{\mathit{F}}}^{-1}{\left({\mathit{P}}_{s-1}^{\star }\right)}^{-1}({\mathit{d}}_s-{\mathit{d}}_{s-1}^{\star })){\widehat{\mathit{F}}}^{-1},\hfill \end{array}$$

(22a)

$$\begin{array}{cc}\hfill & {\mathit{d}}_s^{\star }={\widehat{\mathit{F}}}^{-1}(\varphi {\left({\mathit{P}}_{s-1}^{\star }\right)}^{-1}{\mathit{d}}_{s-1}^{\star }+(1-\varphi ){\mathit{P}}_s^{-1}{\mathit{d}}_s),\hfill \end{array}$$

(22b)

$$\begin{array}{cc}\hfill & {\displaystyle \mathcal{D}}=\mathbb{S}({\mathit{d}}_s^{\star },{\mathit{P}}_s^{\star }),\hfill \end{array}$$

(22c)

where $\widehat{\mathit{F}}=\varphi {\left({\mathit{P}}_{s-1}^{\star }\right)}^{-1}+(1-\varphi ){\mathit{P}}_s^{-1}$. ${\mathit{d}}_s^{\star }$ and ${\mathit{P}}_s^{\star }$ give the mean and the covariance matrix of an over-approximating ellipsoid which over-approximates the intersection between $\mathbb{D}({\mathit{d}}_{s-1}^{\star },{\mathit{d}}_{s-1}^{\star })$ and $\mathbb{D}({\mathit{d}}_s,{\mathit{d}}_s)$, and are obtained using Equations (22a) and (22b). At time $t=0$, ${\mathit{d}}_{s-1}^{\star }={\mathit{d}}_0$ and ${\mathit{P}}_{s-1}^{\star }={\mathit{P}}_0$. $\varphi$ is a degree of freedom of the A-MS-SB and A-MS-CB NMPC optimization problem and is bounded between 0 and 1. The scenario tree is generated using the parameter combinations given as the sigma points of the over-approximating ellipsoid $\mathbb{D}({\mathit{d}}^{\star },{\mathit{P}}^{\star })$, see Equation (22c).

Remark 6.

The key difference between the adaptive version of multi-stage NMPC (A-MS NMPC and A-MS-VA NMPC) and multi-stage NMPC using sigma points (MS-SB NMPC and MS-CB NMPC) lies in the approximation of the intersection region between two ellipsoids to generate the scenario tree. Standard adaptive multi-stage NMPC approaches over-approximate the intersection region by a minimum perimeter box which can be obtained by solving the optimization problem Equation (19). The adaptive version of multi-stage NMPC using sigma points over-approximates the intersection region by an ellipsoid, which is computed as a part of the NMPC optimization problem. In the NMPC optimization problem an over-approximating ellipsoid from a set of candidate ellipsoids is computed that is optimal for adaptive multi-stage NMPC using the sigma points approaches. This comes at the expense of an additional computational cost.

Remark 7.

The tuning parameters ${\kappa }_s$, ${\kappa }_c$, and ${\kappa }_o$ that are obtained by solving an optimization problem a-priori as mentioned in Section 3.2.1 and Section 3.2.2 are valid only for the robust multi-stage NMPC using sigma points. In the A-MS-SB and A-MS-CB optimization problems, ellipsoidal over-approximations of the intersection region are computed as part of the robust NMPC optimization problem. Here, the scaling factors ${\kappa }_s$, ${\kappa }_c$, and ${\kappa }_o$ have to be tuned based on trial-error using simulation studies.

5. Case Studies

The performance of the different robust multi-stage NMPC controllers is compared using two case studies. A simple benchmark semi-batch reactor example adapted from [66] is chosen to perform a detailed analysis of the performance of robust multi-stage NMPC for different values of the tuning parameters. A more complex industrial semi-batch polymerization reactor [67] is chosen to demonstrate the performance of different control schemes. The controllers described in Section 3 are referred to as non-adaptive controllers and the controllers described in Section 4 as adaptive controllers.

5.1. Case Study 1: Benchmark Semi-Batch Reactor

The semi-batch reactor benchmark example from [66] is adapted to investigate the advantages of the proposed schemes. The chemical reaction under consideration is given by

$$\begin{array}{c}\hfill \mathrm{A}+\mathrm{B}\to \mathrm{C}.\end{array}$$

The nonlinear model is obtained from the mass balance of the reactor, molar balances of the reactants A and B, and energy balances of the reactor and the jacket. The model equations are given below Equation (23):

$$\begin{array}{cc}\hfill {\dot{V}}_{\mathrm{R}}& ={\dot{V}}_{\mathrm{in}},\hfill \end{array}$$

(23a)

$$\begin{array}{cc}\hfill {\dot{c}}_{\mathrm{A}}& =-\frac{{\dot{V}}_{\mathrm{in}}}{V_{\mathrm{R}}}{c}_{\mathrm{A}}-K{c}_{\mathrm{A}}{c}_{\mathrm{B}},\hfill \end{array}$$

(23b)

$$\begin{array}{cc}\hfill {\dot{c}}_{\mathrm{B}}& =\frac{{\dot{V}}_{\mathrm{in}}}{V_{\mathrm{R}}}(c_{\mathrm{B},}\mathrm{in}-c_{\mathrm{B}})-K{c}_{\mathrm{A}}{c}_{\mathrm{B}},\hfill \end{array}$$

(23c)

$$\begin{array}{cc}\hfill {\dot{T}}_{\mathrm{R}}& =\frac{{\dot{V}}_{\mathrm{in}}}{V_{\mathrm{R}}}(T_{\mathrm{in}}-T)-\frac{\alpha A_{\mathrm{w}}(T_{\mathrm{R}}-T_{\mathrm{J}})}{\rho V_{\mathrm{R}}{c}_p}-\frac{K{c}_{\mathrm{A}}{c}_{\mathrm{B}}H}{\rho c_p},\hfill \end{array}$$

(23d)

$$\begin{array}{cc}\hfill {\dot{T}}_{\mathrm{J}}& =\frac{{\dot{Q}}_{\mathrm{K}}+\alpha A_{\mathrm{w}}(T_{\mathrm{R}}-T_{\mathrm{J}})}{\rho V_{\mathrm{J}}{c}_p},\hfill \end{array}$$

(23e)

with

$$\begin{array}{cc}\hfill c_{\mathrm{C}}& =\frac{c_{\mathrm{A},0}{V}_{\mathrm{R},0}+c_{\mathrm{C},0}{V}_{\mathrm{R},0}-c_{\mathrm{A}}{V}_{\mathrm{R}}}{V_{\mathrm{R}}},\hfill \end{array}$$

(23f)

$$\begin{array}{cc}\hfill A_{\mathrm{w}}& =\pi r^2+\frac{0.002V_{\mathrm{R}}}{r},\hfill \end{array}$$

(23g)

where $V_{\mathrm{R}}$ denotes the volume of the reactor, $c_{\mathrm{A}}$, $c_{\mathrm{B}}$, and $c_{\mathrm{C}}$ represent the concentrations of the reactants A and B and the product C. $T_{\mathrm{R}}$, and $T_{\mathrm{J}}$ represent the temperature of the contents inside the reactor and the jacket. $A_{\mathrm{w}}$ denotes the inner surface area of the reactor covered with the reaction mixture. The feed $\left(V_{\mathrm{in}}\right)$ and the cooling power of the jacket $\left({\dot{Q}}_k\right)$ are the control inputs.

5.1.1. Formulation of the Optimal Control Problem

The control task is to maximize the number of moles of product C that are produced along the prediction horizon while satisfying the constraints until the specified end time of $1.0\mathrm{h}$ is reached. The sampling time $t_s$ is chosen as $0.05\mathrm{h}$. The temperature of the reactor $T_{\mathrm{R}}$ must be kept between $322\mathrm{K}$ and $326\mathrm{K}$. The volume of the reactor contents must not exceed $7\mathrm{L}$. It is assumed the specific reaction rate constant K and the reaction enthalpy H are not known precisely. The parameters are contained in an ellipsoidal confidence region described by their nominal value and the parameter covariance matrix (see Equation (24)).

$$\begin{array}{c}\hfill {\mathit{d}}_0=\left(\begin{array}{c}H\\ K\end{array}\right)=\left(\begin{array}{c}-355\frac{\mathrm{kJ}}{\mathrm{mol}}\\ 1.205\frac{\mathrm{l}}{\mathrm{mol}\mathrm{h}}\end{array}\right),{\mathit{P}}_0=\left(\begin{array}{cc}11300& -7.7\\ -7.7& 0.131\end{array}\right).\end{array}$$

(24)

All other parameters in the model equations are considered to be known and are given along with a short description in

Table 2. Case study 1: Model parameters.

Parameter	Description	Value	Unit
$\alpha$	heat-transfer coefficient between the reactor and jacket	1700	$\mathrm{kJ}{\mathrm{K}}^{-1}{\mathrm{h}}^{-1}{\mathrm{m}}^{-2}$
r	radius of the cross-section of the inner part	$0.092$	m
$\rho$	density of the reactor contents	1000	$\mathrm{g}{\mathrm{L}}^{-1}$
$c_p$	specific heat capacity of the reactor contents	$4.2$	$\mathrm{J}{\mathrm{g}}^{-1}{\mathrm{K}}^{-1}$
$c_{\mathrm{B},\mathrm{in}}$	input concentration of reactant B	3	$\mathrm{mol}{\mathrm{L}}^{-1}$
$V_{\mathrm{J}}$	volume of the contents inside the cooling jacket	$2.22$	L
$T_{\mathrm{in}}$	temperature of the flows entering the reactor	300	K
$c_{\mathrm{C},0}$	initial concentration of the product C	0	$\mathrm{mol}{\mathrm{L}}^{-1}$

. All the states ($\mathit{x}={(V_{\mathrm{R}},c_{\mathrm{A}},c_{\mathrm{B}},T_{\mathrm{R}},T_{\mathrm{J}})}^T$) are assumed to be measured with measurement noise of standard deviation $\mathit{\sigma }={(0.0001\mathrm{L},0.01\mathrm{mol}{\mathrm{L}}^{-1},0.01\mathrm{mol}{\mathrm{L}}^{-1},0.1\mathrm{K},0.1\mathrm{K})}^T$. The control vector ($\mathit{u}$) is represented as ${({\dot{V}}_{\mathrm{in}},{\dot{Q}}_k)}^T$. The bounds on the control inputs are given in

Table 3. Case study 1: Bounds on the control inputs.

Control	Lower Bound	Upper Bound	Unit
${\dot{V}}_{\mathrm{in}}$	0	$32.4$	$\mathrm{L}{\mathrm{h}}^{-1}$
${\dot{Q}}_{\mathrm{K}}$	$-9000$	0	$\mathrm{kJ}{\mathrm{h}}^{-1}$

. The states are not constrained but reasonable bounds on the states are chosen to reduce the computational effort. The bounds on the states along with their initial values are given in

Table 4. Case study 1: Initial conditions of the states along with practical bounds.

State	Initial Condition	Lower Bound	Upper Bound	Unit
$V_{\mathrm{R}}$	$3.5$	0	8	L
$c_{\mathrm{A}}$	2	0	5	$\mathrm{mol}{\mathrm{L}}^{-1}$
$c_{\mathrm{B}}$	0	0	5	$\mathrm{mol}{\mathrm{L}}^{-1}$
$T_{\mathrm{R}}$	325	273	350	K
$T_{\mathrm{J}}$	325	273	350	K

.

The nominal MPC optimization problem that is solved at time t ($s^{\mathrm{th}}$ iteration) reads as Equation (25):

$$\begin{array}{c}\hfill \underset{{\mathit{x}}_k,{\mathit{u}}_k,{\mathit{ϵ}}_k}{min}\sum _{k=s}^{s+N_p-1}-c_{\mathrm{C}}{V}_{\mathrm{R}}+0.0154{(\Delta {\dot{V}}_{\mathrm{in},k})}^2+5.5\times {10}^{-5}{(\Delta {\dot{Q}}_k)}^2+{10}^6{\mathit{ϵ}}_{k,\left[1\right]}^2+{10}^{10}{\mathit{ϵ}}_{k,\left[2\right]}^2,\end{array}$$

(25a)

subject to

$$\begin{array}{cc}\hfill & {\mathit{x}}_{k+1}=\mathit{f}({\mathit{x}}_k,{\mathit{u}}_k,{\mathit{d}}_k),\hfill \end{array}$$

(25b)

$$\begin{array}{cc}\hfill & 322\le T_{\mathrm{R},k}+{\mathit{ϵ}}_{k,\left[1\right]}\le 326,\hfill \end{array}$$

(25c)

$$\begin{array}{cc}\hfill & V_{\mathrm{R},k}+{\mathit{ϵ}}_{k,\left[2\right]}\le 7,\hfill \end{array}$$

(25d)

$$\begin{array}{cc}\hfill & -1\le {\mathit{ϵ}}_{k,\left[1\right]}\le 1,\hfill \end{array}$$

(25e)

$$\begin{array}{cc}\hfill & -0.01\le {\mathit{ϵ}}_{k,\left[2\right]}\le 0.01,\hfill \end{array}$$

(25f)

$$\begin{array}{cc}\hfill & \underline{\mathit{u}}\le {\mathit{u}}_k\le \overline{\mathit{u}},\hfill \end{array}$$

(25g)

$$\begin{array}{cc}\hfill & {\mathit{x}}_s={\mathit{x}}_s^m,\hfill \end{array}$$

(25h)

where $\Delta {\dot{V}}_{\mathrm{in},k}={\dot{V}}_{\mathrm{in},k-}-{\dot{V}}_{\mathrm{in},k-1}$, and $\Delta {\dot{Q}}_k={\dot{Q}}_k-{\dot{Q}}_{k-1}$. The stage cost is chosen to maximize the number of moles of product C produced along the prediction horizon and to penalize the control moves and the constraint violations. The weights for penalizing the control moves are chosen to prevent drastic changes of the inputs. The model Equation (23) is discretized using orthogonal collocation on the finite elements [68,69] and is given in Equation (25b). The collocation points are obtained using the Legendre method [70]. The number of finite elements and the degree of the polynomial are chosen as 3 and 1. The additional constraints that must be satisfied are given by Equations (25c) and (25d). These constraints are implemented as soft constraints to prevent the MPC optimization from becoming infeasible due to the influence of measurement noise. The maximum constraint violation that can happen is bounded by Equations (25e) and (25f). The control bounds are enforced using Equation (25g), where ${\underline{\mathit{u}}}_k$ and ${\overline{\mathit{u}}}_k$ represent the lower and upper bounds that can be obtained from Table 3. The plant is initialized at the current plant measurements using Equation (25h), where ${\mathit{x}}_s^m\in {\mathbb{R}}^5$ represents the measurement vector obtained at time t.

5.1.2. Results Obtained Using the Non-Adaptive and the Adaptive Controllers for One Set of Parameters

The reference used for the comparison of the different multi-stage NMPC strategies considers the nominal value of the uncertain parameter to be the true realization of the uncertain parameter. The prediction horizon $\left(N_p\right)$ and the robust horizon $\left(N_r\right)$ of the variants of multi-stage NMPC are chosen as 5 and 2. All the nodes in the scenario trees of multi-stage NMPC are equally weighted (i.e., $w_k^i=\frac{1}{N_n},\mathrm{s}.\mathrm{t}.{\sum }_{i=1}^{N_n}{w}_k^i=1$, where $N_n$ is the number of nodes at time step k in the prediction horizon). The scaling factors ${\kappa }_s$ and ${\kappa }_c$ of the multi-stage NMPC using sigma points were obtained as mentioned in Section 3.2. The optimization problems are solved using the global optimization toolbox from MATLAB [62]. The optimal values of the tuning parameters ${\kappa }_s^{\star }$ and ${\kappa }_c^{\star }$ resulted as $1.57$ and $1.56$. The scaling increase factor $\left(\beta \right)$ is chosen as $1.02$. The value of $\alpha$ is chosen according to the $3\sigma$ range of the variables for the adaptive NMPC approaches. The plant measurements are obtained by adding white Gaussian noise with standard deviation $\sigma$ to the true plant states. The weights on the sigma points $\left(\mathit{v}\right)$ chosen to build the covariance matrix of the MS-SB and MS-CB NMPC are given as ${(0.2,0.2,0.2,0.2,0.2)}^T$.

We use the number of moles of the product C after $0.3$ h of the batch time as an indicator of the performance because this is a measure of the efficiency of the process in the first phase. When the full amount of B has been fed, the further evolution of the process is determined by the kinetics alone and the final batch time depends on the specified conversion. The results obtained when the plant is controlled using the different NMPC strategies are shown in Figure 5. The green plots show the simulation results obtained when the plant is controlled using nonlinear model predictive control under the assumption that the true plant model is known. The optimal operation is to feed as much reactant B as possible while respecting the constraints Equations (25c) and (25d). The temperature of reactant B entering into the reactor $\left(T_{\mathrm{in}}\right)$ is less than the reactor temperature $T_{\mathrm{R}}$, hence the reference NMPC at the beginning tries to feed as much material as possible while satisfying the lower bound on the reactor temperature. Once the reactor is filled, the reactor temperature increases due to the exothermic reaction taking place inside the reactor and the reactor temperature hits the upper bound. The reference NMPC solution maintains the reactor temperature within its limits until the batch end by manipulating its cooling power $\left({\dot{Q}}_K\right)$ as the true plant and the nominal plant model are identical.

MS NMPC feeds a smaller amount of reactant B due to the presence of large uncertainty in the parameters and the tight specification on the admissible reactor temperature. The scenarios generated using the upper bound on the reaction enthalpy and the lower bound on the reaction rate hit the lower constraint on the reactor temperature, whereas the scenarios generated using the lower bound on the reaction enthalpy and the upper bound on the reaction rate hit the upper constraint on the reactor temperature in the predictions and are shown in Figure 6, where the blue lines indicate the states and the control inputs predicted by the scenario tree of the MS NMPC optimization problem solved at time $t=0.15\mathrm{h}$. The lower bound constraint on the reactor temperature prevents MS NMPC from using the full cooling power of the plant to increase the feed. The number of moles of reactant B fed into the reactor using MS-VA NMPC is similar to the number of moles of reactant B fed using the MS NMPC approach because the scenario trees of both formulations include the scenarios built using the upper bound on the reaction enthalpy and the lower bound on the reaction rate as well as using the lower bound on the reaction enthalpy and the upper bound on the reaction rate which prevent the controllers from feeding more reactants into the reactor.

The number of moles of reactant B fed into the reactor is increased when using the multi-stage NMPC variants based on sigma points. There is a $29\%$ increase in the number of moles of product C produced for the MS-SB and MS-CB NMPC variants over MS and MS-VA NMPC. The MS-SB and MS-CB NMPC variants build the scenario tree using the sigma points and tightly approximate the uncertainty set in contrast to MS and MS-VA NMPC where the scenario tree is built based on the box over-approximation of the uncertainty set. The constraints Equations (25d) and (25c) are linear with respect to the states and concern only one state ($T_{\mathrm{R}}$ or $V_{\mathrm{R}}$), hence the performance of the MS-SB NMPC is similar to the MS-CB NMPC. The tuning parameter ${\kappa }_s$ was obtained a-priori by solving the optimization problem Equation (8) and this guarantees robust constraint satisfaction of MS-SB NMPC since the constraints are linear in the states. All the non-adaptive robust NMPC schemes satisfy the constraints on the reactor temperature and reactor volume until the batch end.

The time taken to solve one iteration of MS NMPC, MS-VA, MS-SB, and MS-CB NMPC are $1.91\mathrm{s}$, $0.56\mathrm{s}$, $0.63\mathrm{s}$, and $0.44\mathrm{s}$, in average. MS NMPC considers 9 branches at each node in the scenario tree in contrast to all the other NMPC approaches where only 5 scenarios are considered. There is a small increase in the computational time of MS-SB NMPC when compared to MS-VA NMPC due to the computation of the state covariance matrix along the robust horizon. The computation time of MS-CB NMPC is less than that of MS-SB NMPC because only two constraints $n_c=2$ are present and the computation of the constraint covariance matrices reduces the computational effort compared to the computation of the state covariance matrix with $n_x=5$.

The number of moles of reactants fed into the reactor in the initial phase can be improved using the adaptive NMPC approaches. There is a $47\%$ increase in the number of moles of product C produced within the first $0.3$ h when using the A-MS and A-MS-VA NMPC strategies over the MS and MS-VA NMPC approach and a $17\%$ increase in the number of moles of product C produced from using the A-MS-SB and A-MS-CB NMPC formulations over the A-MS and A-MS-VA NMPC.

The confidence regions of the uncertain parameters at time $t=0.05\mathrm{h}$ ($2^{\mathrm{nd}}$ NMPC iteration) is shown in Figure 7. The confidence region considered by A-MS-SB NMPC is similar to that of A-MS-CB NMPC, hence it is not plotted. A-MS and A-MS-VA NMPC schemes approximate the intersection between the initial confidence region and the confidence region obtained using the plant measurements by a box whereas the A-MS-SB and A-MS-CB NMPC schemes approximate the intersection region by an ellipsoid. The parameter combinations that are used to build the scenario trees of the multi-stage NMPC formulations are shown by green squares (■). The box over-approximation includes additional uncertainty by the upper and lower bounds on the uncertain parameters which result in a performance loss. The tuning parameters ${\kappa }_{s,0}$, and ${\kappa }_{c,0}$ that are computed by the non-adaptive NMPC approaches by solving an optimization problem are assumed to be valid for the adaptive approaches as well. The scaling factors are validated using the plant measurements, where it is verified whether the box over-approximation of the reachable states and the constraint function set predicted at stage $k+1$ of A-MS-SB and A-MS-CB NMPC optimization problems solved at ${(s-1)}^{\mathrm{th}}$ iteration contains the plant measurement ($x_s^m$) observed at time t and the constraint function value obtained using the plant measurements and the applied control input, respectively.

The times taken to solve one iteration of A-MS NMPC, A-MS-VA, A-MS-SB, and A-MS-CB NMPC on average are $1.85\mathrm{s}$, $0.57\mathrm{s}$, $3.4\mathrm{s}$, and $1\mathrm{s}$. The computation time of A-MS and A-MS-VA NMPC is similar to that of MS and MS-VA NMPC because the box over-approximation of the intersection region is computed by solving the optimization problem Equation (21) separately and then the scenario tree of the multi-stage NMPC is updated. The increase in the computational time of A-MS-SB and A-MS-CB NMPC when compared to MS-SB and MS-CB NMPC results from the computation of an ellipsoidal over-approximation of the intersection region between two ellipsoids as a part of the NMPC optimization problem.

5.1.3. Controller Performance Analysis for Different Values of the Tuning Parameters

The results obtained for different choices of the tuning parameters and 100 realizations of the uncertain parameters are presented in this section. The realizations are marked by black dots (●) in Figure 8. All tuning parameters other than the tuning parameter for which the controller performance is evaluated are the same as in the reference case. We evaluate the performance of the controllers by the amount of product obtained at $0.3\mathrm{h}$. The measurements of the plant states were assumed to reduce the computational effort. The results are plotted using a violin plot [71]. The widths of the violins correspond to the normalized histogram plot. The blue color (■) represents multi-stage NNPC (MS NMPC, A-MS NMPC) using the approach in [9], the magenta color (■) represents multi-stage NMPC based on the vertex approximation (MS-VA NMPC, A-MS-VA NMPC), the black color (■) represents multi-stage NMPC based on the box over-approximation of the reachable states set (MS-SB NMPC, A-MS-SB NMPC) and the golden color (■) represents the proposed multi-stage NMPC based on the box over-approximation of the reachable set of the constraint function (MS-CB NMPC, A-MS-CB NMPC). The green diamond (◆) and the red line (━) represent the mean and the median value of the number of moles of product C that are produced at $0.3$ h of batch time (after 6 NMPC iterations) using the different robust NMPC strategies.

Effect of the Length of the Prediction Horizon

The amount of product C produced during the first $0.3$ h using the non-adaptive and adaptive NMPC schemes for different choices of the prediction horizon length $\left(N_p\right)$ is shown using a violin plot in Figure 9. The performance of the non-adaptive and of the adaptive controllers increases significantly from $N_p=2$ to $N_p=3$ and then slightly up to $N_p=5$. The NMPC formulations with shorter prediction horizon (i.e., $N_p\le 3$) were not able to predict the performance gain achieved at the end of the batch correctly and result in a loss of performance. The adaptive NMPC schemes using the sigma points fill the reactor at $0.25\mathrm{h}$ as shown in Figure 5, hence a prediction horizon of length 5 is sufficient for them to approximate the maximization of the number of moles of product C produced at $0.3$ h. The average computation time needed for solving one NMPC iteration for different lengths of the prediction horizon is indicated in Figure 10. The computational times of the adaptive and non-adaptive schemes increase with the length of the prediction horizon length.

Effect of the Length of the Robust Horizon

The amount of product C produced during the first $0.3$ h using the non-adaptive and adaptive NMPC schemes for different choices of the robust horizon length $\left(N_r\right)$ is shown in

Table 5. Case study 1: Minimum (min), mean and maximum (max) number of moles of product C produced in the first $0.3$ h using the non-adaptive and adaptive NMPC schemes with different lengths of robust horizon $\left(N_r\right)$ in the absence of measurement noise. A prediction horizon $\left(N_p\right)$ of length 5 is chosen.

	Robust	Non-Adaptive				Adaptive
	Horizon	MS	MS-VA	MS-SB	MS-CB	A-MS	A-MS-VA	A-MS-SB	A-MS-CB
min	2	$0.83$	$0.82$	$1.07$	$1.06$	$1.37$	$1.38$	$1.52$	$1.52$
	3	$0.74$	$0.76$	$0.98$	$0.97$	$1.13$	$1.21$	$1.34$	$1.34$
mean	2	$1.15$	$1.14$	$1.49$	$1.47$	$1.71$	$1.74$	$2.00$	$2.00$
	3	$1.02$	$1.07$	$1.37$	$1.36$	$1.50$	$1.60$	$1.82$	$1.82$
max	2	$1.52$	$1.52$	$1.99$	$1.97$	$2.26$	$2.27$	$2.50$	$2.5$
	3	$1.35$	$1.41$	$1.83$	$1.81$	$2.00$	$2.15$	$2.36$	$2.35$

. The number of moles of product C produced decreases with an increase in the length of the robust horizon. The multi-stage NMPC with $N_r=1$ assumes that the uncertainty realization remains constant after the first stage and provides an optimistic control strategy. The non-adaptive schemes with $N_r=1$ solved at $s=0$ feed more reactant B into the reactor under assumptions that the uncertainty realization becomes known and remains constant after the first stage. The multi-stage NMPC solved at the next time steps $s>1$ cannot find an optimal control input that satisfies the constraints for all scenarios along the robust horizon because of the control decision taken in the previous time step, the tight bounds on the reactor temperature and large initial confidence region of the uncertain parameters.

For multi-stage NMPC with $N_r=2$ the NMPC optimization problem for $s>0$ has an optimal control input that satisfies the constraints on the reactor temperature for all the scenarios considered in its scenario tree. Multi-stage NMPC with $N_r=3$ results in a small performance loss. A disadvantage of choosing a larger robust horizon is the exponential increase in the computational time of the multi-stage NMPC optimization problem. The computational times taken by the non-adaptive and adaptive NMPC schemes for different choices of the robust horizon are also reported in Figure 10. MS and A-MS NMPC with $N_r=2$ consider 81 scenarios and MS-VA, A-MS-VA, MS-SB, A-MS-SB, MS-CB and A-MS-CB NMPC with $N_r=2$ consider 25 scenarios. MS and A-MS NMPC for $N_r=3$ consider 729 scenarios and The MS-VA, A-MS-VA, MS-SB, A-MS-SB, MS-CB and A-MS-CB NMPC with $N_r=3$ consider 125 scenarios.

Effect of the Scaling Factor ${\kappa }_k$

The covariance matrix of the multi-stage NMPC using sigma points obtained using the unscented transformation (MS-SB, A-MS-SB, MS-CB, and A-MS-CB NMPC) is scaled using the scaling factor ${\kappa }_k$ such that the box over-approximation of the ellipsoids represented by the covariance matrix and mean encloses the set of reachable states and constraint function values. The scaling factor consists of two parts ${\kappa }_0$ and $\beta$. The scaling factor ${\kappa }_0$ is tuned such that reachable sets at the first stage are enclosed in the prediction horizon. The scaling increase factor $\beta$ reflects the uncertainty in the initial condition of the future states after the first stage.

Effect of Change in the Scaling Factor ${\kappa }_0$

The scaling factor ${\kappa }_0$ can be obtained using a trial-error method based on posterior analysis on simulation studies or by solving an optimization problem so that it is valid for the entire operating region of the plant. The number of moles of product C produced up to $0.3\mathrm{h}$ obtained using the MS-SB, A-MS-SB, MS-CB, and A-MS-CB NMPC formulations for different values of ${\kappa }_0$ are shown in Figure 11, where ${\kappa }_0={\kappa }^{\star }$ corresponds to the value of the scaling factor that was obtained by solving the optimization problem Equation (8) (i.e., ${\kappa }_{s,0}=1.57$, and ${\kappa }_{c,0}=1.56$).

As can be expected, the amount of product C produced in the initial phase of the batch run decreases with an increase in the value of scaling factor ${\kappa }_0$. The scaling factor ${\kappa }_0=1.25$ is less than ${\kappa }^{\star }$, this implies that the box over-approximation predicted at the first stage $(k=s+1)$ is not valid for the entire operating region of the plant but it is valid for the relevant operating region since the scaling factor ${\kappa }_0=1.25$ is not invalidated by the observed measurements. It can be seen that larger values of ${\kappa }_0$ reduce the performance.

Effect of change in scaling increase factor $\beta$

Figure 12 shows the number of moles of product C produced during the initial part of the batch run using non-adaptive and adaptive NMPC using sigma points for different values of the scaling increase factor $\beta$. The amount of product C decreases significantly with an increase in the value of $\beta$ for the non-adaptive versions, while for the adaptive cases, the influence is minor.

5.2. Case Study 2: Industrial Semi-Batch Polymerization Reactor

The industrial semi-batch polymerization reactor example is taken from [48]. It was provided by BASF SE as a case study for NMPC algorithms in the EU-project [67]. The process flow diagram of the industrial semi-batch reactor under consideration is shown in Figure 13. Monomer A is fed into the reactor, where it combines and produces the polymer P. The reaction is highly exothermic. The reactor is equipped with an external heat exchanger (EHE) and a cooling jacket to control the temperature of the reactor contents.

The model of the plant was obtained from mass and energy balances. It comprises 8 ordinary differential equations and is given below Equation (26):

$$\begin{array}{ccc}\hfill & {\dot{m}}_{\mathrm{W}}\hfill & ={\dot{m}}_{\mathrm{F}}{\omega }_{\mathrm{W},\mathrm{F}},\hfill \end{array}$$

(26a)

$$\begin{array}{ccc}\hfill & {\dot{m}}_{\mathrm{A}}\hfill & ={\dot{m}}_{\mathrm{F}}{\omega }_{\mathrm{A},\mathrm{F}}-k_{\mathrm{R}1}{m}_{\mathrm{A},\mathrm{R}}-\frac{k_{\mathrm{R}2}{m}_{\mathrm{AWT}}{m}_{\mathrm{A}}}{m_{\mathrm{ges}}},\hfill \end{array}$$

(26b)

$$\begin{array}{ccc}\hfill & {\dot{m}}_{\mathrm{P}}\hfill & =k_{\mathrm{R}1}{m}_{\mathrm{A},\mathrm{R}}+\frac{k_{\mathrm{R}2}{m}_{\mathrm{AWT}}{m}_{\mathrm{A}}}{m_{\mathrm{ges}}},\hfill \end{array}$$

(26c)

$$\begin{array}{ccc}\hfill & {\dot{T}}_{\mathrm{R}}\hfill & =\frac{{\dot{m}}_{\mathrm{F}}{c}_{\mathrm{p},\mathrm{F}}(T_{\mathrm{F}}-T_{\mathrm{R}})}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{ges}}}+\frac{H_{\mathrm{R}}{k}_{\mathrm{R}1}{m}_{\mathrm{A},\mathrm{R}}}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{ges}}}-\frac{k_{\mathrm{K}}A(T_{\mathrm{R}}-T_{\mathrm{S}})}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{ges}}}-\frac{{\dot{m}}_{\mathrm{AWT}}(T_{\mathrm{R}}-T_{\mathrm{EK}})}{m_{\mathrm{ges}}},\hfill \end{array}$$

(26d)

$$\begin{array}{ccc}\hfill & {\dot{T}}_{\mathrm{S}}\hfill & =\frac{k_{\mathrm{K}}A(T_{\mathrm{R}}-T_{\mathrm{S}})}{c_{\mathrm{p},\mathrm{S}}{m}_{\mathrm{S}}}-\frac{k_{\mathrm{WS}}A(T_{\mathrm{S}}-T_{\mathrm{M}})}{c_{\mathrm{p},\mathrm{S}}{m}_{\mathrm{S}}},\hfill \end{array}$$

(26e)

$$\begin{array}{ccc}\hfill & {\dot{T}}_{\mathrm{M}}\hfill & =\frac{{\dot{m}}_{\mathrm{M},\mathrm{KW}}(T_{\mathrm{M}}^{\mathrm{IN}}-T_{\mathrm{M}})}{m_{\mathrm{M},\mathrm{KW}}}+\frac{k_{\mathrm{WS}}A(T_{\mathrm{S}}-T_{\mathrm{M}})}{c_{\mathrm{p},\mathrm{W}}{m}_{\mathrm{M},\mathrm{KW}}},\hfill \end{array}$$

(26f)

$$\begin{array}{ccc}\hfill & {\dot{T}}_{\mathrm{EK}}\hfill & =\frac{{\dot{m}}_{\mathrm{AWT}}(T_{\mathrm{R}}-T_{\mathrm{EK}})}{m_{\mathrm{AWT}}}-\frac{{\alpha }_{exp}(T_{\mathrm{EK}}-T_{\mathrm{AWT}})}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{AWT}}}+\frac{k_{\mathrm{R}2}{m}_{\mathrm{A}}{m}_{\mathrm{AWT}}{H}_{\mathrm{R}}}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{AWT}}{m}_{\mathrm{ges}}},\hfill \end{array}$$

(26g)

$$\begin{array}{ccc}\hfill & {\dot{T}}_{\mathrm{AWT}}\hfill & =\frac{{\dot{m}}_{AWT,KW}(T_{AWT}^{\mathrm{IN}}-T_{\mathrm{AWT}})}{m_{\mathrm{AWT},\mathrm{KW}}}-\frac{{\alpha }_{exp}(T_{AWT}-T_{\mathrm{EK}})}{c_{\mathrm{p},\mathrm{W}}{m}_{\mathrm{AWT},\mathrm{KW}}},\hfill \end{array}$$

(26h)

where

$$\begin{array}{ccc}\hfill & U\hfill & =\frac{m_{\mathrm{P}}}{m_{\mathrm{A}}+m_{\mathrm{P}}},\hfill \end{array}$$

(26i)

$$\begin{array}{ccc}\hfill & m_{\mathrm{ges}}\hfill & =m_{\mathrm{W}}+m_{\mathrm{A}}+m_{\mathrm{P}},\hfill \end{array}$$

(26j)

$$\begin{array}{ccc}\hfill & k_{R1}\hfill & =k_0{e}^{\frac{-E_{\mathrm{a}}}{R(T_{\mathrm{R}}+273.15)}}(k_{\mathrm{U}1}(1-U)+k_{\mathrm{U}2}U),\hfill \end{array}$$

(26k)

$$\begin{array}{ccc}\hfill & k_{R2}\hfill & =k_0{e}^{\frac{-E_{\mathrm{a}}}{R(T_{\mathrm{EK}}+273.15)}}(k_{\mathrm{U}1}(1-U)+k_{\mathrm{U}2}U),\hfill \end{array}$$

(26l)

$$\begin{array}{ccc}\hfill & k_{\mathrm{K}}\hfill & =\frac{m_{\mathrm{W}}{k}_{\mathrm{WS}}}{m_{\mathrm{ges}}}+\frac{m_{\mathrm{A}}{k}_{\mathrm{AS}}}{m_{\mathrm{ges}}}+\frac{m_{\mathrm{P}}{k}_{\mathrm{PS}}}{m_{\mathrm{ges}}},\hfill \end{array}$$

(26m)

$$\begin{array}{ccc}\hfill & m_{\mathrm{A},\mathrm{R}}\hfill & =m_{\mathrm{A}}-\frac{m_{\mathrm{A}}{m}_{\mathrm{AWT}}}{m_{\mathrm{ges}}}.\hfill \end{array}$$

(26n)

The total mass of water, monomer, and polymer present in the reactor and in the external heat exchanger are denoted by $m_{\mathrm{W}}$, $m_{\mathrm{A}}$ and $m_{\mathrm{P}}$. The temperatures of the contents inside the reactor, vessel, and jacket are denoted by $T_{\mathrm{R}}$, $T_{\mathrm{S}}$, and $T_{\mathrm{M}}$. The temperature of the material leaving the external heat exchanger and the coolant inside the external heat exchanger are denoted by $T_{\mathrm{EK}}$ and $T_{\mathrm{AWT}}$. U gives the ratio of the total mass of the product to the sum of the total mass of the monomer and the product. $m_{\mathrm{ges}}$ represents the total mass of the contents of the reactor and the external heat exchanger. $k_{\mathrm{R}1}$ and $k_{\mathrm{R}2}$ represent the reaction rates of the conversion of the monomer to the polymer inside the reactor and external heat exchanger, respectively. $k_{\mathrm{K}}$ results from an approximation of the heat-transfer coefficient between the reaction mixture and the reactor vessel. The amount of monomer present inside the reactor is given by $m_{\mathrm{A},\mathrm{R}}$. The input feed $\left({\dot{m}}_{\mathrm{F}}\right)$, the temperature of the coolant entering the jacket $\left(T_{\mathrm{M}}^{\mathrm{IN}}\right)$ and the temperature of the coolant entering the external heat exchanger $\left(T_{\mathrm{AWT}}^{\mathrm{IN}}\right)$ are the control inputs. The bounds on the control inputs are given in

Table 6. Case study 2: Bounds on the control inputs.

Control	Lower Bound	Upper Bound	Unit
${\dot{m}}_{\mathrm{F}}$	0	30,000	$\mathrm{k}\mathrm{g}{\mathrm{h}}^{-1}$
$T_{\mathrm{M}}^{\mathrm{IN}}$	60	100	$°\mathrm{C}$
$T_{\mathrm{AWT}}^{\mathrm{IN}}$	60	100	$°\mathrm{C}$

.

5.2.1. Formulation of the Optimal Control Problem

The control task is to minimize the batch time while satisfying crucial quality and safety constraints. The temperature of the reactor $T_{\mathrm{R}}$ has to be maintained between 88 ${}^{°}\mathrm{C}$ and 92 ${}^{°}\mathrm{C}$ to obtain the desired product quality. The temperature of the reactor content must not exceed 109 ${}^{°}\mathrm{C}$ in the event of a cooling failure. The maximum temperature of the reactor content that can result in the event of complete cooling failure $\left(T_{\mathrm{CF}}\right)$ is given by the so-called adiabatic temperature Equation (27):

$$\begin{array}{c}\hfill T_{\mathrm{CF}}=\frac{H_{\mathrm{R}}{m}_{\mathrm{A}}}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{ges}}}+T_{\mathrm{R}}.\end{array}$$

(27)

The total mass of the contents present in the reactor and the external heat exchanger must not exceed 40,879.5 $\mathrm{kg}$. It is assumed the specific reaction rate constant $\left(k_0\right)$ and the reaction enthalpy $\left(H_{\mathrm{R}}\right)$ are not known precisely. The parameters are assumed here to be contained in an ellipsoidal confidence region described by their nominal values and the parameter covariance matrix Equation (28):

$$\begin{array}{c}\hfill {\mathit{d}}_0=\left(\begin{array}{c}{H}_{\mathrm{R}}\\ k_0\end{array}\right)=\left(\begin{array}{c}950\frac{\mathrm{kJ}}{\mathrm{kg}}\\ 7\end{array}\right),{\mathit{P}}_0=\left(\begin{array}{cc}36100& -159.6\\ -159.6\times {10}^2& 1.96\end{array}\right).\end{array}$$

(28)

All other parameters present in the model equations are considered to be known and are given in

Table 7. Case study 2: Model parameters.

Parameter	Description	Value	Unit
R	Universal gas constant	$8.314$	$\mathrm{kJ}{\mathrm{kmol}}^{-1}{\mathrm{K}}^{-1}$
$c_{\mathrm{p},\mathrm{W}}$	Specific heat capacity of the water (coolant)	$4.2$	$\mathrm{kJ}{\mathrm{kg}}^{-1}{\mathrm{K}}^{-1}$
$c_{\mathrm{p},\mathrm{S}}$	Specific heat capacity of the steel	$0.47$	$\mathrm{kJ}{\mathrm{kg}}^{-1}{\mathrm{K}}^{-1}$
$c_{\mathrm{p},\mathrm{F}}$	Specific heat capacity of the feed	3	$\mathrm{kJ}{\mathrm{kg}}^{-1}{\mathrm{K}}^{-1}$
$c_{\mathrm{p},\mathrm{R}}$	Specific heat capacity of the contents inside reactor	5	$\mathrm{kJ}{\mathrm{kg}}^{-1}{\mathrm{K}}^{-1}$
$k_{\mathrm{WS}}$	Heat-transfer coefficient between water and steel	17,280	$\mathrm{kJ}{\mathrm{h}}^{-1}{\mathrm{m}}^{-2}{\mathrm{K}}^{-1}$
$T_{\mathrm{F}}$	Feed temperature	25	$°\mathrm{C}$
A	Heat-transfer surface of the jacket	65	${\mathrm{m}}^2$
$m_{\mathrm{M},\mathrm{KW}}$	Mass of coolant in the jacket	5000	$\mathrm{kg}$
$m_{\mathrm{S}}$	Mass of the reactor (steel)	39,000	$\mathrm{kg}$
$m_{\mathrm{AWT}}$	Mass of the product in external heat exchanger	200	$\mathrm{kg}$
$m_{\mathrm{AWT},\mathrm{KW}}$	Mass of the coolant in external heat exchanger	1000	$\mathrm{kg}$
${\dot{m}}_{\mathrm{M},\mathrm{KW}}$	Mass of the coolant flow into the jacket	300,000	$\mathrm{kg}{\mathrm{h}}^{-1}$
${\dot{m}}_{\mathrm{AWT},\mathrm{KW}}$	Mass of the coolant flow into the external heat exchanger	$100,000$	$\mathrm{kg}{\mathrm{h}}^{-1}$
${\dot{m}}_{\mathrm{AWT}}$	Mass of the product flow into the external heat exchanger	$20,000$	$\mathrm{kg}{\mathrm{h}}^{-1}$
$E_{\mathrm{a}}$	Activation energy	8500	$\mathrm{kJ}{\mathrm{kmol}}^{-1}$
$k_{\mathrm{U}1}$	Reaction parameter 1	32	−
$k_{\mathrm{U}2}$	Reaction parameter 2	4	−
${\omega }_{\mathrm{W},\mathrm{F}}$	Mass fraction of coolant water in the feed	$0.333$	−
${\omega }_{\mathrm{A},\mathrm{F}}$	Mass fraction of monomer A in the feed	$0.667$	−
$k_{\mathrm{AS}}$	Heat-transfer coefficient between monomer and steel	3600	$\mathrm{kJ}{\mathrm{h}}^{-1}{\mathrm{m}}^{-2}{\mathrm{K}}^{-1}$
$k_{\mathrm{PS}}$	Heat-transfer coefficient between product and steel	360	$\mathrm{kJ}{\mathrm{h}}^{-1}{\mathrm{m}}^{-2}{\mathrm{K}}^{-1}$
${\alpha }_{exp}$	Experimental coefficient	3,600,000	${\mathrm{h}}^{-1}$

. All the states ($\mathit{x}={(m_{\mathrm{w}},m_{\mathrm{A}},m_{\mathrm{P}},T_{\mathrm{R}},T_{\mathrm{S}},T_{\mathrm{M}},T_{\mathrm{EK}},T_{\mathrm{AWT}})}^T$) are considered to be measured directly from the plant with measurement noise of standard deviation $\mathit{\sigma }={(50\mathrm{kg},50\mathrm{kg},50\mathrm{kg},0.1°\mathrm{C},0.1°\mathrm{C},0.1°\mathrm{C},0.1°\mathrm{C},0.1°\mathrm{C})}^T$. The initial condition of the states is given as ${\mathit{x}}_0={(1000\mathrm{kg},853\mathrm{kg},26.5\mathrm{kg},90°\mathrm{C},90°\mathrm{C},90°\mathrm{C},35°\mathrm{C},35°\mathrm{C})}^T$. The sampling time of the controller $t_s$ is chosen as $0.2\mathrm{h}$. The batch is completed if the mass of product P produced reaches $20,700\mathrm{kg}$. The control vector ($\mathit{u}$) is represented as ${({\dot{m}}_{\mathrm{F}},T_{\mathrm{M}}^{\mathrm{IN}},T_{\mathrm{AWT}}^{\mathrm{IN}})}^T$. The MPC optimization problem which is solved at time t ($s^{\mathrm{th}}$ iteration) reads as Equation (29):

$$\begin{array}{c}\hfill \underset{{\mathit{x}}_k,{\mathit{u}}_k,{\mathit{ϵ}}_k}{min}\sum _{k=s}^{s+N_p-1}-m_{\mathrm{P},k}+\frac{3.33}{{10}^3}{(\Delta {\dot{m}}_{\mathrm{F},k})}^2+2.5{(\Delta T_{\mathrm{M},k}^{\mathrm{IN}})}^2+2.5{(\Delta T_{\mathrm{AWT},k}^{\mathrm{IN}})}^2+{10}^5({\mathit{ϵ}}_{k,1}^2+{\mathit{ϵ}}_{k,\left[2\right]}^2)+{10}^{10}{\mathit{ϵ}}_{k,\left[3\right]}^2,\end{array}$$

subject to

$$\begin{array}{cc}\hfill & {\mathit{x}}_{k+1}=\mathit{f}({\mathit{x}}_k,{\mathit{u}}_k,{\mathit{d}}_k),\hfill \end{array}$$

(29a)

$$\begin{array}{cc}\hfill & 88\le T_{\mathrm{R},k}+{ϵ}_{k,\left[1\right]}\le 92,\hfill \end{array}$$

(29b)

$$\begin{array}{cc}\hfill & \frac{H_{\mathrm{R}}{m}_{\mathrm{A}}}{c_{\mathrm{p},\mathrm{R}}{m}_{\mathrm{ges}}}+T_{\mathrm{R}}+{ϵ}_{k,2}\le 109,\hfill \end{array}$$

(29c)

$$\begin{array}{cc}\hfill & m_{\mathrm{W},k}+m_{\mathrm{A},k}+m_{\mathrm{P},k}+{ϵ}_{k,3}\le 40879.5,\hfill \end{array}$$

(29d)

$$\begin{array}{cc}\hfill & -1\le {\mathit{ϵ}}_{k,\left[1\right]}\le 1,\hfill \end{array}$$

(29e)

$$\begin{array}{cc}\hfill & -1\le {\mathit{ϵ}}_{k,\left[2\right]}\le 1,\hfill \end{array}$$

(29f)

$$\begin{array}{cc}\hfill & -150\le {\mathit{ϵ}}_{k,\left[3\right]}\le 150,\hfill \end{array}$$

(29g)

$$\begin{array}{cc}\hfill & \underline{\mathit{u}}\le {\mathit{u}}_k\le \overline{\mathit{u}},\hfill \end{array}$$

(29h)

$$\begin{array}{cc}\hfill & {\mathit{x}}_s={\mathit{x}}_s^m,\hfill \end{array}$$

(29i)

where $\Delta {\dot{m}}_{\mathrm{F},k}={\dot{m}}_{\mathrm{F},k}-{\dot{m}}_{\mathrm{F},k-1}$, $\Delta T_{\mathrm{M},k}^{\mathrm{IN}}=T_{\mathrm{M},k}^{\mathrm{IN}}-T_{\mathrm{M},k-1}^{\mathrm{IN}}$, $T_{\mathrm{AWT},k}^{\mathrm{IN}}-T_{\mathrm{AWT},k-1}^{\mathrm{IN}}$, and ${\mathit{ϵ}}_k\in {\mathbb{R}}^{n_c}$. The objective function aims to maximize the number of moles of product P produced along the prediction horizon, and penalizes the control moves and the constraint violations. We assume that maximizing the mass of product P produced along the prediction horizon implicitly minimizes the batch time. The model Equation (26) is discretized using orthogonal collocation on finite elements [68,69] and this leads to Equation (29a). The number of finite elements and the degree of the polynomials are chosen as 4 and 2. The collocation points are obtained using the Radau method [72]. The constraints are given in Equations (29b)–(29d). The constraints are implemented as soft constraints to prevent the MPC optimization from becoming infeasible due to the influence of measurement noise. The maximum constraint violation that can happen is bounded by Equations (29e)–(29g). The control bounds are enforced using Equation (29h), where ${\underline{\mathit{u}}}_k$ and ${\overline{\mathit{u}}}_k$ represent the lower and upper bounds on the control and can be obtained from Table 6. The controller is reinitialized at the current plant measurements using Equation (29i), where ${\mathit{x}}_s^m\in {\mathbb{R}}^8$ represents the measurement vector obtained at time t.

5.2.2. Results

All multi-stage NMPC strategies consider a prediction of horizon $\left(N_p\right)$ of length 5 and a robust horizon $\left(N_r\right)$ of length 2. All the nodes in the scenario tree of the multi-stage NMPC are equally weighted (i.e., $w_k^i=\frac{1}{N_n},\mathrm{s}.\mathrm{t}.{\sum }_{i=1}^{N_n}{w}_k^i=1$, where $N_n$ is the number of nodes at time step k in the prediction horizon). The scaling factors ${\kappa }_s^{\star }$ and ${\kappa }_c^{\star }$ are chosen as $1.75$. The scaling increase factor $\left(\beta \right)$ is chosen as $1.05$. The scaling factors and the scaling increase factors were chosen based on a posterior analysis using simulation studies such that the observed measurements do not invalidate the scaling factors and the NMPC optimization problem is recursively feasible. The value of $\alpha$ was chosen corresponding to a $6\sigma$ confidence level for the adaptive NMPC approaches. The weight on the sigma points $\left(\mathit{v}\right)$ chosen to build the covariance matrix of the MS-SB and MS-CB NMPC formulations is given as ${(0.2,0.2,0.2,0.2,0.2)}^T$. The nominal values of the uncertain parameters are considered to be the true realizations of the uncertain parameters. The batch is stopped when the measured mass of product P reaches 20,750$\mathrm{kg}$ which guarantees that at least $20,700\mathrm{kg}$ of product P ($m_{\mathrm{P}}$) is present at the batch end. The simulation results obtained when the reactor is controlled using NMPC with the true plant model, the non-adaptive and the adaptive NMPC approaches are shown in Figure 14. The dashed vertical lines indicate the end times of the batches.

It can be seen that the NMPC with the correct model fully exploits the range of operation and meets the constraints, apart from the effect of measurement errors. The end of the batch is reached at $1.4\mathrm{h}$. MS and MS-VA NMPC take into account the presence of uncertainty in the reaction rate constant and in the reaction enthalpy while satisfying the constraint on the reactor temperature for all scenarios in their scenario trees and are shown in Figure 15. The upper and lower bounds on the reactor temperature are active in the predictions of the MS and MS-VA NMPC until the end of the batch. The large range of the uncertain parameters along with the tight bounds on the reactor temperature results in a significant increase in the batch time to $3.2\mathrm{h}$. The box over-approximation of the ellipsoidal confidence region adds uncertainty to the parameter estimates which contributes to the increased batch time.

The batch time in the presence of the uncertainties can be reduced using MS-SB and MS-CB NMPC. The sigma points more tightly approximate the uncertainty set. MS-SB and MS-CB NMPC take $2.6\mathrm{h}$ and $2.4\mathrm{h}$ to complete the batch. The sensitivity of the states $m_{\mathrm{W},k}$, $m_{\mathrm{A},k}$$m_{\mathrm{P},k}$ with respect to the uncertain parameters is not zero. This enlarges the box over-approximation of the reachable sets computed using the MS-SB NMPC. This results in an additional performance loss when compared to MS-CB NMPC, since the sensitivity of $m_{\mathrm{W},k}+m_{\mathrm{A},k}+m_{\mathrm{P},k}$ in Equation (29d) with respect to the uncertain parameters is zero.

The batch time can be further reduced significantly using the adaptive NMPC approaches which use the plant measurements to improve the knowledge about the uncertain parameters. A-MS NMPC and A-MS-VA NMPC finish the batch at $1.8\mathrm{h}$, close to the optimal batch time if the exact model is used. A-MS-CB and A-MS-SB NMPC require only $1.6\mathrm{h}$ to finish the batch. There is a $38\%$ and $33\%$ reduction in the batch time when using A-MS-SB and A-MS-CB NMPC over MS-SB and MS-CB NMPC.

Finally, the performance of the controllers was evaluated for 100 random realizations of the uncertain parameter values within the initial confidence region of the uncertain parameters (marked by black dots in Figure 16a). The results are shown in Figure 16b. The end times of the batches controlled using MS and A-MS NMPC are similar to those of MS-VA and A-MS-VA NMPC because the scenarios generated using the extreme values of the uncertain parameters prevent the controller from feeding more reactants into the reactor and are considered in the scenario tree of both variants. The batch time is considerably reduced using the proposed multi-stage NMPC variants using sigma points when compared to the standard multi-stage NMPC because the uncertainty set is more tightly approximated using the sigma points. MS-CB NMPC results in a better performance when compared to MS-SB NMPC because the performance is restricted by the constraints and their approximation is tighter in MS-CB NMPC. The performance is further improved using the adaptive approaches. A-MS-CB NMPC results in the minimum batch times when compared to other the robust NMPC schemes presented in this paper. The times taken to solve one iteration of the MS, MS-VA, MS-SB, and MS-CB NMPC are $8.89\mathrm{s}$, $2.74\mathrm{s}$, $12.90\mathrm{s}$, and $2.88\mathrm{s}$. The times taken to solve one iteration of A-MS, A-MS-VA, A-MS-SB, and A-MS-CB NMPC are $12.47\mathrm{s}$, $3.84\mathrm{s}$, $28.17\mathrm{s}$, and $4.27\mathrm{s}$.

6. Conclusions

In this paper, we proposed two novel multi-stage NMPC formulations, where the scenario trees are generated using the sigma points, and box over-approximations of the sets of reachable states or constraint function values are computed along the prediction horizon using the unscented transformation. The proposed scheme tightly over-approximates the uncertainty set and results in a better performance for ellipsoidal uncertainty sets when compared to the traditional multi-stage NMPC where the scenario tree is generated using a box over-approximation. In addition, adaptive approaches are presented where the plant measurements are used to reduce the uncertainty. This significantly reduces the conservatism introduced by the non-adaptive robust NMPC approaches. A simulation study was carried out to analyze the influence of the tuning parameters in the performance of the proposed multi-stage NMPC approaches. An industrial benchmark example is used to show the benefits of the proposed approaches for an example of a realistic size. It confirms the advantages of the new methods. Our future work will focus on dual approaches where a trade-off problem between using excitation signals (probing actions) and performance-oriented control actions is solved. We will also consider improving the performance of robust multi-stage NMPC using sigma points further by over-approximating the reachable sets of the constraint functions and states using an ellipsoidal set instead of a box.