An Optimal Feedback Control Strategy for Nonlinear, Distributed-Parameter Processes

Abstract

In this work, an optimal state feedback control strategy is proposed for non-linear, distributed-parameter processes. For different values of a given parameter susceptible to upsets, the strategy involves off-line computation of a repository of optimal open-loop states and gains needed for the feedback adjustment of control. A gain is determined by minimizing the perturbation of the objective functional about the new optimal state and control corresponding to a process upset. When an upset is encountered in a running process, the repository is utilized to obtain the control adjustment required to steer the process to the new optimal state. The strategy is successfully applied to a highly non-linear, gas-based heavy oil recovery process controlled by the gas temperature with the state depending non-linearly on time and two spatial directions inside a moving boundary, and subject to pressure upsets. The results demonstrate that when the process has a pressure upset, the proposed strategy is able to determine control adjustments with negligible time delays and to navigate the process to the new optimal state.

1. Introduction

In a bid to attain top efficiencies amid increasing competition, reduced profit margins, and rising concerns to protect energy and environment, modern day process operations mandate increasingly greater performance from control methods. This is a major challenge when dealing with the ubiquitous, non-linear and non-uniform processes, which are described by sophisticated mathematical models. In these cases, it is very difficult to obtain closed-form control laws and stability characteristics.

In general, the control of non-linear processes requires the solution of the associated optimal control problem [1,2,3]. This is done in non-linear model predictive control (NMPC) at each sample time of the process over a future time interval of fixed duration, or receding horizon. In a multi-layered scheme, real-time optimization (RTO) enables the determination of optimal set points to be tracked by faster acting closed loop control based on a simplified process model. These approaches provide a flexible structure to cover a wide range of optimality criteria [2,3,4]. In Refs. [5,6,7], it is shown that the knowledge of the necessary conditions of optimality (NCO) aids in devising efficient and robust real-time optimization schemes by utilizing the gradient information of the process under uncertainty.

However, in all aforementioned cases, there is always an inherent computational overhead associated with the optimal control determination, which is proportional to process non-linearity and non-uniformity. This overhead is often significant enough to cause feedback delays when the computation time exceeds the sample time interval. Such delays result in suboptimal solutions, or worse, infeasible operations [8]. Development of control techniques to surmount this challenge is an active area of research. Reviewed in 2016, the techniques include fast but typcially sub-optimal updates with negligible delays using a few iterations of the associated non-linear programs in serial, parallel and two-layer schemes [9]. Approaches that implement intensive computations in the off-line mode range from those exploiting neighborhood extremals [10,11] to those enabling the determination of control laws in critical regions of control-affine systems [12]. Recently, the indirect open-loop method applied to closed loop control problems has been found to be computationally more efficient than direct methods [13].

In this work, we develop an optimal state, feedback control strategy for non-linear, distributed-parameter processes. The strategy is based on a perturbation method exploiting Riccati transformation [14] to determine optimal control adjustment to compensate for an upset in a non-linear, distributed-parameter system. The perturbation approach is similar to NCO tracking in that the variation of the optimal objective functional is minimized by control action [15]. The control adjustment results from a proportional feedback law depending on the perturbation from an optimal state corresponding to the upset. Both the gain and optimal state are determined and stored off-line from the solution of the associated open loop, optimal control problem for different upset values within a range. The stored states and gains are utilized in a running process to determine control adjustment from quick interpolation for any upset within or sufficiently close to the range. The developed method is applied to a gas-based heavy oil recovery process in which the recovery is maximized by controlling the gas temperature with time in presence of pressure upsets.

2. Theory

Consider the open-loop problem of finding the control $\mathbf{u}(t)$ at the optimum of the objective functional

$$I=\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\underset{0}{\overset{x_{0\mathrm{f}}}{\int }}\underset{0}{\overset{x_{1\mathrm{f}}}{\int }}\underset{0}{\overset{x_{2\mathrm{f}}}{\int }}F[\mathbf{y}(t),\mathbf{u}(t)]\mathrm{d}{x}_2\mathrm{d}{x}_1\mathrm{d}{x}_0\mathrm{d}t;\mathbf{y}\in {\mathbb{R}}^n,\mathbf{x}\in {\mathbb{R}}^3,\mathbf{u}\in {\mathbb{R}}^m$$

(1)

subject to the process model, or state equation

$$\frac{\partial \mathbf{y}}{\partial t}=\mathbf{f};\mathbf{y}(0)={\mathbf{y}}_0,\mathbf{p}=\mathbf{0},$$

(2)

where $\mathbf{f}$ depends on $\mathbf{u}$ (the control vector), $\mathbf{y}$ (the state vector), and its spatial derivatives; and $\mathbf{p}$ is a vector of functions representing boundary conditions. They depend on $\mathbf{y}$ at spatial boundaries, ${\mathbf{y}}_{\partial }$, and its derivatives with respect to $\mathbf{x}$ (the vector of Cartesian co-ordinates $x_0$, $x_1$ and $x_2$) and $\mathbf{u}$. While $\mathbf{y}$ is constrained by Equation (2), $\mathbf{u}$ is undetermined over space and time. It may be noted that $\mathbf{f}$ represents a general, non-linear distributed-parameter process, and is not restricted to any particular structure. It is, however, assumed that the first and second order derivatives of $\mathbf{f}$ and F exist with respect to $\mathbf{y}$ and $\mathbf{u}$. The problem is equivalent to finding the optimum of the augmented functional,

$$J=\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\underset{0}{\overset{x_{0\mathrm{f}}}{\int }}\underset{0}{\overset{x_{1\mathrm{f}}}{\int }}\underset{0}{\overset{x_{2\mathrm{f}}}{\int }}\left[F+\mathit{\lambda }\left(-\frac{\partial \mathbf{y}}{\partial t}+\mathbf{f}\right)\right]\mathrm{d}{x}_2\mathrm{d}{x}_1\mathrm{d}{x}_0\mathrm{d}t,$$

(3)

where $\mathit{\lambda }$ is the costate variable. Note that for simplicity, we have presented the objective functional in Lagrange form for fixed final time, and not included any (in)equality constraints since they could be handled by appropriately augmenting the associated Hamiltonian [16].

At the optimum of J, its variation, $\delta J=0$. Using the principles of variational calculus, this result eventually provides the necessary conditions—differential costate equation with zero-valued costates at the final time $t_{\mathrm{f}}$ along with boundary conditions, and the stationarity condition, i.e., the variational derivative of J with respect to $\mathbf{u}$ being zero—which must be satisfied to obtain the optimal control policy $\widehat{\mathbf{u}}(t)$ and the corresponding optimal state $\widehat{\mathbf{y}}(t)$.

2.1. Optimal Process Under Upset

For a process running with an optimal control policy, a process upset (i.e., a change in a parameter value) renders the state non-optimal. Given a sufficiently small upset, the change in J from the optimal state $(\widehat{\mathbf{y}},\widehat{\mathbf{u}})$ can be expressed as

$$\Delta J=J(\mathbf{y},\mathbf{u})-J(\widehat{\mathbf{y}},\widehat{\mathbf{u}})=\underset{{\displaystyle =0}}{\underbrace{\delta J(\widehat{\mathbf{y}},\widehat{\mathbf{u}})}}+{\delta }^{2}J(\widehat{\mathbf{y}},\widehat{\mathbf{u}};\delta \mathbf{y},\delta \mathbf{u}),$$

where $\delta \mathbf{y}=\mathbf{y}-\widehat{\mathbf{y}}$, and $\delta \mathbf{u}=\mathbf{u}-\widehat{\mathbf{u}}$. Thus, the restoration of the process to optimality under the upset can be posed as the optimal control problem of finding the control adjustment $\delta \mathbf{u}$ that minimizes ${\delta }^{2}J$—the second variation of J—subject to the perturbed state governed by

$$\frac{\partial (\delta \mathbf{y})}{\partial t}={\left.\frac{\partial \mathbf{f}}{\partial \mathbf{y}}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}}\delta \mathbf{y}+{\left.\frac{\partial \mathbf{f}}{\partial \mathbf{u}}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}}\delta \mathbf{u};\delta \mathbf{y}(0)=\delta \mathbf{p}=\mathbf{0},$$

(4)

where $\partial \mathbf{f}/\partial \mathbf{y}$ is the total derivative taking into account the spatial derivatives of $\mathbf{y}$ as functions of $\mathbf{y}$.

2.2. Incorporation of State Feedback

Discretizing the spatial domain using uniform grid point spacing $\Delta x_i$ along each $x_i$-direction, we obtain

$$\begin{array}{cc}\hfill {\delta }^{2}J(\widehat{\mathbf{y}},\widehat{\mathbf{u}};\delta \mathbf{y},\delta \mathbf{u})& =\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\left(\frac{1}{2}\delta {\mathbf{u}}^{\top }{\left.\frac{{\partial }^{2}H}{\partial {\mathbf{u}}^2}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}}\delta \mathbf{u}+\delta {\mathbf{y}}^{\top }{\left.\frac{{\partial }^{2}H}{\partial \mathbf{y}\partial \mathbf{u}}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}}\delta \mathbf{u}+\frac{1}{2}\delta {\mathbf{y}}^{\top }{\left.\frac{{\partial }^{2}H}{\partial {\mathbf{y}}^2}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}}\delta \mathbf{y}\right)\mathrm{d}t,\hfill \end{array}$$

(5)

where

$$\begin{array}{cc}\hfill H& ={\mathit{\lambda }}^{\top }\mathbf{f}+{\displaystyle \sum _{i=0}^{L-1}\sum _{j=0}^{M-1}\sum _{k=0}^{N-1}}{F}_{ijk}\Delta x_0\Delta x_1\Delta x_2\hfill \end{array}$$

(6)

is the Hamiltonian corresponding to the spatially discretized J in Equation (3) using L, M, and N grid points along, respectively, $x_0$-, $x_1$-, and $x_2$-directions. While $F_{ijk}$s are F values, $\mathit{\lambda }$ and $\mathbf{f}$ are vectors of their respective elements at these grid points. Thus, for n state variables,

$$\begin{array}{cc}\hfill \mathit{\lambda }& ={\left[{\mathit{\lambda }}_0{\mathit{\lambda }}_1\cdots {\mathit{\lambda }}_{n-1}\right]}^{\top }\mathrm{and}\hfill \\ \hfill {\mathit{\lambda }}_i& =\begin{array}{ccccccccccc}\left[{\lambda }_{i,0,0,0}\right.& {\lambda }_{i,0,0,1}& \cdots & {\lambda }_{i,0,0,N-1}& {\lambda }_{i,0,1,0}& {\lambda }_{i,0,1,1}& \cdots & {\lambda }_{i,0,1,N-1}& \cdots & {\lambda }_{i,0,M-1,0}& {\lambda }_{i,0,M-1,1}\end{array}\hfill \\ & \begin{array}{ccccccc}\cdots & {\lambda }_{i,0,M-1,N-1}& \cdots & {\lambda }_{i,L-1,M-1,0}& {\lambda }_{i,L-1,M-1,1}& \cdots & {\left.{\lambda }_{i,L-1,M-1,N-1}\right]}^{\top },\end{array}\hfill \end{array}$$

(7)

$$\begin{array}{c}\hfill i=0,1,\cdots ,n-1.\end{array}$$

(8)

Note that the minimization of ${\delta }^{2}J$, subject to Equation (4) with $\delta \mathbf{u}(t)$ as the perturbed control function, is the linear quadratic regulator problem of optimal control with finite time horizon [14]. In the necessary condition for the minimum of ${\delta }^{2}J$, we introduce the Riccatti transfromation,

$$\delta \mathit{\lambda }=\mathbf{S}\delta \mathbf{y},$$

(9)

where $\delta \mathit{\lambda }$, $\mathbf{y}$ and $\delta \mathbf{y}$ have the same structure as that of $\mathit{\lambda }$ in Equations (7) and (8), and

$$\mathbf{S}=\left[\begin{array}{cccc}{\mathbf{S}}_{0,0}& {\mathbf{S}}_{0,1}& \cdots & {\mathbf{S}}_{0,n-1}\\ {\mathbf{S}}_{1,0}& {\mathbf{S}}_{1,1}& \cdots & {\mathbf{S}}_{1,n-1}\\ ⋮& ⋮& ⋮& ⋮\\ {\mathbf{S}}_{n-1,0}& {\mathbf{S}}_{n-1,1}& \cdots & {\mathbf{S}}_{n-1,n-1}\end{array}\right]$$

(10)

is a block positive definite Riccati matrix, each element of which is a symmetric matrix of size $LMN\times LMN$. Finally, we incorporate into the necessary condition, the feedback control law

$$\delta \mathbf{u}=-\mathbf{K}\delta \mathbf{y},$$

(11)

where $\mathbf{K}$ is the $m\times n$ gain matrix with elements similar to ${\mathbf{S}}_{i,j}$. With the above considerations, the necessary condition for the minimum of ${\delta }^{2}J$ results in the differential Riccati equation

$$\frac{\mathrm{d}\mathbf{S}}{\mathrm{d}t}=-\mathbf{S}\mathbf{A}-{\mathbf{A}}^{\top }\mathbf{S}+\mathbf{S}\mathbf{B}{\mathbf{D}}^{-1}{\mathbf{B}}^{\top }\mathbf{S}-\mathbf{C};\mathbf{S}(t_{\mathrm{f}})={\mathbf{S}}_{\partial }=\mathbf{0},$$

(12)

where

$$\mathbf{A}={\left.\frac{\partial \mathbf{f}}{\partial \mathbf{y}}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}},\mathbf{B}={\left.\frac{\partial \mathbf{f}}{\partial \mathbf{u}}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}},\mathbf{C}={\left.\frac{{\partial }^{2}H}{\partial {\mathbf{y}}^2}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}},\mathbf{D}={\left.\frac{{\partial }^{2}H}{\partial {\mathbf{u}}^2}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}},\mathrm{and} \mathbf{E}={\left.\frac{{\partial }^{2}H}{\partial \mathbf{y}\partial \mathbf{u}}\right|}_{\widehat{\mathbf{y}},\widehat{\mathbf{u}}}$$

(13)

and a stationarity condition for $\delta \mathbf{u}$ that provides the equation for the gain,

$$\mathbf{K}={\mathbf{D}}^{-1}\left({\mathbf{E}}^{\top }+{\mathbf{B}}^{\top }\mathbf{S}\right).$$

(14)

In the above equations, $\mathbf{A}$ and $\mathbf{C}$ have the same structure as that of $\mathbf{S}$, while $\mathbf{B}$ and $\mathbf{E}$ have the same structure as that of $\mathbf{K}$, and $\mathbf{D}$ is an $m\times m$ symmetric matrix.

2.3. Feedback Optimal Control Strategy

Based on the above development, following is the strategy to apply feedback control adjustment to a process susceptible to upsets in a parameter value:

Offline Calculations

For each, different value of a parameter expected to undergo upsets, determine and store a repository of the following:

• the optimal state $\widehat{\mathbf{y}}(t)$ by solving the open-loop optimal control problem, and
• the gain $\mathbf{K}(t)$ by solving the differential Riccati equation and the gain equation

Real-Time Control

Execute the process with a nominal value of the parameter, as well as pre-determined $\widehat{\mathbf{u}}(t)$. Whenever there is an upset in the parameter, do the following:

• obtain from the repository the $\widehat{\mathbf{y}}(t)$ and $\mathbf{K}(t)$ interpolated at the current parameter value, and
• obtain and apply the improved control

  $$\widehat{\mathbf{u}}(t)=\mathbf{u}(t)-\mathbf{K}(t)[\widehat{\mathbf{y}}(t)-\mathbf{y}(t)],$$

  which is the control law given by Equation (11).

It may be noted that the above strategy is different from model predictive control, which requires online solution of the optimal control problem at each sample time of a running process over a finite, receding time horizon. The past control approaches on distributed-parameter systems have been either open loop or focused on processes described by specific types of equations, such as linear, parabolic, hyperbolic, and elliptic partial differential equations [17,18,19,20,21,22,23].

3. Application

We apply the feedback control strategy proposed above to a distributed-parameter system of a labscale, cylindrical heavy oil reservoir subjected to gas injection for heavy oil recovery. The gas penetrates the vertical reservoir and reduces the viscosity of the oil, which in turn drains to the bottom and is recovered. The objective is to keep the oil recovery maximum despite disturbances in pressure (P) by adjusting the gas temperature versus time $[T_{\mathrm{int}}(t)]$, which is the control function. Finally, we execute and validate the optimal feedback control strategy using computer simulation of the detailed process model, which was experimentally validated in an earlier work [24].

The upset considered for this process is pressure, P. Practical oil field applications often experience a phenomenon of forced fluid flow into the wellbore resulting from the deviation in mud hydrostatic pressure. This well control problem is termed as “kick” and it may result in severe operational difficulties [25]. A well kick needs to be properly compensated for so that it does not result in production irregularities.

Upsets in P affect the gas solubility (see Appendix A) at the cylindrical surface where $T_{\mathrm{int}}(t)$ is controlled to help maintain oil recovery at the maximum level. Thus, the control function is a boundary condition for the equation of change of temperature in the system, which is simultaneously described by the equation of change of the gas mass fraction as well as the system height. While the system temperature and gas mass fraction vary with time and radial as well as axial directions, the height varies with time and the radial direction only.

Figure 1 shows the schematic of the experimental setup in which the gas (nitrogen) at a given pressure and temperature is injected to vessel containing the cylindrical heavy oil reservoir suspended from a load cell. With gas penetration, the viscosity of heavy oil drops, and it starts draining out from the bottom of the model into a collection funnel. The accumulated oil is directed through an online viscometer to a flash separation tank. The gas dissolved in the oil gets separated in a flash tank, and the gas volume is measured in two water columns. The gas-free oil from the flash tank is weighed and is the oil recovered from the reservoir.

3.1. Process Model

Based on first principles, the process model is given by the following three state equations:

$$\begin{array}{cc}\hfill \frac{\partial T}{\partial t}& =\frac{k}{\rho {\widehat{C}}_{\mathrm{P}}}\left(\frac{1}{r}\frac{\partial T}{\partial r}+\frac{{\partial }^{2}T}{\partial r^2}+\frac{{\partial }^{2}T}{\partial z^2}\right)+\frac{\alpha }{\varphi \mu }\frac{\partial T}{\partial z}-\frac{\alpha T}{\varphi {\mu }^2}\left(\frac{\partial \mu }{\partial w}\frac{\partial w}{\partial z}+\frac{\partial \mu }{\partial T}\frac{\partial T}{\partial z}\right)\equiv f_1,\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill \frac{\partial w}{\partial t}& =\frac{D}{r}\frac{\partial w}{\partial r}+D\frac{{\partial }^{2}w}{\partial r^2}+\frac{\partial D}{\partial w}{\left(\frac{\partial w}{\partial r}\right)}^2+\frac{\partial D}{\partial T}\frac{\partial w}{\partial r}\frac{\partial T}{\partial r}+\frac{\alpha }{\varphi \mu }\frac{\partial w}{\partial z}-\frac{\alpha w}{\varphi {\mu }^2}\left(\frac{\partial \mu }{\partial w}\frac{\partial w}{\partial z}+\frac{\partial \mu }{\partial T}\frac{\partial T}{\partial z}\right)\equiv f_2,\hfill \end{array}$$

(16)

$$\begin{array}{cc}\frac{\partial Z}{\partial t}& =-\frac{\alpha }{\varphi \overline{\mu }(r)}\equiv f_3,\hfill \end{array}$$

(17)

where T is the temperature inside the reservoir, w is the mass fraction of the dissolved gas in the reservoir, Z is the time-dependent reservoir height along the vertical direction, t is time, r is radial distance, and z is the vertical or axial distance. Moreover, $\alpha =K_{\mathrm{r}}K\rho g$ where $K_{\mathrm{r}}$ and K are, respectively, the relative permeability and permeability of the reservoir, g is gravity, $\rho$ is oil density, and $\varphi$ is reservoir porosity with ${\widehat{C}}_{\mathrm{P}}$ as specific heat capacity and k as thermal conductivity.

It may be noted that $D=D(w,P,T)$ is the dispersion coefficient of gas in heavy oil (refer to Appendix B), $\mu =\mu (w,P,T)$ is the heavy oil viscosity (see Appendix C), and $\overline{\mu }(r)$ is the average viscosity at $z=0$, and is a function of r. Thus, $\overline{\mu }(r)$ at any time is the average of the viscosity values at the corners $[(r,0)$, $(r+\mathrm{d}r,0)$, $(r,\mathrm{d}z)$ and $(r+\mathrm{d}r,\mathrm{d}z)]$ of a differential element at a given r and $z=0$.

It is assumed that the flow of the live oil through the porous medium along the vertical direction is governed by Darcy’s law, gas diffusion is dominant only along the radial direction, and the porous medium has constant $\alpha$, $\rho$, ${\widehat{C}}_{\mathrm{P}}$, and k in the range of temperature variation [26].

Initial and Boundary Conditions

Initially, the reservoir is at room temperature $T_0$ with height $Z_0$, and no gas mass fraction, except at the surface. Thus, initial conditions are:

$$\begin{array}{cc}\hfill T(r,z,0)& =T_0\forall 0\le z\le Z(r,0)\mathrm{and}0\le r\le R\hfill \end{array}$$

(18)

$$\begin{array}{cc}\hfill w(r,z,0)& =\left\{\begin{array}{cc}{w}_{\mathrm{int}}\hfill & \mathrm{at}\left\{\begin{array}{ll}r=R& \forall 0\le z\le Z(r,0)\\ z=0,Z(r,0)& \forall 0\le r<R,\end{array}\right.\hfill \\ 0\hfill & \forall 0<z<Z(r,0)\mathrm{and}0\le r<R,\hfill \end{array}\right.\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill Z(r,0)& =Z_{0}0\le r\le R.\hfill \end{array}$$

(20)

The boundary conditions are the specifications of T and w at the surface, and the radial symmetric conditions as follows:

$$\begin{array}{cc}\hfill T(r,z,t)& =T_{\mathrm{int}}\mathrm{at}\left\{\begin{array}{ll}r=R& \forall 0\le z\le Z(r,t)\\ z=0,Z(r,t)& \forall 0\le r<R,\end{array}\right.\hfill \end{array}$$

(21)

$$\begin{array}{cc}\hfill w(r,z,t)& =w_{\mathrm{int}}\mathrm{at}\left\{\begin{array}{ll}r=R& \forall 0\le z\le Z(r,t)\\ z=0,Z(r,t)& \forall 0\le r<R,\end{array}\right.\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \frac{\partial T}{\partial r}=\frac{\partial w}{\partial r}& =0\mathrm{at}r=0\forall 0\le z\le Z(r,t).\hfill \end{array}$$

(23)

In the above equations, $w_{\mathrm{int}}$ is the interfacial gas mass fraction (i.e., gas solubility) while $T_{\mathrm{int}}$ is the interfacial temperature. The latter is a control function depending on time, and influences $w_{\mathrm{int}}$.

Equations (15)–(23) constitute the highly non-linear, distributed-parameter model of the heavy oil recovery process. This model is the basis for the optimal control problem that is formulated next.

3.2. Open Loop Optimal Control Problem

This problem involves the maximization of heavy oil production, i.e., the objective functional

$$I=\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\dot{m}(t)\mathrm{d}t=\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\underset{0}{\overset{R}{\int }}\frac{2\pi \alpha \rho r}{\overline{\mu }}\mathrm{d}r\mathrm{d}t$$

(24)

using interfacial temperature versus time, $T_{\mathrm{int}}(t)$, as a control function. Here, $\dot{m}(t)$ is the rate of recovery of the heavy oil from the bottom of the reservoir.

The maximization of I is equivalent to the maximization of the augmented functional

$$J=I+\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\underset{0}{\overset{R}{\int }}\underset{0}{\overset{Z}{\int }}\left[{\lambda }_1\left(-\frac{\partial T}{\partial t}+f_1\right)+{\lambda }_2\left(-\frac{\partial w}{\partial t}+f_2\right)\right]\mathrm{d}z\mathrm{d}r\mathrm{d}t+\underset{0}{\overset{t_{\mathrm{f}}}{\int }}\underset{0}{\overset{R}{\int }}{\lambda }_3\left(-\frac{\partial Z}{\partial t}+f_3\right)\mathrm{d}r\mathrm{d}t,$$

(25)

which is obtained by adjoining I to the state equations, Equations (15)–(17), with the help of the three undetermined costate variables—${\lambda }_1(r,z,t)$, ${\lambda }_2(r,z,t)$, and ${\lambda }_3(r,t)$.

3.2.1. Control Function

It may be noted that the control function $T_{\mathrm{int}}(t)$ is the temperature at:

• the curved surface, i.e., $T(R,z,t)$ for any height and time;
• the bottom surface, i.e., $T(r,0,t)$ for any radial distance and time; and
• the top surface, i.e., $T[r,Z(t),t]$ for any radial distance and time.

3.2.2. Necessary Conditions for Optimality

At the optimum, $\delta J=0$. This gives rise to the necessary conditions that must be satisfied by the control function, $T_{\mathrm{int}}(t)$, to be optimal. The conditions are the state equations, costate equations, and the stationarity conditions as follows.

• State Equations These are given by the process model, i.e., Equations (15)–(23).
• Costate Equations These equations are:

  $$\begin{array}{ccc}\hfill \frac{\partial {\lambda }_1}{\partial t}& =& -{\lambda }_1\frac{\partial f_1}{\partial T}-{\lambda }_2\frac{\partial f_2}{\partial T}+\frac{\partial }{\partial r}\left({\lambda }_1\frac{\partial f_1}{\partial T_r}\right)+\frac{\partial }{\partial z}\left({\lambda }_1\frac{\partial f_1}{\partial T_z}\right)+\frac{\partial }{\partial r}\left({\lambda }_2\frac{\partial f_2}{\partial T_r}\right)\hfill \\ & & +\frac{\partial }{\partial z}\left({\lambda }_2\frac{\partial f_2}{\partial T_z}\right)-\frac{{\partial }^2}{\partial r^2}\left({\lambda }_1\frac{\partial f_1}{\partial T_{rr}}\right)-\frac{{\partial }^2}{\partial z^2}\left({\lambda }_1\frac{\partial f_1}{\partial T_{zz}}\right)-\frac{{\partial }^2}{\partial r^2}\left({\lambda }_2\frac{\partial f_2}{\partial T_{rr}}\right)\hfill \\ & & -\frac{{\partial }^2}{\partial z^2}\left({\lambda }_2\frac{\partial f_2}{\partial T_{zz}}\right)+\left(2\pi r\rho -\frac{{\lambda }_3}{\varphi }\right)\frac{\alpha }{Z_0{\overline{\mu }}^2}\frac{\partial \overline{\mu }}{\partial \overline{T}}\frac{\partial \overline{T}}{\partial T},\hfill \end{array}$$

  (26)

  $$\begin{array}{ccc}\hfill \frac{\partial {\lambda }_2}{\partial t}& =& -{\lambda }_1\frac{\partial f_1}{\partial w}-{\lambda }_2\frac{\partial f_2}{\partial w}+\frac{\partial }{\partial r}\left({\lambda }_1\frac{\partial f_1}{\partial w_r}\right)+\frac{\partial }{\partial z}\left({\lambda }_1\frac{\partial f_1}{\partial w_z}\right)+\frac{\partial }{\partial r}\left({\lambda }_2\frac{\partial f_2}{\partial w_r}\right)\hfill \\ & & +\frac{\partial }{\partial z}\left({\lambda }_2\frac{\partial f_2}{\partial w_z}\right)-\frac{{\partial }^2}{\partial r^2}\left({\lambda }_1\frac{\partial f_1}{\partial w_{rr}}\right)-\frac{{\partial }^2}{\partial z^2}\left({\lambda }_1\frac{\partial f_1}{\partial w_{zz}}\right)-\frac{{\partial }^2}{\partial r^2}\left({\lambda }_2\frac{\partial f_2}{\partial w_{rr}}\right)\hfill \\ & & -\frac{{\partial }^2}{\partial z^2}\left({\lambda }_2\frac{\partial f_2}{\partial w_{zz}}\right)+\left(2\pi r\rho -\frac{{\lambda }_3}{\varphi }\right)\frac{\alpha }{Z_0{\overline{\mu }}^2}\frac{\partial \overline{\mu }}{\partial \overline{w}}\frac{\partial \overline{w}}{\partial w},\hfill \end{array}$$

  (27)

  $$\begin{array}{cc}\hfill \frac{\partial {\lambda }_3}{\partial t}& =0,\hfill \end{array}$$

  (28)

  where for a property x, $\overline{x}$ denotes the average x in the differential element at a given r and $z=0$; and subscripted r and z denote partial differentiation, e.g., $T_r={\partial }^{}T/\partial r$, $T_{rr}={\partial }^{2}T/\partial r^2$, etc.
  The costate equations are subject to the following terminal and boundary conditions:

  $$\begin{array}{cc}\hfill {\lambda }_i(r,z,t_{\mathrm{f}})& =0\forall 0\le z\le Z(r,t_{\mathrm{f}})\mathrm{and}0\le r\le R;i=1,2,\hfill \end{array}$$

  (29)

  $$\begin{array}{cc}\hfill {\lambda }_3(r,t_{\mathrm{f}})& =0\forall 0\le r\le R,\hfill \end{array}$$

  (30)

  $$\begin{array}{cc}\hfill {\lambda }_1(r,z,t)={\lambda }_2(r,z,t)& =0\mathrm{at}\left\{\begin{array}{ll}r=R& \forall 0\le z\le Z(r,t)\\ z=0,Z(r,t)& \forall 0\le r<R,\end{array}\right.\hfill \end{array}$$

  (31)

  $$\begin{array}{cc}\hfill \frac{\partial {\lambda }_1}{\partial r}=\frac{\partial {\lambda }_2}{\partial r}& =0\mathrm{at}r=0\forall 0\le z\le Z(r,t).\hfill \end{array}$$

  (32)
• Stationarity Condition This stems from zeroing out the variational derivative of J with respect to $T_{\mathrm{int}}(t)$, and is as follows:

  $$\begin{array}{l}-\frac{1}{R}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}\frac{\partial }{\partial r}{\left({\lambda }_1\frac{\partial f_1}{\partial w_{rr}}\right)}_R-\frac{1}{R}\frac{\partial }{\partial r}{\left({\lambda }_1\frac{\partial f_1}{\partial T_{rr}}\right)}_R-\frac{1}{Z_0}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}\frac{\partial }{\partial z}{\left({\lambda }_1\frac{\partial f_1}{\partial w_{zz}}\right)}_{Z_0}\\ +\frac{1}{Z_0}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}\frac{\partial }{\partial z}{\left({\lambda }_1\frac{\partial f_1}{\partial w_{zz}}\right)}_{z=0}-\frac{1}{Z_0}\frac{\partial }{\partial z}{\left({\lambda }_1\frac{\partial f_1}{\partial T_{zz}}\right)}_{Z_0}+\frac{1}{Z_0}\frac{\partial }{\partial z}{\left({\lambda }_1\frac{\partial f_1}{\partial T_{zz}}\right)}_0\hfill \\ -\frac{1}{R}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}\frac{\partial }{\partial r}{\left({\lambda }_2\frac{\partial f_2}{\partial w_{rr}}\right)}_R-\frac{1}{R}\frac{\partial }{\partial r}{\left({\lambda }_2\frac{\partial f_2}{\partial T_{rr}}\right)}_R-\frac{1}{Z_0}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}\frac{\partial }{\partial z}{\left({\lambda }_2\frac{\partial f_2}{\partial w_{zz}}\right)}_{Z_0}\hfill \\ +\frac{1}{Z_0}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}\frac{\partial }{\partial z}{\left({\lambda }_2\frac{\partial f_2}{\partial w_{zz}}\right)}_{z=0}-\frac{1}{Z_0}\frac{\partial }{\partial z}{\left({\lambda }_2\frac{\partial f_2}{\partial T_{zz}}\right)}_{Z_0}+\frac{1}{Z_0}\frac{\partial }{\partial z}{\left({\lambda }_2\frac{\partial f_2}{\partial T_{zz}}\right)}_{z=0}\hfill \\ +\frac{{\lambda }_3}{Z_0}\frac{\partial f_3}{\partial \overline{w}}\frac{\partial \overline{w}}{\partial w_{\mathrm{int}}}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}+\frac{{\lambda }_3}{Z_0}\frac{\partial f_3}{\partial \overline{T}}\frac{\partial \overline{T}}{\partial T_{\mathrm{int}}}-\frac{2\pi \alpha \rho r}{Z_0{\overline{\mu }}^2}\left(\frac{\partial \overline{\mu }}{\partial \overline{w}}\frac{\partial \overline{w}}{\partial w_{\mathrm{int}}}\frac{\partial w_{\mathrm{int}}}{\partial T_{\mathrm{int}}}+\frac{\partial \overline{\mu }}{\partial \overline{T}}\frac{\partial \overline{T}}{\partial T_{\mathrm{int}}}\right)=0.\hfill \end{array}$$

  (33)

3.2.3. Computational Algorithm

The following computational algorithm was executed to determine optimal, open loop control policy at a given pressure:

• Provide an initial guess for control values at each sample time instant in the interval.
• Integrate the state equations forward using the initial and boundary conditions and the control values. Save the values of state variables at all sample time instants and spatial grid points.
• Evaluate the objective functional. If the relative increase in its value is less than ${10}^{-6}$, quit with the control values, which are optimal.
• Integrate the costate equations backward using the terminal and boundary conditions, and the values of control and the saved state variables. Save the values of costate variables at all sample time instants and spatial grid points.
• Compute the left hand side of the stationarity condition at each sample time to obtain the gradient, and use it to improve the control values.
• Go to Step 2 and repeat computations with the improved control values.

We applied the second order finite-difference approximations along the radial and axial directions to the partial differential, state and costate equations (PDEs). This step yielded a set of ordinary differential equations (ODEs) for each PDE at the grid points. The ODEs were then integrated using the fifth order Runge–Kutta–Fehlberg method with adaptive step-size control [27]. Grid-independence analysis was done to determine adequate number of grid points to be used in computations [24]. Broyden–Fletcher–Goldfarb–Shanno (BFGS) method [28] was utilized to improve the control values. The computational algorithm was programmed in C++, and executed on a desktop computer with Intel Core-I7 processor (64 bit, 3.60 GHz and 15.9 GB usable RAM). Appendix D provides the parameter values used in the computations.

3.2.4. Open Loop Results

The above algorithm was applied for five different pressures—24.5, 43.5, 50, 58.5, and 74.5 psi. In addition, operating temperature ranged from 25.0 °C to 90.0 °C. These operating ranges for pressure and temperature were selected based on the design limitations of the experimental apparatus used to validate the process model in previous studies [26,29,30]. At each pressure, the algorithm was executed several times with different initial guesses using 30 control values equispaced over the 163.9 min time interval. It was found that the algorithm converged to the same, final values of control versus time at a given pressure.

Figure 2 shows the control values thus obtained. The corresponding values of optimal oil mass, and computation time and iterations taken by the algorithm are shown in

Table 1. Optimal I (i.e., recovered oil mass) under open loop control at different pressures, and the corresponding computational overhead.

P (psi)	Optimal I (g)	Computation Time (min)	Iterations
24.5	19.3	20.3	21
43.5	35.0	18.1	16
50.0	42.6	18.8	17
58.5	52.8	23.0	27
74.5	58.3	21.4	20

.

It is observed that as pressure increases from 24.5 psi to 74.5 psi, value of optimal I increases from 19.3 $\mathrm{g}$ to 58.3 $\mathrm{g}$, whereas the average optimal $T_{\mathrm{int}}$ decreases from 77.9 °C to 56.6 °C. This result indicates that a higher pressure causes more gas to penetrate the reservoir and increase oil recovery. In addition, a lower gas temperature is more conducive to oil recovery at a higher pressure.

It may be noted that after quickly achieving a maximum, the optimal $T_{\mathrm{int}}$ becomes mildly wavy in nature. Such behavior leads to temporary reversals in temperature and concentration gradients, which have been found to augment oil recovery in past studies [26,29,31].

Based on these optimal open loop results, we now implement closed loop control in presence of pressure upsets in the range, 24.5–74.5 psi.

3.3. Closed Loop Optimal Control

Following the development of Section 2, the Hamiltonian associated with Equation (25) is given by

$$\begin{array}{ccc}\hfill H& =& {\displaystyle \sum _{i=0}^{L-1}}\left(2\pi \rho r_i-\frac{{\lambda }_{3i}}{\varphi }\right)\frac{\alpha }{\overline{\mu }}\Delta r+{\displaystyle \sum _{i=0}^{L-1}\sum _{j=0}^{M-1}}{\lambda }_{1ij}\left[\gamma \left(\frac{T_{rij}}{r_i}+T_{rrij}+T_{zzij}\right)+\frac{\alpha T_{zij}}{\varphi {\mu }_{ij}}\right.\\ & & \left.-\frac{\alpha T_{ij}}{\varphi {\mu }_{ij}^2}\left({\mu }_{wij}{w}_{zij}+{\mu }_{Tij}{T}_{zij}\right)\right]\Delta r\Delta z+{\displaystyle \sum _{i=0}^{L-1}\sum _{j=0}^{M-1}}{\lambda }_{2ij}\left[D_{ij}\left(\frac{w_{rij}}{r_i}+w_{rrij}\right)\right.\\ & & \left.+D_{wij}{w}_{rij}^2+D_{Tij}{w}_{rij}{T}_{rij}+\frac{\alpha w_{zij}}{\varphi {\mu }_{ij}}-\frac{\alpha w_{ij}}{\varphi {\mu }_{ij}^2}\left({\mu }_{wij}{w}_{zij}+{\mu }_{Tij}{T}_{zij}\right)\right]\Delta r\Delta z,\end{array}$$

(34)

where the two states (T and w), costates (${\lambda }_1$ and ${\lambda }_2$) as well as their spatial derivatives span the radial and axial directions, which are discretized using equispaced grid points indexed, respectively, by subscripts i and j. The third state (Z) and the costate (${\lambda }_3$) span the radial direction only.

3.3.1. Control Methodology

For the online control of the process, we proceeded as follows. Right after the convergence of the open loop algorithm in Section 3.2.3 for a given pressure, the differential Riccati equation, Equation (12), was integrated based on the Hamiltonian given by Equation (34), and the gain matrix was computed from Equation (14). In this way, a repository of the optimal states and gains was obtained for five pressures, i.e., 24.5, 43.5, 50, 58.5, and 74.5 psi.

Next, the following online computational algorithm was executed for the closed loop control of the process, which was subjected to single or multiple step changes in pressure:

• Begin the process simulation at a specified pressure, and apply the open loop, optimal control values determined earlier.
• When there is or has been a pressure upset, replace the current control $T_{\mathrm{int}}$ with

  $$T_{\mathrm{int}}-\mathbf{K}(\widehat{\mathbf{y}}-\mathbf{y}),$$

  where $\mathbf{y}$ is the current state while $\mathbf{K}$ and $\widehat{\mathbf{y}}$ are the gain and optimal state at the new pressure. The last two items are interpolated from the pre-determined repository of optimal states and gains.
• Go to Step 2 at the next sample time.

Both open loop and closed loop algorithms are portrayed in Figure 3.

3.3.2. Results

In this section, we present the results obtained from the application of the developed control methodology on the heavy oil recovery process subjected to one or more pressure upsets or step changes.

Figure 4 shows the results for the closed loop control of the process after a step change of 17% in the initial pressure—from 50 psi to 58.5 psi—at 22.6 min. The open loop, optimal control policies at the five different pressures are also shown. It is observed that the control $T_{\mathrm{int}}$ after the pressure upset approaches in about 27 min to the optimal policy corresponding to the new pressure. This adjustment leads to an improvement in the objective functional value, i.e., oil recovery. It gets raised from 42.6 g (at 50 psi without upset) to 51.0 g, which is close to 52.8 g (at 58.5 psi without upset).

Figure 5 shows the results for a process run similar to the previous one but with the upset delayed to 73.5 min. It is observed that the control takes about the same time to approach the optimal policy for the new pressure, and yields 48 g of oil. This value is higher than that for 50 psi but lower than that in Figure 4. This is reasonable as the process is subjected to higher pressure (which facilitates oil recovery) but for a shorter duration though, due to the delayed pressure upset.

Figure 6 shows the results for the process initiated at 50 psi, and then subjected to five pressure upsets of 52.5, 56, 58.5, 62.5, and 67.5 psi at, respectively, 28.3, 39.6, 56.5, 73.5, and 84.8 min. It is observed that the control begins to decrease with the onset of pressure upsets, which are all positive. This is expected since lower control values are optimal at higher pressures. The oil recovery is 54.8 g, which lies between objective functional values of the neighborhood pressures, i.e., 52.8 g (for 58.5 psi) and 58.3 g (for 74.5 psi).

Tracking Property

To examine the tracking property of the control methodology, we consider the process control run of Figure 5, which is subjected to 17% pressure increment at 73 min. Figure 7 plots the reservoir temperature at the grid point: ($i=2,j=3$). It is observed that the temperature under feedback control progressively approaches the optimal reservoir temperature for that grid point at the new pressure.

For all grid points, Figure 8 plots the square of the error between the reservoir temperature under feedback control, and the corresponding optimal temperature at the new pressure. It is observed that the errors tend to insignificant values with time as the controlled state tracks the optimal state corresponding to the new pressure. The time derivative of the plotted error is of the order ${10}^{-5}$ in the end. These results demonstrate that the developed control methodology has the capability to steer the process to the new optimum in case of a pressure upset.

Computational Overhead

With the modest computer configuration described in Section 3.2.3, it took less than a microsecond to carry out online computations to determine and apply a feedback correction to control.

Thus, the online feedback control has negligible time delay, which is very beneficial since it helps minimize the propagation of the effects of upsets on a process. As to storage requirements, it may be noted that the repository, which was determined offline, required a total space of just 0.185 MB.

Overall, the results indicate that the proposed control methodology has a potential for real-time, feedback control of non-linear and distributed-parameter processes. This methodology has the ability to take control action with minimal time delay, and track the optimal state. Of course, the field implementation would require suitable instrumentation including soft sensors for online state measurement/estimation at different grid point locations of the system.

4. Conclusions

Based on the perturbation from an optimal state, a closed loop control strategy was developed for non-linear, distributed-parameter processes. The strategy incorporates offline computation of a repository of open-loop optimal states and gains corresponding to different values of a parameter susceptible to upsets. When an upset in the parameter occurs, control adjustments are quickly determined from the repository to attain the new optimal state.

The strategy was applied to a highly non-linear process of gas-based heavy oil recovery with three states varying with space and time in a moving boundary scenario. The objective was to maximize the oil recovery using gas temperature versus time as the control function with pressure subjected to upsets. Single and multiple pressure upsets were applied in process simulations, and relevant control adjustments were made on the basis of the pre-computed repository of optimal states and gains, which occupied a modest space of 0.185 MB. In each case, the process attained the final, optimal state corresponding to the last value of the disturbed parameter. The time taken to compute control adjustments was less than a microsecond on a desktop computer with Intel Core-I7 processor (64 bit, 3.60 GHz and 15.9 GB usable RAM). These results suggest that the developed strategy can be promising for the real-time control of non-linear, distributed-parameter processes.