LSTM-CNN Network-Based State-Dependent ARX Modeling and Predictive Control with Application to Water Tank System

Abstract

Industrial process control systems commonly exhibit features of time-varying behavior, strong coupling, and strong nonlinearity. Obtaining accurate mathematical models of these nonlinear systems and achieving satisfactory control performance is still a challenging task. In this paper, data-driven modeling techniques and deep learning methods are used to accurately capture a category of a smooth nonlinear system’s spatiotemporal features. The operating point of these systems may change over time, and their nonlinear characteristics can be locally linearized. We use a fusion of the long short-term memory (LSTM) network and convolutional neural network (CNN) to fit the coefficients of the state-dependent AutoRegressive with the eXogenous variable (ARX) model to establish the LSTM-CNN-ARX model. Compared to other models, the hybrid LSTM-CNN-ARX model is more effective in capturing the nonlinear system’s spatiotemporal characteristics due to its incorporation of the strengths of LSTM for learning temporal characteristics and CNN for capturing spatial characteristics. The model-based predictive control (MPC) strategy, namely LSTM-CNN-ARX-MPC, is developed by utilizing the model’s local linear and global nonlinear features. The control comparison experiments conducted on a water tank system show the effectiveness of the developed models and MPC methods.

1. Introduction

MPC is an effective control method developed in industrial practice. It can adequately deal with the problems of multivariable, multi-constraint, strong coupling, and strong nonlinearity existing in actual industrial process objects; it has been a widespread in academia and industry and resulted in numerous theoretical research and application results [1,2]. The MPC method predicts the system’s future behavior based on the controlled object’s dynamic model to achieve the goal of optimal control. Therefore, the predictive model’s capacity to describe the system has a substantial effect on the MPC controller’s control performance. Due to the fact that some key parameters of complex nonlinear systems cannot be determined or obtained, it is difficult to obtain a physical model that accurately represents the system’s nonlinear dynamic characteristics by establishing differential or difference equations via Newtonian mechanics analysis or the Lagrange energy method [3]. Therefore, data-driven identification is a highly effective modeling approach for realizing the MPC of nonlinear systems. This method uses input and output data to construct the system model and does not need to analyze the complex interrelationship between the various physical variables of the system [4,5].

When designing predictive control algorithms based on the identified nonlinear system models, using segmented linearization [6] or local linearization [7] models can simplify the controller’s design but not adequately represent the complex system’s nonlinear dynamic characteristics, which will affect the controller’s control performance. To address the accuracy loss issue caused by linearization models, direct use of nonlinear models such as bilinear models [8,9], Volterra series models [10,11], and neural network models [12,13] can provide a better description of the system’s nonlinear dynamic characteristics. However, predictive control algorithms based on these nonlinear models need to solve non-convex optimization problems online, which may increase the computational burden and not guarantee feasible solutions. To address these issues, in recent years, many scholars have started studying combined models, including the Hammerstein model [14], Wiener model [15], Hammerstein-Wiener model [16,17], and SD-ARX model [18,19,20,21,22,23,24,25]. Among them, the SD-ARX model outperforms the Hammerstein and Wiener models in capturing the nonlinear system’s dynamic properties. The SD-ARX model uses state-dependent coefficients to represent the system’s nonlinear dynamic properties, and its local linear and global nonlinear structural characteristics make it easy to design MPC controllers. In recent years, SD-ARX models obtained by fitting model coefficients using RBF neural networks [18,19,20,21,22,23], deep belief networks [24], and wavelet neural networks [25] have been widely employed for complex industrial system modeling and control. However, these neural networks belong to feedforward neural networks, and their data information can only be transmitted forward. The connection between network nodes at each layer does not form a cycle, so their ability to describe nonlinear systems is limited.

With the fast advancement of AI technology, deep learning has attained success in numerous fields [26]. Its core idea is to learn the high-dimensional features of data via multi-level nonlinear transformation and a back-propagation algorithm. LSTM and CNN are popular deep learning neural networks, but their design objectives and application scenarios are different. LSTM is primarily employed for time series modeling, has strong memory ability and long-term dependence, and has achieved great success in natural language processing [27,28,29], speech recognition [30,31,32], time series prediction [33,34,35,36], etc. Because the LSTM neural network introduces a gating mechanism, it successfully solves the issue of gradient vanishing and exploding in a standard recurrent neural network (RNN), allowing it to process long sequence data more efficiently, which is conducive to nonlinear systems modeling. For example, Wu et al. [37] developed a dropout LSTM and co-teaching LSTM strategy for nonlinear systems. Terzi et al. [38] developed an MPC method based on the LSTM network and carried out numerical tests on the pH reactor simulator. Zarzycki et al. [39] developed a model prediction algorithm based on the LSTM network that has achieved good modeling accuracy and control effects by using online advanced tracking linearization. Although these modeling methods based on the LSTM neural network can properly learn the time dimension features in nonlinear system data, their ability to learn the spatial dimension features is limited, which affects the accuracy of system modeling. In addition, the validation of these methods has been proved by numerical simulation only and has not been applied to real objects.

The CNN can autonomously learn the spatial features of input data via convolution and pooling operations and is mainly used in computer vision [40,41,42,43]. Therefore, the composite neural network consisting of LSTM and CNN can effectively learn the spatiotemporal features of data and improve the accuracy of complex nonlinear modeling. Its research mainly focuses on emotion analysis [44,45,46], text classification [47,48,49], and electricity forecasting [50,51], but has not been found to be used for modeling and MPC of real industrial objects.

This article established the LSTM-ARX and LSTM-CNN-ARX models to represent a category of smooth nonlinear systems whose operating point may change over time and whose nonlinear characteristics can be locally linearized by using LSTM or/and CNN to fit the SD-ARX model’s coefficients. Furthermore, according to the pseudo-linear structure of the SD-ARX model, which exhibits both local linearization and global nonlinearity, two model predictive controllers have been designed, namely LSTM-ARX-MPC and LSTM-CNN-ARX-MPC. To evaluate the performance of these models and control algorithms, a real-time control comparative study was conducted on the commonly used water tank system in industrial process control. The outcomes showed that the developed models and MPC algorithms are effective and feasible. Particularly, the LSTM-CNN-ARX-MPC demonstrates excellent comprehensive control performance by leveraging the strengths of LSTM for learning temporal dimension characteristics and CNN for learning spatial dimension characteristics, enabling it to efficiently and accurately learn the nonlinear system’s spatiotemporal characteristics from large volumes of data. This article’s major contributions are summarized below.

(1)

The LSTM-ARX and LSTM-CNN-ARX models are proposed to describe the system’s nonlinear features.

(2)

The predictive controller was developed using the model’s pseudo-linear structure features.

(3)

Control comparison experiments were conducted on the water tank system, which is a commonly used industrial process control device, to validate the efficiency of the developed models and control algorithms. To our knowledge, there are currently no reports on using deep learning algorithms for nonlinear system modeling and the real-time control of actual industrial equipment. This study demonstrates how to establish deep learning-related models for nonlinear systems, design the MPC algorithms, and achieve real-time control rather than only doing a numerical simulation as in the relevant literature.

The article’s structure is as follows: Section 2 describes the related work. Section 3 studies three combination models. Section 4 designs the model-based predictive controllers. Section 5 presents the results of real-time control comparative experiments on the water tank system. Section 6 summarizes the research content.

2. Related Work

The SD-ARX, LSTM, and CNN models are summarized in this part.

2.1. SD-ARX Model

Taylor expansion and local linearization models are often used to deal with nonlinear systems [52,53]. Using these ideas, Priestley [54] proposed the SD-AR model structure of a class of nonlinear time series and pointed out that the SD-AR model can fit nonlinear systems without any prior conditions. Peng et al. [22] extended the SD-AR model to the SD-ARX model. The nonlinear ARX model is used to represent a category of smooth nonlinear systems, as shown below:

$$\{\begin{array}{l}y(a)=\eta (\epsilon (a-1))+\theta (a)\\ \epsilon (a-1)=[y(a-1{)}^{\mathrm{T}},\mathrm{\dots },y(a-s_y{)}^{\mathrm{T}},u(a-1{)}^{\mathrm{T}},\mathrm{\dots },u(a-s_u{)}^{\mathrm{T}}{]}^{\mathrm{T}}\end{array}$$

(1)

where $y(a)$ represents output, $u(a)$ represents input, $\theta (a)$ represents modeling error, $s_y$ and $s_u$ represent model orders. At an operating point ${\epsilon }_0$, $\eta (•)$ is expanded into the following Taylor polynomial

$$\begin{array}{l}\eta (\epsilon (a-1))=\eta ({\epsilon }_0)+{\eta }^{\prime }({\epsilon }_0{)}^{\mathrm{T}}(\epsilon (a-1)-{\epsilon }_0)+ \\ \frac{1}{2} (\epsilon (a-1)-{\epsilon }_0{)}^{\mathrm{T}}{\eta }^{″}({\epsilon }_0)(\epsilon (a-1)-{\epsilon }_0)+\dots +\gamma (\epsilon (a-1))\end{array}$$

(2)

Then, substituting Equation (2) into Equation (1) yields the SD-ARX model as follows.

$$\{\begin{array}{l}y(a)={\rho }_0\left(\epsilon (a-1)\right)+{\displaystyle \sum _{i=1}^{s_y}{\rho }_{y,i}\left(\epsilon (a-1)\right)y(a-i)}+{\displaystyle \sum _{j=1}^{s_u}{\rho }_{u,j}\left(\epsilon (a-1)\right)u(a-j)}+\theta (a)\\ {\mathit{\ell }}_0=\eta ({\epsilon }_0)-{\eta }^{\prime }({\epsilon }_0{)}^{\mathrm{T}}{\epsilon }_0+\frac{1}{2}{\epsilon }_0^{\mathrm{T}}{\eta }^{″}({\epsilon }_0){\epsilon }_0+\dots \\ {\mathit{\ell }}_1(\epsilon (a-1))=\gamma (\epsilon (a-1))\\ {\mathrm{{\rm T}}}_0={\eta }^{\prime }({\epsilon }_0{)}^{\mathrm{T}}-\frac{1}{2}{\epsilon }_0^{\mathrm{T}}{\eta }^{″}({\epsilon }_0)-\frac{1}{2}{\epsilon }_0^{\mathrm{T}}{\eta }^{″}({\epsilon }_0{)}^{\mathrm{T}}+\mathrm{\dots }\\ {\mathrm{{\rm T}}}_1(\epsilon (a-1))=\frac{1}{2}\epsilon (a-1{)}^{\mathrm{T}}{\eta }^{″}({\epsilon }_0)+\dots \\ {\mathit{\ell }}_0+{\mathit{\ell }}_1(\epsilon (a-1))={\rho }_0\left(\epsilon (a-1)\right)\\ {\mathrm{{\rm T}}}_0+{\mathrm{{\rm T}}}_1(\epsilon (a-1))=[{\rho }_{y,1}\left(\epsilon (a-1)\right),\mathrm{\dots },{\rho }_{y,s_y}\left(\epsilon (a-1)\right), {\rho }_{u,1}\left(\epsilon (a-1)\right),\mathrm{\dots },{\rho }_{u,s_u}\left(\epsilon (a-1)\right)]\end{array}$$

(3)

where $\left\{{\rho }_{y,i}(\epsilon (a-1))|i=1,\mathrm{\dots },s_y\right\}$, $\left\{{\rho }_{u,j}(\epsilon (a-1))|j=1,\mathrm{\dots },s_u\right\}$ and ${\rho }_0(\epsilon (a-1))$ represent the regression coefficients that can be obtained by fitting using neural networks, including BRF neural networks [18,19,20,21,22,23] and wavelet neural networks [24]. $\epsilon (a-1)$ represents the operating state at time $a$, which may correspond to the system’s input or/and output. When fixing $\epsilon (a-1)$, Equation (3) is considered a locally linearized model. When $\epsilon (a-1)$ varies with the system’s operating point, Equation (3) represents a model capable of globally describing the system’s nonlinear properties. This model has local linear and global nonlinear features, making it advantageous for MPC design. In this article, we use LSTM or/and CNN to approximate the model’s regression coefficients and obtain a category of SD-ARX models that can effectively represent the system’s nonlinear properties.

2.2. LSTM

LSTM and GRU are commonly used RNN variants for handling time series data. In comparison to the GRU network, LSTM possesses stronger memory and can capture longer dependencies. which may be more effective in handling long data sequences and modeling intricate systems. LSTM networks can not only solve the problem that traditional RNNs cannot handle the dependence of long sequence data but also the issue that gradient exploding or vanishing is easy to occur with increasing training time and network layers of the neural networks. The LSTM network consists of multiple LSTM units, and its internal structure and chain structure are shown in Figure 1, with its core being the cell state $C$, represented by a horizontal line running through the entire cell at the top layer. It controls the addition or removal of information via gates. LSTM mainly consists of an input gate, a forget gate, and an output gate. The input gate controls which input information needs to be added to the current time step’s storage unit. The forget gate controls which information needs to be forgotten in the storage unit of the previous time step. The output gate controls which information in the current time step’s storage unit needs to be output to the next time step. LSTM introduces a gate mechanism to control information transmission, remember content that needs to be memorized for a long time, and forget unimportant information, making it particularly effective in handling and forecasting critical events with long intervals and temporal delays in sequential data. The formula for an LSTM unit is as follows.

$$\{\begin{array}{l}{c}_l=f_l\odot c_{l-1}+i_l\odot {\sigma }_5\left(x_l{W}_{cx}+h_{l-1}{W}_{ch}+b_c\right)\\ h_l=o_l\odot {\sigma }_5\left(c_l\right)\\ f_l={\sigma }_4\left(x_l{W}_{fx}+h_{l-1}{W}_{fh}+b_f\right)\\ i_l={\sigma }_4\left(x_l{W}_{ix}+h_{l-1}{W}_{ih}+b_i\right)\\ o_l={\sigma }_4\left(x_l{W}_{ox}+h_{l-1}{W}_{oh}+b_o\right)\end{array}$$

(4)

where $l$ represents the time step of the input sequence, referred to as the number of rounds; $\odot$ represents matrix dot multiplication; $c_l$ and $h_l$ denote cell state vector and hidden state vector, respectively; ${\sigma }_4(•)$ and ${\sigma }_5(•)$ denote the nonlinear activation functions; $x_l$ represents input information; $f_l$ represents the forget gate’s output, which decides the amount of information from the preceding state $c_{l-1}$ needs to be forgotten. The value of the output ranges from 0 to 1, with a smaller value indicating more information to be forgotten, 0 meaning total forgetting, and 1 meaning total retention; $i_l$ indicates the input gate’s output, which controls the current output state. Its output value is usually mapped to a range of 0 to 1 using the ${\sigma }_4(•)$ function, indicating the degree of activation of the output state, with 1 indicating full activation and 0 indicating no activation; $o_l$ represents the output gate’s output, which decides the information to be output in this round;$W_{cx}$, $W_{ch}$, $W_{fx}$, $W_{fh}$, $W_{ix}$, $W_{ih}$, $W_{ox}$, and $W_{oh}$ denote weight coefficients; $b_f$, $b_i$, $b_o$, and $b_c$ denote offsets.

Figure 1. The schematic diagram of LSTM [32].

2.3. CNN

CNN has outstanding spatial feature extraction capabilities. CNN can automatically learn the spatial characteristics of liquid level distribution and liquid pressure in water tank systems. For example, CNN can learn the liquid level distribution at different positions, that is, the changes in water surface height. Meanwhile, by learning the pressure changes hidden in the input and output data, CNN can learn the liquid pressure information at different positions. CNN is mainly composed of the input layer, convolutional layer, pooling layer, fully connected layer, and output layer, as demonstrated in Figure 2. The input layer is employed for receiving raw data. The convolutional layer is utilized for extracting features from the input data. The pooling layer downsamples the convolution layer’s output to lower the data’s dimension. The fully connected layer turns the pooling layer’s output into a one-dimensional vector and outputs it through the output layer. Compared with the full connection mode of the traditional neural network, CNN uses a convolution kernel of appropriate size to extract the local features of this layer’s input, that is, this CNN convolution layer’s neurons are only connected to some of the upper layer’s neurons. This local connectivity method reduces the neural network’s parameter count and overfitting risk. At the same time, CNN employs weight sharing to enable convolutional kernels to capture identical features at varying places, resulting in reduced model training complexity and imparting the model with the characteristics of translation invariance. The convolution operation formula is below:

$$x_{j_1}^{l_1}={\sigma }_1\left({\displaystyle \sum _{i_1=1}^{g_1}{x}_{i_1}^{l_1-1}\otimes W_{i_1{j}_1}^{l_1}+b_{j_1}^{l_1}}\right)$$

(5)

where $\otimes$ represents convolution calculation; $x_{j_1}^{l_1}$ denotes the $j_1$-th feature map of the $l_1$-th convolution layer; $b_{j_1}^{l_1}$ represents offset; $w_{i_1{j}_1}^{l_1}$ represents convolution kernel matrix; ${\sigma }_1(•)$ represents the nonlinear activation functions; $g_1$ indicates the number of input feature maps.

Figure 2. The CNN’s framework.

3. Hybrid Models

This section describes three combination models, namely the LSTM-ARX model, CNN-ARX model, and LSTM-CNN-ARX model.

3.1. LSTM-ARX Model

LSTM neural networks utilize recurrent connections that allow information to flow in a directed cycle, enabling them to effectively capture the temporal dependencies in data during modeling. Unlike feedforward neural networks, which only propagate information forward, LSTM can retain and utilize information from previous time steps, enabling them to learn and represent long-term dependencies in the input sequences. This property makes LSTM particularly well-suited for building sequential models of nonlinear systems. Simultaneously, a gating mechanism is introduced in LSTM networks to control the flow and discarding of historical information and input characteristics, which solves the long-term dependence problem in simple RNN networks. In the actual modeling process, single-layer LSTM is usually unable to express complex temporal nonlinear characteristics, so a series approach is often adopted to stack multiple LSTM layers to deepen the network and bolster modeling capabilities. The LSTM-ARX model can be obtained by approximating the function coefficient of the model (3) with LSTM. It combines the benefits of LSTM for handling long sequence data with the SD-ARX model’s nonlinear expression capabilities to fully represent the nonlinear system’s dynamic features. Figure 3 describes the LSTM-ARX model’s architecture and its expression below:

$$\{\begin{array}{l}y(a)={\rho }_0\left(\epsilon (a-1)\right)+{\displaystyle \sum _{i=1}^{s_y}{\rho }_{y,i}\left(\epsilon (a-1)\right)y(a-i)}+{\displaystyle \sum _{j=s_d}^{s_u+s_d-1}{\rho }_{u,j}\left(\epsilon (a-1)\right)u(a-j)}+\theta (a)\\ [{\mathbf{\rho }}_0(\mathbf{\epsilon }(a-1)) ,{\mathbf{\rho }}_{y,i}(\epsilon (a-1)),{\rho }_{u,j}(\epsilon (a-1))]=W\left(f(h_d^n(a)W_{hy}+b_y)\right)+b\\ h_l^r(a)={\sigma }_4\left(h_l^{r-1}(a)W_{ox}^r+h_{l-1}^r(a)W_{oh}^r+b_o^r\right)\odot {\sigma }_5\left(c_l^r(a)\right),\mathrm{r}=2,3,\mathrm{\dots },n\\ c_l^r(a)={\sigma }_4\left(h_l^{r-1}(a)W_{fx}^r+h_{l-1}^r(a)W_{fh}^r+b_f^r\right)\odot c_{l-1}^r(a)\\ \hspace{1em}\hspace{1em}\hspace{1em}+{\sigma }_4\left(h_l^{r-1}(a)W_{ix}^r+h_{l-1}^r(a)W_{ih}^r+b_i^r\right)\odot {\sigma }_5\left(h_l^{r-1}(a)W_{cx}^r+h_{l-1}^r(a)W_{ch}^r+b_c^r\right)\\ h_l^1(a)={\sigma }_4\left({\epsilon }_l{W}_{ox}^1+h_{l-1}^1(a)W_{oh}^1+b_o^1\right)\odot {\sigma }_5\left(c_l^1(a)\right), l=1,2,\dots ,d \\ c_l^1(a)={\sigma }_4\left({\epsilon }_l{W}_{fx}^1+h_{l-1}^1(a)W_{fh}^1+b_f^1\right)\odot c_{l-1}^1(a)\\ \hspace{1em}\hspace{1em}\hspace{1em}+{\sigma }_4\left({\epsilon }_l{W}_{ix}^1+h_{l-1}^1(a)W_{ih}^1+b_i^1\right)\odot {\sigma }_5\left({\epsilon }_l{W}_{cx}^1+h_{l-1}^1(a)W_{ch}^1+b_c^1\right)\\ h_0^k(a)=0, c_0^k(a)=0,k=1,2,\mathrm{\dots },n;\epsilon (a-1)=[{\epsilon }_1,\mathrm{\dots },{\epsilon }_d{]}^T\end{array}$$

(6)

where $y(a)$ represents output; $u(a)$ represents input; $\theta (a)$ represents white noise; $s_y$$s_u$, $d$ are model’s orders; $s_d$ denotes time delay; $\epsilon (a-1)$ represents the input data at time $a$, which may correspond to the system’s input or/and output; ${\rho }_0(\epsilon (a-1))$, ${\rho }_{y,i}(\epsilon (a-1))$, and ${\rho }_{u,j}(\epsilon (a-1))$ represent the model’s function coefficient; n denotes the number of hidden layers; $h_l^r(a)$ and $c_l^r(a)$ represent the hidden layer state and cell state of the $r$-th hidden layer of time step $l$ at time $a$; ${\sigma }_4(•)$, ${\sigma }_5(•)$, and $f(•)$ represent activation functions (e.g., Hard Sigmoid and Tanh) that are employed to improve the model’s capability in capturing the system’s nonlinearity; $\{b_f^k,b_i^k,b_o^k,b_c^k|k=1,\mathrm{\dots },n\}$,$b_y$, and $b$ represent offsets; $\{W_{fx}^k,W_{fh}^k,W_{ix}^k,W_{ih}^k,W_{ox}^k,W_{oh}^k,W_{cx}^k,W_{ch}^k|k=1,\mathrm{\dots },n\}$, $W_{hy}$, and $W$ represent weight coefficients.

Figure 3. The LSTM-ARX model’s architecture.

3.2. CNN-ARX Model

CNN has strong nonlinear feature extraction capabilities. The CNN-ARX model is derived by fitting the SD-ARX model’s coefficients to CNN. This model combines the ability of CNN’s local feature extraction and the SD-ARX model’s expression. Compared to traditional neural networks, CNN has the characteristics of local connections, weight sharing, pooling operations, etc., which can further decrease the complexity and overfitting risks of the model and enable the model to possess a certain level of robustness and fault tolerance. Figure 4 describes the CNN-ARX model’s architecture and its expression below:

$$\{\begin{array}{l}y(a)={\rho }_0\left(\epsilon (a-1)\right)+{\displaystyle \sum _{i=1}^{s_y}{\rho }_{y,i}\left(\epsilon (a-1)\right)y(a-i)}+{\displaystyle \sum _{j=s_d}^{s_u+s_d-1}{\rho }_{u,j}\left(\epsilon (a-1)\right)u(a-j)}+\theta (a)\\ [{\mathbf{\rho }}_0\left(\mathbf{\epsilon }(a-1)\right),{\rho }_{y,i}\left(\epsilon (a-1)\right),{\rho }_{u,j}\left(\epsilon (a-1)\right)]=W_1\left(f_1({\tilde{x}}^{n_1}(a)W_{xy}+b_x)\right)+b_1\\ {\tilde{x}}^{n_1}(a)={\sigma }_3\left({\widehat{x}}_{j_1}^{n_1}(a)\right); {\widehat{x}}_{j_1}^{n_1}(a)={\sigma }_2\left(x_{j_1}^{n_1}(a)\right)\\ x_{j_1}^{l_1}(a)={\sigma }_1\left({\displaystyle \sum _{i_1=1}^{g_1}{x}_{i_1}^{l_1-1}(a)\otimes W_{i_1{j}_1}^{l_1}+b_{j_1}^{l_1}}\right),1\le l_1\le n_1\\ x^0(a)=\epsilon (a-1)=[{\epsilon }_1,\dots ,{\epsilon }_d{]}^T\end{array}$$

(7)

where $x^0(a)$ represents input; ${\sigma }_1(•)$ and $f_1(•)$ represent activation functions; ${\sigma }_2(•)$ and ${\sigma }_3(•)$ indicate pooling operation and flattening operation; ${\tilde{x}}^{n_1}(a)$ denotes the one-dimensional vector obtained by flattening the output feature maps ${\widehat{x}}_{j_1}^{n_1}(a)$; $\left\{W_{i_1{j}_1}^{l_1}\right|l_1=1,\mathrm{\dots },n_1\}$, $W_{xy}$, and $W_1$ represent weight coefficients; $\left\{b_{j_1}^{l_1}\right|l_1=1,\mathrm{\dots },n_1\}$, $b_x$, and $b_1$ represent the offsets.

Figure 4. The CNN-ARX model’s architecture.

3.3. LSTM-CNN-ARX Model

The LSTM-CNN-ARX model is obtained by combining LSTM and CNN to approximate SD-ARX model coefficients, as shown in Figure 5. First, employ LSTM for learning the temporal characteristics of the input data, followed by the utilization of CNN to capture the spatial characteristics, and finally calculate the SD-ARX model’s correlation coefficients through a full connection layer to obtain the LSTM-CNN-ARX model. This model integrates the spatiotemporal characteristic extraction ability of LSTM-CNN with the expressive power of the SD-ARX model; this approach effectively captures the nonlinear system’s dynamic features. The LSTM-CNN-ARX model’s mathematical expression is obtained from Formulas (6) and (7), as follows:

$$\{\begin{array}{l}y(a)={\rho }_0\left(\epsilon (a-1)\right)+{\displaystyle \sum _{i=1}^{s_y}{\rho }_{y,i}\left(\epsilon (a-1)\right)y(a-i)}+{\displaystyle \sum _{j=s_d}^{s_u+s_d-1}{\rho }_{u,j}\left(\epsilon (a-1)\right)u(a-j)}+\theta (a)\\ [{\mathbf{\rho }}_0\left(\mathbf{\epsilon }(a-1)\right),{\mathbf{\rho }}_{y,i}\left(\epsilon (a-1)\right),{\rho }_{u,j}\left(\epsilon (a-1)\right)]=W_1\left(f_1({\tilde{x}}^{n_1}(a)W_{xy}+b_x)\right)+b_1\\ {\tilde{x}}^{n_1}(a)={\sigma }_3({\widehat{x}}_{j_1}^{n_1}(a)); {\widehat{x}}_{j_1}^{n_1}(a)={\sigma }_2\left(x_{j_1}^{n_1}(a)\right)\\ x_{j_1}^{l_1}(a)={\sigma }_1\left({\displaystyle \sum _{i_1=1}^{g_1}{x}_{i_1}^{l_1-1}(a)\otimes W_{i_1{j}_1}^{l_1}+b_{j_1}^{l_1}}\right),1\le l_1\le n_1\\ x^0(a)=h_l^n(a)\\ h_l^r(a)={\sigma }_4\left(h_l^{r-1}(a)W_{ox}^r+h_{l-1}^r(a)W_{oh}^r+b_o^r\right)\odot {\sigma }_5\left(c_l^r(a)\right),\mathrm{r}=2,3,\mathrm{\dots },n\\ c_l^r(a)={\sigma }_4\left(h_l^{r-1}(a)W_{fx}^r+h_{l-1}^r(a)W_{fh}^r+b_f^r\right)\odot c_{l-1}^r(a)\\ \hspace{1em}\hspace{1em}\hspace{1em}+{\sigma }_4\left(h_l^{r-1}(a)W_{ix}^r+h_{l-1}^r(a)W_{ih}^r+b_i^r\right)\odot {\sigma }_5\left(h_l^{r-1}(a)W_{cx}^r+h_{l-1}^r(a)W_{ch}^r+b_c^r\right)\\ h_l^1(a)={\sigma }_4\left({\epsilon }_l{W}_{ox}^1+h_{l-1}^1(a)W_{oh}^1+b_o^1\right)\odot {\sigma }_5\left(c_l^1(a)\right), l=1,2,\mathrm{\dots },d \\ c_l^1(a)={\sigma }_4\left({\epsilon }_l{W}_{fx}^1+h_{l-1}^1(a)W_{fh}^1+b_f^1\right)\odot c_{l-1}^1(a)\\ \hspace{1em}\hspace{1em}\hspace{1em}+{\sigma }_4\left({\epsilon }_l{W}_{ix}^1+h_{l-1}^1(a)W_{ih}^1+b_i^1\right)\odot {\sigma }_5\left({\epsilon }_l{W}_{cx}^1+h_{l-1}^1(a)W_{ch}^1+b_c^1\right)\\ h_0^k(a)=0, c_0^k(a)=0,k=1,2,\mathrm{\dots },n;\epsilon (a-1)=[{\epsilon }_1,\dots ,{\epsilon }_d{]}^T\end{array}$$

(8)

Figure 5. The LSTM-CNN-ARX model’s architecture.

The aforementioned deep learning-based SD-ARX models adopt data-driven techniques for modeling. First, select appropriate and effective input and output data as identification data for modeling. Second, based on the complexity of the identified system, select the model’s initial orders, the network’s layer count, and the node count per layer. Third, choose activation functions (such as Sigmoid, Tanh, and Relu) and optimizers (such as Adam and SGD) and configure other hyperparameters of the model. Fourth, train the model and evaluate its performance using a test set, computing the mean square error (MSE). Fifthly, adjust the orders while keeping the other structures of the model unchanged to train the model with new orders again, and select the orders of the model with the minimum MSE as the optimal orders. Next, keep the other structures of the model unchanged, adjust the network’s layer count, and select the network’s layer count with the smallest MSE as the optimal number of layers. Using the same method, determine the other model structures, i.e., the node count, activation function, and hyperparameters, and ultimately choose the model structure with the smallest MSE as the optimal model.

4. MPC Algorithm Design

This section designs the predictive controller based on models (6)–(8). By utilizing the model’s local linear and global nonlinear features, the predictive controller uses current state operating point information to locally linearize the model, calculates the nonlinear system’s predicted output, and obtains the control law by solving a quadratic programming (QP) problem.

For the convenience of MPC algorithm development, models (6)–(8) can be converted to the following forms:

$$\{\begin{array}{l}y(a)={\rho }_0\left(a-1\right)+{\displaystyle \sum _{q=1}^{k_g}{\stackrel{⌢}{\gamma }}_{q,a-1}y(a-q)}+{\displaystyle \sum _{q=1}^{k_g}{\stackrel{⌢}{\lambda }}_{q,a-1}u(a-q)}+\theta (a)\\ {\rho }_0\left(a-1\right)={\rho }_0\left(\epsilon (a-1)\right)\\ k_g=\max \left(s_y,s_u+s_d-1\right)\\ {\stackrel{⌢}{\gamma }}_{q,a-1}^{}=\{\begin{array}{l}{\rho }_{y,q}\left(\epsilon (a-1)\right),(q\le s_y)\\ 0\hspace{0.17em}\hspace{0.17em},(q>s_y)\end{array} ,{\stackrel{⌢}{\lambda }}_{q,a-1}^{}=\{\begin{array}{l}{\rho }_{u,q}\left(\epsilon (a-1)\right),(s_d\le q\le s_u+s_d-1)\\ 0\hspace{0.17em}\hspace{0.17em}, \mathrm{else}\end{array}\end{array}$$

(9)

where ${\rho }_0\left(a-1\right)$, ${\stackrel{⌢}{\gamma }}_{q,a-1}$, and ${\stackrel{⌢}{\lambda }}_{q,a-1}$ indicate regression coefficients; $k_g$ represents the maximum value of $s_u+s_d-1$ and $s_y$; $\theta (a)$ denotes the white noise.

To transform the above equation into the state-space form, design the vectors below:

$$\{\begin{array}{l}X(a)=[x_{1,a}^{\mathrm{T}}, x_{2,a}^{\mathrm{T}}\cdots x_{k_g,a}^{\mathrm{T}}]\mathrm{T},\\ x_{1,a}^{}=y(a)\\ x_{p,a}^{}={\displaystyle \sum _{i=1}^{k_g+1-p}{\stackrel{⌢}{\gamma }}_{i+p-1,a-1}^{}y(a-i)}+{\displaystyle \sum _{j=1}^{k_g+1-p}{\stackrel{⌢}{\lambda }}_{j+p-1,a-1}^{}u(a-j)}\\ p=2,3,\cdots ,k_g\end{array}$$

(10)

Therefore, Equation (9) becomes the following form:

$$\{\begin{array}{l}X(a+1)=G_{a}X(a)+H_{a}u(a)+{\rho }_a+\Lambda (a+1)\\ y(a)=QX(a)\end{array}$$

(11)

where

$$\{\begin{array}{l}{G}_a={\left[\begin{array}{ccccc}{\stackrel{⌢}{\gamma }}_{1,a}^{}& I& 0& \cdots & 0\\ {\stackrel{⌢}{\gamma }}_{2,a}^{}& 0& I& \cdots & ⋮\\ ⋮& ⋮& ⋮& \ddots & 0\\ {\stackrel{⌢}{\gamma }}_{k_g-1,a}^{}& 0& 0& \cdots & I\\ {\stackrel{⌢}{\gamma }}_{k_g,a}^{}& 0& 0& \cdots & 0\end{array}\right]}_{k_g\times k_g},\\ H_a=[{\stackrel{⌢}{\lambda }}_{1,a}^{} {\stackrel{⌢}{\lambda }}_{2,a}^{} \cdots {\stackrel{⌢}{\lambda }}_{k_g,a}^{}{]}^{\mathrm{T}},Q=[I 0 \cdots 0], \\ {\rho }_a=[{\rho }_0 0 \cdots 0{]}^{\mathrm{T}},\Lambda (a+1)=[\theta (a+1) 0 \cdots 0{]}^{\mathrm{T}}\end{array}$$

(12)

Note that the coefficient matrix above is related to the working state $\epsilon (a)$ at time $a$. For different state-dependent ARX models, the values of matrices $G_a$, $H_a$, and ${\rho }_a$ in Equation (12) are different and can be calculated using Equation (9). To further develop the model predictive controller, define the vectors below:

$$\{\begin{array}{l}\stackrel{⌢}{X}(a)=[\stackrel{⌢}{\mathbf{X}}{(a+1|a)}^{\mathrm{T}}\stackrel{⌢}{\mathbf{X}}{(a+2|a)}^{\mathrm{T}}\cdots \stackrel{⌢}{\mathbf{X}}{(a+M_y|a)}^T{]}^{\mathrm{T}}\\ \stackrel{⌢}{\mathbf{Y}}(a)=[\stackrel{⌢}{\mathbf{y}}{(a+1|a)}^{\mathrm{T}}\stackrel{⌢}{\mathbf{y}}{(a+2|a)}^{\mathrm{T}}\cdots \stackrel{⌢}{\mathbf{y}}{(a+M_y|a)}^{\mathrm{T}}{]}^{\mathrm{T}}\\ \stackrel{⌢}{\mathbf{U}}(a)=[\mathbf{u}{(a)}^{\mathrm{T}}\mathbf{u}{(a+1)}^{\mathrm{T}}\cdots \mathbf{u}{(a+M_u-1)}^{\mathrm{T}}{]}^{\mathrm{T}}\\ {\stackrel{⌢}{\mathbf{\rho }}}_a=[{\mathbf{\rho }}_a^{\mathrm{T}}{\mathbf{\rho }}_{a+1}^{\mathrm{T}}\cdots {\mathbf{\rho }}_{a+M_y-1}^{\mathrm{T}}{]}^{\mathrm{T}}\end{array}$$

(13)

where $\stackrel{⌢}{\mathbf{X}}(a)$, $\stackrel{⌢}{\mathbf{Y}}(a)$, $\stackrel{⌢}{\mathbf{U}}(a)$, and ${\stackrel{⌢}{\mathbf{\rho }}}_a$ represent the multi-step forward prediction state vector, multi-step forward prediction output vector, multi-step forward prediction control vector, and multi-step forward prediction offset, respectively; $\left\{\stackrel{⌢}{\mathbf{X}}{(a+q|a)}^{\mathrm{T}}|q=1,\mathrm{\dots },M_y\right\}$ and $\left\{\stackrel{⌢}{\mathbf{y}}{(a+q|a)}^{\mathrm{T}}|q=1,\mathrm{\dots },M_y\right\}$ represent the $q$-step forward prediction of the state and output; $M_y$ and $M_u$ represent the prediction and control time domains, respectively. The model’s multi-step forward prediction control is presented below:

$$\{\begin{array}{l}\stackrel{⌢}{\mathbf{X}}(a)={\stackrel{⌢}{\mathbf{G}}}_a\mathbf{X}(a)+{\stackrel{⌢}{\mathbf{H}}}_a\stackrel{⌢}{\mathbf{U}}(a)+{\stackrel{⌢}{\mathsf{\Xi }}}_a{\stackrel{⌢}{\mathbf{\rho }}}_a,\\ \stackrel{⌢}{\mathbf{Y}}(a)=\stackrel{⌢}{\mathbf{Q}}\stackrel{⌢}{X}(a)\end{array}$$

(14)

where

$${\stackrel{⌢}{\mathbf{G}}}_a=\left[\begin{array}{c}{\displaystyle \coprod _{i=0}^0{G}_{a+i}}\\ {\displaystyle \coprod _{i=0}^1{G}_{a+i}}\\ ⋮\\ {\displaystyle \coprod _{i=0}^{M_y-1}{G}_{a+i}}\end{array}\right] , {\displaystyle \coprod _{i=j}^q{G}_{a+i}}=\{\begin{array}{l}{G}_{a+q}{G}_{a+q-1}\cdots G_{a+j}, j\le q\\ \begin{array}{cc}I,& j>q\end{array}\end{array}$$

(15)

$${\stackrel{⌢}{\Xi }}_a=\left[\begin{array}{ccccc}I& 0& 0& \cdots & 0\\ {\displaystyle \coprod _{i=1}^1{G}_{a+i}}& I& 0& \cdots & 0\\ {\displaystyle \coprod _{i=1}^2{G}_{a+i}}& {\displaystyle \coprod _{i=2}^2{G}_{a+i}}& I& \cdots & 0\\ ⋮& ⋮& \ddots & \ddots & ⋮\\ {\displaystyle \coprod _{i=1}^{M_y-1}{G}_{a+i}}& {\displaystyle \coprod _{i=2}^{M_y-1}{G}_{a+i}}& \cdots & {\displaystyle \coprod _{i=M_y-1}^{M_y-1}{G}_{a+i}}& I\end{array}\right]$$

(16)

$${\stackrel{⌢}{H}}_a=\left[\begin{array}{ccccc}{H}_a& 0& \cdots & \cdots & 0\\ \left({\displaystyle \coprod _{i=1}^1{G}_{a+i}}\right)H_a& H_{a+1}& 0& \cdots & ⋮\\ ⋮& ⋮& \ddots & \ddots & 0\\ \left({\displaystyle \coprod _{i=1}^{M_u-1}{G}_{a+i}}\right)H_a& \left({\displaystyle \coprod _{i=2}^{M_u-1}{G}_{a+i}}\right)H_{a+1}& \cdots & \left({\displaystyle \coprod _{i=M_u-1}^{M_u-1}{G}_{a+i}}\right)H_{a+M_u-2}& H_{a+M_u-1}\\ \left({\displaystyle \coprod _{i=1}^{M_u}{G}_{a+i}}\right)H_a& \left({\displaystyle \coprod _{i=2}^{M_u}{G}_{a+i}}\right)H_{a+1}& \cdots & \left({\displaystyle \coprod _{i=M_u-1}^{M_u}{G}_{a+i}}\right)H_{a+M_u-2}& {\displaystyle \sum _{j=M_u-1}^{M_u}\left({\displaystyle \coprod _{i=j+1}^{M_u}{G}_{a+i}}\right)H_{a+j}}\\ ⋮& ⋮& ⋮& ⋮& ⋮\\ \left({\displaystyle \coprod _{i=1}^{M_y-1}{G}_{a+i}}\right)H_a& \left({\displaystyle \coprod _{i=2}^{M_y-1}{G}_{a+i}}\right)H_{a+1}& \cdots & \left({\displaystyle \coprod _{i=M_u-1}^{M_y-1}{G}_{a+i}}\right)H_{a+M_u-2}& {\displaystyle \sum _{j=M_u-1}^{M_y-1}\left({\displaystyle \coprod _{i=j+1}^{M_y-1}{G}_{a+i}}\right)H_{a+j}}\end{array}\right]$$

(17)

$$\stackrel{⌢}{Q}=\left[\begin{array}{cccc}Q& 0& \cdots & 0\\ 0& Q& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & Q\end{array}\right]$$

(18)

In Formula (14), the coefficient matrices ${\stackrel{⌢}{G}}_a$, ${\stackrel{⌢}{\Xi }}_a$, ${\stackrel{⌢}{H}}_a$, and $\stackrel{⌢}{Q}$ change with the system state and can be obtained by solving the system’s future working point $\epsilon (a+q|a)$ $(q=1,2,\cdots ,M_y-1)$. Nevertheless, in the actual control process, it may be challenging to obtain information on the future state operating point. Therefore, compute using the current operating point $\epsilon (a)$ instead of $\epsilon (a+q|a)$ for the computer. Obtain the locally linearized model according to Equation (11) and use it to develop the following MPC method.

To facilitate the design of the objective function, Equation (14) is converted into the following forms:

$$\{\begin{array}{l}\stackrel{⌢}{Y}(a)=R_a\stackrel{⌢}{U}(a)+Y_0(a)\\ R_a=\stackrel{⌢}{Q}{\stackrel{⌢}{H}}_a\\ Y_0(a)=\stackrel{⌢}{Q}{\stackrel{⌢}{G}}_{a}X(a)+\stackrel{⌢}{Q}{\stackrel{⌢}{\Xi }}_a{\stackrel{⌢}{\rho }}_a\end{array}$$

(19)

Then, the desired output ${\stackrel{⌢}{Y}}_r(a)$ and control increment $\Delta \stackrel{⌢}{U}(a)$ are defined:

$$\{\begin{array}{l}\mathsf{\Delta }\stackrel{⌢}{U}(a)=[\mathsf{\Delta }u{(a)}^T\mathsf{\Delta }u{(a+1)}^T\cdots \mathsf{\Delta }u{(a+M_u-1)}^{\mathrm{T}}{]}^{\mathrm{T}}\\ {\stackrel{⌢}{Y}}_r(a)=[y_r{(a+1)}^{\mathrm{T}} y_r{(a+2)}^T\cdots y_r{(a+M_y)}^{\mathrm{T}}{]}^T\end{array}$$

(20)

where $\mathsf{\Delta }u(a)=u(a)-u(a-1)$. The optimization objective function design for MPC as below:

$$\{\begin{array}{l}\underset{\stackrel{⌢}{U}\left(a\right)}{\min }J={\Vert \stackrel{⌢}{Y}(a)-{\stackrel{⌢}{Y}}_r(a)\Vert }_{A_1}^2+{\Vert \stackrel{⌢}{U}(a)\Vert }_{B_1}^2+{\Vert \mathsf{\Delta }\stackrel{⌢}{U}(a)\Vert }_{B_2}^2\\ s.t.Y_{\min }\le \stackrel{⌢}{Y}(a)\le Y_{\max },U_{\min }\le \stackrel{⌢}{U}(a)\le U_{\max },\mathsf{\Delta }{U}_{\min }\le \mathsf{\Delta }\stackrel{⌢}{U}(a)\le \mathsf{\Delta }{U}_{\max }\end{array}$$

(21)

where ${\Vert X\Vert }_{\mathsf{\Delta }}^2=X^T\Delta X$; $A_1$, $B_1$, and $B_2$ represent weight coefficient matrices. Substitute Equations (19) and (20) into Equation (21) to eliminate the constant term and convert it into the following QP problem:

$$\{\begin{array}{l}\underset{\stackrel{⌢}{U}\left(a\right)}{\min }\stackrel{⌢}{J}=\frac{1}{2}\stackrel{⌢}{U}(a{)}^T\left[R_a^{\mathrm{T}}{A}_1{R}_a+B_1+N^{-\mathrm{T}}{B}_2{N}^{-1}\right]\stackrel{⌢}{U}(a)\\ \hspace{1em}\hspace{1em}\hspace{1em}+\left[Y_0{(a)}^{\mathrm{T}}{A}_1{R}_a-Y_r{(a)}^{\mathrm{T}}{A}_1{R}_a-U_0{(a-1)}^{\mathrm{T}}{N}^{-\mathrm{T}}{B}_2{N}^{-1}\right]\stackrel{⌢}{U}(a)\\ s.t.\left[\begin{array}{c}{R}_a\\ -R_a\end{array}\right]\stackrel{⌢}{U}(a)\le \left[\begin{array}{c}{Y}_{\max }-Y_0(a)\\ -Y_{\min }+Y_0(a)\end{array}\right],U_{\min }\le \stackrel{⌢}{U}(a)\le U_{\max },\\ \hspace{1em}\hspace{1em}\hspace{1em}{U}_0(a-1)+N\mathsf{\Delta }{U}_{\min }\le \stackrel{⌢}{U}(a)\le U_0(a-1)+N\mathsf{\Delta }{U}_{\max }.\end{array}$$

(22)

where

$$\{\begin{array}{l}\stackrel{⌢}{U}(a)=U_0(a-1)+N\Delta \stackrel{⌢}{U}(a),\\ U_0(a-1)=[\begin{array}{cccc}U{(a-1)}^{\mathrm{T}}& U{(a-1)}^T& \cdots & U{(a-1)}^{\mathrm{T}}\end{array}{]}^{\mathrm{T}}\\ N=\left[\begin{array}{ccccc}I& 0& 0& \cdots & 0\\ I& I& 0& \cdots & 0\\ I& I& I& \ddots & ⋮\\ ⋮& ⋮& \ddots & \ddots & 0\\ I& I& I& \cdots & I\end{array}\right]\end{array}$$

(23)

Equation (22) represents a convex QP problem with constraints. If the feasible set defined by the constraint condition is not empty and the objective function has a lower bound within the feasible set, then the QP problem has a globally optimal solution. If Equation (22) has no feasible solution in a certain control period, the feasible solution obtained in the previous period is used for control in the practical control. In addition, in practical control, in order to avoid control sequence deviations caused by environmental or system inherent biases, the optimal control sequence’s initial value is employed as the control input, and the system output is observed at the next moment for feedback correction, thus realizing online rolling optimization.

5. Control Experiments

This part uses the water tank system shown in Figure 6 as the experimental object, which is a commonly used process control device in industrial production. The deep learning models are established using the data-driven modeling approach, and the predictive controllers are designed based on the model’s distinctive structure for real-time control comparative experiments on water tank systems.

Figure 6. Water tank system [21].

5.1. Water Tank System

The water tank system is a representative dual-input, dual-output process control piece of experimental equipment with strong coupling, large time delay, and strong nonlinearity. The water inflows of water tanks 1 and 2 are controlled by electric control valves EV1 and EV2, the water outflows are regulated by proportional valves V1 and V2, and the water level heights are detected by the liquid level sensors LV1 and LV2. Control experiments were performed on the water tank system, and models (6)–(8) were designed and are shown below:

$$\{\begin{array}{l}Y(a)={\displaystyle \sum _{i=1}^{s_y}{G}_{i,a-1}Y(a-i)}+{\displaystyle \sum _{j=s_d}^{s_u+s_d-1}{H}_{j,a-1}U(a-j)}+{\rho }_{0,a-1}+\theta (a)\\ G_{i,a-1}=\left[\begin{array}{l}{\gamma }_{i,a-1}^{11}{\gamma }_{i,a-1}^{12}\\ {\gamma }_{i,a-1}^{21}{\gamma }_{i,a-1}^{22}\end{array}\right],H_{j,a-1}=\left[\begin{array}{l}{\lambda }_{j,a-1}^{11}{\lambda }_{j,a-1}^{12}\\ {\lambda }_{j,a-1}^{21}{\lambda }_{j,a-1}^{22}\end{array}\right],{\rho }_{0,a-1}=\left[\begin{array}{l}{\rho }_{0,a-1}^1\\ {\rho }_{0,a-1}^2\end{array}\right]\end{array}$$

(24)

where $Y(a)={[y_1(a),y_2(a)]}^{\mathrm{T}}$ represents the liquid level heights of tank 1 and tank 2; $U(a)={[u_1(a),u_2(a)]}^{\mathrm{T}}$ represents the openings (0%~100%) of electric control valves EV1 and EV2; $\left\{G_{i,a-1}\right|i=1,\dots ,s_y\}$, $\left\{H_{j,a-1}\right|j=s_d,\dots ,s_u+s_d-1\}$, and ${\rho }_{0,a-1}$ are the weight coefficients, which can be obtained by models (6)–(8); $\theta (a)$ indicates the white noise signal. Among them, we have set the working point $\epsilon (a-1)={[Y{(a-1)}^{\mathrm{T}},\cdots ,Y{(a-d)}^{\mathrm{T}}]}^{\mathrm{T}}$ because the change in water level is an important factor leading to the strongly nonlinear dynamic characteristics of the water tank system.

5.2. Estimation of Model

In order to make the collected modeling data contain the various nonlinear dynamic characteristics of the water tank system to the maximum extent, this article uses the expected signal, including sine and step, to control the large fluctuation of the water tank level in the normal working range via the PID controller to obtain effective input/output sampling data. The time is 2 s. Figure 7 presents the structure of the PID controller used, which is mainly used to generate the system identification data. The PID controller output is limited between 0 and 100 (%).

Figure 7. The structure of MPC and PID control systems.

The sampling data given in Figure 8 are segregated into different proportions for model training and testing. The best proportion of the final modeling results is that the first 3500 data points (sampling time 2 s) are utilized for estimating the model, while the subsequent 1000 data points are utilized for verifying the model.

Figure 8. Identification data of water tank system.

The parameter optimization algorithms of the deep learning-based SD-ARX models are all the sampling adaptive motion estimation (Adam) algorithms, which integrate the advantages of momentum optimization and adaptive learning rate with high computational efficiency and can accelerate parameter convergence. The Tanh function is the full connecting layer activation function for all models, which gives the models a faster convergence rate and a stronger nonlinear expression ability. The hidden layer activation functions ${\sigma }_4(•)$ and ${\sigma }_5(•)$ of the LSTM are the Hard Sigmaid function and Tanh function, respectively. Compared with the standard Sigmoid function, the application of the Hard Sigmaid function in the neural network is more stable. It can reduce the gradient vanishing problem and has a faster calculation speed. The CNN employs the Relu as its convolutional activation function, which assists in reducing redundant calculation and parameter quantity between neurons, thus enhancing the model’s efficiency and generalization performance. The CNN pool function selects the average pool, which is beneficial for reducing the model’s complexity, minimizing the overfitting risk, and improving the feature’s robustness. Its convolutional and pooling layers have kernel sizes of 3 and 4. The MSE of different model structures is calculated and compared. When the model complexity meets the requirements, select the model structure and parameters with the lowest MSE for real-time control experiments. In addition, the search range for hyperparameters of all deep learning models includes the learning rate between 0.0001 and 0.1, the batch size between 4 and 128, and the epochs between 100 and 600. The hyperparameters finally selected include the Learning rate, batch size, and epochs of 0.001, 16, and 400, respectively. The parameter settings of the LSTM-CNN-ARX model proposed in this article are shown in Table 1.

Table 1. Parameter setting of the proposed model.

Proposed Model	Configuration
LSTM-CNN-ARX	LSTM	Units 1	Units = 8	Epochs = 400, Batch size = 16; Optimizer = ‘Adam’; Learning rate = 0.001.
		Units 2	Units = 8
		Units 3	Units = 8
	CNN	Convolution 1	Filter = 16; Stride = 1; Kernel size = 3
		Convolution 2	Filter = 16; Stride = 1; Kernel size = 3
		Convolution 3	Filter = 16; Stride = 1; Kernel size = 3
		Average-pooling	Stride = 1; Kernel size = 4

The model training in this study was implemented using the Keras [55] framework in the Jupyter Notebook 5.5.0 software development environment on a PC (Intel i9-11900k 3.5 GHz, 16 GB RAM). The supervisory control software used for the real-time control experiments was Kingview 6.55. Data communication between Matlab/Simulink 2022b software and Kingview 6.55 was achieved via the Dynamic Data Exchange (DDE) protocol, enabling real-time control of the water tank system. Taking the LSTM-CNN-ARX model as an example, Figure 9 and Figure 10 show that the model’s training and testing residuals are very small and the models have high modeling accuracy. Table 2 illustrates that different models have varying structures because they have distinct characteristics and nonlinear description capabilities. At the same time, the model with more parameters and deeper network layers has stronger nonlinear expression ability, but this also increases the computational burden, resulting in a longer training time. The findings suggest that the ARX model exhibits the maximum MSE and cannot capture the nonlinear system’s dynamic behavior well. The deep learning-based ARX model has a lower MSE than the RBF-ARX model because the multi-layer deep learning network has stronger nonlinear expression capabilities than the single-layer RBF neural network. The LSTM-ARX model has less MSE than the CNN-ARX model because LSTM can maintain long-term dependencies on sequence data. In addition, because the LSTM-CNN-ARX model incorporates the strengths of LSTM for learning temporal characteristics and CNN for capturing spatial characteristics, its MSE is smaller than the LSTM-ARX and CNN-ARX models, which can accurately capture the water tank system’s nonlinear behavior.

Figure 9. LSTM-CNN-ARX model’s training result.

Figure 10. LSTM-CNN-ARX model’s testing result.

Table 2. Comparison of modeling results.

$\mathbf{Models} \left({\mathit{s}}_{\mathit{y}},{\mathit{s}}_{\mathit{u}},{\mathit{s}}_{\mathit{d}},\mathit{d},\mathit{n},{\mathit{n}}_1\right)$	Number of Nodes in Each Layer	Number of Parameters	Training Time	MSE of Training Data		MSE of Testing Data
				${\mathit{y}}_1(\mathit{a})$	${\mathit{y}}_2\left(\mathit{a}\right)$	${\mathit{y}}_1(\mathit{a})$	${\mathit{y}}_2\left(\mathit{a}\right)$
ARX (18,18,6,2,/,/) [19]	/	146	18 s	0.4910	0.3526	0.4176	0.4101
RBF-ARX (23,20,6,2,/,/) [19]	2	562	169 s	0.4525	0.3158	0.3804	0.3712
CNN-ARX (19,22,6,2,0,3)	8,16,8	3142	205 s	0.4277	0.2925	0.3562	0.3448
LSTM-ARX (18,20,6,2,3,0)	16,32,16	23,738	1152 s	0.4013	0.2634	0.3353	0.3163
LSTM-CNN-ARX (22,21,6,2,3,3)	8,8,8,16,16,16	9710	1160 s	0.3732	0.2437	0.3076	0.2866

5.3. Real-Time Control Experiments

This study was conducted in a laboratory environment using the water tank system experimental equipment shown in Figure 1, which uses the deep learning ARX model-based MPC method to compare with PID, ARX-MPC, and RBF-ARX-MPC methods, demonstrating the developed method’s superiority. The PID parameters were tuned for good performance at the given state of the water tank system. Figure 7 shows the control system’s structure, where the PID controller’s parameters are $K_{\mathrm{P}1}=18, K_{\mathrm{I}1}=1.2, K_{\mathrm{D}1}=0.2$ and $K_{\mathrm{P}2}=20, K_{\mathrm{I}2}=1.1, K_{\mathrm{D}2}=0.2$. Because the water tank system operates in different liquid levels, it exhibits varying dynamic behaviors. Thus, control experiments are performed in different water level regions, i.e., low, medium, and high water level zones. The parameter setting of the objective optimization function (22) for the predictive control method is $B_1=\mathrm{diag}\left[0.0001, 0.0001\right]$, $B_2=\mathrm{diag}\left[0.80, 0.80\right]$, $\mathrm{A}1=\mathrm{diag}\left[1, 1\right]$. The control experimental results of the water tank system are shown in Figure 11, where $y_1(a)$ and $y_2\left(a\right)$ represent the liquid level heights, $u_1(a)$ and $u_2\left(a\right)$ represent the control valve opening (ranging from 0% to 100%), and $y_r(a)$ indicates that the expected output is a pink dashed line. In addition, Table 3, Table 4 and Table 5 display each algorithm’s control outcomes, encompassing overshoot (O), peak time (PT), and adjustment time (AT); $\Uparrow$ and $\Downarrow$ represent the rising and falling step response, respectively.

Figure 11. Low liquid level zone’s experimental results ($y_1$/$u_1$).

Table 3. Low liquid level zone’s control performance.

Control Strategy	${\mathit{y}}_1$			${\mathit{y}}_2$
	PT (s)	O (%)	AT (s)	PT (s)	O (%)	AT (s)
	⇓/⇑	⇓/⇑	⇓/⇑	⇓/⇑	⇓/⇑	⇓/⇑
PID	280/162	39.4/13.3	692/616	258/170	39.5/14.2	814/782
ARX-MPC	278/194	36.2/8.6	478/256	256/194	34.2/9.6	732/376
RBF-ARX-MPC	272/208	30.2/5.6	420/242	242/186	29.6/7.2	534/234
CNN-ARX-MPC	264/218	25.0/4.6	352/172	240/268	23.2/5.8	340/248
LSTM-ARX-MPC	262/248	19.6/3.8	348/186	236/238	19.0/4.0	326/184
LSTM-CNN-ARX-MPC	252/234	15.6/3.2	336/166	224/222	14.0/3.6	320/160

Table 4. Medium liquid level zone’s control performance.

Control Strategy	${\mathit{y}}_1$			${\mathit{y}}_2$
	PT (s)	O (%)	AT (s)	PT (s)	O (%)	AT (s)
	⇑/⇓	⇑/⇓	⇑/⇓	⇑/⇓	⇑/⇓	⇑/⇓
PID	332/486	5.1/14.7	342/606	404/480	5.0/14.3	414/604
ARX-MPC	336/484	2.4/12.4	268/566	448/476	2.5/13.1	330/558
RBF-ARX-MPC	334/474	1.8/11.0	280/552	436/466	1.8/12.4	334/548
CNN-ARX-MPC	374/456	1.7/10.6	276/528	440/466	1.7/11.8	332/546
LSTM-ARX-MPC	372/450	1.3/9.1	278/502	454/454	1.4/10.3	328/512
LSTM-CNN-ARX-MPC	366/444	1.1/7.5	274/490	430/448	1.1/8.5	326/502

Table 5. High liquid level zone’s control performance.

Control Strategy	${\mathit{y}}_1$			${\mathit{y}}_2$
	PT (s)	O (%)	AT (s)	PT (s)	O (%)	AT (s)
	⇑/⇓	⇑/⇓	⇑/⇓	⇑/⇓	⇑/⇓	⇑/⇓
PID	168/222	14.0/44.0	776/726	190/228	12.3/43.8	942/958
ARX-MPC	188/218	9.0/36.0	532/498	216/226	8.2/37.2	546/748
RBF-ARX-MPC	176/214	7.4/28.4	456/330	206/220	6.0/30.0	322/526
CNN-ARX-MPC	230/210	5.0/23.6	236/282	262/216	5.6/26.0	280/294
LSTM-ARX-MPC	258/214	4.0/16.8	184/272	272/208	4.2/18.0	206/278
LSTM-CNN-ARX-MPC	216/196	3.2/13.8	162/260	266/202	3.6/12.6	164/272

5.3.1. Low Liquid Level Zone Control Experiments

The control outcomes of distinct algorithms in a low liquid level zone (50–100 mm) are presented in Figure 11 and Figure 12 and Table 3, and the LSTM-CNN-ARX-MPC approach shows significantly superior control performance compared to other algorithms. In the falling step response of LSTM-CNN-ARX-MPC, the PTs of liquid levels $y_1(a)$ and $y_2\left(a\right)$ are 252 s and 224 s, the Os are 15.6% and 14.0%, and the ATs are 336 s and 320 s; in the rising step response, the Os are 3.2% and 3.6%, and the ATs are 166 s and 160 s, respectively, which are superior to other methods. This is due to the LSTM-CNN-ARX model incorporating the strengths of LSTM for learning temporal characteristics and CNN for capturing spatial characteristics, which can efficiently capture the water tank system’s nonlinear characteristics and achieve outstanding control performance. It should be noted that the PT of LSTM-CNN-ARX-MPC may not always outperform other methods. However, the evaluation of control performance is not just about looking at PT; we also need to see other indicators, such as O and AT. In fact, the O and AT of LSTM-CNN-ARX-MPC are much smaller than those of other methods, so overall, LSTM-CNN-ARX-MPC outperforms other methods. Usually, for controllers, their smaller PT may result in a larger O and AT, and there may also be significant oscillations at the beginning of the control process.

Figure 12. Low liquid level zone’s experimental results ($y_2$/$u_2$).

5.3.2. Medium Liquid Level Zone Control Experiments

Figure 13 and Figure 14 and Table 4 present the experimental results of various controllers in a medium liquid level zone (100–250 mm), indicating that the PID’s control performance is not as good as other methods; it has the maximum overshoot and AT. The control effectiveness of RBF-ARX-MPC outperforms ARX-MPC, exhibiting lower O and PT. However, RBF-ARX-MPC demonstrates inferior O and AT compared to the three deep learning-based MPC algorithms. Multi-layer deep learning networks have stronger nonlinear expression and generalization capabilities than single-layer RBF neural networks and can adaptively learn the water tank system’s nonlinear features. Therefore, the predictive control effect and modeling accuracy based on the deep learning ARX models are better than those of the RBF-ARX models. Additionally, LSTM-ARX-MPC’s comprehensive control effect is marginally superior to CNN-ARX-MPC because LSTM belongs to a recurrent neural network, which has storage function and advantages in time series data modeling, so it can maintain long-term correlation in sequence data. LSTM-CNN-ARX-MPC outperforms all other algorithms in the falling step response, exhibiting the lowest PT, AT, and O. Additionally, it has the smallest O in the rising step response, which enables the water tank system’s liquid level to effectively follow the reference trajectory.

Figure 13. Medium liquid level zone’s experimental results ($y_1$/$u_1$).

Figure 14. Medium liquid level zone’s experimental results ($y_2$/$u_2$).

5.3.3. High Liquid Level Zone Control Experiments

Figure 15 and Figure 16 and Table 5 present the experimental results in a high liquid level zone (250–300 mm). It is apparent that the PID controller exhibits the poorest control performance with the largest O and AT. ARX-MPC’s control performance is also poor because the ARX model has limited capability to capture nonlinear features. LSTM-ARX-MPC demonstrates superior overall control performance compared to ARX-MPC, RBF-ARX-MPC, and CNN-ARX-MPC due to the advantage of LSTM in maintaining long-term dependence on sequence data in time series modeling. LSTM-CNN-ARX-MPC demonstrates optimal comprehensive control effectiveness, particularly exhibiting a considerably lower O in the falling step. This is attributable to the LSTM-CNN-ARX model’s powerful spatiotemporal feature capture capability, which enable it to effectively capture the water tank system’s nonlinear properties.

Figure 15. High liquid level zone’s experimental results ($y_1$/$u_1$).

Figure 16. High liquid level zone’s experimental results ($y_2$/$u_2$).

In conclusion, PID and ARX-MPC exhibit the poorest control performance, failing to effectively capture the nonlinear characteristics of the water tank system, thereby limiting their applicability. Additionally, the overall control effectiveness of RBF-ARX-MPC is not as good as deep learning-based MPC. This is because RBF is a single-layer network with lesser nonlinear descriptive capabilities compared to multi-layer deep learning networks. Furthermore, LSTM-ARX-MPC’s superior control performance compared to CNN-ARX-MPC can be attributed to LSTM’s memory function, which preserves the long-term dependence on sequential data. In almost all cases, LSTM-CNN-ARX-MPC demonstrates the best control effectiveness, especially in the low and high water level regions with strong nonlinearity. This is attributed to the powerful spatiotemporal feature learning capability of the LSTM-CNN-ARX model, which enable it to capture all the nonlinear features of the water tank system.

6. Conclusions

This study aims to solve the modeling and predictive control problems of a category of smooth nonlinear systems. The studied system’s operating point may change over time, and its dynamic characteristics can be locally linearized. We utilized the spatiotemporal feature extraction capabilities of the LSTM and CNN, as well as the SD-ARX model’s pseudo-linear ARX structural feature, to establish the LSTM-CNN-ARX model to describe the studied system’s nonlinear properties. The developed modeling and MPC algorithms were validated in real-time control rather than digital simulation on an actual multivariable water tank system, confirming their effectiveness. The proposed methods can be useful and can generate better results for a category of smooth nonlinear plants than some well-established, simpler, but proven efficiency approaches.

This article used the deep learning model to fit the state-dependent ARX model’s autoregressive coefficients to obtain the LSTM-ARX model and the LSTM-CNN-ARX model that accurately describe the system’s nonlinear properties. These models have local linear and global nonlinear characteristics, which decompose the model’s complexity into the autoregressive part, making it easier to design the MPC controllers. Control comparative experiments demonstrate that compared with the PID, ARX-MPC, and RBF-ARX-MPC algorithms, the three deep learning-based MPC algorithms can achieve more precise trajectory tracking control of the water tank system’s liquid level. This is because of the stronger nonlinear expression and generalization ability of deep learning networks, which can adaptively learn system features and model parameters and better represent the nonlinear behavior of a water tank system. Furthermore, the comprehensive control performance of LSTM-CNN-ARX-MPC is superior to LSTM-ARX-MPC and CNN-ARX-MPC due to its incorporation of the strengths of LSTM for learning temporal characteristics and CNN for capturing spatial characteristics, which enables it to capture the multi-dimensional spatiotemporal nonlinear features of water tank systems more effectively.

While LSTM neural networks may effectively address the gradient vanishing or exploding problem in traditional RNN modeling and better handle long sequence data, the complex structure of the LSTM model with numerous unidentified parameters results in significant computational demands during model training, convergence difficulties, and the risk of overfitting. Consequently, future studies will concentrate on improving the model structure by replacing the LSTM structure with gated recurrent units or related improved structures to enhance modeling efficiency and expand the model’s applicability in scenarios with limited computational resources.