Virtual Sensing of Key Variables in the Hydrogen Production Process: A Comparative Study of Data-Driven Models

Abstract

Hydrogen is an ideal energy carrier manufactured mainly by the natural gas steam reforming hydrogen production process. The concentrations of ${\mathrm{CH}}_4$, $\mathrm{CO}$, ${\mathrm{CO}}_2$, and ${\mathrm{H}}_2$ in this process are key variables related to product quality, which thus need to be controlled accurately in real-time. However, conventional measurement methods for these concentrations suffer from significant delays or huge acquisition and upkeep costs. Virtual sensors effectively compensate for these shortcomings. Unfortunately, previously developed virtual sensors have not fully considered the complex characteristics of the hydrogen production process. Therefore, a virtual sensor model, called “moving window-based dynamic variational Bayesian principal component analysis (MW-DVBPCA)” is developed for key gas concentration estimation. The MW-DVBPCA considers complicated characteristics of the hydrogen production process, involving dynamics, time variations, and transportation delays. Specifically, the dynamics are modeled by the finite impulse response paradigm, the transportation delays are automatically determined using the differential evolution algorithm, and the time variations are captured by the moving window method. Moreover, a comparative study of data-driven virtual sensors is carried out, which is sporadically discussed in the literature. Meanwhile, the performance of the developed MW-DVBPCA is verified by the real-life natural gas steam reforming hydrogen production process.

1. Introduction

Hydrogen is an ideal energy storage medium with zero carbon emissions, essential in energy systems’ low-carbon transformation [1]. Currently, over 90% of the world’s large-scale industrial hydrogen production process is the fossil energy reforming process, of which more than 50% is the natural gas steam reforming process [2,3]. The natural gas steam reforming hydrogen production process consists of feedstock purification, steam reforming, medium temperature conversion, and pressure swing adsorption. In this process, ${\mathrm{CH}}_4$, $\mathrm{CO}$, ${\mathrm{CO}}_2$, and ${\mathrm{H}}_2$ are primary process gases, and the concentrations of these gases are key variables (KVs) related to the product quality. Infrequent and inaccurate measurements of these KVs risk substantial production losses, poor control performance, and even hidden dangers to safety. Consequently, real-time measurements of these KVs are essential and desirable.

Currently, measurements of the hydrogen production process’ KVs are classified into offline laboratory analyses, hardware sensors, and virtual sensors. The offline laboratory analyses give accurate measurements of the KVs, but result in significant measurement delays [4]. The hardware sensors measure the KVs in real-time, but need colossal investment and maintenance costs [5,6]. Virtual sensors are actually predictive mathematical models which use explanatory variables (EVs, i.e., easily measurable variables like pressure and flow rate) as inputs and estimates of the KVs as outputs, having the benefits of no measurement delays and low costs [7,8]. Therefore, the virtual sensor effectively compensates for the shortcomings of the offline laboratory analysis and hardware sensor [9,10]. As a result, virtual sensors have been intensively studied and widely used in the hydrogen production process.

Virtual sensors can generally be grouped into first-principle-based and data-driven virtual sensors. The first-principle-based virtual sensor is established by analyzing and explaining the physicochemical mechanisms of the hydrogen production process. For example, Yang et al. established a predictive model of ammonia pyrolysis decomposition rate through theoretical derivation for the reaction in the hydrogen production process [11]. Huang et al. developed a mathematical model between the EVs and the conversion rate of methane by coupling conduction, convection, thermal radiation, and chemical reaction kinetics [12]. Zhou et al. simulated the chemical looping hydrogen production process by analyzing the process mechanisms [13]. However, due to the complex reaction dynamics, precise first-principle-based models of the hydrogen production process are challenging to obtain. Usually, laboratory-scale studies or ideal steady-state analyses are the main applications of these first-principle-based virtual sensors. On the contrary, a data-driven virtual sensor is constructed by process-measured data, and does not rely on accurate mechanisms [14,15]. Therefore, it is closer to reality, and better describes the actual operations of the hydrogen production process [16,17]. As a result, data-driven virtual sensors are widely applied in the hydrogen production process. For example, Tong et al. built the multi-layer perceptron (MLP) neural network to estimate the ${\mathrm{CH}}_4$ concentration in the hydrogen production process [18]. Zamaniyan et al. employed the MLP model to predict the concentration of ${\mathrm{H}}_2$ and $\mathrm{CO}$ of the hydrogen production process [19]. Ögren et al. established deep neural networks (DNNs) to estimate the concentration of ${\mathrm{CH}}_4$, $\mathrm{CO}$, ${\mathrm{CO}}_2$, and ${\mathrm{H}}_2$, which have large network layers [20]. Considering the challenges of artificial neural network models in model selection and parameter adjustment with limited data, small-data-oriented models have been applied to the hydrogen production process. Fang et al. used a support vector machine (SVM) to predict the hydrogen yield [21]. Zhao et al. proposed a predictive model based on the least squares SVM (LSSVM) employed in wind power hydrogen production [22].

Despite these achievements on virtual sensors for the KVs, the hydrogen production process is usually complicated because of practical operations. Firstly, due to the feedstock variations and external disturbances, the hydrogen production process shows strong dynamics [23]. Secondly, the hydrogen production process involves many series-wound devices (e.g., pre-reformers and pre-heaters), which requires significant time to deliver the energies and feedstocks to the reformer [24]. That means the EVs and the KVs are recorded simultaneously and are mismatched (i.e., there are transportation delays between the EVs and the KVs). Thirdly, the hydrogen production process exhibits time-varying properties because of the process drifts caused by mechanical abrasions, catalyst deactivation, or even climatic variations [25].

The above complex properties of the hydrogen production process make it challenging to develop high-precision virtual sensors for the KVs. As far as we know, the existing virtual sensors developed for the hydrogen production process do not account for these complicated characteristics. Moreover, the hydrogen production process has multiple KVs to estimate, and the existing virtual sensors constructed multiple independent single-output virtual sensors (one for each KV), ignoring the correlations between the KVs. To this end, we first develop a multi-output virtual sensor model called “dynamic variational Bayesian principal component analysis (DVBPCA)” for real-time prediction of the KVs in the hydrogen production process. As a further contribution, a moving window-based DVBPCA (MW-DVBPCA) is developed to improve estimation performance while considering the time variations of the hydrogen production process. The main contributions we make in the article are organized as follows.

(1)

The developed DVBPCA considers both process dynamics and transportation delays of energies and materials. Concretely, the finite impulse response (FIR) method is employed to model the dynamics of the hydrogen production process. And the transportation delays related to the EVs are automatically determined by differential evolution (DE). Moreover, the DVBPCA is able to make full use of the correlations between KVs for performance enhancement.

(2)

The moving window (MW) approach is employed for updating the DVBPCA with the latest online process information, which effectively captures the time-varying characteristics of the hydrogen production process in real-time.

(3)

A comparative study of data-driven virtual sensors is implemented for the hydrogen production process, which is sparsely mentioned in the predecessors’ research. Furthermore, the performance of the developed MW-DVBPCA is verified by the real-life natural gas steam reforming hydrogen production process.

This article is structured as follows. The variational inference (VI) is introduced in Section 2. Section 3 describes the developed DVBPCA and MW-DVBPCA in detail. In Section 4, the performance of the DVBPCA and MW-DVBPCA is evaluated using the operational data from the distributed control system (DCS) database of a real-life hydrogen production process. Finally, Section 5 concludes the article.

2. Variational Inference

The VI and Monte Carlo–Markov Chain are common methods to study posterior distributions over RVs in probabilistic models [26]. And the VI was used to learn the developed DVBPCA, considering its advantages in convergence diagnosis and computational efficiency.

Denote the random variables (RVs) with unknown posterior distribution as $\mathbf{\Theta }$. The equation shown as Equation (1) decomposes the log of model evidence, as follows:

$$\begin{array}{cc}\hfill & lnp\left(\mathit{D}\right)=\mathcal{L}\left(q\right)+\mathrm{KL}\left(q\Vert p\right)\hfill \\ \hfill & \mathrm{KL}\left(q\Vert p\right)=\int q\left(\mathbf{\Theta }\right)ln\left\{\frac{q\left(\mathbf{\Theta }\right)}{p\left(\mathbf{\Theta }\mid \mathit{D}\right)}\right\}d\mathbf{\Theta }\hfill \\ \hfill & \mathcal{L}\left(q\right)=\int q\left(\mathbf{\Theta }\right)ln\left\{\frac{p\left(\mathit{D},\mathbf{\Theta }\right)}{q\left(\mathbf{\Theta }\right)}\right\}d\mathbf{\Theta }\hfill \end{array}$$

(1)

where $\mathrm{KL}\left(q\Vert p\right)$ means the Kullback–Leibler (KL) divergence between $q\left(\mathbf{\Theta }\right)$ and $p\left(\mathbf{\Theta }\mid \mathit{D}\right)$, $\mathit{D}$ is training dataset, $q\left(\mathbf{\Theta }\right)$ represents any form of probability distribution over $\mathbf{\Theta }$, $p\left(\mathbf{\Theta }\mid \mathit{D}\right)$ represents the actual posterior distribution over $\mathbf{\Theta }$, and $\mathcal{L}\left(q\right)$ means the variational evidence lower bound (ELBO) of the log of model evidence $lnP\left(\mathit{D}\right)$ as $\mathrm{KL}\left(q\Vert p\right)\ge 0$.

Therefore, the posterior distributions over $\mathbf{\Theta }$ can be found by maximizing the ELBO $\mathcal{L}\left(q\right)$ as shown in Equation (2), i.e.,

$$p(\mathbf{\Theta }\mid \mathit{D})=q^{*}(\mathbf{\Theta })=\underset{q(\mathbf{\Theta })}{argmax}\mathcal{L}\left(q(\mathbf{\Theta })\right)$$

(2)

where $q^{*}(\mathbf{\Theta })$ represents the variational solution of $q(\mathbf{\Theta })$.

Assume $q(\mathbf{\Theta })$ can factorize into the product ${\prod }_{j=1}^J{q}_j\left({\mathsf{\theta }}_j\right)$, where $\mathbf{\Theta }=\{{\mathsf{\theta }}_1,{\mathsf{\theta }}_2,\cdots ,{\mathsf{\theta }}_J\}$. The general rule to find the variational distribution $q_j\left({\mathsf{\theta }}_j\right)$ is obtained as Equation (3):

$$ln{q}_j^{*}\left({\mathsf{\theta }}_j\right)=\int lnp\left(\mathit{D},\mathbf{\Theta }\right)\prod _{i\ne j}{q}_i^{*}\left({\mathsf{\theta }}_i\right)d{\mathsf{\theta }}_i+\mathbb{C}={\mathbb{E}}_{q_i\left({\mathsf{\theta }}_{i\ne j}\right)}\left[lnp\left(\mathit{D},\mathbf{\Theta }\right)\right]+\mathbb{C}$$

(3)

where ${\mathbb{E}}_{q\left(\mathsf{\theta }\right)}\left[f\left(\mathsf{\theta }\right)\right]$ is the expectation of a function $f\left(\mathsf{\theta }\right)$ of the RV $\mathsf{\theta }$ with respect to the distribution $q\left(\mathsf{\theta }\right)$ over $\mathsf{\theta }$, ${\mathsf{\theta }}_{i\ne j}$ means factors in $\mathbf{\Theta }$ except ${\mathsf{\theta }}_j$, and $\mathbb{C}$ means constant terms.

3. Dynamic Variational Bayesian Principal Component Analysis Based on Moving Window

3.1. Time-Delayed Moving Average Model

Due to the flexibility of combining various regression models, the FIR paradigm is generally employed for capturing dynamics. Moreover, the KVs are measured at a lower frequency than the EVs in the hydrogen production process, which implies that the model structure based on autoregression from the FIR family is unsuitable. More importantly, as mentioned earlier, the transportation delays between the EVs and the KVs are unknown, and require consideration. Based on these considerations, a time-delayed moving average (MA) model structure is embedded in the virtual sensor, which is given by

$$\mathit{y}\left(n\right)=f\left[\begin{array}{c}{x}_1\left(n-l_1\right),x_1\left(n-l_1-1\right),\cdots ,x_1\left(n-l_1-p_1\right),\\ x_2\left(n-l_2\right),x_2\left(n-l_2-1\right),\cdots ,x_2\left(n-l_2-p_2\right),\\ ⋮\\ x_S\left(n-l_S\right),x_S\left(n-l_S-1\right),\cdots ,x_S\left(n-l_S-p_S\right)\end{array}\right]$$

(4)

where $\mathit{y}\left(n\right)\in {\mathbb{R}}^{K_y}$ are the n-th measurements of the $K_y$-dimensional KVs, $x_i\left(n\right)$ is n-th measurement of the i-th EV, S is the number of the EVs, $l_i$ and $p_i$ refer to the transportation delay and the order of $x_i$, respectively, and both are non-negative integers, and $f(·)$ means the mathematical relationship between the EVs and KVs (but there may exist strong colinearities between the EVs of the model $f(·)$). Denote $\mathit{l}=(l_1,l_2,\cdots ,l_S)$ and $\mathit{p}=(p_1,p_2,\cdots ,p_S)$ for conciseness.

3.2. Dynamic Variational Bayesian Principal Component Analysis

Variational Bayesian principal component analysis (VBPCA) is a widely used model to deal with the colinearities between the EVs, owing to its advantage in the automatic determination of appropriate model dimensionality [27]. Therefore, it is ideally suited as the model $f(·)$. In this article, name the VBPCA combined with the model structure shown in Equation (4) as the dynamic VBPCA (DVBPCA) due to the capability of capturing process dynamics. Moreover, the DVBPCA can handle the overfitting resulting from highly dimensional variable augmentation in Equation (4) through penalizing parameter values, which are detailed as follows.

Denote the observed sequential dataset as $\mathit{D}={({\mathit{d}}_1,{\mathit{d}}_2,\cdots ,{\mathit{d}}_N)}^{\mathrm{T}}$, where ${\mathit{d}}_n=\left(\begin{array}{c}{\mathit{x}}_n\\ {\mathit{y}}_n\end{array}\right)\in {\mathbb{R}}^K$ means the n-th set of samples, ${\mathit{x}}_n={({\mathit{v}}_1^{l_1,p_1}\left(n\right),{\mathit{v}}_2^{l_2,p_2}\left(n\right)\cdots ,{\mathit{v}}_{K_x}^{l_{K_x},p_{K_x}}\left(n\right))}^{\mathrm{T}}\in {\mathbb{R}}^{K_k}$ and ${\mathit{y}}_n=\mathit{y}\left(n\right)\in {\mathbb{R}}^{K_y}$ are inputs and outputs, respectively, and ${\mathit{v}}_i^{l_i,p_i}\left(n\right)=\left(x_i\left(n-l_i\right),x_i\left(n-l_i-1\right),\cdots ,x_i\left(n-l_i-p_i\right)\right)$. Figure 1 graphically represents the DVBPCA. where, ${\mathit{W}}_x\in {\mathbb{R}}^{K_x\times M}$ is the loading matrix of the inputs ${\mathit{x}}_n$, ${\mathit{W}}_y\in {\mathbb{R}}^{K_y\times M}$ is the loading matrix of the outputs ${\mathit{y}}_n$, $\mathit{W}=\left(\begin{array}{c}{\mathit{W}}_x\hfill \\ {\mathit{W}}_y\hfill \end{array}\right)$, ${\mathit{z}}_n\in {\mathbb{R}}^M$ are the M-dimensional hidden variables corresponding to the sample at time instant n, ${\mu }_x\in {\mathbb{R}}^{K_x}$ are the mean values of the inputs ${\mathit{x}}_n$, ${\mu }_y\in {\mathbb{R}}^{K_y}$ are the mean values of the outputs ${\mathit{y}}_n$, $\mu =\left(\begin{array}{c}{\mathsf{\mu }}_x\\ {\mathsf{\mu }}_y\end{array}\right)$, $\tau$ is the accuracy parameter of the noise, and $\mathit{\alpha }={({\alpha }_1,{\alpha }_2,\cdots ,{\alpha }_M)}^{\mathrm{T}}$ is the accuracy parameter set of the loading matrix. Moreover, denote each column of $\mathit{W}$ as ${\mathit{w}}_m$, the transpose of each row of $\mathit{W}$ as ${\mathit{w}}_k^{*}$, and $\mathit{Z}={\left({\mathit{z}}_1,{\mathit{z}}_2,\cdots ,{\mathit{z}}_N\right)}^{\mathrm{T}}$. As illustrated in Equation (5), the observed data point ${\mathit{d}}_n$ is assumed to be generated by

$${\mathit{d}}_n=\mathit{W}{\mathit{z}}_n+\mu +{\epsilon }_n$$

(5)

where ${\epsilon }_n$ is the measurement noise.

As shown in Equation (6), assign conjugate prior to the hidden variables

$$p\left({\mathit{z}}_n\right)=\mathcal{N}\left({\mathit{z}}_n\mid {\mathbf{0}}_M,{\mathbf{I}}_M\right)$$

(6)

where $\mathcal{N}(·)$ represents the normal distribution probability density function, ${\mathbf{0}}_M$ represents an M-dimensional column vector with element 0, and ${\mathbf{I}}_M$ represents an M-dimensional unit matrix.

According to Figure 1, which shows that the samples are independently collected, the conditional distributions of the observed samples are designed as shown in Equations (7); that is,

$$p(\mathit{D}\mid \mathit{Z},\mathit{W},\mu ,\tau )=\prod _{n=1}^N\mathcal{N}\left({\mathit{d}}_n\mid \mathit{W}{\mathit{z}}_n+\mu ,{\tau }^{-1}{\mathbf{I}}_K\right)$$

(7)

where ${\mathbf{I}}_K$ represents a K-dimensional unit matrix.

To conform with common sense and to simplify subsequent calculations, the distributions over all RVs are selected as conjugate priors as detailed in Equation (8)–(11); that is,

$$p(\mathit{W}\mid \alpha )=\prod _{m=1}^M\mathcal{N}\left({\mathit{w}}_m\mid {\mathbf{0}}_K,{\alpha }_m^{-1}{\mathbf{I}}_K\right)$$

(8)

$$p\left(\mu \right)=\mathcal{N}\left(\mu \mid {\mathbf{0}}_K,{\beta }^{-1}{\mathbf{I}}_K\right)$$

(9)

$$p\left(\alpha \right)=\prod _{m=1}^M\mathcal{G}\left({\alpha }_m\mid a_{\alpha },b_{\alpha }\right)$$

(10)

$$p\left(\tau \right)=\mathcal{G}\left(\tau \mid a_{\tau },b_{\tau }\right)$$

(11)

where ${\mathbf{0}}_K$ represents a K-dimensional column vector with element 0, $\beta$ is the accuracy parameter of the mean, $a_{\alpha }$, $b_{\alpha }$, $a_{\tau }$, $b_{\tau }$ are the hyper-parameters of the distributions over these RVs, and $\mathcal{G}(·)$ is the gamma distribution probability density function. Particularly, when little or no prior information is obtained for these priors, it is suggested to set noninformative priors for the RVs to minimize the effect on the posterior distribution. That is, the values of hyperparameters $a_{\alpha }$, $b_{\alpha }$, $a_{\tau }$, and $b_{\tau }$ are set as small as possible so that the posteriors do not rely on the priors, but only on the data.

Remark 1.

To facilitate model training, rewrite Equation (8) as $p(\mathit{W}\mid \alpha )={\prod }_{k=1}^K\mathcal{N}\left({\mathit{w}}_k^{*}\mid \mathbf{0},{\mathit{A}}^{-1}\right)$, where $\mathit{A}=diag\left\{{\alpha }_1,{\alpha }_2,\Vert ,{\alpha }_M\right\}$. Each inverse variance of the $w_m$ is controlled by the corresponding ${\alpha }_m$. Thus, if the posterior distribution of a particular ${\alpha }_m$ is focused on larger values, the corresponding $w_m$ will converge to be tiny, effectively deactivating this particular direction in the latent space. Therefore, the effective dimensionality of the potential space is determined automatically.

According to Figure 1, the equation shown as Equation (12) represents the joint probability density function between training data $\mathit{D}$ and the RVs $\mathbf{\Theta }$ as follows:

$$p(\mathit{D},\mathbf{\Theta })=p(\mathit{D}\mid \mathit{Z},\mathit{W},\mathsf{\mu },\tau )p\left(\mathit{Z}\right)p(\mathit{W}\mid \mathsf{\alpha })p\left(\mathsf{\mu }\right)p\left(\mathsf{\alpha }\right)p\left(\tau \right).$$

(12)

The VI is employed to train the DVBPCA to obtain the posterior distributions $q(\mathbf{\Theta })$ of the RVs $\mathbf{\Theta }=\{\mathit{Z},\mathit{W},\mathsf{\mu },\mathsf{\alpha },\tau \}$.

Assume the posterior distribution over $\mathbf{\Theta }$, $q(\mathbf{\Theta })$, can factorize into

$$q(\mathbf{\Theta })=q\left(\mathit{Z}\right)q\left(\mathit{W}\right)q\left(\mathsf{\mu }\right)q\left(\mathsf{\alpha }\right)q\left(\tau \right)$$

(13)

According to Equation (2), the posterior distribution $q^{*}\left({\mathit{z}}_n\right)$ over ${\mathit{z}}_n$ is detailed as

$$\begin{array}{cc}\hfill ln{q}^{*}\left(\mathit{Z}\right)& ={\mathbb{E}}_{\mathit{W},\mathsf{\mu },\mathsf{\alpha },\tau }\left[lnp\left(\mathit{D},\mathbf{\Theta }\right)\right]+\mathbb{C}\hfill \\ \hfill & ={\mathbb{E}}_{\mathit{W},\mathsf{\mu },\tau }[lnp(\mathit{D}\mid \mathit{Z},\mathit{W},\mathsf{\mu },\tau )]+\mathbb{E}\left[p\left(\mathit{Z}\right)\right]+\mathbb{C}\hfill \\ \hfill & =\sum _{n=1}^N{\mathbb{E}}_{\mathit{W},\mathsf{\mu },\tau }[-\frac{\tau }{2}{({\mathit{d}}_n-\mathit{W}{\mathit{z}}_n-\mathsf{\mu })}^{\mathrm{T}}({\mathit{d}}_n-\mathit{W}{\mathit{z}}_n-\mathsf{\mu })]-\sum _{n=1}^N\mathbb{E}\left[\frac{1}{2}{\mathit{z}}_n^{\mathrm{T}}{\mathit{z}}_n\right]+\mathbb{C}\hfill \\ \hfill & =\sum _{n=1}^N\{\langle \tau \rangle {({\mathit{d}}_n-\langle \mathsf{\mu }\rangle )}^{\mathrm{T}}\langle \mathit{W}\rangle {\mathit{z}}_n-\frac{1}{2}(\langle \tau \rangle \langle {\mathit{W}}^{\mathrm{T}}\mathit{W}\rangle +{\mathbf{I}}_M){\mathit{z}}_n\}+\mathbb{C}\hfill \\ & =\sum _{n=1}^{N}ln\mathcal{N}\left({\mathit{z}}_n\mid {\mathit{m}}_z^{\left(n\right)},{\Sigma }_z\right)\hfill \end{array}$$

(14)

where ${\mathit{m}}_z^{\left(n\right)}=\langle \tau \rangle {\Sigma }_z〈{\mathit{W}}^{\mathrm{T}}〉\left({\mathit{d}}_n-\langle \mathsf{\mu }\rangle \right)$, ${\Sigma }_z={\left(\langle \tau \rangle 〈{\mathit{W}}^{\mathrm{T}}\mathit{W}〉+{\mathbf{I}}_M\right)}^{-1}$, and $\langle ·\rangle$ is the expectation.

Similarly, by using the general rule given by Equation (2), we can find the variational posterior distributions over other RVs, as shown on the right side of Equation (13) one by one, which are given as follows.

The posterior distribution $q^{*}\left(\mathit{W}\right)$ over $\mathit{W}$ is obtained as

$$q^{*}\left(\mathit{W}\right)=\prod _{k=1}^K\mathcal{N}\left({\mathit{w}}_k^{*}\mid {\mathit{m}}_w^{\left(k\right)},{\Sigma }_w\right)$$

(15)

where ${\Sigma }_w={\left({\sum }_{n=1}^N\langle \tau \rangle 〈{\mathit{z}}_n{\mathit{z}}_n^{\mathrm{T}}〉+\mathit{A}\right)}^{-1}$, and ${\mathit{m}}_w^{\left(k\right)}={\sum }_{n=1}^N\langle \tau \rangle {\Sigma }_w〈{\mathit{z}}_n〉\left(d_{nk}-〈{\mathsf{\mu }}_k〉\right)$.

The posterior distribution $q^{*}\left(\mathsf{\mu }\right)$ over $\mathsf{\mu }$ is obtained as

$$q^{*}\left(\mathsf{\mu }\right)=\mathcal{N}\left(\mathsf{\mu }\mid {\mathit{m}}_{\mu },{\Sigma }_{\mu }\right)$$

(16)

where ${\Sigma }_{\mu }={(\langle \tau \rangle N+\beta )}^{-1}{\mathbf{I}}_K$, and ${\mathit{m}}_{\mu }={\sum }_{n=1}^N\langle \tau \rangle {\Sigma }_{\mu }\left({\mathit{d}}_n-\langle \mathit{W}\rangle 〈{\mathit{z}}_n〉\right)$.

The posterior distribution $q^{*}\left(\alpha \right)$ over $\alpha$ is obtained as

$$q^{*}\left(\mathsf{\alpha }\right)=\prod _{m=1}^M\mathcal{G}\left({\alpha }_m\mid c_{\alpha },d_{\alpha }\right)$$

(17)

where $c_{\alpha }=\frac{K}{2}+a_{\alpha }$, and $d_{\alpha }=b_{\alpha }+\frac{1}{2}〈{\mathit{w}}_m^{\mathrm{T}}{\mathit{w}}_m〉$.

The posterior distribution $q^{*}\left(\tau \right)$ over $\tau$ is obtained as

$$q^{*}\left(\tau \right)=\mathcal{G}\left(\tau \mid c_{\tau },d_{\tau }\right)$$

(18)

where $d_{\tau }=b_{\tau }+\frac{1}{2}{\sum }_{n=1}^N\left\{{\mathit{d}}_n^{\mathrm{T}}{\mathit{d}}_n-2{\mathit{d}}_n^{\mathrm{T}}\left(\langle \mathit{W}\rangle 〈{\mathit{z}}_n〉+\langle \mathsf{\mu }\rangle \right)+〈{\mathit{z}}_n^{\mathrm{T}}{\mathit{W}}^{\mathrm{T}}\mathit{W}{\mathit{z}}_n〉+2〈{\mathit{z}}_n^{\mathrm{T}}〉〈{\mathit{W}}^{\mathrm{T}}〉\langle \mathsf{\mu }\rangle +〈{\mathsf{\mu }}^{\mathrm{T}}\mathsf{\mu }〉\right\}$, and $c_{\tau }=\frac{NK}{2}+a_{\tau }$.

The ELBO $\mathcal{L}\left(q\right)$ is calculated using the updated posterior distributions, i.e.,

$$\begin{array}{cc}\hfill \mathcal{L}\left(q\right)& =\mathbb{E}[lnp(\mathit{D},\mathit{Z},\mathit{W},\mathsf{\mu },\mathsf{\alpha },\tau )]-\mathbb{E}[ln{q}^{*}(\mathit{Z},\mathit{W},\mathsf{\mu },\mathsf{\alpha },\tau )]\hfill \\ \hfill & =\mathbb{E}[lnp(\mathit{D}\mid \mathit{Z},\mathit{W},\mathsf{\mu },\tau \left)\right]+\mathbb{E}[lnp(\mathit{Z}\left)\right]+\mathbb{E}[lnp(\mathit{W}\mid \mathsf{\alpha }\left)\right]+\mathbb{E}[lnp(\mathsf{\mu }\left)\right]\hfill \\ \hfill & +\mathbb{E}[lnp\left(\mathsf{\alpha }\right)]+\mathbb{E}[lnp\left(\tau \right)]-\mathbb{E}[ln{q}^{*}\left(\mathit{Z}\right)]-\mathbb{E}[ln{q}^{*}\left(\mathit{W}\right)]-\mathbb{E}[ln{q}^{*}\left(\mathsf{\mu }\right)]\hfill \\ \hfill & -\mathbb{E}[ln{q}^{*}\left(\mathsf{\alpha }\right)]-\mathbb{E}[ln{q}^{*}\left(\tau \right)]\hfill \end{array}$$

(19)

The convergence of the training procedure can be diagnosed with ELBO. The termination criteria for the parameter learning process are defined as detailed in Equation (20), as follows:

$$|\frac{{\mathcal{L}}^{iter+1}-{\mathcal{L}}^{iter}}{{\mathcal{L}}^{iter+1}}|<\delta$$

(20)

where ${\mathcal{L}}^{iter+1}$ and $L^{iter}$ are the values of ELBO at the $(iter+1)$-th and $iter$-th iterative steps, respectively, and $\delta$ is the threshold.

The main steps in training the DVBPCA model are summarized in Algorithm 1.

Algorithm 1 Pseudocode for the DVBPCA.

1: Input: The observed dataset $\mathit{D}$, M, $\beta$, tol, maximum number of iterations $Ite{r}_{max}$, prior hyper-parameters $\left(a_{\alpha },b_{\alpha },a_{\tau },b_{\tau }\right)$.     2: Initialization: initialize ${\mathit{m}}_z$, ${\Sigma }_z$, ${\mathit{m}}_w$, ${\Sigma }_w$, ${\mathit{m}}_{\mu }$, ${\Sigma }_{\mu }$, $c_{\alpha }$, $d_{\alpha }$, $c_{\tau }$, $d_{\tau }$.     3: for iter=1 to $Ite{r}_{max}$ do   4:     VB-E-Step      5:    Calculate ${\mathit{m}}_z$, ${\Sigma }_z$ using Equation (14).     6:     VB-M-Step     7:     Calculate ${\mathit{m}}_w$, ${\Sigma }_w$ using Equation (15).     8:     Calculate ${\mathit{m}}_{\mathit{\mu }}$, ${\Sigma }_{\mu }$ using Equation (16).     9:     Calculate ${\mathit{c}}_{\alpha }$, ${\mathit{d}}_{\alpha }$ using Equation (17).   10:     Calculate ${\mathit{c}}_{\tau }$, ${\mathit{d}}_{\tau }$ using Equation (18).   11:     Calculate $\mathcal{L}\left(q\right)$ using Equation (19).   12:     if the termination condition is satisfied then 13:         Break   14:     end if  15: end for  16: Output: the posterior distributions of RVs $\mathbf{\Theta }$.

3.3. Moving Window-Based Dynamic Variational Bayesian Principal Component Analysis

The moving window (MW) method is used to trace time variations and reject disturbances [28]. It captures the latest process information by discarding the oldest sample after the latest sample is obtained and rebuilding a model with all data in the window. Specially, for the first local model, as mentioned above, set noninformation priors for the RVs. For subsequent local modeling, the prior distributions of the RVs in the MW-DVBPCA are updated one by one and replaced with the trained posterior distributions of the RVs at the previous moment. The MW-DVBPCA is detailed in this subsection.

Firstly, the training dataset (i.e., historical data) is preset in the MW, and the DVBPCA is trained on the training dataset to obtain the posterior distributions of the RVs $\mathbf{\Theta }$. When the test samples (i.e., online samples) are acquired, the DVBPCA generated earlier is used to estimate the KVs, and the posterior distributions of RVs are used as the prior for the new DVBPCA model, which is elaborated as follows.

Let ${\mathit{x}}_{n^{\prime }}$ and ${\mathit{y}}_{n^{\prime }}$ denote the observations of inputs and outputs at certain online time instant $n^{\prime }$, respectively, where ${\mathit{y}}_{n^{\prime }}$ is unknown. The hidden RVs ${\mathit{z}}_{n^{\prime }}$ are first introduced.

Since the outputs ${\mathit{y}}_{n^{\prime }}$ are unknown, the posterior distribution over ${\mathit{z}}_{n^{\prime }}$ is calculated with the observation of ${\mathit{x}}_{n^{\prime }}$ which, based on Equation (14), is given by the equation as shown in Equation (21) as follows:

$$q^{*}\left({\mathit{z}}_{n^{\prime }}\right)=\mathcal{N}\left({\mathit{z}}_n\mid {\mathit{m}}_z^{\left(n^{\prime }\right)},{\Sigma }_z^{n^{\prime }}\right)$$

(21)

where ${\Sigma }_z^{n^{\prime }}={\left(\langle \tau \rangle 〈{\mathit{W}}_x^{\mathrm{T}}{\mathit{W}}_x〉+{\mathbf{I}}_M\right)}^{-1}$, and ${\mathit{m}}_z^{\left(n^{\prime }\right)}=\langle \tau \rangle {\Sigma }_z^{n^{\prime }}〈{\mathit{W}}_x^{\mathrm{T}}〉\left({\mathit{x}}_{n^{\prime }}-\langle \mu \rangle \right)$.

Then, the conditional distribution of ${\mathit{y}}_{n^{\prime }}$ given ${\mathit{x}}_{n^{\prime }}$ is obtained as

$$\begin{array}{cc}\hfill & p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }}\right)=\int p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{z}}_{n^{\prime }},{\mathit{W}}_y,{\mathsf{\mu }}_y,\tau \right)p\left({\mathit{z}}_{n^{\prime }}\mid {\mathit{x}}_q\right)q^{*}\left({\mathit{W}}_y\right)q^{*}\left({\mathsf{\mu }}_y\right)q^{*}\left(\tau \right)d\mathit{z}d\mathit{W}d\mathsf{\mu }d\tau \hfill \\ \hfill & =\int \mathcal{N}\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{W}}_y^{\mathrm{T}}{\mathit{z}}_{n^{\prime }}+{\mathsf{\mu }}_y,{\tau }^{-1}{\mathbf{I}}_K\right)\mathcal{N}\left({\mathit{z}}_{n^{\prime }}\mid {\mathit{m}}_z^{\left(n^{\prime }\right)},{\Sigma }_z^{n^{\prime }}\right)\mathcal{N}\left({\mathit{W}}_y\mid {\mathit{m}}_w^y,{\Sigma }_w\right)\mathcal{N}\left({\mathsf{\mu }}_y\mid {\mathit{m}}_{\mu }^y,{\Sigma }_{\mu }^{yy}\right)\mathcal{G}\left(\tau \mid c_z,d_z\right)d\mathit{z}d\mathit{W}d\mathsf{\mu }d\tau \hfill \end{array}$$

(22)

where ${\mathit{m}}_w^y$ and ${\Sigma }_w$ are the mean and covariance calculated from the posterior distribution $q^{*}\left({\mathit{W}}_y\right)$, respectively, and ${\mathit{m}}_{\mu }^y$ and ${\Sigma }_{\mu }^{yy}$ are the mean values and covariance matrix calculated from the posterior distribution $q^{*}\left({\mathsf{\mu }}_y\right)$, respectively.

Equation (22) can be transformed into

$$p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }}\right)=\int \mathcal{N}\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{m}}_w^{y\mathrm{T}}{\mathit{m}}_z^{\left(n^{\prime }\right)}+{\mathit{m}}_{\mu }^y,tr\left[{\Sigma }_w{\Sigma }_z^{n^{\prime }}+{\Sigma }_w{\mathit{m}}_z^{\left(n^{\prime }\right)}{\mathit{m}}_z^{\left(n^{\prime }\right)\mathrm{T}}+{\mathit{m}}_w^y{\mathit{m}}_w^{y\mathrm{T}}{\Sigma }_z^{n^{\prime }}\right]+{\Sigma }_{\mu }^{yy}+{\tau }^{-1}{\mathbf{I}}_K\right)\mathcal{G}\left(\tau \mid c_{\tau },d_{\tau }\right)d\tau$$

(23)

where $tr[·]$ represents the trace.

The mean vector and covariance matrix of Gaussian distribution in Equation (23) are obtained as illustrated in Equation (24) and Equation (25), respectively, as follows:

$${\mathbb{E}}_{p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }},\tau \right)}\left[{\mathit{y}}_{n^{\prime }}\right]=\mathbb{E}[{\mathit{W}}_y^{\mathrm{T}}{\mathit{z}}_{n^{\prime }}+{\mathsf{\mu }}_y+{\mathsf{\epsilon }}_y]={\mathit{m}}_w^{y\mathrm{T}}{\mathit{m}}_z^{\left(n^{\prime }\right)}+{\mathit{m}}_{\mu }^y$$

(24)

$$\begin{array}{cc}\hfill {\mathbb{V}}_{p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }},\tau \right)}\left[{\mathit{y}}_{n^{\prime }}\right]& =\mathbb{V}[{\mathit{W}}_y^{\mathrm{T}}{\mathit{z}}_{n^{\prime }}+{\mathsf{\mu }}_y+{\mathsf{\epsilon }}_y]\hfill \\ \hfill & =\mathbb{V}\left[{\mathit{W}}_y^{\mathrm{T}}{\mathit{z}}_{n^{\prime }}\right]+\mathbb{V}\left[{\mathsf{\mu }}_y\right]+\mathbb{V}\left[{\mathsf{\epsilon }}_y\right]\hfill \\ \hfill & =\mathbb{E}\left[{\mathit{W}}_y^{\mathrm{T}}{\mathit{z}}_{n^{\prime }}{\mathit{z}}_{n^{\prime }}^{\mathrm{T}}{\mathit{W}}_y\right]-{\mathit{m}}_w^{y\mathrm{T}}{\mathit{m}}_z^{\left(n^{\prime }\right)}{\mathit{m}}_z^{\left(n^{\prime }\right)\mathrm{T}}{\mathit{m}}_w^y+{\Sigma }_{\mu }^{yy}+{\left(\tau \right)}^{-1}{\mathbf{I}}_K\hfill \\ \hfill & =tr(\mathbb{E}\left[{\mathit{W}}_y{\mathit{W}}_y^{\mathrm{T}}{\mathit{z}}_{n^{\prime }}{\mathit{z}}_{n^{\prime }}^{\mathrm{T}}\right]-{\mathit{m}}_w^y{\mathit{m}}_w^{y\mathrm{T}}{\mathit{m}}_z^{\left(n^{\prime }\right)}{\mathit{m}}_z^{\left(n^{\prime }\right)\mathrm{T}})+{\Sigma }_{\mu }^{yy}+{\left(\tau \right)}^{-1}{\mathbf{I}}_K\hfill \\ \hfill & =tr\left[{\Sigma }_w{\Sigma }_z^{n^{\prime }}+{\Sigma }_w{\mathit{m}}_z^{\left(n^{\prime }\right)}{\mathit{m}}_z^{\left(n^{\prime }\right)\mathrm{T}}+{\mathit{m}}_w^y{\mathit{m}}_w^{y\mathrm{T}}{\Sigma }_z^{n^{\prime }}\right]+{\Sigma }_{\mu }^{yy}+{\Sigma }_{\mu }^{yy}+{\left(\tau \right)}^{-1}{\mathbf{I}}_K\hfill \end{array}$$

(25)

where $p\left({\epsilon }_y\right)=\mathcal{N}({\epsilon }_y\mid 0,{\left(\tau \right)}^{-1}{\mathbf{I}}_K)$.

By replacing the distribution of $\tau$ with its expectation, the probability distribution of the predicted KVs is approximated as

$$p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }}\right)\approx \mathcal{N}\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{m}}_w^{y\mathrm{T}}{\mathit{m}}_z^{\left(n^{\prime }\right)}+{\mathit{m}}_{\mu }^y,tr\left[{\Sigma }_w{\Sigma }_z^{n^{\prime }}+{\Sigma }_w{\mathit{m}}_z^{\left(n^{\prime }\right)}{\mathit{m}}_z^{\left(n^{\prime }\right)\mathrm{T}}+{\mathit{m}}_w^y{\mathit{m}}_w^{y\mathrm{T}}{\Sigma }_z^{n^{\prime }}\right]+{\Sigma }_{\mu }^{yy}+{\left(\frac{c_{\tau }}{d_{\tau }}\right)}^{-1}{\mathbf{I}}_K\right).$$

(26)

Therefore, based on Equation (26), the estimations ${\widehat{\mathit{y}}}_{n^{\prime }}$ of ${\mathit{y}}_{n^{\prime }}$ are given by

$${\widehat{\mathit{y}}}_{n^{\prime }}={\mathbb{E}}_{p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }}\right)}\left[{\mathit{y}}_{n^{\prime }}\right]={\mathit{m}}_w^{y\mathrm{T}}{\mathit{m}}_z^{\left(n^{\prime }\right)}+{\mathit{m}}_{\mu }^y$$

(27)

and the prediction uncertainty is quantified by the variance matrix of ${\mathit{y}}_{n^{\prime }}$ denoted by ${\mathbb{V}}_{p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }}\right)}\left[{\mathit{y}}_{n^{\prime }}\right]$, which is given by Equation (28), i.e.,

$${\mathbb{V}}_{p\left({\mathit{y}}_{n^{\prime }}\mid {\mathit{x}}_{n^{\prime }}\right)}\left[{\mathit{y}}_{n^{\prime }}\right]=tr\left[{\Sigma }_w{\Sigma }_z^{n^{\prime }}+{\Sigma }_w{\mathit{m}}_z^{\left(n^{\prime }\right)}{\mathit{m}}_z^{\left(n^{\prime }\right)\mathrm{T}}+{\mathit{m}}_w^y{\mathit{m}}_w^{y\mathrm{T}}{\Sigma }_z^{n^{\prime }}\right]+{\Sigma }_{\mu }^{yy}+{\left(\frac{c_{\tau }}{d_{\tau }}\right)}^{-1}{\mathbf{I}}_K.$$

(28)

Once the KVs predictions for the new sample are completed, the historical data window is slid down to include the latest measured sample set $\{{\mathit{x}}_{n^{\prime }},{\mathit{y}}_{n^{\prime }}\}$ and to eliminate the oldest sample. The DVBPCA is then retrained according to the current window of the historical dataset for online prediction. This process repeats once new samples are obtained online.

3.4. Differential Evolutionary-Based Model Selection

The DE is an effective and simple intelligent optimization algorithm commonly used for solving NP-hard problems in real number fields [29]. Associated studies have demonstrated that DE has fast convergence performance as an excellent global optimization algorithm. Therefore, the DE is utilized for model selection (i.e., selecting the best parameter) of the DVBPCA and MW-DVBPCA, involving the dynamic orders $\mathit{p}$ and time delays $\mathit{l}$ given in Equation (4). Equation (29) shows the optimization problem, as follows:

$$\begin{array}{cc}\hfill & ming(\mathit{p},\mathit{l})\hfill \\ \hfill & s.t.{\mathit{p}}_{min}\le \mathit{p}\le {\mathit{p}}_{max}\hfill \\ \hfill & {\mathit{l}}_{min}\le \mathit{l}\le {\mathit{l}}_{max}\hfill \end{array}$$

(29)

where $g(·)$ is the fitness function, ${\mathit{p}}_{min}$ and ${\mathit{p}}_{max}$ are the lower and upper bounds of the dynamic orders $\mathit{p}$, respectively, and ${\mathit{l}}_{min}$ and ${\mathit{l}}_{max}$ are the lower and upper bounds of the delays $\mathit{l}$, respectively.

Based on Equation (4), $\{\mathit{p},\mathit{l}\}$ are non-negative integers. Therefore, the individuals generated by the DE need to be rounded. And the DE algorithm contains three core evolutionary operations, i.e., mutation, crossover and selection, which are introduced in detail as follows.

Initialization. Set the population size $NP$, dimension of individual $\left(2S\right)$, and the upper bounds $({\mathsf{\rho }}_1^{\left(max\right)},{\mathsf{\rho }}_2^{\left(max\right)},\cdots ,{\mathsf{\rho }}_{2S}^{\left(max\right)})$ and the lower bounds $({\mathsf{\rho }}_1^{\left(min\right)},{\mathsf{\rho }}_2^{\left(min\right)},\cdots ,{\mathsf{\rho }}_{2S}^{\left(min\right)})$ of the dynamic orders and time delays, where individuals $({\mathsf{\rho }}_1,{\mathsf{\rho }}_2,\cdots ,{\mathsf{\rho }}_{2S})$ in the population are randomly generated in integer field by Equation (30), i.e.,

$${\mathsf{\rho }}_e={\mathsf{\rho }}_e^{\left(min\right)}+({\mathsf{\rho }}_e^{\left(max\right)}-{\mathsf{\rho }}_e^{\left(min\right)})·rand(NP,1)$$

(30)

Mutation. In the r-th generation, DE generates a mutation vector ${\mathbf{v}}_e$ for the e-th individual, given by Equation (31); that is,

$${\mathbf{v}}_e^{r+1}={\mathsf{\rho }}_{e_0}^r+F·\left({\mathsf{\rho }}_{e_1}^r-{\mathsf{\rho }}_{e_2}^r\right)$$

(31)

where ${\mathsf{\rho }}_{e_0}^r$, ${\mathsf{\rho }}_{e_1}^r$, and ${\mathsf{\rho }}_{e_2}^r$ are individuals that randomly selected from population with $e_0\ne e_1\ne e_2\ne e$, and F is the mutation rate.

Crossover. A trial individual vector ${\mathit{u}}_e$ is generated by the original individual vector or the mutation vector based on the crossover rate $CR$ ranged in $(0,1)$, which is shown in Equation (32), i.e.,

$$u_{i,e}^{r+1}=\left\{\begin{array}{cc}{v}_{i,e}^{r+1},\hfill & \mathrm{if}rand(0,1)\le CR\mathrm{or}e={rand}_e\hfill \\ {\rho }_{i,e}^r,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(32)

where ${rand}_e$ is a random integer ranged in $(1,2S)$.

Selection. According to the fitness function $g(·)$, select the individual with a lower value as detailed in Equation (33) as follows:

$${\mathsf{\rho }}_e^{r+1}=\left\{\begin{array}{cc}{\mathit{u}}_e^{r+1},\hfill & \mathrm{if}g\left({\mathit{u}}_e^{r+1}\right)\le g\left({\mathsf{\rho }}_e^r\right)\hfill \\ {\mathsf{\rho }}_e^r,\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(33)

Termination. The evolution continues until it reaches the given maximum generation or the change in the fitness value is small. Otherwise, the above three operators should be performed repeatedly.

Figure 2 shows the flowchart of the DVBPCA and MW-DVBPCA combined with the DE for parameter optimization. Note that for the multi-output models, DVBPCA and MW-DVBPCA, the average prediction error of four KVs is set as the fitness function.

4. Case Studies and Comparisons

A comparative study of data-driven virtual sensors upon a real-life natural gas steam reforming hydrogen production process is provided in this section. Meanwhile, a case study is useful for understanding the hydrogen production processes and data features. The state-of-the-art virtual sensors and developed models (DVBPCA and MW-DVBPCA) are investigated, including partial least squares (PLS) [30], VBPCA, long short-term memory (LSTM) [31], echo state networks (ESN) [32], and dynamic PLS (DPLS) [33]. The PLS is a classic static model with the advantages of modeling data colinearities and simple data structure. The VBPCA is a static probabilistic model. Different from the PLS, the VBPCA determines the number of principal components automatically, which is also the basis of our developed models (DVBPCA and MW-DVBPCA). The LSTM and ESN are advanced dynamic neural network models that consider the dynamics. The DPLS is the improvement of the PLS based on the FIR paradigm, which also accounts for the dynamics of the hydrogen production process.

4.1. Natural Gas Steam Reforming Hydrogen Production Process

The natural gas steam reforming hydrogen production process is the largest industrial source of hydrogen, and it consists of four main processes: feedstock purification, steam reforming, medium temperature conversion, and pressure swing adsorption. Among these processes, steam reforming is the dominant reaction process, which is schematically shown in Figure 3 [34]. Processed gases are mixed with steam in the pre-reformer, where all hydrocarbons and some ${\mathrm{CH}}_4$ are converted to $\mathrm{CO}$, ${\mathrm{CO}}_2$ and ${\mathrm{H}}_2\mathrm{O}$. The temperature then decreases, which is not conducive to promoting hydrogen generation. The pre-reformer output is heated in a pre-heater and then continuously fed to the primary reformer for complete reforming.

The main chemical reactions of this process are shown as follows:

$$\begin{array}{cc}\hfill {\mathrm{C}}_{\mathrm{n}}{\mathrm{H}}_{2\mathrm{n}+2}+\mathrm{n}{\mathrm{H}}_2\mathrm{O}& \stackrel{\Delta }{⟷}\mathrm{n}\mathrm{CO}+(2\mathrm{n}+1){\mathrm{H}}_2\hfill \\ \hfill {\mathrm{CH}}_4+{\mathrm{H}}_2\mathrm{O}& \stackrel{\Delta }{⟷}\mathrm{CO}+3{\mathrm{H}}_2\hfill \\ \hfill \mathrm{CO}+{\mathrm{H}}_2\mathrm{O}& \stackrel{\Delta }{⟷}{\mathrm{CO}}_2+{\mathrm{H}}_2.\hfill \end{array}$$

(34)

According to Equation (34), the exit gases of the hydrogen production process consist of ${\mathrm{CH}}_4$, $\mathrm{CO}$, ${\mathrm{CO}}_2$, and ${\mathrm{H}}_2$, and the concentrations of these gases are KVs (as labeled “Y” in Figure 3) related to product quality. Therefore, these KVs need to be strictly monitored. In practice, offline laboratory analysis and hardware sensors are traditional methods for measuring the KVs, but have delays and high investment. Meanwhile, series-wound devices (such as pre-reformers and pre-heaters) introduce considerable transportation delays, which must be considered in virtual sensor modeling. Therefore, for giving real-time predictions of the KVs, a virtual sensor considering transportation delays is desirable.

4.2. Explanatory Variable Selection and Data Collection

A total of 13 EVs for the virtual sensor development are selected based on the first principles and the operation experience of the engineers. These EVs are easily measured variables, closely linked to the KVs and highly correlated with process variations, as shown by labels U1–U13 in Figure 3. The descriptions of these EVs are listed in

Table 1. EVs in the hydrogen production process.

EV	Description
U1	Flow of fuel natural gas into primary reformer
U2	Flow of fuel off gas into primary reformer
U3	Pressure of fuel off gas at the exit of heat exchanger 3
U4	Pressure of furnace flue gas at primary reformer’s exit
U5	Temperature of fuel off gas at the exit of heat exchanger 3
U6	Temperature of fuel natural gas at pre-heater’s exit
U7	Temperature of process gas at primary reformer’s entrance
U8	Temperature of furnace flue gas at primary reformer’s top left
U9	Temperature of furnace flue gas at primary reformer’s top right
U10	Temperature of mixed furnace flue gas at primary reformer’s top
U11	Temperature of transformed gas at primary reformer’s left exit
U12	Temperature of transformed gas at primary reformer’s right exit
U13	Temperature of transformed gas at primary reformer’s exit

.

The samples used to develop the virtual sensor of the KVs were obtained from the DCS database. The KVs’ sampling rate is 10 min, and 2500 samples were collected. These were divided into three parts in order of collection time: the training set, the validation set, and the testing set, which consist of 1500 samples, 300 samples, and 700 samples, respectively. Figure 4 and Figure 5 show sample division and the changing trend of these EVs and four KVs, respectively. In this article, we use scaled dimensionless data for modeling.

4.3. Evaluation Metrics

The root mean squares error (RMSE), the coefficient of determination (${\mathrm{R}}^2$), and the mean absolute error (MAE) are chosen to quantify different models’ performance, which are defined as

$$\begin{array}{cc}\hfill & \mathrm{RMSE}=\sqrt{\frac{1}{N^{\prime }}\sum _{n^{\prime }=1}^{N^{\prime }}{\left({\widehat{\mathit{y}}}_{n^{\prime }}-{\mathit{y}}_{n^{\prime }}\right)}^2}\hfill \\ \hfill & {\mathrm{R}}^2=1-\sum _{n^{\prime }=1}^{N^{\prime }}{\left({\widehat{\mathit{y}}}_{n^{\prime }}-{\mathit{y}}_{n^{\prime }}\right)}^2/\sum _{n^{\prime }=1}^{N^{\prime }}{\left({\mathit{y}}_{n^{\prime }}-\overline{\mathit{y}}\right)}^2\hfill \\ \hfill & \mathrm{MAE}=\frac{1}{N^{\prime }}\sum _{n^{\prime }=1}^{N^{\prime }}\left|{\widehat{\mathit{y}}}_{n^{\prime }}-{\mathit{y}}_{n^{\prime }}\right|\hfill \end{array}$$

(35)

where ${\widehat{\mathit{y}}}_{n^{\prime }}$ and ${\mathit{y}}_{n^{\prime }}$ are the estimated values and true values of the $n^{\prime }$th testing sample, respectively, $N^{\prime }$ is the size of the testing dataset, and $\overline{\mathit{y}}={\sum }_{n^{\prime }=1}^{N^{\prime }}{\mathit{y}}_{n^{\prime }}/N^{\prime }$ are the mean of the KVs in the testing dataset. The RMSE and MAE indexes characterize the average and largest prediction errors on the test set, respectively, and the ${\mathrm{R}}^2$ index characterizes the correlation between the predicted and true values. Therefore, the smaller the RMSE and MAE or the bigger the ${\mathrm{R}}^2$, the higher the predictive accuracy.

4.4. Parameter Selection

All models’ optimal parameters need to be chosen to obtain different models’ best prediction performance. Select the DE algorithm to minimize the average RMSE (i.e., the mean of the RMSE of the four KVs) on the validation set for parameter optimization of different models. For the PLS, the number of principal components and the time delays are the parameters to be optimized. For the DPLS, the number of principal components, the time delays and the dynamic orders are the parameters to be optimized. For the VBPCA-based models, the time delays and the dynamic orders are the parameters to be optimized. For the LSTM and ESN, the time delays are the parameters to be optimized.

For the VBPCA-based models, the key issue of automatic parameter determination of the principal component number is whether each column of the loading matrix $\mathit{W}$ is either insignificant or significant, i.e., ${\mathit{w}}_m=0$ or ${\mathit{w}}_m\ne 0$, which can be controlled by $\mathsf{\alpha }$ according to Equation (8). Take DVBPCA for example, the fact that the variance of each ${\mathit{w}}_m$ can be quantified by $1/\mathbb{E}\left({\alpha }_m\right)$ for $m=1,\Vert ,M$, which is displayed in Figure 6. The color distinction of the points in Figure 6 means the value of $1/\mathbb{E}\left({\alpha }_m\right)$ is either insignificant or significant by setting threshold, where red means significant and blue means insignificant. As shown in Figure 6, the appropriate dimensionality of the principal component subspace is selected as 39.

For the ESN, set the input regulation scale to 0.1, set the reservoir size to 50, and set the spectral radius to 0.8. Considering that the ESN involves the random weights in the reservoir computing step, 20 modeling tests were performed. For each trial, the random weights are saved and fixed, and the DE algorithm is used to optimize the model delays further. Then, the best results from 20 tests are picked for model performance comparison. Note that the ESN is a single-output model; we therefore construct four ESN models, one for each KV. For the LSTM, based on the debugging experience, set the time step to 1, set the learning rate to 0.1, set the hidden layer to 1, and set the neuron number in the hidden layer to 100, which performs favorably empirically in each of the replicated experiments of this work.

For the MW-DVBPCA, the impact of MW size is detailed in Figure 7. The MW size is in units of 10 min, i.e., setting the MW size to 50 means that the MW contains 500 min of data. As shown in Figure 7, for small MW sizes, the data in the window may not appropriately represent the relationship between process variables. In contrast, excessive MW size covers too much outdated sample data so that the MW-DVBPCA fails to track the process change adequately. Therefore, according to Figure 7, the MW size was selected as 50.

4.5. Results and Analysis

The estimations on the test set of the KVs obtained by the investigated seven virtual sensors are visualized in Figure 8, Figure 9 and Figure 10. In Figure 8, obviously, the PLS and VBPCA have the poor estimation performance. Because of the significant dynamics of the hydrogen production process, the estimation accuracy of the static models, PLS and VBPCA, is not as satisfactory as that of the other five dynamic models (such as around the 120-th sample of the ${\mathrm{CO}}_2$ concentration). Figure 9 shows that the estimated values of the DVBPCA tracks real values better than the other three models (such as around the 300-th sample of the $\mathrm{CO}$ concentration and around the 300-th sample of the ${\mathrm{CO}}_2$ concentration). That is because, on the one hand, the DVBPCA can deal with the colinearities between the EVs compared with the LSTM and ESN. On the other hand, the DVBPCA can tackle the overfitting results from high-order variable augmentation compared with the DPLS. Moreover, the ESN constructs four virtual sensors, one for each KV for estimation, but the DVBPCA is a multi-output model that considers the inherent relationships between the KVs. Figure 10 illustrate that the estimations of the four KVs by the MW-DVBPCA match the real values much better (particularly in the localized area of the ${\mathrm{CH}}_4$ concentration around the 440-th sample) than other models, revealing the importance of considering time variation properties in the virtual sensor modeling of the hydrogen production process. Moreover, although the overall predicted values do match well with the true ones when the proposed MW-DVBPCA method was used, some discrepancies can be observed between test sample number 200 and 300 for all the four KVs. The possible reasons for these differences in the predicted and true values are as follows. Firstly, the characteristics of the samples between test sample number 200 and 300 are changing rapidly, and model learning does not accommodate such changes in time. Secondly, the samples between test sample number 200 and 300 have nonlinearities, but the local model constructed by MW-DVBPCA is linear. Overall, it is recognized that the proposed models show noticeable advantages over the benchmark models.

Figure 11 compares the seven models in terms of scatter plots. Based on Figure 11, a further comparison of the prediction of the data-driven models can be made. As shown in Figure 11a, the predictions of the PLS for ${\mathrm{CH}}_4$ component deviate obviously from the real values. Figure 11b,c shows that the VBPCA model improves the prediction accuracy of $\mathrm{CO}$ and ${\mathrm{CO}}_2$ concentrations somewhat compared to the PLS model. However, the overall prediction accuracy is still very low. Figure 11d reveals that all models have relatively poor prediction accuracy for the ${\mathrm{H}}_2$ concentration, but the MW-DVBPCA presents a better result. As indicated in Figure 11, the scatters by MW-DVBPCA are more closely and clearly located around the diagonal line than those of other models, thus illustrating better performance.

The estimation performance of all data-driven virtual sensors is quantitatively tabulated in

Table 2. Quantitative estimation results of various models for the concentrations of ${\mathrm{CH}}_4$ and $\mathrm{CO}$.

		${\mathbf{CH}}_4$			$\mathbf{CO}$
		RMSE	${\mathbf{R}}^{\mathbf{2}}$	MAE	RMSE	${\mathbf{R}}^{\mathbf{2}}$	MAE
➀	PLS	0.1915	−0.1329	0.1643	0.1944	−0.0975	0.1483
	VBPCA	0.2061	−0.2566	0.1703	0.2054	−0.2259	0.1592
	LSTM	0.1567	0.2733	0.1373	0.1351	0.5370	0.1080
	ESN	0.1150	0.6088	0.0965	0.1148	0.6173	0.0968
	DPLS	0.0942	0.7373	0.0770	0.0956	0.7348	0.0764
	DVBPCA	0.0827	0.7975	0.0704	0.0826	0.8018	0.0662
	MW-DVBPCA	0.0525	0.9186	0.0388	0.0709	0.8538	0.0533
➁	PLS	0.1348	0.4623	0.1077	0.1352	0.4687	0.1067
	VBPCA	0.1339	0.4694	0.1092	0.1315	0.4976	0.1052
	LSTM	0.1227	0.5543	0.1051	0.0829	0.8128	0.0606
	ESN	0.0955	0.7301	0.0759	0.0912	0.7582	0.0730
	DPLS	0.0777	0.8215	0.0647	0.0732	0.8444	0.0587
	DVBPCA	0.0673	0.8659	0.0536	0.0719	0.8497	0.0574
	MW-DVBPCA	0.0511	0.9228	0.0368	0.0665	0.8717	0.0515

➀ The time delays are not considered, ➁ the time delays are considered.

and

Table 3. Quantitative estimation results of various models for the concentrations of ${\mathrm{CO}}_2$ and ${\mathrm{H}}_2$.

		${\mathbf{CO}}_2$			${\mathbf{H}}_2$
		RMSE	${\mathbf{R}}^{\mathbf{2}}$	MAE	RMSE	${\mathit{R}}^{\mathbf{2}}$	MAE
➀	PLS	0.1930	0.0095	0.1536	0.1882	−0.0729	0.1612
	VBPCA	0.1913	0.0264	0.1521	0.1973	−0.1792	0.1660
	LSTM	0.1562	0.3949	0.1293	0.1302	0.4863	0.1118
	ESN	0.1152	0.6471	0.0946	0.1057	0.6616	0.0895
	DPLS	0.1346	0.5185	0.1099	0.0955	0.7237	0.0779
	DVBPCA	0.1145	0.6514	0.0906	0.0796	0.8080	0.0676
	MW-DVBPCA	0.1103	0.6765	0.0858	0.0548	0.9090	0.0425
➁	PLS	0.1724	0.2094	0.1400	0.1320	0.4722	0.1090
	VBPCA	0.1648	0.2777	0.1361	0.1289	0.4964	0.1065
	LSTM	0.1352	0.5137	0.1094	0.1292	0.4943	0.1015
	ESN	0.1230	0.5978	0.1007	0.0823	0.7948	0.0678
	DPLS	0.1156	0.6448	0.0934	0.0790	0.8107	0.0669
	DVBPCA	0.1119	0.6670	0.0859	0.0752	0.8285	0.0605
	MW-DVBPCA	0.1056	0.7036	0.0835	0.0531	0.9146	0.0403

➀ The time delays are not considered, ➁ the time delays are considered.

. For a further comparison, Table 2 and Table 3 show the estimations of the data-driven virtual sensors not considering transportation delays. These models’ parameter selection is consistent with the corresponding model considering transportation delays. Overall, the estimation results in Table 2 and Table 3 provide an initial validation of the effectiveness of the data-driven virtual sensors. However, the performances of the different data-driven virtual sensors vary considerably. The performance of the models that consider delays is better than that of the corresponding models that do not, as shown in Table 2 and Table 3. Take the DVBPCA as an instance. The predictive performance based on the RMSE index of the four KVs by the model accounting for the time delays is improved by 18.6%, 13.0%, 2.3%, and 5.5%, respectively, compared to the model ignoring time delays. This is because the hydrogen production process has substantial time delays, and it has been proven that ignoring the delays could result in significantly deteriorated performance [29]. The ${\mathrm{R}}^2$ values for two static models, i.e., the PLS and VBPCA, are as low as below 0.5; in contrast, the dynamic models, such as LSTM, ESN, and DPLS, show significantly better performance than the PLS and VBPCA, indicating the dynamic model better fits the data features of the actual hydrogen production process. Moreover, due to the capability of dealing with overfitting, the DVBPCA performs better than the DPLS. Concretely, compared to the DPLS, the RMSEs of the four KVs obtained by the DVBPCA are decreased by 13.4%, 1.8%, 3.2%, and 4.8%, respectively. Moreover, the MW-DVBPCA further improves the estimations of the four KVs. The ${\mathrm{R}}^2$s of the ${\mathrm{CH}}_4$ and ${\mathrm{H}}_2$ concentrations reach as high as up to 0.9. Compared with the DVBPCA, the predictive performance on the four KVs by the MW-DVBPCA improves by 1.3%, 10.3%, 2.8%, and 33.4%, respectively, in terms of the MAE index.

Furthermore, to check whether the MW-DVBPCA’s performance is significantly different from that of other models, the Wilcoxon test is employed for statistical testing [35]. The Wilcoxon test is a non-parametric testing method which is used to examine whether there is significant difference in the median values of the squared estimation errors obtained by the two virtual sensors. In Wilcoxon’s test, the likelihood that the corresponding hypothesis will be accepted is measured by calculating the p-value. The smaller the value of p-value, the lower the probability that the corresponding hypothesis will be accepted. Typically, the hypothesis should be rejected if the p-value is less than the given significance level $\eta$, the hypothesis should be rejected; that is, statistically the median values of two virtual sensors are different.

The Wilcoxon test results are given in

Table 4. Results of the Wilcoxon test.

KV	Hypothesis	$\mathit{p}-\mathbf{Value}$	Decision
${\mathrm{CH}}_4$ concentration	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{PLS}}^2$	$2.79\times {10}^{-70}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{VBPCA}}^2$	$1.48\times {10}^{-71}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{LSTM}}^2$	$1.15\times {10}^{-81}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{ESN}}^2$	$5.39\times {10}^{-42}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DPLS}}^2$	$6.32\times {10}^{-30}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DVBPCA}}^2$	$1.03\times {10}^{-56}$	$\mathrm{Reject}{\mathcal{H}}_0$
$\mathrm{CO}$ concentration	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{PLS}}^2$	$7.67\times {10}^{-45}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{VBPCA}}^2$	$3.15\times {10}^{-44}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{LSTM}}^2$	$8.1\times {10}^{-3}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{ESN}}^2$	$1.43\times {10}^{-18}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DPLS}}^2$	$2.3\times {10}^{-3}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DVBPCA}}^2$	$2.58\times {10}^{-11}$	$\mathrm{Reject}{\mathcal{H}}_0$
${\mathrm{CO}}_2$ concentration	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{PLS}}^2$	$3.00\times {10}^{-35}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{VBPCA}}^2$	$1.69\times {10}^{-36}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{LSTM}}^2$	$8.67\times {10}^{-15}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{ESN}}^2$	$1.19\times {10}^{-5}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DPLS}}^2$	$9.95\times {10}^{-4}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DVBPCA}}^2$	$2.78\times {10}^{-4}$	$\mathrm{Reject}{\mathcal{H}}_0$
${\mathrm{H}}_2$ concentration	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{PLS}}^2$	$6.52\times {10}^{-74}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{VBPCA}}^2$	$4.60\times {10}^{-69}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{LSTM}}^2$	$3.04\times {10}^{-60}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{ESN}}^2$	$2.97\times {10}^{-29}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DPLS}}^2$	$4.25\times {10}^{-30}$	$\mathrm{Reject}{\mathcal{H}}_0$
	${\mathcal{H}}_0:{\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2={\overline{E}}_{\mathrm{DVBPCA}}^2$	$3.19\times {10}^{-50}$	$\mathrm{Reject}{\mathcal{H}}_0$

, where ${\overline{E}}_{\mathrm{PLS}}^2$, ${\overline{E}}_{\mathrm{VBPCA}}^2$, ${\overline{E}}_{\mathrm{LSTM}}^2$, ${\overline{E}}_{\mathrm{ESN}}^2$, ${\overline{E}}_{\mathrm{DPLS}}^2$, ${\overline{E}}_{\mathrm{DVBPCA}}^2$, and ${\overline{E}}_{\mathrm{MW}-\mathrm{DVBPCA}}^2$ mean the median values of squared estimated errors obtained by the PLS, VBPCA, LSTM, ESN, DPLS, DVBPCA, and MW-DVBPCA, respectively. Additionally, set the significance level $\eta$ at 5%. As shown in Table 4, all hypothesized p-values are far below $\eta$. Hence, all hypotheses are rejected. In other words, there is statistical significance in comparing the MW-DVBPCA with other virtual sensors in the hydrogen production process.

4.6. Computational Efficiency Analysis

Since this article is concerned with real-time estimation, examining the runtime of the model is desirable. The offline and online computational efficiency of the virtual sensors is evaluated using the average over 10 independent simulations, including the CPU time consumed offline for parameter optimization (${\mathrm{CPT}}^{\mathrm{opt}}$) and the CPU time consumed online (${\mathrm{CPT}}^{\mathrm{online}}$). All experiments were computed on a Core i5 (2.90 GHz × 2) with 8 GB RAM, Windows 10 and R2021a version of MATLAB.

Table 5. Comparison of computational efficiency of various models (seconds).

	${\mathrm{CPT}}^{\mathbf{opt}}$	${\mathrm{CPT}}^{\mathbf{online}}$
PLS	3.44	<0.01
VBPCA	11.49	<0.01
LSTM	3893.01	0.04
ESN	264.38	<0.01
DPLS	255.83	<0.01
DVBPCA	210.16	<0.01
MW-DVBPCA	1877.91	0.06

lists the time taken by each virtual sensor on parameter determination. As can be observed, the ${\mathrm{CPT}}^{\mathrm{opt}}$ index for the DVBPCA model is much smaller than that for the LSTM, due to its more concise structure. The ${\mathrm{CPT}}^{\mathrm{opt}}$ indices for other dynamic global models are almost the same as those for the DVBPCA, but other dynamic global models have lower accuracy than the DVBPCA given in Table 2 and Table 3. Note that the ${\mathrm{CPT}}^{\mathrm{opt}}$ index for the MW-DVBPCA is much larger than that for the DVBPCA, which is because the MW-DVBPCA needs to rebuild the model each time it predicts a new valid sample. Fortunately, the parameter determination processes are carried out offline. In other words, this process hardly affects the online calculative efficiency of the MW-DVBPCA. The last column of Table 5 illustrates the online computation time of MW-DVBPCA. Consequently, the online computational efficiency of the developed models is not an issue. In practice, the ${\mathrm{CPT}}^{\mathrm{opt}}$ indices for all virtual sensors are less than 0.1 s/sample, significantly faster than the minimum sampling period for the KVs in the hydrogen production process. The results show that all data-driven virtual sensors meet the time requirements for real-time estimation, including the developed DVBPCA and MW-DVBPCA.

5. Conclusions and Outlook

Considering the complicated properties of the hydrogen production process, we develop virtual sensors for the KVs of the hydrogen production process in this article. The MW-DVBPCA is developed to model complicated properties of the hydrogen production process, including dynamics, time variations, and transportation delays. To be specific, the FIR paradigm and MW technique are employed to extract process dynamics and to deal with time variations, respectively. The time delays are determined automatically by the DE. A comparative study of developed virtual sensors and other state-of-the-art virtual sensors is carried out. Meanwhile, the performance of MW-DVBPCA is demonstrated by the real-life natural gas steam reforming hydrogen production process.

From the industrial point of view, the online estimations of the KVs of the hydrogen production process need further research. Some future works are given to further improve the estimation performance of the KVs.

• Robust methods. The probabilistic model in this article is based on the traditional Gaussian distribution assumption, which is susceptible to outliers. Therefore, the training set must be cleaned to remove outliers. However, some outliers are indistinctive and challenging to detect and remove. To this end, finding a probability distribution insensitive to the noise and outliers can help improve the generalization performance of predictive models. Typically, Student’s t distribution with heavier tails is a candidate choice. As a result, designing a robust virtual sensor based on Student’s t distribution is worth investigating.
• Data-driven approaches fused with process knowledge. In fact, the states (or the hidden variables) of the system are influenced by variables characterizing materials and energies fed into the process. Conventional virtual sensors take all observed variables as inputs and the KVs as outputs, which makes it difficult to describe the true causality between variables of the hydrogen production process, weakening the interpretability and generalization abilities. A causal virtual sensor can better reflect the process mechanism and thus estimate the KVs more accurately. Therefore, equipping the MW-DVBPCA model with the causality of process variables of the hydrogen production process is desirable.