Anomaly Detection Method for Multivariate Time Series Data of Oil and Gas Stations Based on Digital Twin and MTAD-GAN

Abstract

Due to the complexity of the oil and gas station system, the operational data, with various temporal dependencies and inter-metric dependencies, has the characteristics of diverse patterns, variable working conditions and imbalance, which brings great challenges to multivariate time series anomaly detection. Moreover, the time-series reconstruction information of data from digital twin space can be used to identify and interpret anomalies. Therefore, this paper proposes a digital twin-driven MTAD-GAN (Multivariate Time Series Data Anomaly Detection with GAN) oil and gas station anomaly detection method. Firstly, the operational framework consisting of digital twin model, virtual-real synchronization algorithm, anomaly detection strategy and realistic station is constructed, and an efficient virtual-real mapping is achieved by embedding a stochastic Petri net (SPN) to describe the station-operating logic of behavior. Secondly, based on the potential correlation and complementarity among time series variables, we present a MTAD-GAN anomaly detection method to reconstruct the error of multivariate time series by combining mechanism of knowledge graph attention and temporal Hawkes attention to judge the abnormal samples by a given threshold. The experimental results show that the digital twin-driven anomaly detection method can achieve accurate identification of anomalous data with complex patterns, and the performance of MTAD-GAN anomaly detection is improved by about 2.6% compared with other methods based on machine learning and deep learning, which proves the effectiveness of the method.

1. Introduction

The anomaly-detection problem is one of the important research topics for data mining [1], network intrusion [2], structural defects [3], text errors [4], industrial production [5] and social networks [6], etc. Considering the actual situation in physical industrial production, it is very difficult to detect anomalies in the samples due to diverse patterns, variable working conditions and imbalance data. Even though some progress [7,8,9,10,11] has been made, the task of multivariate time series anomaly detection is still very challenging.

Traditional anomaly detection methods have been extensively studied, with various strategies such as density estimation [12], tree based [13], cluster based [14], proximity based [15], probability based [16] and dimension reduction based [17]. However, faced with diverse patterns, variable working conditions and imbalance data, those methods cannot achieve good results. Another kind of anomaly detection methods with multivariate time series is based on deep learning. Among them, the long short term memory (LSTM) provide effective methods for analyzing time correlation features of time-series datasets, i.e., VLSTM [18], AMF-LSTM [19], DBN-LSTM [20], LSTM-AE [21,22,23], etc. Although these methods have improved accuracy and scope of application compared with traditional methods, they still cannot resolve the anomaly detection for the reasons of diverse patterns, variable working conditions and imbalance samples.

In this paper, we construct a digital twin framework by giving definitions of consistency, completeness and soundness for digital twin system, and using stochastic Petri net to realize the virtual and reality mapping. The MTAD-GAN anomaly detection method is designed to enhance the spatio-temporal correlation of multivatiate time serier by combining the attention mechanism of knowledge graph and temporal Hawkes. For resolving the problem of diverse patterns, variable working conditions and imbalance data, we propose a knowledge-aware transfer learning by using the data from digital twin space.

In Figure 1, we present an overview of this paper. In the knowledge graph section, we divide the physical parts and the relationships into several components, and use the knowledge graph for mapping the physical and virtual models. In the Petri net section, we describe the realistic oil and gas station operation behavior logically and achieve an efficient virtual-real mapping. In the evolution rules section, different nodes in the knowledge graph can be set according to different risk factors. In the anomaly detection section, we propose an MTAD-GAN anomaly detection method that reconstructs multivatiate time-series by combing knowledge graph attention and temporal Hawkes attention mechanism, and then uses knowledge transfer learning to detect anomalies in realistic oil and gas stations. Accordingly, our main contributions can be summarized as follows:

• We propose a digital twin operation framework, which is mapped to a virtual station by decomposing the knowledge graph of the physical station. The stochastic Petri net is designed to describe the station behavior logic and achieve an efficient virtual mapping.
• In order to resolve the problems of of diverse patterns, variable working conditions and imbalance samples, the method of MTAD-GAN is proposed by using the potential relationship between time-series variables, which enhances the features of multivatiate time-series by combining knowledge graph attention and temporal Hawkes attention mechanism. The ADGS scoring loss function is designed to estimate the probability distribution of network learning samples to complete the anomaly detection.
• Experiments on accuracy, precision, F1 and AUCROC with different datasets have shown great improvements with integrating the proposed MTAD-GAN. It demonstrates that the MTAD-GAN can effectively detect anomalies and outperforms the state-of-the-art deep learning methods as well as traditional methods.

We will introduce the related works in Section 2 and digital twin framework in Section 3.1. We describe the proposed MTAD-GAN in Section 3.2. In Section 4, we present the experimental results and compare our algorithm with others. In Section 5, we discuss the reason for the achieved results. Section 6 gives the conclusions of this paper.

2. Related Works

2.1. Digital Twin

With the rapid evolution of digital twins, the application of digital twins in various industries is becoming more and more widespread.

Table 1. A Comparative Study for Digital Twin in open literature.

References	Type	Broad Area	Specific Area	Technology
Tao et al. [29] (2019)	Case Study	Manufacturing	Product Design	Big Data
Sacks et al. [30] (2020)	Conception	Smart City	Construction Industry	BIM, Construction Planning and Control
Guo et al. [28] (2021)	Case Study	Manufacturing	Production Line	Transfer Learning, IRF
Salem and Dragomir [27] (2022)	Conception	Smart City	Construction Industry	BIM, AI, Monitoring
Li et al. [24] (2022)	Case Study	Manufacturing	Smart Manufacturing	Analytics, Evaluation
Priyanka et al. [25] (2022)	Case Study	Manufacturing	Oil Pipeline	Risk Estimation, SVM
Yang et al. [26] (2022)	Case Study	Manufacturing	Log Data Mining	GRU, AI
Ours	Case Study	Manufacturing	Oil and Gas Station	Petri Net, Knowledge Graph

shows the key sources from manufacturing and smart cities based on digital twin. The digital twin serves as a bridge between the virtual and real worlds and is often used by industry to enhance the processing and management of information related to virtual and physical entities. Li et al. [24] uses quantitative green performance evaluation of smart manufacturing (GPEoSM) to quantify and evaluate green performance. which focuses more on the evaluation of the completed digital twin for iterative optimization. Priyanka et al. [25] proposes the digital twin framework to detect the failure precursors with the predictive algorithms. Yang et al. [26] uses process transformation technology to build digital twin framework, which is used to mine event logs for characterization and then predicts the remaining cycle values of a manufacturing process based on subsequent states by using a GRU neural network. Salem and Dragomir [27] present a framework of digital twins to manage activities and services related to the operation of buildings and urban structures. Guo et al. [28] introduces an improved random forest algorithm and maps the trained model to the physical production line for fault diagnosis by use transfer learning. Tao et al. [29] demonstrates that the framework of digital twin-driven product design (DTPD) can facilitate the iterative optimisation of bicycle design.

2.2. Deep Learning

Deep learning has been the most popular method in recent years. Among them, LSTM networks and GAN networks, as the basic network models for time series prediction, can effectively deal with the problems that cannot be solved by traditional methods. We find the outcomes and limitations of anomaly detection approaches based on LSTM and GAN in

Table 2. Comparative Analysis of the proposed deep learning scheme with related research.

References	Techniques	Outcomes	Limitations
LSTM-GAN [35] (2019)	LSTM, GAN	Higher accuracy rate compared to traditional methods	Relationships between multidimensional variables are ignored
SAE-LSTM [31] (2019)	Stack AE, LSTM	Higher detection rate in multiple features sequence	Unbalanced data have not been considered
VAE-LSTM [32] (2020)	VAE, LSTM	Higher recall rate and F1 score compared to standard methods	The relationship between multivariates has not been considered
DAGAN [34] (2020)	Dual AE, GAN	High AUC is maintained even with a small quantity of training data	Unbalanced sample distribution has not been considered
ART-GAN [33] (2021)	Data augmentation, GAN	Using data augmentation to solve data imbalances	Data augmentation only considers a single sensor
Improved LSTM [36](2022)	LSTM	Time series data anomaly detection for diverse distributions	Unable to detect anomalies for small quantity of data
LSTM-FS [37] (2022)	LSTM, FS, Transfer learning	Using transfer learning to reduce the differences between data distributions	Power status has a large impact on monitoring data
MTAD-GAN (proposed)	LSTM, GAN, Digital twin, Transfer learning	Using digital twin to solve problem of data imbalances and small quantity	Performance can be improved with exploring relationships of time series and adjusting the network structure for higher anomaly detection rate

. Li et al. [31] combine a stacked autoencoder (SAE) and a long short-term memory (LSTM) neural network to identify anomalies in a completely unsupervised manner. The VAE-LSTM hybrid model [32] as an unsupervised method for time series anomaly detection utilizes VAE modules to form robust local features over short windows and LSTM modules to estimate long-term correlations in the series on top of the features. To reduce training bias, the ATR-GAN [33] is used to generate more effective artificial samples for training supervised learning models. The DAGAN [34] is designed to solve the sample unevenness problem by using a jump-joint and dual self-encoder architecture, which shows excellent image reconstruction capability and training stability. Using the combination of the advantages of LSTM gate structure in processing time series and the advantages of GAN network in acquiring data depth features, the LSTM-GAN [35] network is proposed. Wang et al. [36] attempts to cope with highly diverse data distributions and lack of labelled anomalous data by proposing an improved long short-term memory (LSTM) based time series anomaly detection scheme. In [37], LSTM-FC uses feature-based transfer learning to reduce the variance between wind turbine data distributions and has been validated for feasibility. However, none of the above methods can completely solve problem of missing data and imbalanced samples in real industrial production. We propose an anomaly detection method combining digital twin and transfer learning, which can effectively solve the problem of insufficient data samples as well as better assist industrial production management.

2.3. Transfer Learning

Transfer learning [38] refers to the transfer of knowledge learned in a large number of adjacent data domains to solve similar problems in new data domains faster and more effectively. Vercruyssen et al. [39] applies transfer learning to anomaly detection, transferring labeled examples from the source domain to the target domain with no available labels. Andrews et al. [40] uses transfer representation learning for anomaly detection using convolutional neural networks. The MSTLN [41] aggregates and transmits diagnostic knowledge from multiple source machines by combining multiple locally distributed adaptive subnets (PDA-Subnets) and a multi-source diagnostic knowledge fusion module. Based on a combination of long and short-term memory, fuzzy synthesis and feature-based approaches to transfer learning, Zhu et al. [37] use three feature-based transfer learning methods to reduce the differences between WTG data distributions. In the actual operation of the oil and gas station, due to the lack of abnormal samples, it is difficult to obtain a relatively balanced datasets to train a classifier with generalization.

3. Proposed Method

3.1. SSUPS Based Digital Twin Framework

Due to the diversity and complexity of station equipment, it is difficult to construct a complete oil and gas station digital twin directly at one time. Therefore, we propose a SSUPS-based approach to the construction of digital twins which includes scenario, system, unit, part and subpart. The physical entity division can be expressed as follows:

$$M_R=\{Scenario,System,Unit,Part,Subpart\}$$

(1)

where scenarios are the collection of real systems, such as oil and gas stations, pipelines, etc. The system is specific to a particular system, e.g., compression system, oil delivery system, etc. Units are classes of specific equipment, such as pipeline equipment and compressor equipment. Part is a component of a piece of equipment, such as the body, midbody, cylinders, crankshaft, connecting rods, etc. Subparts are tiny, non-subdividable parts such as bearings and screws. We display the knowledge graph as a semantic network of relationships between physical entities on digital twin framework, as shown in Figure 1. Based on SSUPS component decomposition, the knowledge graph of oil and gas station can deduct the industrial accidents, and judge the early warning and anomaly detection.

3.1.1. Virtual and Reality Mapping Based on SPN

Based on the knowledge graph’s classification of the different components and their relationships, the digital twin (DT) are divided into three classes: entity DT, attribute DT and relationship DT. Therefore, the digital twin mapping of an oil and gas station can be represented as a mapping of the various types of TE, we have:

$$M_{DT}=M_{EDT}\cup M_{ADT}\cup M_{RDT}$$

(2)

The mapping between entities is a topology connection relationship. Entities include equipment, pipeline components, buildings, as well as sensor data collections and business processes, etc. All digital twins after entity mapping form a finite set. This finite set can be expressed as:

$$EDT=\left\{TE\right|M_{EDT}\left[n\right]\in Se{t}_{EDT}=\{s_1,s_2,\cdots ,s_{n_{EDT}}\},\forall n\in [0,N_{EDT}]\}.$$

(3)

where N is the number of entities class, $TE$ is the twinning entity, $EDT$ is the entity digital twin, $Se{t}_{EDT}$ is the entity mapping set. The mapping function of the entity digital twin can be expressed as:

$$\begin{array}{c}{M}_{EDT}=\{f_{TE1},f_{TE2},f_{T{E}_{n_{ETE}}}|f_{T{E}_i}:D_{E_i}^R\mapsto D_{T{E}_i}^{DT},\hfill \\ s.t.\left|D_{E_i}^R\right|=\left|D_{T{E}_i}^{DT}\right|\in Se{t}_{EDT},\hfill \\ \forall i=1,2,\cdots ,n_{EDT}\}.\hfill \end{array}$$

(4)

where $M_{EDT}$ is the mapping entity digital twin, f is the mapping function, $\left|D\right|$ is a value operation that ignores data structures, formats, timestamps, etc.

The attribute mapping in the knowledge graph triplet has a unique identifier of the device with device information and status, which is represented as follows:

$$ADT=\left\{TA\right|M_{ADT}\in Se{t}_{ADT}=\{ID,State,\cdots ,A_{DT}\}\}.$$

(5)

where N is the number of attribute class, $TA$ is the twinning attribute, $ADT$ is the attribute digital twin, $Se{t}_{EADT}$ is the attribute mapping set. The mapping function of the attribute digital twin can be expressed as:

$$\begin{array}{c}{M}_{ADT}=\{f_{TA1},f_{TA2},f_{T{E}_{n_{DTA}}}|f_{T{E}_i}:D_{T{E}_i}^R\mapsto D_{T{E}_i}^{DT},\hfill \\ s.t.\left|D_{T{E}_i}^R\right|=\left|D_{T{E}_i}^{DT}\right|\in Se{t}_{ADT},\hfill \\ \forall i=1,2,\cdots ,n_{ADT}\}.\hfill \end{array}$$

(6)

where $M_{ADT}$ is the mapping entity digital twin, f is the mapping function.

Relationships are the bridges for connecting entities in knowledge graph, which can be expressed as:

$$\begin{array}{c}{M}_{RDT}=\{f_{TR1},f_{TR2},f_{T{R}_{n_{RDT}}}|f_{T{E}_i}:D_{T{E}_i}^R\mapsto D_{T{R}_i}^{DT},\hfill \\ s.t.\left|D_{T{R}_i}^R\right|=\left|D_{T{R}_i}^{DT}\right|,\hfill \\ \forall i=1,2,\cdots ,n_{RDT}\}.\hfill \end{array}$$

(7)

where N is the number of Relationship classes, $TA$ is the twinning attribute, and $ADT$ is the attribute digital twin.

The division of the above three types for TE conducts to the integration of twin body data processing, and facilitates the design of simulation behaviors in the digital twin platform according to the digital twin representation methods of different TEs. In order to enhance virtual and reality interaction performace, our digital twin framework integrates SPN and evolution rules to improve better generalization on anomaly detection task. The SPN is defined as a seven-tuple $N=(P,T,F,K,W,M_0,L)$. Based on the recursive combination of elements such as places and transitions, a logical network model of virtual-real interaction is constructed, as shown in Figure 2. The meanings of all places and transitions in the logical Petri net model are shown in

Table 3. The meaning of place in Petri net.

Meaning of Place
P1 data collection	P2 abnormal data	P3 low level warning
P4 normal data	P5 advanced warning	P6 detection feedback information
P7 On-site monitoring notice	P8 Alert information	P9 On-site warning information
P10 Historical information data	P11 Pre-plan library	P12 accident data
P13 Plan confirmed	P14 loss estimate	P15 Plan optimization
P16 resume operation

.

3.1.2. Digital Twin Definitions

To provide the virtual-real synchronization of consistency, completeness and soundness for the operating logic of behavior of oil and gas station, we provide precise definitions for these three aspects.

Definition 1.

(Consistency): For$\forall s\in E_R$, exist only one$m\in E_{DT}$, such that$G(s,r,v)=G(m,r,v)$. Where G is the knowledge graph, s is one of the triples in the knowledge graph. This definition reflects entities in physical space are the same as entities in virtual space.

Definition 2.

(Completeness): For$\forall s\in E_R$, $\exists m\in E_{DT}$, such that$G(s,r,v)=G(m,r,v)$, and for$s⊢\phi$, $G(\phi ,r,v)=G(m^{\prime },r,v)$is also satisfied. Where$s^{\prime }$is the derivation of s through different relations in the knowledge graph.

Definition 3.

(Soundness): If$\exists m\in E_{DT}$, $f_m:\mu \to {\mu }^{\prime }$, then$G\left(m\right)=f_m\left(G\left(m^{\prime }\right)\right)$. Where$m^{\prime }=s|s⊢,s\in E_{RW}$, f is the mapping function.

When the constructed digital twin satisfies the above three confirmatory definitions, it can be called a qualified digital twin that meets the demand of anomaly detection.

3.2. MTAD-GAN Network

3.2.1. Network Structure

To achieve anomaly detection of multivariate time series data, we designed a multi-level crossover GAN network (MTAD-GAN), as shown in Figure 3. During the MTAD-GAN model training phase, we input random noise with the same dimension as the training sample into the knowledge graph attention and Hawkes attention long short-term memory (KH-LSTM) to obtain the generated sample. Then the generated samples and real samples are simultaneously transmitted to the discriminator for iterative training until the network reaches Nash equilibrium. In the anomaly detection phase, the test samples are mapped to the random latent space, and the reconstruction samples are generated through iterative training by calculating the reconstruction loss. In order to strengthen the feature difference between normal samples and abnormal samples, a scoring network is introduced here to calculate the score loss of the reconstructed samples. Combining the Gaussian joint distribution, the adaptive dynamic Gaussian score (ADGS) is constructed by fusing the scoring loss, reconstruction loss and discrimination loss to detect and judge the anomaly of the data.

3.2.2. KH-LSTM Network Structure

In the anomaly detection task, the knowledge graph attention module is introduced to enhance the spatial correlation of features. We incorporated the temporal Hawkes attention layer into the LSTM to suppress interference from unrelated features, as shown in Figure 4. The knowledge graph attention module is used to spatially weight the original input sequence $X^i=\{x_1^i,x_2^i,\cdots ,x_T^i\}\in R^{T\times D}$. Based on the hidden layer space state of LSTM, combined with temporal hawked attention, the temporal correlation characteristics of the input signal are enhanced and obtain the spatiotemporal feature sequence $Y^i=\{y_1^i,y_2^i,\cdots ,y_T^i\}\in R^{T\times D}$.

3.2.3. Knowledge Graph Attention Module

Set $X\in R^{T\times D}$ as the input time series data, D is the sensor variable dimension, T is the time series length. Divide the input X time series data into T D-dimension vectors, the single-layer knowledge graph attention network update node is expressed as follows:

$$e_{ij}=LeakyRuLU\left(a^T\left[W{x}_i∥W{x}_j\right]\right)$$

(8)

$${\alpha }_{ij}=softmax\left(e_{ij}\right)=\frac{exp\left(e_{ij}\right)}{{\sum }_{k\in N_j}exp\left(e_{ik}\right)}$$

(9)

$${x_i}^{\prime }=\sigma \left(\sum _{j\in N_i}({\alpha }_{ij}W{x}_i\right)$$

(10)

where $e_{ij}$ is a single-layer feed-forward neural network, a and W are learnable parameters in the neural network, ‖ represents the concatenation of vectors, $\alpha$ is the attention weight. The node feature representation after the update is obtained by Equation (10).

3.2.4. Hawkes Attention Module

Considering that each spatial feature may have noise, temporal Hawkes attention mechanism is introduced to describe temporal correlation features. Set $s_{(t-1)}$ as LSTM hidden state at time t, we have:

$$g_{\left(t\right)}^k=V_2^{k}tanh(W_2^k{s}_{(t-1)}+U_2^k{h}_{(t-T+k)}+b_2^k),1\le k\le M$$

(11)

$${\beta }_{\left(t\right)}^k=\frac{\left|g_{\left(t\right)}^k\right|}{{\sum }_{m=1}^T\left|g_{\left(t\right)}^m\right|},1\le k\le M$$

(12)

where $V_2^k$, $W_2^k$, $U_2^k$ and $b_2^k$ are the training model parameters, $g_{{\left(t\right)}^k}$ is the weight of the kth LSTM hidden state of the sub window at time t, M is the size of sub window.

The Hawkes process [42] is a self-excited point-in-time process that models discrete sequences of events in continuous time. We compute the temporal features B as:

$$B={\beta }_t+\sum _{\tau =0,\Delta t_{\tau }\ge 0}\kappa {\beta }_t{e}^{-\delta \Delta t_{\tau }}$$

(13)

where $\tau$ is the time point and $\delta$ is the decay parameter. The context vector obtained by weighted summation of hidden states using attention weight is:

$$c_{\left(t\right)}={\sum }_{k=1}^T{h}_{(t-T+k)}B$$

(14)

where $c_{\left(t\right)}$ is the context vector at time t, ${\beta }_{\left(t\right)}^k$ is the temporal attention weight.

3.2.5. Knowledge-Aware Transfer Learning

As shown in the Figure 5, transfer learning can map knowledge from the source domain to the target domain, and the data simulated from our digital twin space may have different effects on the physical devices. Therefore, we apply an attention mechanism to better learn the knowledge relation between two domains. When the device correlation of two domains is low, the shared knowledge will be penalized, and the attention network will learn more device-specific domain knowledge.The formulas for knowledge-aware similarity are as follows:

$${\gamma }_{(K,S)}^{*}=h_{\gamma }^{T}ReLU(W_{\gamma }{k}_{\gamma }+b_{\gamma });{\gamma }_{(C,S)}^{*}=h_{\gamma }^{T}ReLU(W_{\gamma }{c}_{\gamma }+b_{\gamma })$$

(15)

$${\mu }_{(K,S)}^{*}=h_{\mu }^{T}ReLU(W_{\mu }{k}_{\mu }+b_{\mu });{\mu }_{(P,S)}^{*}=h_{\mu }^{T}ReLU(W_{\mu }{p}_{\mu }+b_{\mu })$$

(16)

where W is the weight matrices, h and bias b are the parameters of the network, $\gamma$ and $\mu$ are the related to C domain and P domain. The attention scores is normalized with a softmax function:

$$\begin{array}{c}{\gamma }_{(K,S)}=\frac{exp\left({\gamma }_{(K,S)}^{*}\right)}{exp\left({\gamma }_{(K,S)}^{*}\right)+exp\left({\gamma }_{(C,S)}^{*}\right)}=1-{\gamma }_{(C,S)}\hfill \\ {\mu }_{(K,S)}=\frac{exp\left({\mu }_{(K,S)}^{*}\right)}{exp\left({\mu }_{(K,S)}^{*}\right)+exp\left({\mu }_{(P,S)}^{*}\right)}=1-{\mu }_{(P,S)}\hfill \end{array}$$

(17)

where $\gamma$ and $\mu$ represent common knowledge between two domains. After calculating the above attention weights, the features of the two domains can be expressed as follows:

$$u_s^C={\gamma }_{(C,S)}{c}_{\gamma }+{\gamma }_{(K,S)}{k}_{\gamma };u_s^P={\mu }_{(P,S)}{c}_{\mu }+{\mu }_{(K,S)}{k}_{\mu }$$

(18)

Transfer learning computes two types of losses, with marginal distribution loss and conditional distribution loss, which are the data distribution and prediction distribution, respectively. The specific losses are as follows:

$$\begin{array}{c}loss={\lambda }_{1}los{s}_{KL}+{\lambda }_{2}los{s}_{MMD}\hfill \\ ={\lambda }_1\sum u_i^{source}log{u}_i^{target}+{\lambda }_2∥\frac{1}{n^s}{\displaystyle \sum _{i=1}^{n^s}}{u}_i^s-\frac{1}{n^t}{\displaystyle \sum _{i=1}^{n^t}}{u}_i^t∥\hfill \end{array}$$

(19)

where ${\lambda }_1$ and ${\lambda }_2$ are the hyper-parameters. We use the grid search method to train the hyper-parameters by the validation set at $\{0.1,0.2,0.3,\cdots ,0.9\}$. The optimal parameter are both set as 0.5 in our experiments.

3.3. Abnormal Score

In the anomaly detection phase, the test set $A^{test}$ is mapped to the potential feature space and entered into the generator to generate the initial reconstruction sample $G\left(Z^1\right)$. The gradient is then updated by calculating the error between the test sample and the reconstructed sample. The similarity measure between the input sample and the potential feature space is defined as follows:

$$\stackrel{min}{z^m}Er(A^{test},G_{rnn}\left(Z^m\right))=1-Sim(A^{test},G_{rnn}\left(Z^m\right))$$

(20)

where $Er\left(·\right)$ is the error measurement function, $Sim\left(·\right)$ is the similarity measure function, and $Z^m$ is the mth reconstruction sample. The Equation (20) is obtained by gradient descent to obtain the optimal solution, then the sample residual function of the moment is expressed as:

$$Z^m=G_{rnn}\left(Z_a\right),s=S\left(Z_a\right)$$

(21)

The minimize-loss function is used to update the parameters in S, using the reconstruction error as a self-supervised signal, defining the score loss as:

$$L_S=\left\{\begin{array}{cc}\left|s-{\mu }_0\right|,& F\left(A_t^{test}\right)<\epsilon \\ {\lambda }_{a}max(0,a-s),& F\left(A_t^{test}\right)\ge \epsilon \end{array}\right.$$

(22)

where a is the threshold and ${\mu }_0$ is the anomaly score.

The MTAD-GAN model is a multi-branch structure, due to the different learning tasks of different branches, the end-to-end training method is adopted, and the overall loss function is the sum of the weighting coefficients of multiple loss functions, we have:

$$L=\alpha (L_R+L_D)+(1-\alpha ){\lambda }_S\frac{1}{N}{L}_S$$

(23)

where $L_R$ is the reconstruction loss, $L_D$ is the discriminant loss, $L_S$ is the score loss. According to the subsequence of multiple test data, we can obtain multiple sets of anomaly loss functions $L=L_{j,s},j=1,2,\cdots ,n;s=1,2,\cdots ,S_w\subseteq R_n\times S_w$. Here, the anomaly detection loss of multiple groups of subsequences is fitted to a Gaussian distribution, and then the adaptive dynamic scoring function ADGS is obtained, which is defined as follows:

$$ADG{S}_t^i=-log(1-\phi (\frac{L_t^i-{\widehat{\mu }}_t^i}{{\widehat{\sigma }}_t^i}))$$

(24)

$${\widehat{\mu }}_t^i=\frac{i}{M}{\sum }_{j=0}^{M-1}{L}_{t-j}^i$$

(25)

$${({\widehat{\sigma }}_t^i)}^2=\frac{1}{M-1}{\sum }_{j=0}^{M-1}(L_{t-j}^i-{\widehat{\mu }}_t^i){}^2$$

(26)

where ${\widehat{\mu }}_t^i$ is the dynamic mean, ${\widehat{\sigma }}_t^i$ is the dynamic variance, M is the window size.

4. Experiments

4.1. Datasets

Experiments were carried out on the KDD99 [43], SWaT [44], WADI [45], SKAB [46], DAMADICS [47], MSL [48], SMAP [48], SMD [49] and J10031 datasets.

Table 4. The details of datasets.

Dataset	No. Sensors	Normal Data	Data with Attacks	Attacks	Attack Duration (mins)
KDD99	34	562,387	494021	2	NA
SWAT	51	496,800	449,919	36	2-25
WADI	127	1,048,571	172,801	15	1.5–30
J10031	29	43,194	42,150	5	2–30
SKAB	20	9401	35,600	12	2.4–9.8
DAMADICS	12	8546	9542	5	2.1–5.6
MSL	10	2160	2731	2	11–1141
SMAP	24	2556	8071	4	31–4218
SMD	16	25,300	25,301	3	2–3160

shows the detail of our pre-processed dataset. For each full dataset, it is divided into the ratio of training data, validation data and test data as 6:2:2 in the process of learning.

4.1.1. KDD99

The KDD99 dataset from the Third International Knowlegde Discovery and Data Mining Tools Competition in 1999 for system evaluation of network security contains 4,898,431 samples with 40 columns of features and 1 column of class labels. In this paper, 562,387 samples for 34 sensors were collected under normal working conditions, and 494,021 samples were collected under various attacked scenario.

4.1.2. SWaT

The Secure Water Treatment (SWaT) dataset is a water treatment testbed, which contains data from the water treatment plant and water treatment process generated. It includes 7 days of normal data with 496,800 samples and 4 days of abnormal data with 449,919 samples. The abnormal data contain 36 attacks in total. The number of sensors and actuators in the testbed for data collection is 51.

4.1.3. WADI

The Water Distribution (WADI) dataset contains data from a water distribution network in a city, which collected by the Singapore Public Utilities Commission. The dataset collected recordings of 123 sensors for 16 days, which contains 1,048,571 samples under the normal scenario for the first 14 days and 172,801 samples under the attacked scenario for the last 2 days. There are 15 attacks in total, generating about 6% abnormal data.

4.1.4. J10031

The J10031 is the measured valve dataset of compressors in an oil and gas station, which contains 43,194 normal samples for 29 sensors and 42,150 data samples under abnormal working condition.

4.1.5. SKAB

The Skoltech Anomaly Benchmark (SKAB) dataset is designed for evaluation algorithms for anomaly detection which contains 35 subsets of data from a water circulation system with the sensors installed on the testbed. The 9401 samples were obtained from the experiments with normal mode and the 35,600 samples were under abnormal scenario.

4.1.6. DAMADICS

The Development and Application of Methods for Actuator Diagnosis in Industrial Control Systems (DAMADICS) is a benchmark which includes data from a real water evaporation process in a boiler of a sugar factory in Poland. In this paper, the DAMADICS simulated 8546 normal samples and 9542 abnormal samples under the operating condition of pneumatic control valves in industrial processes.

4.1.7. MSL and SMAP

The Mars Science Laboratory rover (MSL) and the Soil Moisture Active Passive satellite (SMAP) are real-word datasets acquired from the spacecraft, which were annotated by experts of NASA. MSL dataset includes the 2160 normal samples and the 2731 samples under attack and SMAP includes the 2556 normal samples and the 8071 samples under attack. There are 55 and 27 sensors in SMAP and MSL datasets, respectively.

4.1.8. SMD

The Server Machine Dataset (SMD) is a five-week dataset, which contains data from 28 server machines. To avoid the effect of service change during the data collection period, we used part of the SMD dataset, containing 25,300 normal samples and 25,301 samples under attack. The data on each server are monitored by 38 sensors.

4.2. Evaluation Indicators

To evaluate our method more intuitively and comprehensively, we calculate accuracy A, precision P, recall R, $F1$ score and AUC-ROC as measure of experimental results. The calculation formula is as follows:

$$P=\frac{TP}{TP+FP}$$

(27)

$$R=\frac{TP}{TP+FN}$$

(28)

$$F1=\frac{2\times P\times R}{TP+FN}$$

(29)

$$A=\frac{TP+TN}{TP+FP+TN+FN}$$

(30)

where $TP$ is a positive sample predicted by the model as a positive value, $TN$ is a negative sample with negative value predicted by the model, $FP$ is a negative sample predicted by the model as a positive value, $FN$ is a positive sample predicted by the model as a negative value, $AUCROC$ is the offline area of $ROC$ curve.

4.3. Results and Analysis

We compared the performance of MTAD-GAN with six popular anomaly detection methods, including: PCA: The method is based on Principal Component Analysis [50]; RawSignal: Raw Signal is a trivial model that reconstructs any signal to 0, so that the error is the same as the normalized signals; UAE: The method is based on Unimodal Autoencoder [51]; LSTM-ED: The method is based on Long Short Term Memory neural network and Encoder-Decoder [52]; AE: The method is based on Autoencoder [53]; TcnED: The method is based on Temporal Convolutional Network and Encoder–Decoder [54].

Figure 6 shows the performance comparison of this method with PCA, RawSignal, UAE, LSTM-ED, AE and TcnED on SMAP, SMD, DAMADICS, SKAB, MSL and J10031 datasets based on different scoring functions. The experimental results show that the choice of detection model and scoring function has different effects on anomaly detection. Compared with other algorithms, the performance of MTAD-GAN is improved by 0.01∼0.4, while ADGS integrates multiple loss items to better identify abnormal situations. Compared with other scoring functions, the model performance is improved by 0.16∼0.4.

Table 5. Performance comparison of the time and accuracy of different baseline methods with MTAD-GAN.

Prediction Model	Train Time/h	Predicted Time/s	A%
PCA	4.5	2.2	80.23
RS	4.3	3.8	88.25
UAE	4.5	2.3	82.61
LSTM-ED	5.1	4.5	87.52
AutoEncoder	5.6	3.1	88.71
TcnED	4.6	3.9	93.21
LSTM-GAN	5.5	4.6	76.44
MTAD-GAN	4.6	3.4	95.70

shows the prediction results based on the ADGS scoring function in the J10031 dataset. It can be seen that the accuracy of MTAD-GAN is improved by 2.6% compared with TcnED algorithm, indicating that the proposed model has good performance. To compare and analyze the differences between the methods in this paper and TcnED,

Table 6. Performance comparison of the accuracy of different datasets.

Dataset	TcnED	MTAD-GAN
SMAP	76.64	88.46
SMD	80.46	86.88
DAMADICS	80.28	87.46
SKAB	78.52	86.59
MSL	80.21	88.62
KDD99	85.72	94.57
SWAT	89.44	97.15
WADI	88.35	98.54

shows the accuracy on SMAP, SMD, DAMADICS, SKAB, MSL, SWAT and WADI datasets. It can be seen that the accuracy of MTAD-GAN algorithm is improved by 6.42∼15.42% compared with TcnED algorithm.

To verify the model’s ability for detecting abnormal data, the SWAT, WADI, MSL and J10031 datasets are evaluated for abnormal scores. It can be seen from Figure 7 that there is a significant difference between normal and abnormal distribution in the scores, and the abnormal score is higher than the normal score with the average score difference between 0.01 and 9.21, which proves that the MTAD-GAN model can reflect the characteristic distribution of data and give more accurate abnormal scores.

We introduce a scoring network to analyze the sensitivity of different weight parameters of MTAD-GAN and optimize the model. The parameters in Equation (22) are divided into two groups, namely $(\epsilon ,a)$ and $({\lambda }_{GS},{\lambda }_a)$. Figure 8 shows the AUCROC indexes used to measure the sensitivity of the two groups of parameters on the MSL and J10031 data sets respectively. It can be seen that different data sets have different sensitivity to parameters. On MSL dataset, when $a<20$ and ${\lambda }_a>4$, the correlation of parameter is strong. On the J10031 dataset, when $a<25$ and ${\lambda }_a>12$, the model performance is relatively stable.

4.4. Ablation Experiment

Experiments were conducted on SWAT, WADI, KDD99 and J10031 respectively by verifying the rationality of MTAD-GAN design. Removal of Hawkes temporal attention module is marked as K-GAN, removal of knowledge graph aware attention module is marked as H-GAN. As shown in the

Table 7. Performance comparison of the P, R, F1 of ablation experiment.

Dataset		KDD99	SWAT	WADI	J10031
LSTM-GAN	P	0.720	0.712	0.765	0.775
	R	0.796	0.720	0.711	0.784
	F1	0.844	0.722	0.624	0.753
K-GAN	P	0.780	0.745	0.782	0.842
	R	0.842	0.775	0.738	0.826
	F1	0.882	0.754	0.652	0.826
H-GAN	P	0.820	0.740	0.794	0.817
	R	0.944	0.842	0.810	0.827
	F1	0.864	0.833	0.645	0.792
MTAD-GAN	P	0.945	0.932	0.956	0.972
	R	0.960	0.955	0.931	0.912
	F1	0.965	0.890	0.796	0.904

, the average accuracy of the MTAD-GAN algorithm has increased by 13.42∼21.31%, the average recall rate has increased by 8.53∼23.45%, and the average F1 has increased by 9.49∼16.6%.

To verify the effectiveness of the loss function used in this paper, the scoring loss network is removed from MTAD-GAN and marked as MTA-GAN, and the scoring loss (i.e., LSTMS-GAN) is introduced into LSTM-GAN for retraining. The abnormal samples are randomly deleted from KDD99, SWAT, WADI and J10031 datasets, and the abnormal rates are adjusted to 5%, 10%, 15%, 20% and 25% respectively. The test results are shown in Figure 9. It can be seen that the performances of LSTMS-GAN and MTAD-GAN are relatively stable when all loss functions are used; on the other hand, the AUCROC performance of MTA-GAN and LSTM-GAN has decreased to varying degrees on the four datasets with an average performance decrease of 15∼87.5%.

5. Discussion

As shown in Table 7, LSTM combined with the Hawkes process can effectively capture the cascading correlations among historical records for the inherent temporal self-exciting characteristic of anomalies. Hence, the output of the LSTM corresponding to self influence of the Hawkes process demonstrates the accumulated results which decay exponentially with time for anomalies during historical intervals for current detection. The knowledge graph of oil and gas station for multivariate time series data, which enhance the semantic information of detecting anomalies, is constructed to capture the latent inter-deependencies between entities and relations. Then, the anomaly information related to the occurrence of sensors is transformed into the feature vector to mine the potential connections among multi-sensors. The feature vectors are used for training LSTM model to improves the ability of high-level feature expression and the accuracy of anomaly detection.

The novel loss functions including reconstruction loss function, discrimination loss function and score loss function is designed to constrain the hidden variable and accelerate the gradient descent speed with few iterations during the training process. The highest results of AUCROC shown in Table 6 demonstrate the efficiency of our MTAD-GAN method with ADGS to improve the accuracy of detecting anomalies for the imbalanced and high-dimensional data in oil and gas stations.

6. Conclusions

Anomaly detection of multivariate time series is an important task for intelligent operation of oil and gas stations. In this paper, an unsupervised multivariate time series anomaly detection method based on digital twin and MTAD-GAN is proposed, which constructs a digital twin system matching with the actual oil and gas station operation logic and realizes an efficient virtual-real mapping through stochastic Petri nets. Based on the correlation between data samples and using twin space for training, the data of multivariate time series are modeled simultaneously by combining knowledge graph attention and temporal Hawkes attention mechanisms, and the probability distribution of the estimated network learning samples is designed based on knowledge-aware transfer learning model to complete anomaly detection. In the future work, we intend to further explore the various implicit relationships of variate time series and adjust the network structure of the knowledge mapping and Hawkes process modules to adapt to more complex and variable application contexts by fusing more levels of feature information.