SCConv-Denoising Diffusion Probabilistic Model Anomaly Detection Based on TimesNet

Abstract

Time series anomaly detection is a significant challenge due to the inherent complexity and diversity of time series data. Traditional methods for time series anomaly detection (TAD) often struggle to effectively address the intricate nature of a complex time series and the composite characteristics of diverse anomalies. In this paper, we propose SCConv-Denoising Diffusion Probabilistic Model Anomaly Detection Based on TimesNet (SDADT), a novel framework that integrates the Spatial and Channel Reconstruction Convolution (SCConv) module and Denoising Diffusion Probabilistic Models (DDPMs) to address these challenges. By transforming 1D time series into 2D tensors via TimesNet, our method captures intra- and inter-period variations, achieving state-of-the-art performance across three real-world datasets: 85.39% F1-score on SMD, 92.76% on SWaT, and 97.36% on PSM, outperforming nine baseline models including Transformers and LSTM. Ablation studies confirm the necessity of both modules, with performance dropping significantly when either SCConv or DDPMs are removed. In conclusion, this paper proposes a novel alternative solution for anomaly detection in the Cyber Physical Systems (CPSs) domain.

1. Introduction

As the number of networked equipment and sensors in Cyber Physical Systems (CPSs)—such as data centers, industrial systems, and automobiles—increases rapidly, there is a growing need to keep monitoring this equipment to prevent attacks [1]. CPSs have grown significantly over the last 20 years. Not only have more critical infrastructures, such as industrial power plants, banking systems, and other essential service providers, embraced CPS technologies, but hacking activities have also become more sophisticated [2]. Massive volumes of time series data are produced by numerous interconnected sensors found in many real-world systems. For instance, each of the numerous components of a water treatment facility has sensors that measure various parameters, such as the level of water, velocity, and quality of water. However, these sensors may encounter problems such as nonlinear responses, sensor drift, or external interference, leading to anomalous data [3]. Therefore, automated anomaly detection methods are crucial for ensuring the stability and safety of industrial systems.

Detecting anomalies in such data is crucial for ensuring system safety and operational efficiency. Anomaly detection models targeting 1D data may encounter certain limitations and challenges in their practical implementation [4]. Canizo et al. proposed a multi-head CNN–RNN architecture for detecting multivariate time series anomalies [5]. Their experimental results show the effectiveness of using a hybrid architecture that combines the strengths of CNN, RNN, and LSTM for tasks with both spatial and temporal dependencies. In this approach, RNN is utilized to concurrently learn temporal patterns after CNN has extracted significant features from raw data. However, 1D data remain susceptible to noise interference, which may complicate feature engineering and negatively impact the stability and accuracy of the model. Additionally, due to the locality of 1D convolution kernels, they are only capable of modeling changes between adjacent time points, thus failing to capture long-term dependencies. Using the Transformer architecture, Kim et al. presented a prediction-based unsupervised time series anomaly detection technique [6]. The prediction-based approach forecasts the values of subsequent time steps and identifies temporal anomalies based on the prediction errors. The model might have trouble detecting trustworthy dependencies from sparse temporal observations and capturing underlying patterns in the data in situations where there is a lack of data. The main reason for this is that temporal dependencies are frequently completely hidden within intricate temporal patterns. To address the challenge of extracting features from 1D data, Haixu Wu et al. proposed a method that involves converting a 1D time series into a set of 2D tensors [7]. In order to create a 2D representation of the time series, this method successfully unifies the intra-period and inter-period variations of the time series into a 2D space, overcoming the representation capability limitations in the original 1D space. However, such approaches often neglect two critical aspects: (1) the cumulative impact of noise on feature extraction, and (2) the redundancy in spatial-channel features introduced by multi-scale convolutions.

Therefore, we propose SCConv-Denoising Diffusion Probabilistic Model Anomaly Detection Based on TimesNet (SDADT) in this study. Specifically, the method uses TimesBlock to adaptively transform a 1D time series into a set of 2D tensors based on learned periodicity. Furthermore, intra- and inter-period fluctuations in the two-dimensional space can be further captured by a parameter-efficient inception block. The technique shows outstanding performance across several datasets by combining DDPMs and SCConv to produce clearer samples and better capture multi-scale characteristics. The following is a summary of our work’s primary contributions:

• We integrated Denoising Diffusion Probabilistic Models (DDPMs) into TimesNet’s architecture. This enables the iterative refinement of noisy input representations during training, enhancing robustness against sensor drift and adversarial perturbations.
• We designed a lightweight Spatial and Channel Reconstruction Convolution (SCConv) module that dynamically suppresses redundant features. This integration enables more effective multi-scale feature extraction, enhances the ability to capture multi-scale data characteristics, and improves anomaly detection performance.
• SDADT jointly optimizes periodicity modeling (via TimesBlock), noise resilience (via DDPMs), and feature efficiency (via SCConv) and demonstrates that our method achieves promising performance in anomaly detection.

The rest of this article is organized as follows: Section 2 reviews the related work in this area. Section 3 provides a detailed description of the proposed method. Section 4 presents the experiments and discusses the results. Section 5 concludes this paper.

2. Related Work

In this section, we give a quick summary of the relevant research that has been conducted recently on anomaly detection in time series data.

Before CPSs were widely adopted in industrial, transportation, and other societal infrastructure sectors, many researchers had already conducted extensive studies on anomaly-based attack detection in CPSs. These anomaly-based detection methods can be broadly classified into three categories [8]:

• Traditional Detection Methods: Conventional time series anomaly detection approaches, which build normal operating models of systems or data to find anomalies based on deviations, mostly rely on statistical and rule-based analytical techniques. For example, Rigatos et al. used a derivative-free nonlinear Kalman filter to estimate system states, measured the deviation between observed and estimated values, and set a confidence interval to recognize attacks [9].
• Machine Learning Detection Methods: These methods detect anomalies or attacks by learning the system’s behavior under normal conditions, even when such behaviors are not included in prior knowledge. In order to detect anomalous traffic patterns with a low false negative rate, Hao et al. created a hybrid machine learning algorithm that combines a dynamic threshold model based on the seasonal autoregressive integrated moving average (SARIMA) with an LSTM model [10].
• Deep Learning Detection Methods: Deep learning’s benefit for CPS attack detection is its capacity to glean useful features from intricate, extensive, and multimodal data, increasing the precision and robustness of identifying different kinds of attacks. By using GANs to train anomaly detection models and recognize network attack behaviors in intricate multisensor industrial networks, Freitas P et al. presented a novel framework to detect malicious behavior in CPSs [11].

Deep learning has emerged as a powerful technique for handling the complexity of CPSs. This approach has applications in CPSs for fault identification, control, optimization, and anomaly detection [12]. CPS applications have made extensive use of deep learning, specifically convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep belief networks (DBNs) [8]. According to their optimized feature representations, Zhou X et al. proposed creating a Siamese CNN encoding network to calculate the distance between input samples [13]. To improve the effectiveness of the training procedure, they subsequently presented a strong cost function design that incorporates three distinct loss functions. Lastly, a clever algorithm for detecting anomalies was suggested. The experiment’s findings demonstrated that this strategy significantly raised the F1-score and false alarm rate (FAR). Akowuah F et al. suggested a real-time adaptive sensor attack detection framework that can dynamically modify the false alarm rate and detection latency based on changing system variables in order to satisfy detection deadlines and enhance usability [14]. J Li et al. proposed SCConv, a method made up of two reconstruction units. While the CRU uses a split-conversion–fusion strategy to reduce channel redundancy, the SRU suppresses spatial redundancy through separation and reconstruction [15]. In order to prove the effectiveness and efficiency of the suggested framework, real sensor data from an automotive CPS were used for implementation and validation. The experimental results show that models incorporating SCConv improve performance while significantly reducing complexity and computational cost. Finally, the proposed framework was implemented and validated with real sensor data from an automotive CPS to show its efficiency and effectiveness.

Although CPS data are frequently limited and costly to collect, deep learning models need vast amounts of data for training [4]. To overcome this difficulty, transfer learning techniques have been used to take advantage of pre-trained models [16]. Deep learning methods can also be utilized to improve CPS security. For example, autoencoders can be used to detect intrusions, and Generative Adversarial Networks (GANs) can be used to create adversarial examples, increasing CPS resilience [17]. Deep learning techniques based on GANs have recently surfaced for the detection of time series anomalies [17]. However, these approaches require identifying the best mapping between the real-time and latent spaces during the anomaly detection stage, which adds more time and introduces new errors. Niu Zijian et al. proposed a time series anomaly detection method based on an LSTM Variational Autoencoder Generative Adversarial Network (VAE-GAN), which successfully resolves the previously mentioned problems [18]. This approach uses the discriminator’s discriminative power and the encoder’s mapping capability to train the discriminator, generator, and encoder together. LSTM networks serve as the discriminator, generator, and encoder in this technique. Reconstruction differences and discriminator findings are used to identify anomalies during anomaly detection. The experimental results demonstrate that this method can quickly and accurately identify anomalies. Recently, DDPMs, as part of a new family of deep generative models, have emerged and outperformed autoencoders and GANs in image synthesis challenges [19]. All things considered, deep learning has demonstrated encouraging outcomes in a number of CPS applications and is anticipated to be essential in developing the newest CPS technologies.

However, the aforementioned methods ignore the limitations of 1D data in deep learning. Complex feature expression, multi-scale anomaly perception, noise robustness, and visualization are all challenges with 1D time series data. Furthermore, using image processing methods in deep learning for such data is challenging. Converting 1D data into a 2D tensor allows for the extraction of spatial features using methods like convolution, the disclosure of hidden spatiotemporal features, and an improvement in noise robustness and detection accuracy. Additionally, 2D tensors are easier to visualize and interpret, which improves anomaly detection’s usability and efficacy. As a result, the model can detect intricate abnormal patterns with greater accuracy.

3. Materials and Methods

In this section, we propose SCConv-Denoising Diffusion Probabilistic Model Anomaly Detection Based on TimesNet (SDADT) for anomaly detection in time series data. The overall architecture is illustrated in Figure 1. In SDADT, (1) the TimeBlock is utilized to transform 1D data into a 2D tensor. (2) The framework incorporates DDPMs and SCConv modules, where DDPMs are employed for generation and denoising, and SCConv is used for multi-scale feature extraction. (3) Finally, MSELoss is applied to compute the anomaly scores, enabling the detection of anomalous instances.

3.1. TimesNet

In time series analysis, periodicity (or seasonality) is a prominent feature of the data, particularly in application scenarios with pronounced cyclical fluctuations, such as power demand analysis, sensor data, and other similar contexts. The utilization of periodic features helps improve the modeling accuracy of time series data. Traditional time series models, such as ARIMA and LSTM, can handle some periodicity but often perform poorly when faced with complex or multiple periodic fluctuations. These models typically only capture single or simple cycles, limiting their ability to model more intricate patterns. The periodicity in time series data can exhibit multiple scales, and these cycles may overlap or vary over time. Traditional models struggle to simultaneously capture these multi-scale periodic patterns, as they are typically designed to model simpler or single-cycle behaviors. In contrast, TimesNet views every time point in the time series as reasonably independent. On the one hand, it emphasizes inter-period differences by accounting for overlapping time segments to capture interactions between segments. Conversely, it examines the connections between distinct time points within each segment, emphasizing intra-period variations—differences between neighboring times within the same period. Multi-periodic variations in a 1D time series often overlap, and direct handling of such data may lead to confusion. However, by transforming the data into a 2D tensor, different periods can be clearly separated for individual processing. Once the time series is converted into a 2D tensor, the model can process the data in a manner similar to image processing. This improves the model’s capacity to represent both local and global features by making it easier to employ two-dimensional convolution and other methods to capture intricate spatiotemporal relationships. To achieve the aforementioned effect, TimesNet converts the 1D time series into a 2D tensor, as shown in Figure 2. Time points within a single period are contained in each column, while time points at the same phase over multiple periods are represented by each row.

The aforementioned procedure is integrated into TimesNet’s TimesBlock, which models both kinds of variations efficiently by using a parameter-efficient beginning block. Below is a detailed description of the TimesNet configuration. Examining the time series’ periodicity is essential to accurately capturing and illustrating the temporal differences within and between periods. The Fast Fourier Transform (FFT), represented as $X_p\in R^{T\ast d_{model}}$, can be applied along the time dimension of the 1D time series to achieve this. Here, T represents the time series’ length, and $d_{model}$ represents the dimensionality of the input features. Using FFT on the time dimension aids in recognizing periodic patterns in the time series. Here is the procedure:

$$A=Avg(Amp(FFT(X_{1D})),\left\{\begin{array}{c}{f}_1,\dots ,f_k\end{array}\right\}{=}_{f_{*}\in \left\{\begin{array}{c}1,\dots ,[{\displaystyle \frac{T}{2}}]\end{array}\right\}}^{argTopk}(A),p_i=[{\displaystyle \frac{T}{f_i}}],i\in \left\{\begin{array}{c}1,\dots ,k\end{array}\right\},$$

(1)

where $A\in R^T$ represents the amplitude of each frequency component in $X_{ip}$. The highest k frequencies $\left\{\begin{array}{c}{f}_1,\dots ,f_k\end{array}\right\}$ correspond to the most significant periodic lengths of k $\left\{\begin{array}{c}{p}_1,\dots ,p_k\end{array}\right\}$ in the time series. Considering conjugate symmetry in the frequency domain, we restrict our analysis to frequencies within the range $\left\{\begin{array}{c}1,\dots ,{\displaystyle \frac{T}{2}}\end{array}\right\}$. Equation (1) can be summarized as follows:

$$A,\left\{\begin{array}{c}{f}_1,\dots ,f_k\end{array}\right\},\left\{\begin{array}{c}{p}_1,\dots ,p_k\end{array}\right\}=Period(X_{1D}).$$

(2)

As illustrated in Figure 3, the original one-dimensional time series $X_{1D}$ is compressed according to the selected periods. This process can be formally expressed as follows:

$$X_{2D}^i={Reshape}_{p_i,f_i}(Padding(X_{1D})i\in \left\{\begin{array}{c}1,\dots ,k\end{array}\right\},$$

(3)

where $Padding()$ is the procedure that adds zeros to the end of the sequence in order to make sure that $p_i$ divides its length. We obtain a series of 2D tensors $\left\{\begin{array}{c}{X}_1^{2D},X_2^{2D},\dots ,X_k^{2D}\end{array}\right\}$ by carrying out this operation, where each $X_i^{2D}$ represents the 2D temporal variations dominated by the period $p_i$.

TimeBlock

The architecture of TimesNet is shown in Figure 4. In the lth layer of the TimesBlock, the input is $X_{1D}^{l-1}\in R^{T\ast d_{model}}$, which can be processed using 2D convolution to extract 2D temporal variations. This process can be represented by the following formula:

$$X_{1D}^l=TimesBlock(X_{1d}^{l-1})+X_{1D}^{l-1}.$$

(4)

Below is a detailed explanation of the full TimesBlock procedure for collecting the 2D temporal fluctuations of a time series. It mostly consists of the four subprocesses listed below:

(1)

Transforming the 1D time series into a 2D tensor.

The first step is to extract and transform the periodicity of the input 1D time feature $X^{\prime }$ into a 2D tensor that will represent the 2D temporal fluctuations. The following is a rewrite of the formula based on the previously discussed procedure:

$$A^{l-1},\left\{\begin{array}{c}{f}_1,\dots ,f_k\end{array}\right\},\left\{\begin{array}{c}{p}_1,\dots ,p_k\end{array}\right\}=Period(X_{1D}^{l-1}),$$

(5)

$$X_{2D}^{l,i}={Reshape}_{p_i,f_i}(Padding(X_{1D}^{l-1}))i\in \left\{\begin{array}{c}1,\dots ,k\end{array}\right\}.$$

(6)

(2)

Capturing the 2D temporal variation representation.

We can use 2D convolution to extract information from the 2D tensors with 2D locality $\left\{\begin{array}{c}{X}_{2D}^{l,1},X_{2D}^{l,2},\dots ,X_{2D}^{l,k}\end{array}\right\}$. In this case, we use the traditional inception model.

$${\widehat{X}}_{2D}^{l,i}=Inception(X_{2D}^{l,i})$$

(7)

(3)

Converting the 2D tensor into a 1D space.

To aggregate the information, we first capture the temporal features and then transform them back into a 1D space.

$${\widehat{X}}_{1D}^{l,i}=Trunc(Reshap{e}_{1,(p_i\ast f_i)}({\widehat{X}}_{2D}^{l,i}))i\in \left\{\begin{array}{c}1,\dots ,k\end{array}\right\},$$

(8)

where $Trunc(·)$ indicates the process of eliminating the zeros added during the $Padding(·)$ operation in step (1), and ${\widehat{X}}_{1D}^{l,i}\in R^{T\ast d_{model}}$.

(4)

Adaptive Aggregation

As in Autoformer [20], the adaptive fusion mechanism generates the final output by performing a weighted sum of the 1D representations obtained $\left\{\begin{array}{c}{\widehat{X}}_{l,1},\dots ,{\widehat{X}}_{l,k}\end{array}\right\}$ according to their corresponding frequency values.

$${\widehat{A}}_{f_1}^{l-1},\dots ,{\widehat{A}}_{f_k}^{l-1}=Softmax(A_{f_1}^{l-1},\dots ,A_{f_k}^{l-1})$$

(9)

$$X_{1D}^l={\sum }_{i=1}^k{\widehat{A}}_{f_i}^{l-1}\ast {\widehat{X}}_{1D}^{l,i}$$

(10)

The TimesBlock can record multi-scale and fine-grained 2D temporal patterns at the same time. Thus, TimesNet achieves more efficient representation learning than directly extracting features from a 1D time series.

3.2. DDPMs

A distribution $p_{\theta }(x)$ that approximates the initial data distribution $q(x)$ of two Markov chain processes is what DDPMs aim to learn. Over a number of steps, noise is gradually added to the input in the forward process. In the opposite procedure, the input is recovered by eliminating the noise that a neural network predicted.

Gaussian noise is gradually injected into the initial input, resulting in a series of latent variables $x_t^{\prime }$ at each diffusion step $t=1,2,\dots ,T$. Each step is controlled by a noise schedule function ${\left\{\begin{array}{c}\beta \_t\in (0,1)\end{array}\right\}}_{t=1}^T$, which can be mathematically expressed as

$$q(x^t|x^{t-1})=N(x^t;\sqrt{1-{\beta }_t}{x}^{t-1},{\beta }_{t}I),$$

(11)

$$q(x^{1:T}|x^0)={\prod }_{t=1}^{T}q(x^t|x^{t-1}).$$

(12)

Given $x^0$, we can sample $x^t$ at any time step in a closed form:

$$x^t=\sqrt{{\overline{\alpha }}_t}{x}^0+\sqrt{1-{\overline{\alpha }}_t}ϵ,withϵ\backsim N(0,I),$$

(13)

where ${\alpha }_t=1-{\beta }_t$ and ${\overline{\alpha }}_t={\prod }_{s=1}^t{\alpha }_s$.

Starting from $X_T\backsim N(0,1)$, we sample from $q(x_{t-1}|x_t)$ over T steps to remove the noise until $x_0$ is reconstructed. For a sufficiently small ${\beta }_t$, $q(x_{t-1}|x_t)$ approximates a Gaussian distribution, and it can be parameterized by the model $p_{\theta }$. This reverse process can be formulated as follows:

$$p_{\theta }(x^{t-1}|x^t)=N(x^{t-1};\mu (x^t,t),{\beta }_{\theta }(x^t,t)),$$

(14)

$$p_{\theta }(x^{0:T})=p(x^T){\prod }_{t=1}^T{p}_{\theta }(x^{t-1}|x^t).$$

(15)

According to Bayes’ theorem, the posterior distribution takes the form of

$$q(x^{t-1}|x^t)=N(x^{t-1};{\tilde{\mu }}_t(x^t),{\tilde{\beta }}_t(x^t)I),$$

(16)

with

$${\tilde{\mu }}_t(x^t)={\displaystyle \frac{1}{\sqrt{{\alpha }_t}}}(x^t-{\displaystyle \frac{\beta }{\sqrt{1-{\overline{\alpha }}_t}\epsilon }}\epsilon ),$$

(17)

$$\widetilde{{\beta }_t}(x^t)=({\displaystyle \frac{1-{\overline{\alpha }}_{t-1}}{1-{\overline{\alpha }}_t}}){\beta }_t.$$

(18)

We train the neural network ${\epsilon }_{\theta }$ to predict $\epsilon$ at each time step t using $x^t$ as the input. The current diffusion time step is specified by sinusoidal positional embeddings. The training objective can be described as optimizing the negative log-likelihood [21], as follows:

$${\displaystyle L=E_q[D_{KL}(q(x^T|x^0)||p_{\theta }(x^T))+\sum _{t=2}^T{D}_{KL}(q(x^{t-1}|x^t,x^0)||p_{\theta }(x^{t-1}|x^t))-{log}_{p_{\theta }}(x^0|x^1)].}$$

(19)

After simplification, the training loss is defined as minimizing:

$$L_{simple}(\theta )=E_{t,x^0,\epsilon }{[||\epsilon -{\epsilon }_{\theta }(\sqrt{\overline{{\alpha }_t}}{x}^0+\sqrt{1-{\overline{\alpha }}_t}\epsilon ,t)||}^2].$$

(20)

3.3. SCConv

As shown in Figure 5, the two units that make up SCConv are the Channel Reconstruction Unit (CRU) and the Spatial Reconstruction Unit (SRU). Whereas the CRU reduces channel redundancy by using a segmentation–transformation–fusion strategy, the SRU suppresses spatial redundancy by using a separation–reconstruction approach. Combining SCConv with TimesNet improves performance by cutting down on redundant features, drastically lowering computational cost and complexity, and successfully enhancing anomaly detection performance.

For SRU, it utilizes separation and reconstruction operations: First, Group Normalization (GN) is used to normalize the input features [22]. Then, the trainable parameters $\gamma$ are re-weighted, and the resulting weights are processed through the $sigmoid$ function to map them to the range [0, 1]. Weights greater than 0.5 are considered information weights $W_1$, while the remaining weights are considered non-informative weights $W_2$. Specifically, as shown in Equation (21):

$$W=Gate(Sigmod(W_{\gamma }(GN(X)))).$$

(21)

Subsequently, the input features X are multiplied by $W_1$ and $W_2$, respectively, resulting in $X_1^w$ with rich feature information and $X_2^w$ with redundant information. These two parts then undergo cross-reconstruction operations to obtain spatially refined features. In the case of the CRU, the feature map $X_w$ is initially divided into two channels, $\alpha C$ and $(1-\alpha )C$, and is then compressed using a convolution $1\times 1$ to improve computational efficiency. One set of features serves as a rich feature extractor and undergoes GWC (Group-wise Convolution) with $g=2$ and a $1\times 1$ operation called PWC (Point-wise Convolution), resulting in a mapping $Y_1$ with rich features. This operation enhances the overall feature extraction process and helps reduce computational complexity while maintaining or improving feature richness for anomaly detection tasks. The other set of features uses $1\times 1$ PWC to extract shallow features, complementing the rich feature extractor, resulting in the mapping $Y_2$. Finally, both $Y_1$ and $Y_2$ undergo global average pooling to reduce dimensionality and the number of parameters, producing the outputs $S_1$ and $S_2$. This process is mathematically represented as

$$S_n=\frac{1}{H\times W}{\sum }_{i=1}^H{\sum }_{j=1}^W{x}_c(i,j)n=1,2.$$

(22)

Then, through a channel-wise soft attention operation, feature vectors ${\beta }_1$ and ${\beta }_2$ are obtained. These two feature vectors are used to combine $Y_1$ and $Y_2$ along the channel dimension, thereby obtaining the channel-refined features. This process is mathematically represented as

$${\beta }_1={\displaystyle \frac{e^{S_2}}{e^{S_1}+e^{S_2}}},{\beta }_1+{\beta }_2=1,$$

(23)

$$Y={\beta }_1{Y}_1+{\beta }_2{Y}_2.$$

(24)

3.4. Anomaly Score

The input time series data are passed through the model to generate a reconstructed output, obtaining the predicted reconstruction values. The actual values at each time point are then compared with the reconstructed values, and an anomaly score is calculated. The Mean Squared Error (MSE) formula, which is a measure of the reconstruction error, is used to calculate the anomaly score, as indicated in Equation (25):

$$score(x)=\frac{1}{n}{\sum }_{i=1}^n{(x_i-{\widehat{x}}_i)}^2,$$

(25)

where $x_i$ is the actual value, ${\widehat{x}}_i$ is the reconstructed value, and n is the sequence length.

In this error distribution, most of the normal data points have a small MSE, while anomalous data points typically have a higher MSE. Therefore, the error distribution reflects the normal and anomalous patterns in the data. An anomaly is defined as one where the anomaly score is greater than a predetermined threshold; if the anomaly score is less than the threshold, it is considered normal.

4. Experiment

4.1. Dataset

The SWaT, PSM, SMD, and SMAP are commonly used datasets for anomaly detection in CPSs due to their relevance in real-world industrial and server monitoring applications. SWaT (Secure Water Treatment) focuses on water treatment systems, while the PSM (Prognostics Server Machine) includes predictive maintenance and system management data. The SMD (Server Machine Dataset) provides data on server health and performance. The SMAP (Soil Moisture Active Passive) dataset is a high-resolution, satellite-derived collection of global soil moisture measurements, designed to monitor and analyze terrestrial water cycles. These datasets are particularly valuable because they contain labeled anomalies and reflect the complex, dynamic behaviors typical in industrial and server environments.

Table 1. Statistics for the three datasets utilized in the experiments.

Dataset	#Features	#Train	#Test	Anomaly%
SWaT	51	495,000	449,919	12.10%
PSM	25	132,481	87,841	27.80%
SMD	38	708,405	708,420	4.2%
SMAP	25	135,183	427,617	13.13%

summarizes the statistics of the three datasets, covering server monitoring as well as water treatment applications:

• SWaT (Secure Water Treatment) is a CPS cyber security research testbed that replicates an actual industrial water treatment system [23]. Developed by the Singapore University of Technology and Design (SUTD), it aims to study potential cyber attacks and defense measures in critical infrastructure.
• The PSM (Prognostics Server Machine) dataset is a time series dataset for server machine metrics, mainly used for anomaly detection tasks [24]. It was collected by eBay and contains server performance indicators recorded every minute over a period of 21 weeks.
• The SMD (Server Machine Dataset) is designed for multivariate time series anomaly detection [25]. It comprises system performance metrics collected by sensors from servers during their regular operation. This dataset is commonly used in both industry and academia for research related to server anomaly detection.
• The SMAP (Soil Moisture Active Passive) was launched by NASA in January 2015; the SMAP mission combines an L-band radar and radiometer to provide accurate, frequent observations of soil moisture at a spatial resolution of approximately 9 km (radiometer) and 3 km (radar) [26]. The dataset is mainly used for tasks such as remote sensing monitoring and anomaly detection.

4.2. Baselines

We evaluate our suggested approach’s performance against nine well-known anomaly detection models, such as the following:

• LSTM [27]: LSTM is used in anomaly detection by learning normal patterns from time series data. It predicts the next step or reconstructs the input, and anomalies are identified when the prediction or reconstruction error exceeds a predetermined threshold.
• Transformers [28]: Transformers model global dependencies in time series data and identify anomalies by using the self-attention mechanism. Anomalies are identified when the prediction error or reconstruction error of the sequence exceeds a predefined threshold after learning the patterns of normal data.
• LogTrans [29]: LogTrans uses a logarithmic sparse self-attention mechanism to capture time series dependencies both locally and globally. By learning the normal patterns of the data, it performs anomaly detection when the prediction error exceeds a predefined threshold.
• Reformer [30]: To manage a lengthy time series, Reformer employs an effective locality-sensitive hashing self-attention mechanism. By comparing the difference between the actual and predicted values, it learns the typical patterns of behavior and identifies anomalies.
• Pyraformer [31]: Pyraformer efficiently captures multi-scale dependencies in a time series through its pyramid-structured attention mechanism. This enables the model to identify anomaly patterns when processing a long time series, facilitating effective anomaly detection.
• ETSformer [32]: ETSformer detects anomalies by decomposing the time series into three components: level, trend, and seasonality. Anomalies are identified by recognizing abnormal changes in these components. If certain patterns in the data deviate from the expected level, trend, or seasonal behavior, they are flagged as anomalies, enabling effective anomaly detection.
• LightTS [33]: LightTS captures anomaly patterns in a time series efficiently through its self-attention mechanism and multi-scale modeling.
• Dlinear [34]: Dlinear breaks down time series data into trend and seasonal components in order to identify anomalies. It then applies linear models to process these components separately. By learning the linear relationships within the time series, Dlinear identifies anomaly points that deviate from the normal trend.
• CARLA [35]: CARLA utilizes self-supervised contrastive learning to generate embedding representations of a time series. It maximizes the similarity between normal data and minimizes the difference for anomalous data, thereby enabling effective anomaly detection.

4.3. Evaluation Metrics

We evaluate the performance of SDADT using the most common metrics, including $Precision,Recall$, and $F1Score$:

$$Precision={\displaystyle \frac{TP}{TP+FP}},$$

(26)

$$Recall={\displaystyle \frac{TP}{TP+FN}},$$

(27)

$$F1Score={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}},$$

(28)

where $TP,FN,$ and $FP$ stand for true positives, false negatives, and false positives, respectively.

4.4. Data Preprocessing

To increase robustness and speed up convergence, we clean and normalize the data before training the model. Data normalization is used on both the training and test sets, as shown below:

$${\tilde{x}}_i={\displaystyle \frac{x_i-min(x_i)}{max(x_i-min(x_i)}},$$

(29)

where $x_i$ represents the ith univariate time series. $max(x_i)$ and $min(x_i)$ represent the maximum and minimum values of $x_i$, respectively. Both reconstruction-based and prediction-based models are susceptible to anomalous and erratic occurrences in the training set. In order to address this issue, we use Spectral Residual (SR) [36], a cutting-edge univariate anomaly identification technique, to find anomalous timestamps in every time series in the training data. We then complete data cleaning by replacing the values of the anomalous timestamps with the surrounding normal values.

4.5. Experimental Setup

The methods and their variants mentioned in this paper were implemented using CUDA 12.7 and PyTorch 12.1 in Python 3.8, and the system used for the experiments had a 12th Gen Intel^® Core™ i5-12400F @ 2.50 GHz CPU and an NVIDIA GeForce RTX 4060 Ti GPU. Following Anomaly Transformer’s preprocessing method, we used a sliding window to segment the dataset into continuous, non-overlapping fragments. In previous research, reconstruction was a traditional task for unsupervised point-wise representation learning, and the standard for anomaly detection was the reconstruction error. We employed MSE as the common anomaly detection criterion for all experiments and only altered the base model for reconstruction in order to provide a fair comparison. Regarding hyperparameters, different datasets corresponded to different parameter settings, as shown in

Table 2. Experiment configuration of SDADT.

Dataset	Epoch	Batch_Size	k	Seq_Len	Group Number	Image_Size
SWaT	3	128	5	15	4	128
PSM	3	128	3	15	4	128
SMD	10	128	3	75	4	64
SMAP	3	128	1	50	4	64

. Here, k represents the variable in Equation (1), $seq\_len$ denotes the length of the input time series for the model, and the group number denotes the number of groups (g) for Group-wise Convolution in the SCConv module.

4.6. Overall Performance

To compare the performance of different models, this paper adopts the widely used Point Adjustment (PA) strategy [37]. The core of this strategy is to adjust errors or biases in the detection results to reduce false positives or false negatives. Particularly when dealing with time-series data, the point-adjustment strategy can optimize the output of detected points.

This paper compares the proposed method with nine baseline models across three real-world datasets, as shown in

Table 3. Anomaly detection accuracy in terms of precision (%), recall (%), and F1-score, on four datasets with ground-truth labeled anomalies. The best results are in bold, and the second-best is indicated by underline.

Method	SMD			SWaT			PSM			SMAP			Avg (F1) (%)
	P	R	F1	P	R	F1	P	R	F1	P	R	F1
LSTM	78.52	65.47	71.41	78.06	91.72	84.34	69.24	99.53	81.67	91.06	57.49	70.48	76.98
Transformer	83.58	76.13	79.56	68.84	96.53	80.37	62.75	96.56	76.07	89.37	57.12	69.70	76.43
LogTrans	83.46	70.13	76.21	68.67	97.32	80.52	63.06	98	76.74	89.15	57.59	69.97	75.86
Reformer	82.58	69.24	75.32	72.5	96.53	82.8	59.93	95.38	73.61	90.91	57.44	70.40	75.53
Pyraformer	85.61	80.61	83.04	87.92	96	91.78	71.67	96.02	82.08	92.54	57.71	71.09	81.99
ETSformer	87.44	79.23	83.13	90.02	80.36	84.91	99.31	85.28	91.76	92.25	55.75	69.50	82.325
LightTS	87.1	78.42	82.53	91.98	94.72	93.33	98.37	95.97	97.15	92.58	55.27	69.21	85.56
Dlinear	83.62	71.52	77.1	80.91	95.3	87.52	98.28	89.26	93.55	92.32	55.41	69.26	81.86
CARLA	67.57	84.65	75.15	98.91	71.32	82.88	98.33	95.93	97.18	92.03	58.10	71.23	81.61
SDADT (ours)	88.25	82.8	85.39	92.38	93.15	94.00	98.66	97.65	98.15	93.60	57.70	71.39	86.73

. In all three datasets, SDADT performs better in terms of precision than almost all baselines. In particular, it performs best on SMD and only marginally worse than one baseline on the other two datasets. In terms of recall, although it does not achieve the best results among the baselines, it lags behind the best baseline by an average of only about 3.6%. With the exception of SWaT, where it performs only 0.58% worse than the best baseline, our model offers the best detection performance from the standpoint of the F1-score on the majority of datasets. The F1-score and precision performance are the best in the SMAP dataset, while the recall performance is the second best. The experimental findings verify that adding convolution operations to DDPMs improves performance in terms of F1-score, precision, and recall.

4.7. Ablation Study

To investigate the necessity of each component in our proposed method and ensure fairness in the ablation study, we progressively excluded these components while keeping the same hyperparameter settings to observe how the model performance degraded. Our main modifications include removing the DDPM module (denoted as w/o DDPM), removing the SCConv module (denoted as w/o SCConv), and removing both the DDPM and SCConv modules simultaneously (denoted as w/o DDPM and SCConv). The results are shown in Figure 6. From the observations, the conclusions can be drawn as follows:

When the SCConv module was removed, a significant drop in performance was observed. This phenomenon highlights the importance of capturing feature relevance, and SCConv is capable of capturing multi-scale features. It also demonstrates that the combination of the DDPM module and the convolution module effectively improves experimental performance. The combination shows a more noticeable improvement in the PSM dataset, indicating that it is more effective for anomaly detection in datasets with complex multivariate dependencies, noise, and short-term and long-term features, as well as irregular change patterns.

4.8. Hyperparameter Sensitivity Analysis

To investigate the sensitivity of the model to hyperparameters, this section selects two hyperparameters to study the hyperparameter sensitivity of SDADT.

We introduce the hyperparameter k in Equation (1) to select the most significant frequencies. The parameter k determines the number of frequency components selected, specifically choosing the k components with the highest amplitudes as the primary periodic features of the sequence. Figure 7 illustrates the experimental results of the model under different values of k while keeping other hyperparameters consistent. As shown in the figure, the model consistently maintains stable F1-scores across different datasets. The model’s capacity to learn from the data distribution is improved by the newly introduced DDPM module, which produces samples that roughly reflect the original data distribution through the diffusion process. In the meantime, the expression is enhanced in both spatial and channel dimensions, and feature redundancy is successfully reduced by the SCConv module. The inclusion of these modules makes the model more robust in handling input features, thereby reducing the model’s sensitivity to the hyperparameter k.

Image_size is an important parameter in DDPMs for adjusting the reconstructed image size. Figure 8 shows that both excessively large and small values of image_size negatively affect the performance of anomaly detection. This indicates that when the image_size is too small, the representation of the data after converting them into a 2D tensor may be overly compact, leading to the insufficient representation of some important time series features. For example, some long-period fluctuations or trend information may be compressed or lost in tensors with small sizes, making it difficult for the model to capture the full patterns in the data, thereby affecting the accuracy of anomaly detection. As the image_size increases, it significantly raises computational costs and model complexity. This can lead to excessively long training times, higher memory requirements, and even issues such as gradient vanishing or explosion, ultimately affecting the model’s training performance and stability.

As shown in Figure 9, the optimal seq_len is inherently tied to the temporal scale of anomalies and the trade-off between context integration and noise suppression. Adaptive seq_len selection is guided by anomaly duration.Future work could explore dynamic sequence length tuning based on real-time anomaly characteristics.

5. Conclusions

In this study, we innovatively combined TimesNet, DDPM, and SCConv to construct a novel model architecture for anomaly detection. We compared this model with a variety of sophisticated baseline models and thoroughly assessed its performance in anomaly detection tasks using rigorous and extensive experiments on several publicly available datasets. The experimental findings clearly show that the suggested model performs noticeably better in anomaly detection. In particular, it outperforms baseline models in important performance metrics like precision, recall, and F1-score. The findings of this study provide a new and effective solution for anomaly detection, which is expected to play an important role in practical applications. However, our research also has certain limitations. For example, the model may face challenges related to computational resources and time costs when handling extremely large datasets. Due to the large number of features in large datasets, it is difficult to quickly and accurately obtain features, resulting in longer training time. We may be looking for new methods to enhance the feature extraction module. In addition, the model’s performance can still be enhanced for data with unique distributions or noise features. In future work, we intend to further optimize the model architecture, investigate more efficient algorithms to improve computational efficiency, and investigate how to improve the model’s adaptability to complex data environments, all with the goal of improving its performance in anomaly detection tasks.