Automatic Grouping in Singular Spectrum Analysis

Abstract

Singular spectrum analysis (SSA) is a non-parametric forecasting and filtering method that has many applications in a variety of fields such as signal processing, economics and time series analysis. One of the four steps of the SSA, which is called the grouping step, plays a pivotal role in the SSA because reconstruction and forecasting of results are directly affected by the outputs of this step. Usually, the grouping step of SSA is time consuming as the interpretable components are manually selected. An alternative more optimized approach is to apply automatic grouping methods. In this paper, a new dissimilarity measure between two components of a time series that is based on various matrix norms is first proposed. Then, using the new dissimilarity matrices, the capabilities of different hierarchical clustering linkages are compared to identify appropriate groups in the SSA grouping step. The performance of the proposed approach is assessed using the corrected Rand index as validation criterion and utilizing various real-world and simulated time series.

1. Introduction

Singular spectrum analysis (SSA) is a non-parametric technique that is increasingly becoming a standard tool in the field of time series analysis. In this model-free method, a time series is decomposed into a number of interpretable components such as trend, various oscillatory components, and a structure-less noise. The remarkable features of SSA are that neither a parametric model nor stationarity-type conditions have to be assumed for the time series [1]. The SSA method is attracting considerable interest due to its widespread capabilities and it has many applications in a variety of fields such as medicine [2,3,4,5], biology and genetics [6,7], finance and economics [8,9,10,11,12,13,14,15,16], engineering [17,18,19,20,21,22,23], and other fields [24,25,26]. Whole and precise details on the theory and applications of SSA can be found in [1,27,28,29]. For a recent comprehensive review of SSA and description of its modifications and extensions, we refer the interested reader to [30].

It worth mentioning that a major difficulty of the SSA technique is identifying the meaningful and interpretable components of a time series in the grouping step. Conventionally, the information concealed in singular values and singular vectors of the trajectory matrix of a time series is used to detect interpretable components such as trend and oscillations. Usually a scree plot of the singular values, one-dimensional and two-dimensional figures of the singular vectors, and the matrix of the absolute values of the weighted correlations enable us to provide a visual tool to identify appropriate components. More details on grouping based on visual tools can be found in [1,28,29].

In the viewpoint of machine learning, the manual identification of groups in SSA can be regarded as an disadvantage since the intervention of an analyst is required. A neater solution to this problem is to apply an automatic grouping technique. In the SSA framework, the automatic grouping methods can be classified into two categories: frequency-based and distance-based methods. In the frequency-based methods, which include two versions, the elementary components are automatically split into disjointed groups using their frequency contributions that are measured by a periodogram. In the first version of the frequency-based method, each frequency interval is considered separately, and in the second version, the set of frequency intervals are simultaneously used. While the review of theory and applications of automatic grouping via the frequency-based method is beyond the scope of this paper, the interested reader is referred to the whole and precise details on this topic that are explained in [1,3,29,31].

In the distance-based method; first, the dissimilarity of elementary components of a time series is measured by means of a distance criterion (e.g., an appropriate function of weighted correlations between components). Then, a proximity matrix is created using distances. Finally, the elementary components are grouped automatically via distance-based clustering techniques such as hierarchical methods. Although it seems that this interesting approach is an straightforward process, one question that needs to be asked is which clustering method can provide an accurate and reasonable grouping. The hierarchical clustering with complete linkage was used in [32], while the reason for selecting the complete linkage was not clear.

The focus of this research revolves around the distance-based method. In this paper, we first propose a new dissimilarity measure between two components of a time series that is based on various matrix norms. Then, using the new dissimilarity matrices, the capabilities of different hierarchical clustering linkages are compared to find appropriate groups in the grouping step of SSA. It is believed that the outputs of this investigation can lead to a more timely and precise grouping with more precise reconstructed series and forecasting results.

To have a general overview of the two separate but complementary stages of SSA, Section 2 briefly presents a review of SSA. The novel dissimilarity measure between two components of a time series based on various matrix norms is proposed in Section 3. Section 4 is dedicated towards comparing the performance of different hierarchical clustering linkages and various dissimilarity measures via simulation study. Applications of real-world time series data are given in Section 5. The conclusions and summary are presented in Section 6.

2. Review of SSA

In brief, the SSA technique consists of two complementary stages: decomposition and reconstruction. Each of these stages includes two separate steps. At the decomposition stage, the series is decomposed into several components such as trend, seasonal, and cyclical components, which enables us to preform signal extraction and noise reduction. At the reconstruction stage, the interpretable components are reconstructed, which can be used to forecast new data points. For more detailed information on the theory of basic SSA; see, for example [27]. The basic SSA is briefly reviewed below and in doing so we mainly follow [3,33].

Stage 1: Decomposition (Embedding and Singular Value Decomposition)

In the embedding step, a time series $Y_N=\{y_1,\dots ,y_N\}$ is transformed to the sub-series $X_1,\dots ,X_K$, where $X_i={(y_i,\dots ,y_{i+L-1})}^T\in {\mathbf{R}}^L$ and $K=N-L+1$. The vectors $X_i$ are called L-lagged vectors. The single choice of this step is the Window Length L, which is an integer such that $2\le L\le N/2$. The output of the embedding step is the trajectory matrix $\mathbf{X}=[X_1:\cdots :X_K]$, which is also a Hankel matrix. It is noteworthy that this embedding method has been introduced by Broomhead and King [34,35].

In the singular value decomposition (SVD) step, the trajectory matrix $\mathbf{X}$ is decomposed into $\mathbf{X}=\mathbf{U}\mathsf{\Sigma }{\mathbf{V}}^T$, where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal and $\mathsf{\Sigma }$ is a diagonal matrix. The diagonal entries of the matrix $\mathsf{\Sigma }$ are called the singular values of $\mathbf{X}$ and denoted by ${\sigma }_1,\dots ,{\sigma }_L$ in decreasing order of magnitude $\left({\sigma }_1\ge \cdots \ge {\sigma }_L\ge 0\right)$. The columns of $\mathbf{U}$ are called left singular vectors and those of $\mathbf{V}$ are called right singular vectors. If $d=\max \{i,\mathrm{such}\mathrm{that}{\sigma }_i>0\}=rank\left(\mathbf{X}\right)$ then the SVD of the trajectory matrix $\mathbf{X}$ can be written as the sum of rank-one elementary matrices:

$$\mathbf{X}={\mathbf{X}}_1+\cdots +{\mathbf{X}}_d,$$

(1)

where ${\mathbf{X}}_i={\sigma }_i{U}_i{V_i}^T$, $U_i$ is the ith left singular vector and $V_i$ is the ith right singular vector ($i=1,\dots ,d$). It is also well known that the left singular vectors of $\mathbf{X}$ are the eigenvectors of $\mathbf{X}{\mathbf{X}}^T$.

Stage 2: Reconstruction (Grouping and Diagonal Averaging)

The grouping step splits the elementary matrices ${\mathbf{X}}_i$ into several groups and sums the matrices within each group. If a group of indices $i_1,\dots ,i_p$ is denoted by $I=\{i_1,\dots ,i_p\}$ then the matrix ${\mathbf{X}}_I$ corresponding to the group I is defined as ${\mathbf{X}}_I={\mathbf{X}}_{i_1}+\cdots +{\mathbf{X}}_{i_p}$. Having the SVD of $\mathbf{X}$, the split of the set of indices $\{1,\dots ,d\}$ into the disjoint subsets $I_1,\dots ,I_m$ corresponds to the following representation:

$$\mathbf{X}={\mathbf{X}}_{I_1}+\cdots +{\mathbf{X}}_{I_m}.$$

(2)

The goal of diagonal averaging is transforming each matrix ${\mathbf{X}}_{I_j}$ of the grouped decomposition (2) to a Hankel matrix so that these can subsequently be transformed to a time series. Suppose that $z_{ij}$ stands for an element of a matrix $\mathbf{Z}$, then the k-th term of the resulting series is obtained by averaging $z_{ij}$ over all $i,j$ such that $i+j=k+2$. This process is also known as Hankelization of the matrix $\mathbf{Z}$. The output of the Hankelization of a matrix $\mathbf{Z}$ is the Hankel matrix $\mathcal{H}\mathbf{Z}$, which is the trajectory matrix corresponding to the series obtained as a result of the diagonal averaging. The Hankel matrix $\mathcal{H}\mathbf{Z}$ uniquely defines the series by relating the value in the anti-diagonals to the values in the series. By applying the Hankelization procedure to all matrix components of (2), this expansion is obtained: $\mathbf{X}={\tilde{\mathbf{X}}}_{I_1}+\cdots +{\tilde{\mathbf{X}}}_{I_m},$ where ${\tilde{\mathbf{X}}}_{I_j}=\mathcal{H}{\mathbf{X}}_{I_j}$, $j=1,\cdots ,m$. This is equivalent to the decomposition of the initial series $Y_N=\{y_1,\dots ,y_N\}$ into a sum of m series: $y_t={\sum }_{k=1}^m{\tilde{y}}_t^{\left(k\right)}(t=1,\dots ,N)$, where ${\tilde{Y}}_N^{\left(k\right)}=\{{\tilde{y}}_1^{\left(k\right)},\dots ,{\tilde{y}}_N^{\left(k\right)}\}$ corresponds to the matrix ${\tilde{\mathbf{X}}}_{I_k}$.

3. Theoretical Background

3.1. Distances Based on Matrix Norms

It is well known that if x and y are two elements of a normed vector space, then ([36], Appendix A).

$$\left|{∥\mathbf{x}∥}_{}-{∥\mathbf{y}∥}_{}\right|\le {∥\mathbf{x}-\mathbf{y}∥}_{}\le {∥\mathbf{x}∥}_{}+{∥\mathbf{y}∥}_{}.$$

Especially, if ${∥\mathbf{x}∥}_{}={∥\mathbf{y}∥}_{}=1$ then $0\le {∥\mathbf{x}-\mathbf{y}∥}_{}\le 2$ or $0\le \frac{1}{2}{∥\mathbf{x}-\mathbf{y}∥}_{}\le 1$. Therefore, we can define the distance function $d_{xy}=\frac{1}{2}{∥\mathbf{x}-\mathbf{y}∥}_{}$ satisfying $0\le d_{xy}\le 1$.

Let $\mathbf{A}={\left(a_{ij}\right)}_{i,j=1}^{m,n}$ be an $m\times n$ matrix and ${∥\mathbf{A}∥}_{}$ denotes the norm of it. Now, suppose ${\mathbf{A}}_i=\frac{{\tilde{\mathbf{X}}}_i}{∥{\tilde{\mathbf{X}}}_i∥}$, where ${\tilde{\mathbf{X}}}_i$ is the Hankelized version of matrix ${\mathbf{X}}_i$ in the SVD (1) ($i=1,\dots ,d$). If we define the distance between two matrices ${\mathbf{A}}_i$ and ${\mathbf{A}}_j$ as $d_{ij}=\frac{1}{2}∥{\mathbf{A}}_i-{\mathbf{A}}_j∥$ then the distance between two components ${\mathbf{X}}_i$ and ${\mathbf{X}}_j$ can be measured by $d_{ij}$. Thus having the distance matrix $\mathbf{D}={\left(d_{ij}\right)}_{i,j=1}^d$ it is possible to cluster the eigentriples by means of distance-based clustering methods such as the hierarchical clustering approach. It is noteworthy that the reason for defining the matrix ${\mathbf{A}}_i$ with unitary norm ($∥{\mathbf{A}}_i∥=1$) is obtaining the distance measure $d_{ij}$ satisfying $0\le d_{ij}\le 1$. This enables us to interpret the distance between two components ${\mathbf{X}}_i$ and ${\mathbf{X}}_j$ easily.

The distance matrix $\mathbf{D}$ used in clustering is calculated by some commonly used matrix norms. The matrix norms applied in this paper are as follows. More details on matrix norms can be found in [37].

• The Frobenius norm
  The most frequently used matrix norm is the Frobenius norm defined as:

  $${∥\mathbf{A}∥}_F=\sqrt{\sum _{i=1}^m\sum _{j=1}^n{a}_{ij}^2}.$$
• The ${\mathit{L}}_{\mathbf{1}}$-norm
  The $L_1$-norm of the matrix $\mathbf{A}$ is defined as:

  $${∥\mathbf{A}∥}_{L_1}=\sum _{i=1}^m\sum _{j=1}^n\left|a_{ij}\right|.$$
• The 1-norm
  The 1-norm of the matrix $\mathbf{A}$ is the maximum of the absolute column sums, that is,

  $${∥\mathbf{A}∥}_1=\underset{1\le j\le n}{max}\left(\sum _{i=1}^m\left|a_{ij}\right|\right).$$
• The infinity norm
  The infinity norm of the matrix $\mathbf{A}$ is the maximum of the absolute row sums, that is,

  $${∥\mathbf{A}∥}_{\infty }=\underset{1\le i\le m}{max}\left(\sum _{j=1}^n\left|a_{ij}\right|\right).$$
• The maximum modulus norm
  In this case, the maximum modulus of all the elements in the matrix $\mathbf{A}$ is computed, that is,

  $${∥\mathbf{A}∥}_M=\underset{\begin{array}{c}1\le i\le m\\ 1\le j\le n\end{array}}{max}\left|a_{ij}\right|.$$
• The 2-norm
  The spectral or 2-norm of the matrix $\mathbf{A}$ is denoted by ${∥\mathbf{A}∥}_2$. It can be shown that

  $${∥\mathbf{A}∥}_2=\mathrm{The}\mathrm{largest}\sin \mathrm{gular}\mathrm{value}\mathrm{of}\mathbf{A}=\sqrt{\mathrm{The}\mathrm{largest}\mathrm{eigenvalue}\mathrm{of}\mathbf{A}{\mathbf{A}}^T}.$$

In addition to these matrix norm-based distances, we also use another dissimilarity measure that is based on the weighted correlation or w-correlation. It shows the quality of decomposition and determines how well different components of a time series are separated from each other. The w-correlation between two time series $Y_N^{\left(1\right)}$ and $Y_N^{\left(2\right)}$ is defined as follows:

$${\rho }_{12}^{\left(w\right)}=\frac{{\sum }_{i=1}^N{w}_i{y}_i^{\left(1\right)}{y}_i^{\left(2\right)}}{\parallel Y_N^{\left(1\right)}{\parallel }_w{\parallel Y_N^{\left(2\right)}\parallel }_w},$$

where $w_i=min\{i,L,N-i+1\}$, $\parallel Y_N^{\left(k\right)}{\parallel }_w=\sqrt{{\sum }_{i=1}^N{w}_i{y}_i^{\left(k\right)}{y}_i^{\left(k\right)}},(k=1,2)$.

We define the w-correlation-based distance between two components ${\mathbf{X}}_i$ and ${\mathbf{X}}_j$ as $d_{ij}=1-\left|{\rho }_{ij}^{\left(\mathbf{w}\right)}\right|$. It is noteworthy that there is another w-correlation-based distance between two components ${\mathbf{X}}_i$ and ${\mathbf{X}}_j$ defined as $d_{ij}=\frac{1}{2}(1-{\rho }_{ij}^{\left(\mathbf{w}\right)})$. This distance measure, which is explained in [29] and employed in the R package Rssa [38,39,40], is not used in this paper.

3.2. Hierarchical Clustering Methods

In this investigation, hierarchical clustering methods are applied to cluster the components of the time series in the grouping step of SSA. Hierarchical clustering is a popular and distance-based method that is widely used to connect objects in order to form clusters based on their distance. In this paper, we use the distances defined in Section 3.1 as a dissimilarity measure between two components ${\mathbf{X}}_i$ and ${\mathbf{X}}_j$.

Hierarchical clustering approaches can generally divided into two types: the divisive and the agglomerative.

• Divisive: In this technique, an initial single cluster of objects is divided into two clusters such that the objects in one cluster are far from the objects in the other cluster. The procedure continues by splitting the clusters into smaller and smaller clusters until each object makes a separate cluster [41,42]. This method is implemented in our research via the function diana from the cluster package [43] of the freely available statistical R software [44].
• Agglomerative: In this method, the individual objects are initially treated as a cluster, and then the most similar clusters are merged according to their similarities. This process proceeds by successive fusions until all clusters are fused into a single cluster. [41,42]. The agglomerative hierarchical clustering methods that are applied in this research are as follows [45].
  • Single: The distance between two clusters $C_i$ and $C_j$ ($D_{ij}$) is the minimum distance between two points x and y, where $x\in C_i$ and $y\in C_j$; that is,

    $$D_{ij}=\underset{x\in C_i,y\in C_j}{min}{d}_{xy}.$$
  • Complete: The maximum distance between two points x and y is treated as the distance between two clusters $C_i$ and $C_j$, where $x\in C_i$ and $y\in C_j$; that is,

    $$D_{ij}=\underset{x\in C_i,y\in C_j}{max}{d}_{xy}.$$
  • Average:$D_{ij}$ is defined as the mean of the distances between the pair of points x and y, where $x\in C_i$ and $y\in C_j$:

    $$D_{ij}=\sum _{x\in C_i,y\in C_j}\frac{d_{xy}}{n_i\times n_j},$$

    where $n_i$ and $n_j$ are the number of elements in clusters $C_i$ and $C_j$, respectively.
  • McQuitty:$D_{ij}$ is defined as the mean of the between-cluster dissimilarities:

    $$D_{ij}=\frac{D_{ik}+D_{il}}{2},$$

    where cluster $C_j$ is formed from the aggregation of clusters $C_k$ and $C_l$.
  • Median:$D_{ij}$ is defined as follows:

    $$D_{ij}=\frac{D_{ik}+D_{il}}{2}-\frac{D_{kl}}{4},$$

    where cluster $C_j$ is formed from the aggregation of clusters $C_k$ and $C_l$.
  • Centroid:$D_{ij}$ is defined as the squared Euclidean distance between the centres of gravity of the two clusters; that is,

    $$D_{ij}={\parallel {\overline{x}}_i-{\overline{x}}_j\parallel }^2,$$

    where ${\overline{x}}_i$ and ${\overline{x}}_j$ are the mean vectors of the two clusters.
  • Ward: This method is based on minimizing the total within-cluster variance. The pair of clusters with the minimum cluster distance is merged at each step of the analysis. This pair of clusters provides a minimum increase in the total within-cluster variance after merging [45]. There are two algorithms ward.D and ward.D2 for this method, which are available in R packages such as stats and NbClust [45]. By implementing the ward.D2 algorithm, the dissimilarities are squared before the cluster updates.

In this research, we apply the function hclust from the stats package of R software to perform agglomerative hierarchical clustering. More details on hierarchical clustering algorithms can be found in [41,42,46,47].

To enable us to measure the similarity between grouping results obtained by a hierarchical clustering method and a given "correct" grouping, we used the corrected Rand (CR) index, which is an external cluster validation index. This index varies from -1 (no similarity) to 1 (perfect similarity), and it has been proven that high values of the CR index indicate great similarity [47,48,49,50]. The CR index can be implemented in the R function cluster.stats from the fpc package [51].

4. Simulation Results

Here, the performance of hierarchical clustering methods based on various dissimilarity measures introduced in Section 3.1 are compared. In this simulation study, the various simulated time series are evaluated in terms of the CR index. In the following simulated series, ${\epsilon }_t$ is the normally distributed noise with zero mean.

(a)

Exponential:$y_t=exp\left(0.03t\right)+{\epsilon }_t,t=1,2,\dots ,100$.

(b)

Linear:$y_t=0.5t+{\epsilon }_t,t=1,2,\dots ,100$.

(c)

Sine:$y_t=sin(\pi t/6)+{\epsilon }_t,t=1,2,\dots ,100$.

(d)

Cosine+Cosine:$y_t=0.7cos(\pi t/2)+0.5cos(\pi t/3)+{\epsilon }_t,t=1,2,\dots ,100$.

(e)

Exponential×Sine:$y_t=exp\left(0.03t\right)sin(2\pi t/3)+{\epsilon }_t,t=1,2,\dots ,100$.

(f)

Exponential+Sine:$y_t=exp\left(0.03t\right)+sin(2\pi t/3)+{\epsilon }_t,t=1,2,\dots ,100$.

In this simulation, four signal to noise ratios (SNRs) including $SNR=0.25,0.75,5,$ and 10 were used to assess the effect of the noise levels on the grouping results. The simulation was repeated 2000 times for each scenario (a–f) and for each SNR. In order to enable a better comparison based on the CR index, a dashed horizontal line $y=1$ is added to all figures of the CR index.

The simulated series (a–f) have various patterns including trend and periodicity. In order to observe the structure of each simulated series, the simulation was done once with $SNR=5$. The time series plots are depicted in Figure 1.

Considering the theory of time series of finite rank proposed in [27], the simulated series (a–f) have finite rank. Therefore, it is possible to determine the “correct” groups.

Table 1. Correct groups of the simulated series.

Simulated Series	Correct Groups
Exponential	$\left\{1\right\},\{2,\dots ,L\}$
Linear	$\{1,2\},\{3,\dots ,L\}$
Sine	$\{1,2\},\{3,\dots ,L\}$
Cosine+Cosine	$\{1,2\},\{3,4\},\{5,\dots ,L\}$
Exponential×Sine	$\{1,2\},\{3,\dots ,L\}$
Exponential+Sine	$\left\{1\right\},\{2,3\},\{4,\dots ,L\}$

shows the “correct” groups for each of the simulated series that are determined based on the rank of the corresponding trajectory matrices. For more details; see, [27].

In Figure 2, the CR index is shown for the exponential series (case a) with different SNRs, which are computed for various hierarchical clustering algorithms and different values of the window length (L). It can be concluded that the ward.D and ward.D2 methods have the worst performance for each L, at each level of the SNR and for all clustering methods and all kinds of distances. However, clustering based on the w-correlation and the Frobenius norm results in the best performance, except for the complete method. In this case, the similarity between the grouping by the complete method and “correct” groups decrease as L increases. In addition, clustering based on $L_1$, Infinity and 2-norm distances exhibit good performance in detecting the “correct” groups for $L=10$. The capability of these distances goes to decline for a larger L except for the single method and 2-norm distance.

Figure 3 depicts the CR index for the linear series (case b). Similar to the case with an exponential series, the ward.D and ward.D2 methods can not provide a satisfactory result for each L, at each level of the SNR and all types of distances. Moreover, the efficiency of the clustering based on the w-correlation and the Frobenius norm is better than the other distances. In addition, the performance of $L_1$, infinity and 2-norm distances are acceptable only for $L=10$. The capability of these distances decreases for a larger L.

In Figure 4, the CR index for the Sine series (case c) is drawn. Similar to the simulated exponential and linear series, the output of ward.D and ward.D2 methods are not good for large L, at each level of the SNR and all types of distances. Additionally, the efficiency of the clustering based on the w-correlation and the Frobenius norm is better than the other distances. As can be seen in these figures, the single, average and McQuitty methods are better than other methods when the Frobenius norm is used to measure the dissimilarity. Another interesting find is that for large L and at each level of the SNR, the single method outperforms other methods if $L_1$, 1-norm Infinity and 2-norm distances be used. However, the average and McQuitty methods are better than other methods for the maximum modulus norm.

Figure 5 presents the CR index for the cosine+cosine series (case d). Similar to the previous simulated series, at each level of the SNR and all types of distances, the utility of ward.D and ward.D2 methods goes to decline as L increases. Additionally, the superiority of the w-correlation and the Frobenius norms over the other distances are more visible for large SNR. As can be seen in these figures, for large L and at each level of the SNR, the single method outperforms other methods when the maximum modulus norm is not used. In the case with the maximum modulus norm, the average, McQuitty and centroid methods present better outputs.

Figure 6 depicts the CR index for the exponential×sine series (case e). This is similar to the previous simulated series; firstly, the efficiency of the clustering based on the w-correlation and the Frobenius norm is better than the other distances, especially for the single, average and McQuitty clustering methods. Secondly, the ward.D and ward.D2 methods can not provide a satisfactory result for large L, at each level of the SNR and all types of distances. It is noteworthy that in this simulated series, the $L_1$ norm shows a good performance, which is not obtained from the previous simulated series. It can be concluded from these figures that the single method is better than other methods for $SNR<1$. In the case with $SNR>1$ and $L=48$, the capability of the single method in detecting the "correct" groups is considerable when the 2-norm is used.

In Figure 7, the CR index for the exponential+sine series (case f) is drawn. The results that can be concluded from these figures are broadly similar to that of the exponential×sine series (case e); and thus, they are not repeated here.

5. Real-World Data

In this section, we compare the efficiency of hierarchical clustering methods along with different matrix norms using three real-world data. These real-world time series are as follows:

• Seasonally non-adjusted food and vehicle products of France from January 1990 to February 2014. These data are taken from INSEE (Institute National de la Statistique et des Etudes Economiques) including 290 observations. These series were previously used in [52,53,54]. Those interested in a summary of the data are referred to [52] instead of replicating this information here. The time series plots for these data, which are depicted in Figure 8, clearly show that they have a seasonal structure along with a non-linear trend.
• Gross domestic product (GDP) of the United States of America (USA) in billions of dollars from January 1947 to January 2019. This quarterly time series contains 289 observations that are taken from Federal Reserve Economic Data available at https://www.quandl.com/data/FRED/GDP. As shown in Figure 8, the GDP series is non-stationary with a non-linear trend that appears to increase exponentially over time.

Table 2. Identified groups of the real time series.

Real Time Series	L	Groups
Food product	144	$\left\{1\right\},\{2,3\},\{4,5\},\{6,7\},\{8,9\},\{10,\dots ,144\}$
Vehicles	144	$\left\{1\right\},\{2,10\},\{3,4\},\{5,6\},\{7,8\},\left\{9\right\},\{11,12\},\{13,14\},\{15,\dots ,144\}$
GDP	144	$\left\{1\right\},\{2,\dots ,144\}$

reports the identified groups and window length (L) for the real-world time series. We chose the groups based on information obtained from one-dimensional figures of the eigenvectors $U_i$, two-dimensional figures of the eigenvectors $(U_i,U_{i+1})$, and the matrix of the absolute values of the w-correlations. For example, consider the food product time series. The two-dimensional figures of the eigenvectors of this time series are shown in Figure 9. This figure, which is the scatter plot of successive eigenvectors, is used to detect the harmonic components with different frequencies. Each regular P-vertex polygon in the scatter plots of eigenvectors denotes a harmonic component with period P [27]. Therefore, for identifying the harmonic component it is sufficient to find regular P-vertex polygons, which may be in a spiral form. Corresponding pairs of eigenvectors identify the harmonic component. Hence, it can be concluded from Figure 9 that the pairs of eigenvectors 2–3, 4–5, 6–7, and 8–9 correspond to the harmonic components of food product time series. More details on the window length selection and group identification can be found in [1,29].

Figure 10 shows the CR index for the real-world time series. It can be concluded from this figure that, similar to the outputs of the simulation study, the ward.D and ward.D2 methods are unable to identify proper groups. In the food product series, centroid linkage with w-correlation-based distance is the best method. Additionally, the single method shows a good performance especially for w-correlation, Frobenius, and 2-norm. However, the average method is better than other methods for the maximum modulus norm.

In the vehicles series, the single linkage with w-correlation and Frobenius norms are the best methods. Note that the centroid linkage performs well only for w-correlation-based distance. Similar to the food product series, the average method is better than other methods for the maximum modulus norm, which is in accordance with the simulation results for harmonic series (cases c and d).

In the GDP series, hierarchical clustering with the single and centroid linkages can exactly split the components into the groups reported in Table 2. This result is also true for the median linkage with the w-correlation and maximum modulus norm.

6. Conclusions

In this paper, we have proposed a novel dissimilarity measure between components of a time series based on various matrix norms such as Frobenius, $L_1$-norm, 1-norm, 2-norm and so on. Various matrix norms result in different distance matrices that can be used to cluster eigentriples at the grouping step of SSA by means of distance-based clustering methods such as hierarchical clustering techniques. In this research, a comparison study has been conducted in order to find a suitable matrix norm and a proper hierarchical clustering method to identify appropriate groups in SSA. In general, the simulation results and real-world data applications indicated that the accuracy of clustering based on the w-correlation and the Frobenius norm is better than other distance measures. Also, the results support the idea that single linkage along with 2-norm can provide satisfactory automatic grouping. However, the evidence from this study implies that the ward.D and ward.D2 linkages could not detect meaningful groups.

Despite these explicit findings, determining the best hierarchical clustering linkage is not a straightforward procedure. It depends on the structure of the time series and the level of SNR. For example, in the exponential synthetic data (case a) and the GDP series, which includes an exponential-like trend, the efficiency of the single, centroid and median linkages are almost similar, and these techniques outperform the other method. Additionally, in a time series with a seasonal pattern, it seems that the single and average linkages are able to separate periodic components from noise. In summary, findings of this investigation suggest that automatic grouping of eigentriples using the single, average, centroid, and median linkages by means of the w-correlation and the Frobenius norm as distance measures, can lead to more accurate results.