Next Article in Journal
Econometrics Best Paper Award 2016
Previous Article in Journal
Market Microstructure Effects on Firm Default Risk Evaluation
Open AccessArticle

Measuring the Distance between Sets of ARMA Models

Department of Computer Engineering, Computer Science and Mathematics, University of L’Aquila, Via Vetoio Coppito, L’Aquila I-67010, Italy
Academic Editor: Kerry Patterson
Econometrics 2016, 4(3), 32; https://doi.org/10.3390/econometrics4030032
Received: 5 May 2016 / Revised: 5 July 2016 / Accepted: 8 July 2016 / Published: 15 July 2016

Abstract

A distance between pairs of sets of autoregressive moving average (ARMA) processes is proposed. Its main properties are discussed. The paper also shows how the proposed distance finds application in time series analysis. In particular it can be used to evaluate the distance between portfolios of ARMA models or the distance between vector autoregressive (VAR) models.
Keywords: ARMA models; distance; time series; VAR models ARMA models; distance; time series; VAR models

1. Introduction

Measuring the similarity and dissimilarity between models is crucial in many fields of time series analysis. We limit ourselves to mentioning some applications: clustering time series data (see [1,2]), data mining problems (see [3]), time series classification (see [4]), selecting between direct and indirect model-based seasonal adjustment (see [5]), the analysis of Granger causality (see [6]), comparing autocorrelation structures of multiple time series (see [7]).
A number of measures of dissimilarity between univariate linear models have been suggested in the literature. Among these, we find the Mahalanobis distance between autoregressive models used in [8], the autoregressive distances introduced in [9,10,11], the cepstral distance proposed in [12] to be of note. In this paper, we introduce a distance measure between sets of invertible autoregressive moving average (ARMA) models and we show how it finds application in time series analysis. In particular, two specific applications are described. The first application consists of measuring the distance between two single ARMA series, x and y, by means of the proposed distance between portfolios of ARMA models providing reasonably good fits of x and y. The second application uses our distance to compute distances between two vector autoregressive (VAR) models once these models are properly represented in terms of univariate ARMA models. The remainder of the paper is organized as follows: Section 2 presents the distance. Section 3 shows how this distance can be used. An empirical application is presented in Section 4. Section 5 concludes.

2. Proposed Distance

The intrinsic nature of a time series is usually that the observations are dependent or correlated. The autoregressive moving average (ARMA) processes are a very general class of parametric models useful for describing such correlations. For a general reference on ARMA models, see Brockwell and Davis [13].
We have that x = x t ; t Z is an ARMA(p, q) process if, for every t,
ϕ ( L ) x t = θ ( L ) ϵ t ,
where ϕ ( L ) and θ ( L ) are polynomials in the lag operator L, with no common factors, and ϵ t is a white noise process with constant variance σ 2 . The ARMA process x is said to be invertible if it admits the representation:
π ( L ) x t = ϵ t ,
where the AR(∞) operator is defined by
π ( L ) = ϕ ( L ) θ ( L ) 1 = 1 i = 1 π i L i
with i = 1 | π i | < .
Let M be the class of ARMA invertible processes and let d ( x , y ) be a metric on M. Now, we consider the family F M of all non-empty finite subsets of M. We want to introduce a metric on F M .
If S is an element of F M and x is a point of M, the distance from x to S is defined as
δ ( x , S ) = min d ( x , s ) ; s S .
Equation (1) represents the smallest distance between x M and any point s S . Given a subset T of M, let us define the function
h ( S , T ) = max δ ( s , T ) ; s S ,
that measures the largest among all distances δ ( s , T ) , with s S . However, this function is not symmetric since in general h ( S , T ) is not equal to h ( T , S ) . Thus, we prefer to define the distance, H ( S , T ) , between two finite sets of invertible ARMA processes, S and T, as their Hausdorff distance. Formally, we have:
H ( S , T ) = max h ( S , T ) , h ( T , S ) .
The distance between two finite sets of invertible ARMA processes, defined in this way satisfies all the desirable metric properties. In fact, we can prove the following:
Proposition 1. 
The function H : F M × F M R defined as
H ( S , T ) = m a x h ( S , T ) , h ( T , S ) S , T F M ,
is a metric on F M .
Proof. 
The function (2) is obviously non-negative and the symmetry follows straight from the definition.
In order to prove the triangle inequality, we note that if h ( S , T ) < α , then δ ( s , T ) < α for all s S , so there exists s S , t T such that d ( s , t ) < α . Let S, F, T be three elements of F M and put h ( S , F ) = α and h ( F , T ) = β . For each ϵ > 0 we have that h ( F , T ) < β + ϵ and there exists f F , t T such that d ( f , t ) < β + ϵ . Analogously, from h ( S , F ) < α + ϵ there exists s S such that d ( s , f ) < α + ϵ . Since d is a metric on M,
d ( s , t ) d ( s , f ) + d ( f , t ) < α + β + 2 ϵ .
It follows that
h ( S , T ) h ( S , F ) + h ( F , T ) .
and
h ( T , S ) h ( T , F ) + h ( F , S ) .
Thus
H ( S , F ) + H ( F , C ) H ( S , T ) .
Finally, we show that H ( S , T ) = 0 if and only if S = T . It is clear that if S = T , then H ( S , T ) = 0 . Now, we assume that H ( S , T ) = 0 . This implies that h ( S , T ) = 0 . It follows that δ ( s , T ) = 0 s S and hence for every s S there exists a t T such that δ ( s , t ) = 0 . Since δ ( s , t ) is a metric on M, this implies that s = t . Thus S T . In an analogous way, we can show that T S . Thus we can conclude that S = T . The non-negative real-valued function (2) is a metric on F M . ☐
In order to make operative the notion of distance between finite sets of invertible ARMA models introduced above, it is necessary to specify a particular distance d between invertible ARMA models. If x , y M , following Piccolo [9], we can consider the Euclidean distance between the corresponding π-weights sequence, π j , that is
d P ( x , y ) = i = 1 ( π x i π y i ) 2 1 2 .
where π x i and π y i denote the sequences of AR weights of x and y, respectively.
The measure d P ( x , y ) satisfies the following properties:
i. Non-negativity: d P ( x , y ) 0 x , y M ;
ii. Symmetry: d P ( x , y ) = d P ( y , x ) x , y M ;
iii. Subadditivity: d P ( x , y ) d P ( x , z ) + d P ( z , y ) x , y , z M .
We note that the distance between two ARMA processes, measured by d P ( x , y ) , is allowed to be zero even if they are generated by different white noise processes, ϵ x and ϵ y .1 This implies that d P is a pseudometric on M. It follows that also H, with d = d P , becomes a pseudometric on F M .

3. Applications

This section discusses two applications of the distance function defined in (2) in the field of time series analysis.

3.1. Distance between Portfolios of ARMA Models

The first application consists of measuring the distance between two univariate ARMA series, x and y, by means of the proposed distance between portfolios of ARMA models providing reasonably good fits of x and y. The notion of portfolio of ARMA models has been introduced by Poskitt and Tremayne [14]. When modeling time series data an analyst typically searches over a range of models. Then a single model is usually selected as a satisfactory representation of the true, but unknown, underlying data generating mechanism. However, given that a wrong model may be selected or that a “best” model may not exist anyway (see Chatfield [15], p. 80), a better strategy could be to allow the possibility that there may be more than one model which may be regarded as a reasonable representation of the data generating mechanism.
Consider a process x = x t ; t Z that admits an invertible ARMA representation
x t ϕ 1 x t 1 . . . ϕ p 1 x t p = u t + θ 1 ϵ t 1 + . . . + θ q ϵ t q ,
ϵ t W N ( 0 , σ 2 ) .
Following Box and Jenkins [16], given a realization x 1 , . . . , x T , a common model building strategy is to initially select plausible values of p and q based on statistics (the Akaike information criterion (AIC), the Bayesian information criterion (BIC), etc.) calculated from the data.
Consider, for example, the BIC criterion
BIC ( p , q ) = log σ ^ 2 + ( p + q ) log T T ,
where σ ^ 2 is the maximum likelihood estimate of σ 2 . Let p 1 and q 1 be the orders of the ARMA selected by using the BIC criterion, that is
BIC ( p 1 , q 1 ) = min p P ¯ , q Q ¯ BIC ( p , q ) ,
where P ¯ = 0 , 1 , . . . , P and Q ¯ = 0 , 1 , . . . , Q . We denote with M j an ARMA( p j , q j ) model (with p j P ¯ and q j Q ¯ ) for the process x and, following Poskitt and Tremayne [14], we consider the quantity
R ( M j ) = exp 1 2 T BIC ( p 1 , q 1 ) BIC ( p j , q j ) .
We note that this quantity is related to the Y parameter introduced and studied in [17,18].
When 1 < R ( M j ) < 10 the ARMA model M j is said to be a “close competitor” to the criterion-minimizing ARMA( p 1 , q 1 ) model. The set of closely competing models
P ( x ) = M j | 1 < R ( M j ) < 10
is termed a model portfolio for the process x.
The concept of model portfolio suggests not only that the model minimizing the criterion should be selected but also that any additional specifications closely competing with ARMA( p 1 , q 1 ) should not be discarded.
Using the Hausdorff distance H, we can evaluate the distance between portfolios of ARMA models. Let x and y be two invertible ARMA processes and let P ( x ) and P ( y ) be the model portfolios of the process x and y, respectively. Since P ( x ) , P ( y ) F M , the distance between the portfolios P ( x ) and P ( y ) is given by H ( P ( x ) , P ( y ) ) .

3.2. Distance between VAR Processes

The vector autoregressive (VAR) process is a generalization of the univariate autoregressive process. The VAR processes has been popularized by Sims [19]. In this subsection, we will show how the proposed distance can be used to evaluate the distance between VAR processes.
Recall that a k-dimensional process y = y t = y 1 t , y k t ; t Z is a VAR ( p ) process if it can be represented as
y t = A 1 y t 1 + A p y t p + ϵ t
where A 1 , . . . A p are k × k matrices od coefficients and ϵ t = ( ϵ 1 t , . . . , ϵ k t ) is a k-dimensional white noise, with non-singular covariance matrix Σ ϵ . Using the lag operator, L, we can rewrite the VAR model in the following way
A ( L ) y t = ϵ t
where A ( L ) = I A 1 L . . . A p L p .
We have
A ( L ) a d j A ( L ) = det A ( L ) I
where A ( L ) a d j and det A ( L ) are, respectively, the adjoint matrix and the determinant of the matrix A ( L ) .
Following Zellner and Palm [20], premultiplying both sides of (3) by A ( L ) a d j , we obtain the “final equations”:
det A ( L ) y t = A ( L ) a d j ϵ t
For the ith element of y t , expression (4) becomes
det A ( L ) y i t = A ( L ) i . a d j ϵ t
where A ( L ) i . a d j denotes the ith row of the adjoint matrix A ( L ) a d j .
Since the right-hand side of (5) is the sum of k finite moving averages, it can also be represented as a finite moving average θ i ( L ) u i t , where u i t is a white noise process, such that
θ i ( L ) u i t = A ( L ) i . a d j ϵ t
The coefficients of the polynomial θ i ( L ) are found by equating the autocovariances in the two representations. In this way we obtain a non-linear system of q * + 1 equations in q * + 1 unknowns θ i 1 , θ i 2 , . . . , θ i q * and σ u i 2 = var ( u i t ) . It is important to note that we consider the invertible solution of this system.
Considering (5) and (6), the univariate models implied by (3) are given by
det A ( L ) y i t = θ i ( L ) u i t i = 1 , . . . , k .
Thus, we can conclude that
y i t ARMA ( p * , q * ) i = 1 , . . . , k ,
where it is well known that p * k p and q * ( k 1 ) p . We denote with A y this set of univariate ARMA processes. Given two VAR processes, y and x, we can obtain their final forms, A y and A x . Since A y , A x F M , we can calculate
H ( A y , A x ) = max max y i A y min x j A x d P y i , x j , max x i A x min y j A y d P y j , x i
and consider it as distance between the VAR processes y and x.

An Illustrative Example

Here, we present an example of the proposed distance. We consider two VAR(1) processes, y and x, defined, respectively, by the following equations
1 0 . 5 L 0 . 66 L 0 . 5 L 1 + 0 . 3 L y 1 t y 2 t = a 1 t a 2 t
with
E a 1 t a 2 t a 1 t a 2 t = 1 0 0 1
and
1 0 . 5 L 0 0 . 5 L 1 + 0 . 3 L x 1 t x 2 t = b 1 t b 2 t
with
E b 1 t b 2 t b 1 t b 2 t = 1 0 0 1
where the bivariate white noise processes are such that
E a 1 t a 2 t b 1 t h b 2 t h = 0 0 0 0 for h = 0 , ± 1 , . . .
The univariate models implied by (7) are given by:
1 0 . 2 L 0 . 48 L 2 y 1 t = ϵ 1 t + 0 . 204 ϵ 1 t 1 with σ ϵ 1 2 = 1 . 469 ,
1 0 . 2 L 0 . 48 L 2 y 2 t = ϵ 2 t 0 . 382 ϵ 2 t 1 with σ ϵ 2 2 = 1 . 309 .
The univariate models implied by (8) are:
1 0 . 5 L x 1 t = e 1 t with σ e 1 2 = 1 ,
1 0 . 2 L 0 . 15 L 2 x 2 t = e 2 t 0 . 382 e 2 t 1 with σ e 2 2 = 1 . 309 .
We have that d P y 1 , x 1 = 0 . 42 , d P y 1 , x 2 = 0 . 67 , d P y 2 , x 1 = 0 . 81 and d P y 2 , x 2 = 0 . 36 , Thus we obtain H ( A y , A x ) = 0 . 42 .

4. An Empirical Application

To show the applicability of the derived results we consider the following time series:
  • the northern hemisphere annual temperature anomalies (N);
  • the southern hemisphere annual temperature anomalies (S);
  • the annual global land temperature anomalies (L);
  • the annual global ocean temperature anomalies (O).
These global temperature anomaly data, with respect to the 20th century average, come from the Global Historical Climatology Network-Monthly (GHCN-M) data set and International Comprehensive Ocean-Atmosphere Data Set (ICOADS). The data span is from 1980 to 2012. These time series, shown in Figure 1, are available at [21]. Various subsets of this data set have been used in a number of studies (see, for example, [22,23]). The Auto Correlation Functions (ACFs) of the series and the Cross Correlation Functions (CCFs) are presented in Figure 2 and Figure 3, respectively. The ACF plots indicate that the series are nonstationary, since the ACFs decay very slowly.
We will build portfolios of ARMA models for these four series, and hence, by estimating the models for each portfolio, an estimate of the Hausdorff distance H among our time series will be obtained. Preliminarily, we have used the augmented Dickey Fuller (ADF) test in order to establish whether the considered series are stationary or non-stationary. We concluded that all four temperature time series are integrated of order one. These results are consistent with those obtained by Stern and Kaufmann ([24]), Liu and Rodriguez ([25]) and Mills ([26]). Now, in order to build a model portfolio for the considered time series, we identify ARMA models for the first differences of our time series. Bayesian Information Criterion (BIC) has been used to find the appropriate orders p and q of the AR and MA polynomials respectively.
The BIC values has been calculated for the ARMA( p , q ) models with p , q 3 . Table 1 shows the orders ( p , q ) of ARMA models (for each time series) for which the value of the BIC, has been found to be minimum. The residual plots, reported in Figure 4, indicate no presence of serial correlation on the error term. The obtained portfolios of ARMA models for the four differenced series are:
  • P ( O ) = A R ( 2 ) , M A ( 2 ) ,
  • P ( S ) = A R ( 2 ) , M A ( 2 ) , A R ( 3 ) ,
  • P ( N ) = M A ( 1 ) , A R ( 3 ) ,
  • P ( L ) = M A ( 1 ) .
By estimating the models for each portfolio, an estimate of the Hausdorff distance H is obtained. Table 2 summarizes the results.
According to our notion of distance, the dynamic structure of the ocean temperature (O) and the dynamic structure of the southern temperature (S) seem to be very similar. On the contrary, both the series, O and S, seem to be very distant by the land temperature (L) and by the northern hemisphere temperature (N). Instead, a smaller distance separates the series L and N. The big thermal inertia of oceans, together with the uneven distribution of the land and sea (the Southern Hemisphere contains 80.9 percent water and 19.1 percent land, while the Northern Hemisphere is 60.7 percent water and 39.3 percent land) represents a possible explanation of our results. This conclusion is attractive since it links a statistical result to a physical mechanism.

5. Conclusions

This manuscript focuses on measuring the distance between two finite sets of invertible ARMA processes using the Hausdorff distance. In order to make this distance operative, a dissimilarity measure between two single ARMA processes (the AR metric) is considered. Of course other dissimilarity measures, e.g., the Mahalanobis distance between autoregressive models or the cepstral distance, could be used. We have chosen the AR metric since it is simple to compute and it is robust with respect to the presence of anomalous behavior in the data. Further, it is implemented for both stationary and non-stationary time series.
In order to show the usefulness of the proposed distance, two specific applications have been described. The first application consists of measuring the distance between two single ARMA series, let’s say x and y, by means of the Hausdorff distance between portfolios of ARMA models providing reasonably good fits of x and y. The second application uses the Hausdorff distance to compute distances between two VAR processes once these models are properly represented in terms of univariate ARMA processes. An empirical application is also presented.

Acknowledgments

We thank three anonymous reviewers and Diane Bergman for their constructive comments, which helped us to improve the manuscript.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. E.A. Maharaj. “Clusters of time series.” J. Classif. 17 (2000): 297–314. [Google Scholar] [CrossRef]
  2. T. Liao. “Clustering time series data—A survey.” Pattern Recognit. 38 (2005): 1857–1874. [Google Scholar] [CrossRef]
  3. R. Agrawal, C. Faloutsos, and A. Swami. “Efficient similarity search in sequence databases.” In Proceedings of the 4th International Conference on Data Organization and Algorithms FODO ’93, Chicago, IL, USA, 13–15 October 1993; New York, NY, USA: Springer Verlag, 1993. Number 730 in Lecture Notes in Computer Science. pp. 69–84. [Google Scholar]
  4. J. Caiado, N. Crato, and D. Peña. “A periodogram-based metric for time series classification.” Comput. Stat. Data Anal. 50 (2006): 2668–2684. [Google Scholar] [CrossRef][Green Version]
  5. E. Otranto, and U. Triacca. “Measures to evaluate the discrepancy between direct and indirect model-based seasonal adjustment.” J. Off. Stat. 18 (2002): 551–530. [Google Scholar]
  6. F. Di Iorio, and U. Triacca. “Testing for Granger non-causality using the autoregressive metric.” Econ. Model. 33 (2013): 120–125. [Google Scholar] [CrossRef][Green Version]
  7. L. Jin. “Comparing autocorrelation structures of multiple time series via the maximum distance between two groups of time series.” J. Stat. Comput. Simul. 85 (2015): 3535–3548. [Google Scholar] [CrossRef]
  8. P.J. Thomson, and P. de Souza. “Speech recognition using LPC distance measures.” In Handbook of Statistics 5 Time Series in the Time Domain. Edited by E.J. Hannan, P.R. Krishnaiah and M.M. Rao. Amsterdam, The Netherlands: North Holland, 1985, pp. 389–412. [Google Scholar]
  9. D. Piccolo. “A distance measure for classifying ARIMA models.” J. Time Ser. Anal. 11 (1990): 153–164. [Google Scholar] [CrossRef]
  10. M. Corduas, and D. Piccolo. “Time series clustering and classification by the autoregressive metric.” Comput. Stat. Data Anal. 52 (2008): 1860–1872. [Google Scholar] [CrossRef]
  11. E.A. Maharaj. “A significance test for classifying ARMA models.” J. Stat. Comput. Simul. 54 (1996): 305–331. [Google Scholar] [CrossRef]
  12. R.J. Martin. “A metric for ARMA processes.” IEEE Trans. Signal Proces. 48 (2000): 1164–1170. [Google Scholar] [CrossRef]
  13. P. Brockwell, and R. Davies. Time Series: Theory and Methods, 2nd ed. New York, NY, USA: Springer-Verlag, 1991. [Google Scholar]
  14. D.S. Poskitt, and A.R. Tremayne. “Determining a portfolio of linear time series models.” Biometrika 74 (1987): 125–137. [Google Scholar] [CrossRef]
  15. C. Chatfield. Time-Series Forecasting. London, UK: Chapman & Hall, 2000. [Google Scholar]
  16. G.E.P. Box, and G.M. Jenkins. Time Series Analysis: Forecasting and Control. San Francisco, CA, USA: Holden-Day, 1970. [Google Scholar]
  17. D. Faranda, B. Dubrulle, F. Daviaud, and F.M.E. Pons. “Probing turbulence intermittency via autoregressive moving-average models.” Phys. Rev. E 90 (2014): 061001. [Google Scholar] [CrossRef] [PubMed]
  18. D. Faranda, F.M.E. Pons, E. Giachino, S. Vaienti, and B. Dubrulle. “Early warnings indicators of financial crises via auto regressive moving average models.” Commun. Nonlinear Sci. Numer. Simul. 29 (2015): 233–239. [Google Scholar] [CrossRef]
  19. A. Sims. “Macroeconomics and reality.” Econometrica 48 (1980): 1–48. [Google Scholar] [CrossRef]
  20. A. Zellner, and F. Palm. “Time series analysis and simultaneous equation econometric models.” J. Econom. 2 (1974): 17–54. [Google Scholar] [CrossRef]
  21. “Climate at a Glance.” NOAA. Available online: http://www.ncdc.noaa.gov/cag/time-series/global (accessed on 10 June 2016).
  22. K.E. Trenberth. “Has there been a hiatus? ” Science 6249 (2015): 691–692. [Google Scholar] [CrossRef] [PubMed]
  23. U. Triacca, A. Pasini, and A. Attanasio. “Measuring persistence in time series of temperature anomalies.” Theor. Appl. Climatol. 118 (2014): 491–495. [Google Scholar] [CrossRef]
  24. D.I. Stern, and R.K. Kaufmann. “Econometric analysis of global climate change.” Environ. Model. Softw. 14 (1999): 597–605. [Google Scholar] [CrossRef]
  25. H. Liu, and G. Rodríguez. “Human activities and global warming: A cointegration analysis.” Environ. Model. Softw. 20 (2005): 761–773. [Google Scholar] [CrossRef][Green Version]
  26. T.C. Mills. “Breaks and unit roots in global and hemispheric temperatures: An updated analysis.” Clim. Chang. 118 (2013): 745–755. [Google Scholar] [CrossRef]
  • 1.It is important to note that this is a good property of the distance measure d P ( x , y ) . The white noise variance is simply a scale parameter and it is not relevant to measure distance between ARMA models.
Figure 1. Global Ocean temperature anomalies, Land temperature anomalies, Northern and Southern Hemisphere temperature anomalies (clockwise from top left).
Figure 1. Global Ocean temperature anomalies, Land temperature anomalies, Northern and Southern Hemisphere temperature anomalies (clockwise from top left).
Econometrics 04 00032 g001
Figure 2. Sample Auto Correlation Functions (ACFs) of the series: Global Ocean temperature anomalies, Land temperature anomalies, Northern and Southern Hemisphere temperature anomalies (clockwise from top left). The lag axes are in terms of years.
Figure 2. Sample Auto Correlation Functions (ACFs) of the series: Global Ocean temperature anomalies, Land temperature anomalies, Northern and Southern Hemisphere temperature anomalies (clockwise from top left). The lag axes are in terms of years.
Econometrics 04 00032 g002
Figure 3. Sample Cross Correlation Functions (CCFs). The lag axes are in terms of years.
Figure 3. Sample Cross Correlation Functions (CCFs). The lag axes are in terms of years.
Econometrics 04 00032 g003
Figure 4. The residuals of the chosen models.
Figure 4. The residuals of the chosen models.
Econometrics 04 00032 g004
Table 1. Selection of the orders ( p , q ) of autoregressive moving average (ARIMA)( p , 1 , q ) models.
Table 1. Selection of the orders ( p , q ) of autoregressive moving average (ARIMA)( p , 1 , q ) models.
SeriesOLNS
Order(2, 0)(0, 1)(0, 1)(2, 0)
Table 2. Estimates of the Hausdorff distance H.
Table 2. Estimates of the Hausdorff distance H.
Differenced SeriesOSNL
O0.0000.1990.9081.128
S0.1990.0000.9111.153
N0.9080.9110.0000.344
L1.1281.1530.3440.000
Back to TopTop