Next Article in Journal
Maximum Entropy Models for Quantum Systems
Previous Article in Journal
A Possible Application of the Contribution of Aromaticity to Entropy: Thermal Switch
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Monitoring Test for Stability of Dependence Structure in Multivariate Data Based on Copula

1
Department of Statistics, Seoul National University, Seoul 08826, Korea
2
Department of Statistics, Yeungnam University, Gyeongsan 38541, Korea
*
Author to whom correspondence should be addressed.
These authors contributed equally to this work.
Entropy 2016, 18(12), 457; https://doi.org/10.3390/e18120457
Submission received: 17 September 2016 / Revised: 31 October 2016 / Accepted: 19 December 2016 / Published: 21 December 2016

Abstract

:
In this paper, we consider a sequential monitoring procedure for detecting changes in copula function. We propose a cusum type of monitoring test based on the empirical copula function and apply it to the detection of the distributional changes in copula function. We investigate the asymptotic properties of the stopping time and show that under regularity conditions, its limiting null distribution is the same as the sup of Kiefer process. Moreover, we utilize the bootstrap method in order to obtain the limiting distribution. A simulation study and a real data analysis are conducted to evaluate our test.

1. Introduction

A copula is a multivariate joint distribution function for which the marginal distribution of each variable is uniform. In classical analysis, the Pearson’s correlation is most frequently used in practice as a measure of dependence. However, it only works well with elliptical distributions while empirical data in finance and insurance mostly has skewed distributions, heavy tails and extreme values. Thus, correlation may not be suitable to model the nonlinear association. The copula function can overcome this drawback due to the fact that a copula can model various types of dependence structure beyond the linear dependence independently of the marginal distributions. For this reason, a copula has become a flexible methodology in applications of financial risk assessment and actuarial analysis (see Cherubini et al. [1,2], McNeil et al. [3], Hougaard [4] and the papers cited therein). Recently, dependence modelling with copula function has been widely applied to various areas such as civil engineering, reliability engineering, climatology, hydrology, biology, etc.
Since all of the information about the dependence is contained in the copula function, estimating the copula function is a crucial task for providing a correct dependence structure. Conventionally, the copula function is assumed to remain constant over time. However, there is empirical evidence suggesting that the dependence structure is likely to change due to some financial adjustments and critical social events (see, for example, Longin and Solnik [5], Patton [6] and Rodriguez [7]). To cope with this, Dias and Embrechts [8] and Guegan and Zhang [9] suggested a likelihood ratio test for copula parameter changes in specific copula families, Harvey [10] and Busetti and Harvey [11] developed a nonparametric stationarity test for a constant copula based on time-varying quantiles, and Na et al. [12] studied a cusum test for detecting the copula parameter change.
Recently, the problem of testing constancy of the copula has been studied by Quessy et al. [13], Bücher and Ruppert [14], and Bücher et al. [15]. All of these approaches are devoted to the change point detection within data sets of fixed size. However, many researchers are also interested in monitoring change over time when new observations are sequentially observed. Chu et al. [16] pointed out that repeated application of the retrospective tests each time new data is observed may result in a high probability of type I error as the number of applications grows. In this study, we focus on the monitoring procedure for the early detection of a copula function change. During the past few decades, the monitoring procedure has been studied by many authors. For instance, Chu et al. [16] and Horváth et al. [17] developed a cusum type monitoring procedure for detecting structural changes in linear regression models. Berkes et al. [18] applied the monitoring test to generalized autoregressive conditional heteroscedastic (GARCH) models based on quasi-likelihood score. Gombay and Serban [19] monitored parameter changes in autoregressive (AR) models using the cusum type method. Na et al. [20] designed the monitoring test in general time series models.
Recently, Na et al. [21] developed a monitoring procedure to detect a change of the copula parameter. However, one major drawback of this test is that the test statistic is derived under the assumption that the copula family does not change. In this regard, we attempt to develop a monitoring procedure that is more general than their test against alternatives that involve a change in copula function. For this task, we extend the approach of Lee et al. [22] who sequentially monitored distributional changes in AR models and proposed the test statistic based on the empirical copula function. It is shown that, under regularity conditions, the stopping time designed for the detection of changes behaves asymptotically the same as the Kiefer process. In practice, however, the Kiefer process depends on the unknown copula and is rather complicated to compute. To overcome this problem, several authors have suggested bootstrap methods for approximating the Kiefer process. We refer to Fermanian et al. [23], Rémillard and Scaillet [24], Bücher and Dette [25], Bouzebda [26] and the references therein for more details. In this study, we also utilize the bootstrap method to deal with this difficulty.
This paper is organized as follows. We introduce the monitoring procedure based on the empirical copula process in Section 2 and establish some asymptotic results of the stopping time in Section 3. In Section 4, we report some simulation results that validate our monitoring procedure, and apply the test to real data analysis. In Section 5, conclusions with a brief discussion of the advantages and limitations of our study are provided.

2. Monitoring Procedure

Let { X t = ( X 1 t , , X d t ) ; t = 1 , 2 , } be a sequence of d-dimensional independent random vectors with joint distribution F and continuous marginal distribution functions F i for i = 1 , , d . If C is a true copula function of { X t } , owing to Sklar’s theorem, we can express
F ( x 1 , , x d ) = C ( F 1 ( x 1 ) , , F d ( x d ) ) for ( x 1 , , x d ) R d .
When the marginal distribution functions F 1 , , F d are continuous, it is well known that the function C is uniquely determined such as
C ( u 1 , , u d ) = F ( F 1 ( u 1 ) , , F d ( u d ) ) for ( u 1 , , u d ) [ 0 , 1 ] d ,
where F i ( u i ) = i n f { x : F i ( x ) u i } for i = 1 , , d . According to Sklar’s theorem, one can always model any multivariate distributions by modelling both marginal distributions and copula functions separately. For details, we refer to Joe [27] and Nelson [28].
Suppose that we have observed X 1 , , X n , which are called the historical data. For each new observation, we wish to test the following hypotheses:
H 0 : C t = C  for all  t > n ,  vs .   H 1 : C t C  for some  t > n ,
where C t is a copula function at time t and C is a time-invariant copula. The copula function is assumed to be stable over the historical period of length n, i.e.,
C t = C , 1 t n .
The historical data is used as a reference for comparison with future observations. The aim of our monitoring procedure is to check a change of copula function each time a new observation is updated in the post-historical period.
In this study, we assume that the marginal distributions do not change under both the null and alternative hypotheses, while the copula function changes under the alternative hypothesis. This assumption is in line with other precedent studies on testing for structural breaks in copula (see, e.g., Harvey [10], Busetti and Harvey [11], Bouzebda [26], Bücher and Ruppert [14] and Na et al. [12,21]). Furthermore, this assumption is very crucial in real practice since one can encounter the situation that the monitoring test detects a change in copula function although the change actually occurs in marginal distributions. Empirical evidence of this situation can be found in Na et al. [29]. Therefore, a change point test for marginal distribution must be conducted in advance of implementing the monitoring test for a copula function change.
The monitoring procedure in a general set-up can be described with the stopping time τ ( n ) defined as follows:
τ ( n ) : = inf { k > n : D k , n b ( k / n ) } ,
where D k , n is a test statistic based on X 1 , , X k and b ( · ) is a boundary function. If τ ( n ) is finite, we reject H 0 and consider that the change has occurred at τ ( n ) . Otherwise, we continue to observe the data. Therefore, in order to perform the monitoring procedure, we have to define a test statistic D k , n and choose a boundary function b ( · ) such that
lim n P { τ ( n ) < | H 0 } = lim n P { D k , n b ( k / n ) for some k > n | H 0 } = α
for a given α ( 0 , 1 ) , and
lim n P { τ ( n ) < | H 1 } = 1 .
We consider the boundary function b ( · ) satisfies:
b ( s ) is continuous on ( 1 , ) and inf 1 < s < b ( s ) > 0
(cf. Chu et al. [16] and Berkes at al. [18]). Here, we focus on the b ( · ) of a specific form such as
b ( s ) = c s a for some c > 0 , a > 0 ,
proposed by Lee et al. [22]. Practically, the constants c and a must be chosen to satisfy (3) for a given α.
For the specific form of test statistic D k , n , Chu et al. [16] considered two types of test statistics. The one is based on the fluctuations of sequential parameter estimates, and the other is based on the cumulative sum of recursive residuals. Berkes [18] proposed the quasi–maximum likelihood estimator of the parameters in the GARCH process for test statistics. Lee et al. [22] used a sequential empirical process of residuals for the detection of distributional changes in AR models.
In this paper, we use a copula function for the test statistic to detect a change in the dependence structure of multivariate random vectors. The main idea of the procedure is based on the changes in the dependence structure upon changes in the copula function. In real practice, the copula function C is usually unknown. Thus, we consider the situation in which the empirical copula estimator is employed to play a role of the true copula function. The empirical copula function can be obtained by replacing the unknown terms in (1) with the joint empirical distribution function and the marginal empirical distribution function, which are defined, respectively, by
F n ( x 1 , , x d ) = 1 n t = 1 n I ( X 1 t x 1 , , X d t x d ) for x i R ,
F i n ( x i ) = 1 n t = 1 n I ( X i t x i ) for i = 1 , , d .
Then, the empirical copula estimator is
C n , n ( u 1 , , u d ) = F n ( F 1 n ( u 1 ) , , F d n ( u d ) ) for u i [ 0 , 1 ] ,
where F i n ( u i ) = i n f { x : F i n ( x ) u i } for i = 1 , , d . Set U i t = F i ( X i t ) , for i = 1 , d . Then, for u i [ 0 , 1 ] , G n and G i n can be expressed as
G n ( u 1 , , u d ) = 1 n t = 1 n I ( U 1 t u 1 , , U d t u d )
and
G i n ( u i ) = 1 n t = 1 n I ( U i t u i ) for i = 1 , , d .
Then, we have
F n ( x 1 , , x d ) = d G n ( F 1 ( x 1 ) , , F d ( x d ) )
and
( F 1 n ( x 1 ) , , F d n ( x d ) ) = d ( G 1 n ( F 1 ( x 1 ) ) , , G d n ( F 1 ( x d ) ) ) .
Thus, using the representation (6), it follows that
C n , n ( u 1 , , u d ) = d G n ( G 1 n ( u 1 ) , , G d n ( u d ) ) ,
where G i n ( u i ) = i n f { x : G i n ( x ) u i } for i = 1 , , d . This implies that the law of C n , n is the same for all F whose associated copula is C. We will propose the test statistics based on the empirical copula estimator in the next section.
Remark 1.
Note that the alternative hypothesis H 1 actually means that H 1 : C t C for some n < t < T ( n ) , where T ( n ) is the predetermined maximal number. Since it is impossible to monitor at an unlimited time horizon and the test for too large t will be meaningless, the maximal number of observations T ( n ) is considered. Here, T ( n ) is considered as lim n T ( n ) n = q < and lim n T ( n ) n = , and this result will be seen in our simulation study.

3. Main Result

For monitoring the copula function, we employ the test statistic D n , k based on the empirical copula functions such as
D k , n = n sup u [ 0 , 1 ] d C k , n ( u ) C n , n ( u ) ,
where u = ( u 1 , , u d ) and C k , n ( u ) = F k ( F 1 n ( u 1 ) , , F d n ( u d ) ) . Suppose that X 1 , , X n are observed which represent available historical data. By observing new data sequentially, we wish to detect if a change occurs in copula function C. This procedure compares the estimates of the copula function obtained based on a growing number of observations, with the estimate obtained based on the historical observations. Note that C k , n is the estimator of C based on the observation up to time k, while C n , n is the estimator obtained based on the historical data. As mentioned earlier, since we assume that the marginal distribution functions are stable, the marginal empirical distribution functions F i n are used in C k , n instead of F i k .
To show the asymptotic behavior of the test statistic, we need to introduce some Gaussian processes. The limiting process K C ( · ) on [ 0 , 1 ] d × [ 0 , ) is a Gaussian process with
E ( K C ( u , n ) ) = 0
and
E ( K C ( u , n ) K C ( v , m ) ) = ( n m ) C ( u v ) C ( u ) C ( v ) ,
for u , v [ 0 , 1 ] d and n , m 0 . The process K C ( · ) on [ 0 , 1 ] d × [ 0 , ) is called the Kiefer process associated with the copula function C. For details on the Kiefer process, we refer to Adler [30] and Piterbarg [31].
Here, we impose the following conditions for the main theorem:
(A1)
C is twice continuously differentiable on ( 0 , 1 ) d ;
(A2)
The second-order partial derivatives of C exist and are continuous on [ 0 , 1 ] d ;
(A3)
lim n sup k n 1 b ( k / n ) ( log k ) 3 / 2 n 1 / 4 d = 0 .
Under the above assumptions, we have the following:
Theorem 1.
Suppose that H 0 is true and conditions (A1) and (A2) hold. In addition, a boundary function b ( · ) satisfies (4), (5) and condition (A3). Then, the stopping time τ ( n ) with a test statistic D k , n in (7) satisfies
lim n P { τ ( n ) < | H 0 } = P { sup u [ 0 , 1 ] d K C ( u , s ) b 1 1 s f o r s o m e 0 < s < 1 } ,
where K C ( · ) is a ( d + 1 )-dimensional Kiefer process on [ 0 , 1 ] d × [ 0 , ) associated with the copula function C ( · ) .
Proof of Theorem 1.
For convenience, we set u = ( u 1 , , u d ) and v = ( v 1 , , v d ) . Then, we can rewrite
C k , n ( u ) = G k ( G 1 n ( u 1 ) , , G d n ( u d ) ) = I k ( u ) + I I n , k ( u ) + I I I n ( u ) ,
where
I k ( u ) = G k ( u ) C ( u ) ,
I I n , k ( u ) = G k ( G 1 n 1 ( u 1 ) , , G d n 1 ( u d ) ) C ( G 1 n 1 ( u 1 ) , , G d n 1 ( u d ) ) G k ( u ) + C ( u ) ,
I I I n ( u ) = C ( G 1 n 1 ( u 1 ) , , G d n 1 ( u d ) ) .
On the other hand, due to (8) and Lemmas 1 and 2 addressed below, we have
sup k n n sup u [ 0 , 1 ] d C k , n ( u ) C n , n ( u ) b ( k / n ) n sup u [ 0 , 1 ] d 1 k K C ( u , k ) 1 n K C ( u , n ) b ( k / n ) sup k n n b ( k / n ) sup u [ 0 , 1 ] d C k , n ( u ) C n , n ( u ) 1 k K C ( u , k ) + 1 n K C ( u , n ) = sup k n n b ( k / n ) sup u [ 0 , 1 ] d I k ( u ) + I I n , k ( u ) I n ( u ) I I n , n ( u ) 1 k K C ( u , k ) + 1 n K C ( u , n ) sup k n 2 n b ( k / n ) sup u [ 0 , 1 ] d I k ( u ) + I I n , k ( u ) 1 k K C ( u , k ) sup k n 2 b ( k / n ) sup u [ 0 , 1 ] d 1 k k I k ( u ) K C ( u , k ) + sup u [ 0 , 1 ] d k I I n , k ( u ) sup k n 2 M b ( k / n ) ( log k ) 3 / 2 n 1 / 4 d .
By condition (A3), the last term converges to 0 a.s. as n .
Therefore, we can express
lim n P { τ ( n ) < | H 0 } = lim n P { n sup u [ 0 , 1 ] d C k , n ( u ) C n , n ( u ) b ( k n ) for some k > n | H 0 } = P { n sup u [ 0 , 1 ] d 1 k K C ( u , k ) 1 n K C ( u , n ) b ( k n ) for some k > n } = P { sup u [ 0 , 1 ] d K C ( u , k n ) k n K C ( u , 1 ) k n b ( k n ) for some k > n } = P { sup u [ 0 , 1 ] d K C ( u , s ) s K C ( u , 1 ) s b ( s ) for some s > 1 } = P { sup u [ 0 , 1 ] d K C ( u , s ) b ( 1 1 s ) for some 0 < s < 1 } .
This validates the theorem. ☐
Lemma 1.
If the assumptions in Theorem 1 hold, then we have
sup u [ 0 , 1 ] d k I k ( u ) K C ( u , k ) = O k 1 / 2 1 / 4 d ( log k ) 3 / 2 .
Proof. 
It follows from Theorem B of Bouzebda [26] (see also Csörgő and Horváth [32]). ☐
Lemma 2.
If the assumptions in Theorem 1 hold, then we have
sup u [ 0 , 1 ] d k I I n , k ( u ) = O ( n 1 / 4 ( log n ) 1 / 2 ( log log n ) 1 / 4 ) .
Proof. 
We follow the lines of the proof of Proposition 4.2 in Segers [33] and the proof of Theorem 4.1 in Tsukahara [34]. For k n and a n 0 , we put
w k ( a n ) = sup { | k ( G k ( u ) C ( u ) ) k ( G k ( v ) C ( v ) ) | : u , v [ 0 , 1 ] d , | u i v i | a n , 1 i d } .
By the Smirnov–Chung law of the iterated logarithm for the empirical distribution functions, we obtain
max 1 i d sup 0 u i 1 | G i n 1 ( u i ) u i | = O ( n 1 / 2 ( log log n ) 1 / 2 ) .
Moreover, we take a n = n 1 / 2 ( log log n ) 1 / 2 and λ n = 2 K 2 1 / 2 n 1 / 4 ( log n ) 1 / 2 ( log log n ) 1 / 4 for K 2 as in Proposition A.1 of Segers [33]. Using Proposition A.1 of Segers [33], there exist constants K 1 such that
n 2 P { w k ( a n ) λ n } n 2 K 1 n 3 / 2 ( log log n ) 1 / 2 < .
By the Borel–Cantelli lemma, we obtain
sup u [ 0 , 1 ] d k I I n , k ( u ) w k ( a n ) = O ( λ n ) , a . s .
 ☐
In practice, given 0 < α < 1 , we reject H 0 if
T k , n = n b ( k / n ) sup u [ 0 , 1 ] d C k , n ( u ) C n , n ( u ) C α ,
where C α is the number such that
P sup 0 < s < 1 sup u [ 0 , 1 ] d K C ( u , s ) b 1 / ( 1 s ) C α = α .
However, the asymptotic limiting distribution is complicated to compute in practice and depends on the unknown copula C. For this reason, it is not directly applicable for the monitoring procedure in practice. To overcome the difficulty that arises due to the computation, we recommend using a bootstrap method. Some precedent studies have used the bootstrap method to approximate the limiting distribution. Bücher and Dette [25] compared the finite sample properties of the various bootstrap methods proposed in the literature and concluded that the procedure proposed by Rémillard and Scaillet [24] yields the best results in most cases. In this study, we consider the multiplier bootstrap approach proposed by Rémillard and Scaillet [24].
Let ϵ 1 , , ϵ n be an i.i.d sequence of random variables with mean zero, variance one, 0 P ( | ϵ 1 | > x ) d x < , and independent of X 1 , , X n . Rémillard and Scaillet [24] defined the bootstrap process
α n ( u ) = 1 n t = 1 n ϵ t I ( F 1 n ( X 1 t ) u 1 , , F d n ( X d t ) u d C n , n ( u ) ] = 1 n t = 1 n ( ϵ t ϵ ¯ ) I F 1 n ( X 1 t ) u 1 , , F d n ( X d t ) u d ,
where u = ( u 1 , , u d ) and ϵ ¯ = t = 1 n ϵ t , and showed that α n ( u ) approximates a Brownian bridge process B C with E ( B C ( u ) ) = 0 , E ( B C ( u ) B C ( v ) ) = C ( u v ) C ( u ) C ( v ) for u , v [ 0 , 1 ] d . (cf. Lemma A.1 of Rémillard and Scaillet [24]). Using the fact s 1 / 2 K C ( u , s ) = B C ( u ) , we can obtain the approximation of K C and calculate an approximate value for C α in (10). The detailed procedure is as follows:
(Step 1)
Based on the data X 1 , , X n , obtain the marginal empirical distribution functions F i n and the empirical copula function C n , n .
(Step 2)
For each j { 1 , , B } , generate ϵ 1 ( j ) , , ϵ n ( j ) that is an i.i.d sequence of random variables with mean zero, variance one and 0 P ( | ϵ 1 ( j ) | > x ) d x < , and calculate α n ( j ) ( u ) obtained through (11) based on these random variables.
(Step 3)
For Z n = { 0 / n , 1 / n , , n / n } , calculate
T n ( j ) = sup 0 < s < 1 s max u Z n d α n ( j ) ( u ) b 1 / ( 1 s ) .
(Step 4)
Repeat the above procedure (Step 2) and (Step 3) B times and calculate the 100 ( 1 α ) % percentile of the obtained B number of T n ( j ) values.
(Step 5)
Starting from time k = n + 1 onward, we reject H 0 if T k , n in (9) is larger than the 100 ( 1 α ) % percentile obtained through (Step 4).
The approximate value for C α in (10) can be obtained by a bootstrap sample and the approximate quantile is copula distribution free. The above bootstrap method is easy to implement and gives satisfactory results, as seen in the next section.
Remark 2.
In this study, we focus on the boundary function of the form in (5). In this case, there is no such rule to choose an optimal a. The test with small a produces large powers compared to that with large a. Unsatisfactory results are obtained if a is either too small or too large. Thus, the choice of a can be an important issue in practice. From the simulation study in Lee et al. [22], it is found that no test with specific a outperforms the others completely in terms of the stability for the test. Here, we recommend using a = 2 and this result will be seen in our simulation study. Furthermore, one can also employ other boundary functions satisfying (4) and condition (A3).

4. Empirical Studies

4.1. Simulation

In this section, we evaluate the performance of the monitoring test proposed in Section 3 through a simulation study. For this task, we use the boundary function in (5) and employ the stopping rule based on (7). In this study, we consider the bivariate Gaussian copula with copula parameter θ 0 as a true copula model. To see an effect from the copula functions with different functional forms allowing degrees of asymmetry and tail dependence, we also consider the Gumbel copula that is asymmetric and has upper tail dependency as a true model. The copula parameters of the Gumbel model are set to be equal to the value of Kendall’s tau τ 0 in Gaussian copula models. For each case, sets of n = 100, 200 and 300 observations are generated from the copula model with marginal distribution N ( 0 , 1 ) . The empirical sizes and powers are calculated by the number of rejections of the null hypothesis “ H 0 : no changes occur in the copula model at t = n + 1 , ”, out of 1000 repetitions. Here, the predetermined maximal number of observations T ( n ) are considered as T ( n ) = n log n for empirical size and T ( n ) = 2 n , 3 n , 4 n , 5 n , n log n for empirical power. In order to examine the power, we consider the following alternative hypotheses. We take into account two elliptical copulas such as the Gaussian and the Student t and the Frank copula as alternative hypotheses.
H 1 ( 1 )
A change occurs from the Gaussian copula with τ 0 = 0.13 to the Gaussian copula with τ 0 = 0.35 and 0.60 at n p .
H 1 ( 2 )
A change occurs from the Gaussian copula with τ 0 = 0.13 to the Student t copula with τ 0 = 0.35 and 0.60 at n p .
H 1 ( 3 )
A change occurs from the Gaussian copula with τ 0 = 0.13 to the Frank copula with τ 0 = 0.35 and 0.60 at n p .
For the Gumbel copula, we consider Archimedean copulas family for alternative hypotheses such as the Gumbel, Clayton and Frank copulas. For this, we consider the following alternative hypotheses.
H 1 ( 1 )
A change occurs from the Gumbel copula with τ 0 = 0.13 to the Gumbel copula with 0.60 at n p .
H 1 ( 2 )
A change occurs from the Gumbel copula with τ 0 = 0.13 to the Clayton copula with 0.60 at n p .
H 1 ( 3 )
A change occurs from the Gumbel copula with τ 0 = 0.13 to the Frank copula with 0.60 at n p .
In each case, the copula parameters are set to be at the same level in terms of Kendall’s tau in different copula families. To examine the power, many cases of changes in the dependence structure are considered, namely changes of the copula parameter and/or changes of the copula family. For H 1 ( 1 ) and H 1 ( 1 ) , we consider the situation involving a change of the copula parameter within a copula family. For H 1 ( 2 ) , H 1 ( 3 ) , H 1 ( 2 ) and H 1 ( 3 ) , we examined power of the case involving a change of copula parameter and copula family at the same time.
Throughout our simulation study, we only consider the change of copula function and assume that the marginal distributions experience no changes. The empirical sizes are calculated at the nominal levels 0.01, 0.05 and 0.10, and the powers are examined at the nominal level 0.10. The bootstrap method is used for the calculation of the critical value at the nominal level. We perform the bootstrap method discussed in Section 3 with B = 500 for n = 100 , 200 and 300, and the constant a of the boudary function in (5) is chosen to be 2.
In particular, our test is compared with the monitoring test proposed by Na et al. [21]. Recall that Na et al. [21]’s test can be applied to detect a copula parameter change when the copula family does not change. Na et al. [21] proposed the detector in (2) based on the difference between estimates of the copula parameter:
D k , n E = n θ ^ k θ ^ n ,
where θ ^ k is the estimator of the true copula parameter θ 0 based on the observation up to time k, while θ ^ n is the estimator obtained based on the historical data.
Empirical sizes and powers are presented in Table 1, Table 2, Table 3 and Table 4. The figures in Table 1, Table 2, Table 3 and Table 4 are for D k , n while the figures in the parentheses are for D k , n E . Table 1 shows that the test procedure has some size distortions when n is small. However, as n increases, the empirical size of the test gets very close to the nominal levels in most cases. Size distortions of tests for small sample sizes can be reduced if a smaller a is chosen. For D k , n E , it can be seen that the test also has some size distortions, but the test is generally able to keep their nominal level, especially when n = 300 . The result is also same for the other copula models such as t-copula, Frank copula, and Clayton copula, although not reported here for brevity. Table 2, Table 3 and Table 4 report the empirical power of H 1 ( 1 ) H 1 ( 3 ) and H 1 ( 1 ) H 1 ( 3 ) with p = 1.1 and 1.5 . Table 2, Table 3 and Table 4 show that our monitoring procedure produces good powers in most cases. It is shown that the powers increase remarkably either as n increases or the more significant change occurs. Moreover, we can see that when the changes in copula function occur earlier, the powers increase remarkably. It can be seen that the powers in the case that n is large and p = 1.1 are very close to 1. As pointed out by Lee et al. [22], our monitoring procedure with boundary function such as (5) detects early changes more effectively than late changes. Due to the curvature of the component ( k / n ) a in the boundary function, the boundary function increases rapidly as the change point moves further away from the point where the monitoring was initiated. This implies that it is more likely to capture small changes early in the sample. Consequently, our test has better power properties for early change points. Similar findings were reported in Na et al. [21] and Lee et al. [22]. This result indicates that it is desirable to renew the historical data appropriately to escalate the power when the null hypothesis appears to be true for a certain period time. Note that for alternative hypothesis H 1 ( 1 ) involving a change of the copula parameter within a copula family, the monitoring procedure based on D k , n E appears to have higher powers than our monitoring test. This result can be explained by the fact that Na et al. [21]’s monitoring test is designed only to detect parameter changes of copula function. However, even if we consider the alternative hypotheses H 1 ( 2 ) and H 1 ( 3 ) that involve a change of copula family, Na et al. [21]’s test also shows good performance in terms of power. This means that Na et al. [21]’s test tends to detect a copula parameter change, even though the change actually occurs in copula function. From this aspect, we were motivated to develop the monitoring procedure for detecting a copula function change. In comparing Table 4 against Table 3, the performance of power appears to be similar. Different functional forms of copula seemed to have no impact on the performance, hence our monitoring test also has good performance in copula models having asymmetry properties or tail dependency. All these results indicate that our test procedure performs adequately to monitor for stability of copula function.

4.2. Real Data Analysis

In this section, we illustrate an example of a real data analysis. We consider bivariate climate data consisting of temperature and precipitation over the contiguous United States. There is a lot of literature studying the association of temperature and precipitation over the United States, and they reported empirical evidence that there is an obvious relationship between two variables (see Zhao and Khalil [35] and Huang and van Den Dool [36] and the papers cited therein). Recently, several authors have used a copula based methodology to model the joint distribution of temperature and precipitation (see, e.g., Favre et al. [37], Shiau et al. [38], Dupuis [39] and Schölzel and Friederichs [40]). However, the precedent studies only focus on the problem as to which copula model best fitted the empirical data. Here, we use the copula functions to model the dependence between temperature and precipitation and attempt to monitor for stability of dependence. Annual mean temperature and annual mean precipitation in summer months (June, July, and August) over the contiguous United States from 1895 to 2015 are used for empirical data. The data can be obtained from NOAA’s National Centers for Environmental Information (NCEI). Figure 1 shows that precipitation and temperature tend to be negatively correlated. It is well known that warmer summers usually result in drier conditions and colder summers are likely to be wetter. For historical data, the data from 1895 to 1975 is used, which has 81 observations. As discussed earlier, since the monitoring test for copula function can be influenced by a change in marginal distribution, the change point tests for marginal distributions are performed in advance of implementing the monitoring test for a copula function change. To this end, we perform the test of Lee et al. [22] who sequentially monitored marginal distributional changes based on the following test statistic:
D k , n D = n sup z F i k ( z ) F i n ( z ) for i = 1 , 2 ,
where F i k is the empirical distribution based on the observation up to time k, while F i n is the empirical distribution obtained based on the historical data. By observing new data sequentially, we first conduct the monitoring test for marginal distributional changes. If there are no changes in marginal distributions, we can perform the monitoring test for the copula function change. Since both of the two series detect no evidence of a change in marginal distributions at the nominal level 0.05, we apply monitoring procedure based on the test statistic in (7) to detect a change of dependence. For this task, we use the boundary function in (5) with a = 2 and perform the bootstrap method in Section 3 with B = 500 . As a result, it appears that the test detects a change in dependence at nominal levels 0.01, 0.05, and 0.10. The location of the stopping time is summarized in Table 5 and Figure 1 illustrates the stopping time in dependence: the solid line corresponds to the end of historical data and the dotted lines identify the detected stopping time.

5. Conclusions

In this study, we designed the monitoring test for a change of copula function on the basis of the empirical copula functions. The test is shown to have its limiting distribution as the supremum of the Kiefer process under certain regularity conditions. The simulation results reported in Section 4 confirms that our test performs adequately. Our method to monitor the change of copula function has several advantages. The procedure is copula model free and we use a bootstrap method to overcome the difficulty that the asymptotic limiting distribution depends on the unknown copula function. For this reason, it is directly applicable in practice even when we do not know the true copula function. Furthermore, finite sample properties are expected to be well behaved since we use a bootstrap method. Our monitoring test has been established under the assumption that each series of random vector is independent and identically distributed. However, this assumption is often violated in practice and one might be able to consider even a broader class of stochastic processes such as autoregressive moving average (ARMA), ARCH and GARCH processes. Recently, Doukhan et al. [41] and Bücher and Volgushev [42] considered the weak convergence of the empirical copula process under serial dependence. These studies form the basis for our new monitoring test for copula function under weak dependence. We leave the task of extending our test to future study.

Acknowledgments

This work was supported by the 2015 Yeungnam University Research Grant (No. 215A580013) and the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (NRF-2015R1C1A1A01052330).

Author Contributions

Both authors contributed equally to this work. Both authors have read and approved the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Cherubini, U.; Luciano, E.; Vecchiato, W. Copula Methods in Finance; John Wiley & Sons: Hoboken, NJ, USA, 2004. [Google Scholar]
  2. Cherubini, U.; Gobbi, F.; Mulinacci, S.; Romagnoli, S. Dynamic Copula Methods in Finance; John Wiley & Sons: Hoboken, NJ, USA, 2012. [Google Scholar]
  3. McNeil, A.J.; Frey, R.; Embrechts, P. Quantitative Risk Management: Concepts, Techniques, and Tools; Princeton Series in Finance: London, UK, 2005. [Google Scholar]
  4. Hougaard, P. Analysis of Multivariate Survival Data; Springer: Berlin/Heidelberg, Germany, 2000. [Google Scholar]
  5. Longin, F.; Solnik, B. Is the correlation in international equity returns constant: 1960–1990? J. Int. Money Financ. 1995, 14, 3–26. [Google Scholar] [CrossRef]
  6. Patton, A.J. Modelling asymmetric exchange rate dependence. Int. Econ. Rev. 2006, 47, 527–556. [Google Scholar] [CrossRef]
  7. Rodriguez, J.C. Measuring financial contagion: A copula approach. J. Empir. Financ. 2007, 14, 401–423. [Google Scholar] [CrossRef]
  8. Dias, A.; Embrechts, P. Change-point analysis for dependence structure in finance and insurance. In Risk Measures for the 21st Century; Szegoe, G., Ed.; John Wiley & Sons: Hoboken, NJ, USA, 2004; pp. 321–335. [Google Scholar]
  9. Guegan, D.; Zhang, J. Change analysis of dynamic copula for measuring dependence in multivariate finance data. Quant. Financ. 2010, 10, 421–430. [Google Scholar] [CrossRef] [Green Version]
  10. Harvey, C. Tracking a changing copula. J. Empir. Financ. 2010, 17, 485–500. [Google Scholar] [CrossRef]
  11. Busetti, F.; Harvey, C. When is a copula constant? A test for changing relationships. J. Financ. Econom. 2011, 9, 106–131. [Google Scholar] [CrossRef]
  12. Na, O.; Lee, J.; Lee, S. Change point detection in copula ARMA-GARCH Models. J. Time Ser. Anal. 2012, 33, 554–569. [Google Scholar] [CrossRef]
  13. Quessy, J.F.; Saïd, M.; Favre, A.C. Multivariate Kendall’s tau for change-point detection in copulas. Can. J. Stat. 2013, 41, 65–82. [Google Scholar] [CrossRef]
  14. Bücher, A.; Ruppert, M. Consistent testing for a constant copula under strong mixing based on the tapered block multiplier technique. J. Multivar. Anal. 2013, 116, 208–229. [Google Scholar] [CrossRef]
  15. Bücher, A.; Kojadinovic, I.; Rohmer, T.; Segers, J. Detecting changes in cross-sectional dependence in multivariate time series. J. Multivar. Anal. 2014, 132, 111–128. [Google Scholar] [CrossRef]
  16. Chu, C.S.J.; Stinchcombe, M.; White, H. Monitoring structural change. Econometrica 1996, 64, 1045–1065. [Google Scholar] [CrossRef]
  17. Horváth, L.; Hušková, M.; Kokoszka, P.; Steinebach, J. Monitoring changes in linear models. J. Stat. Plan. Inference 2004, 126, 225–251. [Google Scholar] [CrossRef]
  18. Berkes, I.; Gombay, E.; Horvárh, L.; Kokoszaka, P. Sequential change-point detection in GARCH(p,q) models. Econ. Theory 2004, 20, 1140–1167. [Google Scholar] [CrossRef]
  19. Gombay, E.; Serban, D. Monitoring parameter change in AR(p) time series models. J. Multivar. Anal. 2009, 100, 715–725. [Google Scholar] [CrossRef]
  20. Na, O.; Lee, Y.; Lee, S. Monitoring parameter change in time series models. Stat. Methods Appl. 2011, 20, 171–199. [Google Scholar] [CrossRef]
  21. Na, O.; Lee, J.; Lee, S. Monitoring test for stability of copula parameter in time series. J. Korean Stat. Soc. 2014, 43, 483–501. [Google Scholar] [CrossRef]
  22. Lee, S.; Lee, Y.; Na, O. Monitoring distributional changes in autoregressive models. Commun. Stat. 2009, 38, 2969–2982. [Google Scholar] [CrossRef]
  23. Fermanian, J.D.; Radulović, D.; Wegkamp, M. Weak convergence of empirical copula processes. Bernoulli 2004, 10, 847–860. [Google Scholar] [CrossRef]
  24. Rémillard, B.; Scaillet, O. Testing for equality between two copulas. J. Multivar. Anal. 2009, 100, 377–386. [Google Scholar] [CrossRef]
  25. Bücher, A.; Dette, H. A note on bootstrap approximations for the empirical copula process. Stat. Probab. Lett. 2010, 80, 1925–1932. [Google Scholar] [CrossRef]
  26. Bouzebda, S. On the strong approximation of bootstrapped empirical copula process with applications. Math. Methods Stat. 2012, 21, 153–188. [Google Scholar] [CrossRef]
  27. Joe, H. Multivariate Models and Dependence Concepts; Chapman and Hall: London, UK, 1997. [Google Scholar]
  28. Nelsen, R.B. An Introduction to Copulas; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
  29. Na, O.; Lee, J.; Lee, S. Change point detection in SCOMDY models. AStA Adv. Stat. Anal. 2013, 97, 215–238. [Google Scholar] [CrossRef]
  30. Adler, R.J. An introduction to continuity, extrema, and related topics for general Gaussian processes. In Institute of Mathematical Statistics Lecture Notes; Monograph Series; Institute of Mathematical Statistics: Hayward, CA, USA, 1990; Volume 12, pp. 1–41. [Google Scholar]
  31. Piterbarg, V.I. Asymptotic Methods in the Theory of Gaussian Processes and Fields; American Mathematical Society: Providence, RI, USA, 1996. [Google Scholar]
  32. Csörgő, M.; Horváth, L. A note on strong approximations of multivariate empirical processes. Stoch. Process. Their Appl. 1988, 28, 101–109. [Google Scholar] [CrossRef]
  33. Segers, J. Asymptotics of empirical copula processes under non-restrictive smoothness assumptions. Bernoulli 2012, 18, 764–782. [Google Scholar] [CrossRef]
  34. Tsukahara, H. Empirical Copulas and Some Applications; Seijo University: Tokyo, Japan, 2000. [Google Scholar]
  35. Zhao, W.; Khalil, M.A.K. The Relationship between Precipitation and Temperature over the Contiguous United States. J. Clim. 1992, 6, 1232–1236. [Google Scholar] [CrossRef]
  36. Huang, J.; Van Den Dool, H.M. Monthly precipitation-temperature relations and temperature prediction over the United States. J. Clim. 1993, 6, 1111–1132. [Google Scholar] [CrossRef]
  37. Favre, A.C.; Adlouni, S.E.; Perreault, L.; Thiémonge, N.; Bobée, B. Multivariate hydrological frequency analysis using copulas. Water Resour. Res. 2004, 40, 1–12. [Google Scholar] [CrossRef]
  38. Shiau, J.T.; Feng, S.; Nadarajah, S. Assessment of hydrological droughts for the Yellow River, China, using copulas. Hydrol. Process. 2007, 21, 2157–2163. [Google Scholar] [CrossRef]
  39. Dupuis, D.J. Using copulas in hydrology: Benefits, cautions, and issues. J. Hydrol. Eng. 2007, 12, 381–393. [Google Scholar] [CrossRef]
  40. Schölzel, C.; Friederichs, P. Multivariate non-normally distributed random variables in climate research—Introduction to the copula approach. Nonlinear Process. Geophys. 2008, 15, 761–772. [Google Scholar] [CrossRef]
  41. Doukhan, P.; Fermanian, J.D.; Lang, G. An empirical central limit theorem with applications to copulas under weak dependence. Stat. Inference Stoch. Process. 2009, 12, 65–87. [Google Scholar] [CrossRef]
  42. Bücher, A.; Volgushev, S. Empirical and sequential empirical copula processes under serial dependence. J. Multivar. Anal. 2013, 119, 61–70. [Google Scholar] [CrossRef]
Figure 1. Annual mean temperature and annual mean precipitation in summer months over the contiguous United States from 1895 to 2015. (a) Annual mean temperature in summer; (b) Annual mean precipitation in summer.
Figure 1. Annual mean temperature and annual mean precipitation in summer months over the contiguous United States from 1895 to 2015. (a) Annual mean temperature in summer; (b) Annual mean precipitation in summer.
Entropy 18 00457 g001
Table 1. Empirical sizes.
Table 1. Empirical sizes.
Modelnα
0 . 01 0 . 05 0 . 10
1000.0070.0370.081
(0.023)(0.080)(0.124)
G a u s s i a n 2000.0150.0440.109
τ 0 = 0 . 13 (0.020)(0.056)(0.096)
3000.0190.0590.124
(0.009)(0.043)(0.086)
1000.0080.0400.101
(0.050)(0.109)(0.158)
G u m b e l 2000.0110.0490.125
τ 0 = 0 . 13 (0.020)(0.080)(0.131)
3000.0150.0570.133
(0.018)(0.061)(0.125)
Table 2. Empirical powers.
Table 2. Empirical powers.
n k * q < q =
T = 2 n T = 3 n T = 4 n T = 5 n T = nlogn
H 1 ( 1 ) : Gaussian with τ 0 = 0 . 13 → Gaussian with τ 0 = 0 . 35
change at k * = 1 . 1 n
1001100.4420.5940.6130.6180.617
(0.268)(0.597)(0.721)(0.793)(0.770)
2002200.6450.7910.8070.8120.813
(0.606)(0.917)(0.976)(0.983)(0.984)
3003300.8240.9330.9470.9470.949
(0.836)(0.983)(0.994)(0.997)(0.997)
change at k * = 1 . 5 n
1001500.2910.4200.4920.5010.495
(0.077)(0.360)(0.570)(0.678)(0.643)
2003000.4880.6010.6230.6330.640
(0.145)(0.693)(0.897)(0.956)(0.961)
3004500.6510.7910.8570.8900.902
(0.224)(0.883)(0.970)(0.991)(0.992)
H 1 ( 2 ) : Gaussian with τ 0 = 0 . 13 → Student t with τ 0 = 0 . 35
change at k * = 1 . 1 n
1001100.4370.5580.5810.5930.593
(0.282)(0.582)(0.694)(0.751)(0.735)
2002200.6020.7630.7890.7990.801
(0.558)(0.884)(0.944)(0.965)(0.969)
3003300.8110.9160.9210.9250.927
(0.774)(0.980)(0.997)(0.998)(1.000)
change at k * = 1 . 5 n
1001500.2150.3260.4610.4750.471
(0.091)(0.357)(0.522)(0.615)(0.587)
2003000.4010.5310.5920.5990.602
(0.127)(0.661)(0.872)(0.936)(0.947)
3004500.6230.7700.8310.8680.881
(0.218)(0.843)(0.972)(0.990)(0.996)
H 1 ( 3 ) : Gaussian with τ 0 = 0 . 13 → Frank with τ 0 = 0 . 35
change at k * = 1 . 1 n
1001100.4490.6010.6380.6440.639
(0.268)(0.554)(0.682)(0.755)(0.734)
2002200.6700.8150.8360.8400.849
(0.539)(0.884)(0.963)(0.980)(0.981)
3003300.8330.9520.9630.9760.977
(0.772)(0.980)(0.994)(0.999)(0.999)
change at k * = 1 . 5 n
1001500.3010.4510.5030.5210.510
(0.064)(0.339)(0.526)(0.634)(0.600)
2003000.4970.6310.6510.6890.689
(0.127)(0.638)(0.859)(0.922)(0.929)
3004500.6710.8110.8830.9150.921
(0.183)(0.836)(0.967)(0.993)(0.996)
Table 3. Empirical powers.
Table 3. Empirical powers.
n k * q < q =
T = 2 n T = 3 n T = 4 n T = 5 n T = nlogn
H 1 ( 1 ) : Gaussian with τ 0 = 0 . 13 → Gaussian with τ 0 = 0 . 60
change at k * = 1 . 1 n
1001100.5580.7080.7600.8350.815
(0.966)(1.000)(1.000)(1.000)(1.000)
2002200.8730.9390.9640.9870.990
(1.000)(1.000)(1.000)(1.000)(1.000)
3003300.9630.9931.0001.0001.000
(1.000)(1.000)(1.000)(1.000)(1.000)
change at k * = 1 . 5 n
1001500.3700.5460.6510.7790.741
(0.333)(0.978)(0.999)(1.000)(1.000)
2003000.6280.8290.9130.9710.979
(0.711)(1.000)(1.000)(1.000)(1.000)
3004500.8820.9620.9851.0001.000
(0.904)(1.000)(1.000)(1.000)(1.000)
H 1 ( 2 ) : Gaussian with τ 0 = 0 . 13 → Student t with τ 0 = 0 . 60
change at k * = 1 . 1 n
1001100.5570.6970.7560.8330.813
(0.940)(0.999)(1.000)(1.000)(1.000)
2002200.8610.9380.9580.9890.992
(1.000)(1.000)(1.000)(1.000)(1.000)
3003300.9600.9910.9971.0001.000
(1.000)(1.000)(1.000)(1.000)(1.000)
change at k * = 1 . 5 n
1001500.3700.5480.6430.7630.733
(0.312)(0.957)(0.996)(1.000)(1.000)
2003000.6310.8320.9120.9770.981
(0.701)(1.000)(1.000)(1.000)(1.000)
3004500.8770.9440.9830.9991.000
(0.888)(1.000)(1.000)(1.000)(1.000)
H 1 ( 3 ) : Gaussian with τ 0 = 0 . 13 → Frank with τ 0 = 0 . 60
change at k * = 1 . 1 n
1001100.6610.7910.8580.9180.901
(0.912)(0.995)(1.000)(1.000)(1.000)
2002200.9220.9770.9900.9991.000
(1.000)(1.000)(1.000)(1.000)(1.000)
3003300.9861.0001.0001.0001.000
(1.000)(1.000)(1.000)(1.000)(1.000)
change at k * = 1 . 5 n
1001500.4430.6380.7210.8200.802
(0.277)(0.930)(0.996)(1.000)(1.000)
2003000.7090.9050.9550.9951.000
(0.595)(1.000)(1.000)(1.000)(1.000)
3004500.9120.9831.0001.0001.000
(0.835)(1.000)(1.000)(1.000)(1.000)
Table 4. Empirical powers.
Table 4. Empirical powers.
n k * q < q =
T = 2 n T = 3 n T = 4 n T = 5 n T = nlogn
H 1 ( 1 ) : Gumbel with τ 0 = 0 . 13 → Gumbel with τ 0 = 0 . 60
change at k * = 1 . 1 n
1001100.5160.6410.7060.7940.750
(0.616)(1.000)(1.000)(1.000)(1.000)
2002200.8060.9070.9350.9690.976
(1.000)(1.000)(1.000)(1.000)(1.000)
3003300.9420.9800.9861.0001.000
(1.000)(1.000)(1.000)(1.000)(1.000)
change at k * = 1 . 5 n
1001500.3340.4870.5880.7170.656
(0.244)(0.925)(0.994)(0.999)(0.999)
2003000.5680.7640.8450.9560.970
(0.787)(1.000)(1.000)(1.000)(1.000)
3004500.7200.9060.9540.9921.000
(0.905)(1.000)(1.000)(1.000)(1.000)
H 1 ( 2 ) : Gumbel with τ 0 = 0 . 13 → Clayton with τ 0 = 0 . 60
change at k * = 1 . 1 n
1001100.6510.6800.7510.8450.799
(0.702)(0.968)(0.993)(1.000)(1.000)
2002200.8610.9430.9710.9941.000
(0.984)(1.000)(1.000)(1.000)(1.000)
3003300.9591.0001.0001.0001.000
(1.000)(1.000)(1.000)(1.000)(1.000)
change at k * = 1 . 5 n
1001500.4340.5790.6270.7850.698
(0.358)(0.901)(0.954)(1.000)(1.000)
2003000.6060.8080.8870.9770.993
(0.602)(0.988)(0.999)(1.000)(1.000)
3004500.8200.9560.9931.0001.000
(0.893)(1.000)(1.000)(1.000)(1.000)
H 1 ( 3 ) : Gumbel with τ 0 = 0 . 13 → Frank with τ 0 = 0 . 60
change at k * = 1 . 1 n
1001100.5450.6750.7290.8230.770
(0.731)(0.958)(0.991)(0.993)(0.993)
2002200.8700.9430.9750.9940.998
(0.977)(0.999)(1.000)(1.000)(1.000)
3003300.9531.0001.0001.0001.000
(0.997)(1.000)(1.000)(1.000)(1.000)
change at k * = 1 . 5 n
1001500.3480.5210.6070.7510.689
(0.381)(0.930)(0.996)(1.000)(1.000)
2003000.6180.820.9750.9940.998
(0.581)(0.988)(0.999)(1.000)(1.000)
3004500.9120.9831.0001.0001.000
(0.881)(1.000)(1.000)(1.000)(1.000)
Table 5. The stopping time.
Table 5. The stopping time.
α τ ^ ( n ) Year
0 . 01 1142008
0 . 05 1092003
0 . 10 1072001

Share and Cite

MDPI and ACS Style

Lee, J.; Kim, B. Monitoring Test for Stability of Dependence Structure in Multivariate Data Based on Copula. Entropy 2016, 18, 457. https://doi.org/10.3390/e18120457

AMA Style

Lee J, Kim B. Monitoring Test for Stability of Dependence Structure in Multivariate Data Based on Copula. Entropy. 2016; 18(12):457. https://doi.org/10.3390/e18120457

Chicago/Turabian Style

Lee, Jiyeon, and Byungsoo Kim. 2016. "Monitoring Test for Stability of Dependence Structure in Multivariate Data Based on Copula" Entropy 18, no. 12: 457. https://doi.org/10.3390/e18120457

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop