1. Introduction and Problem Formulation
The correlation coefficient is a principal statistical tool and the most widely used measure of a relationship between two variables [
1]. When using this tool, it is assumed that the nature of the relationship is linear and can be modelled by a simple linear regression. Large natural systems, however, such as the climate system, exhibit behaviors that are far more complex and seem to require equally complex, possibly nonlinear, models to simulate them adequately. An alternative view, which is adopted here, is that such systems may be governed by simple rules, but the parameters of those rules are changing, possibly in an abrupt fashion. Indeed, quasi-permanent regimes with abrupt changes between them are characteristic of the climate system and have been observed on different time scales [
2,
3,
4,
5,
6,
7]. In this paradigm, a major task would be to determine the timing of those abrupt changes, also known as breaks, regime shifts, discontinuities, or change-points. In practice, when new data arrive over time, it is also important to allow for monitoring of those shifts and detect them as soon as possible.
Given this framework, a relationship between two variables can be written in a sequential manner as a dynamic regression model [
8]:
where
and
are observations at time
of the independent and dependent variables, respectively,
are the regression coefficients and
are the error terms or residuals. The sample
is called the historic sample, for which observations are available, and new observations are expected to arrive after time
. Let this historic sample be a random sample from a bivariate normal distribution with means
variances
and population correlation coefficient
. The idea is to estimate the regression coefficient just once for the history period
assuming it is constant during all that time,
i.e.,
. Alternatively, one can estimate the correlation coefficient instead, because in a simple linear regression, the values of the ordinary least squares (OLS) estimate
and the Pearson’s correlation coefficient
are the same when the time series
and
are normalized by their respective standard deviations
. Then a process is devised that captures fluctuations either in estimates or in residuals of a regression model during the monitoring period. Whenever there are excessive fluctuations in these processes, that is,
or
for
become significantly different from
or
respectively, or the residuals
deviate systematically from their zero mean, the null hypothesis of stability is rejected and a regime shift is declared.
This approach works well in statistical quality control applications, when the underlying process is known and the constancy of the correlation coefficient during the history period can be assured [
9]. In the case of natural systems, however, there is no guarantee that the data collected came from the same population. On the contrary, it is quite possible that there were already multiple regime shifts during the history period that occurred at unknown times. Therefore, a more appropriate model would be
where each change point
is the first point of regime
. The initial task here is to find the number of regimes
and locations of change-points
before the monitoring phase can begin. To resolve this task, both sequential and non-sequential (also known as retrospective or “ex-post”) approaches can be used. The retrospective tests usually fall into one of the two categories: (1) Cumulative sum (CUSUM) and fluctuation tests or (2)
-tests, such as the Chow test [
8]. The classic CUSUM tests are based on the behavior of recursive residuals
[
10]. Fluctuation tests are CUSUM-type tests, but deal with changes in estimates of the correlation coefficients
[
8]. While the tests in the first category can be used in both retrospective and monitoring settings, the
-tests cannot be applied to monitor out-of-sample stability each time new data arrive [
11]. Due to the law of the iterated logarithm, repeated application of such tests yields a procedure that rejects a true null hypothesis of no change with probability approaching one as the number of applications grows [
12]. In addition, all retrospective tests have a common problem: the drastic deterioration of the test statistics toward the end of time-series. Therefore, the null hypothesis of constancy of the regression coefficient for the last regime in the historic sample,
, would be weak and the transition to the monitoring phase would be problematic.
Staying within the sequential framework, one can use subsamples of the data (e.g., in a form of a sliding window as in Leisch
et al. [
13]), assuming that the subsample size is equal or smaller than the length of anticipated regimes. This will improve data homogeneity within the subsamples, but increase the standard error of the correlation coefficient. It is known that the Pearson’s
is a biased estimator of
, and the bias increases as the sample size decreases [
14]. The bias can be substantially reduced by an estimator suggested by Fisher [
15] and almost completely eliminated by a related estimator recommended by Olkin and Pratt [
16]. A much bigger issue, however, is that even if the correlation coefficient remains constant within a subsample, there might be changes in the means and variances in one or both variables occurring at unknown times. In these circumstances, an accurate estimation of the correlation coefficient directly from data, without addressing this issue first, is not feasible.
A solution suggested here represents a three-step procedure discussed in detail in
Section 2. In
Section 3, this procedure is tested using synthetic time series with predefined statistics. It is demonstrated that failure to remove regime shifts in the mean and variance first may result in spurious regime shifts in the correlation, whereas true regime shifts may not be detected at all.
Section 4 discusses a real world example of structural changes in the Bering Sea climate. This example was previously used by Wang
et al. [
17] who found a shift in the Arctic influence on the region in the 1960s. Since they used smoothed (by 5-yr running means) temperature data series and 25-yr running correlation coefficients as a tool, the timing of the shift was approximate. The shift is reexamined in
Section 4 using the proposed method. The results are summarized in
Section 5.
3. An Example with Synthetic Time Series
The above three-step algorithm was implemented as a software package called the Sequential Regime Shift Detector (SRSD). It is written in VBA for MS Excel and is available at [
33].
To illustrate the software capabilities, two correlated random time series of size
= 70 were generated from the bivariate normal distribution with
and
. The change-point for the correlation coefficient from regime 1 to regime 2 was set in the middle of the series at
= 36, with
= −0.6 in the first half and
= 0.6 in the second half. Then the shifts in the variance were added to time series
from
= 1 to
= 9 at
= 51 and to time series
from
= 9 to
= 1 at
= 21. Finally, the shifts in the mean were superimposed to
from
= −1 to
= 1 at
= 26 and to
from
= 1 to
= −1 at
= 41. The resultant time series are presented in
Figure 1.
Figure 1.
Randomly generated time series with shifts in the correlation coefficient, variance and mean as described in the text.
Figure 1.
Randomly generated time series with shifts in the correlation coefficient, variance and mean as described in the text.
The addition of regime shifts in the variance and, particularly, in the mean significantly changes the direct OLS estimates of
. As shown in
Figure 2, in the presence of those shifts, sample correlation coefficient
tends to underestimate
by as much as 0.6. In some other cases, when the shifts in the mean and variance are placed differently, it may lead to an overestimation of
. Therefore, in order to detect shifts in the correlation coefficient accurately, it is necessary to remove shifts in the mean and variance first.
The SRSD starts by detecting shifts in the mean. In this example, the target significance level and the cut-off length are set at
= 0.05 and
= 20. It is always good to know an approximate length of the regimes in time series to set these two parameters right, but since there is only one strong regime shift in the mean in each of the series, some variations in
and
do not change the results.
Figure 3a,b show that the detected change points
= 26 and
= 41 in
and
, respectively, coincide with the theoretical change points. The calculated
–values in both cases are much lower than 0.05.
After the stepwise trends are removed, the detection of regime shifts in the variance is performed using the same target significance level
= 0.05 and the cut-off length
= 20. The increase in the variance in
(
Figure 3c) and its decrease in
(
Figure 3d) were both accurately detected at
= 51 and
= 21, respectively.
Figure 2.
Running 21-point cross-correlations before and after the shifts in the variance and mean are added to the time series.
Figure 2.
Running 21-point cross-correlations before and after the shifts in the variance and mean are added to the time series.
Figure 3.
Regime shifts detected in the mean (top row) and variance (bottom row) in time series (left column) and (right column). In all four cases the detection is performed with the target significance level = 0.05 and cut-off length = 20.
Figure 3.
Regime shifts detected in the mean (top row) and variance (bottom row) in time series (left column) and (right column). In all four cases the detection is performed with the target significance level = 0.05 and cut-off length = 20.
In its final step, the SRSD uses the normalized time series to detect regime shifts in the correlation coefficient. As shown in
Figure 4a, the regime shift in our synthetic time series is found at
= 36, which is precisely where it is supposed to be. Interestingly, when the first two steps are skipped, that is the regime shifts in the mean and variance are not removed, a shift in
is incorrectly detected at
= 21 (
Figure 4b). Although the difference in
before and after this spurious shift appears to be statistically significant (the exact
-value cannot be calculated due to violation of the assumptions for
), its timing is way off.
Figure 4.
Regime shifts in the correlation coefficient (
a) correctly detected at
= 36, after shifts in the mean and variance are removed; and (
b) a spurious shift at
= 21, when the SRSD is applied directly to the original series in
Figure 1.
Figure 4.
Regime shifts in the correlation coefficient (
a) correctly detected at
= 36, after shifts in the mean and variance are removed; and (
b) a spurious shift at
= 21, when the SRSD is applied directly to the original series in
Figure 1.
4. Change in the Climatological Structure of the Bering Sea
Wang
et al. [
17] reported a change of unknown origin in the climatological structure of the Bering Sea that occurred sometime in the 1960s. Using the running 25-yr correlation coefficients between winter (DJFM) surface air temperatures (SAT) at two Alaskan stations, Barrow (71.3°N, 156.8°W) and St. Paul (57.1°N, 170.2°W), they found a shift from about 0.2 to 0.7 between the 1940s and 1980s. They concluded that the shift was statistically significant at the 95% confidence level, based on their Monte Carlo simulations.
With the approach they used, it was difficult to determine the exact timing of the shift, especially when they smoothed the SAT time series with running 5-yr means, which only increased the autocorrelation in the series. In addition, a visual inspection of their
Figure 2a reveals a strong shift in SAT at Barrow, from a cold regime before the late 1970s to a warm regime thereafter. As shown in
Section 3, it may lead to large errors in estimation of the correlation coefficient.
The station SAT data used in this study was obtained from GISS [
34]. To facilitate a comparison between SATs at Barrow and St. Paul, the data for both stations is presented as anomalies from the 1971–2000 base period normalized by the standard deviations for the same period.
Figure 5a shows that SAT at Barrow experienced an abrupt, and statistically highly significant, shift upward in 1978, the
-value of which was
. It coincided with a major step in the North Pacific climate in the late 1970s [
35]. Interestingly, no shift at that time was detected in SAT at St. Paul, despite its proximity to the North Pacific Ocean (
Figure 5b). It should be noted that, using the same method, a weak shift in 1977 was detected for that station in [
36]. The discrepancy can be explained by the difference in the data sources used, that is, the raw data from NCDC in [
36] and adjusted data from GISS here. Apparently, the homogeneity adjustment technique used in GISS made the 1977 shift in SAT at St. Paul even weaker. In any case, the absence of a pronounced shift in SAT at St. Paul is a reflection of the complex climate dynamics in the region and the lack of a clear-cut connection with the major modes of climate variability in the Pacific. As shown in [
37], the correlation coefficients between winter SAT at St. Paul and such indices as the Pacific Decadal Oscillation, Aleutian Low Pressure, Pacific-North American, and Southern Oscillation are all statistically insignificant.
Figure 5.
Regime shifts in the mean winter (DJFM) surface air temperatures (SATs) at (
a) Barrow and (
b) St. Paul, 1921–2015. The detection was performed using the target significance level
= 0.1 and cut-off length
= 15. The autocorrelation coefficients, estimated using the IP4 method [
30], are close to zero in both series. The
-values for the change-points in 1978 for Barrow and in 2007 for St. Paul are
and 0.07, respectively.
Figure 5.
Regime shifts in the mean winter (DJFM) surface air temperatures (SATs) at (
a) Barrow and (
b) St. Paul, 1921–2015. The detection was performed using the target significance level
= 0.1 and cut-off length
= 15. The autocorrelation coefficients, estimated using the IP4 method [
30], are close to zero in both series. The
-values for the change-points in 1978 for Barrow and in 2007 for St. Paul are
and 0.07, respectively.
The relative stability of temperature fluctuations at St. Paul was interrupted in 2007, with the onset of a very cold period that lasted through 2013 and included a record cold winter of 2012. This period was characterized by a deep and persistent low-pressure center located in the Gulf of Alaska [
38], which implies the advection of cold continental air over the southeastern Bering Sea [
36].
When the regime shifts in the mean were removed, the maximum r between SAT residuals at Barrow and St. Paul for any 25-yr sample decreased from 0.7 to 0.6. Since no regime shifts in the variance were detected, step two in the three-step procedure was skipped, and the test for shifts in the correlation coefficient was performed using the residuals after the first step. The target significance level and cut-off length were the same as for the tests of shifts in the mean (i.e., target = 0.1 and = 15).
Figure 6 shows two strong correlation regimes, 1921–1939 (
= 0.59, 90% confidence interval: 0.25–0.79) and 1967–2015 (0.69, 0.54–0.80), separated by a period with no correlation between Barrow and St. Paul. The
-values for the regime shifts in 1940 and 1967 are 0.04 and 0.001, respectively. Here the focus will be on the latter shift due to its higher statistical significance and more readily available meteorological data to explore it further.
Figure 6.
Regime shifts in the correlation coefficient between winter SATs at Barrow and St. Paul detected using the target significance level = 0.1 and cut-off length = 15. The -values for the change-points in 1940 and 1967 are 0.04 and 0.001, respectively. The correlation coefficients, along with their 90% confidence intervals, for the three detected regimes are 0.59 (0.25 0.79) for 1921–1939, −0.01 (−0.33 0.31) for 1940–1966, and 0.69 (0.54 0.80) for 1967–2015.
Figure 6.
Regime shifts in the correlation coefficient between winter SATs at Barrow and St. Paul detected using the target significance level = 0.1 and cut-off length = 15. The -values for the change-points in 1940 and 1967 are 0.04 and 0.001, respectively. The correlation coefficients, along with their 90% confidence intervals, for the three detected regimes are 0.59 (0.25 0.79) for 1921–1939, −0.01 (−0.33 0.31) for 1940–1966, and 0.69 (0.54 0.80) for 1967–2015.
Baines and Folland [
5] reported a number of rapid climate changes in various parts of the globe centered on the late 1960s. In the Bering Sea region, that was the time when atmospheric circulation changed from a generally zonal flow to a meridional pattern [
17]. Wang
et al. [
17] demonstrated that Arctic and Pacific air had fewer meridional excursions before the late 1960s. They alluded, however, that since the increase in the north/south correlation structure did not begin near the well-known shift in the late 1970s, but about a decade earlier, it might be more related to changes in the Siberian High.
The Siberian High is part of the so-called Siberian-Alaskan Index (SAI) designed specifically to characterize the effect of atmospheric circulation on thermal conditions in the Bering Sea. The SAI is defined as a difference between the mean winter (DJFM) normalized 700-hPa anomalies in two regions, Siberia (55°N–70°N, 90°E–150°E) and Alaska/Yukon (60°N–70°N, 130°W–160°W). Positive (negative) values of the index indicate anomalously strong north-westerly (south-easterly) winds and colder (warmer) than normal winters in the Bering Sea. The SAI is available at the Bering Climate web site [
39].
It should be emphasized that the SAI is not a dipole, as, for example, the North Atlantic Oscillation. The two parts of the SAI, the Siberian Index (SI) and Alaskan Index (AI), are completely uncorrelated, and therefore, represent two independent sources of influence on Bering Sea climate. The SI reflects the strength of the Siberian High (
Figure 7a) and the advection of cold Siberian air into the Bering Sea. Judging from the correlations between the SI and winter temperatures computed for the entire period of observations, 1949–2015 (
Figure 7c), its effect on winter conditions in the Bering Sea is rather modest. Note that the correlation maps in
Figure 7 are computed for the first differences rather than for absolute values. Taking first differences serves as a high-pass filter that practically eliminates the effect of regime shifts in the mean on the correlation coefficient.
Figure 7.
Correlation maps for the Siberian (left column) and Alaskan (right column) indices in the mean winter (DJFM) 700-hPa (a,b) height and (c,d) SAT fields. The correlation coefficients were calculated using the entire dataset, 1949–2015. The first differences filter () was applied. Areas where the correlation coefficients are statistically significant at the 90% confidence level are colored.
Figure 7.
Correlation maps for the Siberian (left column) and Alaskan (right column) indices in the mean winter (DJFM) 700-hPa (a,b) height and (c,d) SAT fields. The correlation coefficients were calculated using the entire dataset, 1949–2015. The first differences filter () was applied. Areas where the correlation coefficients are statistically significant at the 90% confidence level are colored.
The 700-hPa correlation map for the AI is shown in
Figure 7b. Fang and Wallace [
40] demonstrated the importance of a high-pressure center over Alaska for thermal conditions in the Bering Sea. Using the singular value decomposition (SVD) technique for sea ice concentration in the North Pacific and the hemispheric 500-hPa field, they showed that the leading SVD mode was characterized by a dominant center of action over Alaska. Blocking over Alaska (positive AI values) prevents storms from entering the Gulf of Alaska and redirects them into the Bering Sea. These storms bring warm Pacific air and push the ice edge northward. Negative AI values are indicative of advection of cold Arctic air and rapid advance of ice edge southward. A comparison of the SAT correlation maps (
Figure 7c,d) shows that the linear effect of the Alaskan center of action on SATs in the Bering Sea is much stronger than that of the Siberian High.
The AI is strongly correlated with SAT at Barrow as well, which can be seen in
Figure 7c. For the entire period of observations, 1949–2015, the correlation coefficient between the AI and residuals of SAT at Barrow, after removing the stepwise trend, is 0.61.
Figure 8.
Regime shifts in the mean (top row) and variance (bottom row) for the Siberian (left column) and Alaskan (right column) indices. In all four cases, the target significance level and cut-off length were set at 0.1 and 15, respectively. The change-points and their -values (in parentheses) are: (a) 1966 (0.003), 2005 (0.006); (b) none detected; (c) 2001 (0.04); and (d) 2012 (not calculated due to a small number of points after the shift).
Figure 8.
Regime shifts in the mean (top row) and variance (bottom row) for the Siberian (left column) and Alaskan (right column) indices. In all four cases, the target significance level and cut-off length were set at 0.1 and 15, respectively. The change-points and their -values (in parentheses) are: (a) 1966 (0.003), 2005 (0.006); (b) none detected; (c) 2001 (0.04); and (d) 2012 (not calculated due to a small number of points after the shift).
The results of regime shift detection in the mean and variance in the SI and AI are presented in
Figure 8. Prior to 1966, the SI was mostly positive, indicating a strong Siberian High (
Figure 8a). Another period of predominantly positive SI started in 2005. No shifts in the mean were detected for the AI (
Figure 8b). As for the variance, a statistically significant decrease in the SI fluctuations since 2001 was detected (
Figure 8c). In contrast, the AI index experienced a remarkable increase in its fluctuations in recent years, when it jumped from a record low value in 2012 to a record high value in 2014 (
Figure 8d). However, due to a small number of observations in this high variance regime, normalization for the AI was not performed.
The results of regime shift detection in the correlation coefficient between SAT at St. Paul and the two atmospheric circulation indices, SI and AI, are presented in
Figure 9. Although the effect of the SI on temperature in the Bering Sea appeared to be rather weak overall (
Figure 7c), there was a strong correlation regime from 1966 through 1997, when
reached −0.54, a statistically significant value at the 99.8% confidence level (
Figure 9a). The linear relationship between the AI and SAT at St. Paul (
Figure 9b) was weak at the beginning of the series (1949–1967), but then
jumped from 0.27 to 0.70. It means that about 50% of SAT variance at St. Paul during the 1968–2015 period can be explained by the AI alone.
Figure 9.
Regime shifts in the correlation coefficient between SAT at St. Paul and (a) Siberian and (b) Alaskan indices. The target significance level and cut-off length were set at 0.3 and 15, respectively, for the pair SI–St. Paul, and 0.1 and 15 for the pair AI–St. Paul. The change-points and their -values (in parentheses) are: (a) 1966 (0.13); 1998 (0.09) and (b) 1968 (0.05).
Figure 9.
Regime shifts in the correlation coefficient between SAT at St. Paul and (a) Siberian and (b) Alaskan indices. The target significance level and cut-off length were set at 0.3 and 15, respectively, for the pair SI–St. Paul, and 0.1 and 15 for the pair AI–St. Paul. The change-points and their -values (in parentheses) are: (a) 1966 (0.13); 1998 (0.09) and (b) 1968 (0.05).
It is interesting to compare atmospheric circulation patterns for the correlation regimes in
Figure 9. As shown in
Figure 10, the most prominent feature of atmospheric circulation during the strong correlation regime for the SI, 1966–1997, is a strong Siberian—North Pacific (NP) dipole in the 700-hPa field, with the correlation coefficient between the two centers reaching 0.8 (
Figure 10c). During a positive phase of the dipole (SI+, NP−), the pressure gradient between the Siberian High and Aleutian low increases that leads to an enhanced advection of Arctic air directed into the Bering Sea. During its negative phase (SI−, NP+), the advection of Arctic air decreases, while the advection of warm Pacific air increases, apparently due to a more active Siberian storm track [
36]. The NP center is weaker during the weak correlation regime of 1949–1965 (
Figure 10a) and completely disappears during the recent weak correlation regime of 1998–2015 (
Figure 10e).
Figure 11 shed some light on why the correlation coefficient between the AI and SAT at St. Paul was low during the period of 1949–1967. A comparison of
Figure 11a,c shows that during the earlier period, the Alaskan center of action was expanded westward. In that situation the advection of warm Pacific air (AI+), or cold Arctic air (AI−) was directed into the western part of the Bering Sea, where the correlation coefficients with SATs reached 0.8, whereas they were statistically insignificant in the eastern part (
Figure 11b). Note that the correlation between the AI and SAT at Barrow remained strong during this earlier regime, and that the shift in the correlation for the pairs AI–St. Paul and Barrow—St. Paul occurred almost at the same time. This indicates that the primary reason for the break in the correlation for the latter pair in 1967 is likely to be the westward expansion and subsequent contraction of the Alaskan center of action, and not the change in the Arctic influence as suggested by Wang
et al. [
17].
Figure 10.
Correlation maps for the Siberian index in the mean winter 700-hPa height (left column) and SAT (right column) fields for three correlation regimes: (1) (a,b) 1949–1965; (2) (c,d) 1966–1997; and (3) (e,f) 1998–2015. The first differences filter was applied. Areas where the correlation coefficients are statistically significant at the 90% confidence level are colored.
Figure 10.
Correlation maps for the Siberian index in the mean winter 700-hPa height (left column) and SAT (right column) fields for three correlation regimes: (1) (a,b) 1949–1965; (2) (c,d) 1966–1997; and (3) (e,f) 1998–2015. The first differences filter was applied. Areas where the correlation coefficients are statistically significant at the 90% confidence level are colored.
Figure 11.
Same as
Figure 10, except for Alaskan index and two correlation regimes: (1) (
a,
b) 1949–1967 and (2) (
c,
d) 1968–2015.
Figure 11.
Same as
Figure 10, except for Alaskan index and two correlation regimes: (1) (
a,
b) 1949–1967 and (2) (
c,
d) 1968–2015.