Changepoint in Error-Prone Relations

Abstract

Linear relations, containing measurement errors in input and output data, are considered. Parameters of these so-called errors-in-variables models can change at some unknown moment. The aim is to test whether such an unknown change has occurred or not. For instance, detecting a change in trend for a randomly spaced time series is a special case of the investigated framework. The designed changepoint tests are shown to be consistent and involve neither nuisance parameters nor tuning constants, which makes the testing procedures effortlessly applicable. A changepoint estimator is also introduced and its consistency is proved. A boundary issue is avoided, meaning that the changepoint can be detected when being close to the extremities of the observation regime. As a theoretical basis for the developed methods, a weak invariance principle for the smallest singular value of the data matrix is provided, assuming weakly dependent and non-stationary errors. The results are presented in a simulation study, which demonstrates computational efficiency of the techniques. The completely data-driven tests are illustrated through problems coming from calibration and insurance; however, the methodology can be applied to other areas such as clinical measurements, dietary assessment, computational psychometrics, or environmental toxicology as manifested in the paper.

1. Introduction and Main Aims

If measured input and output data are supposed to be in some linear relations, then it is of particular interest to detect whether impact of the input characteristics has changed over time on the output observables. Moreover, only error-prone surrogates of the unobservable input-output characteristics are in hand instead of a precise measurement. Despite the fact that the relations and, consequently, suitable underlying stochastic models are linearly defined, the possible estimates and the corresponding inference may be highly non-linear [1]. It becomes even more challenging to handle measurement errors in input and output data simultaneously, when the linear relations are subject to change at some unknown time point—changepoint.

There is a vast literature aimed at linear relations modeled through so-called measurement error models or errors-in-variables models (for an overview, see [2,3,4,5], or [6]), but very little has been explored in the changepoint analysis for these models yet. A change in regression has been explored thoroughly, cf. [7] or [8]. However, such a framework does not cover the case of measurement error models. Maximum likelihood approach [9,10] and Bayesian approach [11,12] to the changepoint estimation in the measurement error models were applied, both requiring parametric distributional assumptions on the errors. The changepoint in the input data only was estimated in [13]. A change in the variance parameter of the normally distributed errors within the measurement error models was investigated in [14]. All of these mentioned contributions dealt with the changepoint estimation solely. Our main goal is to test for a possible change in the parameters relating the input and output data, both encumbered by some errors. Consequently, if a change is detected, we aim to estimate it. By our best knowledge, we are not aware of any similar results even for the independent and identically distributed errors. Additionally to that, our changepoint tests are supposed to be nuisance-parameter-free, distributional-free, and to allow for very general error structures.

Outline

The paper is organized as follows: In the next section, our data model for the changepoint in errors-in-variables is introduced and several practical motivations for such a model are given. Section 3 contains a spectral weak invariance principle for weakly dependent and non-stationary random variables. It serves as the main theoretical tool for the consequent inference. The technical assumptions are discussed as well. Two test statistics for the changepoint detection are proposed in Section 4. Consequently, their asymptotic behavior is derived under the null as well as under the alternative hypothesis. Moreover, a consistent changepoint estimator is introduced. Section 5 contains a simulation study that compares finite sample performance of the investigated tests. It numerically emphasizes the advantages of the proposed detection procedures. A practical application of the developed approach to a calibration problem is presented in Section 6.1. On the other hand, an actuarial application concerning randomly spaced time series is performed in Section 6.2. Afterwards, our conclusion follows. Proofs are given in the Appendix A.

2. Changepoint in Errors-In-Variables

Errors-in-variables (EIV) or also called measurement error model

$$\mathit{X}=\mathit{Z}+\mathbf{\Theta }$$

($ℳ$)

and

$$\mathit{Y}=\mathit{Z}\mathit{\beta }+\mathit{\epsilon }$$

($\mathscr{H}$₀)

is considered, where $\mathit{\beta }\in {\mathbb{R}}^p$ is a vector of unknown regression parameters possibly subject to change, $\mathit{X}\in {\mathbb{R}}^{n\times p}$ and $\mathit{Y}\in {\mathbb{R}}^{n\times 1}$ consist of observable random variables ($\mathit{X}$ are covariates and $\mathit{Y}$ is a response), $\mathit{Z}\in {\mathbb{R}}^{n\times p}$ consists of unknown constants and has full rank, $\mathit{\epsilon }\in {\mathbb{R}}^{n\times 1}$ and $\mathbf{\Theta }\in {\mathbb{R}}^{n\times p}$ are random errors. This setup can be extended to a multivariate case, where $\mathit{\beta }\in {\mathbb{R}}^{p\times q}$, $\mathit{Y}\in {\mathbb{R}}^{n\times q}$, and $\mathit{\epsilon }\in {\mathbb{R}}^{n\times q}$, $q\ge 1$, see Section 3.3.

The EIV model $\mathcal{M}$–${\mathcal{H}}_0$ with non-random unknown constants $\mathit{Z}$ is sometimes called functional EIV model [2,15]. On the other hand, a different approach may handle $\mathit{Z}$ as random covariates, which is called structural EIV model [9]. As stated by [16]: ‘However, functional models played an important role in the study of measurement error models and in statistics more generally.’ In addition, here, we will concentrate on the functional EIV model not because of this matter-of-fact quote, but because we wish to demonstrate a distributional-free approach, where ‘no, or only minimal, assumptions are made about the distribution of the $\mathit{X}$s’ [4], as challenged in the introduction. Nevertheless with respect to derivation of the forthcoming theory for the functional EIV model, changing some technical assumptions would allow proving suitable results for the structural case as well.

To estimate the unknown parameter $\mathit{\beta }$, one usually minimizes the Frobenius matrix norm of the errors $[\mathbf{\Theta },\mathit{\epsilon }]$, see [17]. This approach leads to a total least squares (TLS) estimate $\widehat{\mathit{\beta }}={({\mathit{X}}^{\top }\mathit{X}-{\lambda }_{min}\left({[\mathit{X},\mathit{Y}]}^{\top }[\mathit{X},\mathit{Y}]\right){\mathit{I}}_p)}^{-1}{\mathit{X}}^{\top }\mathit{Y}$, where ${\lambda }_{min}\left(\mathit{M}\right)$ is the smallest eigenvalue of an matrix $\mathit{M}$ and ${\mathit{I}}_p$ is a $(p\times p)$ identity matrix. Geometrically speaking, the Frobenius norm tries to minimize the orthogonal distance between the observations and the fitted hyperplane. Therefore, the TLS are usually known as orthogonal regression. One can generalize this method by replacing the Frobenius norm by any unitary invariance matrix norm, which surprisingly yields the same TLS estimate, with interesting invariance and equivariance properties [18]. The TLS estimate is shown to be strongly and weakly consistent [1,19,20] as well as to be asymptotically normal [21,22,23] under various conditions.

We aim to detect a possible change in the linear relation parameter $\mathit{\beta }$. The interest lies in testing the null hypothesis ${\mathcal{H}}_0$ of all observations $Y_i$’s being random variables with expectations ${\mathit{Z}}_{i,•}\mathit{\beta }$’s. Our goal is to test against the alternative of the first $\tau$ outcome observations have expectations ${\mathit{Z}}_{i,•}\mathit{\beta }$’s and the remaining $n-\tau$ observations come from distributions with expectations ${\mathit{Z}}_{i,•}(\mathit{\beta }+\mathit{\delta })$’s, where $\mathit{\delta }\ne \mathbf{0}$. A ‘row-column’ notation for a matrix $\mathit{M}$ is used in this manner: ${\mathit{M}}_{i,•}$ denotes the ith row of $\mathit{M}$ and ${\mathit{M}}_{•,j}$ corresponds to the jth column of $\mathit{M}$. Furthermore, if $i\in {\mathbb{N}}_0$, then ${\mathit{M}}_i$ stays for the first i rows of $\mathit{M}$ and ${\mathit{M}}_{-i}$ represents the remaining $n-i$ rows of $\mathit{M}$, when the first i rows are deleted. Now more precisely, our alternative hypothesis is

$${\mathit{Y}}_{\tau }={\mathit{Z}}_{\tau }\mathit{\beta }+{\mathit{\epsilon }}_{\tau } \mathrm{and} {\mathit{Y}}_{-\tau }={\mathit{Z}}_{-\tau }(\mathit{\beta }+\mathit{\delta })+{\mathit{\epsilon }}_{-\tau }.$$

($\mathscr{H}$_𝒜)

Here, $\mathit{\delta }\equiv \mathit{\delta }\left(n\right)\ne \mathbf{0}$ is an unknown vector parameter representing the size of change and is possibly depending on n. The changepoint $\tau \equiv \tau \left(n\right)<n$ is also an unknown scalar parameter, which depends on n as well, although $\mathit{\beta }$ is considered to be independent of n. One may also think of the changepoint in errors-in-variables framework as segmented regression with measurement errors, cf. [10].

2.1. Intercept and Fixed Regressors

Please note that the EIV model $\mathcal{M}$–${\mathcal{H}}_0$ has no intercept and all the covariates are encumbered by some errors. To overcome such a restriction, one can think of an extended regression model, where some explanatory variables are subject to error and some are measured precisely. i.e., $\mathit{Y}=\mathit{W}\mathit{\gamma }+\mathit{Z}\mathit{\beta }+\mathit{\epsilon }$, where $\mathit{W}$ are observable true and $\mathit{Z}$ are unobservable true constants, both with full rank. Regression parameters $\mathit{\gamma }$ and $\mathit{\beta }$ remain unknown. Then, the non-random (fixed) intercept can be incorporated into the regression model by setting one column of the matrix $\mathit{W}$ equal to ${[1,\dots ,1]}^{\top }$. Consequently, we may project out exact observations using projection matrix $\mathit{R}:={\mathit{I}}_n-\mathit{W}{\left({\mathit{W}}^{\top }\mathit{W}\right)}^{-1}{\mathit{W}}^{\top }$. Notice that $\mathit{R}$ is symmetric and idempotent. Finally, one may work with $\mathit{R}\mathit{Y}=\mathit{R}\mathit{Z}\mathit{\beta }+\mathit{R}\mathit{\epsilon }$ instead of ${\mathcal{H}}_0$.

2.2. Motivations

The proposed class of models—errors-in-variables with changepoint—is very rich and general. Our approach and results are motivated in the context of several applications taken from chemistry, biological sciences, medicine, epidemiological studies, and finance.

Application 1: Assessing agreement in clinical measurement

Direct measurement of cardiac stroke volume or blood pressure without adverse effects is difficult or even impossible. The true values remain unknown. Indirect methods are, therefore, used instead. When a new measurement technique is developed, it has to be evaluated by comparison with an established technique rather than with the true quantity [24]. Clinicians need to test whether both measurement techniques agree sufficiently. Thereafter, the old technique may be replaced by the new one.

Application 2: Nutritional epidemiology

Data from a nutritional study were analyzed in [10], where the relation between dietary folate intake (calories adjusted $\mathsf{\mu }$g/day) on plasma homocysteine concentration ($\mathsf{\mu }$mol/liter of blood) was investigated. There exists a suspicion that serum homocysteine is significantly elevated when ingested folate is below a certain changepoint. Moreover, the analysis used estimates of folate that were developed with a food frequency questionnaire, which is recognized to be imperfect.

Application 3: Psychometric testing

Let us think of two psychometric instruments: unspeeded 15-item vocabulary tests and highly speeded 75-item vocabulary tests, cf. [25]. The results of both tests are error-prone. Within a group of people, there is a speculation that individuals with an unspeeded test’s result exceeding some unknown level should perform dramatically better in the highly speeded test.

Application 4: Environmental toxicology

A threshold limiting value in toxicology is the dose of a toxin or a substance under which there is harmless or insignificant influence on some response. In a dose-response relationship, both of them are measured with errors. In addition, the goal is to set the threshold limiting value. Such a problem was dealt in [12] using fully Bayesian approach. Moreover, a similar task regarding the NO${}_2$ concentration is discussed in [16].

Application 5: Device calibration

Later on in Section 6.1, we concentrate in more details on the calibration task and exemplify the proposed methodology through analysis of data from a calibrated device and a casual device (needs to be calibrated) in order to demonstrate practical efficiency of our detection method.

Application 6: Randomly spaced time series

Randomly spaced time series can come from a situation when the observation times are driven by the series itself. For instance, cumulative counts of occurrences of a disease in a given area [26] or aggregated claim amounts for a specific line of business in non-life insurance as illustrated in Section 6.2.

Besides that, there are many other applications of the changepoint within the linear relations framework in, for instance, glaciology [27], empirical economics [9], dietary assessment [11], image forensics [28], or healthcare industry [29,30,31].

3. Spectral Weak Invariance Principle

A theoretical device is going to be developed in order to construct the changepoint tests. The smallest eigenvalue of ${\mathsf{\Sigma }}^{-1}{[\mathit{X},\mathit{Y}]}^{\top }[\mathit{X},\mathit{Y}]$—the squared smallest singular value of the matrix $[\mathit{X},\mathit{Y}]{\mathsf{\Sigma }}^{-1/2}$, i.e., the data matrix $[\mathit{X},\mathit{Y}]$ multiplied by the inverse of a matrix square root from the error variance structure (cf. subsequent Assumption $\mathcal{E}$)—plays a key role. We proceed to the assumptions that are needed for deriving forthcoming asymptotic results. Henceforth, $\stackrel{\mathrm{P}}{\to }$ denotes convergence in probability, $\stackrel{\mathrm{D}}{\to }$ convergence in distribution, $\underset{n\to \infty }{\overset{\mathrm{D}[0,1]}{\to }}$ weak convergence in the Skorokhod topology $\mathrm{D}[0,1]$ of càdlàg functions on $[0,1]$, and $\left[x\right]$ denotes the integer part of the real number x.

3.1. Assumptions

Firstly, a design assumption on the unobservable regressors is needed.

Assumption $\mathcal{D}$.

${\displaystyle {\Delta }_t:=\underset{n\to \infty }{\lim }{n}^{-1}{\mathit{Z}}_{\left[nt\right]}^{\top }{\mathit{Z}}_{\left[nt\right]}}$, ${\displaystyle {\Delta }_{-t}:=\underset{n\to \infty }{\lim }{n}^{-1}{\mathit{Z}}_{-\left[nt\right]}^{\top }{\mathit{Z}}_{-\left[nt\right]}}$ for every $t\in (0,1)$, and ${\displaystyle \Delta :=\underset{n\to \infty }{\lim }{n}^{-1}{\mathit{Z}}^{\top }\mathit{Z}}$ are positive definite.

It basically says that the error-free design points do not concentrate to close to each other (i.e., strict positive definiteness) and, simultaneously, they do not spread-out too far (i.e., existence of limits). For example in one-dimensional case (i.e., $p=1$), a simple equidistant design, where $Z_{i,1}=i/(n+1)$, provides ${\Delta }_t=t^3/3$ and $\Delta =1/3$.

Prior to postulating an errors’ assumption, we summarize the notion of strong mixing ($\alpha$-mixing) dependence in more detail, which will be imposed on the model’s errors. Suppose that ${\left\{{\xi }_n\right\}}_{n=1}^{\infty }$ is a sequence of random elements on a probability space $(\mathsf{\Omega },\mathcal{F},\mathrm{P})$. For sub-$\sigma$-fields $\mathcal{A},\mathcal{B}\subseteq \mathcal{F}$, let $\alpha \left(\mathcal{A}\right|\mathcal{B}):={\sup }_{A\in \mathcal{A},B\in \mathcal{B}}\left|\mathrm{P}(A\cap B)-\mathrm{P}\left(A\right)\mathrm{P}\left(B\right)\right|$. Intuitively, $\alpha (·|·)$ measures the dependence of the events in $\mathcal{B}$ on those in $\mathcal{A}$. There are many ways in which one can describe weak dependence or, in other words, asymptotic independence of random variables, see [32]. Considering a filtration ${\mathcal{F}}_m^n=\sigma \{{\xi }_i\in \mathcal{F},m\le i\le n\}$, sequence ${\left\{{\xi }_n\right\}}_{n=1}^{\infty }$ of random variables is said to be strong mixing ($\alpha$-mixing) if $\alpha ({\xi }_{\circ },n)={\sup }_{k\in \mathbb{N}}\alpha \left({\mathcal{F}}_1^k\right|{\mathcal{F}}_{k+n}^{\infty })\to 0$ as $n\to \infty$. A class of m-dependent processes was comprehensively analyzed in [33]. They are $\alpha$-mixing, since they are finite order ARMA processes with innovations satisfying Doeblin’s condition ([34], p. 168). Finite order processes, which do not satisfy Doeblin’s condition, can be shown to be $\alpha$-mixing ([35], pp. 312–313). General conditions under which stationary Markov processes are $\alpha$-mixing are provided in [36]. Since functions of mixing processes are themselves mixing [32], time-varying functions of any of the processes just mentioned are mixing as well. This means that the class of the $\alpha$-mixing processes is sufficiently large for the further practical applications and that is why we chose such a mixing condition.

Assumption $\mathcal{E}$.

${\left\{[{\mathbf{\Theta }}_{n,•},{\epsilon }_n]\right\}}_{n=1}^{\infty }$ is a sequence of α-mixing absolutely continuous random vectors with zero mean and a variance matrix ${\sigma }^2\mathsf{\Sigma }$ with an unknown ${\sigma }^2>0$ and a known positive definite $\mathsf{\Sigma }=\left[\begin{array}{cc}{\mathsf{\Sigma }}_{\mathbf{\Theta }}& {\mathsf{\Sigma }}_{\mathbf{\Theta },\mathit{\epsilon }}\\ {\mathsf{\Sigma }}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }& 1\end{array}\right]$ such that $\alpha ([{\mathbf{\Theta }}_{\circ ,•},{\epsilon }_{\circ }],n)=\mathcal{O}\left(n^{-1-\varpi }\right)$ as $n\to \infty$ for some $\varpi >0$, ${\displaystyle \underset{n\in \mathbb{N}}{\sup }{Z}_{n,j}^2<\infty }$, ${\displaystyle \underset{n\in \mathbb{N}}{\sup }{\mathrm{E}\left|{\mathbf{\Theta }}_{n,j}\right|}^{4+\omega }<\infty }$, $j\in \{1,\dots ,p\}$, and ${\displaystyle \underset{n\in \mathbb{N}}{\sup }{\mathrm{E}\left|{\epsilon }_n\right|}^{4+\omega }<\infty }$ for some $\omega >0$ such that $\omega \varpi >2$.

Let us emphasize that the sequence of the errors do not have to be stationary. The assumption of an unknown ${\sigma }^2$ and a known $\mathsf{\Sigma }$ implies that we know the ratio of any pair of covariances in advance. In the simplest situation, a homoscedastic covariance structure of the within-individual errors $[{\mathbf{\Theta }}_{n,•},{\epsilon }_n]$ can be assumed (i.e., $\mathsf{\Sigma }={\mathit{I}}_{p+1}$), if prior experience or essence of the analyzed problem allow for that. On the other hand, if the covariance matrix $\mathsf{\Sigma }$ is unknown, it can be estimated when possessing replicate measurements or validation data as commented in [16]. There are various approaches proposed to serve this purpose. In order to mention at least some of them, we refer to [22,37,38], or [39]. On the top of that, we have to bear in mind that $\mathsf{\Sigma }$ cannot be completely unspecified. If $\mathsf{\Sigma }$ is unrestricted, no strongly consistent estimator for $\mathit{\beta }$ can exist even under normally distributed errors [40].

Relaxations of Assumption $\mathcal{E}$ with respect to the known and constant $\mathsf{\Sigma }$ are provided and discussed in Section 4.4—unknown covariance matrix—and in Section 4.5—changing covariance structure.

Furthermore, a variance assumption for the misfit disturbances is stated. It can be considered to be a typical assumption for the long-run variance of residuals. Let us denote ${\mathsf{\Sigma }}^{-1/2}=\left[\begin{array}{cc}{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}& {\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}\\ {\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }& {\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}\end{array}\right]$ a symmetric square root of ${\mathsf{\Sigma }}^{-1}$, where ${\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}\in \mathbb{R}$ is a scalar.

Assumption $\mathcal{V}$.

There exist $\varphi :={\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}-{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }{({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top })}^{-1}({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }})\ne 0$ and $\upsilon :={\lim }_{n\to \infty }{n}^{-1}ꓦ\mathrm{ar}{\parallel \mathit{Y}-\mathit{X}\mathit{\beta }\parallel }_2^2>0$.

Let us remark that ${\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}=\mathbf{0}$ for the uncorrelated error structure and, then, $\varphi ={\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}$.

3.2. Swip

Finally, the spectral weak invariance principle (SWIP) [41] for the smallest eigenvalues is provided. Let us denote ${\lambda }_i:={\lambda }_{min}\left({\mathsf{\Sigma }}^{-1}{[{\mathit{X}}_i,{\mathit{Y}}_i]}^{\top }[{\mathit{X}}_i,{\mathit{Y}}_i]\right)$ for $2\le i\le n$, ${\lambda }_0:={\lambda }_1:=0$ and ${\tilde{\lambda }}_i:={\lambda }_{min}\left({\mathsf{\Sigma }}^{-1}{[{\mathit{X}}_{-i},{\mathit{Y}}_{-i}]}^{\top }[{\mathit{X}}_{-i},{\mathit{Y}}_{-i}]\right)$ for $0\le i\le n-2$, ${\tilde{\lambda }}_n:={\tilde{\lambda }}_{n-1}:=0$. Please note that ${\lambda }_n\equiv {\tilde{\lambda }}_0$.

Proposition 1

(SWIP). Let $\mathcal{M}$ and ${\mathcal{H}}_0$ hold. Under Assumptions $\mathcal{D}$, $\mathcal{E}$, and $\mathcal{V}$,

$${\left\{\frac{1}{\sqrt{n}}\left({\lambda }_{\left[nt\right]}-\left[nt\right]{\sigma }^2\right)\right\}}_{t\in [0,1]}\underset{n\to \infty }{\overset{\mathrm{D}[0,1]}{{\displaystyle \to }}}{\left\{\frac{{\varphi }^2\upsilon }{1+{\parallel \mathit{\alpha }\parallel }_2^2}\mathcal{W}\left(t\right)\right\}}_{t\in [0,1]}$$

and

$${\left\{\frac{1}{\sqrt{n}}\left({\tilde{\lambda }}_{\left[n\right(1-t\left)\right]}-\left[n(1-t)\right]{\sigma }^2\right)\right\}}_{t\in [0,1]}\underset{n\to \infty }{\overset{\mathrm{D}[0,1]}{{\displaystyle \to }}}{\left\{\frac{{\varphi }^2\upsilon }{1+{\parallel \mathit{\alpha }\parallel }_2^2}\tilde{\mathcal{W}}\left(t\right)\right\}}_{t\in [0,1]},$$

where ${\left\{\mathcal{W}\left(t\right)\right\}}_{t\in [0,1]}$ is a standard Wiener process, $\tilde{\mathcal{W}}\left(t\right)=\mathcal{W}\left(1\right)-\mathcal{W}\left(t\right)$, and $\mathit{\alpha }={({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top })}^{-1}({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }})$.

3.3. Extension to Multivariate Case

Suppose that $\mathit{\beta }\in {\mathbb{R}}^{p\times q}$, $\mathit{Y}\in {\mathbb{R}}^{n\times q}$, and $\mathit{\epsilon }\in {\mathbb{R}}^{n\times q}$, $q\ge 1$. Let the singular value decomposition (SVD) of the partial transformed data be

$$[{\mathit{X}}_{\left[nt\right]},{\mathit{Y}}_{\left[nt\right]}]{\mathsf{\Sigma }}^{-1/2}=\mathit{U}\left(t\right)\mathsf{\Gamma }\left(t\right){\mathit{V}}^{\top }\left(t\right)={\displaystyle \sum }_{i=1}^{p+q}\varsigma {\left(t\right)}^{\left(i\right)}\mathit{u}{\left(t\right)}^{\left(i\right)}\mathit{v}{\left(t\right)}^{\left(i\right)\top },$$

where $\mathit{u}{\left(t\right)}^{\left(i\right)}$’s are the left-singular vectors, $\mathit{v}{\left(t\right)}^{\left(i\right)}$’s are the right-singular vectors, and $\varsigma {\left(t\right)}^{\left(i\right)}$’s are the singular values in the non-increasing order. One may replace ${\lambda }_{\left[nt\right]}$ by

$${\mathsf{\Lambda }}_{\left[nt\right]}:=\sum _{j=1}^q{\left(\varsigma {\left(t\right)}^{(p+j)}\right)}^2$$

in Proposition 1 (and analogously for ${\tilde{\lambda }}_{\left[n\right(1-t\left)\right]}$). Then, the SWIP can be derived again (see the proof of Proposition 1), provided adequately extended assumptions on the errors ${\left\{{\mathit{\epsilon }}_{n,1}\right\}}_{n=1}^{\infty },\dots ,{\left\{{\mathit{\epsilon }}_{n,q}\right\}}_{n=1}^{\infty }$ instead of the original ones ${\left\{{\epsilon }_n\right\}}_{n=1}^{\infty }$. However, the consequent proofs would become more technical.

4. Nuisance-Parameter-Free Detection

Consistent estimation of $\mathit{\beta }$ can be performed via the generalized TLS approach [19,42]. The optimizing problem

$$[\mathit{b},\widehat{\mathbf{\Theta }},\widehat{\mathit{\epsilon }}]:=\underset{\left[\mathbf{\Theta },\mathit{\epsilon }\right]\in {\mathbb{R}}^{n\times (p+1)},\mathit{\beta }\in {\mathbb{R}}^p}{\arg \min }{∥\left[\mathbf{\Theta },\mathit{\epsilon }\right]{\mathsf{\Sigma }}^{-1/2}∥}_F \mathrm{s}.\mathrm{t}. \mathit{Y}-\mathit{\epsilon }=(\mathit{X}-\mathbf{\Theta })\mathit{\beta },$$

where ${\parallel ·\parallel }_F$ stands for the Frobenius matrix norm, has a solution consisting of the estimator

$$\mathit{b}={({\mathit{X}}^{\top }\mathit{X}-{\lambda }_n{\mathsf{\Sigma }}_{\mathbf{\Theta }})}^{-1}({\mathit{X}}^{\top }\mathit{Y}-{\lambda }_n{\mathsf{\Sigma }}_{\mathbf{\Theta },\mathit{\epsilon }})$$

(1)

and the fitted errors $[\widehat{\mathbf{\Theta }},\widehat{\mathit{\epsilon }}]$ such that

$$\parallel [\widehat{\mathbf{\Theta }},\widehat{\mathit{\epsilon }}]{\mathsf{\Sigma }}^{-1/2}{\parallel }_F^2={\lambda }_n.$$

(2)

We construct the changepoint test statistics based on property (2).

4.1. Changepoint Test Statistics

Let us think of two TLS estimates of $\mathit{\beta }$: The first one based on the first i data lines $[{\mathit{X}}_i,{\mathit{Y}}_i]$ and the second one based on the first k data lines $[{\mathit{X}}_k,{\mathit{Y}}_k]$ such that $1\le i\le k\le n$. Under the null ${\mathcal{H}}_0$, these two TLS estimates should be close to each other. On the other hand, under the alternative ${\mathcal{H}}_A$ such that $\tau \in \{i,\dots ,k\}$, they should be somehow different. A similar conclusion can be made for the goodness-of-fit statistics coming from (2). It means that

$${\lambda }_i-\frac{i}{k}{\lambda }_k$$

should be reasonably small under the null ${\mathcal{H}}_0$. Under the alternative ${\mathcal{H}}_A$ such that $\tau \in \{i,\dots ,k\}$, it should be relatively large. For the multivariate case described in previous Section 3.3, one has to replace ${\lambda }_k$ by ${\mathsf{\Lambda }}_k={\sum }_{j=1}^q{\left(\varsigma {(k/n)}^{(p+j)}\right)}^2$.

We rely on self-normalized test statistics introduced in [43], because the unknown quantity ${\varphi }^2{\upsilon /(1+\parallel \mathit{\alpha }\parallel }_2^2)$ from Proposition 1 cancels out in the test statistics. Our supremum-type self-normalized test statistic based on the goodness-of-fit is defined as

$${\mathcal{S}}_n:=\underset{1\le k<n}{\max }\frac{|{\lambda }_k-\frac{k}{n}{\lambda }_n|}{{\max }_{1\le i<k}|{\lambda }_i-\frac{i}{k}{\lambda }_k|+{\max }_{k<i\le n}|{\tilde{\lambda }}_i-\frac{n-i}{n-k}{\tilde{\lambda }}_k|}$$

(3)

and the integral-type self-normalized test statistic is defined as

$${\mathcal{T}}_n:=\sum _{k=1}^{n-1}\frac{{\left({\lambda }_k-\frac{k}{n}{\lambda }_n\right)}^2}{{\sum }_{i=1}^{k-1}{\left({\lambda }_i-\frac{i}{k}{\lambda }_k\right)}^2+{\sum }_{i=k+1}^n{\left({\tilde{\lambda }}_i-\frac{n-i}{n-k}{\tilde{\lambda }}_k\right)}^2}.$$

(4)

Let us note that evaluations of the above defined test statistics require just several singular value decompositions, which is reasonably quick. Our new test statistics involve neither nuisance parameters nor tuning constants (i.e., nuisance-parameter-free) and will work for non-stationary and weakly dependent data. On the top of that, no boundary issue is present meaning that the tests can detect the change close to the beginning or to the end of the studied regime. Furthermore, the power at the boundaries can be improved by introducing weight functions, see, e.g., [44].

Under the null hypothesis and the technical assumptions from Section 3.1, the test statistics defined in (3) and (4) converge to non-degenerate limit distributions (their quantiles are in Section 4.2).

Theorem 1

(Under the null). Let $\mathcal{M}$ and ${\mathcal{H}}_0$ hold. Under Assumptions $\mathcal{D}$, $\mathcal{E}$, and $\mathcal{V}$,

$${\mathcal{S}}_n\underset{n\to \infty }{\overset{\mathrm{D}}{{\displaystyle \to }}}\underset{t\in [0,1]}{\sup }\frac{\left|\mathcal{W}\right(t)-t\mathcal{W}(1\left)\right|}{{\sup }_{s\in [0,t]}|\mathcal{W}\left(s\right)-\frac{s}{t}\mathcal{W}\left(t\right)|+{\sup }_{s\in [t,1]}|\tilde{\mathcal{W}}\left(s\right)-\frac{1-s}{1-t}\tilde{\mathcal{W}}\left(t\right)|}$$

(5)

and

$${\mathcal{T}}_n\underset{n\to \infty }{\overset{\mathrm{D}}{{\displaystyle \to }}}{\int }_0^1\frac{{\left\{\mathcal{W}\left(t\right)-t\mathcal{W}\left(1\right)\right\}}^2}{{\int }_0^t{\left\{\mathcal{W}\left(s\right)-\frac{s}{t}\mathcal{W}\left(t\right)\right\}}^{2}ds+{\int }_t^1{\left\{\tilde{\mathcal{W}}\left(s\right)-\frac{1-s}{1-t}\tilde{\mathcal{W}}\left(t\right)\right\}}^{2}ds}dt,$$

(6)

where ${\left\{\mathcal{W}\left(t\right)\right\}}_{t\in [0,1]}$ is a standard Wiener process and $\tilde{\mathcal{W}}\left(t\right)=\mathcal{W}\left(1\right)-\mathcal{W}\left(t\right)$.

The null hypothesis is rejected at significance level $\alpha$ for large values of ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$. The critical values can be obtained as the $(1-\alpha )$-quantiles of the asymptotic distributions from (5) and (6). To describe limit behavior of the test statistics under the alternative, an additional changepoint assumption is required.

Assumption $\mathcal{C}$.

For some $\zeta \in (0,1)$, as $n\to \infty$,

$${\parallel \mathit{\delta }\parallel }_2\to 0 and (\eta \kappa -{\mathit{\phi }}^{\top }\mathit{\phi })\sqrt{n}\to \infty ,$$

(7)

where

$$\begin{array}{cc}\hfill \kappa & :=({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }+{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}{\mathit{\beta }}^{\top }){\Delta }_{\zeta }({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }})+({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }+{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}{\tilde{\mathit{\beta }}}^{\top }){\Delta }_{-\zeta }({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}+\tilde{\mathit{\beta }}{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}),\hfill \\ \hfill \mathit{\phi }& :=({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}{\mathit{\beta }}^{\top }){\Delta }_{\zeta }({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }})+({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}{\tilde{\mathit{\beta }}}^{\top }){\Delta }_{-\zeta }({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}+\tilde{\mathit{\beta }}{\overline{\mathsf{\Sigma }}}_{\mathit{\epsilon }}),\hfill \\ \hfill \eta & :={\lambda }_{min}(({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}{\mathit{\beta }}^{\top })\Delta ({\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top })+{\sigma }^2{\mathit{I}}_p)-{\sigma }^2,and\tilde{\mathit{\beta }}:=\mathit{\beta }+\mathit{\delta }.\hfill \end{array}$$

This assumption may be considered to be a changepoint detectability requirement for local alternatives, because it manages the relationship between the size of the change, the location of the change, and the noisiness of the data in order to be able to detect the changepoint. In case of uncorrelated error structure, the previous formulae become simpler due to ${\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}=\mathbf{0}$. Assumption $\mathcal{C}$ is automatically fulfilled, for instance, for an arbitrary $\delta \to 0$ and the one-dimensional equidistant design points $Z_i$’s on $(0,1)$ with homoscedastic error structure, because then $\eta \kappa -{\mathit{\phi }}^{\top }\mathit{\phi }={\beta }^2\{{\zeta }^3+{(1-\zeta )}^3\}\{1-{\zeta }^3-{(1-\zeta )}^3\}/9+O\left(\delta \right)$ as $\delta \to 0$. Furthermore, let us remark that $\mathit{\vartheta }:={\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta }}+\mathit{\beta }{\overline{\mathsf{\Sigma }}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }$ has full rank under Assumption $\mathcal{V}$.

A practical purpose should not lie in the detection of changes that are eminent or evident, but rather concealed or hidden. The tests based on ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$ are shown to be consistent, as the test statistics converge to infinity under some local alternatives, provided that the size of the change does not convergence to zero too fast, cf. Assumption $\mathcal{C}$ where $\kappa$ and $\mathit{\phi }$ depend on $\mathit{\delta }$.

Theorem 2

(Under local alternatives). Let $\mathcal{M}$ and ${\mathcal{H}}_A$ hold such that $\tau =\left[n\zeta \right]$ for some $\zeta \in (0,1)$. Under Assumptions $\mathcal{C}$, $\mathcal{D}$, $\mathcal{E}$, and $\mathcal{V}$,

$${\mathcal{S}}_n\underset{n\to \infty }{\overset{\mathrm{P}}{\to }}\infty \underset{n\to \infty }{\overset{\mathrm{P}}{\leftarrow }}{\mathcal{T}}_n.$$

(8)

Assumption $\mathcal{C}$ can be sharpened as remarked below with the corresponding proof in the Appendix A.

Remark 1.

The second part of relation (7) can be replaced by

$$\sqrt{n}\left\{\kappa +\eta -\sqrt{{(\kappa +2{\sigma }^2+\eta )}^2-4(\kappa +{\sigma }^2-{\mathit{\phi }}^{\top }{({\mathit{\vartheta }}^{\top }\Delta \mathit{\vartheta }+{\sigma }^2{\mathit{I}}_p)}^{-1}\mathit{\phi })({\sigma }^2+\eta )}\right\}\to \infty$$

(9)

and the assertion of Theorem 2 still holds.

Basically, Theorem 2 discloses that in presence of the structural change in linear relations, the test statistics explode above all bounds. Hence, the asymptotic distributions from Theorem 1 can be used to construct the tests, although explicit forms of those distributions stated in (5) and (6) are unknown.

4.2. Asymptotic Critical Values

The critical values may be determined by simulations from the limit distributions ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$ from Theorem 1. Theorem 2 ensures that we reject the null hypothesis for large values of the test statistics. We have simulated the asymptotic distributions (5) and (6) by discretizing the standard Wiener process and using the relationship of a random walk to the standard Wiener process. We considered 1000 as the number of discretization points within $[0,1]$ interval and the number of simulation runs equals to $100,000$. In

Table 1. Simulated asymptotic critical values for ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$.

$100(1-\mathit{\alpha })\%$	$90\%$	$95\%$	$97.5\%$	$99\%$	$99.5\%$
$\mathcal{S}$-based	$1.209008$	$1.393566$	$1.571462$	$1.782524$	$1.966223$
$\mathcal{T}$-based	$5.700222$	$7.165705$	$8.807070$	$10.597625$	$11.755233$

, we present several critical values for the test statistics ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$.

4.3. Changepoint Estimator

If a change is detected, it is of interest to estimate the changepoint. It is sensible to use

$${\widehat{\tau }}_n:=\underset{1\le k\le n-1}{\mathrm{argmax}}\frac{|{\lambda }_k-\frac{k}{n}{\lambda }_n|+|{\tilde{\lambda }}_k-\frac{n-k}{n}{\tilde{\lambda }}_0|}{{\max }_{1\le i<k}|{\lambda }_i-\frac{i}{k}{\lambda }_k|+{\max }_{k<i\le n}|{\tilde{\lambda }}_i-\frac{n-i}{n-k}{\tilde{\lambda }}_k|}$$

as a changepoint estimator. Our next theorem shows that under the alternative, the changepoint $\tau$ is consistently estimated by the estimator ${\widehat{\tau }}_n$ with the corresponding optimal convergence rate.

Corollary 1

(Consistency). Let the assumptions of Theorem 2 hold. If

$$\begin{array}{cc}\hfill {\psi }_n& :=\sqrt{n}\underset{t\in (\zeta ,1)}{\inf }\{\eta \left(t\right)\kappa \left(t\right)-\mathit{\phi }{\left(t\right)}^{\top }\mathit{\phi }\left(t\right)\}\stackrel{n\to \infty }{\to }\infty ;\hfill \end{array}$$

(10)

$$\begin{array}{cc}\hfill {\tilde{\psi }}_n& :=\sqrt{n}\underset{t\in (0,\zeta )}{\inf }\{\tilde{\eta }\left(t\right)\tilde{\kappa }\left(t\right)-\tilde{\mathit{\phi }}{\left(t\right)}^{\top }\tilde{\mathit{\phi }}\left(t\right)\}\stackrel{n\to \infty }{\to }\infty ,\hfill \end{array}$$

(11)

where $\kappa \left(t\right)$, $\tilde{\kappa }\left(t\right)$, $\mathit{\phi }\left(t\right)$, $\tilde{\mathit{\phi }}\left(t\right)$, $\eta \left(t\right)$, and $\tilde{\eta }\left(t\right)$ are defined in the proof, then

$${\widehat{\tau }}_n/n-\zeta =O_{\mathrm{P}}\left({({\psi }_n\wedge {\tilde{\psi }}_n)}^{-1}\right), n\to \infty .$$

(12)

Conditions (10) and (11) serve as a uniform intermediary between the size of the change, the location of the change, the sample size, and the heteroscedasticity of the disturbances (cf. the proof of Corollary 1 in the Appendix A) for assuring changepoint estimator’s consistency. These assumptions are again automatically fulfilled for the case discussed below Assumption $\mathcal{C}$ . Other concepts of the changepoint estimator’s construction relying on the ratio type statistics [45] or on the different dependency structures [46] can be adopted as well.

4.4. Unknown Covariance Matrix

If the true covariance matrix of the errors is unknown, possessing a surrogate covariance matrix, which is not far away from the original one, can help to overcome such an issue.

Assumption $\mathcal{E}1$.

The same as Assumption $\mathcal{E}$, except that the matrix Σ is unknown. Moreover, there exists a known positive definite $\underline{\mathsf{\Sigma }}$ such that ${\lim }_{n\to \infty }{\parallel {\underline{\mathsf{\Sigma }}}^{-1/2}\mathsf{\Sigma }{\underline{\mathsf{\Sigma }}}^{-1/2}-{\mathit{I}}_{p+1}\parallel }_2=0$.

Here, ${\parallel ·\parallel }_2$ is the induced matrix norm by the Euclidean vector norm. Please note that ${\parallel \mathit{M}\parallel }_2\le {\parallel \mathit{M}\parallel }_F$ and ${\parallel \mathit{M}\parallel }_2^2={\lambda }_{max}\left({\mathit{M}}^{\top }\mathit{M}\right)$, where ${\lambda }_{max}(·)$ stands for the largest eigenvalue. For example, if $\mathsf{\Sigma }=\left[\begin{array}{cc}1& \frac{1}{\log n}\\ \frac{1}{\log n}& 1\end{array}\right]$ and $\underline{\mathsf{\Sigma }}={\mathit{I}}_2$, then $\parallel {\underline{\mathsf{\Sigma }}}^{-1/2}\mathsf{\Sigma }{\underline{\mathsf{\Sigma }}}^{-1/2}-{\mathit{I}}_2{\parallel }_2=1/\log n\to 0$.

To practically achieve Assumption $\mathcal{E}1$ without any prior knowledge about the correlation structure, it is reasonable to re-scale (e.g., normalize or standardize) the original data, to choose $\underline{\mathsf{\Sigma }}={\mathit{I}}_{p+1}$, and, then, to suppose that we deal with transformed errors with common variance and covariances vanishing sufficiently fast to zero as the sample size increases.

Let us denote ${\underline{\lambda }}_i:={\lambda }_{min}\left({\underline{\mathsf{\Sigma }}}^{-1}{[{\mathit{X}}_i,{\mathit{Y}}_i]}^{\top }[{\mathit{X}}_i,{\mathit{Y}}_i]\right)$ for $2\le i\le n$, ${\underline{\lambda }}_0:={\underline{\lambda }}_1:=0$ and ${\tilde{\underline{\lambda }}}_i:={\lambda }_{min}\left({\underline{\mathsf{\Sigma }}}^{-1}{[{\mathit{X}}_{-i},{\mathit{Y}}_{-i}]}^{\top }[{\mathit{X}}_{-i},{\mathit{Y}}_{-i}]\right)$ for $0\le i\le n-2$, ${\tilde{\underline{\lambda }}}_n:={\tilde{\underline{\lambda }}}_{n-1}:=0$. Please note that ${\underline{\lambda }}_n\equiv {\tilde{\underline{\lambda }}}_0$.

Lemma 1.

If Assumption $\mathcal{E}1$ holds instead of Assumption $\mathcal{E}$, then Proposition 1, Theorem 1, Theorem 2, and Corollary 1 are valid when ${\lambda }_i$’s and ${\tilde{\lambda }}_i$’s are replaced by ${\underline{\lambda }}_i$’s and ${\tilde{\underline{\lambda }}}_i$’s, respectively.

4.5. Simultaneously Changing Relation and Covariance Structure

Practically speaking, it is plausible that the covariance structure could disrupt alongside the abruptly changed parameter of the linear relations. Suppose that the covariance matrix of the errors is ${\sigma }^2\underleftarrow{\mathsf{\Sigma }}$ up to the changepoint $\tau$, such that $\underleftarrow{\mathsf{\Sigma }}\equiv \mathsf{\Sigma }$, and it becomes ${\sigma }^2\underrightarrow{\mathsf{\Sigma }}$ after the changepoint $\tau$, where the bottom-right element of $\underrightarrow{\mathsf{\Sigma }}$ does not have to be equal 1. Now, $\underleftarrow{\mathsf{\Sigma }}$ and $\underrightarrow{\mathsf{\Sigma }}$ are considered to be unknown.

Let us construct a transformed data matrix

$$[\underset{˜}{\mathit{X}},\underset{˜}{\mathit{Y}}]:=\left[\begin{array}{c}[{\mathit{X}}_{\tau },{\mathit{Y}}_{\tau }]{\underleftarrow{\mathsf{\Sigma }}}^{-1/2}\\ {\mathit{X}}_{-\tau },{\mathit{Y}}_{-\tau }]{\underrightarrow{\mathsf{\Sigma }}}^{-1/2}\end{array}\right].$$

In the analogous manner, we define $[\underset{˜}{\mathbf{\Theta }},\underset{˜}{\mathit{\epsilon }}]$. It holds that $\mathrm{Var}[{\underset{˜}{\mathbf{\Theta }}}_{i,•},{\underset{˜}{\epsilon }}_i]={\sigma }^2{\mathit{I}}_{p+1}$ for $i=1,\dots ,n$. Consequently, put $\underset{˜}{\mathit{Z}}:=\underset{˜}{\mathit{X}}-\underset{˜}{\mathbf{\Theta }}$. The EIV model $\mathcal{M}$–${\mathcal{H}}_0$ implies $\left([\mathit{X},\mathit{Y}]-[\mathbf{\Theta },\mathit{\epsilon }]\right)\left[\begin{array}{c}\mathit{\beta }\\ -1\end{array}\right]=\mathbf{0}$. With respect to the alternative ${\mathcal{H}}_A$, we define $\underset{˜}{\mathit{\beta }}$ such that $\left([{\underset{˜}{\mathit{X}}}_{\tau },{\underset{˜}{\mathit{Y}}}_{\tau }]-[{\underset{˜}{\mathbf{\Theta }}}_{\tau },{\underset{˜}{\mathit{\epsilon }}}_{\tau }]\right)\left[\begin{array}{c}\underset{˜}{\mathit{\beta }}\\ -1\end{array}\right]=\mathbf{0}$. Thus,

$$\underset{˜}{\mathit{\beta }}=\left({\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta }}\mathit{\beta }-{\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}\right)/\left({\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathit{\epsilon }}-{\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }\mathit{\beta }\right),$$

(13)

where ${\underleftarrow{\mathsf{\Sigma }}}^{1/2}:=\left[\begin{array}{cc}{\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta }}& {\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}\\ {\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }& {\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathit{\epsilon }}\end{array}\right]$ is a symmetric square root of $\underleftarrow{\mathsf{\Sigma }}$. Similarly, we define $\underset{˜}{\mathit{\delta }}$ such that $\left([{\underset{˜}{\mathit{X}}}_{-\tau },{\underset{˜}{\mathit{Y}}}_{-\tau }]-[{\underset{˜}{\mathbf{\Theta }}}_{-\tau },{\underset{˜}{\mathit{\epsilon }}}_{-\tau }]\right)\left[\begin{array}{c}\underset{˜}{\mathit{\beta }}+\underset{˜}{\mathit{\delta }}\\ -1\end{array}\right]=\mathbf{0}$. Hence,

$$\underset{˜}{\mathit{\delta }}=\left({\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta }}(\mathit{\beta }+\mathit{\delta })-{\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}\right)/\left({\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathit{\epsilon }}-{\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }(\mathit{\beta }+\mathit{\delta })\right)-\underset{˜}{\mathit{\beta }},$$

(14)

where ${\underrightarrow{\mathsf{\Sigma }}}^{1/2}:=\left[\begin{array}{cc}{\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta }}& {\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}\\ {\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }& {\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathit{\epsilon }}\end{array}\right]$ is a symmetric square root of $\underrightarrow{\mathsf{\Sigma }}$. A technical requirement assuring that $\underset{˜}{\mathit{\beta }}$ and $\underset{˜}{\mathit{\delta }}$ are well-defined is needed, cf. (13) and (14).

Assumption $\mathcal{B}$.

${\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathit{\epsilon }}-{\widetilde{\underleftarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }\mathit{\beta }\ne 0\ne {\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathit{\epsilon }}-{\widetilde{\underrightarrow{\mathsf{\Sigma }}}}_{\mathbf{\Theta },\mathit{\epsilon }}^{\top }(\mathit{\beta }+\mathit{\delta })$.

This assumption basically implies that the changepoint EIV problem can be correctly postulated for the transformed data $[\underset{˜}{\mathit{X}},\underset{˜}{\mathit{Y}}]$.

Furthermore, let us denote ${\underset{˜}{\lambda }}_i:={\lambda }_{min}\left({[{\underset{˜}{\mathit{X}}}_i,{\underset{˜}{\mathit{Y}}}_i]}^{\top }[{\underset{˜}{\mathit{X}}}_i,{\underset{˜}{\mathit{Y}}}_i]\right)$ for $2\le i\le n$, ${\underset{˜}{\lambda }}_0:={\underset{˜}{\lambda }}_1:=0$ and ${\widetilde{\underset{˜}{\lambda }}}_i:={\lambda }_{min}\left({[{\underset{˜}{\mathit{X}}}_{-i},{\underset{˜}{\mathit{Y}}}_{-i}]}^{\top }[{\underset{˜}{\mathit{X}}}_{-i},{\underset{˜}{\mathit{Y}}}_{-i}]\right)$ for $0\le i\le n-2$, ${\widetilde{\underset{˜}{\lambda }}}_n:={\widetilde{\underset{˜}{\lambda }}}_{n-1}:=0$. Please note that ${\underset{˜}{\lambda }}_n\equiv {\widetilde{\underset{˜}{\lambda }}}_0$ and all these $\underset{˜}{\lambda }$’s are unknown. Clearly, if one appropriately changes Assumptions $\mathcal{C}$, $\mathcal{D}$, $\mathcal{E}$, and $\mathcal{V}$ for the ‘under-tilde’ versions of the involved entities, the corresponding Proposition 1, Theorem 1, Theorem 2, and Corollary 1 come into force. On one hand, Assumptions $\mathcal{C}$, $\mathcal{E}$, and $\mathcal{V}$ become even simpler due to the homoscedastic covariance matrix of $[{\underset{˜}{\mathbf{\Theta }}}_{i,•},{\underset{˜}{\epsilon }}_i]$. On the other hand, the transformed data $[\underset{˜}{\mathit{X}},\underset{˜}{\mathit{Y}}]$ are simply not observable.

Since a local alternative for the change in the parameter $\mathit{\beta }$ is considered, a local alternative for the change of the unknown covariance matrix is assumed as well. Besides that, a known covariance matrix close in some sense to the true changing covariance matrix is required.

Assumption $\mathcal{E}2$.

The same as Assumption $\mathcal{E}$ , except the fact that $\mathit{Z}$ is replaced by $\underset{\tilde{}}{\mathit{Z}}$, $\mathrm{Var}[{\mathbf{\Theta }}_{i,•},{\epsilon }_i]={\sigma }^2\underleftarrow{\mathsf{\Sigma }}\equiv {\sigma }^2\mathsf{\Sigma }$ under the null hypothesis ${\mathcal{H}}_0$, and $\mathrm{Var}[{\mathbf{\Theta }}_{i,•},{\epsilon }_i]={\sigma }^2(\underleftarrow{\mathsf{\Sigma }}𝟙\{i\le \tau \}+\underrightarrow{\mathsf{\Sigma }}𝟙\{i>\tau \})$ under the alternative ${\mathcal{H}}_A$ for $i=1,\dots ,n$. The matrices $\underleftarrow{\mathsf{\Sigma }}$, $\underrightarrow{\mathsf{\Sigma }}$ are unknown positive definite and ${\sigma }^2>0$ is also unknown. Moreover, there exists a known positive definite $\underline{\mathsf{\Sigma }}$ such that ${\lim }_{n\to \infty }\parallel {\underline{\mathsf{\Sigma }}}^{-1/2}\underleftarrow{\mathsf{\Sigma }}{\underline{\mathsf{\Sigma }}}^{-1/2}-{\mathit{I}}_{p+1}{\parallel }_2=0={\lim }_{n\to \infty }{\parallel {\underline{\mathsf{\Sigma }}}^{-1/2}\underrightarrow{\mathsf{\Sigma }}{\underline{\mathsf{\Sigma }}}^{-1/2}-{\mathit{I}}_{p+1}\parallel }_2$.

One wishes to work with the observable data $[\mathit{X},\mathit{Y}]{\underline{\mathsf{\Sigma }}}^{-1/2}$, to calculate the smallest eigenvalues from ${\underline{\mathsf{\Sigma }}}^{-1}{[{\mathit{X}}_i,{\mathit{Y}}_i]}^{\top }[{\mathit{X}}_i,{\mathit{Y}}_i]$, and consequently to plug-in them into the original test statistics and changepoint estimator preserving the derived properties.

Theorem 3

(Under local alternatives of relation and covariance). Let $\mathcal{M}$ and ${\mathcal{H}}_A$ hold such that $\tau =\left[n\zeta \right]$ for some $\zeta \in (0,1)$. Suppose that Assumption $\mathcal{E}2$ is satisfied instead of Assumption $\mathcal{E}$. Moreover, suppose that Assumptions $\mathcal{C}$, $\mathcal{D}$, and $\mathcal{V}$ are fulfilled for $\underset{˜}{\mathit{Z}}$ and the covariance matrix ${\sigma }^2{\mathit{I}}_{p+1}$ instead of $\mathit{Z}$ and ${\sigma }^2\mathsf{\Sigma }$, respectively. If Assumption $\mathcal{B}$ holds and $\underset{˜}{\mathit{\delta }}\ne \mathbf{0}$, then Theorem 2 and Corollary 1 are valid when ${\lambda }_i$’s and ${\tilde{\lambda }}_i$’s are replaced by ${\underline{\lambda }}_i$’s and ${\tilde{\underline{\lambda }}}_i$’s, respectively.

5. Simulation Study

We are interested in the performance of the tests based on the self-normalized test statistics ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$ that are completely nuisance-parameter-free. We focused on the comparison of the accuracy of critical values obtained by the simulation from the limit distributions.

In Figure 1, Figure 2, Figure 3 and Figure 4, one may see size-power plots considering the test statistics ${\mathcal{S}}_n$ and ${\mathcal{T}}_n$ under the null hypothesis and under the alternative. Figure 1 and Figure 2 correspond to one input covariate (i.e., $p=1$) with choices of $\beta =1$ and $Z_{i,1}=100i/(n+1)$. A case with four error-prone regressors (i.e., $p=4$) is illustrated in Figure 3 and Figure 4 for choices of $\mathit{\beta }={[1,1,1,1]}^{\top }$ and $Z_{i,•}=100\times [i/(n+1),{(i/(n+1))}^{3/2},{(i/(n+1))}^{4/3},{(i/(n+1))}^{5/4}]$. Next, $n\in \{200,1000\}$ and $\tau \in \{n/4,n/2\}$. The size of change is $\delta \in \{0.1,0.5\}$ for $p=1$ and $\mathit{\delta }\in \{{[0.1,0.1,0.1,0.1]}^{\top },{[0.5,0.5,0.5,0.5]}^{\top }\}$ for $p=4$. Especially smaller values of the break should represent the situations under the local alternatives. In Figure 1 and Figure 3, the empirical rejection frequency under the null hypothesis (actual $\alpha$-errors) is plotted against the theoretical size (theoretical $\alpha$-errors with $\alpha \in \{1\%,5\%,10\%\}$), illustrating the size of the tests. The ideal situation under the null hypothesis is depicted by the straight diagonal dotted line. The empirical rejection frequencies ($1-$errors of the second type) under the alternative (with different changepoints and values of the change) are shown in Figure 2 and Figure 4, illustrating the power of the tests. Under the alternative, the desired situation would be a steep function with values close to 1. For more details on the size-power plots we may refer, e.g., to [47]. The standard deviation of the random disturbances was set to $\sigma \in \{0.5,1.0\}$ and the random error terms ${\left\{{\mathsf{\Theta }}_{n,1}\right\}}_{n=1}^{\infty },\dots ,{\left\{{\mathsf{\Theta }}_{n,p}\right\}}_{n=1}^{\infty }$, and ${\left\{{\epsilon }_n\right\}}_{n=1}^{\infty }$ were independently simulated as three time series:

• IID … independent and identically distributed random variables;
• AR(1) … autoregressive (AR) process of order one with a coefficient of autoregression equal $0.5$;
• ARCH(1) … autoregressive conditional heteroscedasticity (ARCH) process with the second coefficient equal $0.5$.

The standard normal distribution and the Student t-distribution with 3 degrees of freedom are used for generating the innovations of the models’ errors. All of the time series are standardized such that they have variance equal ${\sigma }^2$. Let us remark that the setup of Student $t_3$-distribution does not satisfy Assumption $\mathcal{E}$. However, it can be considered to be a misspecified model and one would like to inspect performance of our procedures on such a model that violates our assumptions. In the simulations of the rejection rates, we used 10,000 repetitions.

In all of the subfigures of Figure 1 and Figure 3 depicting a situation under the null hypothesis, we may see that comparing the accuracy of $\alpha$-levels (sizes) for different self-normalized test statistics, the integral-type ($\mathcal{T}$-based) method seems to keep the theoretical significance level more firmly than the supremum-type ($\mathcal{S}$-based) method. Comparing the case of $\mathrm{N}(0,1)$ innovations with the case of $t_3$ innovations, the rejection rates under the null tend to be slightly higher for the $t_3$-distribution. Despite the fact that the $t_3$-distributed errors violate Assumption $\mathcal{E}$, the performance of our tests is still surprisingly satisfactory in such case. As expected, the accuracy of the critical values tends to be better for larger n and smaller p. The more complicated dependence structure of errors is assumed, the worse performance of the tests is obtained. Furthermore, the less volatile errors are set, the better tests’ sizes are attained.

The $\mathcal{T}$-method performs better under the null. However under the alternative, the $\mathcal{S}$-method has a tendency to have slightly higher power than the $\mathcal{T}$-method (see Figure 2 and Figure 4). We may also conclude that under ${\mathcal{H}}_A$ with less volatile errors, the power of the test increases. The power decreases when the changepoint is closer to the beginning or the end of the input-output data. The heavier tails ($t_3$ against $\mathrm{N}(0,1)$) give worse results in general for both test statistics. Moreover, ‘more dependent’ scenarios reveal worsening of the test statistics’ performance. Furthermore, the smaller size of the change is considered, the lower power of the test is achieved. In addition, again, the power gets higher for smaller p and larger n.

Afterwards, a simulation experiment is performed to study the finite sample properties of the changepoint estimator for a change in the linear relations’ parameter. We numerically present only the case of $p=1$. In particular, the interest lies in the empirical distributions of the proposed estimator visualized via boxplots, see Figure 5 (bars correspond to the interquartile range). The simulation setup is kept the same as described above.

It can be concluded that the precision of our changepoint estimate is satisfactory even for relatively small sample sizes regardless of the errors’ structure. Less volatile model errors provide more precise changepoint estimate. The less complicated dependence structure is assumed, the higher accuracy of the estimator is obtained. Furthermore, the disturbances with heavier tails yield less precise estimates than innovations with light tails. One may notice that higher precision is obtained when the changepoint is closer to the middle of the data. It is also clear that the precision of ${\widehat{\tau }}_n$ improves markedly as the size of change increases.

6. Applications

6.1. Device Calibration

A company has two industrial devices, where the first one is calibrated according to some institute of standards and the second one is just a casual device. We want to test whether the second device is calibrated according to the first one. In this calibration problem, it means to know whether the second device has approximately the same performance up to some unknown multiplication constant as the first one. Consequently, other devices of the same type are needed to be calibrated as well. For some reasons, e.g., economic or logistic, it is only possible to calibrate one device by the official authorities.

Our data set, provided by a Czech steelmaker, contains 100 couples of speed values of two hammer rams (see Figure 6), where the first forging hammer is calibrated. We set the same power level on both hammers and measure the speed of each hammer ram repeatedly changing only the power level. Our measurements of the speed are encumbered with errors of the same variability in both cases, because we use the same device for measuring the speed and both forging hammers are of the same type. Since the power set for the forging hammer is directly proportional to the speed of the hammer ram, our goal is to test whether the ratio of two hammer rams’ speeds is kept constant over changing the power level or not. Therefore, our changepoint in the EIV model is very suitable for this setup—a linear dependence and errors in both measured speeds (with the same variance).

Both our changepoint tests—${\mathcal{S}}_n=28.42$ and ${\mathcal{T}}_n=127.53$—reject the null hypothesis of a constant linear coefficient between two hammer rams’ speed values at the significance level of $\alpha =5\%$ (cf. Table 1), indicating a changed performance of the second non-calibrated hammer ram, although there is no visible change in trend present—the left part of Figure 6.

As an estimate for our change, we obtain ${\widehat{\tau }}_n=60$ (depicted by a vertical line in Figure 6—right), which corresponds to the 60th measurement of pair of speeds. After this particular measurement, we have background information that a technical issue appeared to the second hammer ram—one of its oil tubes started to leak. Our procedure is indeed capable of detecting and, consequently, estimating the (hardly visible, but present) changepoint in the ratio of the hammer rams’ speeds. In addition, this is done fully automatically without expert knowledge about the oil tube issue and also without setting tuning parameters. Moreover, the estimated ratio via the TLS approach before the change is $1.000891$ (the slope of the green line in Figure 6), which basically says that the hammer rams work approximately in the same way. However, the estimated ratio via the TLS approach after the change is $0.9941487$ (the slope of the red line in Figure 6), which is significantly different from constant 1 (see a formal statistical test from [22]). In contrast, a significant change in trend is not detected using the ratio type changepoint test for the regression parameter [48]. A possible reason could be that such a detection approach misleadingly considers the covariates as error-free (i.e., measured precisely).

Other calibration examples, where our methodology is applicable, can be found in, e.g., [37] or [38].

6.2. Randomly Spaced Time Series in Insurance

An insurance company records developments of the paid claim amounts coming from car accidents over time. Two types of claims with respect to car insurance are considered: material damage and bodily injury. The interest lies in disorder recognition between aggregated claim amounts paid for these two types of claims comprising two lines of insurance business.

Two claims (material damage and bodily injury) can come of one car accident and the corresponding claim amounts are expected to be related. We possess a data set from the Guarantee Fund of the Czech Insurers’ Bureau for car insurance. The aggregated claim amounts from the bodily injuries are monitored every time after the aggregated claim payments from the material damages exceed a threshold of CZK 2.5M (≈€100,000). This is performed by the bureau from the beginning of 2018 up to the end of 2019. The observation times ($n=96$) of the cumulative bodily injury claim amounts are random and irregularly spaced, because these time points are driven by a series of the cumulative material damage claim amounts.

Identification of structural breaks in the claim amounts and their development helps us to tie in specific legal, economic, or natural changes to the time epochs in which they occurred [49,50,51]. Our task is to test whether there is a common linear time trend of the cumulative bodily injury claim amounts or this trend has abruptly changed inside the monitoring regime [52,53,54,55]. Both our changepoint tests—${\mathcal{S}}_n=4.55$ and ${\mathcal{T}}_n=22.75$—reject the null hypothesis of a constant linear trend at the significance level of $\alpha =5\%$ (cf. Table 1), indicating a changed payment development of the bodily injury claims.

As an estimate for the change, we obtain ${\widehat{\tau }}_n=63$ (depicted by a vertical line in Figure 7), which corresponds to 18 April 2019. The bureau for car insurance set some regulatory procedures forcing the insurance companies to change their reporting policy since the second quarter of 2019. This can result in a change of the bodily injury claim payments’ development and our detection procedures automatically reveal such regime switching. Additionally to that, Easter 2019 could also contribute to the change in the claim amounts. For example, Easter customs in the Central Europe and occasional drivers after the winter period cause more car accidents in general.

The data were firstly re-scaled in order to reach a common empirical variance yielding satisfiable Assumption $\mathcal{D}$ and, then, inversely transformed back. The estimated linear time trend via the TLS approach before the change is $189,599.2$ (the slope of the green line in Figure 7) and after the change becomes $211,885.2$ (the slope of the red line in Figure 7). In contrast, a significant change in trend is not detected using the ratio type changepoint test for the regression parameter [48]. Again, a possible explanation could be that such a traditional detection approach misleadingly considers the time points as deterministic and error-free.

7. Conclusions

Our changepoint problem in linear relations is linearly defined, but comes with a highly non-linear solution and inference. We have proposed two tests for changepoints with desirable theoretical properties: The asymptotic size of the tests is guaranteed by a limit theorem even under non-stationarity and weak dependency; the tests and the related changepoint estimator are consistent. We are not aware of any similar results even for independent and identically distributed errors. By combining self-normalization and the proposed spectral weak invariance principle, there are neither tuning constants nor nuisance parameters involved in the whole testing procedure. Unknown or changing covariance structure of the errors is allowed. Therefore, the detection methods are completely data-driven, which makes this framework effortlessly applicable as demonstrated. In our simulations, the tests show reliable performance. Practical implementations of the developed detection techniques are demonstrated on two problems from calibration and insurance.