Some Results on ℓ₁ Polynomial Trend Filtering

Abstract

${\ell }_1$ polynomial trend filtering, which is a filtering method described as an ${\ell }_1$-norm penalized least-squares problem, is promising because it enables the estimation of a piecewise polynomial trend in a univariate economic time series without prespecifying the number and location of knots. This paper shows some theoretical results on the filtering, one of which is that a small modification of the filtering provides not only identical trend estimates as the filtering but also extrapolations of the trend beyond both sample limits.

1. Introduction

The ${\ell }_1$-norm penalized least-squares problem, defined as:

$$\begin{array}{c}\hfill \underset{x_1,\dots ,x_T}{min}\sum _{t=1}^T{(y_t-x_t)}^2+\lambda \sum _{t=3}^T\left|{\Delta }^2{x}_t\right|,\end{array}$$

(1)

where $y_1,\dots ,y_T$ are observed time-series data, was developed by Kim et al. (2009), who called it ${\ell }_1$ trend filtering.1 Here, $\lambda >0$ is a tuning parameter and $\Delta$ denotes the backward difference operator such that $\Delta x_t=x_t-x_{t-1}$. Accordingly, ${\Delta }^2{x}_t=\Delta (\Delta x_t)=x_t-2x_{t-1}+x_{t-2}$. Recall that ${\sum }_{t=3}^T\left|{\Delta }^2{x}_t\right|$ in (1) is ${\ell }_1$-norm of ${[{\Delta }^2{x}_3,\dots ,{\Delta }^2{x}_T]}^{\mathsf{\top }}$. Unlike Hodrick and Prescott (1997) filtering, which is defined as the following squared ${\ell }_2$-norm penalized least-squares problem:

$$\begin{array}{c}\hfill \underset{x_1,\dots ,x_T}{min}\sum _{t=1}^T{(y_t-x_t)}^2+\psi \sum _{t=3}^T{\left({\Delta }^2{x}_t\right)}^2,\end{array}$$

(2)

where $\psi >0$ is a smoothing/tuning parameter, the solution of ${\ell }_1$ trend filtering becomes a continuous piecewise linear trend. The relationship between HP filtering and ${\ell }_1$ trend filtering corresponds to that between ridge regression of Hoerl and Kennard (1970) and Lasso (least absolute shrinkage and selection operator) regression of Tibshirani (1996)/BPDN (basis pursuit denoising) of Chen et al. (1998). Econometric applications of ${\ell }_1$ trend filtering include Yamada and Jin (2013), Yamada and Yoon (2014), Winkelried (2016), and Yamada (2017a).

It has been well-known that HP filtering is a form of the Whittaker–Henderson (WH) method of graduation, which is defined as:

$$\begin{array}{c}\hfill \underset{x_1,\dots ,x_T}{min}\sum _{t=1}^T{(y_t-x_t)}^2+\psi \sum _{t=p+1}^T{\left({\Delta }^p{x}_t\right)}^2.\end{array}$$

(3)

For historical surveys of WH filtering, see Weinert (2007), Phillips (2010), and Nocon and Scott (2012). Likewise, as shown in Kim et al. (2009), Tibshirani and Taylor (2011), and Tibshirani (2014), ${\ell }_1$ trend filtering may be generalized as:

$$\begin{array}{c}\hfill \underset{x_1,\dots ,x_T}{min}\sum _{t=1}^T{(y_t-x_t)}^2+\lambda \sum _{t=p+1}^T\left|{\Delta }^p{x}_t\right|.\end{array}$$

(4)

We refer to it as ${\ell }_1$ polynomial trend filtering.2 This filtering method is promising because it enables us to estimate a piecewise $(p-1)$-th order polynomial trend of a univariate economic time series without prespecifying the number and location of knots. For more details, see Yamada (2017b).

Let ${\widehat{x}}_1,\dots ,{\widehat{x}}_T$ denote the solution of (3) and define ${\widehat{x}}_{T+1},\dots ,{\widehat{x}}_{T+h}$, where h denotes the length of extrapolation by:

$$\begin{array}{c}\hfill {\Delta }^p{\widehat{x}}_{T+j}=0,(j=1,\dots ,h).\end{array}$$

(5)

Recently, Yamada and Du (2018) introduced the following three modifications of the WH method of graduation:3

$$\begin{array}{cc}\hfill \left(\mathrm{a}\right)& \underset{x_1,\dots ,x_{T+h}}{min}\sum _{t=1}^T{(y_t-x_t)}^2+\psi \sum _{t=p+1}^{T+h}{\left({\Delta }^p{x}_t\right)}^2,\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill \left(\mathrm{b}\right)& \underset{x_1,\dots ,x_{T+h}}{min}\sum _{t=1}^{T+h}{(y_t-x_t)}^2+\psi \sum _{t=p+1}^{T+h}{\left({\Delta }^p{x}_t\right)}^2,\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill \left(\mathrm{c}\right)& \underset{x_1,\dots ,x_{T+h}}{min}\sum _{t=1}^{T+h}{(y_t-x_t)}^2+\psi \sum _{t=p+1}^T{\left({\Delta }^p{x}_t\right)}^2,\hfill \end{array}$$

(8)

where $y_{T+j}={\widehat{x}}_{T+j}$ for $j=1,\dots ,h$. Denote the solution of (a), (b), and (c) by ${\widehat{x}}_t^{\left(i\right)}$ for $i=\mathrm{a},\mathrm{b},\mathrm{c}$ and $t=1,\dots ,T+h$. Yamada and Du (2018) showed that, for $i=\mathrm{a},\mathrm{b},\mathrm{c}$ and $t=1,\dots ,T+h$, it follows that:

$$\begin{array}{c}\hfill {\widehat{x}}_t^{\left(i\right)}={\widehat{x}}_t.\end{array}$$

(9)

Among the above results, ${\widehat{x}}_t^{\left(\mathrm{a}\right)}={\widehat{x}}_t$ is of practical use because it provides not only a smoothed series identical to that of the WH graduation, but also an extrapolation beyond the sample limit of current data. Also, ${\widehat{x}}_t^{\left(\mathrm{b}\right)}={\widehat{x}}_t$ is of interest because it shows that ${\widehat{x}}_{T+1},\dots ,{\widehat{x}}_{T+h}$ based on (5) are useless to reduce the end-point problem of the WH graduation.4 In addition, Yamada and Du (2018) proved that, for $i=\mathrm{a},\mathrm{b},\mathrm{c}$ and $t=1,\dots ,T+h$:

$$\begin{array}{c}\hfill \underset{\psi \to \infty }{lim}{\widehat{x}}_t^{\left(i\right)}={\widehat{\beta }}_0{t}^0+\cdots +{\widehat{\beta }}_{p-1}{t}^{p-1},\end{array}$$

(10)

where $({\widehat{\beta }}_0,\dots ,{\widehat{\beta }}_{p-1})={\arg \min }_{{\beta }_0,\dots ,{\beta }_{p-1}}{\sum }_{t=1}^T{(y_t-{\beta }_0{t}^0-\cdots -{\beta }_{p-1}{t}^{p-1})}^2$.

In this paper, we present three modifications of ${\ell }_1$ polynomial trend filtering and show that they provide not only identical trend estimates as ${\ell }_1$ polynomial trend filtering, but also extrapolations of the trend beyond both sample limits. In addition, we show some other results on the modified filtering. We also provide a MATLAB function for calculating the solution of one of the modified filtering methods.

The paper is organized as follows. In Section 2, we present three modifications of ${\ell }_1$ polynomial trend filtering. In Section 3, we state the main results of the paper. In Section 4, we make some remarks on the results provided in Section 3. Section 5 provides some concluding remarks.

Notation. Let $\mathit{y}={[y_1,\dots ,y_T]}^{\mathsf{\top }}$ and ${\mathit{I}}_T$ be the $T\times T$ identity matrix. For an n-dimensional column vector, $\mathbf{\eta }={[{\eta }_1,\dots ,{\eta }_n]}^{\mathsf{\top }}$, ${\parallel \mathbf{\eta }\parallel }_1={\sum }_{i=1}^n\left|{\eta }_i\right|$, ${\parallel \mathbf{\eta }\parallel }_2^2={\sum }_{i=1}^n{\eta }_i^2$, and ${\parallel \mathbf{\eta }\parallel }_{\infty }=max\left(\right|{\eta }_1|,\dots ,|{\eta }_n\left|\right)$. ${\mathit{D}}_n$ is the $(n-p)\times n$p-th order difference matrix such that ${\mathit{D}}_n\mathbf{\eta }={[{\Delta }^p{\eta }_{p+1},\dots ,{\Delta }^p{\eta }_n]}^{\mathsf{\top }}$. We denote ${\mathit{D}}_T$ by $\mathit{D}$. ${\mathsf{\Pi }}_{g+T+h}$ is a $(g+T+h)\times p$ Vandermonde matrix, defined by

$${\mathsf{\Pi }}_{g+T+h}=\left[\begin{array}{cccc}{(1-g)}^0& {(1-g)}^1& \cdots & {(1-g)}^{p-1}\\ ⋮& ⋮& ⋮\\ 1^0& 1^1& \cdots & 1^{p-1}\\ ⋮& ⋮& ⋮\\ T^0& T^1& \cdots & T^{p-1}\\ ⋮& ⋮& ⋮\\ {(T+h)}^0& {(T+h)}^1& \cdots & {(T+h)}^{p-1}\end{array}\right],$$

and we denote ${\mathsf{\Pi }}_{0+T+0}$, which is a $T\times p$ matrix, by $\mathsf{\Pi }$.

2. Three Modifications of ${\ell }_1$ Polynomial Trend Filtering

Let ${\tilde{x}}_1,\dots ,{\tilde{x}}_T$ denote the solution of (4) and define ${\tilde{x}}_{1-g},\dots ,{\tilde{x}}_{1-1}$ and ${\tilde{x}}_{T+1},\dots ,{\tilde{x}}_{T+h}$, where g and h denote the length of extrapolations:

$$\begin{array}{rr}\hfill {\Delta }^p{\tilde{x}}_{p+1-i}& =0,(i=1,\dots ,g),\hfill \end{array}$$

(11)

$$\begin{array}{rr}\hfill {\Delta }^p{\tilde{x}}_{T+j}& =0,(j=1,\dots ,h).\hfill \end{array}$$

(12)

For example, ${\tilde{x}}_{T+1},\dots ,{\tilde{x}}_{T+h}$, defined by (12) for $p=1,2,3$, are explicitly expressed as follows:

$$\begin{array}{l}(p=1){\tilde{x}}_{T+j}={\tilde{x}}_T,(j=1,\dots ,h),\hfill \end{array}$$

(13)

$$\begin{array}{l}(p=2){\tilde{x}}_{T+j}={\tilde{x}}_T+j(\Delta {\tilde{x}}_T),(j=1,\dots ,h),\hfill \end{array}$$

(14)

$$\begin{array}{c}(p=3){\tilde{x}}_{T+j}={\tilde{x}}_T+j(\Delta {\tilde{x}}_T)+\frac{j(j+1)}{2}\left({\Delta }^2{\tilde{x}}_T\right),(j=1,\dots ,h).\hfill \end{array}$$

(15)

For a proof of (15), see the Appendix A.

Consider the following three modifications of ${\ell }_1$ polynomial trend filtering:

$$\begin{array}{cc}\hfill \left(\mathrm{d}\right)& \underset{x_{1-g},\dots ,x_{T+h}}{min}\sum _{t=1}^T{(y_t-x_t)}^2+\lambda \sum _{t=p+1-g}^{T+h}\left|{\Delta }^p{x}_t\right|,\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill \left(\mathrm{e}\right)& \underset{x_{1-g},\dots ,x_{T+h}}{min}\sum _{t=1-g}^{T+h}{(y_t-x_t)}^2+\lambda \sum _{t=p+1-g}^{T+h}\left|{\Delta }^p{x}_t\right|,\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill \left(\mathrm{f}\right)& \underset{x_{1-g},\dots ,x_{T+h}}{min}\sum _{t=1-g}^{T+h}{(y_t-x_t)}^2+\lambda \sum _{t=p+1}^T\left|{\Delta }^p{x}_t\right|,\hfill \end{array}$$

(18)

where $y_{1-i}={\tilde{x}}_{1-i}$ for $i=1,\dots ,g$ and $y_{T+j}={\tilde{x}}_{T+j}$ for $j=1,\dots ,h$. Note that (16) is equivalent to ${\ell }_1$ polynomial trend filtering if $g=h=0$. We denote the solution of (d), (e), and (f) by ${\tilde{x}}_t^{\left(i\right)}$ for $i=\mathrm{d},\mathrm{e},\mathrm{f}$ and $t=1-g,\dots ,T+h$.

Among (16)–(18), the objective function of (16) may be represented in matrix notation as:

$$\begin{array}{c}\hfill \parallel \mathit{y}-\mathit{S}{\mathit{x}}_{g+T+h}{\parallel }_2^2+\lambda {\parallel {\mathit{D}}_{g+T+h}{\mathit{x}}_{g+T+h}\parallel }_1,\end{array}$$

(19)

where $\mathit{S}=[\mathbf{0},{\mathit{I}}_T,\mathbf{0}]$ is a $T\times (g+T+h)$ matrix and ${\mathit{x}}_{g+T+h}$ is a $(g+T+h)$-dimensional column vector. Let ${\tilde{\mathit{x}}}_{g+T+h}^{\left(\mathrm{d}\right)}={[{\tilde{\mathit{x}}}_g^{\left(\mathrm{d}\right)\mathsf{\top }},{\tilde{\mathit{x}}}^{\left(\mathrm{d}\right)\mathsf{\top }},{\tilde{\mathit{x}}}_h^{\left(\mathrm{d}\right)\mathsf{\top }}]}^{\mathsf{\top }}$, where ${\tilde{\mathit{x}}}_g^{\left(\mathrm{d}\right)}={[{\tilde{x}}_{1-g}^{\left(\mathrm{d}\right)},\dots ,{\tilde{x}}_{1-1}^{\left(\mathrm{d}\right)}]}^{\mathsf{\top }}$, ${\tilde{\mathit{x}}}^{\left(\mathrm{d}\right)}={[{\tilde{x}}_1^{\left(\mathrm{d}\right)},\dots ,{\tilde{x}}_T^{\left(\mathrm{d}\right)}]}^{\mathsf{\top }}$, and ${\tilde{\mathit{x}}}_h^{\left(\mathrm{d}\right)}={[{\tilde{x}}_{T+1}^{\left(\mathrm{d}\right)},\dots ,{\tilde{x}}_{T+h}^{\left(\mathrm{d}\right)}]}^{\mathsf{\top }}$. The MATLAB function for calculating ${\tilde{\mathit{x}}}_g^{\left(\mathrm{d}\right)}$, ${\tilde{\mathit{x}}}^{\left(\mathrm{d}\right)}$, and ${\tilde{\mathit{x}}}_h^{\left(\mathrm{d}\right)}$, which depends on CVX developed by Grant and Boyd (2013), is as follows:

function [x_g,x,x_h]=m_l1_pt_filtering(y,lambda,p,g,h)

% y: T-dimensional column vector

% lambda: positive real number

% p, g, h: positive integer

% x_g: g-dimensional column vector

% x: T-dimensional column vector

% x_h: h-dimensional column vector

   T=length(y);

   S=[sparse(T,g),speye(T),sparse(T,h)];

   D=diff(speye(g+T+h),p);

   cvx_begin

      variables z(g+T+h)

      minimize(sum((y-S*z).^2)+lambda*norm(D*z,1))

   cvx_end

   x_g=z(1:g); x=z(g+1:g+T); x_h=z(g+T+1:g+T+h);

end

3. Main Results

Theorem 1.

Denote the solution of (d), (e), and (f) by ${\tilde{x}}_t^{\left(i\right)}$ for $i=\mathrm{d},\mathrm{e},\mathrm{f}$. For $i=\mathrm{d},\mathrm{e},\mathrm{f}$, and $t=1-g$, …, $T+h$, it follows that:

$$\begin{array}{c}\hfill {\tilde{x}}_t^{\left(i\right)}={\tilde{x}}_t,\end{array}$$

(20)

where ${\tilde{x}}_1,\dots ,{\tilde{x}}_T$ are the solution of (4) and ${\tilde{x}}_{1-g},\dots ,{\tilde{x}}_{1-1}$ and ${\tilde{x}}_{T+1},\dots ,{\tilde{x}}_{T+h}$ are defined by (11) and (12).

Proof. 

Because the objective function of (4) is coercive and strictly convex with respect to $x_1,\dots ,x_T$, ${\tilde{x}}_1,\dots ,{\tilde{x}}_T$ are the unique global minimizer of the function. It follows that:

$$\begin{array}{c}\hfill \sum _{t=1}^T{(y_t-x_t)}^2+\lambda \sum _{t=p+1}^T|{\Delta }^p{x}_t|\ge \sum _{t=1}^T{(y_t-{\tilde{x}}_t)}^2+\lambda \sum _{t=p+1}^T\left|{\Delta }^p{\tilde{x}}_t\right|,\end{array}$$

(21)

where the equality holds only if $x_t={\tilde{x}}_t$ for $t=1,\dots ,T$.5 In addition, from (11) and (12), $y_{1-i}={\tilde{x}}_{1-i}$ for $i=1,\dots ,g$, and $y_{T+j}={\tilde{x}}_{T+j}$ for $j=1,\dots ,h$, we have the following inequalities:

$$\begin{array}{cc}\hfill \lambda \sum _{t=p+1-g}^{p+1-1}\left|{\Delta }^p{x}_t\right|& \ge 0=\lambda \sum _{t=p+1-g}^{p+1-1}\left|{\Delta }^p{\tilde{x}}_t\right|,\hfill \end{array}$$

(22)

$$\begin{array}{cc}\hfill \lambda \sum _{t=T+1}^{T+h}\left|{\Delta }^p{x}_t\right|& \ge 0=\lambda \sum _{t=T+1}^{T+h}\left|{\Delta }^p{\tilde{x}}_t\right|,\hfill \end{array}$$

(23)

$$\begin{array}{cc}\hfill \sum _{t=1-g}^{1-1}{(y_t-x_t)}^2& \ge 0=\sum _{t=1-g}^{1-1}{(y_t-{\tilde{x}}_t)}^2,\hfill \end{array}$$

(24)

$$\begin{array}{cc}\hfill \sum _{t=T+1}^{T+h}{(y_t-x_t)}^2& \ge 0=\sum _{t=T+1}^{T+h}{(y_t-{\tilde{x}}_t)}^2.\hfill \end{array}$$

(25)

Combining (21)–(23) yields

$$\begin{array}{c}\hfill \sum _{t=1}^T{(y_t-x_t)}^2+\lambda \sum _{t=p+1-g}^{T+h}|{\Delta }^p{x}_t|\ge \sum _{t=1}^T{(y_t-{\tilde{x}}_t)}^2+\lambda \sum _{t=p+1-g}^{T+h}\left|{\Delta }^p{\tilde{x}}_t\right|,\end{array}$$

(26)

where the equality in (26) holds only if $x_t={\tilde{x}}_t$ for $t=1-g,\dots ,T+h$, which proves that ${\tilde{x}}_t^{\left(\mathrm{d}\right)}={\tilde{x}}_t$ for $t=1-g,\dots ,T+h$. Likewise, combining (21)–(25) proves that ${\tilde{x}}_t^{\left(\mathrm{e}\right)}={\tilde{x}}_t$ for $t=1-g,\dots ,T+h$ and combining (21), (24) and (25) proves that ${\tilde{x}}_t^{\left(\mathrm{f}\right)}={\tilde{x}}_t$ for $t=1-g,\dots ,T+h$. ☐

As an illustration of the above theorem, we give a numerical example. Consider the case where $T=5$, $g=1$, and $h=2$. Suppose that we obtained

$${\tilde{x}}_1=3,\Delta {\tilde{x}}_2=2,{[{\Delta }^2{\tilde{x}}_3,{\Delta }^2{\tilde{x}}_4,{\Delta }^2{\tilde{x}}_5]}^{\mathsf{\top }}={[0,-1,0]}^{\mathsf{\top }}$$

by applying ${\ell }_1$ polynomial trend filtering of order 2 (i.e., ${\ell }_1$ trend filtering) to a T-dimensional time-series data.6 Because $2=\Delta {\tilde{x}}_2=\Delta {\tilde{x}}_3\ne \Delta {\tilde{x}}_4=\Delta {\tilde{x}}_5=1$, the line plot of $(t,{\tilde{x}}_t)$ for $t=1,\dots ,5$ becomes a continuous piecewise linear line such that $(3,{\tilde{x}}_3)$ is a knot. ${\tilde{x}}_t$ for $t=1,\dots ,5$ are explicitly ${[{\tilde{x}}_1,{\tilde{x}}_2,{\tilde{x}}_3,{\tilde{x}}_4,{\tilde{x}}_5]}^{\mathsf{\top }}={[3,5,7,8,9]}^{\mathsf{\top }}$. Then, from the above theorem, in the case, ${\tilde{x}}_t^{\left(i\right)}$ for $i=\mathrm{d},\mathrm{e},\mathrm{f}$ and $t=1-1,\dots ,5+2$ are as follows:

$$\begin{array}{c}\hfill {[{\tilde{x}}_{1-1}^{\left(i\right)},{\tilde{x}}_1^{\left(i\right)},{\tilde{x}}_2^{\left(i\right)},{\tilde{x}}_3^{\left(i\right)},{\tilde{x}}_4^{\left(i\right)},{\tilde{x}}_5^{\left(i\right)},{\tilde{x}}_{5+1}^{\left(i\right)},{\tilde{x}}_{5+2}^{\left(i\right)}]}^{\mathsf{\top }}={[1,3,5,7,8,9,10,11]}^{\mathsf{\top }}.\end{array}$$

Theorem 2.

If $\lambda \ge 2\parallel {\left(\mathit{D}{\mathit{D}}^{\mathsf{\top }}\right)}^{-1}{\mathit{D}\mathit{y}\parallel }_{\infty }$, for $i=\mathrm{d},\mathrm{e},\mathrm{f}$ and $t=1-g,\dots ,T+h$, it follows that

$$\begin{array}{c}\hfill {\tilde{x}}_t^{\left(i\right)}={\widehat{\beta }}_0{t}^0+\cdots +{\widehat{\beta }}_{p-1}{t}^{p-1},\end{array}$$

(27)

where $({\widehat{\beta }}_0,\dots ,{\widehat{\beta }}_{p-1})={\arg \min }_{{\beta }_0,\dots ,{\beta }_{p-1}}{\sum }_{t=1}^T{(y_t-{\beta }_0{t}^0-\cdots -{\beta }_{p-1}{t}^{p-1})}^2$.

Proof. 

Because ${\mathit{D}}_{g+T+h}$ is a $(g+T+h-p)\times (g+T+h)$$(p+1)$-diagonal Toeplitz matrix, such that:

$$\begin{array}{c}\hfill {\mathit{D}}_{g+T+h}=\left[\begin{array}{cccccc}{a}_0& \cdots & a_p& 0& \cdots & 0\\ 0& \ddots & \ddots & \ddots & ⋮\\ ⋮& \ddots & \ddots & \ddots & 0\\ 0& \cdots & 0& a_0& \cdots & a_p\end{array}\right],\end{array}$$

where $a_k={(-1)}^{p-k}\left(\genfrac{}{}{0pt}{}{p}{k}\right)$ for $k=0,\dots ,p$, it may be expressed as

$$\begin{array}{c}\hfill {\mathit{D}}_{g+T+h}=\left[\begin{array}{ccc}{\mathit{G}}_1& {\mathit{G}}_2& \mathbf{0}\\ \mathbf{0}& \mathit{D}& \mathbf{0}\\ \mathbf{0}& {\mathit{H}}_1& {\mathit{H}}_2\end{array}\right],\end{array}$$

where ${\mathit{G}}_1$ is a $g\times g$ upper triangular matrix, ${\mathit{G}}_2$ is a $g\times T$ matrix, ${\mathit{H}}_1$ is an $h\times T$ matrix, and ${\mathit{H}}_2$ is an $h\times h$ unit lower-triangular matrix. For example, when $p=3$, $g=h=2$, and $T=5$:

$$\begin{array}{c}\hfill {\mathit{D}}_{2+5+2}=\left[\begin{array}{ccccccccc}-1& 3& -3& 1& 0& 0& 0& 0& 0\\ 0& -1& 3& -3& 1& 0& 0& 0& 0\\ 0& 0& -1& 3& -3& 1& 0& 0& 0\\ 0& 0& 0& -1& 3& -3& 1& 0& 0\\ 0& 0& 0& 0& -1& 3& -3& 1& 0\\ 0& 0& 0& 0& 0& -1& 3& -3& 1\end{array}\right].\end{array}$$

(28)

Let ${\tilde{\mathit{x}}}_g={[{\tilde{x}}_{1-g},\dots ,{\tilde{x}}_{1-1}]}^{\mathsf{\top }}$, $\tilde{\mathit{x}}={[{\tilde{x}}_1,\dots ,{\tilde{x}}_T]}^{\mathsf{\top }}$, ${\tilde{\mathit{x}}}_h={[{\tilde{x}}_{T+1},\dots ,{\tilde{x}}_{T+h}]}^{\mathsf{\top }}$, and ${\tilde{\mathit{x}}}_{g+T+h}={[{\tilde{\mathit{x}}}_g^{\mathsf{\top }},{\tilde{\mathit{x}}}^{\mathsf{\top }},{\tilde{\mathit{x}}}_h^{\mathsf{\top }}]}^{\mathsf{\top }}$, which is a $(g+T+h)$-dimensional column vector. Then, by definition of ${\tilde{\mathit{x}}}_g$ and ${\tilde{\mathit{x}}}_h$, it follows that:

$$\begin{array}{cc}\hfill {\mathit{G}}_1{\tilde{\mathit{x}}}_g+{\mathit{G}}_2\tilde{\mathit{x}}& =\mathbf{0},\hfill \end{array}$$

(29)

$$\begin{array}{cc}\hfill {\mathit{H}}_1\tilde{\mathit{x}}+{\mathit{H}}_2{\tilde{\mathit{x}}}_h& =\mathbf{0},\hfill \end{array}$$

(30)

which leads to:

$$\begin{array}{c}\hfill {\mathit{D}}_{g+T+h}{\tilde{\mathit{x}}}_{g+T+h}=\left[\begin{array}{c}\mathbf{0}\\ \mathit{D}\tilde{\mathit{x}}\\ \mathbf{0}\end{array}\right].\end{array}$$

(31)

From Kim et al. (2009), if $\lambda \ge 2\parallel {\left(\mathit{D}{\mathit{D}}^{\mathsf{\top }}\right)}^{-1}{\mathit{D}\mathit{y}\parallel }_{\infty }$, it follows that $\tilde{\mathit{x}}=\mathsf{\Pi }\widehat{\mathbf{\beta }}$, where $\widehat{\mathbf{\beta }}={\left({\mathsf{\Pi }}^{\mathsf{\top }}\mathsf{\Pi }\right)}^{-1}{\mathsf{\Pi }}^{\mathsf{\top }}\mathit{y}$. Recalling that $\mathit{D}\mathsf{\Pi }=\mathbf{0}$, we obtain ${\mathit{D}}_{g+T+h}{\tilde{\mathit{x}}}_{g+T+h}=\mathbf{0}$ if $\lambda \ge 2\parallel {\left(\mathit{D}{\mathit{D}}^{\mathsf{\top }}\right)}^{-1}{\mathit{D}\mathit{y}\parallel }_{\infty }$, which implies that ${\tilde{\mathit{x}}}_{g+T+h}$ may be represented as ${\mathsf{\Pi }}_{g+T+h}\mathbf{\gamma }$. Because $\tilde{\mathit{x}}=\mathsf{\Pi }\widehat{\mathbf{\beta }}$, $\mathbf{\gamma }$ must equal $\widehat{\mathbf{\beta }}$. Therefore, if $\lambda \ge 2\parallel {\left(\mathit{D}{\mathit{D}}^{\mathsf{\top }}\right)}^{-1}{\mathit{D}\mathit{y}\parallel }_{\infty }$, then ${\tilde{\mathit{x}}}_{g+T+h}={\mathsf{\Pi }}_{g+T+h}\widehat{\mathbf{\beta }}$. ☐

Theorem 3.

Suppose that $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$, where $\mathbf{\alpha }\ne \mathbf{0}$ is a p-dimensional column vector. Then, for $i=\mathrm{d},\mathrm{e},\mathrm{f}$, it follows that:

$$\begin{array}{c}\hfill {\tilde{\mathit{x}}}_{g+T+h}^{\left(i\right)}={\mathsf{\Pi }}_{g+T+h}\mathbf{\alpha },\end{array}$$

(32)

where ${\tilde{\mathit{x}}}_{g+T+h}^{\left(i\right)}={[{\tilde{x}}_{1-g}^{\left(i\right)},\dots ,{\tilde{x}}_{T+h}^{\left(i\right)}]}^{\mathsf{\top }}$.

Proof. 

If $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$, it follows that: $\tilde{\mathit{x}}=\mathsf{\Pi }\mathbf{\alpha }$. Accordingly, ${\mathit{D}}_{g+T+h}{\tilde{\mathit{x}}}_{g+T+h}=\mathbf{0}$, which indicates that ${\tilde{\mathit{x}}}_{g+T+h}$ may be represented as ${\mathsf{\Pi }}_{g+T+h}\mathbf{\gamma }$. Because $\tilde{\mathit{x}}=\mathsf{\Pi }\mathbf{\alpha }$ if $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$, $\mathbf{\gamma }$ must equal $\mathbf{\alpha }$. Therefore, we obtain ${\tilde{\mathit{x}}}_{g+T+h}={\mathsf{\Pi }}_{g+T+h}\mathbf{\alpha }$ if $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$. ☐

Corollary 1.

Let ${\tilde{\mathit{x}}}_{g+T+h}^{\left(i\right)}={[{\tilde{x}}_{1-g}^{\left(i\right)},\dots ,{\tilde{x}}_{T+h}^{\left(i\right)}]}^{\mathsf{\top }}$ for $i=d,e,f$.

(i) 

Denote the $(j+1)$-th column of Π and that of ${\mathsf{\Pi }}_{g+T+h}$, respectively, by ${\mathbf{\tau }}_j$ and by ${\mathbf{\tau }}_{g+T+h,j}$ for $j=0,\dots ,p-1$. If $\mathit{y}={\mathbf{\tau }}_j$, then ${\tilde{\mathit{x}}}_{g+T+h}^{\left(i\right)}={\mathbf{\tau }}_{g+T+h,j}$ for any $\lambda >0$.

(ii) 

Let $\mathbf{z}$ be a T-dimensional column vector. If $\mathit{y}=\mathsf{\Pi }{\left({\mathsf{\Pi }}^{\mathsf{\top }}\mathsf{\Pi }\right)}^{-1}{\mathsf{\Pi }}^{\mathsf{\top }}\mathbf{z}$, then ${\tilde{\mathit{x}}}_{g+T+h}^{\left(i\right)}={\mathsf{\Pi }}_{g+T+h}{\left({\mathsf{\Pi }}^{\mathsf{\top }}\mathsf{\Pi }\right)}^{-1}{\mathsf{\Pi }}^{\mathsf{\top }}\mathbf{z}$ for any $\lambda >0$.

4. Some Remarks on the Main Results

First, we make a remark on Theorem 1. Because $|{\mathit{G}}_1{|=(-1)}^{g·p}$, from (29), ${\tilde{\mathit{x}}}_g$ may be expressed with $\tilde{\mathit{x}}$ as ${\tilde{\mathit{x}}}_g=-{\mathit{G}}_1^{-1}{\mathit{G}}_2\tilde{\mathit{x}}$. Likewise, because $|{\mathit{H}}_2|=1$, from (30), ${\tilde{\mathit{x}}}_h$ may be expressed with $\tilde{\mathit{x}}$ as ${\tilde{\mathit{x}}}_h=-{\mathit{H}}_2^{-1}{\mathit{H}}_1\tilde{\mathit{x}}$. Thus, the modified ${\ell }_1$ polynomial trend filtering, (16), may be characterized as a filtering that calculates

$$\begin{array}{c}\hfill \left[\begin{array}{c}-{\mathit{G}}_1^{-1}{\mathit{G}}_2\\ {\mathbf{I}}_T\\ -{\mathit{H}}_2^{-1}{\mathit{H}}_1\end{array}\right]\tilde{\mathit{x}}\end{array}$$

(33)

from $\mathit{y}$.7 In addition, from Kim et al. (2009), it follows that $\tilde{\mathit{x}}\to \mathit{y}$ as $\lambda \to 0$. Therefore, we obtain:

$$\begin{array}{c}\hfill {\tilde{\mathit{x}}}_{g+T+h}^{\left(\mathrm{d}\right)}\to \left[\begin{array}{c}-{\mathit{G}}_1^{-1}{\mathit{G}}_2\\ {\mathbf{I}}_T\\ -{\mathit{H}}_2^{-1}{\mathit{H}}_1\end{array}\right]\mathit{y},(\lambda \to 0).\end{array}$$

(34)

Second, we provide a remark on Theorems 2 and 3. Yamada (2017b) recently showed that:

$$\begin{array}{c}\hfill \tilde{\mathit{x}}=\mathsf{\Pi }\widehat{\mathbf{\beta }}+\mathit{x}\tilde{\mathbf{\varphi }},\end{array}$$

(35)

where $\mathit{x}={\mathit{D}}^{\mathsf{\top }}{\left(\mathit{D}{\mathit{D}}^{\mathsf{\top }}\right)}^{-1}$ and $\tilde{\mathbf{\varphi }}$, which is a $(T-p)$-dimensional column vector, is the solution of the following Lasso regression/BPDN:

$$\begin{array}{c}\hfill \underset{\mathbf{\varphi }}{min}{\parallel \mathit{y}-\mathit{x}\mathbf{\varphi }\parallel }_2^2+\lambda {\parallel \mathbf{\varphi }\parallel }_1.\end{array}$$

(36)

Because ${\mathit{x}}^{\mathsf{\top }}\mathsf{\Pi }=\mathbf{0}$, $\mathsf{\Pi }\widehat{\mathbf{\beta }}+\mathit{x}\tilde{\mathbf{\varphi }}$ in (35) represents an orthogonal decomposition of $\tilde{\mathit{x}}$. Here, we show that we may prove Theorems 2 and 3 by using (35) and (36). Premultiplying (35) by $\mathit{D}$ yields $\mathit{D}\tilde{\mathit{x}}=\tilde{\mathbf{\varphi }}$. We accordingly obtain:

$$\begin{array}{c}\hfill {\mathit{D}}_{g+T+h}{\tilde{\mathit{x}}}_{g+T+h}=\left[\begin{array}{c}\mathbf{0}\\ \tilde{\mathbf{\varphi }}\\ \mathbf{0}\end{array}\right].\end{array}$$

(37)

(i)

From (Osborne et al. 2000, p. 324), if $\lambda \ge 2\parallel {\mathit{x}}^{\mathsf{\top }}{\mathit{y}\parallel }_{\infty }$, then $\tilde{\mathbf{\varphi }}=\mathbf{0}$. Therefore, we obtain $\tilde{\mathit{x}}=\mathsf{\Pi }\widehat{\mathbf{\beta }}$ and ${\mathit{D}}_{g+T+h}{\tilde{\mathit{x}}}_{g+T+h}=\mathbf{0}$, which proves Theorem 2.

(ii)

If $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$, where $\mathbf{\alpha }\ne \mathbf{0}$, then ${\mathit{x}}^{\mathsf{\top }}\mathit{y}=\mathbf{0}$, which implies that $\lambda >2\parallel {\mathit{x}}^{\mathsf{\top }}{\mathit{y}\parallel }_{\infty }=0$. Again, from Osborne et al. (2000), we obtain $\tilde{\mathbf{\varphi }}=\mathbf{0}$ if $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$. Therefore, if $\mathit{y}=\mathsf{\Pi }\mathbf{\alpha }$, it follows that $\tilde{\mathit{x}}=\mathsf{\Pi }\widehat{\mathbf{\beta }}=\mathsf{\Pi }\mathbf{\alpha }$ and ${\mathit{D}}_{g+T+h}{\tilde{\mathit{x}}}_{g+T+h}=\mathbf{0}$, which proves Theorem 3.

Third, we give an example of Corollary 1 (i). For the case where $\mathit{y}={[1,\dots ,5]}^{\mathsf{\top }}$ and $p=g=h=2$, it follows that ${\tilde{\mathit{x}}}_{2+5+2}^{\left(\mathrm{d}\right)}={[-1,0,1,\dots ,5,6,7]}^{\mathsf{\top }}$ for any $\lambda >0$.

5. Concluding Remarks

The ${\ell }_1$ polynomial trend filtering method is a promising piecewise polynomial curve-fitting method because it does not require prespecifying the number and location of knots. We have shown some theoretical results on this method. One of them is that a small modification of the filtering provides identical trend estimates and also extrapolations of the trend beyond both sample limits. Another is that ${\tilde{x}}_{T+1},\dots ,{\tilde{x}}_{T+h}$ based on (12) are useless to improve the trend estimates of ${\ell }_1$ polynomial trend filtering. We also provided a MATLAB function for calculating the solution of one of the modified filtering methods. The main results of the paper are summarized in Theorems 1–3 and Corollary 1.

Finally, we remark that applying the modified ${\ell }_1$ polynomial trend filtering (16)–(18) requires specification of the value of $\lambda$. For this purpose, the methods proposed in Yamada and Yoon (2016) and Yamada (2018) are applicable.