Constructive Approximation of Nonlinear Operators Based on Piecewise Interpolation Technique

Abstract

Suppose $K_Y$ and $K_X$ are the image and the preimage of a nonlinear operator $\mathcal{F}:K_Y\to K_X$. It is supposed that the cardinality of each $K_Y$ and $K_X$ is N and N is large. We provide an approximation to the map $\mathcal{F}$ that requires prior information only on a few elements p from $K_Y$, where $p\ll N$, but still effectively represents $\mathcal{F}\left(K_Y\right)$. It is achieved under Lipschitz continuity assumptions. The device behind the proposed method is based on a special extension of the piecewise linear interpolation technique to the case of sets of stochastic elements. The proposed technique provides a single operator that transforms any element from the arbitrarily large set $K_Y$. The operator is determined in terms of pseudo-inverse matrices so that it always exists.

1. Introduction

A purpose of the proposed methodology is to provide an effective way to transform large data sets. The methodology is motivated by the problems arising in signal processing, where the nonlinear operator $\mathcal{F}:K_Y\to K_X$ is interpreted as a nonlinear system (or a nonlinear filter) transforming a set of stochastic signals $K_Y$ to the set of stochastic signals $K_X$. Therefore, below we refer to this terminology.

The device behind the proposed method is based on a special extension of the piecewise linear interpolation technique to the case of stochastic signal sets. The device is not straightforward and requires the careful substantiation presented in Section 2.2, Section 3.4, Section 4.2 and Section 4.4 below.

1.1. Motivations

1.1.1. Filtering of Large Arrays of Stochastic Signals

We consider the case when the cardinality N of signal sets $K_Y$ and $K_X$ is large. Members of $K_Y$ and $K_X$ are represented by stochastic vectors. The problem of finding a filter that effectively transforms the large set $K_Y$ to the large set $K_X$ has been considered in a number of works such as those represented in [1,2,3,4,5,6,7,8,9,10,11,12,13,14]. The approaches considered in these references require prior information on each reference signal. This requirement poses a significant difficulty on the applicability of the methods mentioned above. For processing of large signal arrays, such restrictions are quite inconvenient. The exception is the techniques provided in [15,16], which exploit the initial information in the form of the vector derived from averaging over the reference signal set.

In this paper, we show how to construct a filter $\mathcal{F}$ that requires prior information only on few signals, $p\ll N$, from $K_X$ but still performs effectively compared to known filters based on prior information on every reference signal from $K_X$. Such a filter is denoted by ${\mathcal{F}}^{(p-1)}$.

1.1.2. Basic Idea of the Proposed Methodology

The specific feature of the proposed filter is an adjustment and extension of the piecewise function interpolation technique [17] to filtering of stochastic signals. It is well known that the piecewise function interpolation [17] often provides better accuracy and faster performance than those associated with the linear and polynomial approximation used in known filtering techniques (such as, for example, those in [5,9]).

The device of the filter under consideration is given in Section 2.2, Section 3.1 and Section 4.2.

1.1.3. Pseudo-Inverse Matrices and Related Matters

The known filter models proposed in [1,2,3,6,7,8,10,13,14], use inverse matrices. In the cases of grossly corrupted signals or erroneous measurements, those inverse matrices may not exist, and thus, those filters cannot be applied. The examples in Section 5 illustrate this case.

The filter proposed here avoids this drawback since its model is based on exploiting pseudo-inverse matrices. As a result, the proposed filter always exists. That is, it processes any kind of noisy signals. Exploiting pseudo-inverse matrices is not straightforward and requires the development of a specific technique. Perhaps it was a reason to not use the pseudo-inverse matrices in the above-mentioned references. Here, an extension of the proposed filtering techniques to the case of implementation of the pseudo-inverse matrices is performed on the basis of theory represented in [5].

1.1.4. Computational Load

Let $\mathbf{x}={[x_1,\dots ,x_m]}^T\in K_X$ and $\mathbf{y}={[y_1,\dots ,y_n]}^T\in K_Y$. An implementation of the filters provided in [1,2,3,4,5,6,7,8,10,13,14] leads to computation of an $n\times n$ inverse or pseudo-inverse matrix for each pair of signals $\mathbf{x}\in K_X$ and $\mathbf{y}\in K_Y$, and a matrix product of $m\times n$ and $n\times n$ matrices. Thus, for the processing of arrays in $K_X$ and $K_Y$, the filters in [1,2,3,4,5,6,7,8,10,13,14] require $O\left(2m{n}^{2}N\right)+O\left(26n^{3}N\right)$ flops [18].

Arrays $K_X$ and $K_Y$ can be represented by vectors, $\chi$ and $\gamma$, each with $mN$ and $nN$ components, respectively. Then an application of the techniques in [1,2,3,4,5,6,7,8,10,13,14] requires $O\left(2m{n}^2{N}^2\right)$ and $O\left(26n^3{N}^3\right)$ flops [18].

Thus, for the case when N is large, the computational work associated with the methods in [1,2,3,4,5,6,7,8,10,13,14] becomes, in practice, unacceptable.

The required computational load for the filter ${\mathcal{F}}^{(p-1)}$ considered below is substantially less. This is implied by an ability to use only p pseudo-inverse matrices, where p is much less than cardinality N. For this reason, ${\mathcal{F}}^{(p-1)}$ requires only $O\left(2m{n}^{2}p\right)+O\left(26n^{3}p\right)$ flops. Clearly, this is less than that required by the filters mentioned above. This issue is illustrated in Section 5.

1.2. Relevant Works

Some particular filtering techniques relevant to the method are proposed below.

1.2.1. Generic Optimal Linear (GOL) Filter [5]

The generic optimal linear (GOL) filter in [5] is a generalization of the Wiener filter to the case when the covariance matrix is not invertible and the observable signal is arbitrarily noisy (i.e., when, in particular, noise is not necessarily additive and Gaussian). The GOL filter has been developed for processing an individual stochastic signal. Some ideas from [5] are used in the proof of Theorem 1 below.

1.2.2. Simplicial Canonical Piecewise Linear Filter [13]

A complex Wiener adaptive filter was developed in [13] from the two-dimensional complex-valued simplicial canonical piecewise linear filter [19]. The filter in [13] was developed for the processing of an individual stochastic signal and can be exploited when the reference signal is known and a ‘covariance-like’ matrix is invertible. The latter precludes an application to the signal types considered, for example, in Section 5: the matrices used in [13] are not invertible for the signals as those in Section 5. Similarly, the filters studied in [8,10] were developed for the processing of a single signal when the covariance matrices are invertible.

For the filter proposed here, these restrictions are removed.

1.2.3. Adaptive Piecewise Linear Filter [12]

A piecewise linear filter in [12] was proposed for a fixed image denoising (given by a matrix), corrupted by an additive Gaussian noise. That is, the method involved a non-stochastic reference signal and required its knowledge. No theoretical justification for the filter was given in [12].

1.2.4. Averaging Polynomial Filter [15,16]

The averaging polynomial filter proposed in [15,16] was developed for the purpose of processing infinite signal sets. The filter was based on an argument involving the ‘averaging’ over sets of signals under consideration. This device allows one to determine a single filter for the processing of infinite signal sets. At the same time, it leads to an increase in the associated error when signals differ considerably from each other. This effect is illustrated in Section 5 below.

1.2.5. Other Relevant Filters

The technique developed in [11] is an extension of the GOL filter to the constraint problem with respect to the filter rank. It concerns data compression.

The methods in [6,7,20,21] have been developed for deterministic signals. Motivated by the results achieved in [20,21], adaptive filters were elaborated in [22]. A theoretical basis for the device proposed in [20,21] is provided in [23].

We note that the idea of piecewise linear filtering has been used in the literature in several very different conceptual frameworks, despite exploiting some very similar terms (as in [12,13,19,20,21,22,23,24,25,26]). At the same time, a common feature of those techniques is that they were developed for the processing of a single signal, not of large signal sets as in this paper. In particular, piecewise linear filters in [24] have been obtained by arranging linear filters and thresholds in a tree structure. Piecewise linear filters discussed in [25] were developed using so-called threshold decomposition, which is a segmentation operator exploited to split a signal into a set of multilevel components. Filter design methods for piecewise linear systems proposed in [26] were based on a piecewise Lyapunov function.

1.3. Difficulties Associated with the Known Filtering Techniques

Basic difficulties associated with applying the known filtering techniques to the case under consideration (i.e., to processing of large signal sets, $K_X$ and $K_Y$) are as follows:

(i)

They require information on each reference signal (in the form of a sample, for example).

(ii)

Matrices used in the known filters can be non-invertible (as in the simulations considered below in Section 5) and then the filter does not exist.

(iii)

The associated computation work may require a very long time. For example, in simulations (Section 5), MATLAB was out of memory for computing the GOL filter [5] when each of the sets $K_X$ and $K_Y$ was represented by a long vector (this option has been discussed in Section 1.1.4 above). The PC used for the simulations was a Dell Latitude 7400 with a CPU Intel Core i5 (8th Gen) 8265U/1.6 GHz and 8 GB RAM.

1.4. Differences from the Known Filtering Techniques

The differences from the known filtering techniques discussed above are as follows:

(i)

We consider a single filter that processes arbitrarily large input-output sets of stochastic signal vectors. The known filters [1,2,3,4,5,6,7,8,9,10,11,12,13,14,19,20,21,22,23,24,25,26] have been developed for the processing of an individual signal vector only. In the case of their application to arbitrarily large signal sets, they imply difficulties described in Section 1.1 and Section 1.3 above.

(ii)

As a result, our piecewise linear filter model (Section 3), the statement of the problem (Section 3.3 below), and consequently, the device of its solution (Section 4 below) are different from those considered in [12,13,19,20,21,22,23,24,25,26]. In this regard, see also Section 1.2.5.

(iii)

The above naturally leads to a new structure of the filter (presented in Section 3.4 and Section 4.2 below), which is very different from the known ones.

1.5. Contribution

In general, for the processing of large data sets, the proposed filter allows us to achieve better results in comparison with the known techniques in [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,19,20,21,22,23,24,25,26]. In particular, it allows us to achieve the following:

(i)

Achieve a desired accuracy in signal estimation This means that any desired accuracy is achieved theoretically, as is shown in Section 4.4 below. In practice, of course, the accuracy is increased to a prescribed reasonable level.

(ii)

Exploit prior information only on few reference signals, p, from the set $K_X$ that contains $N\gg p$ signals or even an infinite number of signals.

(iii)

Find a single filter to process any signal from the arbitrarily large signal set.

(iv)

Determine the filter in terms of pseudo-inverse matrices so that the filter always exists.

(v)

Decrease the computational load compared to the related known techniques.

2. Some Preliminaries

2.1. Notation

The signal sets we consider are, in fact, special representations of time series.

Let $(\mathsf{\Omega },\mathsf{\Sigma },\mu )$ be a probability space (usually, $\mathsf{\Omega }=\left\{\omega \right\}$ is the set of outcomes, $\mathsf{\Sigma }$ is a $\sigma$-field of measurable subsets in $\mathsf{\Omega }$ and $\mu :\mathsf{\Sigma }\to [0,1]$ is an associated probability measure on $\mathsf{\Sigma }$; in particular, $\mu \left(\mathsf{\Omega }\right)=1.$) and $K_X$ and $K_Y$ be arbitrarily large sets of signals such that

$$K_X=\{\mathbf{x}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^m)|t\in T\}\mathrm{and} K_Y=\{\mathbf{y}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^n)|t\in T\}$$

where $T:=[a,b]\subseteq \mathbb{R}.$ We interpret $\mathbf{x}(t,·)$ as a reference signal and $\mathbf{y}(t,·)$ as an observable signal, an input to the filter $\mathcal{F}$ studied below. In an intuitive way, $\mathbf{y}$ can be regarded as a noise-corrupted version of $\mathbf{x}$. For example, $\mathbf{y}$ can be interpreted as $\mathbf{y}=\mathbf{x}+\mathbf{n}$, where $\mathbf{n}$ is white noise. In this paper, we do not restrict ourselves to this simplest version of $\mathbf{y}$ and assume that the dependence of $\mathbf{y}$ on $\mathbf{x}$ and $\mathbf{n}$ is arbitrary. The variable $t\in T\subseteq \mathbb{R}$ represents time. More generally, T can be considered as a set of parameter vectors $\alpha ={({\alpha }^{\left(1\right)},\dots ,{\alpha }^{\left(q\right)})}^T\in C^q\subseteq {\mathbb{R}}^q$, where $C^q$ is a q-dimensional cube, i.e., $\mathbf{y}=\mathbf{y}(\alpha ,·)$ and $\mathbf{x}=\mathbf{x}(\alpha ,·)$. One coordinate, say ${\alpha }^{\left(1\right)}$ of $\alpha$, could be interpreted as time. Then, for example, the stochastic signal $\mathbf{x}(t,·)$ can be interpreted as an arbitrary stationary time series.

Let ${\left\{t_k\right\}}_1^p\subset T$ be a sequence of fixed time-points such that

$$a=t_1<\dots <t_p=b.$$

(1)

Because of Equation (1), the sets $K_Y$ and $K_X$ are divided in ‘smaller’ subsets $K_{X,1},\dots ,K_{X,p-1}$ and $K_{Y,1},\dots ,K_{Y,p-1}$, respectively, so that, for each $j=1,\dots ,p$,

$$K_{X,j}=\left\{\mathbf{x}(t,·)\right|t_j\le t\le t_{j+1}\}\mathrm{and} K_{Y,j}=\left\{\mathbf{y}(t,·)\right|t_j\le t\le t_{j+1}\}.$$

(2)

Therefore, $K_Y$ and $K_X$ can now be represented as

$$K_X=\bigcup _{j=1}^{p-1}{K}_{X,j}\mathrm{and} K_Y=\bigcup _{j=1}^{p-1}{K}_{Y,j}.$$

(3)

2.2. Brief Description of the Method

We wish to determine a single filter that estimates $\mathbf{x}\in K_X$ from $\mathbf{y}\in K_Y$ with a controlled, associated error. Array $K_Y$ can be finite or infinite.

To this end, the proposed filter ${\mathcal{F}}^{(p-1)}:K_Y\to K_X$ is represented by a sum of sub-filters ${\mathcal{F}}_1,\dots ,{\mathcal{F}}_{p-1}$ (see Equations (4) and (5) below). The sub-filter ${\mathcal{F}}_j$ transforms signals of subset $K_{Y,j}$ of array $K_Y$ to signals of subset $K_{X,j}$ of array $K_X$, i.e., ${\mathcal{F}}_j:K_{Y,j}\to K_{X,j}$. The prime idea is to determine ${\mathcal{F}}_j$ separately, for each $j=1,\dots ,p-1$, from an associated minimization problem (Equation (11)) (see Section 3.4 and Section 4.2 below). This procedure provides an estimate ${\mathcal{F}}_j\left[\mathbf{y}(t,·)\right]$ that interpolates $\mathbf{x}(t,·)\in K_{X,j}$ at $t=t_j$ and $t=t_{j+1}$. It is natural to expect that the processing of a ‘smaller’ subset, $K_{Y,j}$, may lead to a smaller error than that associated with the processing of the whole array $K_Y$.

Therefore, the advantages of ${\mathcal{F}}^{(p-1)}\left[\mathbf{y}(t,·)\right]$ are similar to those that follow from an application of the piecewise interpolation procedure, such as, for example, the high accuracy of estimation.

Section 4.4 confirms this observation. Importantly, it is shown in Section 4.5 and Section 5 that the proposed technique avoids the bottlenecks of the known methods discussed in Section 1.3 above.

3. Description of the Problem

3.1. Piecewise Linear Filter Model

Let ${\mathcal{F}}^{(p-1)}:K_Y\to K_X$ be a filter such that, for each $t\in T$,

$${\mathcal{F}}^{(p-1)}\left[\mathbf{y}(t,·)\right]=\sum _{j=1}^{p-1}{\delta }_j{\mathcal{F}}_j\left[\mathbf{y}(t,·)\right],$$

(4)

where

$${\mathcal{F}}_j\left[\mathbf{y}(t,·)\right]={\alpha }_j+{\mathcal{B}}_j\left[\mathbf{y}(t,·)\right]\mathrm{and} {\delta }_j=\left\{\begin{array}{cc}1,& if t_j\le t\le t_{j+1},\hfill \\ 0,& \mathrm{otherwise}.\hfill \end{array}\right.$$

(5)

Here, ${\mathcal{F}}_j$ is a sub-filter defined for $t_j\le t\le t_{j+1}$. In Equation (5), ${\alpha }_j={[{\alpha }_j^{\left(1\right)},\dots ,{\alpha }_j^{\left(m\right)}]}^T\in {\mathbb{R}}^m$ and ${\mathcal{B}}_j:L^2(\mathsf{\Omega },{\mathbb{R}}^n)\to L^2(\mathsf{\Omega },{\mathbb{R}}^m)$ is a linear operator given by a matrix $B_j\in {\mathbb{R}}^{m\times n}$, so that

$$\left[{\mathcal{B}}_j\left(\mathbf{y}\right)\right](t,\omega )=B_j\left[\mathbf{y}(t,\omega )\right].$$

Thus, ${\mathcal{F}}_j$ is defined by an operator $F_j:{\mathbb{R}}^n\to {\mathbb{R}}^m$ such that

$$F_j\left[\mathbf{y}(t,\omega )\right]={\alpha }_j+B_j\left[\mathbf{y}(t,\omega )\right].$$

(6)

Filter ${\mathcal{F}}^{(p-1)}$ defined by Equations (4)–(6) is called the piecewise filter. Hereinafter, we will use a non-curly symbol to denote an operator and associated matrix (e.g., the operator ${\mathcal{F}}_j:L^2(\mathsf{\Omega },{\mathbb{R}}^n)\to L^2(\mathsf{\Omega },{\mathbb{R}}^m)$ and the associated matrix $F_j\in {\mathbb{R}}^{m\times n}$ are denoted by $F_j$).

3.2. Assumptions

In the known approaches related to filtering of stochastic signals (e.g., see [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]), it is assumed that covariance matrices formed from the reference signal and observed signal are known or can be estimated.

The assumption used here is similar. The covariance matrices that are assumed to be known or can be estimated are formed from selected signal pairs $\{\mathbf{x}(t_j,·),\mathbf{y}(t_j,·)\}$ with $j=1,\dots ,p$ and p to be a small number. It is worthwhile to note that it is not assumed that the covariance matrices are known for each signal pair from $K_X\times K_Y$, $\left\{\mathbf{x}\right(t,·),\mathbf{y}(t,·\left)\right\}$ with $t\in [a,b]$), $p\ll N$, where N is the number of signals in $K_X$ or $K_Y$.

3.3. The Problem

In Equations (4)–(6), parameters of the filter ${\mathcal{F}}^{(p-1)}$, i.e., vector ${\alpha }_j$ and matrix $B_j$, for $j=1,\dots ,p-1$, are unknown. Therefore, under the assumptions described in Section 3.2, the problem is to determine ${\alpha }_j$ and $B_j$, for $j=1,\dots ,p-1$. The related problem is to estimate an error associated with the filter ${\mathcal{F}}^{(p-1)}$.

Solutions to both problems are given in Section 4.2 and Section 4.4, respectively. In particular, in the following Section 3.4, interpolation Equations (8) and (11) are introduced that lead to a determination of ${\alpha }_j$ and $B_j$.

3.4. Interpolation Conditions

Let us denote

$$\parallel \mathbf{x}(t_j,·){\parallel }_{\mathsf{\Omega }}^2={\int }_{\mathsf{\Omega }}{\parallel \mathbf{x}(t_j,\omega )\parallel }_2^{2}d\mu \left(\omega \right)$$

(7)

where $\parallel \mathbf{x}(t_j,\omega ){\parallel }_2$ is the Euclidean norm of $\mathbf{x}(t_j,\omega )\in {\mathbb{R}}^m$.

For $t=t_1$, let $\widehat{\mathbf{x}}(t_1,·)$ be an estimate of $\mathbf{x}(t_1,·)$ determined by known methods [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]. This is the initial condition of the proposed technique.

For $j=1,\dots ,p-1$, each sub-filter $F_j$ in Equations (5) and (6) is defined so that ${\alpha }_j$ and ${\mathcal{B}}_j$ satisfy the conditions as follows:

Sub-filter ${\mathcal{F}}_1$: For $j=1$, ${\alpha }_1$ and ${\mathcal{B}}_1$ solve

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t_1,·)={\alpha }_1+{\mathcal{B}}_1\left[\mathbf{y}(t_1,·)\right]\text{and}\underset{{\mathcal{B}}_1}{min}{∥[\mathbf{x}(t_2,·)-{\alpha }_1]-{\mathcal{B}}_1\left[\mathbf{y}(t_2,·)\right]∥}_{\mathsf{\Omega }}^2,\end{array}$$

(8)

respectively. Then an estimate of $\mathbf{x}(t,·)$, $\widehat{\mathbf{x}}(t,·)$, for $t\in [t_1,t_2]$, is determined as

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t,·)={\mathcal{F}}_1\left[\mathbf{y}(t,·)\right]={\alpha }_1+{\mathcal{B}}_1\left[\mathbf{y}(t,·)\right]=\widehat{\mathbf{x}}(t_1,·)+{\mathcal{B}}_1[\mathbf{y}(t,·)-\mathbf{y}(t_1,·)]\end{array}$$

(9)

where ${\alpha }_1$ and ${\mathcal{B}}_1$ satisfy Equation (8). In particular, ${\alpha }_1=\widehat{\mathbf{x}}(t_1,·)-{\mathcal{B}}_1\left[\mathbf{y}(t_1,·)\right]$ and

$$\widehat{\mathbf{x}}(t_2,·)={\mathcal{F}}_1\left[\mathbf{y}(t_2,·)\right].$$

Extending this procedure up to $j=k-1$, where $k=3,\dots ,p$, we set the following: Let $\widehat{\mathbf{x}}(t_{k-1},·)$ be an estimate of $\mathbf{x}(t_{k-1},·)$ defined by the preceding steps as

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t_{k-1},·)={\mathcal{F}}_{k-2}\left[\mathbf{y}(t_{k-1},·)\right].\end{array}$$

(10)

Then sub-filter ${\mathcal{F}}_{k-1}$ is defined as follows:

Sub-filter ${\mathcal{F}}_{k-1}$: For $j=k-1$, ${\alpha }_{k-1}$ and ${\mathcal{B}}_{k-1}$ solve

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t_{k-1},·)={\alpha }_{k-1}+{\mathcal{B}}_{k-1}\left[\mathbf{y}(t_{k-1},·)\right]\mathrm{and}\underset{{\mathcal{B}}_{k-1}}{min}{∥[\mathbf{x}(t_k,·)-{\alpha }_{k-1}]-{\mathcal{B}}_{k-1}\left[\mathbf{y}(t_k,·)\right]∥}_{\mathsf{\Omega }}^2,\end{array}$$

(11)

respectively. Then an estimate of $\mathbf{x}(t,·)$, $\widehat{\mathbf{x}}(t,·)$, for $t\in [t_{k-1},t_k]$, is determined as

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t,·)={\mathcal{F}}_{k-1}\left[\mathbf{y}(t,·)\right]={\alpha }_{k-1}+{\mathcal{B}}_{k-1}\left[\mathbf{y}(t,·)\right]=\widehat{\mathbf{x}}(t_{k-1},·)+{\mathcal{B}}_1[\mathbf{y}(t,·)-\mathbf{y}(t_{k-1},·)].\end{array}$$

(12)

Equations (8) and (11) are motivated by the device of piecewise function interpolation and associated advantages [17].

Filter ${\mathcal{F}}^{(p-1)}$ of the form of Equations (4) and (5) with ${\alpha }_j$ and ${\mathcal{B}}_j$ satisfying Equations (8) and (11) is called the piecewise linear interpolation filter. The pair of signals $\{\mathbf{x}(t_k,·),\mathbf{y}(t_k,·)\}$ associated with time $t_k$ defined by Equation (1) is called the interpolation pair.

Remark 1.

In general, the above procedure might be sensitive to the choice of the initial estimate $\mathbf{x}(t_1,·)$ determined by known methods [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]. If the initial estimate is poor, then the associated error increases.

4. Main Results

4.1. General Device

In accordance with the scheme presented in Section 3.1 and Section 3.4 above, an estimate of the reference signal $\mathbf{x}(t,·)$, for any $t\in T=[a,b]$, by the piecewise linear interpolation filter ${\mathcal{F}}^{(p-1)}$, is given by

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t,·)={\mathcal{F}}^{(p-1)}\left[\mathbf{y}(t,·)\right]=\sum _{j=1}^{p-1}{\delta }_j{\mathcal{F}}_j\left[\mathbf{y}(t,·)\right],\end{array}$$

(13)

where, for each $j=1,\dots ,p-1,$ the sub-filter ${\mathcal{F}}_j$ is given by Equation (5), and is defined from interpolation Equations (8) and (11).

Below, we show how to determine ${\mathcal{F}}_j$ to satisfy the conditions of Equations (8) and (11).

4.2. Filter Model

We write $\mathbf{z}(t_j,t_{j+1},·)={[{\mathbf{z}}^{\left(1\right)}(t_j,t_{j+1},·),\dots ,{\mathbf{z}}^{\left(m\right)}(t_j,t_{j+1},·)]}^T$ and $\mathbf{w}(t_j,t_{j+1},·)={[{\mathbf{w}}^{\left(1\right)}(t_j,t_{j+1},·),\dots ,{\mathbf{w}}^{\left(n\right)}(t_j,t_{j+1},·)]}^T,$ where

$$\begin{array}{c}\hfill \mathbf{z}(t_j,t_{j+1},·)=\mathbf{x}(t_{j+1},·)-\widehat{\mathbf{x}}(t_j,·)\mathrm{and}\mathbf{w}(t_j,t_{j+1},·)=\mathbf{y}(t_{j+1},·)-\mathbf{y}(t_j,·).\end{array}$$

(14)

Here, ${\mathbf{z}}^{\left(j\right)}(t_j,t_{j+1},·)\in L^2(\mathsf{\Omega },\mathbb{R})$ and ${\mathbf{w}}^{\left(i\right)}(t_j,t_{j+1},·)\in L^2(\mathsf{\Omega },\mathbb{R})$, for all $j=1,\dots ,m$, are stochastic variables.

The associated covariance matrix is defined by

$$\begin{array}{c}\hfill E_{z_j{w}_j}={\left\{〈{\mathbf{z}}^{\left(i\right)}(t_j,t_{j+1},·),{\mathbf{w}}^{\left(k\right)}(t_j,t_{j+1},·)〉\right\}}_{i,k=1}^{m,n},\end{array}$$

(15)

where $〈{\mathbf{z}}^{\left(i\right)}(t_j,t_{j+1},·),{\mathbf{w}}^{\left(k\right)}(t_j,t_{j+1},·)〉={\int }_{\mathsf{\Omega }}{\mathbf{z}}^{\left(i\right)}(t_j,t_{j+1},\omega ){\mathbf{w}}^{\left(k\right)}(t_j,t_{j+1},\omega )d\mu \left(\omega \right).$

The Moore–Penrose generalized inverse of a matrix M is denoted by $M^{†}$.

The main results are represented below.

Theorem 1.

Let $K_X=\{\mathbf{x}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^m)|t\in T=[a,b]\}$ and $K_Y=\{\mathbf{y}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^n)|t\in T=[a,b]\}$ be sets of reference signals and observed signals, respectively. Let $t_j\in [a,b]$, for $j=1,\dots ,p$, be such that

$$a=t_1<\dots <t_p=b.$$

For $t=t_1,$ let $\widehat{\mathbf{x}}(t_1,·)$ be a known estimate of $\mathbf{x}(t_1,·)$. As has been mentioned in Section 3.4, $\widehat{\mathbf{x}}(t_1,·)$ can be determined by the known methods. Then, for any $t\in [a,b]$, the proposed piecewise linear interpolation filter ${\mathcal{F}}^{(p-1)}:L^2(\mathsf{\Omega },{\mathbb{R}}^n)\to L^2(\mathsf{\Omega },{\mathbb{R}}^m)$ transforms any signal $\mathbf{y}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^m)$ to an estimate of $\mathbf{x}(t,·)$, $\widehat{\mathbf{x}}(t,·)$, is given by

$$\widehat{\mathbf{x}}(t,·)={\mathcal{F}}^{(p-1)}\left[\mathbf{y}(t,·)\right]=\sum _{j=1}^{p-1}{\delta }_j{\mathcal{F}}_j\left[\mathbf{y}(t,·)\right]$$

(16)

where

$$\begin{array}{c}\hfill {\mathcal{F}}_j\left[\mathbf{y}(t,·)\right]=\widehat{\mathbf{x}}(t_j,·)+{\mathcal{B}}_j[\mathbf{y}(t,·)-\mathbf{y}(t_j,·)],\end{array}$$

(17)

$$\begin{array}{c}\hfill \widehat{\mathbf{x}}(t_j,·)={\mathcal{F}}_{j-1}\left[\mathbf{y}(t_j,·)\right](for j=2,\dots ,p-1),\end{array}$$

(18)

$$\begin{array}{ccc}& & B_j=E_{z_j{w}_j}{E}_{w_j{w}_j}^{†}+M_{B_j}[I_n-E_{w_j{w}_j}{E}_{w_j{w}_j}^{†}],\hfill \end{array}$$

(19)

where $I_n$ is the $n\times n$ identity matrix and $M_{B_j}$ is an $m\times n$ arbitrary matrix.

Proof. 

The proof of Theorem 1 is given in Appendix A.    □

It is worthwhile to observe that, due to an arbitrary matrix $M_{B_j}$ in Equation (19), the filter ${\mathcal{F}}^{(p-1)}$ is not unique. In particular, $M_{B_j}$ can be chosen as the zero matrix $\mathbb{O}$ similarly to the generic optimal linear [5] (which is also not unique for the same reason).

4.3. Numerical Realization of Filter ${\mathcal{F}}^{(p-1)}$ and Associated Algorithm

4.3.1. Numerical Realization

In practice, the set $T=[a,b]$ (see Section 2.1) is represented by a finite set ${\left\{{\tau }_k\right\}}_{k=1}^N$, i.e., $[a,b]=[{\tau }_1,{\tau }_2,\dots ,{\tau }_N]$ where $a\le {\tau }_1<{\tau }_2<\dots <{\tau }_N\le b$.

For $k=1,\dots ,N$, the estimate of $\mathbf{x}({\tau }_k,·)$, $\widehat{\mathbf{x}}({\tau }_k,·)$, and observed signal $\mathbf{y}({\tau }_k,·)$ are represented by $m\times q$ and $n\times q$ matrices

$$\begin{array}{c}\hfill {\widehat{X}}^{\left(k\right)}=[\widehat{\mathbf{x}}({\tau }_k,{\omega }_1),\dots ,\widehat{\mathbf{x}}({\tau }_k,{\omega }_q)]\mathrm{and}{Y}^{\left(k\right)}=[\mathbf{y}({\tau }_k,{\omega }_1),\dots ,\mathbf{y}({\tau }_k,{\omega }_q)].\end{array}$$

(20)

The sequence of fixed time-points ${\left\{t_k\right\}}_1^p\subset [a,b]$ introduced in Equation (1) is such that

$${\tau }_1=t_1<\dots <t_p={\tau }_N,$$

(21)

where $t_1={\tau }_{n_0},t_2={\tau }_{n_0+n_1},\dots ,t_p={\tau }_{n_0+n_1\dots +n_{p-1}},$ and where $n_0=1$ and $n_1,\dots ,n_{p-1}$ are positive integers such that $N=n_0+n_1+\dots +n_{p-1}$.

For $j=1,\dots ,p$, vectors $\widehat{\mathbf{x}}(tj,·)$ and $\mathbf{y}(tj,·)$ associated with $t_j$ in Equation (21) are represented, respectively, by

$${\widehat{X}}_j^{\left(k\right)}=[\widehat{\mathbf{x}}(t_j,{\omega }_1),\dots ,\widehat{\mathbf{x}}(t_j,{\omega }_N)]\mathrm{and}{Y}_j=[\mathbf{y}(t_j,{\omega }_1),\dots ,\mathbf{y}(t_j,{\omega }_N)].$$

4.3.2. Algorithm

As it has been mentioned in Section 3.4, it is supposed that, for $t=t_1,$ an estimate of $X_1$, ${\widehat{X}}_1$, is known and can be determined by the known methods. This is the initial condition of the proposed technique.

On the basis of the results obtained in Section 3.4 and Section 4.2, the performance algorithm of the proposed filter consists of the following steps (Algorithm 1). For $j=1\dots ,p$, we write $N_j=n_0+n_1+\dots +n_{j-1}$.

Algorithm 1 Computation of ${\widehat{X}}^{\left(2\right)}$, ${\widehat{X}}^{\left(3\right)}$, …, ${\widehat{X}}^{\left(N\right)}$ given by Theorem 1

Initial parameters: $Y^{\left(1\right)},\dots ,Y^{\left(N\right)}$, ${\left\{t_j\right\}}_{j=1}^p$ (see Equation (21)), ${\left\{E_{z_j{w}_j}\right\}}_{j=1}^p$, ${\left\{E_{w_j{w}_j}\right\}}_{j=1}^p$ (see Equations (14) and (15)), ${\widehat{X}}_1$, $n_0=1$, $N_0=0$ and $M_{B_j}=\mathbb{O}$, for $j=1,\dots ,p-1$.) The condition $M_{B_j}=\mathbb{O}$, for $j=1,\dots ,p-1$, is motivated by a desire to consider the minimal norm representation of $B_j$, for $j=1,\dots ,p-1$, in Equation (19). Possible ways to obtain estimates of $E_{z_j{w}_j}$ and $E_{w_j{w}_j}$ are discussed below in Section 4.5. Final parameters: ${\widehat{X}}^{\left(2\right)}$, ${\widehat{X}}^{\left(3\right)}$, …, ${\widehat{X}}^{\left(N\right)}$. for $j\leftarrow 1,\dots ,p$ do       $B_j\leftarrow E_{z_j{w}_j}{E}_{w_j{w}_j}^{†}$       for $k\leftarrow N_{j-1}+1,\dots ,N_j$ do             ${\widehat{X}}^{\left(k\right)}\leftarrow {\widehat{X}}_j+B_j(Y^{\left(k\right)}-Y_j)$             ${\widehat{X}}_{j+1}\leftarrow {\widehat{X}}^{\left(N_j\right)}$       end for end for

4.4. Error Analysis

It is natural to expect that the error associated with the piecewise interpolating filter ${\mathcal{F}}^{(p-1)}$ decreases when ${\displaystyle \underset{j=1,\dots ,p-1}{max}\mathsf{\Delta }{t}_j}$ decreases. Below, in Theorem 3, we justify that this observation is true. To this end, first, in the following Theorem 2, we establish an estimate of the error associated with the filter F.

Let us introduce the norm by

$$\begin{array}{c}\hfill {\parallel \mathbf{x}(t,·)\parallel }_{T,\mathsf{\Omega }}^2={\displaystyle \frac{1}{b-a}}{\int }_T{\parallel \mathbf{x}(t,·)\parallel }_{\mathsf{\Omega }}^{2}dt.\end{array}$$

(22)

We also denote ${\parallel \mathbf{x}(t,\omega )\parallel }_{T,\mathsf{\Omega }}^2={\parallel \mathbf{x}(t,·)\parallel }_{T,\mathsf{\Omega }}^2$.

Let us suppose that $\mathbf{x}(·,\omega )$ and $\mathbf{y}(·,\omega )$ are Lipschitz continuous signals, i.e., that there exist real non-negative constants ${\lambda }_j$ and ${\gamma }_j$, with $j=1,\dots ,p$, such that, for $t\in [t_j,t_{j+1}]$,

$$\begin{array}{c}\hfill \parallel \mathbf{x}(t,\omega )-\mathbf{x}(t_j,\omega ){\parallel }_{T,\mathsf{\Omega }}^2\le {\lambda }_j\mathsf{\Delta }{t}_j\mathrm{amd}{\parallel \mathbf{y}(t,\omega )-\mathbf{y}(t_{j+1},\omega )\parallel }_{T,\mathsf{\Omega }}^2\le {\gamma }_j\mathsf{\Delta }{t}_j\end{array}$$

(23)

where $\mathsf{\Delta }{t}_j=|t_{j+1}-t_j|$.

Theorem 2.

Under the conditions (Equation (23)) the error associated with the piecewise interpolation filter, $\parallel \mathbf{x}(t,\omega )-F^{(p-1)}{\left[\mathbf{y}(t,\omega )\right]\parallel }_{T,\mathsf{\Omega }}^2$, is estimated as follows:

$$\begin{array}{ccc}& & \parallel \mathbf{x}(t,\omega )-F^{(p-1)}{\left[\mathbf{y}(t,\omega )\right]\parallel }_{T,\mathsf{\Omega }}^2\hfill \\ & & \le \underset{j=1,\dots ,p-1}{max}\left[({\lambda }_j+{\gamma }_j\parallel B_j{\parallel }^2)\mathsf{\Delta }{t}_j+\parallel E_{z_j{z}_j}^{1/2}{\parallel }^2-{\parallel E_{z_j{w}_j}{\left(E_{w_j{w}_j}^{1/2}\right)}^{†}\parallel }^2\right].\hfill \end{array}$$

(24)

Proof. 

The proof of Theorem 2 is given in Appendix A. □

Further, to show that the error of the reference signal estimate tends to zero, we need to assume that, for $t\in [t_1,t_2]$, the known estimate $\widehat{\mathbf{x}}(t_1,\omega )$ differs from $\mathbf{x}(t,\omega )$ for the value of the order $\mathsf{\Delta }{t}_1$, i.e. that, for some constant $c_1\ge 0$,

$$\begin{array}{c}\hfill \parallel \mathbf{x}(t,\omega )-\widehat{\mathbf{x}}(t_1,\omega ){\parallel }_{\mathsf{\Omega }}^2\le c_1\mathsf{\Delta }{t}_1,\mathrm{for} t\in [t_1,t_2].\end{array}$$

(25)

Theorem 3.

Let the conditions (23) and (25) be true. Then the error associated with the piecewise interpolating filter F, $\parallel \mathbf{x}(t,\omega )-F^{(p-1)}{\left[\mathbf{y}(t,\omega )\right]\parallel }_{T,\mathsf{\Omega }}^2$, decreases in the following sense:

$$\begin{array}{c}\hfill {\displaystyle \parallel \mathbf{x}(t,\omega )-F^{(p-1)}{\left[\mathbf{y}(t,\omega )\right]\parallel }_{T,\mathsf{\Omega }}^2\to 0as\underset{j=1,\dots ,p-1}{max}\mathsf{\Delta }{t}_j\to 0andp\to \infty .}\end{array}$$

(26)

Proof. 

The proof of Theorem 3 is given in Appendix A. □

Remark 2.

We would like to emphasize that the statement of Theorem 3 is fulfilled only under Equations (23) and (25). At the same time, Equations (23) and (25) are not restrictive from a practical point of view. Equation (23) is true for Lipschitz continuous signals $\mathbf{x}$ and $\mathbf{y}$, i.e., for a very wide class of signals. Equation (25) is achieved by choosing an appropriate known method (e.g., see [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]) to find the estimate $\widehat{\mathbf{x}}(t_1,\omega )$ used in the proposed filter ${\mathcal{F}}^{(p-1)}$ (see Equation (8) and Theorem 1).

4.5. Estimation of Covariance Matrices in Equation (19)

Matrix $E_{z_j{w}_j}$ used in Equation (19), for $j=1,\dots ,p$, can be estimated as follows:

• A popular method of estimating $E_{z_j{w}_j}$ is provided, for example, in [27]—it is based on the use of samples of $z_j$ and $w_j$, for $j=1,\dots ,p$.
• In the case of incomplete observations, the method proposed in [28,29] can be used.
• Let $E_{{\widehat{z}}_j{w}_j}$ be a matrix obtained from matrix $E_{z_j{w}_j}$ where the term $\mathbf{x}(t_{j+1},·)$ is replaced by $\widehat{\mathbf{x}}(t,·)$ with $t\in [t_{j-1},t_j]$. Since $\widehat{\mathbf{x}}(t,·)$ with $t\in [t_{j-1},t_j]$ is known, matrix $E_{{\widehat{z}}_j{w}_j}$ can be considered as an estimate of $E_{z_j{w}_j}$.
• In the important case of an additive noise, $E_{z_j{w}_j}$ can be represented in the explicit form. Indeed, if

  $$\mathbf{y}(t,·)=\mathbf{x}(t,·)+\xi (t,·)$$

  where $\xi (t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^m)$ is a random noise, then $\mathbf{z}(t_j,t_{j+1},·)=\mathbf{y}(t_{j+1},·)-\xi (t_{j+1},·)-\widehat{\mathbf{x}}(t_j,·)$ and matrix $E_{z_j{w}_j}$ can be represented as follows:

  $$\begin{array}{c}\hfill E_{z_j{w}_j}=E_{(y_{j+1}-{\xi }_{j+1})(y_{j+1}-y_j)}-E_{{\widehat{x}}_j(y_{j+1}-y_j)}\end{array}$$

  (27)

We note that the RHS of Equation (27) depends only on observed signals $\mathbf{y}(t_j,·)$, $\mathbf{y}(t_{j+1},·)$, estimated signal $\widehat{\mathbf{x}}(t_j,·)$, and noise $\xi (t_{j+1},·)$, not on the reference signal $\mathbf{x}(t_{j+1},·)$. In particular, in Equation (27), the term $E_{{\xi }_{j+1}(y_{j+1}-y_j)}$ can be estimated as $\pm {\left(E\left[{\xi }_{j+1}^2\right]\right)}^{1/2}{\left(E\left[{(y_{j+1}-y_j)}^2\right]\right)}^{1/2}$ where ${\displaystyle E\left[{\xi }_{j+1}^2\right]={\int }_{\mathsf{\Omega }}{\left[\xi (t_{j+1},\omega )\right]}^{2}d\mu \left(\omega \right)}$. It is motivated by the Holder’s inequality for integrals. The second term in Equation (27), $E_{{\widehat{x}}_j(y_{j+1}-y_j)}$, can be estimated from the samples of $\widehat{\mathbf{x}}(t_{j+1},·)$ and $\mathbf{y}(t_{j+1},·)-\mathbf{y}(t_j,·)$.

We also note that the first term in the RHS of Equation (27), $E_{(y_{j+1}-{\xi }_{j+1})(y_{j+1}-y_j)}$, is similar to the related covariance matrix in the Wiener filtering approach [5].

5.

Other known ways to estimate $E_{{\xi }_{j+1}(y_{j+1}-y_j)}$ can be found in [5], Section 5.3.

In general, an estimation of covariance matrices is a special research topic that is not a subject of this paper. The relevant references can be found, for example, in [5,29].

5. Simulations

5.1. General Consideration

The simulations represented below have been performed by Matlab R14 (Version 7.0). In accordance with Section 4.3.1, signal sets $K_X$ and $K_Y$ (see Section 2.1) are given by

$$K_X=\{\mathbf{x}({\tau }_1,·),\mathbf{x}({\tau }_2,·),\dots ,\mathbf{x}({\tau }_N,·)\}\mathrm{and}{K}_Y=\{\mathbf{y}({\tau }_1,·),\mathbf{y}({\tau }_2,·,\dots ,\mathbf{y}({\tau }_N,·)\},$$

where, for $k=1,\dots ,N$, $\mathbf{x}({\tau }_k,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^m)$ and $\mathbf{y}({\tau }_k,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^n)$. In many practical problems (arising, for example, in a DNA analysis the number N is quite large; for instance, $N=\mathcal{O}\left({10}^4\right)$.

We set $N=141$ and $m=n=116$. Thus, in these simulations, the interval $T=[a,b]$ (see Section 2.1 and Section 4.3.1) is modeled as 141 points ${\tau }_k$ with $k=1,\dots ,141$ so that $[a,b]=[{\tau }_1,{\tau }_2,\dots ,{\tau }_{141}]$.

The sequence of fixed time-points ${\left\{t_k\right\}}_1^p\subset T$ in Equation (1) is now such that

$${\tau }_1=t_1<\dots <t_p={\tau }_{141}.$$

(28)

Below, in Examples 1–12, four particular choices of the specific interpolation signal pairs $\{\mathbf{x}(t_j,·),$$\mathbf{y}(t_j,·){\}}_1^p$ (introduced in Section 3.4) are considered, for $p=5,8,15$ and 28. Points $t_1,\dots ,t_p$ are as follows.

For $p=5,8,15$, if $j=1,\dots ,p$, then $t_j=t_j^{\left(p\right)}={\tau }_1+(j-1){\mathsf{\Delta }}_p$, respectively, where ${\mathsf{\Delta }}_5=35,$ ${\mathsf{\Delta }}_8=20$ and ${\mathsf{\Delta }}_{15}=10$.

For $p=28$, if $j=1,\dots ,p-1$, then $t_j=t_j^{\left(p\right)}={\tau }_1+(j-1){\mathsf{\Delta }}_{28}$, and if $j=p$, then $t_{28}=t_{28}^{\left(28\right)}=t_{27}+6=141$, where ${\mathsf{\Delta }}_{28}=5$.

Signals $\mathbf{x}({\tau }_k,·)$ and $\mathbf{y}({\tau }_k,·)$ have been simulated as digital images represented by $116\times 256$ matrices

$$\begin{array}{c}\hfill X^{\left(k\right)}=[\mathbf{x}({\tau }_k,{\omega }_1),\dots ,\mathbf{x}({\tau }_k,{\omega }_{256})]\text{and}{Y}^{\left(k\right)}=[\mathbf{y}({\tau }_k,{\omega }_1),\dots ,\mathbf{y}({\tau }_k,{\omega }_{256})],\end{array}$$

(29)

respectively, for $k=1,\dots ,141$, so that $X^{\left(k\right)}$ represents an image that should be estimated from an observed image $Y^{\left(k\right)}$. A column of matrices $X^{\left(k\right)}$ and $Y^{\left(k\right)}$, $\mathbf{x}({\tau }_k,{\omega }_i)\in {\mathbb{R}}^{116}$ and $\mathbf{y}({\tau }_k,{\omega }_i)\in {\mathbb{R}}^{116}$, for $i=1,\dots ,256$, represents a realization of signals $\mathbf{x}({\tau }_k,·)$ and $\mathbf{y}({\tau }_k,·)$, respectively.

Note that $X^{\left(1\right)},\dots ,X^{\left(141\right)}$ did not used in the piecewise linear filter $F^{(p-1)}$ below since they are not supposed to be known. They are represented here for illustration purposes only. In particular, $X^{\left(1\right)},\dots ,X^{\left(141\right)}$ are used to compare their estimates by different filters.

Observed noisy signals $Y^{\left(1\right)},\dots ,Y^{\left(141\right)}$ have been simulated in different forms presented by Equations (40) and (49)–(51) in the Examples 1–12 below. We note that the considered observed signals are grossly corrupted.

To estimate the signals $X^{\left(1\right)}$, …, $X^{\left(141\right)}$ from the observed signals $Y^{\left(1\right)}$, …, $Y^{\left(141\right)}$, the proposed piecewise linear filter $F^{(p-1)}$, the generic optimal linear (GOL) filters [5] and the averaging polynomial filter [16] have been used.

The filters proposed in [11,12,13,16] have not been applied here by the reasons discussed in Section 1. In particular, the filter in [13] cannot be applied to signals represented by $Y^{\left(1\right)}$, …, $Y^{\left(141\right)}$ in the form Equations (40), (49)–(51) below because the associated inverse matrices used in [13] do not exist.

For signals under consideration (given by matrices $X^{\left(k\right)}$ and $Y^{\left(k\right)}$ with $k=1,\dots ,141$), the filter $F^{(p-1)}$, the generic optimal linear (GOL) filters [5] and the averaging polynomial filter [15,16] are represented as follows:

(i)

Piecewise linear filter $F^{(p-1)}$. For $j=1,\dots ,p$, $\{X_j,Y_j\}$ designates an interpolation pair defined similarly to that in Section 3.4. Each $X_j$ and $Y_j$ is associated with $t_j$ in Equation (28) so that

$$X_j=[\mathbf{x}(t_j,{\omega }_1),\dots ,\mathbf{x}(t_j,{\omega }_{256})]\mathrm{and}{Y}_j=[\mathbf{y}(t_j,{\omega }_1),\dots ,\mathbf{y}(t_j,{\omega }_{256})].$$

The estimate ${\widehat{X}}^{\left(k\right)}$ of $X^{\left(k\right)}$ by the filter $F^{(p-1)}$ is given by

$${\widehat{X}}^{\left(k\right)}=F^{(p-1)}\left[Y^{\left(k\right)}\right],$$

(30)

where Equations (16)–(19) are presented in Section 4.2,

$$\begin{array}{ccc}& & F^{(p-1)}\left[Y^{\left(k\right)}\right]=\sum _{j=1}^{p-1}{\delta }_j{F}_j\left[Y^{\left(k\right)}\right],{\delta }_j=\left\{\begin{array}{cc}1,& if t_j\le {\tau }_k\le t_{j+1},\hfill \\ 0,& \mathrm{otherwise},\hfill \end{array}\right.\hfill \end{array}$$

(31)

$$\begin{array}{ccc}& & F_j^{(p-1)}\left[Y^{\left(k\right)}\right]={\widehat{X}}_j+B_j[Y^{\left(k\right)}-Y_j],\hfill \end{array}$$

(32)

$$\begin{array}{ccc}& & {\widehat{X}}_j=F_{j-1}\left[Y_j\right],{\widehat{X}}_1\mathrm{is} \mathrm{given},\hfill \end{array}$$

(33)

$$B_j=E_{ZjWj}{\left(E_{WjWj}\right)}^{†},$$

(34)

and where $E_{ZjWj}$ and $E_{WjWj}$ are estimates of matrices $E_{z_j{w}_j}$ and $E_{w_j{w}_j}$ in Equation (19), respectively. In particular, $E_{W_j{W}_j}$ can be represented in the form

$$E_{WjWj}=W_j{W}_j^T,\mathrm{where} W_j=Y_{j+1}-Y_j.$$

(35)

Further, matrix $E_{ZjWj}$ depends on $Z_j=X_{j+1}-{\widehat{X}}_j$ where $X_{j+1}$ is unknown. Therefore, a determination of $E_{ZjWj}$ is reduced, in fact, to finding an estimate of $X_{j+1}$. Since it is customary to find $E_{ZjWj}$ in terms of signal samples [5], $E_{ZjWj}$ has been presented as

$$E_{ZjWj}={\tilde{Z}}_j{W}_j^T,\mathrm{where} {\tilde{Z}}_j={\tilde{X}}_{j+1}-{\widehat{X}}_j$$

(36)

and ${\tilde{X}}_{j+1}$ has been constructed from a sample of $X_{j+1}$ as follows. The sample of $X_{j+1}$ is a $116\times 128$ matrix presented by odd columns of $X_{j+1}$. Then an estimate of $X_{j+1}$ is chosen as a $116\times 256$ matrix ${\tilde{X}}_{j+1}$ where each odd column is a related odd column of $X_{j+1}$, and each even column is an average of two adjacent columns. The last column in ${\tilde{X}}_{j+1}$ is the same as its preceding column.

This way of estimating $E_{ZjWj}$ was chosen for illustration purposes only. Other related methods have been considered in Section 4.5.

The errors associated with the filter $F^{(p-1)}$ are given by

$${\epsilon }_{k,F}^{(p-1)}={∥X^{\left(k\right)}-F^{(p-1)}\left[Y^{\left(k\right)}\right]∥}_F^2,\mathrm{for} k=1,\dots ,141.$$

(37)

(ii)

Generic optimal linear (GOL) filters [5]. To each signal $Y^{\left(k\right)}$, an individual GOL filter $W_k$ has also been applied, so that $W_k$ estimates $X^{\left(k\right)}$ from $Y^{\left(k\right)}$ in the form

$$W_k{Y}^{\left(k\right)}=E_{X^{\left(k\right)}{Y}^{\left(k\right)}}{E}_{Y^{\left(k\right)}{Y}^{\left(k\right)}}^{†}{Y}^{\left(k\right)},$$

for each $k=1,\dots ,141$. Thus, the GOL filter $W_k$ requires an estimate of 141 matrices $E_{X^{\left(k\right)}{Y}^{\left(k\right)}}$, for each $k=1,\dots ,141$.

Similarly to matrix $E_{Z_j{W}_j}$ in the filter $F^{(p-1)}$ above, the matrix $E_{X^{\left(k\right)}{Y}^{\left(k\right)}}$ has been estimated from samples of each $X^{\left(k\right)}$, ${\tilde{X}}^{\left(k\right)}$, for each $k=1,\dots ,141$.

One of the advantages of the proposed filter $F^{(p-1)}$ is that $F^{(p-1)}$ requires a smaller number, p, of samples of $X_j$, ${\tilde{X}}_j$, to be known (where $j=1,\dots ,p$).

The errors associated with filters $W_k$ are given by

$${\displaystyle {ϵ}_{k,w}={\parallel X^{\left(k\right)}-W_k{Y}^{\left(k\right)}\parallel }_F^2.}$$

(38)

(iii)

Averaging polynomial filters [15,16]. By the methodology in [15], the averaging polynomial filter W is based on the use of the estimates of the covariance matrices, $E_{XY}$ and $E_{YY}$, in the form

$$E_{XY}={\displaystyle \frac{1}{141}}\sum _{k=1}^{141}{\tilde{X}}^{\left(k\right)}{\left(Y^{\left(k\right)}\right)}^T\mathrm{and}{E}_{YY}={\displaystyle \frac{1}{141}}\sum _{k=1}^{141}{Y}^{\left(k\right)}{\left(Y^{\left(k\right)}\right)}^T.$$

Then, for each, $k=1,\dots ,141$, the estimate of $X^{\left(k\right)}$ is given by

$$W{Y}^{\left(k\right)}=E_{XY}{E}_{YY}^{†}{Y}^{\left(k\right)}.$$

The errors associated with the filter W are given by

$${\displaystyle {\epsilon }_{k_W}={\parallel X^{\left(k\right)}-W{Y}^{\left(k\right)}\parallel }_F^2,\mathrm{for} k=1,\dots ,141.}$$

(39)

5.2. Simulations with Signals Modelled from Images ‘Plant’: Application of Piecewise Interpolation Filter and GOL Filters

Here, results of simulations for reference signals represented by matrices $X^{\left(1\right)},$ $\dots ,$$X^{\left(141\right)}$ (see Equation (29) above) formed from images ‘plant’ from http://sipi.usc.edu/services/database.html (accessed on 21 February 2025). Typical selected images $X^{\left(k\right)}$ are shown in Figure 1.

Observed noisy images $Y^{\left(1\right)},\dots ,Y^{\left(141\right)}$ have been simulated in the form

$$Y^{\left(k\right)}=X^{\left(k\right)}•{\mathtt{randn}}_{\left(k\right)}•{\mathtt{rand}}_{\left(k\right)},$$

(40)

for each $k=1,\dots ,141.$ Here, • means the Hadamard product, and ${\mathtt{randn}}_{\left(k\right)}$ and ${\mathtt{rand}}_{\left(k\right)}$ are $116\times 256$ matrices with random entries. The entries of ${\mathtt{randn}}_{\left(k\right)}$ are normally distributed with mean zero, variance one and standard deviation one. The entries of ${\mathtt{rand}}_{\left(k\right)}$ are uniformly distributed in the interval $(0,1)$. A typical example of such images is given in Figure 2a. Examples of signal $X^{\left(95\right)}$ estimates by different filters are represented in Figure 2b–d.

To demonstrate the effectiveness of the proposed filter $F^{(p-1)}$, sub-filters $F_j^{(p-1)}$ and associated interpolation signal pairs ${\{X_j,Y_j\}}_{j=1}^p$ have been chosen in four different ways as follows.

Example 1.

First, for $p=5$, the interpolation signal pairs are

$$\begin{array}{ccc}& & \{X_1,Y_1\}=\{X^{\left(1\right)},Y^{\left(1\right)}\},\{X_2,Y_2\}=\{X^{\left(35\right)},Y^{\left(35\right)}\},\{X_3,Y_3\}=\{X^{\left(70\right)},Y^{\left(70\right)}\},\hfill \end{array}$$

(41)

$$\begin{array}{ccc}& & \{X_4,Y_4\}=\{X^{\left(105\right)},Y^{\left(105\right)}\},\{X_5,Y_5\}=\{X^{\left(141\right)},Y^{\left(141\right)}\}.\hfill \end{array}$$

(42)

The error values ${\left\{{\epsilon }_{k,F}^{\left(4\right)}\right\}}_1^{141}$ associated with filter $F^{\left(4\right)}$ are evaluated by Equation (37). The graph of ${\left\{{\epsilon }_{k,F}^{\left(4\right)}\right\}}_1^{141}$ is presented in Figure 3a.

Example 2.

For $p=8$, the interpolation signal pairs are

$$\begin{array}{ccc}& & \{X_1,Y_1\}=\{X^{\left(1\right)},Y^{\left(1\right)}\},\{X_j,Y_j\}=\{X^{\left(20\right(j-1\left)\right)},Y^{\left(20\right(j-1\left)\right)}\},for j=2,\dots ,7;\hfill \end{array}$$

(43)

$$\begin{array}{ccc}& & and\{X_8,Y_8\}=\{X^{\left(141\right)},Y^{\left(141\right)}\}.\hfill \end{array}$$

(44)

The error magnitudes ${\left\{{\epsilon }_{k,F}^{\left(7\right)}\right\}}_1^{141}$ associated with the piecewise interpolation filter $F^{\left(7\right)}$ constructed by Equations (31)–(36) with the interpolation signal pairs given by Equations (43) and (44) are diagrammatically shown in Figure 3b.

It follows from Figure 3b that the errors associated with filter $F^{\left(7\right)}$ is less than those of filter $F^{\left(4\right)}$. This is a confirmation of Theorem 3.

Example 3.

Further, for $p=15$, the interpolation pairs are

$$\begin{array}{ccc}& & \{X_1,Y_1\}=\{X^{\left(1\right)},Y^{\left(1\right)}\},\{X_j,Y_j\}=\{X^{\left(10\right(j-1\left)\right)},Y^{\left(10\right(j-1\left)\right)}\}for j=2,\dots ,14;\hfill \end{array}$$

(45)

$$\begin{array}{ccc}& & and\{X_{15},Y_{15}\}=\{X^{\left(141\right)},Y^{\left(141\right)}\}.\hfill \end{array}$$

(46)

In Figure 3c, the errors ${\left\{{\epsilon }_{k,F}^{\left(15\right)}\right\}}_1^{141}$ associated with the piecewise interpolation filter $F^{\left(15\right)}$ are presented. Figure 3c demonstrates a further confirmation of Theorem 3: the errors associated with the piecewise interpolation filter diminish as p increases.

Example 4.

Finally, the number of interpolation signal pairs ${\{X_j,Y_j\}}_{j=1}^p$ is $p=29$ so that

$$\begin{array}{ccc}& & \{X_1,Y_1\}=\{X^{\left(1\right)},Y^{\left(1\right)}\},\{X_j,Y_j\}=\{X^{\left(5\right(j-1\left)\right)},Y^{\left(5\right(j-1\left)\right)}\}for j=2,\dots ,28;\hfill \end{array}$$

(47)

$$\begin{array}{ccc}& & and\{X_{29},Y_{29}\}=\{X^{\left(141\right)},Y^{\left(141\right)}\}.\hfill \end{array}$$

(48)

In this case, when p is greater than in the previous Examples 1–3, the errors ${\left\{{\epsilon }_{k,F}^{\left(29\right)}\right\}}_1^{141}$ associated with the piecewise interpolation filter $F^{\left(29\right)}$ are smaller than those associated with filters $F^{\left(4\right)}$, $F^{\left(8\right)}$ and $F^{\left(15\right)}$—see Figure 3d.

The diagrams of errors associated with the GOL filters [5] are also presented in Figure 3, which shows that the proposed filters $F^{\left(4\right)}$, $F^{\left(8\right)}$, $F^{\left(15\right)}$ and $F^{\left(29\right)}$ provide better accuracy then that of the GOL filters.

At the same time, the filter $F^{(p-1)}$ is easier to implement since it requires less initial information compared to GOL filters, as it has been discussed in Section 1.1.1 and Section 1.1.4.

Results of the application of the averaging polynomial filter [15,16] are discussed in Section 5.4 below.

5.3. Simulations with Signals Modelled from Images ‘Boat’: Application of Piecewise Interpolation Filter and GOL Filters

In this section, results of the simulations for a different type of signal than those considered in Section 5.2 above are presented. Here, the reference signals $X^{\left(1\right)},$$\dots ,$$X^{\left(141\right)}$ are formed from images ‘boat’ (http://sipi.usc.edu/services/database.html, accessed 21 February 2025).

Observed noisy signals $Y^{\left(1\right)},\dots ,Y^{\left(141\right)}$ have been simulated in the form

$$Y^{\left(k\right)}=X^{\left(k\right)}•{\mathtt{randn}}_{\left(k\right)},$$

(49)

for each $k=1,\dots ,141.$ The noise term is different from that in Equation (40).

Typical selected images $X^{\left(k\right)}$ and $Y^{\left(k\right)}$ are shown in Figure 4 and Figure 5, respectively.

As in Section 5.2, the piecewise interpolation filter $F^{(p-1)}$ is constructed by Equations (31)–(36). In Examples 5–8 below, the number $p-1$ of sub-filters $F_j^{(p-1)}$ and associated interpolation signal pairs ${\{X_j,Y_j\}}_{j=1}^p$ have been chosen in four different ways.

Example 5.

First, similar to Example 1, the number of interpolation signal pairs ${\{X_j,Y_j\}}_{j=1}^p$ has been chosen as $p=5$, and $X_j$ and $Y_j$ have been presented as in Equations (41) and (42).

The error values ${\left\{{\epsilon }_{k,F}^{\left(5\right)}\right\}}_1^{141}$ associated with the piecewise interpolation filter $F^{\left(5\right)}$ applied to these data are presented in Figure 6a.

Example 6.

For the greater number of interpolation signal pairs than that in Example 5, $p=8$, and for $X_j$ and $Y_j$ ($j=1,\dots ,8$) chosen as in Equations (43) and (44), the error magnitudes ${\left\{{\epsilon }_{k,F}^{\left(8\right)}\right\}}_1^{141}$ associated with the piecewise interpolation filter $F^{\left(8\right)}$ are diagrammatically shown in Figure 6b. A comparison between Figure 6a,b demonstrates that the increase in p implies the decrease in the errors associated with the filter $F^{(p-1)}$.

Example 7.

For $p=15$, and for $X_j$ and $Y_j$ ($j=1,\dots ,15$) chosen in Equations (45) and (46), the errors ${\left\{{\epsilon }_{k,F}^{\left(15\right)}\right\}}_1^{141}$ associated with the piecewise interpolation filter $F^{\left(15\right)}$ are further less than those for filters $F^{\left(5\right)}$ and $F^{\left(8\right)}$. See Figure 6c in this regard.

Example 8.

The further increase in p to $p=29$ confirms this tendency. The piecewise interpolation filter $F^{\left(29\right)}$ with $X_j$ and $Y_j$ ($j=1,\dots ,29$) chosen similar to Equations (47) and (48) produces the associated errors ${\left\{{\epsilon }_{k,F}^{\left(29\right)}\right\}}_1^{141}$ represented in Figure 6d. They are, clearly, less than the errors associated with filters $F^{\left(5\right)}$, $F^{\left(8\right)}$ and $F^{\left(15\right)}$.

The errors associated with the GOL filters are also presented in Figure 6a–d. The figures clearly demonstrate the advantage of the piecewise interpolation filter $F^{(p-1)}$.

Results of the application of the averaging polynomial filter [16] are discussed in Section 5.4 below.

5.4. Results of Simulations for Averaging Polynomial Filter [15,16]

To further illustrate the effectiveness of the proposed piecewise interpolation filter, in this Section, results of simulations for the averaging polynomial filter [15,16] are presented. The filter has been applied to two different types of data considered in Section 5.2 and Section 5.3.

Example 9.

The filter [15,16] applied to signals considered in Section 5.2 gives the associated errors ${\left\{{ϵ}_{k_W}\right\}}_{k=1}^{141}$ (see Equation (39)) represented in Figure 7a. For a comparison, the errors associated with the piecewise interpolation filter $F^{\left(29\right)}$ and the GOL filters [5] are also given in Figure 7a.

A typical example of the estimated signal by the averaging polynomial filter [15,16] is presented in Figure 2d above.

Example 10.

The averaging polynomial filter [16] applied to signals considered in Section 5.3 produces the associated errors ${\left\{{ϵ}_{k_W}\right\}}_{k=1}^{141}$ shown in Figure 7b. The errors associated with the piecewise interpolation filter $F^{\left(29\right)}$ are much smaller, and they are not discerned in Figure 7b.

The extreme failure of the averaging polynomial filter (the errors of magnitude ${10}^9$) in Examples 9 and 10 illustrates the observation made in Section 1.2.4: The associated error increases when signals differ considerably from each other. This is the case in Examples 9 and 10.

Together, Figure 2, Figure 3, Figure 5, Figure 6 and Figure 7a,b illustrate the advantage of the piecewise interpolation filter.

5.5. Further Simulations with Different Type of Noise

In Examples 11 and 12 below, a different type of noise is considered. Unlike the multiplicative noise in Equations (40) and (49), here, the noise is additive.

Example 11.

First, the piecewise interpolation filter $F^{\left(28\right)}$, the GOL filters [5] and the averaging polynomial filter [16] have been applied to the observed signals given by

$$Y^{\left(k\right)}=X^{\left(k\right)}+900\times {\mathtt{randn}}_{\left(k\right)},for k=1,\dots ,141.$$

(50)

where $X^{\left(k\right)}$ is as in Section 5.2, i.e., $X^{\left(k\right)}$ is formed from the images ’plant’. In Figure 8a, the diagrams of the errors associated with filter $F^{\left(28\right)}$ and the GOL filters [5] are given. The errors associated with the averaging polynomial filter [16], ${\left\{{ϵ}_{k_W}\right\}}_{k=1}^{141}$, are much greater (of order $\mathcal{O}\left({10}^9\right)$) and they are not presented in Figure 8a.

Example 12.

In this example, the reference signals $X^{\left(1\right)},\dots ,X^{\left(141\right)}$ are as those in Section 5.3, i.e., they are formed from the image ’boat’. The observed signals are given by

$$Y^{\left(k\right)}=X^{\left(k\right)}+1000\times {\mathtt{randn}}_{\left(k\right)},for k=1,\dots ,141.$$

(51)

The piecewise interpolation filter $F^{\left(29\right)}$ and the GOL filters [5] estimate the reference signals with the associated errors represented in Figure 8b. As in Example 11 above, in this case, the errors associated with the averaging polynomial filter [15,16] are much greater (of order $\mathcal{O}\left({10}^{10}\right)$), and they are not presented in Figure 8b.

Examples 11 and 12 further demonstrate the advantages of the proposed piecewise interpolation filter.

5.6. Summary of Simulations

The above simulations confirm the theoretical results obtained in Theorems 1–3. In particular, Figure 3 and Figure 6 demonstrate that the error associated with the piecewise interpolation filter $F^{(p-1)}$ decreases when the number of sub-filters $F_1,\dots ,F_p$, p, increases.

A comparison between the proposed filter $F^{(p-1)}$ and the known related filters [5,12,13,15,16] has been performed. The filter $F^{(p-1)}$ estimates the reference signals with the accuracies that are much better than those of the generic optimal linear (GOL) filters [5] and the averaging polynomial filter [15,16]. Further, the filters proposed in [12,13] fail in processing the signals under consideration. This is because the observed signals in Equations (40), (49), (50) and (51) are grossly corrupted, and, therefore, the inverse matrices used in the filter structures in [13] do not exist. The technique in [12] requires the use of the reference signal in the proposed filter, which is supposed to be unknown in the simulations above.

The filters have been applied to the different signal sets (presented in Section 5.2 and Section 5.3), using different forms of noise (given in Equations (40), (49), (50) and (51)).

Computational work associated with the proposed filter $F^{(p-1)}$ is substantially less than that associated with the known filters discussed in Section 1 (in particular, with the filters in [10,11,12,13,16,17,20,21,22,23,24,25,26]). This is because, for the processing of a data set containing N signals, filter $F^{(p-1)}$ requires computation of p covariance matrices with $p\ll N$ while the known filters require computation of N matrices (in the above examples, $p=5,7,15,28$, respectively, and $N=141$).

Table 1. Time in seconds used by PILF, GOL and APF in the above examples.

Initial Parameters	Time in Seconds
	PILF	COL	APF
‘Plant’, $p=8$, $N=256$	0.37	3.72	1.25
‘Boat’, $p=8$, $N=256$,	0.36	3.81	1.17
‘Plant’, $p=15$, $N=256$	0.70	3.72	1.25
‘Boat’, $p=15$, $N=256$,	0.67	3.81	1.17
‘Plant’, $p=29$, $N=256$	1.23	3.72	1.25
‘Boat’, $p=29$, $N=256$,	1.18	3.81	1.17

represents a typical time in seconds used by the piecewise linear interpolation filter (PILF), GOL and the averaging polynomial filter (APF) [16] in the above examples. It follows from Table 1 that the PILF is faster than the GOL up to ten times. While the time used by the PILF and APF for $p=29$ is almost the same, the accuracy associated with the PILF is much better than that associated with the APF.

Table 2. Summary of computational and structural advantages of the proposed method.

	PILF	COL	APF	[10,11,12,13,16,17,20,21,22,23,24,25,26]
PINV	Yes	Yes	Yes	No
Error	${\epsilon }_{PILF}$	${\epsilon }_{GOL}\ge {\epsilon }_{PILF}$	${\epsilon }_{APF}\ge {\epsilon }_{PILF}$	N/A
Comp. load	p by $n\times n$ PINV, $p\ll N$	N by $n\times n$ PINV, $N\gg p$	one $n\times n$ PINV	N/A
Stability	Yes	Yes	No	No

summarizes the computational and structural advantages of the proposed method. The abbreviation ‘PINV’ means the use of the pseudo-inverse matrices (not inverse matrices), and ${\epsilon }_{PILF}$, ${\epsilon }_{GOL}$ and ${\epsilon }_{APF}$ denote the errors associated with the PILF, GOL and APF, correspondingly. ‘Stability’ means numerical stability. The last column illustrates features related to filters considered in [10,11,12,13,16,17,20,21,22,23,24,25,26]. As mentioned above, these filters cannot be applied to the data under consideration because of the singularity of matrices used in [10,11,12,13,16,17,20,21,22,23,24,25,26]. Therefore, their errors and a computational complexity are marked by N/A as not applicable.

6. Conclusions

The technique of the constructive approximation of a nonlinear operator $\mathcal{F}:K_Y\to K_X$ has been provided. Here, $K_X=\{\mathbf{x}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^m)|t\in T\}$ and $K_Y=\{\mathbf{y}(t,·)\in L^2(\mathsf{\Omega },{\mathbb{R}}^n)|t\in T\}$ where $T:=[a,b]\subseteq \mathbb{R}$ and $\mathsf{\Omega }=\left\{\omega \right\}$ is the set of outcomes of a probability space. The device behind the proposed method is based on a special extension of the piecewise linear interpolation technique to the case of stochastic sets $K_X$ and $K_Y$. The proposed methodology is motivated by the problems arising in signal processing, where the nonlinear operator $\mathcal{F}$ is interpreted as a nonlinear system (or a nonlinear filter) transforming a set of stochastic signals $K_Y$ to the set of stochastic signals $K_X$. Therefore, the provided technique provides an effective way to transform large data sets.

Distinctive features of the approach are as follows:

(i)

The proposed filter ${\mathcal{F}}^{(p-1)}:K_Y\to K_X$ is nonlinear and is presented in the form of a sum with $p-1$ terms, where each term, ${\mathcal{F}}_j:K_{Y,j}\to K_{X,j}$, is interpreted as a particular sub-filter. Here, $K_{Y,j}$ and $K_{X,j}$ are ‘small’ pieces of $K_Y$ and $K_X$, respectively.

(ii)

The prime idea is to exploit prior information only on a few reference signals, p, from the set $K_X$ that contains $N\gg p$ signals (or even an infinite number of signals) and determine ${\mathcal{F}}_j$ separately for each piece $K_{Y,j}$ and $K_{X,j}$, so that the associated error is minimal. In other words, the filter ${\mathcal{F}}^{(p-1)}$ is flexible to changes in the sets of observed and reference signals $K_Y$ and $K_X$, respectively.

(iii)

Due to the specific way of determining ${\mathcal{F}}_j$, the filter ${\mathcal{F}}^{(p-1)}$ provides a smaller associated error than that for the processing of the whole set $K_Y$ by a filter that is not specifically adjusted to each particular piece $K_{Y,j}$. Moreover, the error associated with our filter decreases when the number of its terms, ${\mathcal{F}}_1,\dots ,{\mathcal{F}}_{p-1}$, increases.

(iv)

While the proposed filter ${\mathcal{F}}^{(p-1)}$ processes arbitrarily large (and even infinite) signal sets, the filter is nevertheless fixed for all signals in the sets.

(v)

The filter ${\mathcal{F}}^{(p-1)}$ is determined in terms of pseudo-inverse matrices so that the filter always exists.

(vi)

computational load associated with the filter ${\mathcal{F}}^{(p-1)}$ is less than that associated with other known filters applied to the processing of large signal sets.