Abstract
Suppose and are the image and the preimage of a nonlinear operator . It is supposed that the cardinality of each and is N and N is large. We provide an approximation to the map that requires prior information only on a few elements p from , where , but still effectively represents . It is achieved under Lipschitz continuity assumptions. The device behind the proposed method is based on a special extension of the piecewise linear interpolation technique to the case of sets of stochastic elements. The proposed technique provides a single operator that transforms any element from the arbitrarily large set . The operator is determined in terms of pseudo-inverse matrices so that it always exists.
MSC:
15A99; 65F05
1. Introduction
A purpose of the proposed methodology is to provide an effective way to transform large data sets. The methodology is motivated by the problems arising in signal processing, where the nonlinear operator is interpreted as a nonlinear system (or a nonlinear filter) transforming a set of stochastic signals to the set of stochastic signals . Therefore, below we refer to this terminology.
The device behind the proposed method is based on a special extension of the piecewise linear interpolation technique to the case of stochastic signal sets. The device is not straightforward and requires the careful substantiation presented in Section 2.2, Section 3.4, Section 4.2 and Section 4.4 below.
1.1. Motivations
1.1.1. Filtering of Large Arrays of Stochastic Signals
We consider the case when the cardinality N of signal sets and is large. Members of and are represented by stochastic vectors. The problem of finding a filter that effectively transforms the large set to the large set has been considered in a number of works such as those represented in [1,2,3,4,5,6,7,8,9,10,11,12,13,14]. The approaches considered in these references require prior information on each reference signal. This requirement poses a significant difficulty on the applicability of the methods mentioned above. For processing of large signal arrays, such restrictions are quite inconvenient. The exception is the techniques provided in [15,16], which exploit the initial information in the form of the vector derived from averaging over the reference signal set.
In this paper, we show how to construct a filter that requires prior information only on few signals, , from but still performs effectively compared to known filters based on prior information on every reference signal from . Such a filter is denoted by .
1.1.2. Basic Idea of the Proposed Methodology
The specific feature of the proposed filter is an adjustment and extension of the piecewise function interpolation technique [17] to filtering of stochastic signals. It is well known that the piecewise function interpolation [17] often provides better accuracy and faster performance than those associated with the linear and polynomial approximation used in known filtering techniques (such as, for example, those in [5,9]).
1.1.3. Pseudo-Inverse Matrices and Related Matters
The known filter models proposed in [1,2,3,6,7,8,10,13,14], use inverse matrices. In the cases of grossly corrupted signals or erroneous measurements, those inverse matrices may not exist, and thus, those filters cannot be applied. The examples in Section 5 illustrate this case.
The filter proposed here avoids this drawback since its model is based on exploiting pseudo-inverse matrices. As a result, the proposed filter always exists. That is, it processes any kind of noisy signals. Exploiting pseudo-inverse matrices is not straightforward and requires the development of a specific technique. Perhaps it was a reason to not use the pseudo-inverse matrices in the above-mentioned references. Here, an extension of the proposed filtering techniques to the case of implementation of the pseudo-inverse matrices is performed on the basis of theory represented in [5].
1.1.4. Computational Load
Let and . An implementation of the filters provided in [1,2,3,4,5,6,7,8,10,13,14] leads to computation of an inverse or pseudo-inverse matrix for each pair of signals and , and a matrix product of and matrices. Thus, for the processing of arrays in and , the filters in [1,2,3,4,5,6,7,8,10,13,14] require flops [18].
Arrays and can be represented by vectors, and , each with and components, respectively. Then an application of the techniques in [1,2,3,4,5,6,7,8,10,13,14] requires and flops [18].
Thus, for the case when N is large, the computational work associated with the methods in [1,2,3,4,5,6,7,8,10,13,14] becomes, in practice, unacceptable.
The required computational load for the filter considered below is substantially less. This is implied by an ability to use only p pseudo-inverse matrices, where p is much less than cardinality N. For this reason, requires only flops. Clearly, this is less than that required by the filters mentioned above. This issue is illustrated in Section 5.
1.2. Relevant Works
Some particular filtering techniques relevant to the method are proposed below.
1.2.1. Generic Optimal Linear (GOL) Filter [5]
The generic optimal linear (GOL) filter in [5] is a generalization of the Wiener filter to the case when the covariance matrix is not invertible and the observable signal is arbitrarily noisy (i.e., when, in particular, noise is not necessarily additive and Gaussian). The GOL filter has been developed for processing an individual stochastic signal. Some ideas from [5] are used in the proof of Theorem 1 below.
1.2.2. Simplicial Canonical Piecewise Linear Filter [13]
A complex Wiener adaptive filter was developed in [13] from the two-dimensional complex-valued simplicial canonical piecewise linear filter [19]. The filter in [13] was developed for the processing of an individual stochastic signal and can be exploited when the reference signal is known and a ‘covariance-like’ matrix is invertible. The latter precludes an application to the signal types considered, for example, in Section 5: the matrices used in [13] are not invertible for the signals as those in Section 5. Similarly, the filters studied in [8,10] were developed for the processing of a single signal when the covariance matrices are invertible.
For the filter proposed here, these restrictions are removed.
1.2.3. Adaptive Piecewise Linear Filter [12]
A piecewise linear filter in [12] was proposed for a fixed image denoising (given by a matrix), corrupted by an additive Gaussian noise. That is, the method involved a non-stochastic reference signal and required its knowledge. No theoretical justification for the filter was given in [12].
1.2.4. Averaging Polynomial Filter [15,16]
The averaging polynomial filter proposed in [15,16] was developed for the purpose of processing infinite signal sets. The filter was based on an argument involving the ‘averaging’ over sets of signals under consideration. This device allows one to determine a single filter for the processing of infinite signal sets. At the same time, it leads to an increase in the associated error when signals differ considerably from each other. This effect is illustrated in Section 5 below.
1.2.5. Other Relevant Filters
The technique developed in [11] is an extension of the GOL filter to the constraint problem with respect to the filter rank. It concerns data compression.
The methods in [6,7,20,21] have been developed for deterministic signals. Motivated by the results achieved in [20,21], adaptive filters were elaborated in [22]. A theoretical basis for the device proposed in [20,21] is provided in [23].
We note that the idea of piecewise linear filtering has been used in the literature in several very different conceptual frameworks, despite exploiting some very similar terms (as in [12,13,19,20,21,22,23,24,25,26]). At the same time, a common feature of those techniques is that they were developed for the processing of a single signal, not of large signal sets as in this paper. In particular, piecewise linear filters in [24] have been obtained by arranging linear filters and thresholds in a tree structure. Piecewise linear filters discussed in [25] were developed using so-called threshold decomposition, which is a segmentation operator exploited to split a signal into a set of multilevel components. Filter design methods for piecewise linear systems proposed in [26] were based on a piecewise Lyapunov function.
1.3. Difficulties Associated with the Known Filtering Techniques
Basic difficulties associated with applying the known filtering techniques to the case under consideration (i.e., to processing of large signal sets, and ) are as follows:
- (i)
- They require information on each reference signal (in the form of a sample, for example).
- (ii)
- Matrices used in the known filters can be non-invertible (as in the simulations considered below in Section 5) and then the filter does not exist.
- (iii)
- The associated computation work may require a very long time. For example, in simulations (Section 5), MATLAB was out of memory for computing the GOL filter [5] when each of the sets and was represented by a long vector (this option has been discussed in Section 1.1.4 above). The PC used for the simulations was a Dell Latitude 7400 with a CPU Intel Core i5 (8th Gen) 8265U/1.6 GHz and 8 GB RAM.
1.4. Differences from the Known Filtering Techniques
The differences from the known filtering techniques discussed above are as follows:
- (i)
- We consider a single filter that processes arbitrarily large input-output sets of stochastic signal vectors. The known filters [1,2,3,4,5,6,7,8,9,10,11,12,13,14,19,20,21,22,23,24,25,26] have been developed for the processing of an individual signal vector only. In the case of their application to arbitrarily large signal sets, they imply difficulties described in Section 1.1 and Section 1.3 above.
- (ii)
- As a result, our piecewise linear filter model (Section 3), the statement of the problem (Section 3.3 below), and consequently, the device of its solution (Section 4 below) are different from those considered in [12,13,19,20,21,22,23,24,25,26]. In this regard, see also Section 1.2.5.
- (iii)
- The above naturally leads to a new structure of the filter (presented in Section 3.4 and Section 4.2 below), which is very different from the known ones.
1.5. Contribution
In general, for the processing of large data sets, the proposed filter allows us to achieve better results in comparison with the known techniques in [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,19,20,21,22,23,24,25,26]. In particular, it allows us to achieve the following:
- (i)
- Achieve a desired accuracy in signal estimation This means that any desired accuracy is achieved theoretically, as is shown in Section 4.4 below. In practice, of course, the accuracy is increased to a prescribed reasonable level.
- (ii)
- Exploit prior information only on few reference signals, p, from the set that contains signals or even an infinite number of signals.
- (iii)
- Find a single filter to process any signal from the arbitrarily large signal set.
- (iv)
- Determine the filter in terms of pseudo-inverse matrices so that the filter always exists.
- (v)
- Decrease the computational load compared to the related known techniques.
2. Some Preliminaries
2.1. Notation
The signal sets we consider are, in fact, special representations of time series.
Let be a probability space (usually, is the set of outcomes, is a -field of measurable subsets in and is an associated probability measure on ; in particular, ) and and be arbitrarily large sets of signals such that
where We interpret as a reference signal and as an observable signal, an input to the filter studied below. In an intuitive way, can be regarded as a noise-corrupted version of . For example, can be interpreted as , where is white noise. In this paper, we do not restrict ourselves to this simplest version of and assume that the dependence of on and is arbitrary. The variable represents time. More generally, T can be considered as a set of parameter vectors , where is a q-dimensional cube, i.e., and . One coordinate, say of , could be interpreted as time. Then, for example, the stochastic signal can be interpreted as an arbitrary stationary time series.
Let be a sequence of fixed time-points such that
Because of Equation (1), the sets and are divided in ‘smaller’ subsets and , respectively, so that, for each ,
Therefore, and can now be represented as
2.2. Brief Description of the Method
We wish to determine a single filter that estimates from with a controlled, associated error. Array can be finite or infinite.
To this end, the proposed filter is represented by a sum of sub-filters (see Equations (4) and (5) below). The sub-filter transforms signals of subset of array to signals of subset of array , i.e., . The prime idea is to determine separately, for each , from an associated minimization problem (Equation (11)) (see Section 3.4 and Section 4.2 below). This procedure provides an estimate that interpolates at and . It is natural to expect that the processing of a ‘smaller’ subset, , may lead to a smaller error than that associated with the processing of the whole array .
Therefore, the advantages of are similar to those that follow from an application of the piecewise interpolation procedure, such as, for example, the high accuracy of estimation.
Section 4.4 confirms this observation. Importantly, it is shown in Section 4.5 and Section 5 that the proposed technique avoids the bottlenecks of the known methods discussed in Section 1.3 above.
3. Description of the Problem
3.1. Piecewise Linear Filter Model
Let be a filter such that, for each ,
where
Here, is a sub-filter defined for . In Equation (5), and is a linear operator given by a matrix , so that
Thus, is defined by an operator such that
3.2. Assumptions
In the known approaches related to filtering of stochastic signals (e.g., see [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]), it is assumed that covariance matrices formed from the reference signal and observed signal are known or can be estimated.
The assumption used here is similar. The covariance matrices that are assumed to be known or can be estimated are formed from selected signal pairs with and p to be a small number. It is worthwhile to note that it is not assumed that the covariance matrices are known for each signal pair from , with ), , where N is the number of signals in or .
3.3. The Problem
In Equations (4)–(6), parameters of the filter , i.e., vector and matrix , for , are unknown. Therefore, under the assumptions described in Section 3.2, the problem is to determine and , for . The related problem is to estimate an error associated with the filter .
Solutions to both problems are given in Section 4.2 and Section 4.4, respectively. In particular, in the following Section 3.4, interpolation Equations (8) and (11) are introduced that lead to a determination of and .
3.4. Interpolation Conditions
Let us denote
where is the Euclidean norm of .
For , let be an estimate of determined by known methods [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]. This is the initial condition of the proposed technique.
For , each sub-filter in Equations (5) and (6) is defined so that and satisfy the conditions as follows:
Sub-filter : For , and solve
respectively. Then an estimate of , , for , is determined as
where and satisfy Equation (8). In particular, and
Extending this procedure up to , where , we set the following: Let be an estimate of defined by the preceding steps as
Then sub-filter is defined as follows:
Sub-filter : For , and solve
respectively. Then an estimate of , , for , is determined as
Equations (8) and (11) are motivated by the device of piecewise function interpolation and associated advantages [17].
Filter of the form of Equations (4) and (5) with and satisfying Equations (8) and (11) is called the piecewise linear interpolation filter. The pair of signals associated with time defined by Equation (1) is called the interpolation pair.
Remark 1.
In general, the above procedure might be sensitive to the choice of the initial estimate determined by known methods [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]. If the initial estimate is poor, then the associated error increases.
4. Main Results
4.1. General Device
In accordance with the scheme presented in Section 3.1 and Section 3.4 above, an estimate of the reference signal , for any , by the piecewise linear interpolation filter , is given by
where, for each the sub-filter is given by Equation (5), and is defined from interpolation Equations (8) and (11).
4.2. Filter Model
We write and where
Here, and , for all , are stochastic variables.
The associated covariance matrix is defined by
where
The Moore–Penrose generalized inverse of a matrix M is denoted by .
The main results are represented below.
Theorem 1.
Let and be sets of reference signals and observed signals, respectively. Let , for , be such that
For let be a known estimate of . As has been mentioned in Section 3.4, can be determined by the known methods. Then, for any , the proposed piecewise linear interpolation filter transforms any signal to an estimate of , , is given by
where
where is the identity matrix and is an arbitrary matrix.
Proof.
The proof of Theorem 1 is given in Appendix A. □
It is worthwhile to observe that, due to an arbitrary matrix in Equation (19), the filter is not unique. In particular, can be chosen as the zero matrix similarly to the generic optimal linear [5] (which is also not unique for the same reason).
4.3. Numerical Realization of Filter and Associated Algorithm
4.3.1. Numerical Realization
In practice, the set (see Section 2.1) is represented by a finite set , i.e., where .
For , the estimate of , , and observed signal are represented by and matrices
The sequence of fixed time-points introduced in Equation (1) is such that
where and where and are positive integers such that .
For , vectors and associated with in Equation (21) are represented, respectively, by
4.3.2. Algorithm
As it has been mentioned in Section 3.4, it is supposed that, for an estimate of , , is known and can be determined by the known methods. This is the initial condition of the proposed technique.
On the basis of the results obtained in Section 3.4 and Section 4.2, the performance algorithm of the proposed filter consists of the following steps (Algorithm 1). For , we write .
| Algorithm 1 Computation of , , …, given by Theorem 1 |
|
4.4. Error Analysis
It is natural to expect that the error associated with the piecewise interpolating filter decreases when decreases. Below, in Theorem 3, we justify that this observation is true. To this end, first, in the following Theorem 2, we establish an estimate of the error associated with the filter F.
Let us introduce the norm by
We also denote .
Let us suppose that and are Lipschitz continuous signals, i.e., that there exist real non-negative constants and , with , such that, for ,
where .
Theorem 2.
Under the conditions (Equation (23)) the error associated with the piecewise interpolation filter, , is estimated as follows:
Proof.
The proof of Theorem 2 is given in Appendix A. □
Further, to show that the error of the reference signal estimate tends to zero, we need to assume that, for , the known estimate differs from for the value of the order , i.e. that, for some constant ,
Theorem 3.
Proof.
The proof of Theorem 3 is given in Appendix A. □
Remark 2.
We would like to emphasize that the statement of Theorem 3 is fulfilled only under Equations (23) and (25). At the same time, Equations (23) and (25) are not restrictive from a practical point of view. Equation (23) is true for Lipschitz continuous signals and , i.e., for a very wide class of signals. Equation (25) is achieved by choosing an appropriate known method (e.g., see [1,2,3,4,5,6,7,8,9,10,11,13,14,15,16]) to find the estimate used in the proposed filter (see Equation (8) and Theorem 1).
4.5. Estimation of Covariance Matrices in Equation (19)
Matrix used in Equation (19), for , can be estimated as follows:
- A popular method of estimating is provided, for example, in [27]—it is based on the use of samples of and , for .
- In the case of incomplete observations, the method proposed in [28,29] can be used.
- Let be a matrix obtained from matrix where the term is replaced by with . Since with is known, matrix can be considered as an estimate of .
- In the important case of an additive noise, can be represented in the explicit form. Indeed, ifwhere is a random noise, then and matrix can be represented as follows:
We note that the RHS of Equation (27) depends only on observed signals , , estimated signal , and noise , not on the reference signal . In particular, in Equation (27), the term can be estimated as where . It is motivated by the Holder’s inequality for integrals. The second term in Equation (27), , can be estimated from the samples of and .
We also note that the first term in the RHS of Equation (27), , is similar to the related covariance matrix in the Wiener filtering approach [5].
- 5.
- Other known ways to estimate can be found in [5], Section 5.3.
In general, an estimation of covariance matrices is a special research topic that is not a subject of this paper. The relevant references can be found, for example, in [5,29].
5. Simulations
5.1. General Consideration
The simulations represented below have been performed by Matlab R14 (Version 7.0). In accordance with Section 4.3.1, signal sets and (see Section 2.1) are given by
where, for , and . In many practical problems (arising, for example, in a DNA analysis the number N is quite large; for instance, .
We set and . Thus, in these simulations, the interval (see Section 2.1 and Section 4.3.1) is modeled as 141 points with so that .
The sequence of fixed time-points in Equation (1) is now such that
Below, in Examples 1–12, four particular choices of the specific interpolation signal pairs (introduced in Section 3.4) are considered, for and 28. Points are as follows.
For , if , then , respectively, where and .
For , if , then , and if , then , where .
Signals and have been simulated as digital images represented by matrices
respectively, for , so that represents an image that should be estimated from an observed image . A column of matrices and , and , for , represents a realization of signals and , respectively.
Note that did not used in the piecewise linear filter below since they are not supposed to be known. They are represented here for illustration purposes only. In particular, are used to compare their estimates by different filters.
Observed noisy signals have been simulated in different forms presented by Equations (40) and (49)–(51) in the Examples 1–12 below. We note that the considered observed signals are grossly corrupted.
To estimate the signals , …, from the observed signals , …, , the proposed piecewise linear filter , the generic optimal linear (GOL) filters [5] and the averaging polynomial filter [16] have been used.
The filters proposed in [11,12,13,16] have not been applied here by the reasons discussed in Section 1. In particular, the filter in [13] cannot be applied to signals represented by , …, in the form Equations (40), (49)–(51) below because the associated inverse matrices used in [13] do not exist.
For signals under consideration (given by matrices and with ), the filter , the generic optimal linear (GOL) filters [5] and the averaging polynomial filter [15,16] are represented as follows:
- (i)
- Piecewise linear filter . For , designates an interpolation pair defined similarly to that in Section 3.4. Each and is associated with in Equation (28) so thatThe estimate of by the filter is given bywhere Equations (16)–(19) are presented in Section 4.2,and where and are estimates of matrices and in Equation (19), respectively. In particular, can be represented in the formFurther, matrix depends on where is unknown. Therefore, a determination of is reduced, in fact, to finding an estimate of . Since it is customary to find in terms of signal samples [5], has been presented asand has been constructed from a sample of as follows. The sample of is a matrix presented by odd columns of . Then an estimate of is chosen as a matrix where each odd column is a related odd column of , and each even column is an average of two adjacent columns. The last column in is the same as its preceding column.This way of estimating was chosen for illustration purposes only. Other related methods have been considered in Section 4.5.The errors associated with the filter are given by
- (ii)
- Generic optimal linear (GOL) filters [5]. To each signal , an individual GOL filter has also been applied, so that estimates from in the formfor each . Thus, the GOL filter requires an estimate of 141 matrices , for each .Similarly to matrix in the filter above, the matrix has been estimated from samples of each , , for each .One of the advantages of the proposed filter is that requires a smaller number, p, of samples of , , to be known (where ).The errors associated with filters are given by
- (iii)
- Averaging polynomial filters [15,16]. By the methodology in [15], the averaging polynomial filter W is based on the use of the estimates of the covariance matrices, and , in the formThen, for each, , the estimate of is given byThe errors associated with the filter W are given by
5.2. Simulations with Signals Modelled from Images ‘Plant’: Application of Piecewise Interpolation Filter and GOL Filters
Here, results of simulations for reference signals represented by matrices (see Equation (29) above) formed from images ‘plant’ from http://sipi.usc.edu/services/database.html (accessed on 21 February 2025). Typical selected images are shown in Figure 1.
Figure 1.
Examples of selected signals to be estimated from observed data.
Observed noisy images have been simulated in the form
for each Here, • means the Hadamard product, and and are matrices with random entries. The entries of are normally distributed with mean zero, variance one and standard deviation one. The entries of are uniformly distributed in the interval . A typical example of such images is given in Figure 2a. Examples of signal estimates by different filters are represented in Figure 2b–d.
Figure 2.
Examples of the observed signal and the estimates obtained by different filters.
To demonstrate the effectiveness of the proposed filter , sub-filters and associated interpolation signal pairs have been chosen in four different ways as follows.
Example 1.
First, for , the interpolation signal pairs are
The error values associated with filter are evaluated by Equation (37). The graph of is presented in Figure 3a.
Figure 3.
Diagrams of errors associated with the piecewise interpolation filters and the GOL filters [5] applied to signals described in Examples 1–4: (a) for ; (b) for ; (c) for ; (d) for .
Example 2.
For , the interpolation signal pairs are
The error magnitudes associated with the piecewise interpolation filter constructed by Equations (31)–(36) with the interpolation signal pairs given by Equations (43) and (44) are diagrammatically shown in Figure 3b.
It follows from Figure 3b that the errors associated with filter is less than those of filter . This is a confirmation of Theorem 3.
Example 3.
Further, for , the interpolation pairs are
Example 4.
Finally, the number of interpolation signal pairs is so that
In this case, when p is greater than in the previous Examples 1–3, the errors associated with the piecewise interpolation filter are smaller than those associated with filters , and —see Figure 3d.
The diagrams of errors associated with the GOL filters [5] are also presented in Figure 3, which shows that the proposed filters , , and provide better accuracy then that of the GOL filters.
At the same time, the filter is easier to implement since it requires less initial information compared to GOL filters, as it has been discussed in Section 1.1.1 and Section 1.1.4.
Results of the application of the averaging polynomial filter [15,16] are discussed in Section 5.4 below.
5.3. Simulations with Signals Modelled from Images ‘Boat’: Application of Piecewise Interpolation Filter and GOL Filters
In this section, results of the simulations for a different type of signal than those considered in Section 5.2 above are presented. Here, the reference signals are formed from images ‘boat’ (http://sipi.usc.edu/services/database.html, accessed 21 February 2025).
Observed noisy signals have been simulated in the form
for each The noise term is different from that in Equation (40).
Figure 4.
Examples of selected signals to be estimated from observed data considered in Examples 5–9.
Figure 5.
Examples of the observed signal and the estimates obtained by different filters.
As in Section 5.2, the piecewise interpolation filter is constructed by Equations (31)–(36). In Examples 5–8 below, the number of sub-filters and associated interpolation signal pairs have been chosen in four different ways.
Example 5.
First, similar to Example 1, the number of interpolation signal pairs has been chosen as , and and have been presented as in Equations (41) and (42).
The error values associated with the piecewise interpolation filter applied to these data are presented in Figure 6a.
Figure 6.
Diagrams of errors associated with the piecewise interpolation filter of order p and the generic optimal linear (GOL) filters [5] applied to signals described in Examples 5–8: (a) for ; (b) for ; (c) for ; (d) for .
Example 6.
For the greater number of interpolation signal pairs than that in Example 5, , and for and () chosen as in Equations (43) and (44), the error magnitudes associated with the piecewise interpolation filter are diagrammatically shown in Figure 6b. A comparison between Figure 6a,b demonstrates that the increase in p implies the decrease in the errors associated with the filter .
Example 7.
Example 8.
The errors associated with the GOL filters are also presented in Figure 6a–d. The figures clearly demonstrate the advantage of the piecewise interpolation filter .
Results of the application of the averaging polynomial filter [16] are discussed in Section 5.4 below.
5.4. Results of Simulations for Averaging Polynomial Filter [15,16]
To further illustrate the effectiveness of the proposed piecewise interpolation filter, in this Section, results of simulations for the averaging polynomial filter [15,16] are presented. The filter has been applied to two different types of data considered in Section 5.2 and Section 5.3.
Example 9.
The filter [15,16] applied to signals considered in Section 5.2 gives the associated errors (see Equation (39)) represented in Figure 7a. For a comparison, the errors associated with the piecewise interpolation filter and the GOL filters [5] are also given in Figure 7a.
Figure 7.
Diagrams of errors associated with the averaging polynomial filters [15,16] in (a) Example 9 and (b) Example 10.
A typical example of the estimated signal by the averaging polynomial filter [15,16] is presented in Figure 2d above.
Example 10.
The averaging polynomial filter [16] applied to signals considered in Section 5.3 produces the associated errors shown in Figure 7b. The errors associated with the piecewise interpolation filter are much smaller, and they are not discerned in Figure 7b.
The extreme failure of the averaging polynomial filter (the errors of magnitude ) in Examples 9 and 10 illustrates the observation made in Section 1.2.4: The associated error increases when signals differ considerably from each other. This is the case in Examples 9 and 10.
5.5. Further Simulations with Different Type of Noise
In Examples 11 and 12 below, a different type of noise is considered. Unlike the multiplicative noise in Equations (40) and (49), here, the noise is additive.
Example 11.
First, the piecewise interpolation filter , the GOL filters [5] and the averaging polynomial filter [16] have been applied to the observed signals given by
where is as in Section 5.2, i.e., is formed from the images ’plant’. In Figure 8a, the diagrams of the errors associated with filter and the GOL filters [5] are given. The errors associated with the averaging polynomial filter [16], , are much greater (of order ) and they are not presented in Figure 8a.
Figure 8.
Diagrams of errors associated with the piecewise interpolation filter and the generic optimal linear (GOL) filters [5] applied to signals described in (a) Example 11 and (b) Example 12.
Example 12.
In this example, the reference signals are as those in Section 5.3, i.e., they are formed from the image ’boat’. The observed signals are given by
The piecewise interpolation filter and the GOL filters [5] estimate the reference signals with the associated errors represented in Figure 8b. As in Example 11 above, in this case, the errors associated with the averaging polynomial filter [15,16] are much greater (of order ), and they are not presented in Figure 8b.
Examples 11 and 12 further demonstrate the advantages of the proposed piecewise interpolation filter.
5.6. Summary of Simulations
The above simulations confirm the theoretical results obtained in Theorems 1–3. In particular, Figure 3 and Figure 6 demonstrate that the error associated with the piecewise interpolation filter decreases when the number of sub-filters , p, increases.
A comparison between the proposed filter and the known related filters [5,12,13,15,16] has been performed. The filter estimates the reference signals with the accuracies that are much better than those of the generic optimal linear (GOL) filters [5] and the averaging polynomial filter [15,16]. Further, the filters proposed in [12,13] fail in processing the signals under consideration. This is because the observed signals in Equations (40), (49), (50) and (51) are grossly corrupted, and, therefore, the inverse matrices used in the filter structures in [13] do not exist. The technique in [12] requires the use of the reference signal in the proposed filter, which is supposed to be unknown in the simulations above.
The filters have been applied to the different signal sets (presented in Section 5.2 and Section 5.3), using different forms of noise (given in Equations (40), (49), (50) and (51)).
Computational work associated with the proposed filter is substantially less than that associated with the known filters discussed in Section 1 (in particular, with the filters in [10,11,12,13,16,17,20,21,22,23,24,25,26]). This is because, for the processing of a data set containing N signals, filter requires computation of p covariance matrices with while the known filters require computation of N matrices (in the above examples, , respectively, and ).
Table 1 represents a typical time in seconds used by the piecewise linear interpolation filter (PILF), GOL and the averaging polynomial filter (APF) [16] in the above examples. It follows from Table 1 that the PILF is faster than the GOL up to ten times. While the time used by the PILF and APF for is almost the same, the accuracy associated with the PILF is much better than that associated with the APF.
Table 1.
Time in seconds used by PILF, GOL and APF in the above examples.
Table 2 summarizes the computational and structural advantages of the proposed method. The abbreviation ‘PINV’ means the use of the pseudo-inverse matrices (not inverse matrices), and , and denote the errors associated with the PILF, GOL and APF, correspondingly. ‘Stability’ means numerical stability. The last column illustrates features related to filters considered in [10,11,12,13,16,17,20,21,22,23,24,25,26]. As mentioned above, these filters cannot be applied to the data under consideration because of the singularity of matrices used in [10,11,12,13,16,17,20,21,22,23,24,25,26]. Therefore, their errors and a computational complexity are marked by N/A as not applicable.
Table 2.
Summary of computational and structural advantages of the proposed method.
6. Conclusions
The technique of the constructive approximation of a nonlinear operator has been provided. Here, and where and is the set of outcomes of a probability space. The device behind the proposed method is based on a special extension of the piecewise linear interpolation technique to the case of stochastic sets and . The proposed methodology is motivated by the problems arising in signal processing, where the nonlinear operator is interpreted as a nonlinear system (or a nonlinear filter) transforming a set of stochastic signals to the set of stochastic signals . Therefore, the provided technique provides an effective way to transform large data sets.
Distinctive features of the approach are as follows:
- (i)
- The proposed filter is nonlinear and is presented in the form of a sum with terms, where each term, , is interpreted as a particular sub-filter. Here, and are ‘small’ pieces of and , respectively.
- (ii)
- The prime idea is to exploit prior information only on a few reference signals, p, from the set that contains signals (or even an infinite number of signals) and determine separately for each piece and , so that the associated error is minimal. In other words, the filter is flexible to changes in the sets of observed and reference signals and , respectively.
- (iii)
- Due to the specific way of determining , the filter provides a smaller associated error than that for the processing of the whole set by a filter that is not specifically adjusted to each particular piece . Moreover, the error associated with our filter decreases when the number of its terms, , increases.
- (iv)
- While the proposed filter processes arbitrarily large (and even infinite) signal sets, the filter is nevertheless fixed for all signals in the sets.
- (v)
- The filter is determined in terms of pseudo-inverse matrices so that the filter always exists.
- (vi)
- computational load associated with the filter is less than that associated with other known filters applied to the processing of large signal sets.
Author Contributions
Conceptualization, methodology, writing—original draft, A.T.; numerical simulations, A.T. and P.P.; algorithm, P.P.; Matlab codes, P.P.; English amelioration, P.P. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors on request.
Conflicts of Interest
The authors declare no conflicts of interest.
Appendix A
Proof of Theorem 1.
Further, for given by Equation (A1),
where is the Frobenius norm. The latter is true because
and
by Lemma 24 in [5]. Thus, the second expression in Equation (11) is reduced to the problem
It is known (see, for example, [5], p. 304) that the solution of Equation (A5) is given by Equation (19).
Theorem 1 is proven. □
Proof of Theorem 2.
Then, Equation (A6) implies
where .
Proof of Theorem 3.
Then,
Let us consider an estimate of , for . To this end, let us denote .
For i.e., for ,
where . In particular, the latter implies
For i.e., for ,
where . In particular, it follows that
On the above basis, let us assume that, for with , i.e., for ,
where is defined by analogy with .
Then, for with , i.e., for ,
where . Thus, the following is true:
Therefore, Equations (A10), (A11) and (A12) imply
where , and then it follows from Equations (A9)–(A11) and (A13) that for all ,
Let us now choose and so that and partition interval by points so that and with . There exists an integrable (bounded) function , such that, for , . Then,
Thus,
References
- Chen, J.; Benesty, J.; Huang, Y.; Doclo, S. New Insights Into the Noise Reduction Wiener Filter. IEEE Trans. Audio Speech Lang. Process. 2006, 14, 1218–1234. [Google Scholar] [CrossRef] [Scilit]
- Spurbeck, M.; Schreier, P. Causal Wiener filter banks for periodically correlated time series. Signal Process. 2007, 87, 1179–1187. [Google Scholar] [CrossRef] [Scilit]
- Goldstein, J.S.; Reed, I.; Scharf, L.L. A Multistage Representation of the Wiener Filter Based on Orthogonal Projections. IEEE Trans. Inf. Theory 1998, 44, 2943–2959. [Google Scholar] [CrossRef] [Scilit]
- Hua, Y.; Nikpour, M.; Stoica, P. Optimal Reduced-Rank estimation and filtering. IEEE Trans. Signal Process. 2001, 49, 457–469. [Google Scholar]
- Torokhti, A.; Howlett, P. Computational Methods for Modelling of Nonlinear Systems; Elsevier: Amsterdam, The Netherlands, 2007. [Google Scholar]
- Sontag, E.D. Polynomial Response Maps; Lecture Notes in Control and Information Sciences; Springer: Cham, Switzerland, 1979; Volume 13. [Google Scholar]
- Chen, S.; Billings, S.A. Representation of non-linear systems: NARMAX model. Int. J. Control 1989, 49, 1013–1032. [Google Scholar] [CrossRef]
- Mathews, V.J.; Sicuranza, G.L. Polynomial Signal Processing; J. Wiley & Sons: Hoboken, NJ, USA, 2001. [Google Scholar]
- Torokhti, A.; Howlett, P. Optimal Transform Formed by a Combination of Nonlinear Operators: The Case of Data Dimensionality Reduction. IEEE Trans. Signal Process. 2006, 54, 1431–1444. [Google Scholar] [CrossRef] [Scilit]
- Vesma, J.; Saramaki, T. Polynomial-Based Interpolation Filters—Part I: Filter Synthesis. Circuits Syst. Signal Process. 2007, 26, 115–146. [Google Scholar] [CrossRef] [Scilit]
- Torokhti, A.; Miklavcic, S. Data Compression under Constraints of Causality and Variable Finite Memory. Signal Process. 2010, 90, 2822–2834. [Google Scholar] [CrossRef] [Scilit]
- Russo, F. Technique for image denoising based on adaptive piecewise linear filters and automatic parameter tuning. IEEE Trans. Instrum. Meas. 2006, 55, 1362–1367. [Google Scholar] [CrossRef]
- Cousseau, J.E.; Figueroa, J.L.; Werner, S.; Laakso, T.I. Efficient Nonlinear Wiener Model Identification Using a Complex-Valued Simplicial Canonical Piecewise Linear Filter. IEEE Trans. Signal Process. 2007, 55, 1780–1792. [Google Scholar] [CrossRef] [Scilit]
- Wigren, T. Recursive Prediction Error Identification Using the Nonlinear Wiener Model. Automatica 1993, 29, 1011–1025. [Google Scholar] [CrossRef] [Scilit]
- Torokhti, A.; Howlett, P. Filtering and Compression for Infinite Sets of Stochastic Signals. Signal Process. 2009, 89, 291–304. [Google Scholar] [CrossRef] [Scilit]
- Torokhti, A.; Manton, J. Generic Weighted Filtering of Stochastic Signals. IEEE Trans. Signal Process. 2009, 57, 4675–4685. [Google Scholar] [CrossRef] [Scilit]
- Babuska, I.; Banerjee, U.; Osborn, J.E. Generalized finite element methods: Main ideas, results, and perspective. Int. J. Comput. Methods 2004, 1, 67–103. [Google Scholar] [CrossRef] [Scilit]
- Golub, G.H.; van Loan, C.F. Matrix Computations; Johns Hopkins University Press: Baltimore, MD, USA, 1996. [Google Scholar]
- Julian, P.; Desages, A.; D’Amico, B. Orthonormal high-level canonical PWL functions with applications to model reduction. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 2000, 47, 702–712. [Google Scholar] [CrossRef] [Scilit]
- Kang, S.; Chua, L. A global representation of multidimensional piecewise-linear functions with linear partitions. IEEE Trans. Circuits Syst. 1978, 25, 938–940. [Google Scholar] [CrossRef]
- Chua, L.O.; Deng, A.-C. Canonical piecewise-linear representation. IEEE Trans. Circuits Syst. 1988, 35, 101–111. [Google Scholar] [CrossRef] [Scilit]
- Lin, J.-N.; Unbehauen, R. Adaptive nonlinear digital filter with canonical piecewise-linear structure. IEEE Trans. Circuits Syst. 1990, 37, 347–353. [Google Scholar] [CrossRef]
- Lin, J.-N.; Unbehauen, R. Canonical piecewise-linear approximations. IEEE Trans. Circuits Syst. I Fundam. Theory Appl. 1992, 39, 697–699. [Google Scholar] [CrossRef] [Scilit]
- Gelfand, S.B.; Ravishankar, C.S. A tree-structured piecewise linear adaptive filter. IEEE Trans. Inf. Theory 1993, 39, 1907–1922. [Google Scholar] [CrossRef] [Scilit]
- Heredia, E.A.; Arce, G.R. Piecewise linear system modeling based on a continuous threshold decomposition. IEEE Trans. Signal Process. 1996, 44, 1440–1453. [Google Scholar] [CrossRef] [Scilit]
- Feng, G. Robust filtering design of piecewise discrete time linear systems. IEEE Trans. Signal Process. 2005, 53, 599–605. [Google Scholar] [CrossRef] [Scilit]
- Anderson, T. An Introduction to Multivariate Statistical Analysis; Wiley: New York, NY, USA, 1984. [Google Scholar]
- Perlovsky, L.I.; Marzetta, T.L. Estimating a Covariance Matrix from Incomplete Realizations of a Random Vector. IEEE Trans. Signal Process. 1992, 40, 2097–2100. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Ledoit, O.; Wolf, M. A well-conditioned estimator for large-dimensional covariance matrices. J. Multivar. Anal. 2004, 88, 365–411. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.







