Prony Method for Two-Generator Sparse Expansion Problem

Abstract

In data analysis and signal processing, the recovery of structured functions from the given sampling values is a fundamental problem. Many methods generalized from the Prony method have been developed to solve this problem; however, the current research mainly deals with the functions represented in sparse expansions using a single generating function. In this paper, we generalize the Prony method to solve the sparse expansion problem for two generating functions, so that more types of functions can be recovered by Prony-type methods. The two-generator sparse expansion problem has some special properties. For example, the two sets of frequencies need to be separated from the zeros of the Prony polynomial. We propose a two-stage least-square detection method to solve this problem effectively.

1. Introduction

The Prony method is a popular tool used to recover the functions represented in sparse expansions using one generating function. For example, the function with the following form

$$f\left(x\right)=\sum _{j=1}^M{c}_j{e}^{ix{\varphi }_j}$$

(1)

can be recovered from $2M$ equispaced sampling values $f\left(lh\right),l=0,...,2M-1$ for an appropriate positive constant h; however, in many real-world applications, we need to deal with the functions represented by more than one generating functions. For example, the harmonic signals with the form

$$f\left(x\right)=\sum _{j=1}^M\left(c_{j}cos\left({\varphi }_{j}x\right)+d_{j}sin\left({\beta }_{j}x\right)\right),$$

(2)

are generated by two generating functions (or simply generators): $cos\left(\varphi x\right)$ and $sin\left(\beta x\right)$, where $\varphi$ and $\beta$ are generic parameters used as the placeholders for the real parameters ${\left\{{\varphi }_j\right\}}_{j=1}^M$ and ${\left\{{\beta }_j\right\}}_{j=1}^M$ to generate the specific terms in the expansion. In this system, we have two sets of coefficients ${\left\{c_j\right\}}_{j=1}^M$ and ${\left\{d_j\right\}}_{j=1}^M$ and two sets of frequencies ${\left\{{\varphi }_j\right\}}_{j=1}^M$ and ${\left\{{\beta }_j\right\}}_{j=1}^M$. Analogous to the original Prony method, we expect to use $4M$ equispaced sampling values $f\left(lh\right),l=0,...,4M-1$ to recover those four sets of parameters.

There are some existing methods to solve this problem. The first one is to convert it to a single-generator problem by the following formulas

$$cosx=\frac{1}{2}(e^{ix}+e^{-ix})\mathrm{and}sinx=\frac{1}{2i}(e^{ix}-e^{-ix}),$$

which results in problem (1) (see [1]). Another way using the same idea is based on the even/odd properties for $cosx$ and $sinx$ (see [2]) as follows

$$f\left(x\right)+f(-x)=2\sum _{j=1}^M{c}_{j}cos\left({\varphi }_{j}x\right).$$

(3)

However, this approach is very restrictive, because the chance that one can make this kind of conversion is very small. In this paper, we are interested in solving the general two-generator sparse expansion problem by a new way of generalized Prony method. More specifically, we study the functions with the following two-generator sparse expansion

$$f\left(x\right)=\sum _{j=1}^{M_1}{c}_{j}u\left({\varphi }_{j}x\right)+\sum _{l=1}^{M_2}{d}_{l}v\left({\beta }_{l}x\right),$$

(4)

where $u\left(\varphi x\right)$ and $v\left(\beta x\right)$ are two different functions used as the generators. In order to make the Prony method work, we need a critical condition for our special technique: There exists a linear operator, such that $u\left(\varphi x\right)$ and $v\left(\beta x\right)$ are both eigenfunctions of this operator.

Another situation that could lead to the two-generator expansion problem is when we apply some special transforms on a sparse expansion. For example, when we apply the short time Fourier transform (STFT), i.e.,

$$\mathrm{STFT}\left\{f\left(x\right)\right\}(\omega ,\tau )={\int }_{-\infty }^{\infty }f\left(x\right)w(x-\tau )e^{-i\omega x}dx$$

(5)

using the Gaussian window function $w\left(x\right)=\frac{1}{\sqrt{2\pi }}{e}^{-\frac{x^2}{2{\sigma }^2}}$ on the sparse cosine expansion

$$f\left(x\right)=\sum _{j=1}^M{c}_{j}cos\left({\varphi }_{j}x\right),$$

(6)

we would obtain a two-generator sparse expansion as follows,

$$f\left(x\right)=\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-x)}^2}+\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+x)}^2}.$$

(7)

In this example, the two generators are $e^{-\beta {(\varphi -x)}^2}$ and $e^{-\beta {(\varphi +x)}^2}$ with $\beta \ne 0$. Actually, the original single-generator problem (6) can be solved directly. For example, one can convert $cos\left(\varphi x\right)$ to $\frac{1}{2}(e^{i\varphi x}+e^{-i\varphi x})$ (see [1]), or use a method based on the Chebyshev polynomials (see [3]). When we solve problem (6) directly, we use the sampling values in the time domain; when we solve the problem in the form of (7), we use the sampling values in the frequency domain. (See [4] for a discussion on sampling values in the frequency domain.) In this paper, we use this example to study the special properties of the two-generator sparse expansion problem.

Since the signals could take various forms, not necessarily in the exponential form studied in the classical Prony method, many researchers generalized the Prony method to handle different types of signals. For example, many results in [1,3,5,6,7,8,9,10,11,12] have been developed over the last few years. In particular, Peter and Plonka in [1,8] generalized the Prony method to reconstruct M-sparse expansions in terms of eigenfunctions of some special linear operators. In [3], Plonka and others reconstructed different signals by exploiting the generalized shift operator. These results provide us the building blocks for our method in this paper.

We organize our presentation in the remaining sections as follows. In Section 2, we quickly review the classical Prony method and one of its generalizations for the Gaussian generating function to establish the foundation of our method. In Section 3, we describe the details of our method using the example with two generators: cosine and sine functions. In Section 4, we apply our method on two different types of Gaussian generating functions, so that we can study an interesting property: When the Hankel matrix for finding the coefficients of the Prony polynomial is singular, what does it really mean? In Section 5, we show two examples that correspond to the two problems solved in Section 3 and Section 4, respectively. Finally, we make conclusions in Section 6 and describe two related research problems to be solved in the future.

2. Review of the Prony Method and One of Its Generalizations

Our method is built on top of the Prony method and one of its generalizations. Before we present our technique, we review these basic methods.

2.1. Classical Prony Method

Let $f\left(x\right)$ be a function in the form of

$$f\left(x\right)=\sum _{j=1}^M{c}_j{e}^{-ix{\varphi }_j}$$

(8)

with $M\ge 1$. Then the coefficients ${\left\{c_j\right\}}_1^M$ and the frequencies ${\left\{{\varphi }_j\right\}}_1^M$ can be recovered from the sampling values $f\left(lh\right),l=0,...,2M-1$, where h is some positive constant. To solve this problem, a special polynomial called the Prony polynomial can help us convert the relatively hard non-linear problem (8) to two linear problems and a simple non-linear problem (finding zeros of a polynomial). The Prony polynomial for (8) is defined as

$$\Lambda \left(z\right)=\prod _{j=1}^M(z-e^{-ih{\varphi }_j})=\sum _{l=0}^M{\lambda }_l{z}^l,$$

(9)

where ${\lambda }_l,l=0,...,M$ are the coefficients of the monomial terms in (9) with the leading coefficient ${\lambda }_M=1$. The technique is based on the following critical property:

$$\begin{array}{c}\hfill \sum _{l=0}^M{\lambda }_{l}f\left(h(l+m)\right)=\sum _{l=0}^M{\lambda }_l\sum _{j=1}^M{c}_j{e}^{-ih(l+m){\varphi }_j}=\sum _{j=1}^M{c}_j{e}^{-ihm{\varphi }_j}\underset{=0}{\underbrace{\sum _{l=0}^M{\lambda }_l{e}^{-ihl{\varphi }_j}}}=0\end{array}$$

(10)

for any $m=0,1,\dots ,M-1$, which can be written as the following linear system

$$\begin{array}{c}\hfill {\left[f\left(h\right(l+m\left)\right)\right]}_{l,m=0}^{M-1}\left[\begin{array}{c}{\lambda }_0\\ ⋮\\ {\lambda }_{M-1}\end{array}\right]=-\left[\begin{array}{c}f\left(hM\right)\\ ⋮\\ f\left(h\right(2M-1\left)\right)\end{array}\right].\end{array}$$

(11)

The coefficient vector $\lambda ={\left[{\lambda }_0,{\lambda }_1,\dots ,{\lambda }_{M-1}\right]}^T$ can be calculated from the $2M$ sampling values $f\left(lh\right),l=0,...,2M-1$. The linear system (11) is guaranteed to have a unique solution under the condition that all ${\varphi }_j$’s are distinct in $(-K,K)\subset \mathbb{R}$ for some $K>0$ (with h in the range $0<h<\frac{\pi }{K}$), and $c_1,\dots ,c_M$ are nonzero in $\mathbb{C}$, which is a natural requirement for problem (8). This property is a direct result of the following matrix factorization

$$\begin{array}{c}\hfill {\left[f\left(h\right(l+m\left)\right)\right]}_{l,m=0}^{M-1}={\mathit{V}}^{T}diag(c_1,...,c_M)\mathit{V},\end{array}$$

(12)

where $\mathit{V}:={\left[e^{-ilh{\varphi }_j}\right]}_{l=0,j=1}^{l=M-1,j=M}$ is a Vandermonde matrix, which is non-singular for distinct ${\varphi }_j$’s and $h{\varphi }_j\in (-\pi ,\pi ]$ for $j=1,...,M$. The frequencies can be extracted from the zeros of $\Lambda \left(z\right)$ (in the form of $z_j=e^{-ih{\varphi }_j}$) using the formula

$${\varphi }_j=\frac{-\mathrm{Im}(ln\left(z_j\right))}{h},j=1,...,M.$$

(13)

Finally, the coefficients $c_j,j=1,...,M$ can be determined by solving the following overdetermined linear system (with M unknowns and $2M$ equations)

$$f\left(lh\right)=\sum _{j=1}^M{c}_j{e}^{-ilh{\varphi }_j},l=0,...,2M-1.$$

(14)

The redundant equations in this overdetermined linear system will play a critical role in our two-generator method to help us separate the frequencies associated with the two generators (see Section 3).

2.2. Sparse Expansions on Shifted Gaussian

In order to solve the two-generator sparse expansion problem (7), we need to apply the technique presented in [3], which solves a single-generator sparse expansion problem with the following form

$$f\left(x\right)=\sum _{j=1}^M{c}_j{e}^{-\beta {(x-{\varphi }_j)}^2},$$

(15)

where $\beta \in \mathbb{C}\setminus \left\{0\right\}$. The technique relies on the following generalized shift operator

$${\mathcal{S}}_{K,h}f\left(x\right)=K(x,h)f(x+h),$$

(16)

where $h\ne 0$, and $K(·,·)$ has the property

$$\begin{array}{cc}\hfill K(x,h_1+h_2)& =K(x,h_1)K(x+h_1,h_2)=K(x,h_2)K(x+h_2,h_1).\hfill \end{array}$$

The $K(x,h)$ function in (16) is chosen to be $e^{\beta h(2x+h)}$, so that we have the following critical property

$$\begin{array}{cc}\hfill \left({\mathcal{S}}_{K,h}{e}^{-\beta {(\varphi -·)}^2}\right)\left(x\right)& =e^{2\beta \varphi h}{e}^{-\beta {(\varphi -x)}^2},\hfill \end{array}$$

(17)

which means that $e^{-\beta ({\varphi }_j-x)}$’s are eigenfunctions of ${\mathcal{S}}_{K,h}$ for all ${\varphi }_j\in \mathbb{R}$.

The sparse expansion $f\left(x\right)$ in (15) can be reconstructed using $2M$ sampling values $f(x_0+hk),k=0,...,2M-1$, and $x_0$ is an arbitrary real number. If $\mathrm{Re}\beta \ne 0$, then $h\in \mathbb{R}\setminus \left\{0\right\}$; while if $\mathrm{Re}\beta =0$, then $0<h\le \frac{\pi }{2\left|\mathrm{Im}\beta \right|L}$ with ${\varphi }_j\in (-L,L)$ for $j=1,...,M$ for some given $L.$ (See [3].) The Prony polynomial for the problem in (15) can be defined as:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \Lambda \left(z\right)& :=\prod _{j=1}^M(z-e^{2h\beta {\varphi }_j})=\sum _{l=0}^M{\lambda }_l{z}^l\hfill \end{array}\end{array}$$

(18)

with ${\lambda }_M=1$. Then, we have the following linear system

$$\sum _{l=0}^{M-1}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}f(x_0+h(l+m))=-e^{\beta h(m+M)(2x_0+h(m+M))}f(x_0+h(m+M))$$

(19)

for $m=0,1,...,M-1$, which can be represented as an inhomogeneous system

$$\mathit{H}\lambda =-\mathit{G},$$

(20)

where $\mathit{G}:={\left[\left({\mathcal{S}}_{K,(M+m)h}f\right)\left(x_0\right)\right]}_{m=0}^{M-1}$, and $\mathit{H}:={\left[\left({\mathcal{S}}_{K,(l+m)h}f\right)(x_0+(l+m)h)\right]}_{l,m=0}^{M-1}$. This $\mathit{H}$ matrix is a Hankel matrix, and it has the following structure

$$\begin{array}{cc}\hfill \mathit{H}:={\left[\left({\mathcal{S}}_{K,(l+m)h}f\right)(x_0+(l+m)h)\right]}_{l,m=0}^{M-1}& ={\left[K(x_0,(l+m)h)f(x_0+(l+m)h)\right]}_{l,m=0}^{M-1}\hfill \\ \hfill & =\mathit{V}diag\left(c_j{e}^{-\beta {({\varphi }_j-x_0)}^2}\right){\mathit{V}}^T,\hfill \end{array}$$

(21)

with the Vandermonde matrix

$$\mathit{V}:=\left[\begin{array}{cccc}1& 1& \cdots & 1\\ e^{2\beta h{\varphi }_1}& e^{2\beta h{\varphi }_2}& \cdots & e^{2\beta h{\varphi }_M}\\ ⋮& ⋮& \cdots & ⋮\\ e^{2(M-1)\beta h{\varphi }_1}& e^{2(M-1)\beta h{\varphi }_2}& \cdots & e^{2(M-1)\beta h{\varphi }_M}\end{array}\right].$$

Thus, $\mathit{H}$ is invertible for distinct ${\varphi }_j$’s in $(-L,L)\subset \mathbb{R}$ for $L>0$, and the vector of the coefficients $\lambda :={[{\lambda }_0,...,{\lambda }_{M-1}]}^T$ are obtained by solving the system (20), which can be used to calculate the parameters $\left\{{\varphi }_j\right\}$’s.

Finally, the coefficients $c_j$’s in the expansion (15) can be computed by solving the following overdetermined linear system:

$$f(x_0+lh)=\sum _{j=1}^M{c}_j{e}^{-\beta {(x_0-{\varphi }_j+lh)}^2},l=0,...,2M-1.$$

(22)

3. The Sparse Expansion Problem with Two Generators: Cosine and Sine

In this section, we present our method for solving the two-generator sparse expansion problem in the following form

$$f\left(x\right)=\sum _{j=1}^{M_1}{c}_{j}cos\left({\varphi }_{j}x\right)+\sum _{l=1}^{M_2}{d}_{l}sin\left({\beta }_{l}x\right)$$

(23)

through a modified Prony method. We present our method in the following theorem.

Theorem 1.

Assume that a function $f\left(x\right)$ has the two-generator sparse expansion form of(23), where the number of terms for two generators $M_1$ and $M_2$ are known, but the two sets of coefficients in $\{c_1,\dots ,c_{M_1}\}$ and $\{d_1,\dots ,d_{M_2}\}$ and the two sets of frequencies in $\{{\varphi }_1,\dots ,{\varphi }_{M_1}\}$ and $\{{\beta }_1,\dots ,{\beta }_{M_2}\}$ are unknown. If $4(M_1+M_2)-1$ equispaced sampling values of the form $f(x_0+kh)$ for $k=-2(M_1+M_2)+1,\dots ,-1,0,1,\dots ,2(M_1+M_2)-1$ are provided, then the original function $f\left(x\right)$ can be uniquely reconstructed under the following conditions:

$1^{\circ }$ All the coefficients $\{c_1,\dots ,c_{M_1},d_1,\dots ,d_{M_2}\}$ are nonzero in $\mathbb{C}$.

$2^{\circ }$ All the frequencies $\{{\varphi }_1,\dots ,{\varphi }_{M_1},{\beta }_1,\dots ,{\beta }_{M_2}\}$ are distinct in a range $[0,K)\subset \mathbb{R}$ for some $K>0$. Furthermore, h is selected from the range $0<h<{\displaystyle \frac{\pi }{K}}$.

$3^{\circ }$ The value of $x_0\in \mathbb{R}$ is selected to make the $(M_1+M_2)$ numbers $cos\left({\varphi }_1{x}_0\right),\dots ,cos\left({\varphi }_{M_1}{x}_0\right)$,

$sin\left({\beta }_1{x}_0\right),\dots ,sin\left({\beta }_{M_2}{x}_0\right)$ nonzero.

Proof. 

First, we choose an appropriate linear operator, such that our two generating functions $cos\left(\varphi x\right)$ and $sin\left(\beta x\right)$ in (23) are both the eigenfunctions of this operator. We consider the symmetric shift operator (see [3])

$${\mathcal{S}}_{h,-h}f\left(x\right):=\left(\frac{{\mathcal{S}}_{-h}+{\mathcal{S}}_h}{2}\right)f\left(x\right)=\frac{f(x-h)+f(x+h)}{2}.$$

(24)

When we apply this operator on $cos\left(\varphi x\right)$ and $sin\left(\beta x\right)$, we obtain

$$\begin{array}{cc}\hfill \left({\mathcal{S}}_{h,-h}\right)cos\left(\varphi x\right)& =cos\left(\varphi h\right)cos\left(\varphi x\right),\hfill \\ \hfill \left({\mathcal{S}}_{h,-h}\right)sin\left(\beta x\right)& =cos\left(\beta h\right)sin\left(\beta x\right),\hfill \end{array}$$

(25)

where $cos\left(\varphi h\right)$ and $cos\left(\beta h\right)$ are the eigenvalues. Now we define the Prony polynomial for problem (23) using all the eigenvalues ${\{cos\left({\varphi }_{j}h\right)\}}_{j=1}^{M_1}$ and ${\{cos\left({\beta }_{l}h\right)\}}_{l=1}^{M_2}$ as follows:

$$\Lambda \left(z\right)=\prod _{j=1}^{M_1}\left(z-cos\left(h{\varphi }_j\right)\right)\prod _{l=1}^{M_2}\left(z-cos\left(h{\beta }_l\right)\right),$$

(26)

which can be written in terms of the Chebyshev polynomials as

$$\Lambda \left(z\right)=\sum _{k=0}^{M_1+M_2}{\lambda }_k{\mathit{T}}_k\left(z\right),$$

(27)

where ${\mathit{T}}_k\left(z\right):=cos\left(k{cos}^{-1}\left(z\right)\right)$. (See [3] for more information on this technique.) Since the leading coefficient of the Chebyshev polynomial ${\mathit{T}}_k\left(z\right)$ is $2^{k-1}$, we choose ${\lambda }_{M_1+M_2}=2^{1-(M_1+M_2)}$, so that $\Lambda \left(z\right)$ in (27) has the leading coefficient 1. This Prony polynomial has the following critical property:

$$\begin{array}{c}\hfill \sum _{k=0}^{M_1+M_2}{\lambda }_k{\mathit{T}}_k(cos\left({\varphi }_{j}h\right))=0\mathrm{and}\sum _{k=0}^{M_1+M_2}{\lambda }_k{\mathit{T}}_k(cos\left({\beta }_{l}h\right))=0\end{array}$$

for $j=1,2,\dots ,M_1$ and $l=1,2,\dots ,M_2$, respectively, which is essential to help us derive the following linear system.

To derive a linear system for ${\left\{{\lambda }_k\right\}}_{k=0}^{M_1+M_2-1}$, we need to calculate the following expression

$$\sum _{k=0}^{M_1+M_2}{\lambda }_k\left({\mathcal{S}}_{kh,-kh}{\mathcal{S}}_{mh,-mh}f\left(x_0\right)\right),$$

which can be shown to be zero. That is,

$$\frac{1}{4}\sum _{k=0}^{M_1+M_2}{\lambda }_k\left(f(x_0+(m+k)h)+f(x_0-(m+k)h)+f(x_0+(m-k)h)+f(x_0-(m-k)h)\right)=0$$

(28)

for $m=0,1,\dots ,M_1+M_2-1$. Indeed, using the right-hand-side expression in (23) for $f\left(x\right)$ in (28) and for a fixed $m\in \{0,1,\dots ,M_1+M_2-1\}$, we obtaining

$$\begin{array}{cc}\hfill & \frac{1}{4}\sum _{k=0}^{M_1+M_2}{\lambda }_k\left[\sum _{j=1}^{M_1}2c_j\left(cos\left({\varphi }_j(x_0+mh)\right)+cos\left({\varphi }_j(x_0-mh)\right)\right)cos\left({\varphi }_{j}kh\right)\right]\hfill \\ \hfill & +\frac{1}{4}\sum _{k=0}^{M_1+M_2}{\lambda }_k\left[\sum _{l=1}^{M_2}2d_l\left(sin\left({\beta }_l(x_0+mh)\right)+sin\left({\beta }_l(x_0-mh)\right)\right)cos\left({\beta }_{l}kh\right)\right]\hfill \\ \hfill & =\sum _{j=1}^{M_1}{c}_{j}cos\left({\varphi }_j{x}_0\right)cos\left({\varphi }_{j}mh\right)\left(\sum _{k=0}^{M_1+M_2}{\lambda }_{k}cos\left({\varphi }_{j}kh\right)\right)+\sum _{l=1}^{M_2}{d}_{l}sin\left({\beta }_l{x}_0\right)cos\left({\beta }_{l}mh\right)\left(\sum _{k=0}^{M_1+M_2}{\lambda }_{k}cos{\beta }_l\left(kh\right)\right)\hfill \\ \hfill & =\sum _{j=1}^{M_1}{c}_{j}cos\left({\varphi }_j{x}_0\right)cos\left({\varphi }_{j}mh\right)\left(\underset{=0}{\underbrace{\sum _{k=0}^{M_1+M_2}{\lambda }_k{\mathit{T}}_k(cos\left({\varphi }_{j}h\right))}}\right)+\sum _{l=1}^{M_2}{d}_{l}sin\left({\beta }_l{x}_0\right)cos\left({\beta }_{l}mh\right)\left(\underset{=0}{\underbrace{\sum _{k=0}^{M_1+M_2}{\lambda }_k{\mathit{T}}_k(cos\left({\beta }_{l}h\right))}}\right)\hfill \\ \hfill & =0.\hfill \end{array}$$

We can reformulate the system (28) as

$$\begin{array}{cc}\hfill \sum _{k=0}^{(M_1+M_2)-1}& {\lambda }_k\left(f(x_0+(m+k)h)+f(x_0-(m+k)h)+f(x_0+(m-k)h)+f(x_0-(m-k)h)\right)\hfill \\ \hfill & =-2^{1-(M_1+M_2)}(f(x_0+((M_1+M_2)+m)h)+f(x_0-((M_1+M_2)+m)h)\hfill \\ \hfill & +f(x_0+((M_1+M_2)-m)h)+f(x_0-((M_1+M_2)-m)h))\hfill \end{array}$$

(29)

for $m=0,1,\dots ,M_1+M_2-1$. To solve this system, we need $4(M_1+M_2)-1$ sampling values in the form of $f(x_0+kh)$ for $k=-2(M_1+M_2)+1,\dots ,-1,0,1,\dots ,2(M_1+M_2)-1$.

In order to see that the linear system in (29) has a unique solution, we study the $(M_1+M_2)\times (M_1+M_2)$ coefficient matrix in (29), which we denote as $\mathit{H}$. As in the classical Prony method, we can factorize $\mathit{H}$ in the following structure

$$\begin{array}{cc}\hfill \mathit{H}:& ={\left[f(x_0+(m+k)h)+f(x_0-(m+k)h)+f(x_0+(m-k)h)+f(x_0-(m-k)h)\right]}_{m,k=0}^{(M_1+M_2)-1}\hfill \\ \hfill & =4{\left[\sum _{j=1}^{M_1}{c}_{j}cos\left({\varphi }_j{x}_0\right)cos\left({\varphi }_{j}mh\right)cos\left({\varphi }_{j}kh\right)+\sum _{l=1}^{M_2}{d}_{l}sin\left({\beta }_l{x}_0\right)cos\left({\beta }_{l}mh\right)cos\left({\beta }_{l}kh\right)\right]}_{m,k=0}^{(M_1+M_2)-1}\hfill \\ \hfill & =4{\mathit{V}}_h\mathit{D}{\mathit{V}}_h^T,\hfill \end{array}$$

where the Vandermonde Block matrix ${\mathit{V}}_h$ can be written as

$${\mathit{V}}_h:=\left[\begin{array}{ccc}\begin{array}{c}\mathit{A}\end{array}& |& \begin{array}{c}\mathit{B}\end{array}\end{array}\right],$$

(30)

with

$$\mathit{A}:={\left[\begin{array}{ccc}1& \cdots & 1\\ T_1(cos{\varphi }_{1}h)& \cdots & T_1(cos{\varphi }_{M_1}h)\\ ⋮& \cdots & ⋮\\ T_{(M_1+M_2)-1}(cos{\varphi }_{1}h)& \cdots & T_{(M_1+M_2)-1}(cos{\varphi }_{M_1}h)\end{array}\right]}_{(M_1+M_2)\times M_1}$$

(31)

and

$$\mathit{B}:={\left[\begin{array}{ccc}1& \cdots & 1\\ T_1(cos{\beta }_{1}h)& \cdots & T_1(cos{\beta }_{M_2}h)\\ ⋮& \cdots & ⋮\\ T_{(M_1+M_2)-1}(cos{\beta }_{1}h)& \cdots & T_{(M_1+M_2)-1}(cos{\beta }_{M_2}h)\end{array}\right]}_{(M_1+M_2)\times M_2},$$

(32)

and the diagonal block matrix $\mathit{D}$ can be written as

$$\mathit{D}:=\left[\begin{array}{cc}\mathit{D}\mathit{1}& \mathit{0}\\ \mathit{0}& \mathit{D}\mathit{2}\end{array}\right]$$

(33)

where

$$\mathit{D}\mathit{1}:=\left[\begin{array}{cc}{c}_{1}cos\left({\varphi }_1{x}_0\right)& \\ & \ddots & \\ & & c_{M_1}cos\left({\varphi }_{M_1}{x}_0\right)\end{array}\right]$$

(34)

and

$$\mathit{D}\mathit{2}:=\left[\begin{array}{cc}{d}_{1}sin\left({\beta }_1{x}_0\right)& \\ & \ddots & \\ & & d_{M_2}sin\left({\beta }_{M_2}{x}_0\right)\end{array}\right].$$

(35)

Thus, $\mathit{H}$ is guaranteed to be invertible by the conditions $2^{\circ }$ and $3^{\circ }$ of the theorem. Then, we can find the unique solution for ${\left\{{\lambda }_k\right\}}_{k=0}^{M_1+M_2-1}$ from the linear system (29).

With these ${\lambda }_k$ values for $\Lambda \left(z\right)$ as in (26), we can determine ${\varphi }_j$’s and ${\beta }_l$’s from the zeros of $\Lambda \left(z\right)$; however, this step is non-trivial, because we do not know what zeros correspond to ${\varphi }_j$’s and what zeros correspond to ${\beta }_l$’s. In order to resolve this ambiguity, we consider all the possible cases: Among $M_1+M_2$ zeros of $\Lambda \left(z\right)$, $M_1$ of them correspond to ${\varphi }_j$’s. Thus, there are a total $\left(\genfrac{}{}{0pt}{}{M_1+M_2}{M_1}\right)$ possible choices for ${\varphi }_j$’s, among which there is exactly one choice for the solution; however, how do we select the right one? We need to go to the next overdetermined linear system for the answer.

When we determine the coefficients $c_j$’s and $d_l$’s in (23), we have the following linear system

$$\begin{array}{cc}\hfill f(x_0+hn)& =\sum _{j=1}^{M_1}{c}_{j}cos\left({\varphi }_j(x_0+hn)\right)+\sum _{l=1}^{M_2}{d}_{l}sin\left({\beta }_l(x_0+hn)\right)\hfill \end{array}$$

(36)

for $n=-2(M_1+M_2)+1,\dots ,-1,0,...,2(M_1+M_2)-1$ corresponding to all the sampling values, which has $4(M_1+M_2)-1$ equations and $M_1+M_2$ unknowns. This overdetermined linear system gives us the extra information we need to select the true-solution case from the remaining non-solution cases.

Our method is based on an observation: The sampling values $\begin{array}{c}{\left\{f(x_0+nh)\right\}}_{n=-2(M_1+M_2)+1}^{2(M_1+M_2)-1}\end{array}$ are calculated using the original ${\varphi }_j$’s and ${\beta }_l$’s (corresponding to the true-solution case), which means that all the $4(M_1+M_2)-1$ equations in (36) are completely satisfied for the true-solution case. In other words, the least-square solution of (36) for the true-solution case should have this property: Its error term is zero theoretically (or very close to zero due to rounding errors in computation). While the least-square solution for any non-solution case would have a significant (with respect to the rounding errors) nonzero error term, which makes the true solution stand out clearly. □

Our experiments have verified this phenomenon. Based on this observation, we develop a two-stage least-square detection method to minimize the computing cost, and in Section 5, we demonstrate the effectiveness of this method using a simple example.

Remark 1.

The overdetermined linear system (36) plays an important role in determining the true solution from certain number of possible cases. Typically, this situation happens in the multi-generator sparse expansion problem. For the single-generator case, we can select same number of linearly independent equations from the overdetermined system as the number of unknowns to find the solution; however, for the multi-generator case, the redundant equations are very useful in the least-square method.

4. The Sparse Expansion Problem with Two Gaussian Generators

In this section, we solve another two-generator sparse expansion problem as in (7) that uses the two Gaussian generating functions, $e^{-\beta {(\varphi -x)}^2}$ and $e^{-\beta {(\varphi +x)}^2}$, in the form of

$$f\left(x\right)=\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-x)}^2}+\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+x)}^2}$$

(37)

for some constant $\beta \in \mathbb{C}\setminus \left\{0\right\}$. In order to recover the coefficients $c_j\in \mathbb{C}\setminus \left\{0\right\}$ and the parameters ${\varphi }_j$’s, we need $4M$ sampling values $f(x_0+kh),k=0,...,4M-1$, where $x_0\in \mathbb{R}$, and h satisfies the same condition as in Section 2.2.

This two-generator sparse expansion problem has a special property: When ${\varphi }_{j_0}=0$ for some $j_0\in \{1,...,M\}$, the two functions $e^{-\beta {({\varphi }_{j_0}-x)}^2}$ and $e^{-\beta {({\varphi }_{j_0}+x)}^2}$ are the same. This property would cause some problem for our method presented in the previous section. In order to make the discussion easier, we separate these two cases, and consider the case that ${\varphi }_j\in \mathbb{R}\setminus \left\{0\right\}$ for all $j=1,...,M$ first.

Theorem 2.

Assume that a function $f\left(x\right)$ has the two-generator sparse expansion form of (37), where the number of terms M and the constant $\beta \in \mathbb{C}\setminus \left\{0\right\}$ are known, but the coefficients in $\{c_1,\dots ,c_M\}$ and the parameters in $\{{\varphi }_1,\dots ,{\varphi }_M\}$ are unknown. If $4M$ equispaced sampling values of the form $f(x_0+kh)$ for $k=0,1,\dots ,4M-1$ are provided, then the original function $f\left(x\right)$ can be uniquely reconstructed under the following conditions:

$1^{\circ }$ The coefficients $\{c_1,\dots ,c_M\}$ are nonzero in $\mathbb{C}$.

$2^{\circ }$ The parameters $\{{\varphi }_1,\dots ,{\varphi }_M\}$ are nonzero in $(-L,L)\subset \mathbb{R}$ for some $L>0$, and they are distinct.

$3^{\circ }$ If $\mathit{Re}\beta \ne 0$, then $h\in \mathbb{R}\setminus \left\{0\right\}$; while if $\mathit{Re}\beta =0$, then $0<h\le \frac{\pi }{2\left|\mathrm{Im}\beta \right|L}$.

Proof. 

Our method relies on existence of some critical linear operator, such that both generating functions are its eigenfunctions. Here we use the operator ${\mathcal{S}}_{K,h}$ as defined in (16) with $K(x,h):=e^{\beta h(2x+h)}$, which has the following properties:

$$\begin{array}{cc}\hfill \left({\mathcal{S}}_{K,h}{e}^{-\beta {(\varphi -·)}^2}\right)\left(x\right)& =e^{2\beta \varphi h}{e}^{-\beta {(\varphi -x)}^2},\hfill \\ \hfill \left({\mathcal{S}}_{K,h}{e}^{-\beta {(\varphi +·)}^2}\right)\left(x\right)& =e^{-2\beta \varphi h}{e}^{-\beta {(\varphi +x)}^2}.\hfill \end{array}$$

(38)

Clearly $e^{-\beta {({\varphi }_j-·)}^2}$ and $e^{-\beta {({\varphi }_j+·)}^2}$ are eigenfunctions of ${\mathcal{S}}_{K,h}$ for all ${\varphi }_j\in \mathbb{R}\setminus \left\{0\right\}$ with corresponding eigenvalues $e^{2\beta {\varphi }_{j}h}$ and $e^{-2\beta {\varphi }_{j}h}$, respectively, for $j=1,...,M$. Hence we can define the Prony polynomial using all these eigenvalues:

$$\begin{array}{cc}\hfill \Lambda \left(z\right)& =\prod _{j=1}^M(z-e^{2h\beta {\varphi }_j})\prod _{j=1}^M(z-e^{-2h\beta {\varphi }_j})=\sum _{l=0}^{2M}{\lambda }_l{z}^l\hfill \end{array}$$

(39)

with ${\lambda }_{2M}=1$. Since the real number ${\varphi }_j\ne 0$, we can assume that ${\varphi }_j>0$ for all $j=1,...,M$ based on the structure in (37) to improve the certainty without loss of generality.

Then for $m=0,1,...,2M-1$, we calculate

$$\begin{array}{cc}\hfill & \sum _{l=0}^{2M}{\lambda }_l\left({\mathcal{S}}_{K,(l+m)h}f\right)\left(x_0\right)=\sum _{l=0}^{2M}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}f(x_0+h(l+m))\hfill \\ \hfill & =\sum _{l=0}^{2M}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-(x_0+h(l+m)))}^2}+\sum _{l=0}^{2M}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+(x_0+h(l+m)))}^2}\hfill \\ \hfill & =\left(\sum _{j=1}^M{c}_j{e}^{-\beta {(x_0+hm-{\varphi }_j)}^2}{e}^{\beta hm(2x_0+hm)}\right)\underset{=0}{\underbrace{\left(\sum _{l=0}^{2M}{\lambda }_l{e}^{2\beta hl{\varphi }_j}\right)}}+\left(\sum _{j=1}^M{c}_j{e}^{-\beta {(x_0+hm+{\varphi }_j)}^2}{e}^{\beta hm(2x_0+hm)}\right)\underset{=0}{\underbrace{\left(\sum _{l=0}^{2M}{\lambda }_l{e}^{-2\beta hl{\varphi }_j}\right)}}=0,\hfill \end{array}$$

which can be written as the following linear system

$$\sum _{l=0}^{2M-1}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}f(x_0+h(l+m))=-e^{\beta h(m+2M)(2x_0+h(m+2M))}f(x_0+h(m+2M))$$

(40)

for $m=0,1,...,2M-1$. To solve this system, we need $4M$ sampling values: $f(x_0+kh)$ for $=0,1,\dots ,4M-1$. To study existence of the solution for this linear system, we would like to simplify it with respect to the unknown vector $\lambda :={[{\lambda }_0,...,{\lambda }_{2M-1}]}^T$ as follows,

$$\mathit{H}\lambda =-\mathit{G},$$

(41)

with $\mathit{G}:={\left[\left({\mathcal{S}}_{K,(M+m)h}f\right)\left(x_0\right)\right]}_{m=0}^{2M-1}$ and

$$\mathit{H}:={\left[\left({\mathcal{S}}_{K,(l+m)h}f\right)\left(x_0\right)\right]}_{l,m=0}^{2M-1}.$$

(42)

The invertibility of $\mathit{H}$ can be seen from the following matrix factorization:

$$\begin{array}{cc}\hfill \mathit{H}& ={\left[K(x_0,h(l+m))f(x_0+h(l+m))\right]}_{l,m=0}^{2M-1}\hfill \\ \hfill & ={\left[\sum _{j=1}^M{c}_j{e}^{\beta h(l+m)(2x_0+h(l+m))}{e}^{-\beta {({\varphi }_j-(x_0+h(l+m)))}^2}+\sum _{j=1}^M{c}_j{e}^{\beta h(l+m)(2x_0+h(l+m))}{e}^{-\beta {({\varphi }_j+(x_0+h(l+m)))}^2}\right]}_{l,m=0}^{2M-1}\hfill \\ \hfill & ={\left[\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-x_0)}^2}{e}^{2\beta h(l+m){\varphi }_j}+\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+x_0)}^2}{e}^{-2\beta h(l+m){\varphi }_j}\right]}_{l,m=0}^{2M-1}\hfill \\ \hfill & ={\mathit{V}}_{h}diag\left(c_j{e}^{-\beta {({\varphi }_j-x_0)}^2}+c_j{e}^{-\beta {({\varphi }_j+x_0)}^2}\right){\mathit{V}}_h^T\hfill \\ \hfill & ={\mathit{V}}_h\mathit{D}{\mathit{V}}_h^T\hfill \end{array}$$

(43)

where the Vandermonde block matrix ${\mathit{V}}_h$ has the following form

$${\mathit{V}}_h:=\left[\begin{array}{ccc}\begin{array}{c}\mathit{A}\end{array}& |& \begin{array}{c}\mathit{B}\end{array}\end{array}\right]$$

(44)

with

$$\mathit{A}:={\left[\begin{array}{ccc}1& \cdots & 1\\ e^{2\beta h{\varphi }_1}& \cdots & e^{2\beta h{\varphi }_M}\\ ⋮& \cdots & ⋮\\ e^{2(2M-1)\beta h{\varphi }_1}& \cdots & e^{2(2M-1)\beta h{\varphi }_M}\end{array}\right]}_{\left(2M\right)\times M}$$

(45)

and

$$\mathit{B}:={\left[\begin{array}{ccc}1& \cdots & 1\\ e^{-2\beta h{\varphi }_1}& \cdots & e^{-2\beta h{\varphi }_M}\\ ⋮& \cdots & ⋮\\ e^{-2(2M-1)\beta h{\varphi }_1}& \cdots & e^{-2(2M-1)\beta h{\varphi }_M}\end{array}\right]}_{\left(2M\right)\times M},$$

(46)

and the diagonal block matrix $\mathit{D}$ is given by

$$\mathit{D}:=\left[\begin{array}{cc}\mathit{D}\mathit{1}& \mathit{0}\\ \mathit{0}& \mathit{D}\mathit{2}\end{array}\right]$$

(47)

with

$$\mathit{D}\mathit{1}:=\left[\begin{array}{ccc}{c}_1{e}^{\beta {({\varphi }_1-x_0)}^2}& & \\ & \ddots & \\ & & c_M{e}^{\beta {({\varphi }_M-x_0)}^2}\end{array}\right]$$

(48)

and

$$\mathit{D}\mathit{2}:=\left[\begin{array}{ccc}{c}_1{e}^{\beta {({\varphi }_1+x_0)}^2}& & \\ & \ddots & \\ & & c_M{e}^{\beta {({\varphi }_M+x_0)}^2}\end{array}\right].$$

(49)

From this structure, we can see that the Vandermonde matrix ${\mathit{V}}_h$ in (44) is invertible by conditions $2^{\circ }$ and $3^{\circ }$ of the theorem, and hence $\mathit{H}$ in (42) is also invertible by condition $1^{\circ }$, which results in the unique solution for $\lambda$.

With all the ${\lambda }_l$ values found from the above linear system, we can find all the ${\varphi }_j$ values by calculating the zeros of the Prony polynomial of (39). In this case, we do not need to deal with the ambiguity that we encountered in the previous section due to the special structure of the pairs $({\varphi }_j,-{\varphi }_j)$’s. Finally, the coefficients $c_j$’s of the sparse expansion (37) can be computed by solving the following overdetermined linear system:

$$f(x_0+lh)=\sum _{j=1}^M{c}_j\left(e^{-\beta {({\varphi }_j+x_0-lh)}^2}+e^{-\beta {({\varphi }_j+x_0+lh)}^2}\right)$$

(50)

for $l=0,...,4M-1$. □

Remark 2.

Our method above only works for the case when ${\varphi }_j\ne 0$ for all j in $\{1,2,\dots ,M\}$; however, in the real-world situation, when we solve a problem of (37) using $4M$ sampling values, how do we know if there exists any ${\varphi }_j=0$ in it or not? We need a detection method to tell us if all the ${\varphi }_j$’s are nonzero before we apply the above method.

Let us investigate the existence of a solution for the linear system (41), which is determined by the invertibility of $\mathit{H}$ in (42). We notice that when ${\varphi }_1=0$, the first column of (45) and the first column of (46) are the same, which causes the matrix ${\mathit{V}}_h$ in (44) to be singular. Then, we conclude that $\mathit{H}$ in (43) is singular if any ${\varphi }_j=0$. In other words, by checking the invertibility of $\mathit{H}$, we can tell if there is any ${\varphi }_j=0$ for problem (37). If $\mathit{H}$ in (42) is singular, our current method does not work. Fortunately we can modify our method to solve the problem for this special situation.

Let us assume that ${\varphi }_0=0$, and the remaining ${\varphi }_j$’s are positive numbers. In this case, we modify (37) to

$$f\left(x\right)=c_0{e}^{-\beta x^2}+\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-x)}^2}+\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+x)}^2},$$

(51)

and its corresponding Prony polynomial is defined as

$$\begin{array}{cc}\hfill \Lambda \left(z\right)& =(z-1)\prod _{j=1}^M(z-e^{2h\beta {\varphi }_j})\prod _{j=1}^M(z-e^{-2h\beta {\varphi }_j})=\sum _{l=0}^{2M+1}{\lambda }_l{z}^l\hfill \end{array}$$

(52)

with ${\lambda }_{2M+1}=1$. Since $\Lambda \left(1\right)=0$, it leads to

$$\sum _{l=0}^{2M+1}{\lambda }_l=0.$$

(53)

Then we can show that

$$\sum _{l=0}^{2M+1}{\lambda }_l\left({\mathcal{S}}_{K,(l+m)h}f\right)\left(x_0\right)=0,\mathrm{for}m=0,1,...,2M,$$

(54)

because we can split the above left-hand-side summation into the following three summations with zero value each:

$$\begin{array}{cc}\hfill & \sum _{l=0}^{2M+1}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}{c}_0{e}^{-\beta {(x_0+h(l+m))}^2}=c_0{e}^{-\beta x_0^2}\underset{=0}{\underbrace{\sum _{l=0}^{2M+1}{\lambda }_l}}=0,\hfill \end{array}$$

$$\begin{array}{cc}\hfill \sum _{l=0}^{2M+1}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}& \sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-(x_0+h(l+m)))}^2}=\left(\sum _{j=1}^M{c}_j{e}^{-\beta {(x_0+hm-{\varphi }_j)}^2}{e}^{\beta hm(2x_0+hm)}\right)\underset{=0}{\underbrace{\left(\sum _{l=0}^{2M+1}{\lambda }_l{e}^{2\beta hl{\varphi }_j}\right)}}=0,\hfill \end{array}$$

and

$$\begin{array}{cc}\hfill \sum _{l=0}^{2M+1}{\lambda }_l{e}^{\beta h(l+m)(2x_0+h(l+m))}& \sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+(x_0+h(l+m)))}^2}=\left(\sum _{j=1}^M{c}_j{e}^{-\beta {(x_0+hm+{\varphi }_j)}^2}{e}^{\beta hm(2x_0+hm)}\right)\underset{=0}{\underbrace{\left(\sum _{l=0}^{2M+1}{\lambda }_l{e}^{-2\beta hl{\varphi }_j}\right)}}=0.\hfill \end{array}$$

The linear system (54) for $\lambda :={[{\lambda }_0,...,{\lambda }_{2M}]}^T$ can be written as

$$\mathit{H}\lambda =-\mathit{G},$$

(55)

with $\mathit{G}:={\left[\left({\mathcal{S}}_{K,(2M+m+1)h}f\right)\left(x_0\right)\right]}_{m=0}^{2M}$ and

$$\mathit{H}:={\left[\left({\mathcal{S}}_{K,(l+m)h}f\right)\left(x_0\right)\right]}_{l,m=0}^{2M}.$$

(56)

We use $(4M+2)$ sampling values: $f(x_0+kh)$ for $k=0,1,\dots ,4M+1$ to solve the system. Similar to 43, we still have

$$\mathit{H}={\mathit{V}}_h\mathit{D}{\mathit{V}}_h^T,$$

but we need to modify ${\mathit{V}}_h$ to

$${\left[\begin{array}{ccccccc}1& 1& \cdots & 1& 1& \cdots & 1\\ 1& e^{2\beta h{\varphi }_1}& \cdots & e^{2\beta h{\varphi }_M}& e^{-2\beta h{\varphi }_1}& \cdots & e^{-2\beta h{\varphi }_M}\\ ⋮& ⋮& \cdots & ⋮& ⋮& \cdots & ⋮\\ 1& e^{4M\beta h{\varphi }_1}& \cdots & e^{4M\beta h{\varphi }_M}& e^{-4M\beta h{\varphi }_1}& \cdots & e^{-4M\beta h{\varphi }_M}\end{array}\right]}_{(2M+1)\times (2M+1)},$$

which is invertible for positive distinct $\{{\varphi }_1,\dots ,{\varphi }_M\}\subset (0,L)$, and the diagonal block matrix $\mathit{D}$ becomes

$$\mathit{D}=\left[\begin{array}{ccc}{c}_0{e}^{-\beta x_0^2}& \mathit{0}& \mathit{0}\\ \mathit{0}& \mathit{D}\mathit{1}& \mathit{0}\\ \mathit{0}& \mathit{0}& \mathit{D}\mathit{2}\end{array}\right]$$

with $\mathit{D}\mathit{1}$ and $\mathit{D}\mathit{2}$ maintaining the same forms of (48) and (49), respectively.

After we solve the linear system of (55), we obtain the Prony polynomial that contains one zero at $z=1$ and the remaining zeros appear in pairs of $(z_j,z_j^{-1})$’s, which correspond to the parameter values 0 and $({\varphi }_j,-{\varphi }_j)$ pairs. Finally, we will solve the following overdetermined linear system for $c_0,c_1,\dots ,c_M$ values

$$f(x_0+lh)=c_0{e}^{-\beta {(x_0-lh)}^2}+\sum _{j=1}^M{c}_j\left(e^{-\beta {({\varphi }_j+x_0-lh)}^2}+e^{-\beta {({\varphi }_j+x_0+lh)}^2}\right)$$

(57)

for $l=0,...,4M+1$. From this example, we can see that the value of $det\left(\mathit{H}\right)$ can give us some important information, that is, which of the two systems in (37) and (51) we should work on. This property could be useful when we consider a problem in which the M value in (51) is unknown, but restricted in certain range. (See discussion in Section 6).

5. Numerical Experiments

In this section, we use two simple examples to illustrate the implementation details of our method for the two-generator sparse expansion problem described in the previous sections. The first example is for version (23) in Section 3. The second example is for version (37) in Section 4.

Example 1.

We consider a function $f\left(x\right)$ (see Figure 1) that is a two-generator expansion with each generator producing 5 terms in the following form

$$f\left(x\right)=\sum _{j=1}^5{c}_{j}cos\left({\varphi }_{j}x\right)+\sum _{j=1}^5{d}_{j}sin\left({\beta }_{j}x\right),$$

(58)

and the 20 parameters we used are listed in the table below to generate the sampling values.

Figure 1. The signal $f\left(x\right)$ in (58) with 39 equispaced sampling values.

How to use the 39 equispaced sampling values (where 39 comes from $4(5+5)-1$) in the form of $f(x_0+kh),k=-19,\dots ,0,\dots ,19$ to recover the original parameters in Table 1?

Table 1. Original parameters of the function $f\left(x\right)$ in (58).

j	${\mathit{c}}_{\mathit{j}}$	${\mathit{d}}_{\mathit{j}}$	${\mathit{\varphi }}_{\mathit{j}}$	${\mathit{\beta }}_{\mathit{j}}$
1	−2	5	2	3
2	3	−6	4	5
3	−4	4	7	6
4	8	−3	8	9
5	7	2	10	11

There are 20 original parameters in two sets: $\{c_1,\dots ,c_5,{\varphi }_1,\dots ,{\varphi }_5\}$ and $\{d_1,\dots ,d_5,{\beta }_1,\dots ,{\beta }_5\}$ corresponding to the two generators, respectively. To recover them, first we solve the following linear system for the coefficients of the Prony polynomial $\{{\lambda }_0,\dots ,{\lambda }_9\}$ based on the Equation (29)

$$\mathit{H}\lambda =-\mathit{G},$$

where

$$\mathit{H}=\left[\begin{array}{cccccccccc}-36.3064& 24.4399& 35.5543& -40.2183& -16.8633& 18.9503& -0.7573& 18.8146& 4.4668& -52.7171\\ 24.4399& -0.3760& -7.8892& 9.3455& -10.6340& -8.8103& 18.8824& 1.8548& -16.9513& -9.4925\\ 35.5543& -7.8892& -26.5849& 21.6951& 17.3985& -10.7019& -6.1982& -16.8834& -12.1046& 24.6062\\ -40.218& 39.3455& 21.6951& -18.5319& 21.6273& 20.0106& -46.4677& -20.1576& 24.6741& 11.4779\\ -16.8633& -10.6340& 17.3985& 21.6273& -15.9198& -14.1386& 6.0512& -4.9102& 3.4249& 17.8445\\ 18.9503& -8.8103& -10.7019& 20.0106& -14.1386& -29.8792& 27.4189& 29.6337& -11.7398& -2.7658\\ -0.7573& 18.8824& -6.1982& -46.4677& 6.0512& 27.4189& -6.2967& 20.5893& 23.4431& -32.0445\\ 18.8146& 1.8548& -16.8834& -20.1576& -4.9102& 29.6337& 20.5893& -12.4873& 0.2846& 0.1035\\ 4.4668& -16.9513& -12.1046& 24.6741& 3.4249& -11.7398& 23.4431& 0.2846& -35.8269& 11.5787\\ -52.7171& -9.4925& 24.6062& 11.4779& 17.8445& -2.7658& -32.0445& 0.1035& 11.5787& -9.3042\end{array}\right]$$

and

$$\mathit{G}={\left[\begin{array}{cccccccccc}0.0458& 0.0218& -0.0275& -0.0347& -0.0103& 0.0048& 0.0510& 0.0405& -0.0520& -0.0412\end{array}\right]}^T.$$

We obtain

$$\lambda ={\left[\begin{array}{cccccccccc}-0.0088& 0.0275& -0.0639& 0.1254& -0.2113& 0.3180& -0.4300& 0.5316& -0.6010& 0.3135\end{array}\right]}^T,$$

which corresponds to the following Prony polynomial

$$\Lambda \left(z\right)=\begin{array}{c}{z}^{10}+0.3135z^9-0.6010z^8+0.5316z^7-0.4300z^6+0.3180z^5-0.2113z^4+0.1254z^3--0.0639z^2+0.0275z-0.0088\end{array}.$$

From the 10 zeros of this polynomial, we obtain 10 parameter values:

$$\{11.0000,2.0000,3.0000,10.0000,4.0000,5.0000,9.0000,6.0000,7.0000,8.0000\},$$

(59)

which correspond to $\{{\varphi }_1,\dots ,{\varphi }_5,{\beta }_1,\dots ,{\beta }_5\}$, but the explicit order is unknown. We must resolve the ambiguity: What five parameter values are for $\{{\varphi }_1,\dots ,{\varphi }_5\}$ (with the remaining five parameter values for $\{{\beta }_1,\dots ,{\beta }_5\}$)?

To separate the ${\varphi }_j$’s from ${\beta }_l$’s, we consider the following overdetermined linear system:

$$\left[\begin{array}{cccccc}cos\left({\varphi }_1{x}_0\right)& \cdots & cos\left({\varphi }_5{x}_0\right)& sin\left({\beta }_1{x}_0\right)& \cdots & sin\left({\beta }_5{x}_0\right)\\ cos\left({\varphi }_1{x}_1\right)& \cdots & cos\left({\varphi }_5{x}_1\right)& sin\left({\beta }_1{x}_1\right)& \cdots & sin\left({\beta }_5{x}_1\right)\\ ⋮& ⋮& ⋮& \cdots & ⋮& ⋮\\ cos\left({\varphi }_1{x}_{19}\right)& \cdots & cos\left({\varphi }_5{x}_{19}\right)& sin\left({\beta }_1{x}_{19}\right)& \cdots & sin\left({\beta }_5{x}_{19}\right)\end{array}\right]\left[\begin{array}{c}{c}_1\\ ⋮\\ c_5\\ d_1\\ ⋮\\ d_5\end{array}\right]=\left[\begin{array}{c}{f}_0\\ f_1\\ ⋮\\ f_{19}\end{array}\right],$$

(60)

where we use the shorthand notations

$$x_n=x_0+nh\mathrm{and}{f}_n=f(x_0+nh)$$

for $n=0,1,\dots ,19$. Note: In this linear system, we only use 20 out of 39 original sampling values, which is adequate for this particular example. It is a trade-off issue between the accuracy of computation and the cost of computation (in time). In general, the more redundant equations we use, the more accuracy we can achieve in searching for the true solution. In other words, if we can obtain adequate accuracy, we focus on cutting the computation cost to the minimum. We do not solve this overdetermined linear system by the least-square method directly. We split these 20 equations into two parts: In the first part, we approximate the coefficients $\{c_1,\dots ,c_5,d_1,\dots ,d_5\}$ in (60) by the least-square method. Then we apply these derived coefficients to the equations in the second part so as to filter out the true solution.

Among the 10 values in (59), every time we select 5 of them for $\{{\varphi }_1,\dots ,{\varphi }_5\}$, the remaining 5 numbers are automatically for $\{{\beta }_1,\dots {\beta }_5\}$. We will have total 252 possible choices (which is the combinatorial number $\left(\genfrac{}{}{0pt}{}{10}{5}\right)$) as the candidates for the solution. Notice that this combinatorial number is a relatively big number. In order to speed up the processing, we reduce the redundant computation to the minimum. Let us use the notations $\{{\varphi }_1^i,\dots ,{\varphi }_5^i,{\beta }_1^i,\dots {\beta }_5^i\}$ with $i=1,2,\dots ,252$ representing those 252 candidates. Our method is based on the property that the information given in the sampling values has a lot of redundancy for selecting the true solution, and we only use just enough information from the given sampling values so as to save the computation time.

First, when we calculate the coefficients $\{c_1,\dots ,c_5,d_1,\dots ,d_5\}$ by the least-square method, we use exactly 10 equations (the same number of the coefficients) out of the 20 equations in (60). Based on our experiments, we do not have to use an overdetermined system for a good approximation by the least-square method. A determined system can give us excellent approximation for the least-square problem, while any underdetermined system usually does not approximate the data well through the least-square solution. For convenience, we select 10 consecutive equations in (60) somewhere in the middle, which we call the least-square block in our discussion, to approximate the coefficients $\{c_1,\dots ,c_5,d_1,\dots ,d_5\}$. Specifically, our least-square block takes the subscripts from 6 through 15, and the corresponding sampling values $\{f_6,f_7,\dots ,f_{15}\}$ should be selected as a reduced linear system given below,

$$\left[\begin{array}{cccccc}cos\left({\varphi }_1{x}_6\right)& \cdots & cos\left({\varphi }_5{x}_6\right)& sin\left({\beta }_1{x}_6\right)& \cdots & sin\left({\beta }_5{x}_6\right)\\ cos\left({\varphi }_1{x}_7\right)& \cdots & cos\left({\varphi }_5{x}_7\right)& sin\left({\beta }_1{x}_7\right)& \cdots & sin\left({\beta }_5{x}_7\right)\\ ⋮& ⋮& ⋮& \cdots & ⋮& ⋮\\ cos\left({\varphi }_1{x}_{15}\right)& \cdots & cos\left({\varphi }_5{x}_{15}\right)& sin\left({\beta }_1{x}_{15}\right)& \cdots & sin\left({\beta }_5{x}_{15}\right)\end{array}\right]\left[\begin{array}{c}{c}_1\\ ⋮\\ c_5\\ d_1\\ ⋮\\ d_5\end{array}\right]=\left[\begin{array}{c}{f}_6\\ f_7\\ ⋮\\ f_{15}\end{array}\right].$$

(61)

Even if our new linear system (61) is a determined system, we still solve it for a least-square solution, because the determinant of the square matrix in (61) could be very close to zero. Then the remaining equations in (60) together with the coefficients derived from (61) will be used to detect which candidate is the true solution based on the error information.

For each set of values $\{{\varphi }_1^i,\dots ,{\varphi }_5^i,{\beta }_1^i,\dots ,{\beta }_5^i\}$ among the 252 candidates, the least-square solution for the linear system (61) would produce the 10 coefficients ${[c_1^i,\dots ,c_5^i,d_1^i,\dots ,d_5^i]}^T$, and we evaluate the following vector

$$\left[\begin{array}{c}{f}_0^i\\ f_1^i\\ ⋮\\ f_{19}^i\end{array}\right]:=\left[\begin{array}{cccccc}cos\left({\varphi }_1^i{x}_0\right)& \cdots & cos\left({\varphi }_5^i{x}_0\right)& sin\left({\beta }_1^i{x}_0\right)& \cdots & sin\left({\beta }_5^i{x}_0\right)\\ cos\left({\varphi }_1^i{x}_1\right)& \cdots & cos\left({\varphi }_5^i{x}_1\right)& sin\left({\beta }_1^i{x}_1\right)& \cdots & sin\left({\beta }_5^i{x}_1\right)\\ ⋮& \ddots & ⋮& ⋮& \ddots & ⋮\\ cos\left({\varphi }_1^i{x}_{19}\right)& \cdots & cos\left({\varphi }_5^i{x}_{19}\right)& sin\left({\beta }_1^i{x}_{19}\right)& \cdots & sin\left({\beta }_5^i{x}_{19}\right)\end{array}\right]\left[\begin{array}{c}{c}_1^i\\ ⋮\\ c_5^i\\ d_1^i\\ ⋮\\ d_5^i\end{array}\right],$$

which is in general different from the original sampling vector ${[f_0,f_1,\dots ,f_{19}]}^T$. Then we will calculate the difference of these two vectors, and see how close they are. We define the error vector as follows:

$$\left[\begin{array}{c}{ϵ}_0^i\\ {ϵ}_1^i\\ ⋮\\ {ϵ}_{19}^i\end{array}\right]:=\left[\begin{array}{c}|f_0^i-f_0|\\ |f_1^i-f_1|\\ ⋮\\ |f_{19}^i-f_{19}|\end{array}\right].$$

(62)

To search for the true solution among the 252 candidates, we discover an intrinsic property, shown in Figure 2 and Figure 3, that can clearly separate the true solution from other candidates.

Figure 2. Display the error vector for one of the 252 candidates.

Figure 3. Display the error vector for the true solution with two reference points at the ends.

In Figure 2, we plot the error vector for one of the 252 candidates to view its typical behavior. The error values in the least-square block (with subscripts from 6 to 15) are very close to zero for a typical candidate; however, the error values that are out of the least-square block (with subscripts from 0 to 5 and from 16 to 19) are not close to zero in general for a candidate that is not the true solution.

This behavior can be explained in this way: The errors in the least-square block are usually very small due to the fact that the least-square solution of the determined system approximates the targeting sampling values $\{f_6,f_7,\dots ,f_{15}\}$ quite well; however, when we consider an error for a sampling value out of the least-square block, since the corresponding equation is not involved in the least-square approximation, there is no reason for this equation to generate a value that is very close to the targeting sampling value.

While for the true solution case, the behavior is different in the sense that the errors for all the equations in the linear system (60) are very close to zero (see Figure 3, and ignore the two reference points at the ends). Let us summarize the key property that helps us to find out the true solution among all the candidates: For a candidate, if the coefficients generated from the determined linear system (61) by the least-square method cannot approximate just one sampling value out of the least-square block well, then it cannot be the true solution.

However, if the coefficients for one candidate can approximate one particular sampling value out of the least-square block well, we can only say that it is highly likely that this candidate could be the true solution, because the probability for a non-solution candidate to approximate some sampling value out of the least-square block well is very small. Based on this observation from our experiments, we design the following strategy for the solution search.

Strategy: Eliminate as many as possible candidates in the first round filtering in two steps: Step 1. Select a determined linear system from the overdetermined linear system in (60) (as the least-square block), and approximate the coefficients $\{c_1,\dots ,c_5,d_1,\dots ,d_5\}$ by the least-square method for each of the 252 candidates. Step 2. Apply the derived coefficients in Step 1 on one of the linear equations out of the least-square block to approximate the targeting sampling value and calculate the error with the targeting sampling value. If the error is greater than certain threshold (we use $0.1$ as our threshold), we drop this candidate from the consideration; otherwise, this candidate passes the first round filtering. If only one candidate survives the first round filtering, it must be the true solution. If more than one candidates pass the first round filtering, we need to do the second round filtering. In the second round filtering, we simply apply the derived coefficients on another linear equation out of the least-square block, and calculate the error for the targeting sampling value. If the error is greater than the threshold, we eliminate this candidate. We keep doing these cycles until we identify the true solution. Since we have plenty of redundant equations out of the least-square block, we should be able to determine the true solution without going through too many cycles in general. Furthermore, those linear equations corresponding to the original sampling values that are not included in the linear system (60) can still be used for the above steps when necessary, but the probability to use those equations out of the linear system (60) will be extremely small. This simple strategy is designed to allow us to detect the true solution without unnecessary computation, while we still preserve the option to use the redundant information when necessary.

Here we would like to point out that as soon as we select values in $\varphi$-group or $\beta$-group, the order of those values in each group is not important, because their corresponding coefficients ($c_j$’s or $d_l$’s) will also be aligned with them accordingly when we solve the determined linear system (61) using the least-square method.

Example 2.

Our second function to be recovered has the following form

$$\begin{array}{c}\hfill f\left(\omega \right)=\frac{c_1}{2}\left(e^{-\frac{1}{2}{({\varphi }_1-\omega )}^2}+e^{-\frac{1}{2}{({\varphi }_1+\omega )}^2}\right)\\ \hfill +\frac{c_2}{2}\left(e^{-\frac{1}{2}{({\varphi }_2-\omega )}^2}+e^{-\frac{1}{2}{({\varphi }_2+\omega )}^2}\right)\\ \hfill +\frac{c_3}{2}\left(e^{-\frac{1}{2}{({\varphi }_3-\omega )}^2}+e^{-\frac{1}{2}{({\varphi }_3+\omega )}^2}\right),\end{array}$$

(63)

which is derived by applying the STFT on the following function

$$g\left(x\right)=\sum _{j=1}^3{c}_{j}cos\left({\varphi }_{j}x\right),$$

(64)

with the parameters of (64) listed in the following Table 2.

Table 2. Parameters of the function $f\left(x\right)$ in (64).

j	${\mathit{c}}_{\mathit{j}}$	${\mathit{\varphi }}_{\mathit{j}}$
1	$0.5000$	$1.0000$
2	$0.2500$	$3.0000$
3	$1.0000$	$4.0000$

To solve this problem, we need to use 12 (i.e., $4M$) sampling values. After we applied the method described in Section 4, we solved a linear system with 6 unknowns, and derived the Prony polynomial of degree 6 as follows

$$\begin{array}{c}\Lambda \left(z\right)=1.0000(z^6+1)-14.4845(z^5+z)+65.9809(z^4+z^2)+108.8070z^3\end{array}.$$

The symmetric structure of this polynomial tells us that its zeros appear in $(z_j,z_j^{-1})$ pairs for $j=1,2,3$, which correspond to three pairs of parameters: $(1.0000,-1.0000),(3.0000,-3.0000)$, and $(4.0000,-4.0000)$ for $({\varphi }_j,-{\varphi }_j),j=1,2,3$. Finally, we can solve another linear system for the coefficients $c_j$’s with the errors listed in the Table 3.

Table 3. Parameters of the function $f\left(x\right)$ in (64) and approximate errors using 12 sampling values with $h=0.5$.

j	${\mathit{c}}_{\mathit{j}}$	${\mathit{\varphi }}_{\mathit{j}}$	$|{\mathit{c}}_{\mathit{j}}-{\mathit{c}}_{\mathit{j}}^{*}|$	$|{\mathit{\varphi }}_{\mathit{j}}-{\mathit{\varphi }}_{\mathit{j}}^{*}|$
1	0.5000	1.0000	$3.7970.{10}^{-2}$	$5.2824.{10}^{-13}$
2	0.2500	3.0000	$5.0987.{10}^{-14}$	$4.5652.{10}^{-13}$
3	1.0000	4.0000	$5.8065.{10}^{-14}$	$1.4211.{10}^{-14}$

6. Conclusions

In this paper, we introduce a method that extends the Prony method to solve the two-generator sparse expansion problem. This method relies on the existence of a special linear operator for which the two generators must be the eigenfunctions. This two-generator problem has a special property: The zeros of its Prony polynomial correspond to two sets of parameters, and there is no straightforward way to separate them. We propose a two-stage least-square detection method on an overdetermined linear system for each candidate to extract the true solution, which relies on an intrinsic property for the true solution: Only the true solution can use the coefficients derived from the least-square block to approximate the targeting sampling values out of the least-square block well. Our method is designed to minimize the computation cost, while still maintain the computation accuracy.

It seems that the idea can be extended to the k-generator sparse expansion problem for $k>2$; however, for the general k-generator case, the requirement that there exists a linear operator such that all the generators must be its eigenfunctions becomes extremely hard to achieve. For example, in the following sparse expansion problem,

$$f\left(x\right)=\sum _{j=1}^{M_1}{c}_{j}cos\left({\varphi }_{j}x\right)+\sum _{l=1}^{M_2}{d}_l{e}^{{\beta }_{l}x},$$

(65)

it is not easy to find a linear operator, such that both $cos\left(\varphi x\right)$ and $e^{\beta x}$ are its eigenfunctions. One may argue that the problem could be solved by converting $cos\left(\varphi x\right)$ to $\frac{1}{2}(e^{i\varphi x}+e^{-i\varphi x})$, and then it becomes a one-generator problem. Notice that converting a two-generator problem to a one-generator problem may not work most of the time. We are interested in developing a general method that can solve the two-generator sparse expansion problem including the one in (65). We can see that there are many difficult problems to be solved in this multi-generator sparse expansion problem, and we would like to see more researchers contribute in this direction.

Our method for the two-generator sparse expansion problem can handle certain degree of uncertainty. For example, in problem (23), if we know the total number of terms (i.e., the value of $M_1+M_2$), but we do not know the number of terms in each summation (i.e., the individual values of $M_1$ and $M_2$), we can still solve the problem using our two-stage least-square detection method described in Section 3 and Section 5. If we increase the uncertainty a little more, can we still solve the problem?

For example, in the problem we considered in Section 4, if we do not know the exact number of terms (it is referred to unknown order of sparsity M in [1]) in the following expansion,

$$f\left(x\right)=\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j-x)}^2}+\sum _{j=1}^M{c}_j{e}^{-\beta {({\varphi }_j+x)}^2},$$

and we are given K equispaced sampling values for some positive integer K. If we are told that these sampling values are sufficient to recover the signal, how do we recover it? In other words, we know that the number of terms M is in the range $1\le M\le \lfloor K/4\rfloor$, but we do not know the exact number M, can we solve the problem? The answer is yes, because we can try all the possible cases: $M=1,2,\dots ,\lfloor K/4\rfloor$, and for each case, we apply our two-stage least-square detection method to tell us if the true solution can be extracted.

However, we are not satisfied with this kind of exhaustive search type solution due to its high cost. We plan to develop an efficient term number detection method, so that when we make a term number prediction, this method can tell us if it is correct or not immediately. In [1], two methods are proposed: One is based on the rank of the $\mathit{H}$ matrix, and the other is based on the singular values of the $\mathit{H}$ matrix. The main issue is: How to obtain a reliable method to determine the M value in the sparse expansion? Only after we obtain the correct term number we will pay the computation cost to go through all the necessary details to find the solution.