Karhunen–Loève Expansion Using a Parametric Model of Oscillating Covariance Function

Abstract

The Karhunen–Loève (KL) expansion decomposes a stochastic process into a set of orthogonal functions with random coefficients. The basic idea of the decomposition is to solve the Fredholm integral equation associated with the covariance kernel of the process. The KL expansion is a powerful mathematical tool used to represent stochastic processes in a compact and efficient way. It has wide applications in various fields including signal processing, image compression, and complex systems modeling. The KL expansion has high computational complexity, especially for large data sets, since there is no single unique transformation for all stochastic processes and there are no fast algorithms for computing eigenvalues and eigenfunctions. One way to solve this problem is to use parametric models for the covariance function. In this paper, an explicit analytical solution of the KL expansion for a parametric model of oscillating covariance function with a small number of parameters is proposed. This model approximates the covariance functions of real images very well. Computer simulation results using a real image are presented and discussed.

1. Introduction

The Karhunen–Loève (KL) expansion [1,2] is an extension of the Fourier series from deterministic functions to random processes. The KL expansion is a representation in which the process is decomposed into a series of orthogonal functions similar to Fourier series. The process of finding the coefficients consists of minimizing the mean square error of the final representation. While for Fourier analysis, the coefficients are deterministic and the orthogonal functions are trigonometric functions, the coefficients for the KL expansion are random and the orthogonal functions are derived from solution of a Fredholm integral equation of the first type, whose kernel is the covariance function of the stationary process associated with the covariance function of the process [3].

The KL expansion is a powerful mathematical tool used to represent random processes in a compact and efficient way. It has several important properties: (i) the KL expansion provides an optimal representation of the random process in terms of mean square error; (ii) the resulting random coefficients are uncorrelated, which simplifies the analysis and modeling of the random process; and (iii) the KL expansion converges in the mean square sense, so in practical applications, it is important to determine the number of terms needed to achieve a certain level of accuracy. The KL expansion has applications in various fields, including signal processing and denoising [4], image compression and feature extraction [5], and modeling of complex systems [6].

Unfortunately, there are no simple methods for computing the eigenfunctions, and there is no single KL expansion for all stochastic processes. However, it is possible to approximate the KL expansion (with an infinite coordinate space) of a stationary process in a wide sense using orthogonal transforms with a finite coordinate space, such as discrete sinusoidal transforms [7,8]. The use of this type of transform was justified by the fact that it belongs to the family of transforms asymptotically equivalent to the KL transform of a first-order Markov process [9].

Another way to solve this problem is to use parametric models to compute the orthogonal eigenfunctions by solving the integral equation analytically. Such solutions have been found for the case of exponential [10,11] and exponential-cosine [12] covariance function models. In this paper, we propose an analytical solution to the integral equation for a parametric model of oscillating covariance function with a small number of parameters. This model approximates the empirical covariance function of real images much more accurately than the exponential and exponential-cosine models. Consequently, the KL expansion is also more accurate, and the obtained explicit eigenfunctions and eigenvalues depend on a small number of model parameters.

The contributions of this work are as follows:

• An analytical solution of the integral KL equation for a parametric model of oscillating covariance function of a stationary process is obtained.
• Based on the solution, explicit eigenfunctions and eigenvalues are derived as functions of a small number of model parameters.
• Using computer simulation, the performance of the proposed method is illustrated using a real image.

The rest of this paper is organized as follows. Section 2 recalls preliminaries on the KL expansion. Section 3 presents an analytical solution of the integral equation. Section 4 illustrates the eigenvalues and eigenfunctions for a real image using computer simulation. Section 5 summarizes our conclusions.

2. Preliminaries on KL Expansion

Let $E\left[.\right]$ denote the ensemble mean and let $m\left(x\right)=E\left[n\left(x,v\right)\right]$ and $K\left(x,u\right)=E\left[\left(n\left(x,v\right)-m\left(x\right)\right)\left(n\left(u,v\right)-m\left(u\right)\right)\right]$. Here, $K\left(x,u\right)$ is the covariance function of the process n(x, v). If the covariance function is continuous in the square $-T\le x,u\le T$, then the process is second-order continuous in $-T\le x\le T$, and if, in addition, $K\left(x,u\right)=K\left(\left|x-u\right|\right)$, then it is second-order stationary. From Karhunen’s theorem [1,3], any second-order random function n(x, v) that is second-order continuous in $-T\le x\le T$ can be expanded as follows:

$$n\left(x,v\right)=m\left(x\right)+{\displaystyle \sum _{k=1}^{\infty }\sqrt{{\lambda }_{\mathrm{k}}}{\xi }_k\left(v\right){\phi }_k\left(x\right)},$$

(1)

with convergence in the mean for x in [−T, T]. The quantities ${\lambda }_k$ and ${\phi }_k\left(x\right)$ are determined from the KL integral equation

$$\lambda \phi \left(x\right)={\displaystyle \underset{-T}{\overset{T}{\int }}K\left(\left|x-u\right|\right)\phi \left(u\right)du},$$

(2)

where K(x) is the covariance function of a continuous stationary second-order process with the power spectral density $S\left(w\right)$, $-T\le x\le T$. Here, $\left\{{\phi }_k\left(x\right)\right\}$ are orthonormal over $-T\le x\le T$ and $\left\{{\xi }_k\left(v\right)\right\}$ are normalized uncorrelated random variables; that is,

$$E\left[{\xi }_k\left(v\right){\xi }_l\left(v\right)\right]={\delta }_{kl};{\displaystyle \underset{-T}{\overset{T}{\int }}{\phi }_k\left(x\right){\phi }_l\left(x\right)dx}={\delta }_{kl},$$

(3)

where k, l ${\mathit{\in } \mathit{\mathbb{N}}, \delta }_{kl}$ is the Kronecker delta.

The covariance function of a second-order stationary process for our parametric oscillatory model has the form

$$K\left(x\right)=P{e}^{-\alpha \left|x\right|}\left[\cos \left(\beta x\right)+\gamma \sin \left(\beta \left|x\right|\right)\right],$$

(4)

where P is the variance of the process, and $\alpha >0,\beta \ge 0$ and $\gamma$ are the model parameters. The power spectral density can be expressed as (see Appendix A)

$$S\left(w\right)={\displaystyle \frac{2P\left(\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\gamma \beta \right)+w^2\left(\alpha -\gamma \beta \right)\right)}{w^4+2w^2\left({\alpha }^2-{\beta }^2\right)+{\left({\alpha }^2+{\beta }^2\right)}^2}},$$

(5)

and it is always a real, non-negative, and even function of the frequency $w$.

It can be shown that the eigenfunctions generated by such kernels using Equation (2) are complete in $-T\le x\le T$, and the eigenvalues are positive.

3. Solution to KL Integral Equation

To solve the KL integral in Equation (2) with a parametric oscillating covariance function, the integral equation is first converted into a differential equation. To do this, we use the power spectral density given in Equation (5) and the Fourier transform property: differentiating a function in the signal domain corresponds to multiplying the function spectrum by $jw$. After finding the solution to the differential equation, we substitute the solution back into the integral equation to satisfy the boundary conditions.

Let us substitute Equation (4) into Equation (2) and eliminate the magnitude sign as follows:

$$\begin{array}{l}\lambda \phi \left(x\right)={\displaystyle \underset{-T}{\overset{T}{\int }}P{e}^{-\alpha \left|x-u\right|}\left[\cos \left(\beta \left(x-u\right)\right)+\gamma \sin \left(\beta \left|x-u\right|\right)\right]}\phi \left(u\right)du=\\ {\displaystyle \underset{-T}{\overset{x}{\int }}P{e}^{-\alpha \left(x-u\right)}\left[\cos \left(\beta \left(x-u\right)\right)+\gamma \sin \left(\beta \left(x-u\right)\right)\right]}\phi \left(u\right)du + ,\\ {\displaystyle \underset{x}{\overset{-T}{\int }}P{e}^{\alpha \left(x-u\right)}\left[\cos \left(\beta \left(u-x\right)\right)+\gamma \sin \left(\beta \left(u-x\right)\right)\right]}\phi \left(u\right)du\end{array}$$

(6)

where $-T\le x,u\le T.$

Applying the Fourier transform property (differentiation of a function in the signal domain corresponds to multiplication of the function spectrum by $jw$) to the power spectral density given in Equation (5), we obtain a fourth-order linear differential equation with constant coefficients (see Appendix B):

$${\phi }^{(4)}\left(x\right)-2\left({\alpha }^2-{\beta }^2+{\displaystyle \frac{\left(\beta \gamma -\alpha \right)P}{\lambda }}\right){\phi }^{(2)}\left(x\right)+\left({\alpha }^2+{\beta }^2\right)\left({\alpha }^2+{\beta }^2-{\displaystyle \frac{2\left(\beta \gamma +\alpha \right)P}{\lambda }}\right)\phi \left(x\right)=0,$$

(7)

for $\lambda \ne 0.$ The integral equation satisfies the following condition: $0<\widehat{\lambda }<2\left(\beta \gamma +\alpha \right)/\left({\alpha }^2+{\beta }^2\right)$ with $\widehat{\lambda }=\lambda /P$. The roots of the characteristic equation are given by the formulas

$${b^2}_{1,2}=\left(\sqrt{D}-\left({\alpha }^2-{\beta }^2+{\displaystyle \frac{\beta \gamma -\alpha }{\widehat{\lambda }}}\right)\right);{b^2}_{3,4}=\left(\left({\alpha }^2-{\beta }^2+{\displaystyle \frac{\beta \gamma -\alpha }{\widehat{\lambda }}}\right)+\sqrt{D}\right),$$

(8)

where $D=\left({\alpha }^2-{\beta }^2+{\displaystyle \frac{\beta \gamma -\alpha }{\widehat{\lambda }}}\right)-\left({\alpha }^2+{\beta }^2\right)\left({\alpha }^2+{\beta }^2-{\displaystyle \frac{2\left(\beta \gamma +\alpha \right)}{\widehat{\lambda }}}\right)>0$.

A general solution of the integral equation can be written as follows:

$$\phi \left(x\right)=C_1{e}^{i{b}_{1}x}+C_2{e}^{i{b}_{2}x}+C_3{e}^{b_{3}x}+C_4{e}^{b_{4}x},$$

(9)

where $b_2=-b_1;b_4=-b_3$ are the imaginary and real solutions of the characteristic equation, respectively. Note that the real solutions are related to the imaginary solution as follows:

$${b_3}^2={\displaystyle \frac{\left({\alpha }^2+{\beta }^2\right)\left(3{\alpha }^2\beta \gamma -{\beta }^3\gamma +{\alpha }^3-3\alpha {\beta }^2+{b_1}^2\left(\alpha +\beta \gamma \right)\right)}{\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\beta \gamma \right)+{b_1}^2\left(\alpha -\beta \gamma \right)}}.$$

(10)

Moreover, the normalized eigenvalues $\widehat{\lambda }$ can be calculated through the imaginary and real roots, respectively, as follows:

$$\widehat{\lambda }={\displaystyle \frac{2\left(\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\beta \gamma \right)+{b_1}^2\left(\alpha -\beta \gamma \right)\right)}{{\left({\alpha }^2+{\beta }^2\right)}^2+{b_1}^2\left(2{\alpha }^2-2{\beta }^2+{b_1}^2\right)}}={\displaystyle \frac{2\left(\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\beta \gamma \right)+{b_3}^2\left(\beta \gamma -\alpha \right)\right)}{{\left({\alpha }^2+{\beta }^2\right)}^2-{b_3}^2\left(2{\alpha }^2-2{\beta }^2-{b_3}^2\right)}}.$$

(11)

Let us introduce the following notations:

$$K{L}_1=\lambda \left(C_1{e}^{i{b}_{1}x}+C_2{e}^{-i{b}_{1}x}\right)-{\displaystyle \underset{-T}{\overset{T}{\int }}K\left(\left|x-u\right|\right)\left(C_1{e}^{i{b}_{1}u}+C_2{e}^{-i{b}_{1}u}\right)du},$$

(12)

and

$$K{L}_2=\lambda \left(C_3{e}^{b_{3}x}+C_4{e}^{-b_{3}x}\right)-{\displaystyle \underset{-T}{\overset{T}{\int }}K\left(\left|x-u\right|\right)\left(C_3{e}^{b_{3}u}+C_4{e}^{-b_{3}u}\right)du},$$

(13)

with $K{L}_1+K{L}_2=0.$

Substituting Equation (11) for imaginary roots into Equations (12) and performing integration, we obtain

$$\begin{array}{ll}K{L}_1=& \left\{\left[i{b}_1\left(\gamma b_1^2+\left({\alpha }^2\gamma -{\beta }^2\gamma -2\alpha \beta \right)\right)\left(C_1{e}^{-iT{b}_1}-C_2{e}^{iT{b}_1}\right)-\right.\right.\\ & \left.\left(\left(\alpha \gamma +\beta \right)b_1^2+\left({\alpha }^2+{\beta }^2\right)\left(\alpha \gamma -\beta \right)\right)\left(C_1{e}^{-iT{b}_1}+C_2{e}^{iT{b}_1}\right)\right]e^{-\alpha \left(T+x\right)}\sin \left(\beta \left(T+x\right)\right)+\\ & \left[i{b}_1\left(\gamma b_1^2+\left({\alpha }^2\gamma -{\beta }^2\gamma -2\alpha \beta \right)\right)\left(C_2{e}^{-iT{b}_1}-C_1{e}^{iT{b}_1}\right)-\right.\\ & \left.\left(\left(\alpha \gamma +\beta \right)b_1^2+\left({\alpha }^2+{\beta }^2\right)\left(\alpha \gamma -\beta \right)\right)\left(C_2{e}^{-iT{b}_1}+C_1{e}^{iT{b}_1}\right)\right]e^{-\alpha \left(T-x\right)}\sin \left(\beta \left(T-x\right)\right)+\\ & \left[i{b}_1\left(b_1^2+\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2\right)\right)\left(C_1{e}^{-iT{b}_1}-C_2{e}^{iT{b}_1}\right)+\right.\\ & \left.\left(b_1^2\left(\beta \gamma -\alpha \right)-\left({\alpha }^2+{\beta }^2\right)\left(\beta \gamma +\alpha \right)\right)\right]\left(C_1{e}^{-iT{b}_1}+C_2{e}^{iT{b}_1}\right)e^{-\alpha \left(T+x\right)}\cos \left(\beta \left(T+x\right)\right)+\\ & \left[i{b}_1\left(b_1^2+\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2\right)\right)\left(C_2{e}^{-iT{b}_1}-C_1{e}^{iT{b}_1}\right)+\right.\\ & \left.\left.\left(b_1^2\left(\beta \gamma -\alpha \right)-\left({\alpha }^2+{\beta }^2\right)\left(\beta \gamma +\alpha \right)\right)\right]\left(C_2{e}^{-iT{b}_1}+C_1{e}^{iT{b}_1}\right)e^{-\alpha \left(T-x\right)}\cos \left(\beta \left(T-x\right)\right)\right\}/\\ & \left(-b_1^4+2\left({\beta }^2-{\alpha }^2\right)b_1^2-{\left({\alpha }^2+{\beta }^2\right)}^2\right).\end{array}$$

(14)

Repeating the procedure for real roots, that is, substituting Equation (11) for real roots into Equation (13) and performing integration, we obtain

$$\begin{array}{ll}K{L}_2=& \left\{\left[\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\left(\alpha \gamma -\beta \right)-2\alpha \gamma b_3^2\right)\left(C_3{e}^{-T{b}_3}+C_4{e}^{T{b}_3}\right)+\right.\right.\\ & \left.\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\left(\alpha \gamma -\beta \right)\alpha b_3\right)\left(C_3{e}^{-T{b}_3}-C_4{e}^{T{b}_3}\right)\right]e^{-\alpha \left(T+x\right)}\sin \left(\beta \left(T+x\right)\right)+\\ & \left\{\left[\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\left(\alpha \gamma -\beta \right)-2\alpha \gamma b_3^2\right)\left(C_4{e}^{-T{b}_3}+C_3{e}^{T{b}_3}\right)+\right.\right.\\ & \left.\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\left(\alpha \gamma -\beta \right)\alpha b_3\right)\left(C_4{e}^{-T{b}_3}-C_3{e}^{T{b}_3}\right)\right]e^{-\alpha \left(T-x\right)}\sin \left(\beta \left(T-x\right)\right)+\\ & \left[\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\left(\beta \gamma +\alpha \right)-2\alpha b_3^2\right)\left(C_3{e}^{-T{b}_3}+C_4{e}^{T{b}_3}\right)+\right.\\ & \left.\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)b_3-2\left(\beta \gamma +\alpha \right)\alpha b_3\right)\left(C_3{e}^{-T{b}_3}-C_4{e}^{T{b}_3}\right)\right]e^{-\alpha \left(T+x\right)}\cos \left(\beta \left(T+x\right)\right)+\\ & \left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\left(\beta \gamma +\alpha \right)-2\alpha b_3^2\right)\left(C_4{e}^{-T{b}_3}+C_3{e}^{T{b}_3}\right)+\\ & \left.\left.\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)b_3-2\left(\beta \gamma +\alpha \right)\alpha b_3\right)\left(C_4{e}^{-T{b}_3}-C_3{e}^{T{b}_3}\right)\right]e^{-\alpha \left(T-x\right)}\cos \left(\beta \left(T-x\right)\right)\right\}/\\ & \left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right).\end{array}$$

(15)

Note that equality $K{L}_1+K{L}_2=0$ holds if and only if one of the conditions, either (1) $\left\{C_1^{+}\equiv C_2=C_1;C_3^{+}\equiv C_4=C_3\right\}$ or (2) $\left\{C_1^{-}\equiv C_2=-C_1;C_3^{-}\equiv C_4=-C_3\right\}$, is satisfied. Combining Equations (14) and (15), for the first condition, we obtain

$$\begin{array}{ll}K{L}_1& +K{L}_2=\left\{\left[\left(\alpha \gamma \left({\alpha }^2+{\beta }^2+b_1^2\right)-\beta \left({\alpha }^2+{\beta }^2-b_1^2\right)\right)\cos \left(T{b}_1\right)-\right.\right.\\ & \left.\left(\gamma \left({\alpha }^2-{\beta }^2+b_1^2\right)-2\alpha \beta \right)b_1\sin \left(T{b}_1\right)\right]e^{-\alpha T}4C_1^{+}\left(\sin \left(T\beta \right)\cos \left(\beta x\right)\cosh \left(\alpha x\right)\right.-\\ & \left.\cos (T\beta )\sin \left(\beta x\right)\sinh \left(\alpha x\right)\right)+\left[\left(\alpha \left({\alpha }^2+{\beta }^2+b_1^2\right)+\beta \gamma \left({\alpha }^2+{\beta }^2-b_1^2\right)\right)\cos \left(T{b}_1\right)-\right.\\ & \left.\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2+b_1^2\right)b_1\sin \left(T{b}_1\right)\right]e^{-\alpha T}4C_1^{+}\left(\cos \left(T\beta \right)\cos \left(\beta x\right)\cosh (\alpha x)\right.+\\ & \left.\left. \sin \left(T\beta \right)\sin \left(\beta x\right)\sinh \left(\alpha x\right)\right)\right\}/\left({\alpha }^2+{\left(\beta +b_1\right)}^2\right)\left({\alpha }^2+{\left(\beta -b_1\right)}^2\right)+\\ & \left\{\left[\left(\left(\alpha \gamma -\beta \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha \gamma b_3^2\right)\cosh \left(T{b}_3\right)-\right.\right.\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\alpha b_3\left(\alpha \gamma -\beta \right)\right)\times \\ & \left.\sinh \left(T{b}_3\right)\right]e^{-\alpha T}4C_3^{+}\left(\sin \left(T\beta \right)\cos \left(\beta x\right)\cosh (\alpha x)\right.-\cos \left.\left(T\beta \right)\sin \left(\beta x\right)\sinh \left(\alpha x\right)\right)+\\ & \left[\left(\left(\beta \gamma +\alpha \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3^2\right)\cosh \left(T{b}_3\right)-\right.\left(b_3\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\alpha b_3\left(\beta \gamma +\alpha \right)\right)\times \\ & \left.\left.\sinh \left(T{b}_3\right)\right]e^{-\alpha T}4C_3^{+}\left(\cos \left(T\beta \right)\cos \left(\beta x\right)\cosh (\alpha x)\right.+\left.\sin \left(T\beta \right)\sin \left(\beta x\right)\sinh \left(\alpha x\right)\right)\right\}/\\ & \left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)=0.\end{array}$$

(16)

The following system of equations ensures that the first condition is met:

$$\left\{\begin{array}{c}\begin{array}{l}{\displaystyle \frac{\left[\left(\left(\alpha \gamma -\beta \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha \gamma b_3^2\right)\cosh \left(T{b}_3\right)-\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\alpha b_3\left(\alpha \gamma -\beta \right)\right)\sinh \left(T{b}_3\right)\right]C_3^{+}}{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}}+\\ {\displaystyle \frac{\left[\left(\alpha \gamma \left({\alpha }^2+{\beta }^2+b_1^2\right)-\beta \left({\alpha }^2+{\beta }^2-b_1^2\right)\right)\cos \left(T{b}_1\right)-\left.\left(\gamma \left({\alpha }^2-{\beta }^2+b_1^2\right)-2\alpha \beta \right)b_1\sin \left(T{b}_1\right)\right]C_1^{+}\right.}{\left({\alpha }^2+{\left(\beta +b_1\right)}^2\right)\left({\alpha }^2+{\left(\beta -b_1\right)}^2\right)}}=0,\end{array}\\ \begin{array}{l}{\displaystyle \frac{\left[\left(\left(\beta \gamma +\alpha \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3^2\right)\cosh \left(T{b}_3\right)-\left(b_3\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3\left(\beta \gamma +\alpha \right)\right)\sinh \left(T{b}_3\right)\right]C_3^{+}}{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}}+\\ {\displaystyle \frac{\left[\left(\alpha \left({\alpha }^2+{\beta }^2+b_1^2\right)+\beta \gamma \left({\alpha }^2+{\beta }^2-b_1^2\right)\right)\cos \left(T{b}_1\right)-\left.\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2+b_1^2\right)b_1\sin \left(T{b}_1\right)\right]C_1^{+}\right.}{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}}=0.\end{array}\end{array}\right.$$

(17)

From the second equation of this system, $C_3^{+}$ can be expressed as a function of $C_1^{+}$ as follows:

$$C_3^{+}=Q^{+}{C}_1^{+},$$

(18)

where

$$\begin{array}{c}{Q}^{+}=-{\displaystyle \frac{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}{\left({\alpha }^2+{\left(\beta +b_1\right)}^2\right)\left({\alpha }^2+{\left(\beta -b_1\right)}^2\right)}}\times \hfill \\ {\displaystyle \frac{\left(\alpha \left({\alpha }^2+{\beta }^2+b_1^2\right)+\beta \gamma \left({\alpha }^2+{\beta }^2-b_1^2\right)\right)\cos \left(T{b}_1\right)-\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2+b_1^2\right)b_1\sin \left(T{b}_1\right)}{\left(\left(\beta \gamma +\alpha \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3^2\right)\cosh \left(T{b}_3\right)-\left(b_3\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3\left(\beta \gamma +\alpha \right)\right)\sinh \left(T{b}_3\right)}}.\hfill \end{array}$$

Now, using the orthonormality property of eigenfunctions (see Equation (3)), the normalized constant $C_1^{+}$ can be expressed in terms of $b_1$ and $b_3$ as follows:

$$C_1^{+}=\sqrt{{\displaystyle \frac{2b_1{b}_3\left(b_1^2+b_3^2\right)}{\left(b_1^2+b_3^2\right)\left(b_1{\left(Q^{+}\right)}^2\sinh \left(2b_{3}T\right)+b_3\sin \left(2b_{1}T\right)+2b_1{b}_{3}T\left({\left(Q^{+}\right)}^2+1\right)\right)+DP}}},$$

(19)

where $DP=8b_1{b}_3{Q}^{+}\left(b_1\sin \left(b_{1}T\right)\cosh \left(b_{3}T\right)+b_3\cos \left(b_{1}T\right)\sinh \left(b_{3}T\right)\right).$

Solving the system consisting of Equation (10) and the first equation in (17), one can numerically calculate the set $\left\{b_1^{+}\left(k\right),b_3^{+}\left(k\right)\right\},k\in \mathbb{N}$ Finally, the eigenfunctions for this case are given by

$${\phi }_k\left(x\right)=C_1^{+}\left(k\right)\cos \left(b_1^{+}\left(k\right)x\right)+C_3^{+}\left(k\right)\cosh \left(b_3^{+}\left(k\right)x\right)$$

(20)

where $-T\le x\le T$ and $k\in \mathbb{N}$.

Similarly, for the second condition $\left\{C_1^{-}\equiv C_2=-C_1;C_3^{-}\equiv C_4=-C_3\right\}$, we can write

$$\begin{array}{ll}K{L}_1& +K{L}_2=\left\{\left[\left(-\left(\alpha \gamma +\beta \right)b_1^2-\left({\alpha }^2+{\beta }^2\right)\left(\alpha \gamma -\beta \right)\right)\sin \left(T{b}_1\right)-\right.\right.\left(\gamma \left({\alpha }^2-{\beta }^2+b_1^2\right)-2\alpha \beta \right)\times \\ & \left.b_1\cos \left(T{b}_1\right)\right]e^{-\alpha T}4i{C}_1^{-}\left(\sin \left(T\beta \right)\cos \left(\beta x\right)\sinh \left(\alpha x\right)\right.-\left.\cos (T\beta )\sin \left(\beta x\right)\cosh \left(\alpha x\right)\right)-\\ & \left[\left(\left(\alpha -\beta \gamma \right)b_1^2+\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\beta \gamma \right)\right)\sin \left(T{b}_1\right)+\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2+b_1^2\right)b_1\cos \left(T{b}_1\right)\right]\times \\ & e^{-\alpha T}4i{C}_1^{-}\left(\cos \left(T\beta \right)\cos \left(\beta x\right)\sinh (\alpha x)\right.+\left.\left.\sin \left(T\beta \right)\sin \left(\beta x\right)\cosh \left(\alpha x\right)\right)\right\}/\\ & \left(-b_1^4+2b_1^2\left({\beta }^2-{\alpha }^2\right)-{\left({\alpha }^2+{\beta }^2\right)}^2\right)+\left\{\left[\left(\left(\alpha \gamma -\beta \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha \gamma b_3^2\right)\sinh \left(T{b}_3\right)-\right.\right.\\ & \left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\alpha b_3\left(\alpha \gamma -\beta \right)\right)\left.\cosh \left(T{b}_3\right)\right]e^{-\alpha T}4C_3^{-}\times \\ & \left(\sin \left(T\beta \right)\cos \left(\beta x\right)\sinh (\alpha x)\right.-\cos \left.\left(T\beta \right)\sin \left(\beta x\right)\cosh \left(\alpha x\right)\right)+\\ & \left[\left(\left(\beta \gamma +\alpha \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3^2\right)\sinh \left(T{b}_3\right)-\right.\left(b_3\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3\left(\beta \gamma +\alpha \right)\right)\times \\ & \left.\left.\cosh \left(T{b}_3\right)\right]e^{-\alpha T}4C_3^{-}\left(\cos \left(T\beta \right)\cos \left(\beta x\right)\sinh (\alpha x)\right.+\left.\sin \left(T\beta \right)\sin \left(\beta x\right)\cosh \left(\alpha x\right)\right)\right\}/\\ & \left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right).\end{array}$$

(21)

The fulfillment of the second condition is ensured by the following system of equations:

$$\left\{\begin{array}{c}\begin{array}{l}{\displaystyle \frac{\left[\left(\left(\alpha \gamma -\beta \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha \gamma b_3^2\right)\sinh \left(T{b}_3\right)-\left(\left({\alpha }^2+{\beta }^2+b_3^2\right)\gamma b_3-2\alpha b_3\left(\alpha \gamma -\beta \right)\right)\cosh \left(T{b}_3\right)\right]C_3^{-}}{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}}+\\ {\displaystyle \frac{\left[\left(-\left(\alpha \gamma +\beta \right)b_1^2-\left({\alpha }^2+{\beta }^2\right)\left(\alpha \gamma -\beta \right)\right)\sin \left(T{b}_1\right)-\left(\gamma \left({\alpha }^2-{\beta }^2+b_1^2\right)-2\alpha \beta \right)b_1\cos \left(T{b}_1\right)\right]C_1^{-}}{-b_1^4+2b_1^2\left({\beta }^2-{\alpha }^2\right)-{\left({\alpha }^2+{\beta }^2\right)}^2}}=0,\end{array}\\ \begin{array}{l}{\displaystyle \frac{\left[\left(\left(\beta \gamma +\alpha \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3^2\right)\sinh \left(T{b}_3\right)-\left(b_3\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3\left(\beta \gamma +\alpha \right)\right)\cosh \left(T{b}_3\right)\right]C_3^{-}}{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}}-\\ {\displaystyle \frac{\left[\left(\left(\alpha -\beta \gamma \right)b_1^2+\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\beta \gamma \right)\right)\sin \left(T{b}_1\right)+\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2+b_1^2\right)b_1\cos \left(T{b}_1\right)\right]C_1^{-}}{-b_1^4+2b_1^2\left({\beta }^2-{\alpha }^2\right)-{\left({\alpha }^2+{\beta }^2\right)}^2}}=0.\end{array}\end{array}\right.$$

(22)

From the second equation of this system, we can obtain $C_3^{-}$ through $C_1^{-}$ as

$$C_3^{-}=Q^{-}{C}_1^{-},$$

(23)

where

$$\begin{gathered} Q^{-}={\displaystyle \frac{\left({\left(\alpha +b_3\right)}^2+{\beta }^2\right)\left({\left(\alpha -b_3\right)}^2+{\beta }^2\right)}{-b_1^4+2b_1^2\left({\beta }^2-{\alpha }^2\right)-{\left({\alpha }^2+{\beta }^2\right)}^2}}\times \\ {\displaystyle \frac{\left(\left(\alpha -\beta \gamma \right)b_1^2+\left({\alpha }^2+{\beta }^2\right)\left(\alpha +\beta \gamma \right)\right)\sin \left(T{b}_1\right)+\left(2\alpha \beta \gamma +{\alpha }^2-{\beta }^2+b_1^2\right)b_1\cos \left(T{b}_1\right)}{\left(\left(\beta \gamma +\alpha \right)\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3^2\right)\sinh \left(T{b}_3\right)-\left(b_3\left({\alpha }^2+{\beta }^2+b_3^2\right)-2\alpha b_3\left(\beta \gamma +\alpha \right)\right)\cosh \left(T{b}_3\right)}}. \end{gathered}$$

Using the orthonormality property of eigenfunctions (see Equation (3)), the normalized constant has the following form:

$$C_1^{-}=\sqrt{{\displaystyle \frac{2b_1{b}_3\left(b_1^2+b_3^2\right)}{-{\left(Q^{-}\right)}^2{b}_1\left(b_1^2+b_3^2\right)\left(2T{b}_3-\sinh \left(2b_{3}T\right)\right)-DN}}},$$

(24)

where $DN=2\left[4b_1{Q}^{-}\left(b_1\cos \left(b_{1}T\right)\sinh \left(b_{3}T\right)-b_3\sin \left(b_{1}T\right)\cosh \left(b_{3}T\right)\right)+\left(\sin \left(2b_{1}T\right)/2-b_{1}T\right)\left(b_1^2+b_3^2\right)\right]b_3.$ Solving the system consisting of Equation (10) and the first equation in (22), one can numerically calculate the set $\left\{b_1^{-}\left(k\right),b_3^{-}\left(k\right)\right\},k\in \mathbb{N}$ Finally, the eigenfunctions for the second condition are given by

$${\phi }_k\left(x\right)=C_1^{-}\left(k\right)\sin \left(b_1^{-}\left(k\right)x\right)+C_3^{-}\left(k\right)\sinh \left(b_3^{-}\left(k\right)x\right),$$

(25)

where $-T\le x\le T$ and $k\in \mathbb{N}$.

Let us sort the solutions $\left\{b_1^{+}\left(k\right),b_1^{-}\left(k\right)\right\}$ in ascending order with respect to their values as follows: $b_1\left(1\right)<b_1\left(2\right)<b_1\left(3\right)<\dots$, $k\in \mathbb{N}$. The odd-numbered solutions correspond to the first condition, while the even-numbered solutions correspond to the second condition. The corresponding real solutions $\left\{b_3^{+}\left(k\right),b_3^{-}\left(k\right)\right\}$, the eigenvalues $\left\{\widehat{\lambda }\left(k\right)\right\}$, can be calculated as functions of $\left\{b_1\left(k\right)\right\}$ using Equations (10) and (11), respectively.

4. Results

In this section, experimental results illustrating the eigenvalues and eigenfunctions for a real aerial photograph are presented. Figure 1 shows a test image of 256 × 256 pixels; each pixel has 256 quantization levels.

The mean and standard deviation of the image are 132 and 38, respectively. If a two-dimensional covariance function can be separated into functions of row and column coordinates, then the two-dimensional eigenfunctions can be expressed as products of one-dimensional eigenfunctions obtained by solving the integral equation using one-dimensional covariance functions computed over rows and columns. We illustrate our results with one-dimensional eigenfunctions using the averaged covariance function constructed over image rows. Assuming that the image rows are realizations of a stationary process, we first calculate the empirical covariance functions in all rows, and then, by averaging these functions, we obtain one experimental covariance function. The argument x is scaled to the interval $x\in \left[-1,1\right]$. Figure 2 shows the experimental covariance curve computed over image rows and its three approximations optimized in terms of mean square error (MSE) using the following covariance functions: exponential $\left(\exp \left(-a\left[x\right]\right)\right)$, exponentially cosine $\left(\exp \left(-\alpha \left|x\right|\right)\cos (\beta x)\right)$, and parametric oscillating $\left(\exp \left(-a\left|x\right|\right)\left[\cos \left(bx\right)+\gamma \sin \left(b\left|x\right|\right)\right]\right)$.

Since the MSE has a minimum as a function of the model parameters, the least squares method was used for numerical optimization. The method determines the best fit function by selecting model parameters to minimize the sum of squared differences between experimental covariance values and computed model values. The optimal approximation parameters for the exponential model are $MSE=1.055,$ $\alpha ==0.098$; for the exponentially cosine model, they are $MSE=0.965,$$\alpha =0.064$, $\beta =0.048;$ and for the parametric oscillatory model, they are $MSE=0.424,$ $\alpha =0.033$, $\beta =0.025$, $\gamma =-1$.

It is obvious that the approximation by the parametric oscillating function is much better with respect to the MSE than the approximation by the other two models. Figure 3 shows the normalized eigenvalue $\widehat{\lambda }$ as a function of the imaginary solution $b_1$ (see Equation (11)). A solution to the integral equation in (2) exists if the normalized eigenvalue $0<\widehat{\lambda }<2\left(\beta \gamma +\alpha \right)/\left({\alpha }^2+{\beta }^2\right)=9.335$ (limit value). Figure 4 shows the real solution as a function of the imaginary solution. The range of the real solution is $0<b_3<0.0154$, while the imaginary solution is $b_1\ge 0.1072$.

The values of $b_3$ and $b_1$ that satisfy the system of Equations (17) and (22) can be determined graphically as shown in Figure 5.

In the graph, the horizontal curve corresponds to Equation (10), while the vertical curves are the graphs of the first formulas of Equations (17) and (22). The odd set of intersections corresponds to the first condition (set $\left\{b_1^{+}\left(k\right),b_3^{+}\left(k\right)\right\}$), and the even set corresponds to the second condition (set $\left\{b_1^{-}\left(k\right),b_3^{-}\left(k\right)\right\}$. The corresponding numerical solution of the Karhunen–Loève integral equation for the first ten eigenfunctions and eigenvalues is given in

Table 1. Numerical solution of the Karhunen–Loève integral equation.

k	$\widehat{\mathit{\lambda }}\left(\mathit{k}\right)$	${\mathit{b}}_1\left(\mathit{k}\right)$	${\mathit{b}}_3\left(\mathit{k}\right)$	${\mathit{C}}_1\left(\mathit{k}\right)$	${\mathit{C}}_3\left(\mathit{k}\right)\times 10^{-4}$
1	1.92421	0.24402	0.01378	0.69704	169.4023
2	0.04564	1.59407	0.01534	0.99277	10.3956
3	0.01162	3.15990	0.01537	0.99712	−1.4092
4	0.00521	4.72024	0.01537	0.99917	−1.1909
5	0.00293	6.29238	0.01537	0.99927	0.3561
6	0.00188	7.85698	0.01537	0.99970	0.4298
7	0.00130	9.43091	0.01537	0.99968	−0.1586
8	0.00096	10.99894	0.01538	0.99985	−0.2194
9	0.00073	12.57097	0.01538	0.99982	0.0893
10	0.00058	14.13979	0.01538	0.99991	0.0327

.

The corresponding eigenfunctions given in Equations (20) and (25) are shown in Figure 6.

It is interesting to note that as $b_1$ increases, $b_3$ tends to a constant (see Figure 4 and Table 1 (4th column)). This means that as the index k increases, the second terms in Equations (20) and (25) also tend to constants. Therefore, the eigenfunctions become approximately a set of periodic sines and cosines with frequencies approximately equal to ${\overline{b}}_1\left(k\right)=\left(k-1\right)\pi /2$. Consequently, the procedure for calculating $b_1\left(k\right)$ and the remaining parameters of the model can be simplified.

Table 2. Approximation of the numerical solution $b_1$.

k	${\mathit{b}}_1\left(\mathit{k}\right)$	${\overline{\mathit{b}}}_1\left(\mathit{k}\right)$	$\left|\mathit{E}\mathit{R}\left(\mathit{k}\right)\right|$
2	1.59407	1.57079	0.03593
3	3.15990	3.14159	0.02365
4	4.72024	4.71238	0.00398
5	6.29238	6.28318	0.00278
6	7.85698	7.85398	0.00143
7	9.43091	9.42477	0.00142
8	10.99894	10.99557	0.00073
9	12.57097	12.56637	0.00072
10	14.13979	14.13716	0.00044

shows the true values $b_1\left(k\right)$, the approximate values ${\overline{b}}_1\left(k\right)$, and the absolute errors $\left|ER\left(k\right)\right|$ in Equations (17) and (22) when using approximate values instead of true values. Since for this model $b_1\ge 0.1072$, the index k in the table starts with k = 2.

5. Conclusions

In this paper, an analytical solution of the Karhunen–Loève integral equation is presented for a special case: the covariance function of a stationary process is a parametric oscillatory model. Based on the solution, explicit eigenfunctions and eigenvalues are derived depending on a small number of model parameters. Based on the solution, explicit eigenfunctions and eigenvalues are derived depending on three model parameters. The results of computer simulation using a real image illustrate all theoretical results using the parametric model of the oscillating covariance function. An approximate solution of the integral equation is proposed, and it is shown that with an increase in the index of the eigenfunction the approximation error decreases.