Association for Scientific Research MULTICHANNEL BLIND DECONVOLUTION USING A GENERALIZED GAUSSIAN SOURCE MODEL

Abstract– In this paper, we present an algorithm for the problem of multi-channel blind deconvolution which can adapt to un-known sources with both sub-Gaussian and superGaussian probability density distributions using a generalized gaussian source model. We use a state space representation to model the mixer and demixer respectively, and show how the parameters of the demixer can be adapted using a gradient descent algorithm incorporating the natural gradient extension. We also present a learning method for the unknown parameters of the generalized Gaussian source model. The performance of the proposed generalized Gaussian source model on a typical example is compared with those of other algorithm, viz the switching nonlinearity algorithm proposed by Lee et al. [8]. Keywords– blind deconvolution, blind source separation, generalized Gaussian source model, multichannel blind deconvolution.


INTRODUCTION
Recently, the problems of Blind Source Separation (BSS) or Independent Component Analysis [2,3,9], and multichannel blind deconvolution (MBD) [6,10] have been extensively studied.For BSS problems, source signals are instantaneously mixed by an unknown matrix, while for MBD problems, the source signals are mixed through a series of unknown dynamical systems.Since the mixing involves dynamical systems in MBD problems, delay is inevitably present, thus rendering it more complex to analyze than BSS.In BSS and MBD problems, both the sources and the mixing system are unknown, it is intuitively clear that the sources cannot be recovered without some necessary restriction on the nature of the sources.
Assumptions on the nature of the sources.The usual assumptions [2] place on the problems of BSS and MBD are as follows: 1.The source signals are statistically independent.2. At the most one of the source signals is Gaussian distributed.In addition, in this paper, we assume that the dynamical mixing systems in MBD problems are linear time invariant (LTI) systems which are modeled by constant parameters.There are many algorithms for tackling BSS and MBD problems.A popular assumption on the source signals is that they have sharply symmetrical distributions, i.e., the source signals are super-Gaussian with positive kurtosis.The classic Bell-Sejnowski infomax algorithm [2] works well when it is applied to mixtures of super-Gaussian signals, but it becomes inefficient when the mixtures include sub-Gaussian A. S. Abu-Taleb,E.M. E. Zayed, W. M. El-Sayed, A. M. Badawy and O. A. Mohammed 2 signals.The reason of the inefficiency in source separation in this case is that the assumed source model is invalid for the sub-Gaussian sources.This gives impetus to find more appropriate source models to BSS and MBD problems.Most researchers focus on modelling the sources with symmetrical unimodal probability density distributions because they are simple to analyze and they can model the source signals well in practice.Douglas et al. [6] presented a simple and efficient extension of a family of algorithms in BSS and MBD problems with mixtures of arbitrary non-Gaussian sources.Their algorithms [6] monitor the statistics of each output signal of the demixer, then selects the appropriate nonlinearity for each recovered source.Their algorithm is based on some necessary BSS stability conditions [3].Choi et al. [4] derived a learning algorithm, called flexible ICA, with a "flexible"nonlinearity.This nonlinear function is controlled by the Gaussian exponent according to the estimated kurtosis of the recovered signals.Hence the algorithm can successfully separate the mixture of both super-Gaussian and sub-Gaussian sources simultaneously.In Lee et al. [8], an extension of the infomax algorithm is derived which is able to separate both super-and sub-Gaussian independent components.A symmetrical strictly sub-Gaussian density is modeled using a symmetrical form of Pearson model or hyperbolic Cauchy density model.A switching criterion based on BSS stability analysis [3] is obtained.The above mentioned methods work well as they assume that the characteristics of the signals do not alter rapidly.In these methods, there is a certain detection latency as most of them require a finite time to elapse before they can react to the characteristics of the recovered source signals.For example, in [4] the kurtosis of the recovered signals need to be estimated while in [6], statistics of the recovered signals need to be monitored.Hence for signals whose characteristics vary rapidly, the above mentioned methods might not be sufficiently sensitive.In this paper, we will consider the generalized Gaussian source model originally proposed in [7] to "automatically" estimate the source signals in the MBD problems.Here "automatic"is taken to mean that our algorithm will estimate the parameters of the generalized gaussian source model from the recovered signals directly.The nonlinearity used in the separation of signals is a function of these estimated parameters.The parameters of the generalized gaussian source model can be easily interpreted, thus adding transparency to the estimated parameters.Hence the proposed algorithm works more "flexibly" in its ability to adapt to the changing nature of the source signals.The paper is organized as follows: State space Approach to Multichannel Blind Deconvolution is described in Section 2. For the sake of completeness, in Section 3 we include a brief description of the switching nonlinearity algorithm [10] adapted to the MBD case.This section also contains a special form of the switching nonlinearity algorithm, viz., the fixed nonlinearity algorithm.Generalized gaussian Source Model is introduced in Section 4. Some experimental results are given in Section 5. Finally we draw some conclusions in Section 6.

STATE SPACE APPROACH TO MULTICHANNEL BLIND DECONVOLUTION
Given a vector of observed signals u(k); k∈ [0,N], we wish to recover the source signals s(k) based on the assumption that the sources are statistically Independent.If we assume that the observations are convolutive version of the sources, the problem can be tackled using state space approach [10].We consider the state space Multichannel Blind Deconvolution using a Generalized Gaussian Source 3 approach instead of the transfer function approach, as the state space approach can be easily extended to nonlinear mixing systems.Moreover, the state space approach not only gives an efficient internal description of the dynamic systems, but also there exist different possible equivalent state space realizations, for instance, canonical controller form [10] which allows us to find "efficient" representations of the demixer .We model the mixing environment of the MBD problem as follows: where s∈R n , u ∈ R n , and x ∈ N R are the source signals, the observations, and the state of the LTI dynamical system, respectively.The system matrices , which are assumed to be constant, are state mixing matrix, input mixing matrix, output mixing matrix, and input-output mixing matrix respectively.The system matrices in the demixer are similarly defined.For simplicity, we assume the number of source n equals to the number of sensors.Normally, the system order N is unknown; we need to estimate its value from the observed data.Here we assume that the system order is "known".Correspondingly, we model the demixer using a similar discrete time dynamical system: where the input vector is the system states of the demixer.For successful separation and deconvolution, we need N N ≥ .Here we assume both the mixer and the demixer exist, in particular, D -1 exists.The condition of the existence of solution in multichannel blind deconvolution is studied in [10].
We measure the dependence among the recovered sources y using mutual information.Given P(y), the probability density function (PDF) of the recovered signal vector y, the mutual information between the recovered signals can be defined as follows: where ] E[log(P(y) -H(y) = is the entropy of y, )] ( [log( ) ( is the marginal entropy of q y .For simplicity, for the remaining part of this paper, the time index k is dropped if there is no risk of confusion.Observe that I(y) = 0, and I(y) = 0 if and only if the components of vector y are statistically independent.Therefore I(y) is an appropriatemeasurement of the dependence among the recovered signals.Unfortunately, mutual information is difficult to compute explicitly, hence we use a cost function similar to [10]: where θ is the set of system parameters and source model parameters, which we will study in Section 3 and 4, det is the determinant.There exist various ways to tackle the optimization problem [10].Here we follow the derivation of information backpropagation approach given in [8].

Gradient-based learning rules
Based on the cost function (7), we can easily obtain the following updating rules.For matrices D and where ) ( y ϕ is a vector nonlinearity related to the source model.This will be discussed further in Section 3. Note, natural gradient [1] is used in (8) to improve the performance of the learning process.Similarly, for matrices A and B, we have: denotes the l-th column vector of the matrix C can be obtained from following: where li δ is the Kronecker delta function.

LEE ET AL. SWITCHING NONLINEARITY ALGORITHM
For the sake of completeness, we will briefly describe the algorithm due to Lee et al. [8] on detecting the kurtosis of the recovered signals, and then switch the nonlinearity based on this information.We will follow closely the development in [8], but adapting their notation to the notations in this paper for its applications to MBD.Consider the learning equation for D, as shown in (8).Consider the reconstructed signal y.In general, a symmetrical form for modelling a strictly sub-Gaussian density can be obtained using Pearson's mixture model: where ) , ( 2σ µ N denotes the Gaussian distribution with mean µ and variance 2  σ .For various values of µ this can be either a unimodal distribution, or a bimodal distribution. The nonlinearity in ( 8) can be expressed as: and 2 σ =1, and substituting this into (8), we have: This is the learning algorithm derived by assuming that the source model is a sub-Gaussian model.And in a similar fashion, we can derive a learning rule for super-Gaussian distributions.The two can be combined together to give: where K is a diagonal matrix with elements kq = 1 if the recovered signal is super-Gaussian, and kq = -1 if the recovered signal is sub-Gaussian.If we assume that K is an identity matrix, then this will be referred to as the fixed nonlinearity case.This is the algorithm derived in [10].Using stability arguments [3], it is possible to estimate the coefficient kq as follows: Thus, q k can be estimated for each recovered signal.

GENERALIZED GAUSSIAN SOURCE MODEL
A generalized Gaussian source model is introduced in [5].This model encompasses both super-and sub-Gaussian sources.The generalized gaussian density is expressed as follows: where 0 r can change from zero to, through 1 (the Laplace distribution) and q r =2 (standard Gaussian distribution), to q r going to infinity (for uniform distribution).Once we model the source with generalized gaussian density, we can choose the nonlinearity as follows: where the sign operator can be defined as follows: The parameters of the model can be estimated online as follows: Consider the cost function (7), we need to learn the parameters of the generalized gaussian source model to maximize the cost function.We can derive the gradient of l with respect to with respect to q q r σ , as follows [7]: 2 ln( q r q q q q q q q q q q q q r q q q q q q q q r y r r r r r r r y y r r r r is the digamma function.Based on the above gradient, we have the following learning rules for q q q q q q q q q q q q r q q q q q q q q q q q σ σ σ σ

EXPERIMENTAL RESULTS
In this experiment, consider the system described by the following two independent soources:

Lee et al.'s Switching Nonlinearity method
In this section, we present results (shown in Figure 1)in applying the method derived in Section 3.

Generalized Gaussian model approach
In this section, we will present results using the generalized gaussian model approach as proposed in this paper (see Figures 2).Table 1 gives the mean squared errors among the two methods.It is noted that the generalized Gaussian model gives the smallest mean squared errors.

CONCLUSION
In this paper, we consider the possibility of "automatic" adaptation to the source model, whether it is super-Gaussian or sub-Gaussian.We use the fixed nonlinearity approach [10] as the baseline for comparison.We extend the generalized gaussian source model proposed in [5] to the MBD case.In addition, we have extended the switching nonlinearity method in [8] to the MBD case as well.The two methods are all applied to

3 .
It is only possible to recover the source signals modulo scale, and polarity.
space model for blind deconvolution Fig.1.A. S. Abu-Taleb,E.M. E. Zayed, W. M. El-Sayed, A. M. Badawy and O measure of variance known as the dispertion of the distribution.The parameter q

D
observed signals u(k) are obtained by passing the source signals through a mixing linear dynamical system defined in(1), and(2), where the source number n = 2, and system order N = 2.The system matrices D exists and the eigenvalues of A are within the unit circle.

Fig. 2
Fig. 2 Signals of Lee et al's method.For explanation of these graphs please see caption for Figure 2

Fig. 3 .
Fig. 3. Signals of the generalized Gaussian model.(for an explanation of the graphs please see the caption for Figure 3)

Table 1 :
A table showing the mean squared errors as obtained by each of the two methods.