Time Domain Spherical Harmonic Processing with Open Spherical Microphones Recording

Featured Application

Spatial Active Noise control.

Abstract

Spherical harmonic analysis has been a widely used approach for spatial audio processing in recent years. Among all applications that benefit from spatial processing, spatial Active Noise Control (ANC) remains unique with its requirement for open spherical microphone arrays to record the residual sound field throughout the continuous region. Ideally, a low delay spherical harmonic recording algorithm for open spherical microphone arrays is desired for real-time spatial ANC systems. Currently, frequency domain algorithms for spherical harmonic decomposition of microphone array recordings are applied in a spatial ANC system. However, a Short Time Fourier Transform is required, which introduces undesirable system delay for ANC systems. In this paper, we develop a time domain spherical harmonic decomposition algorithm for the application of spatial audio recording mainly with benefit to ANC with an open spherical microphone array. Microphone signals are processed by a series of pre-designed finite impulse response (FIR) filters to obtain a set of time domain spherical harmonic coefficients. The time domain coefficients contain the continuous spatial information of the residual sound field. We corroborate the time domain algorithm with a numerical simulation of a fourth order system, and show the proposed method to have lower delay than existing approaches.

1. Introduction

Spherical harmonic analysis has been widely used for spatial acoustic signal processing for years [1]. Sound field recordings can be decomposed into a set of orthogonal spatial basis functions and respective coefficients when an appropriately designed spherical microphone array is used [2,3]. The spherical harmonic decomposition has the advantage that a given sound field can be analyzed over a continuous spatial region rather than a set of distributed points [4]. This has embraced a wide range of algorithms in three-dimensional (3D) audio signal processing such as: sound intensity analysis [5], sound field diffusive analysis [6], beamforming [7,8], source localization [9,10], and spatial Active Noise Control (ANC) [11,12].

A spatial ANC system aims to reduce the unwanted acoustic noise [13] over a space in order to create a silent zone for people. Multiple microphones are used to record the residual noise, and multiple loudspeakers are used to generate the anti-noise field. The recording’s accuracy of the residual sound field can highly influence the performance of an ANC system. Furthermore, recording efficiency is also important, as ANC usually focuses on low frequency and time-variant noise. As a result, an accurate and low latency algorithm for residual sound field recording is desired [14].

The sound field recording step in a spatial ANC system focuses on obtaining the location independent spherical harmonic coefficients that represent the residual sound field inside a region of interest. This is different to real time spherical harmonic beamforming or directivity analysis which focuses on extracting source location information from the spatial recording. Moreover, spatial ANC mainly focuses on reducing the sound field inside the spherical microphone array (the region of interest). While other spatial recording applications may focus on analysing the sound field exterior to the array. Additionally, although most of the spatial audio applications utilize a rigid spherical array [15,16,17] for its convenience to build and use, an open spherical array is considered to be more suitable for a spatial ANC system. This is because users should be able to enter and move within the ANC region of interest that is surrounded by the spherical microphone array [12,18]. Furthermore, there exists previous work focusing on optimising the open array for spherical harmonic recording [19,20], and for spatial ANC systems [21]. However, we consider the optimisation of the open microphone array design to be outside of the scope of this paper, and instead focus on a time-domain recording algorithm.

Real-time spatial beamforming systems illustrate that applications with strict delay requirements can highly benefit from the small latency and efficient computation of time domain processing [22,23]. By posing the signal processing algorithm in the time domain, system performance can be optimized with real-valued lower order filters [24], and lower modeling delays [25]. Specifically, for a spatial ANC system, the system delay which includes the filter group delay (signal processing algorithm), the A/D and D/A converter, and the data processing delay, should be less than the acoustic delay from the reference microphones to the secondary loudspeakers in order to satisfy causality [26]. Furthermore, a longer signal processing delay slows down the convergence speed of the adaptive filtering and may lead to an unstable system [27,28]. Therefore, it is worthwhile to consider a time domain spherical harmonic decomposition method to achieve sound field recording with an open spherical array for the application of spatial ANC.

Frequency domain spherical harmonic recording has been well developed with various optimised filters [29,30,31]. One benefit of developing the method in the frequency domain is that the influence of the spherical Bessel zeros can be easily removed by avoiding the estimation of the coefficients at these erroneous frequency bins [19,32,33,34]. However, when we consider a time domain method, we can not simply avoid the Bessel zeros because we do not apply a Fourier Transformation to separate the Bessel zero frequency components from the others.

Meanwhile, there are also several works relate to time domain spatial audio signal processing. In [35], Poletti and Abhayapala give a time domain description of the free-space Green’s function in the spherical harmonic domain. This provides a solution to decompose the free-space channel between a loudspeaker and microphone into the time-space domain. This work only targets the free-space Green’s function, and as a result, the method is highly limited to the application of free space sound field reproduction. In [36], a time domain wave field synthesis method is presented. Although an IFFT is applied to derive the time domain solution, the work still demonstrates that time-domain wave field synthesis can be beneficial to time-varying spatial acoustic applications. In [37], Hahn and Spors offer a time domain representation of the spherical harmonic equation. They relate the time domain spherical harmonic coefficients to the sound pressure, but do not include the method of obtaining the time domain coefficients from a given recording. Time domain beamformers are designed in [38,39] with the IFFT of spherical harmonics. These papers show certain advantages for finite impulse response (FIR) filtering based signal processing systems. Overall, these time domain approaches illustrate the advantages of time domain signal processing, however, they remain unable to obtain location-independent spherical harmonic coefficients. This makes them ill-suited for spatial ANC systems, as these location-independent coefficients provide necessary information about the continuous residual sound field inside the region of interest.

In this paper, we propose a FIR filter based time domain spherical harmonic analysis method to accurately record spatial sound fields with an open spherical microphone array for the purpose of spatial ANC. We note that this work focuses solely on the problem of sound field recording, and that the spatial ANC application acts purely as motivation to our problem. Therefore, with spatial ANC in mind, the recording method prioritizes a minimum processing delay, a bandwidth of interest (low frequencies for typical noise scenarios), and a practical array geometry (open sphere surrounding a quite zone). Employing the recording method in an actual ANC system, and its evaluation, is out of the scope of this paper. The novelty of the presented work is the investigation of time domain spherical harmonic coefficients. These time domain coefficients match the properties of conventional frequency domain spherical harmonic coefficients. That is, the coefficients are location independent within the region of interest, and they represent the continuous sound field over the space. Additionally, these coefficients are obtained in time domain, which relieves the block processing constraint (and can do sample-by-sample processing) and results in lower system delay. Hence, the proposed method is considered to be highly beneficial to spatial ANC systems.

We organise the main body of this paper as follows: In Section 2 we detail the background of the frequency domain spherical harmonic algorithm for spatial sound field recording. Additionally, we introduce the time domain equation of spherical harmonic decomposition, while addressing the challenges of recording time domain spherical harmonic coefficients. The filter’s design and implementation to obtain time domain spherical harmonic coefficients is presented in Section 3, along with error analysis. Effects of truncation and filter length are shown in Section 4 via initial simulations of filter performance. Section 5 presents simulation results for the proposed method’s estimation of spherical harmonic coefficients, as well as sound field reconstruction performance at a point and over space, verifying the effectiveness of the proposed theory and design. We conclude the findings and insights gained from this work in Section 6.

2. Problem Formulation

We begin this section by reviewing the well-known frequency domain spherical harmonic decomposition method. We then introduce the corresponding time domain formulation, and detail the Fourier Transform relationship between the components in the frequency domain equation and the time domain equivalent. Finally, we show the difficulties in obtaining spherical harmonic coefficients in the time domain.

2.1. Spherical Harmonic Decomposition of Sound Field in Frequency-Space Domain

An incident sound field at any arbitrary point $\mathit{x}=(r,\theta ,\varphi )$ inside a source free 3D spherical region $\mathsf{\Omega }$, where r refers to the distance between the point $\mathit{x}$ and the origin, $\theta$ and $\varphi$ denote elevation and azimuth angles, respectively [40], can be expressed in the frequency domain as [1,41]

$$S(\mathit{x},k)=\sum _{n=0}^N\sum _{m=-n}^n{\alpha }_{nm}\left(k\right)j_n\left(kr\right)Y_{nm}(\theta ,\varphi ),$$

(1)

where order n$(n\ge 0)$ and mode m are integers, $N=\lceil kR\rceil$[1], $k=2\pi f/c$ is the wave number, f is frequency, c is the speed of sound, R is the radius of $\mathsf{\Omega }$, ${\alpha }_{nm}\left(k\right)$ is a set of spherical harmonic coefficients representing the sound field inside $\mathsf{\Omega }$, $j_n\left(kr\right)$ is the nth order spherical Bessel function of the first kind, $Y_{nm}(\theta ,\varphi )$ are the spherical harmonic functions. For convenience, we use real spherical harmonics in this paper, given by [42]

$$\begin{array}{cc}\hfill Y_{nm}(\theta ,\varphi )=& {(-1)}^{\left|m\right|}\sqrt{\frac{2n+1}{4\pi }\frac{(n-|m\left|\right)!}{(n+|m\left|\right)!}}\hfill \\ \hfill & \times \left\{\begin{array}{cc}\hfill P_{nm}(cos\theta )cos\left(m\varphi \right)& m\ge 0\\ \hfill P_{nm}(cos\theta )sin\left(m\varphi \right)& m<0\end{array}\right.,\hfill \end{array}$$

(2)

where $P_{nm}(·)$ is the associated Legendre function. Real spherical harmonics have the orthogonality property of

$${\int }_0^{2\pi }{\int }_0^{\pi }{Y}_{nm}(\theta ,\varphi )Y_{n^{\prime }{m}^{\prime }}(\theta ,\varphi )sin\theta d\theta d\varphi ={\delta }_{n{n}^{\prime }}{\delta }_{m{m}^{\prime }}.$$

(3)

If the spherical harmonic coefficients ${\alpha }_{nm}\left(k\right)$ are available for a sound field, then these coefficients can fully describe the sound field over the continuous spatial region of interest. Traditionally, when spatial harmonic processing is used to record a spatial sound field $S(\mathit{x},k)$, it is recorded over a spherical surface of radius $R_Q$$(R_Q\ge r)$. The corresponding ${\alpha }_{nm}\left(k\right)$ are extracted by integrating (1) over the spherical surface while exploiting the orthogonality property of $Y_{nm}(·)$ in (3), which gives [2]

$${\alpha }_{nm}\left(k\right)=\frac{1}{j_n\left(kr\right)}{\int }_0^{2\pi }{\int }_0^{\pi }S(r,\theta ,\varphi ,k)Y_{nm}(\theta ,\varphi )sin\theta d\theta d\varphi .$$

(4)

In practice, this integration is realized using an equivalent discrete summation of spatial samples over the sphere.

2.2. Equivalent Spherical Harmonic Decomposition of a Sound Field in Time-Space Domain

While the frequency domain spatial sound field capture is well established as explained in Section 2.1, in this paper, our objective is to investigate the possibility of an analogous spherical harmonic analysis in time domain. In a similar fashion to (1) and (4), we now consider the relationship between sound pressure $s(\mathit{x},t)$ recorded by a spherical microphone array and the time domain spherical harmonic coefficients, denoted as $a_{nm}\left(t\right)$. It is desirable to have these time domain coefficients $a_{nm}\left(t\right)$ independent of the measurement radius. Thus, we only need to record $a_{nm}\left(t\right)$ to obtain the sound field over the entire region of interest $\mathsf{\Omega }$. A time domain method can directly extract $a_{nm}\left(t\right)$, thus avoiding the Fourier transformation of signals.

As a time domain analysis is usually with real-valued components, we rewrite (1) in the form of

$$S(\mathit{x},k)=\sum _{n=0}^N\sum _{m=-n}^n{i}^n{\alpha }_{nm}\left(k\right)\frac{j_n\left(kr\right)}{i^n}{Y}_{nm}(\theta ,\varphi ),$$

(5)

where $i=\sqrt{-1}$, in order to make the inverse Fourier transform of all terms to be real. Taking the inverse Fourier transformation of (5), we obtain

$$s(\mathit{x},t)=\sum _{n=0}^N\sum _{m=-n}^n{a}_{nm}\left(t\right)\ast p_n(t,r)Y_{nm}(\theta ,\varphi ),$$

(6)

where * denotes the convolution operation,

$$a_{nm}\left(t\right)\stackrel{\mathcal{F}}{⟶}{i}^n{\alpha }_{nm}\left(k\right),$$

(7)

where $\stackrel{\mathcal{F}}{⟶}$ denotes the Fourier transform operator,

$$p_n(t,r)\stackrel{\mathcal{F}}{⟶}\frac{j_n\left(kr\right)}{i^n},$$

(8)

which is given by

$$p_n(t,r)=\left\{\begin{array}{ccc}\hfill \frac{c}{2r}{P}_n\left(\frac{tc}{r}\right)& & -\frac{r}{c}\le t\le \frac{r}{c}\hfill \\ \hfill 0& & \left|t\right|>\frac{r}{c}\hfill \end{array}\right.,$$

(9)

where $P_n(·)$ is the Legendre function. The proof of (9) is given in Appendix A. We note that every component in (6) is real valued.

Equation (6) shows how to reconstruct the sound pressure at $\mathit{x}=(r,\theta ,\varphi )$ with the recorded time domain spherical harmonic coefficients $a_{nm}\left(t\right)$. We consider an alternative time domain filter to obtain $a_{nm}\left(t\right)$ from the recorded signals rather than taking the inverse Fourier transform of (4) since $1/j_n\left(kr\right)$ is unbounded when $j_n\left(kr\right)=0$. Note that $j_n\left(kr\right)$ as a filter has order dependent zeros when $j_n\left(kr\right)=0$. As a result, $1/j_n\left(kr\right)$ approaches infinity at these frequencies, making it unstable to have an inverse Fourier transform. In other words, the z-transform of $p_n(t,r)$ given in (9), has zeros on the unit circle because of Bessel zeros, refers to a non-minimum phase system. In this case, the inverse system of $p_n(t,r)$, with the frequency response of $1/j_n\left(kr\right)$ is not stable. As a result, we first define

$$b_{nm}(t,r)\triangleq a_{nm}\left(t\right)\ast p_n(t,r),$$

(10)

which has a frequency response of

$$b_{nm}(t,r)\stackrel{\mathcal{F}}{⟶}{i}^n{\alpha }_{nm}\left(k\right)\frac{j_n\left(kr\right)}{i^n}={\alpha }_{nm}\left(k\right)j_n\left(kr\right).$$

(11)

Since $Y_{nm}(\theta ,\varphi )$ is independent to both frequency and time, $b_{nm}(t,r)$ can be obtained by integrating (6) over a sphere of radius r such that

$$b_{nm}(t,r)={\int }_0^{2\pi }{\int }_0^{\pi }S(r,\theta ,\varphi ,t)Y_{nm}(\theta ,\varphi )sin\left(\theta \right)d\theta d\varphi .$$

(12)

If we regularly place $Q\ge {(N+1)}^2$ omni-directional microphones on a sphere of radius $R_Q$, we can estimate the integration in (12) with a finite summation such that

$$b_{nm}(t,R_Q)\approx \sum _{q=1}^{Q}S({\mathit{x}}_q,t)Y_{nm}({\theta }_q,{\varphi }_q).$$

(13)

To simplify the implementation, we sample the signals with sampling time T such that $t=\nu T=\nu /F_s$, where $\nu$ is the time index and $F_s$ is the sampling frequency. We rewrite (10) as

$$\begin{array}{cc}\hfill b_{nm}(\nu ,R_Q)& =a_{nm}\left(\nu \right)\ast p_n(\nu ,R_Q)\hfill \\ \hfill & =\sum _{\mu =-L_p}^{L_p}{p}_n(\mu ,R_Q)a_{nm}(\nu -\mu ),\hfill \end{array}$$

(14)

where

$$p_n(\nu ,R_Q)=\left\{\begin{array}{ccc}\hfill \frac{c}{2R_Q}{P}_n\left(\frac{\nu c}{R_Q{F}_s}\right)& & -\frac{R_Q{F}_s}{c}\le \nu \le \frac{R_Q{F}_s}{c}\hfill \\ \hfill 0& & \left|\nu \right|>\frac{R_Q{F}_s}{c}\hfill \end{array}\right.,$$

(15)

is a time limited function with $p_n(\nu ,r)\ne 0$ when $-L_p\le \nu \le L_p$, $L_p=\lceil R_Q{F}_s/c\rceil$ such that the length of $p_n(\nu ,R_Q)$ is $2L_p+1$.

With (14) in hand, our problem reduces to obtaining $a_{nm}\left(\nu \right)$ from the measured $b_{nm}(\nu ,R_Q)$. This is not achievable since it is an under-determined problem. We always have $2L_p+1$ more unknowns ($a_{nm}\left(\nu \right)$) than knowns ($b_{nm}(\nu ,R_Q)$). Moreover, this is not practically feasible because the z-transform of $p_n(\nu ,R_Q)$ has zeros on the unit circle, resulting in poles on the unit circle in its direct inverse, making the system unstable. Alternatively, $a_{nm}\left(\nu \right)$ can be extracted from $b_{nm}(\nu ,R_Q)$ using an appropriately designed filter.

In this paper, we attempt to design a filtering solution while overcoming the above challenges. It is important to note that the Fourier transform relationship discussed in this section were solely used to formulate the definition of the time-domain spherical harmonic decomposition of a sound field. From this point onward, we will focus on signal processing of the captured sound field only in the time domain.

3. Filter Design for Obtaining Time Domain Spherical Coefficients

In Section 2, we have presented a method to obtain $b_{nm}(\nu ,R_Q)$ from recorded sound pressure $S({\mathit{x}}_q,\nu )$ with a spherical microphone array. In this section, we design a series of FIR filters to obtain $a_{nm}\left(\nu \right)$ from given $b_{nm}(\nu ,R_Q)$.

3.1. Stability of Ideal Inverse Filter

Due to the challenges mentioned in Section 2, rather than directly using (14), we pre-design a series of filters ${\rho }_n(\nu ,r)$ such that

$$\begin{array}{cc}\hfill & b_{nm}(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)\hfill \\ \hfill & =a_{nm}\left(\nu \right)\ast p_n(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)=a_{nm}\left(\nu \right),\hfill \end{array}$$

(16)

where

$$p_n(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)\approx \delta \left(\nu \right).$$

(17)

We note here that ${\rho }_n(\nu ,r)$ should be order n dependent but mode m independent, as is the same property with $p_n(\nu ,r)$.

However, we can never achieve a precise $\delta \left(\nu \right)$ in (17), as the energy of measured sound pressure at the frequency bins of Bessel zeros has been filtered to zero by $p_n(\nu ,R_Q)$. Therefore, we refrain from designing the inverse filter at these zero positions. Instead, we modify $\delta \left(\nu \right)$ to ${\widehat{z}}_n\left(\nu \right)$ such that its frequency response ${\widehat{Z}}_n\left(f\right)$ is given by

$${\widehat{Z}}_n\left(f\right)=\left\{\begin{array}{ccc}\hfill 1& & |j_n\left(kr\right)|\ge ϵ\hfill \\ \hfill 0& & |j_n\left(kr\right)|<ϵ\hfill \end{array}\right.,$$

(18)

where $ϵ$ is a small positive constant threshold which satisfies $j_n\left(kr\right)\approx 0$ when $|j_n\left(kr\right)|<ϵ$. For a fixed $R_Q$, both $j_n(2\pi f{R}_Q/c)$ and ${\widehat{Z}}_n\left(f\right)$ can be seen as a function of f. Figure 1 shows $j_n(2\pi f{R}_Q/c)$ and ${\widehat{Z}}_n\left(f\right)$ with $ϵ=1/40$ of the first four orders of n.

From Figure 1, we can see that ${\widehat{Z}}_n\left(f\right)$ is a superposition of a series of rectangular windows, meaning its inverse Fourier transformation, ${\widehat{z}}_n\left(\nu \right)$, should be a superposition of $\mathrm{sinc}$ functions. In practice, due to inherent properties of $j_n(2\pi f{R}_Q/c)$, for a given maximum frequency $f_{\max }$, the number of active spherical harmonic orders is up to $N\approx \lceil k{R}_Q\rceil$[1]. We use the same truncation limit when designing ${\widehat{Z}}_n\left(f\right)$, resulting in ${\widehat{z}}_n\left(\nu \right)$ to be a superposition with a finite number of $\mathrm{sinc}$ functions. The necessity and the influence of this truncation on frequency $f_{\max }$ will be further discussed in Section 4.1.

Let us define $w^{\left(n\right)}$ in radian (rad), such that $j_n(w^{\left(n\right)}{F}_s{R}_Q/c)=ϵ$, where $ϵ$ is the positive threshold we explained in the last paragraph. Therefore, $w^{\left(n\right)}$ can be considered as the edges of window in ${\widehat{Z}}_n\left(f\right)$ (see Figure 1). Given the vector of $[w_1^{\left(n\right)},w_2^{\left(n\right)},w_3^{\left(n\right)},\cdots ]$, we can write ${\widehat{z}}_n$ as

$$\begin{array}{cc}\hfill {\widehat{z}}_n\left(\nu \right)=& \sum _{s=1}^S(w_{2s}^{\left(n\right)}-w_{2s-1}^{\left(n\right)})\mathrm{sinc}\left(\frac{w_{2s}^{\left(n\right)}-w_{2s-1}^{\left(n\right)}}{2}\nu \right)\hfill \\ \hfill & \times cos\left(\frac{w_{2s}^{\left(n\right)}+w_{2s-1}^{\left(n\right)}}{2}\nu \right),\hfill \end{array}$$

(19)

where S is the number of rectangular windows in ${\widehat{Z}}_n\left(f\right)$ for $-f_{\max }\le f\le f_{\max }$. Furthermore, $w^{\left(n\right)}$s are dependent on the radius of the microphone array $R_Q$, sampling frequency $F_s$ and the speed of sound c, but the value of $w^{\left(n\right)}{F}_s{R}_Q/c$ remains constant for each order n such that $|j_n(w^{\left(n\right)}{F}_s{R}_Q/c)|=ϵ$. The first four order of $w^{\left(n\right)}$ is given in

Table 1. The first four order of $w^{\left(n\right)}$ to derive ${\widehat{z}}_n\left(\nu \right)$ with $f_{\max }\approx 2047$ Hz, $ϵ=1/40$, $R_Q=0.16$ m and $c=343$ m/s.

n (Order)	w${}_1^{\left(\mathit{n}\right)}$	w${}_2^{\left(\mathit{n}\right)}$	w${}_3^{\left(\mathit{n}\right)}$	w${}_4^{\left(\mathit{n}\right)}$
0	0.0009	0.1369	0.1439	0.2680
1	0.0033	0.1957	0.2059	0.2680
2	0.0276	0.2509	0.3643	0.2680
3	0.0640	0.2680	-	-

with the highest frequency limit of $f_{\max }=2047$ Hz and sampling frequency $F_s=48,000$ Hz. Note that for the zero-th order, we set $w_1=8.9\times {10}^{-4}$ to block DC component in practice.

If we have a series of concentric spherical microphone arrays with the radii of $r_1,r_2,\cdots$, the value of $w^{\left(n\right)}{F}_s{r}_q/c$ would be different from a single sphere model, which can be calculated by $|j_n(w^{\left(n\right)}{r}_1/F_{s}c)+j_n(w^{\left(n\right)}{r}_2/F_{s}c)+\cdots |=ϵ$.

3.2. Modified Inverse Filter

Now that the design for ${\widehat{z}}_n\left(\nu \right)$ is established, our next step is to design filters ${\rho }_n(\nu ,R_Q)$ which satisfies

$$p_n(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)={\widehat{z}}_n\left(\nu \right).$$

(20)

We notice in (20) that $p_n(\nu ,R_Q)$ is a finite length vector and we would like ${\rho }_n(\nu ,R_Q)$ also to be a finite length vector. However, ${\widehat{z}}_n\left(\nu \right)$ is infinitely long with a series of $\mathrm{sinc}$ functions. If we perform linear convolution of $p_n(\nu ,R_Q)$ with ${\rho }_n(\nu ,R_Q)$, we would obtain a vector with the length of $2(L+L_p)+1$ samples, where $2L+1$ is the filter length of ${\rho }_n$, such that ${\rho }_n(\nu ,R_Q)$ has none-zero values for $-L\le \nu \le L$. Thus, we need to truncate the infinite length ${\widehat{z}}_n\left(\nu \right)$ to $2(L+L_p)+1$ samples for every order of n where

$$z_n\left(\nu \right)\triangleq \left\{\begin{array}{ccc}\hfill {\widehat{z}}_n\left(\nu \right)& & -(L+L_p)\le \nu \le L+L_p\hfill \\ \hfill 0& & \mathrm{otherwise}\hfill \end{array}\right..$$

(21)

We can then write (20) in a finite summation form as

$$\begin{array}{cc}\hfill z_n\left(\nu \right)& =p_n(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)\hfill \\ \hfill & =\sum _{\mu =-L}^L{p}_n(\nu -\mu ,R_Q){\rho }_n(\mu ,R_Q).\hfill \end{array}$$

(22)

We rewrite (22) into matrix form

$${\mathit{z}}_n={\mathit{P}}_n{\mathit{\rho }}_n,$$

(23)

where ${\mathit{z}}_n={[z_n(-(L+L_p)),z_n(-(L+L_p)+1),\cdots ,z_n\left((L+L_p)\right)]}^T$, ${\mathit{\rho }}_n=[{\rho }_n(-L,R_Q)$, ${\rho }_n(-L+1,R_Q),\cdots ,{\rho }_n(L,R_Q){]}^T$, and ${\mathit{P}}_n$ is the convolution matrix based on the Toeplitz structure of $p_n(\nu ,R_Q)$, given in (24).

$$\begin{array}{cc}& {\mathit{P}}_n=\hfill \\ & \left[\begin{array}{ccccccc}{p}_n(-L_p,R_Q)& 0& \cdots & \cdots & \cdots & \cdots & 0\\ p_n(-L_p+1,R_Q)& p_n(-L_p,R_Q)& 0& \cdots & \cdots & \cdots & 0\\ ⋮& & & \ddots & & & ⋮\\ p_n(L_p,R_Q)& \cdots & p_n(-L_p,R_Q)& 0& \cdots & \cdots & 0\\ 0& p_n(L_p,R_Q)& \cdots & p_n(-L_p,R_Q)& 0& \cdots & 0\\ ⋮& & & \ddots & & & ⋮\\ 0& \cdots & 0& p_n(L_p,R_Q)& \cdots & p_n(-L_p+1,R_Q)& p_n(-L_p,R_Q)\\ 0& \cdots & \cdots & 0& p_n(L_p,R_Q)& \cdots & p_n(-L_p+1,R_Q)\\ ⋮& & & \ddots & & & ⋮\\ 0& \cdots & \cdots & \cdots & 0& p_n(L_p,R_Q)& p_n(L_p-1,R_Q)\\ 0& \cdots & \cdots & \cdots & \cdots & 0& p_n(L_p,R_Q)\end{array}\right]\hfill \end{array}.$$

(24)

The size of matrix ${\mathit{P}}_n$ is $[2(L+L_p)+1,2L+1]$, where we choose the filter length $2L+1$ of ${\rho }_n(\nu ,R_Q)$ to be significantly larger than both $2L_p+1$ and the main lobe width of function $z_n\left(\nu \right)$, to avoid ${\mathit{P}}_n$ being ill-conditioned and minimize the error of truncating $z_n\left(\nu \right)$ into a finite length signal. The influence of choosing L will be detailed in Section 4.2.

Since (23) is an over-determined system of equations, we apply LMS method to (23) to obtain

$${\mathit{\rho }}_n={\mathit{P}}_n^{+}{\mathit{z}}_n,$$

(25)

where ${\mathit{P}}_n^{+}$ refers to the Moore-Penrose inverse of ${\mathit{P}}_n$. As a result, with (16) and (25), $a_{nm}\left(\nu \right)$ can be estimated by

$$\begin{array}{cc}\hfill a_{nm}\left(\nu \right)& \approx b_{nm}(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)\hfill \\ \hfill & =\left(\sum _{q=1}^{Q}S({\mathit{x}}_q,t)Y_{nm}({\theta }_q,{\varphi }_q)\right)\ast {\rho }_n(\nu ,R_Q).\hfill \end{array}$$

(26)

In this way we obtain $a_{nm}\left(\nu \right)$ while overcoming the challenges listed in Section 2.

3.3. Practical Considerations of Filter Implementation

In (26), $a_{nm}\left(\nu \right)$ is obtained by filtering $b_{nm}(\nu ,R_Q)$ with ${\rho }_n(\nu ,R_Q)$, where we get

$$b_{nm}(\nu ,R_Q)\ast {\rho }_n(\nu ,R_Q)=a_{nm}\left(\nu \right)\ast z_n\left(\nu \right)\approx a_{nm}\left(\nu \right).$$

(27)

Naturally $a_{nm}\left(\nu \right)$ at time index $\nu$ is only influenced by $[b_{nm}(\nu -L_p,R_Q),b_{nm}(\nu -L_p+1,R_Q),\cdots ,b_{nm}(\nu +L_p,R_Q)]$ because of the Legendre function in $p_n(\nu ,R_Q)$. However, with the influence of $\mathrm{sinc}$ functions in $z_n\left(\nu \right)$ in our proposed filters ${\rho }_n(\nu ,R_Q)$, we now need the past L samples and the future L samples of $b_{nm}(\nu ,R_Q)$ to obtain $a_{nm}\left(\nu \right)$ at time index $\nu$. For offline signal processing, L samples of zeros should be added both in the beginning and the end of the vector of $b_{nm}$ before filtering it with pre-designed ${\rho }_n(\nu ,R_Q)$. Moreover, an overlap of $2L+1$ samples is needed for frame based signal processing. For on-line real time signal processing, we cannot obtain future samples of $b_{nm}(\nu ,R_Q)$. As a result, we add L samples of zeros in front of the filter ${\rho }_n(\nu ,R_Q)$, and create a buffer of the past $2L+1$ samples of $b_{nm}(\nu ,R_Q)$. At time index $\nu$, we obtain $a_{nm}(\nu -L)$ with the buffer of $[b_{nm}(\nu -2L,R_Q),\cdots ,b_{nm}(\nu ,R_Q)]$. Thus, there is a L samples of group delay of the system. We further discuss and compare the group delay with frequency domain method in Section 5.5.

3.4. Error Analysis

We define the error $e_{nm}\left(v\right)$ as the difference between the desired time domain spherical harmonic and the coefficients we obtained by the proposed method, which can be decomposed to:

$$e_{nm}\left(v\right)=e_{\mathrm{filter}}(n,m,\nu )+e_{\mathrm{position}}(n,m)+e_{\mathrm{truncation}}\left(n\right),$$

(28)

where $e_{\mathrm{filter}}(n,m,\nu )$ is filtering error introduced by ${\rho }_n(\nu ,R_Q)$, $e_{\mathrm{truncation}}\left(n\right)$ is the truncation error of order N, and $e_{\mathrm{position}}(n,m)$ is due to the microphones position error. The qualitative analysis of $e_{\mathrm{truncation}}\left(n\right)$ and $e_{\mathrm{position}}(n,m)$ based on the frequency domain method are addressed in [3], where we draw a similar conclusion in time domain that with increasing number of microphones and fixed N, $e_{\mathrm{truncation}}\left(n\right)$ decreases. Meanwhile, $e_{\mathrm{position}}(n,m)$ depends on the nature of inaccurate microphone positioning, referring to the distance between the desired point and microphone location. We mainly focus on $e_{\mathrm{filter}}(n,m,\nu )$ here as it is the main error contribution due to the proposed filtering approach.

According to (27), $e_{\mathrm{filter}}(n,m,\nu )$ at a specific order n and mode m can be expressed as

$$\begin{array}{cc}\hfill e_{\mathrm{filter}}(n,m,\nu )& =|a_{nm}\left(\nu \right)\ast z_n\left(\nu \right)-a_{nm}\left(\nu \right)|\hfill \\ \hfill & =|a_{nm}\left(\nu \right)\ast e_n\left(\nu \right)|,\hfill \end{array}$$

(29)

where

$$e_n\left(\nu \right)\triangleq \delta \left(\nu \right)-z_n\left(\nu \right).$$

(30)

Using (18) and (21), the Fourier transform of $e_n\left(\nu \right)$ is

$$e_n\left(\nu \right)\stackrel{\mathcal{F}}{⟶}{E}_n\left(f\right)=\left\{\begin{array}{ccc}\hfill 1& & |j_n\left(kr\right)|\le ϵ,kr<N\hfill \\ \hfill 0& & \mathrm{otherwise}\hfill \end{array}\right.,$$

(31)

with the same truncation in frequency as ${\widehat{Z}}_n\left(f\right)$. Thus, $e_n\left(\nu \right)$ can be expressed as

$$\begin{array}{cc}\hfill e_n\left(\nu \right)=& \sum _{s=0}^{S-1}\left(w_{2s+1}^{\left(n\right)}-w_{2s}^{\left(n\right)}\right)\mathrm{sinc}\left(\frac{w_{2s+1}^{\left(n\right)}-w_{2s}^{\left(n\right)}}{2}\nu \right)\hfill \\ \hfill & \times cos\left(\frac{w_{2s+1}^{\left(n\right)}+w_{2s}^{\left(n\right)}}{2}\nu \right),\hfill \end{array}$$

(32)

where S and $w^{\left(n\right)}$ have the same definition as in (19) and $w_0^{\left(n\right)}=0$.

With (29) and (32) we can quantitatively calculate the filtering error $e_{\mathrm{filter}}(n,m,\nu )$ introduced by ${\rho }_n(\nu ,R_Q)$. The total error caused by filtering can be calculated by a summation of $e_{\mathrm{filter}}(n,m,\nu )$ over every order of n and mode of m. As this filtering error is mainly due to Bessel zeros, it can be reduced by limiting the highest order N of the system, where a smaller N results in lower Bessel zeros hence a smaller $e_{\mathrm{filter}}(n,m,\nu )$. Also, N depends on the highest wave number k and the radius of the microphone array $R_Q$. By choosing N with a pre-knowledge of the frequency limit of the input signals and $R_Q$ also helps to minimize the filtering error $e_{\mathrm{filter}}(n,m,\nu )$.

4. A Filter Design Example

To provide a further understanding of the filter design process, we present a design example of a fourth ($N=4$) order spherical microphone array of $R_Q=0.16$ m, designed to record the time domain spherical harmonic coefficients within the spatial region enclosed by the array with a desired frequency band of $[20,1360]$ Hz. Let $F_s=48,000$ Hz and $c=343$ m/s. Before we apply the proposed method to recording signals, we first analyze the influence of several steps in designing the proposed filter ${\rho }_n(\nu ,R_Q)$.

4.1. Effect of Frequency Truncation of $Z_n\left(\nu \right)$

As audio signals are often band limited in ANC applications [14], we can have a finite truncation on spherical harmonic decomposition with order $N=\lceil k{R}_Q\rceil$. In other words, if we have a fixed N-th order system, the highest frequency that the system can successfully capture is given by $f_{\max }=Nc/\left(2\pi R_Q\right)\approx 1360$ Hz. Figure 2 shows the frequency response of ${\rho }_n(\nu ,R_Q)$, refers to ${\mathsf{\Phi }}_n(f,R_Q)$, which is designed using (25) with $z_n\left(\nu \right)$ truncated at $f_1=1023.6$ Hz (Figure 2a), $f_2=1364.8$ Hz (Figure 2b), $f_3=2047.1$ Hz (Figure 2c), respectively. The filter length is set to be 500. To obtain the frequency response of ${\rho }_n(\nu ,R_Q)$, a FFT of $I=4096$ points is applied with zero padding to ${\rho }_n(\nu ,R_Q)$. We remind here that $z_n\left(\nu \right)$ is given by (19) in time domain, which does not rely on any frequency domain processing.

We observe that for a $N=4\mathrm{th}$ order system, the truncation at $f_1$ is not enough to get an accurate frequency response of ${\rho }_n(\nu ,R_Q)$, as the frequency response ${\mathsf{\Phi }}_n(f,R_Q)$ begins to decline at $f_1$. In this case, ${\rho }_n(\nu ,R_Q)$ can not provide an acceptable filtering result with signals containing higher frequency components. Truncation at both $f_2$ and $f_3$ can give a satisfied frequency response when $f<f_{\max }$. As the frequency range of the system is also limited by $N=\lceil k{R}_Q\rceil$, it is not necessary to look at the frequency response when $f>f_{\max }$. So in both cases ${\rho }_n(\nu ,R_Q)$ can give an acceptable filtering output. As a result, we choose to truncate $z_n\left(\nu \right)$ at $f_{\max }$, where $2\pi f_{\max }{R}_Q/c=N$. If the recorded signal is known as a band limited signal where its highest frequency component is less than $f_{\max }$, an alternative choice of the frequency truncation of $z_n\left(\nu \right)$ is at this highest frequency to reduce the computation complexity. Meanwhile, if $z_n\left(\nu \right)$ has been designed with a higher frequency truncation, it can also be used in a lower order system with a lower requirement of frequency truncation.

4.2. Choice of Filter Length of ${\rho }_n(\nu ,R_Q)$

Intuitively, a longer filter often brings us less error and better performance. Figure 3 supports this idea by showing the result of ${\rho }_n(\nu ,R_Q)\ast p_n(\nu ,R_Q)-z_n\left(\nu \right)$ with different choices of L, which refers to the error introduced into the system by the filtering processing. We observe that the filtering error decreases across all of the orders with a higher L. This is due to the time truncation of $z_n\left(\nu \right)$ (length of vector ${\mathit{z}}_n$ in (25)), being related to L. Thus, a higher L leads to less information loss in the time truncation of $z_n\left(\nu \right)$, hence smaller error in ${\rho }_n(\nu ,R_Q)$. However, Figure 4 shows the time domain filter ${\rho }_n(\nu ,R_Q)$ with different lengths. We observe that a longer filter results in a higher group delay of filtering. This is not desirable because it leads to a higher system delay of our proposed method, while lowering the system delay is one of the most important motivations that we develop the proposed time domain method.

As a result, we need to balance the noise tolerance, group delay, and the filtering error when we choose L. We suggest that filter length $2L+1$ should be significantly larger than the main lobe width of $z_n\left(\nu \right)$ and $2L_P+1$, the length of $p_n(\nu ,R_Q)$, but no more than 50 times of $2L_P+1$. Additionally, L should be less than the maximum tolerance of the delay of the system. Based on these guidelines, for the current example, we choose $2L+1=501$.

5. Simulation Results and Analysis

In this section, we evaluate the result of the proposed algorithm for time domain spherical harmonic analysis using a fourth order ($N=4$) system. We consider 32 microphones regularly placed on an open spherical array of $R_Q=0.16$ m, where the analysis region of interest is inside the array. A point source is placed at $[1,2,1]$ m with respect to the origin which coincides with the origin of the microphone array. The sampling frequency is 48,000 Hz, and the filter length $2L+1$ is 501. A noise signal at 40 dB SNR is added to each microphone to reflect thermal noise. Considering the application of the proposed method to be spatial ANC, we construct the desired frequency band to cover the target noise band, and construct the radius of the region to be wide enough to fit one human head.

It is difficult to validate our method in time domain directly because the coefficients are time dependent and no ground truth has been given. Therefore, we first validate our proposed time domain spherical harmonic coefficients in the frequency domain. Thus, we compare the Fourier transformation of the time domain coefficients to the theoretical frequency domain coefficients given in (4). Next, to clarify that our proposed method has the ability to record a sound field in the region of interest in the time domain, we reconstruct sound pressure at an arbitrary point as well as over a plane inside the region of interest with the captured time domain spherical harmonic coefficients by (6). Finally, the time delay of the proposed method is given.

5.1. Comparison between the Time Domain and the Frequency Domain Spherical Harmonic Coefficients

We use a narrow band signal at 1200 Hz to test if our proposed method can obtain time domain spherical harmonic coefficients $a_{nm}\left(\nu \right)$ correctly with (26). In (11), we give the relationship between $a_{nm}\left(\nu \right)$ and ${\alpha }_{nm}\left(k\right)$. We compare the Fourier transformation result of our obtained time domain spherical harmonic coefficients $\mathcal{FT}\left\{a_{nm}\left(\nu \right)\right\}$ with the desired frequency domain spherical harmonic coefficients ${\alpha }_{nm}\left(k\right)$, obtained by Equation (4) in frequency domain. Fourier transformations use $J=1024$ points. We do not compare the phase of these coefficients since the group delay of the time domain method and the frequency domain method is different. Instead, we compare the phase difference, given by ${\alpha }_{nm}\left(k\right)-{\alpha }_{n(m-1)}\left(k\right)$. The results of both amplitude and phase difference are shown in Figure 5.

In Figure 5 we see that there is little to no difference on both amplitude and phase difference between the Fourier Transformed time domain coefficients and the frequency domain coefficients over all the order and modes. Thus, our proposed time domain method successfully obtained the time domain spherical harmonic coefficients, which can be related to the frequency domain coefficients by Fourier transformation.

Next, we compare the coefficients over different frequencies with a wide band test signal within the frequency limited of $[20,1300]$ Hz. In Figure 6, we show the comparison of amplitude at $FT\left\{a_{00}\left(\nu \right)\right\}$ and ${\alpha }_{00}\left(k\right)$, $FT\left\{a_{11}\left(\nu \right)\right\}$ and ${\alpha }_{11}\left(k\right)$, and $FT\left\{a_{31}\left(\nu \right)\right\}$ and ${\alpha }_{31}\left(k\right)$ over frequencies respectively while Figure 7 shows the phase difference.

A huge error is observed in Figure 6a at the $46\mathrm{th}$ frequency bin. This error is due to that there is a Bessel zero of the zeroth order at this frequency bin (around 1072 Hz). We see the frequency domain spherical harmonic coefficients ${\alpha }_{00}\left(k\right)$ has a much higher amplitude, while our proposed method suppressed the amplitude at this certain frequency bin. Meanwhile, we can see in Figure 6 and Figure 7 that the error at $a_{31}\left(\nu \right)$ is higher compared to the other two modes. As order increases, the error increases. This error can be decreased by applying more microphones on the array. We also obtain a non-negligible error before the $30\mathrm{th}$ frequency bin of the coefficients amplitude for $(n,m)=(3,1)$ in Figure 6c and Figure 7c. This error is because our time domain proposed method and conventional frequency domain method have different processing on suppressing Bessel zeros. During the reconstruction process, the high pass property of spherical Bessel function removes the information at this frequency bin. Thus, this error will not influence the reconstruction of the sound field.

5.2. Sound Pressure Comparison at a Point Of Interest

In this section, we reconstruct the sound field with the captured time domain spherical harmonic coefficients at a point in the region of interest, and compare it with the desired sound field at the same point of interest. We use a signal containing three frequency components of 600 Hz, 850 Hz, and 1300 Hz. Figure 8 shows the desired sound pressure and the reconstructed sound pressure calculated by $a_{nm}\left(\nu \right)$ at the point $[-0.13,0.07,0.02]$ m and $[-0.03,0.01,0.1]$ m inside the region of interest in time domain. The desired sound field has been manually delayed for 272 samples to match the group delay of the reconstructed sound field, where the details of this delay will be shown in Section 5.5.

We note here that when reconstructing the sound-field with (6), we face the problem that at a point $\mathit{x}=(r,\theta ,\varphi )$ where the radius r is very small, the filter $p_n(\nu ,r)$, whose filter length dependents on $r{F}_s/c$, is too short to perform efficient filtering. To overcome this problem, we up-sample the obtained $a_{nm}\left(\nu \right)$ with a rate of $R_Q/r$ and construct corresponding $p_n(\nu ,r)$ with the same length of $L_p=2\ast R_Q{F}_s/c+1$. We then down-sample the resulting $b_{nm}(\nu ,r)$ with a rate of $r/R_Q$ to keep the sampling frequency consistent with $F_s$. We can see that the obtained $a_{nm}\left(\nu \right)$ by our proposed method can successfully reconstruct the sound pressure at a point inside the region of interest with a tolerable error. This supports that our time domain coefficients contain certain spatial information of the sound field that the sound pressure at an arbitrary point inside the region of interest can be properly calculated with the measurements only being taken on the boundary of the region.

5.3. Sound Field Comparison over a Plane

To further evaluate our method on reconstructing sound field over space, we now reconstruct the sound field by $a_{nm}\left(\nu \right)$ over a plane. We use a narrow band signal of 1200Hz here that the sound field in the region of interest is simple and clearly understood. Although the sound field is reconstructed over time, a 2D plot can only show the result of one time index. Figure 9 shows the reconstructed sound field and the desired sound field over the plane parallel to the x-y plane, with $z=0.02$ m at $t=0.3$ s. The 272 samples group delay is manually fixed and will be discussed later in the next subsection.

The white line in Figure 9 bounds the region of interest. We can see that the reconstructed sound field inside this region in Figure 9a is roughly the same as the desired sound field in Figure 9b. This confirms that the coefficients recorded by our proposed method are able to capture the sound field inside the region of interest.

5.4. Sound Field Error Estimation over The Region

To evaluate the reconstructed sound field over time, we calculate the instantaneous average squared spatial error over time, which is defined by

$$e\left(\nu \right)\triangleq \frac{{\sum }_{\mathsf{\Omega }}\parallel S_r(\mathit{x},\nu )-S_d{(\mathit{x},\nu )\parallel }^2}{{\sum }_{\mathsf{\Omega }}}.$$

(33)

Figure 10 shows how the error fluctuates with time in a tolerable range (no more than $5\times {10}^{-4}$) with a 900 Hz tone and a 1072 Hz tone. We have already observed in Figure 8 that the error of the sound pressure at a point of interest is proportional to the desired sound pressure. We observe the same trend when we evaluate the error over the region that the error increases when the sound field inside the region of interest is at peak amplitude. We also observe in Figure 10 that the error with 1072 Hz signal is higher than 900 Hz signal. This is due to that there is a Bessel zero of the zeroth order ($j_0\left(kr\right)$) at 1072 Hz in the proposed spatial ANC system. Hence, the amplitude of $a_{00}\left(\nu \right)$ is suppressed by the proposed method, leading to a higher error in reconstructing the sound field.

5.5. Processing Delay Analysis

In this section, we indicate the group delay of our method. Figure 11 shows the desired sound pressure and the reconstructed sound pressure of a signal containing three frequency components of $[600,850,1300]$ Hz at the point $[-0.13,0.07,0.02]$ m. We can obtain from Figure 11 that the processing delay of the system is $1046-774=272$ samples, which equals to $L+\lceil R_Q{F}_s/c\rceil$. The L samples of the delay is from the group delay of the proposed filter ${\rho }_n(\nu ,R_Q)$, while $R_Q{F}_s/c$ is the delay introducing by the Legendre function within filter $p_n(\nu ,r)$ to reconstruct the sound pressure at a point with the time domain spherical harmonic coefficients. Comparing to a conventional frequency domain scenario with 512 frame-size and $75\%$ of overlap Short Time Fourier transformation, which refers to a 2048 samples of delay [43], our proposed method can significantly reduce the processing delay.

Comparing to one of the start-of-art frequency domain spherical harmonic filter designs [44], which states a 75 ms delay with a 900 sample long filter, our method can achieve a 972 samples (20.25 ms with 48k Hz sampling frequency) delay with the same length of filter. Meanwhile, as our method is processed in time domain, there is nothing to stop us from doing a sample by sample signal processing instead of frame based signal processing. This sample based processing considerably extends the application of spherical harmonic analysis.

6. Conclusions

In this paper, a time domain spherical harmonic analysis method for spatial sound field recording over 3D space has been developed with the goal to minimize processing delay. This favours the specific application of spatial ANC. With the proposed FIR filter design, the time domain spherical harmonic coefficients can be obtained from the sound pressure measurements of an open spherical microphone array. The filters are designed based on the inverse of the Legendre function. Additionally, the filters are modified with considerations of stability and practical implementation. We have provided simulation results proving the validity of the proposed method.

We note that by obtaining the proposed time domain spherical harmonic coefficients, the desired sound field can be efficiently captured and reconstructed over space. The proposed time domain spherical harmonic coefficients can be related to the conventional frequency domain coefficients, where both have the same location independent property. The proposed method has the prominent advantage of lower delay since it is developed in the time domain without the introduction of a Fourier transformation or inverse Fourier transformation. Furthermore, the proposed time domain filtering method can support sample based signal processing instead of frame based, which indicates that the frame size can be one sample if necessary. As a result, we consider the proposed time domain spherical harmonic analysis method to be highly suitable for a spatial ANC system where accurate spatial recording with low delay is desired.

The most important future work is practically introducing the proposed spatial recording method to a spatial ANC system. Currently the proposed method utilizes open spherical microphone arrays, where the difficulties of constructing this array limit the potential applications. Hence, applying the proposed method to alternative optimised open microphone arrays is another direction for future work.