Spectral Object Recognition in Hyperspectral Holography with Complex-Domain Denoising

Abstract

In this paper, we have applied a recently developed complex-domain hyperspectral denoiser for the object recognition task, which is performed by the correlation analysis of investigated objects’ spectra with the fingerprint spectra from the same object. Extensive experiments carried out on noisy data from digital hyperspectral holography demonstrate a significant enhancement of the recognition accuracy of signals masked by noise, when the advanced noise suppression is applied.

1. Introduction

Hyperspectral imaging (HSI) is used to obtain the spectral distributions for each pixel of the image of a scene. One of the main goals of this spectral analysis is to classify an image and its fragments. This is possible due to availability of the detailed spectral data with thousands of spectrum lines typical for HSI. The specific spectra, called fingerprints or spectrum signatures, are used for object recognition and classification.

HSI is extremely effective in obtaining spectral data in many applications such as earth remote sensing [1], terahertz imaging [2], and medical imaging [3]. For example, highly spectrally resolved images provide spectral signatures unique to facial skin tissue that may not be detected using common CCD cameras, which cannot distinguish well between different spectral signals [4]. Spectral recognition is well known in Fourier transform infrared spectroscopy (FTIR), which provides a biochemical fingerprint of a biopsy sample and, together with advanced data analysis techniques, can classify cells. However, one of the challenges when dealing with FTIR imaging is the slow recording of the data such that a one square centimeter tissue section requires several hours of image recording [5].

Hyperspectral Digital Holography (HSDH) is a technique based on FT spectrometer but uses multipixel detectors instead of single-pixel ones like in FTIR, with that advantage scanning of tissues is possible ${10}^6$ faster just due to the larger area of the detector. As an additional advantage, HSDH provides information on both object amplitude and phase in each image pixel [6,7]. With that information, one can obtain spectra from the area of interest and provide a recognition and/or process detection. However, HS data are severely corrupted by noise, which drastically changes the shape of spectra and ruins the recognition process. It is a major challenge that analytical biosensing platforms face, as the signal from ensemble measurements, at low concentrations, is masked by the background noise [8].

Traditionally, many rooting techniques are used for denoising in HSDH, but they do not help much in noise suppression, working for low noise levels [7,9], but failing in the high ones [10]. It is explained by a slice-wise separate filtering which does not process all HS cube slices jointly. In particular, they fail due to high noise levels on spectral components with low intensity of a radiation source, which causes a low signal-to-noise ratio (SNR). An additional challenge in HSDH is a phase filtering, which can not be done by a traditional image noise suppression due to the angle-like behavior of the phase. Recently, a noise suppression technique for complex-domain HS cube ($\mathcal{CCF}$) was developed for an additive Gaussian noise [10,11,12]. It processes all slices of the whole HS cube jointly, significantly improving the noise suppression and retrieving HS data that were lost due to noise. It is demonstrated in [10] that $\mathcal{CCF}$ algorithm produces good quality HS object reconstruction from extremely noisy data with SNR down to -8 dB. A great advantage of the $\mathcal{CCF}$ algorithm is that it is applicable for any type of complex-domain HS data, regardless of how it was obtained.

The contribution of this paper concerns an application of $\mathcal{CCF}$ filtering for HS object recognition and classification. We show that with the proper noise suppression by $\mathcal{CCF}$, it is possible to significantly improve HSDH processing and to overcome problems of signal masking by background noise despite a high level of noise.

2. Problem Formulation

Let $U(x,y,\lambda )\subset {\mathbb{C}}^{N\times M}$ be a slice of the complex-valued HS cube of the size $N\times M$ on $(x,y)$ provided a fixed wavelength $\lambda$, and $Q_{\Lambda }(x,y)=\{U(x,y,\lambda ),$ $\lambda \subset \Lambda \}$, $Q_{\Lambda }\subset {\mathbb{C}}^{N\times M\times L_{\Lambda }}$, be a whole cube composed of a set of wavelengths $\Lambda$ with the number of individual wavelengths $L_{\Lambda }$. Thus, the total size of the cube is $N\times M\times L_{\Lambda }$ pixels.

The observations of the hyperspectral denoising problem under the additive noise assumption can be written as:

$${\mathrm{Z}}_{\Lambda }(x,y)=Q_{\Lambda }(x,y)+{\epsilon }_{\Lambda }(x,y),$$

(1)

where ${\mathrm{Z}}_{\Lambda },Q_{\Lambda }$, ${\epsilon }_{\Lambda }\subset {\mathbb{C}}^{N\times M\times L_{\Lambda }}$ represent the recorded noisy HS data, clean HS data and additive noise, respectively. Accordingly to the notation for the clean image, the noisy cube can be represented as ${\mathrm{Z}}_{\Lambda }(x,y)=\{\mathrm{Z}(x,y,\lambda ),$ $\lambda \in \Lambda \}$, ${\mathrm{Z}}_{\Lambda }\subset {\mathbb{C}}^{N\times M\times L_{\Lambda }}$ with the slices $\mathrm{Z}(x,y,\lambda )$.

The denoising problem is formulated as a reconstruction of unknown $Q_{\Lambda }(x,y)$ from the given ${\mathrm{Z}}_{\Lambda }(x,y)$. The properties of the clean HS $Q_{\Lambda }(x,y)$ and the noise ${\epsilon }_{\Lambda }(x,y)$ are essential for the algorithm development.

The denoising algorithm relies upon three basic and essential assumptions [13]: Similarity of the HS slices $U(x,y,\lambda )$ for close values of $\lambda$ follows from the fact that $U(x,y,\lambda )$ are slowly varying functions of $\lambda$; Sparsity of the HS images $U(x,y,\lambda )$ as functions of $(x,y)$ means that there are bases such that $U(x,y,\lambda )$ can be represented with a small number of elements of these bases; Noise ${\epsilon }_{\Lambda }(x,y)$ is zero mean Gaussian with unknown correlation matrix $L_{\Lambda }\times L_{\Lambda }$. Clean image subspace identification is a crucial first step in the developed algorithm. The signal and noise correlation matrices are estimated and then used to select the subset of eigenvectors that best represents the signal subspace in the least mean squared error sense.

Complex Cube Filter Algorithm for HSI Denoising

Here we briefly describe Complex Cube Filter ($\mathcal{CCF}$) algorithm (for more detailed description please see the paper [10]). The algorithm is used with the following notations:

$${\widehat{U}}_{\overline{\Lambda }}(x,y)=\mathcal{CCF}\{{\mathrm{Z}}_{\overline{\Lambda }}(x,y),\overline{\Lambda }\subset \Lambda \}.$$

(2)

Here, $\overline{\Lambda }$ is a set of slices to be denoised. $\mathcal{CCF}$ processes data of the noisy cube ${\mathrm{Z}}_{\overline{\Lambda }}(x,y)$ jointly and provides the estimates ${\widehat{U}}_{\overline{\Lambda }}(x,y)$ for all $\lambda \in$ $\overline{\Lambda }$. A principal flow work of the $\mathcal{CCF}$ algorithm is presented in Figure 1 and is composed of the following steps.

• Calculation of the orthonormal transform matrix $E\subset {\mathbb{C}}^{L_{\overline{\Lambda }}\times p}$ and the 2D transform domain eigenimage ${\mathrm{Z}}_{2D,eigen}$ as:

  $$[E,{\mathrm{Z}}_{2D,eigen},p]=HySime\left(\mathrm{Z}\right),$$

  (3)

  where HySime stays for Hyperspectral signal Subspace Identification by Minimum Error [14] based on Singular Value Decomposition (SVD). p is a length of the eigenspace, which is automatically identified by a selection of the subset of eigenvalues that best represents the signal subspace in the least squared error sense. When E is given, the eigenimage is calculated as:

  $${\mathrm{Z}}_{2D,eigen}=E^H\mathrm{Z}.$$

  (4)
• Filtering of each of the $N\times M$ 2D images (slices) of ${\mathrm{Z}}_{3D,eigen}$ by Complex-Domain Block-Matching 3D (CDBM3D) algorithm [15]:

  $${\widehat{\mathrm{Z}}}_{3D,eigen}(x,y,{\lambda }_s)=CDBM3D\left({\mathrm{Z}}_{3D,eigen}(x,y,{\lambda }_s)\right).$$

  (5)

  where p eigenvalues ${\lambda }_s$ belong to the eigenspace.
• Returning from the eigenimages of the transform domain to the 2D original image space as follows

  $${\widehat{\mathrm{Z}}}_{2D}=E{\widehat{\mathrm{Z}}}_{2D,eigen}.$$

  (6)

Forward and backward passages 2D⇆3D signed between the algorithm steps define the reshapes from 2D to 3D and vice versa for proper processing of the HS data, the subscript (2D or 3D) denotes the number of dimensions. It is needed to produce SVD transform since it works only with 2D data, and to produce the CDBM3D filtering in the corresponding 3D domain ${\mathrm{Z}}_{3D,eigen}$ slice-by-slice. However, in order to return these filtered data ${\widehat{\mathrm{Z}}}_{3D,eigen}$ into the original image space we need to use 2D transform (6) and, thus again to reshape data into 2D transform space. Optimization in step 1 of the HS data results in minimization of its size and usually obtained subspace dimension much smaller than the initial one $p\ll L_{\overline{\Lambda }}$. This leads to a shorter processing time of the algorithm. Thus, the CDBM3D filtering is produced only for p eigenimages but the backward transform (6) gives the estimates for all $L_{\overline{\Lambda }}$ spectral images.

3. Spectral Analysis

Traditionally, for spectral analysis of an object, unknown spectra obtained in the experiments are compared with the known standard spectral fingerprint databases [16]. Otherwise, standard spectral fingerprints might be obtained directly from the experimental HS cube. For this purpose, averaging of spectra from an area with known objects’ material (or process) is produced. Such fingerprint creation is free from external databases and, therefore, is useful for the new processes or object detection which are not recognized before and do not have standard databases. However, this approach is sensitive to noise in data and, thus, a recognition becomes complicated and might even fail.

A comparison of the fingerprint spectrum with the spectrum in question can be produced based on the correlation analysis. In this paper, for this analysis we use the formula:

$$\rho (A,B)=\frac{1}{N-1}\sum _{i=1}^N\left(\frac{A_i-{\mu }_A}{{\sigma }_A}\right)\left(\frac{B_i-{\mu }_B}{{\sigma }_B}\right).$$

(7)

Here A and B are two compared spectra; ${\mu }_A$ and ${\sigma }_A$ are the mean and standard deviation of a spectrum A, respectively, and ${\mu }_B$ and ${\sigma }_B$ are the mean and standard deviation of fingerprint spectrum B; N is a number of points (spectral lines) in the spectrum. In this comparison, the averaging is produced over the whole spectral interval and it should be done for all pixels of the images in a pixel-by-pixel manner. Thus, as a result, we obtain an image of the correlations $\rho (A,B)$.

4. Spectral Object Recognition

For objects recognition demonstration, we took experimental HS data obtained by transmissive HS digital holography technique [7], we do not give a detailed description of the HSDH technique here to emphasize that $\mathcal{CCF}$ algorithm does not depend on the way how HS data is obtained. For the given data the lowest SNR $=6.7$ dB is for the slice $\lambda =787$ nm, the highest SNR $=20.5$ dB is for the slice $\lambda =576$ nm, and average SNR $=12.6$ dB. The investigated object was a transparent sample with different color letters, which correspond to different spectra. The object slice is demonstrated in Figure 2 and the used light source spectrum in Figure 3. As it is seen from Figure 2, the investigated object has a simple structure with 3 colors (red, green, blue) and the transparent area which should be recognized. In the right image of Figure 2, the small squares show areas of the object of different colors selected in order to define the standard fingerprints. An HS cube corresponding to that object consists of $807\times 807\times 226$ pixels, which in turn corresponds to 651249 spectra to be recognized.

Using $\mathcal{CCF}$ algorithm, we produced a noise suppression for the initial noisy HS cube (left column) and obtain filtered HS cube (see Figure 4, middle column). The top row of Figure 4 demonstrates amplitudes and the bottom one - phases. The HS slice in Figure 4 corresponds to 599 nm and illustrates effective noise suppression and object details recovery from the noisy slice by the $\mathcal{CCF}$ algorithm. To emphasis spectral recognition improvement, we compare $\mathcal{CCF}$ filtering results not only with initial noisy data but also with the Fast Hyperspectral Denoising (FastHyDe) technique [13], which is developed for HS imaging but only for real-valued data. The filtering result by FastHyDe is in the right column of Figure 4, it produces also visually good noise suppression for the amplitudes, but do not provide correct phase filtering.

In Figure 5, the normalized spectrum fingerprints for each color of the object are presented, they are obtained from noisy (solid lines with circles), filtered by $\mathcal{CCF}$ (dash lines with triangles), and filtered by FastHyDe (dash-dot lines with squares) HS data. The fingerprints for each color have distinctive peaks in the corresponding range of the wavelengths, however, it is clearly seen that for the fingerprints obtained from the noisy HS data background signal is higher, especially for the red and blue spectra. Due to this noisy background, the true signal information might be lost. For the FastHyDe spectra, the extra peaks are registered for red, green, and blue colors, which might be explained by a leakage of the signal during the SVD transform. For $\mathcal{CCF}$ fingerprints we do not observe such behavior due to effective noise suppression which along with the SVD transform utilizes amplitude-phase correlations in CDBM3D.

Figure 6 demonstrates correlation maps (images) obtained by (7): left column is for noisy HS data, central column is for filtered by $\mathcal{CCF}$, and the right one is for the filtered by FastHyDe. Colorbar relates to all images and represents correlation value, the white color corresponds to high correlation, black to the low one. Even for such a simple object, it is hard to recognize different spectral regions due to the high noise level in the HS data. From noisy data, object recognition in the blue and red regions is failed since the noise level in these spectrum regions was high compared to the level of intensity of the used LED masking the true signal. Green and white regions are recognized better but with low correlation values with the standard fingerprints.

For FastHyDe correlation maps, we observe higher correlation values for the whole object with barely separated object features. It is explained by the similarity of the fingerprints provided by FastHyDe (see Figure 5). Completely different correlation maps are for $\mathcal{CCF}$ filtered HS data, where all different colors are well recognized. For the red, green, and blue colors of the object, the recognition improvement due to $\mathcal{CCF}$ filtering is obvious since the corresponding features have the highest correlations.

To evaluate the performance of the recognition, in Figure 7 we demonstrate Receiver Operating Characteristic (ROC) [17] curves for recognition from each data set for every color of the object with different correlation thresholds from 0 to 1, step 0.1. To estimate a recognition probability, logistic regression was used [18]. In Figure 7, solid curves with circles correspond to recognition from noisy HS data, dash curves with triangles correspond to recognition from filtered by $\mathcal{CCF}$ HS data, and dash-dot curves with squares—to filtered by FastHyDe. According to ROC curves, the best recognition is performed for blue color features from filtered by $\mathcal{CCF}$ HS data, the second-best is for recognition from green color also filtered by $\mathcal{CCF}$ HS data. Significant recognition improvement comparatively to noisy data is demonstrated for the red color, where for noisy data it is almost a random guess. In the region of low false-positive rate values, the highest true positive rate values for each color except white are also for the $\mathcal{CCF}$ filtered data. For the white color, no improvement is obtained which is explained by the good-enough fingerprint spectrum obtained from the noisy data. FastHyDe filtering has not improved recognition for white, blue, and green colors, however, it significantly improved red color recognition, which is supported also by a high value of area under ROC curves (AUC), presented in

Table 1. AUC.

Color	Noisy Data	CCF	FastHyDe
red	0.6416	0.8293	0.8986
green	0.8767	0.9102	0.8443
blue	0.8428	0.9478	0.8008
white	0.8054	0.7652	0.7838

. The highest AUC values are for filtered data by $\mathcal{CCF}$ for green and blue colors, which confirms detection improvement due to $\mathcal{CCF}$ noise suppression. For the white region, AUC is higher for noisy data, which means that for the transparent regions of the object (places without features) there is no need in noise suppression since, for the given spectral analysis, the fingerprint spectrum correlation with the estimated ones is high.

5. Conclusions

Application of the complex domain HS cube filter to the HS object recognition is performed. It is demonstrated that, with a proper noise suppression, it is possible to significantly improve the classification process. It becomes possible to recognize spectral features masked by a noise. We would like to emphasize that due to general denoising problem formulation $\mathcal{CCF}$ algorithm works with any type of HS data with additive Gaussian noise, therefore it is applicable for a wide range of tasks. Due to such flexibility, it is compatible with other more sophisticated recognition techniques, as for example with the use of convolutional neural networks [19].