# Spectral Object Recognition in Hyperspectral Holography with Complex-Domain Denoising

^{1}

^{2}

^{3}

^{4}

^{*}

Faculty of Information Technology and Communication Sciences, Tampere University, FI-33101 Tampere, Finland

Department of Photonics and Optical Information Technology, ITMO University, 197101 St. Petersburg, Russia

Institut für Lasertechnologien in der Medizin und Messtechnik, Helmholtzstraße 12, 89081 Ulm, Germany

Institut für Technische Optik (ITO), Universität Stuttgart, Pfaffenwaldring 9, 70569 Stuttgart, Germany

Author to whom correspondence should be addressed.

Received: 7 October 2019 / Revised: 22 November 2019 / Accepted: 24 November 2019 / Published: 26 November 2019

(This article belongs to the Section Optical Sensors)

In this paper, we have applied a recently developed complex-domain hyperspectral denoiser for the object recognition task, which is performed by the correlation analysis of investigated objects’ spectra with the fingerprint spectra from the same object. Extensive experiments carried out on noisy data from digital hyperspectral holography demonstrate a significant enhancement of the recognition accuracy of signals masked by noise, when the advanced noise suppression is applied.

Hyperspectral imaging (HSI) is used to obtain the spectral distributions for each pixel of the image of a scene. One of the main goals of this spectral analysis is to classify an image and its fragments. This is possible due to availability of the detailed spectral data with thousands of spectrum lines typical for HSI. The specific spectra, called fingerprints or spectrum signatures, are used for object recognition and classification.

HSI is extremely effective in obtaining spectral data in many applications such as earth remote sensing [1], terahertz imaging [2], and medical imaging [3]. For example, highly spectrally resolved images provide spectral signatures unique to facial skin tissue that may not be detected using common CCD cameras, which cannot distinguish well between different spectral signals [4]. Spectral recognition is well known in Fourier transform infrared spectroscopy (FTIR), which provides a biochemical fingerprint of a biopsy sample and, together with advanced data analysis techniques, can classify cells. However, one of the challenges when dealing with FTIR imaging is the slow recording of the data such that a one square centimeter tissue section requires several hours of image recording [5].

Hyperspectral Digital Holography (HSDH) is a technique based on FT spectrometer but uses multipixel detectors instead of single-pixel ones like in FTIR, with that advantage scanning of tissues is possible ${10}^{6}$ faster just due to the larger area of the detector. As an additional advantage, HSDH provides information on both object amplitude and phase in each image pixel [6,7]. With that information, one can obtain spectra from the area of interest and provide a recognition and/or process detection. However, HS data are severely corrupted by noise, which drastically changes the shape of spectra and ruins the recognition process. It is a major challenge that analytical biosensing platforms face, as the signal from ensemble measurements, at low concentrations, is masked by the background noise [8].

Traditionally, many rooting techniques are used for denoising in HSDH, but they do not help much in noise suppression, working for low noise levels [7,9], but failing in the high ones [10]. It is explained by a slice-wise separate filtering which does not process all HS cube slices jointly. In particular, they fail due to high noise levels on spectral components with low intensity of a radiation source, which causes a low signal-to-noise ratio (SNR). An additional challenge in HSDH is a phase filtering, which can not be done by a traditional image noise suppression due to the angle-like behavior of the phase. Recently, a noise suppression technique for complex-domain HS cube ($\mathcal{CCF}$) was developed for an additive Gaussian noise [10,11,12]. It processes all slices of the whole HS cube jointly, significantly improving the noise suppression and retrieving HS data that were lost due to noise. It is demonstrated in [10] that $\mathcal{CCF}$ algorithm produces good quality HS object reconstruction from extremely noisy data with SNR down to -8 dB. A great advantage of the $\mathcal{CCF}$ algorithm is that it is applicable for any type of complex-domain HS data, regardless of how it was obtained.

The contribution of this paper concerns an application of $\mathcal{CCF}$ filtering for HS object recognition and classification. We show that with the proper noise suppression by $\mathcal{CCF}$, it is possible to significantly improve HSDH processing and to overcome problems of signal masking by background noise despite a high level of noise.

Let $U(x,y,\lambda )\subset {\mathbb{C}}^{N\times M}$ be a slice of the complex-valued HS cube of the size $N\times M$ on $(x,y)$ provided a fixed wavelength $\lambda $, and ${Q}_{\Lambda}(x,y)=\{U(x,y,\lambda ),$ $\lambda \subset \Lambda \}$, ${Q}_{\Lambda}\subset {\mathbb{C}}^{N\times M\times {L}_{\Lambda}}$, be a whole cube composed of a set of wavelengths $\Lambda $ with the number of individual wavelengths ${L}_{\Lambda}$. Thus, the total size of the cube is $N\times M\times {L}_{\Lambda}$ pixels.

The observations of the hyperspectral denoising problem under the additive noise assumption can be written as:
where ${\mathrm{Z}}_{\Lambda},{Q}_{\Lambda}$, ${\epsilon}_{\Lambda}\subset {\mathbb{C}}^{N\times M\times {L}_{\Lambda}}$ represent the recorded noisy HS data, clean HS data and additive noise, respectively. Accordingly to the notation for the clean image, the noisy cube can be represented as ${\mathrm{Z}}_{\Lambda}(x,y)=\{\mathrm{Z}(x,y,\lambda ),$ $\lambda \in \Lambda \}$, ${\mathrm{Z}}_{\Lambda}\subset {\mathbb{C}}^{N\times M\times {L}_{\Lambda}}$ with the slices $\mathrm{Z}(x,y,\lambda )$.

$${\mathrm{Z}}_{\Lambda}(x,y)={Q}_{\Lambda}(x,y)+{\epsilon}_{\Lambda}(x,y),$$

The denoising problem is formulated as a reconstruction of unknown ${Q}_{\Lambda}(x,y)$ from the given ${\mathrm{Z}}_{\Lambda}(x,y)$. The properties of the clean HS ${Q}_{\Lambda}(x,y)$ and the noise ${\epsilon}_{\Lambda}(x,y)$ are essential for the algorithm development.

The denoising algorithm relies upon three basic and essential assumptions [13]: Similarity of the HS slices $U(x,y,\lambda )$ for close values of $\lambda $ follows from the fact that $U(x,y,\lambda )$ are slowly varying functions of $\lambda $; Sparsity of the HS images $U(x,y,\lambda )$ as functions of $(x,y)$ means that there are bases such that $U(x,y,\lambda )$ can be represented with a small number of elements of these bases; Noise ${\epsilon}_{\Lambda}(x,y)$ is zero mean Gaussian with unknown correlation matrix ${L}_{\Lambda}\times {L}_{\Lambda}$. Clean image subspace identification is a crucial first step in the developed algorithm. The signal and noise correlation matrices are estimated and then used to select the subset of eigenvectors that best represents the signal subspace in the least mean squared error sense.

Here we briefly describe Complex Cube Filter ($\mathcal{CCF}$) algorithm (for more detailed description please see the paper [10]). The algorithm is used with the following notations:
Here, $\overline{\Lambda}$ is a set of slices to be denoised. $\mathcal{CCF}$ processes data of the noisy cube ${\mathrm{Z}}_{\overline{\Lambda}}(x,y)$ jointly and provides the estimates ${\widehat{U}}_{\overline{\Lambda}}(x,y)$ for all $\lambda \in $ $\overline{\Lambda}$. A principal flow work of the $\mathcal{CCF}$ algorithm is presented in Figure 1 and is composed of the following steps.

$${\widehat{U}}_{\overline{\Lambda}}(x,y)=\mathcal{CCF}\{{\mathrm{Z}}_{\overline{\Lambda}}(x,y),\phantom{\rule{4.pt}{0ex}}\overline{\Lambda}\subset \Lambda \}.$$

- Calculation of the orthonormal transform matrix $E\subset {\mathbb{C}}^{{L}_{\overline{\Lambda}}\times p}$ and the 2D transform domain eigenimage ${\mathrm{Z}}_{2D,eigen}$ as:$$[E,{\mathrm{Z}}_{2D,eigen},p]=HySime\left(\mathrm{Z}\right),$$$${\mathrm{Z}}_{2D,eigen}={E}^{H}\mathrm{Z}.$$
- Filtering of each of the $N\times M$ 2D images (slices) of ${\mathrm{Z}}_{3D,eigen}$ by Complex-Domain Block-Matching 3D (CDBM3D) algorithm [15]:$${\widehat{\mathrm{Z}}}_{3D,eigen}(x,y,{\lambda}_{s})=CDBM3D\left({\mathrm{Z}}_{3D,eigen}(x,y,{\lambda}_{s})\right).$$
- Returning from the eigenimages of the transform domain to the 2D original image space as follows$${\widehat{\mathrm{Z}}}_{2D}=E{\widehat{\mathrm{Z}}}_{2D,eigen}.$$

Forward and backward passages 2D⇆3D signed between the algorithm steps define the reshapes from 2D to 3D and vice versa for proper processing of the HS data, the subscript (2D or 3D) denotes the number of dimensions. It is needed to produce SVD transform since it works only with 2D data, and to produce the CDBM3D filtering in the corresponding 3D domain ${\mathrm{Z}}_{3D,eigen}$ slice-by-slice. However, in order to return these filtered data ${\widehat{\mathrm{Z}}}_{3D,eigen}$ into the original image space we need to use 2D transform (6) and, thus again to reshape data into 2D transform space. Optimization in step 1 of the HS data results in minimization of its size and usually obtained subspace dimension much smaller than the initial one $p\ll {L}_{\overline{\Lambda}}$. This leads to a shorter processing time of the algorithm. Thus, the CDBM3D filtering is produced only for p eigenimages but the backward transform (6) gives the estimates for all ${L}_{\overline{\Lambda}}$ spectral images.

Traditionally, for spectral analysis of an object, unknown spectra obtained in the experiments are compared with the known standard spectral fingerprint databases [16]. Otherwise, standard spectral fingerprints might be obtained directly from the experimental HS cube. For this purpose, averaging of spectra from an area with known objects’ material (or process) is produced. Such fingerprint creation is free from external databases and, therefore, is useful for the new processes or object detection which are not recognized before and do not have standard databases. However, this approach is sensitive to noise in data and, thus, a recognition becomes complicated and might even fail.

A comparison of the fingerprint spectrum with the spectrum in question can be produced based on the correlation analysis. In this paper, for this analysis we use the formula:

$$\rho (A,B)=\frac{1}{N-1}\sum _{i=1}^{N}\left(\frac{{A}_{i}-{\mu}_{A}}{{\sigma}_{A}}\right)\left(\frac{{B}_{i}-{\mu}_{B}}{{\sigma}_{B}}\right).$$

Here A and B are two compared spectra; ${\mu}_{A}$ and ${\sigma}_{A}$ are the mean and standard deviation of a spectrum A, respectively, and ${\mu}_{B}$ and ${\sigma}_{B}$ are the mean and standard deviation of fingerprint spectrum B; N is a number of points (spectral lines) in the spectrum. In this comparison, the averaging is produced over the whole spectral interval and it should be done for all pixels of the images in a pixel-by-pixel manner. Thus, as a result, we obtain an image of the correlations $\rho (A,B)$.

For objects recognition demonstration, we took experimental HS data obtained by transmissive HS digital holography technique [7], we do not give a detailed description of the HSDH technique here to emphasize that $\mathcal{CCF}$ algorithm does not depend on the way how HS data is obtained. For the given data the lowest SNR $=6.7$ dB is for the slice $\lambda =787$ nm, the highest SNR $=20.5$ dB is for the slice $\lambda =576$ nm, and average SNR $=12.6$ dB. The investigated object was a transparent sample with different color letters, which correspond to different spectra. The object slice is demonstrated in Figure 2 and the used light source spectrum in Figure 3. As it is seen from Figure 2, the investigated object has a simple structure with 3 colors (red, green, blue) and the transparent area which should be recognized. In the right image of Figure 2, the small squares show areas of the object of different colors selected in order to define the standard fingerprints. An HS cube corresponding to that object consists of $807\times 807\times 226$ pixels, which in turn corresponds to 651249 spectra to be recognized.

Using $\mathcal{CCF}$ algorithm, we produced a noise suppression for the initial noisy HS cube (left column) and obtain filtered HS cube (see Figure 4, middle column). The top row of Figure 4 demonstrates amplitudes and the bottom one - phases. The HS slice in Figure 4 corresponds to 599 nm and illustrates effective noise suppression and object details recovery from the noisy slice by the $\mathcal{CCF}$ algorithm. To emphasis spectral recognition improvement, we compare $\mathcal{CCF}$ filtering results not only with initial noisy data but also with the Fast Hyperspectral Denoising (FastHyDe) technique [13], which is developed for HS imaging but only for real-valued data. The filtering result by FastHyDe is in the right column of Figure 4, it produces also visually good noise suppression for the amplitudes, but do not provide correct phase filtering.

In Figure 5, the normalized spectrum fingerprints for each color of the object are presented, they are obtained from noisy (solid lines with circles), filtered by $\mathcal{CCF}$ (dash lines with triangles), and filtered by FastHyDe (dash-dot lines with squares) HS data. The fingerprints for each color have distinctive peaks in the corresponding range of the wavelengths, however, it is clearly seen that for the fingerprints obtained from the noisy HS data background signal is higher, especially for the red and blue spectra. Due to this noisy background, the true signal information might be lost. For the FastHyDe spectra, the extra peaks are registered for red, green, and blue colors, which might be explained by a leakage of the signal during the SVD transform. For $\mathcal{CCF}$ fingerprints we do not observe such behavior due to effective noise suppression which along with the SVD transform utilizes amplitude-phase correlations in CDBM3D.

Figure 6 demonstrates correlation maps (images) obtained by (7): left column is for noisy HS data, central column is for filtered by $\mathcal{CCF}$, and the right one is for the filtered by FastHyDe. Colorbar relates to all images and represents correlation value, the white color corresponds to high correlation, black to the low one. Even for such a simple object, it is hard to recognize different spectral regions due to the high noise level in the HS data. From noisy data, object recognition in the blue and red regions is failed since the noise level in these spectrum regions was high compared to the level of intensity of the used LED masking the true signal. Green and white regions are recognized better but with low correlation values with the standard fingerprints.

For FastHyDe correlation maps, we observe higher correlation values for the whole object with barely separated object features. It is explained by the similarity of the fingerprints provided by FastHyDe (see Figure 5). Completely different correlation maps are for $\mathcal{CCF}$ filtered HS data, where all different colors are well recognized. For the red, green, and blue colors of the object, the recognition improvement due to $\mathcal{CCF}$ filtering is obvious since the corresponding features have the highest correlations.

To evaluate the performance of the recognition, in Figure 7 we demonstrate Receiver Operating Characteristic (ROC) [17] curves for recognition from each data set for every color of the object with different correlation thresholds from 0 to 1, step 0.1. To estimate a recognition probability, logistic regression was used [18]. In Figure 7, solid curves with circles correspond to recognition from noisy HS data, dash curves with triangles correspond to recognition from filtered by $\mathcal{CCF}$ HS data, and dash-dot curves with squares—to filtered by FastHyDe. According to ROC curves, the best recognition is performed for blue color features from filtered by $\mathcal{CCF}$ HS data, the second-best is for recognition from green color also filtered by $\mathcal{CCF}$ HS data. Significant recognition improvement comparatively to noisy data is demonstrated for the red color, where for noisy data it is almost a random guess. In the region of low false-positive rate values, the highest true positive rate values for each color except white are also for the $\mathcal{CCF}$ filtered data. For the white color, no improvement is obtained which is explained by the good-enough fingerprint spectrum obtained from the noisy data. FastHyDe filtering has not improved recognition for white, blue, and green colors, however, it significantly improved red color recognition, which is supported also by a high value of area under ROC curves (AUC), presented in Table 1. The highest AUC values are for filtered data by $\mathcal{CCF}$ for green and blue colors, which confirms detection improvement due to $\mathcal{CCF}$ noise suppression. For the white region, AUC is higher for noisy data, which means that for the transparent regions of the object (places without features) there is no need in noise suppression since, for the given spectral analysis, the fingerprint spectrum correlation with the estimated ones is high.

Application of the complex domain HS cube filter to the HS object recognition is performed. It is demonstrated that, with a proper noise suppression, it is possible to significantly improve the classification process. It becomes possible to recognize spectral features masked by a noise. We would like to emphasize that due to general denoising problem formulation $\mathcal{CCF}$ algorithm works with any type of HS data with additive Gaussian noise, therefore it is applicable for a wide range of tasks. Due to such flexibility, it is compatible with other more sophisticated recognition techniques, as for example with the use of convolutional neural networks [19].

I.S. and V.K. developed the algorithm; D.C. and G.P. conceived and designed the experiments; V.K., I.S., N.V.P. and K.E. analyzed the data and wrote the paper.

Jane and Aatos Erkko Foundation and Finland Centennial Foundation funded Computational Imaging without Lens (CIWIL) project; Russian Ministry of Education and Science (project within the state mission for institutions of higher education, agreement 3.1893.2017/4.6).

The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

The following abbreviations are used in this manuscript:

CCD | Charge-coupled device |

CCF | Complex Cube Filter |

HSI | HyperSpectral Imaging |

FTIR | Fourier Transform Infrared Spectroscopy |

HSDH | HyperSpectral Digital Holography |

HySime | Hyperspectral signal Subspace Identification by Minimum Error |

SVD | Singular Value Decomposition |

CDBM3D | Complex-Domain Block-Matching 3D |

LED | Light Emitting Diode |

FastHyDe | Fast Hyperspectral Denoising |

SNR | Signal-to-Noise Ratio |

ROC | Receiver Operating Characteristic |

AUC | Area Under Curve |

- Pandey, P.C.; Anand, A.; Srivastava, P.K. Spatial distribution of mangrove forest species and biomass assessment using field inventory and earth observation hyperspectral data. Biodivers. Conserv.
**2019**, 28, 2143–2162. [Google Scholar] [CrossRef] - Kulya, M.S.; Balbekin, N.S.; Gredyuhina, I.V.; Uspenskaya, M.V.; Nechiporenko, A.P.; Petrov, N.V. Computational terahertz imaging with dispersive objects. J. Mod. Opt.
**2017**, 64, 1283–1288. [Google Scholar] [CrossRef] - Lu, G.; Fei, B. Medical hyperspectral imaging: A review. J. Biomed. Opt.
**2014**, 19, 010901. [Google Scholar] [CrossRef] - Chang, H.; Koschan, A.; Abidi, M.; Kong, S.G.; Won, C.H. Multispectral visible and infrared imaging for face recognition. In Proceedings of the 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, Anchorage, AK, USA, 23–28 June 2008; pp. 1–6. [Google Scholar]
- Ali, M.H.; Rakib, F.; Al-Saad, K.; Al-Saady, R.; Lyng, F.M.; Goormaghtigh, E. A simple model for cell type recognition using 2D-correlation analysis of FTIR images from breast cancer tissue. J. Mol. Struct.
**2018**, 1163, 472–479. [Google Scholar] [CrossRef] - Kalenkov, S.G.; Kalenkov, G.S.; Shtanko, A.E. Hyperspectral holography: An alternative application of the Fourier transform spectrometer. J. Opt. Soc. Am. B
**2017**, 34, B49. [Google Scholar] [CrossRef] - Claus, D.; Pedrini, G.; Buchta, D.; Osten, W. Accuracy enhanced and synthetic wavelength adjustable optical metrology via spectrally resolved digital holography. J. Opt. Soc. Am. A Opt. Image Sci. Vis.
**2018**, 35, 546–552. [Google Scholar] [CrossRef] [PubMed] - Yesilkoy, F.; Arvelo, E.R.; Jahani, Y.; Liu, M.; Tittl, A.; Cevher, V.; Kivshar, Y.; Altug, H. Ultrasensitive hyperspectral imaging and biodetection enabled by dielectric metasurfaces. Nat. Photonics
**2019**, 13, 390–396. [Google Scholar] [CrossRef] - Kalenkov, G.S.; Kalenkov, S.G.; Meerovich, I.G.; Shtanko, A.E.; Zaalishvili, N.Y. Hyperspectral holographic microscopy of bio-objects based on a modified Linnik interferometer. Laser Phys.
**2019**, 29, 016201. [Google Scholar] [CrossRef] - Shevkunov, I.; Katkovnik, V.; Claus, D.; Pedrini, G.; Petrov, N.; Egiazarian, K. Hyperspectral phase imaging based on denoising in complex-valued eigensubspace. arXiv
**2019**, arXiv:1907.03104. [Google Scholar] - Katkovnik, V.; Shevkunov, I.; Claus, D.; Pedrini, G.; Egiazarian, K. Hyperspectral phase imaging with denoising in SVD image subspace. In Proceedings of the Digital Holography and Three-Dimensional Imaging 2019, Bordeaux, France, 19–23 May 2019; Optical Society of America: Washington, DC, USA, 2019; p. W1B.2. [Google Scholar]
- Katkovnik, V.; Shevkunov, I.; Claus, D.; Pedrini, G.; Egiazarian, K. Non-local similarity complex domain denoising for hyperspectral phase imaging. In Proceedings of the OPAL Conference, Amsterdam, The Netherlands, 24 April 2019. [Google Scholar]
- Zhuang, L.; Bioucas-Dias, J.M. Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2018**, 11, 730–742. [Google Scholar] [CrossRef] - Bioucas-Dias, J.M.; Nascimento, J.M. Hyperspectral subspace identification. IEEE Trans. Geosci. Remote Sens.
**2008**, 46, 2435–2445. [Google Scholar] [CrossRef] - Katkovnik, V.; Egiazarian, K. Sparse phase imaging based on complex domain nonlocal BM3D techniques. Digit. Signal Process. A Rev. J.
**2017**, 63, 72–85. [Google Scholar] [CrossRef] - Haghighat, M.; Abdel-Mottaleb, M.; Alhalabi, W. Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition. IEEE Trans. Inf. Forensics Secur.
**2016**, 11, 1984–1996. [Google Scholar] [CrossRef] - Fawcett, T. ROC graphs: Notes and practical considerations for researchers. Mach. Learn.
**2004**, 31, 1–38. [Google Scholar] - Jessen, H.C.; Menard, S. Applied Logistic Regression Analysis. Statistician
**1996**, 45, 534–535. [Google Scholar] [CrossRef] - Xu, X.; Li, W.; Ran, Q.; Du, Q.; Gao, L.; Zhang, B. Multisource remote sensing data classification based on convolutional neural network. IEEE Trans. Geosci. Remote Sens.
**2017**, 56, 937–949. [Google Scholar] [CrossRef]

Color | Noisy Data | CCF | FastHyDe |
---|---|---|---|

red | 0.6416 | 0.8293 | 0.8986 |

green | 0.8767 | 0.9102 | 0.8443 |

blue | 0.8428 | 0.9478 | 0.8008 |

white | 0.8054 | 0.7652 | 0.7838 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).