# Hyperspectral Face Recognition with Patch-Based Low Rank Tensor Decomposition and PFFT Algorithm

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

_{d}and varying the other indices. Next, we discuss several different tensor products as follows. The mode-d product of a tensor and a matrix is expressed as $\underset{\_}{T}=\underset{\_}{G}{\times}_{d}S$, which is equivalent to ${T}_{\left(d\right)}=G{S}_{\left(d\right)}$. An outer product of a tensor is defined as $\underset{\_}{C}=\underset{\_}{S}\circ \underset{\_}{T}={s}_{{n}_{1}{n}_{2}\dots {n}_{Q}}{t}_{{l}_{1}{l}_{2}\dots {l}_{P}}$, where $\underset{\_}{T}\in {\mathbb{R}}^{{L}_{1}\times {L}_{2}\times \dots \times {L}_{d}\times \dots \times {L}_{P}}$ and $\underset{\_}{S}\in {\mathbb{R}}^{{N}_{1}\times {N}_{2}\times \dots \times {N}_{d}\times \dots \times {N}_{Q}}$ yields $\underset{\_}{C}\in {\mathbb{R}}^{{N}_{1}\times \dots \times {N}_{Q}\times {L}_{1}\dots \times {L}_{P}}$. Since CPD decomposes a high-order tensor into a multiple tensor sum of rank-1, it is typically used for factorizing data into easy-to-interpret components. For example, considering rank-1 decomposition for a diagonal tensor $\underset{\_}{T}\in {\mathbb{R}}^{{L}_{1}\times {L}_{2}\times \dots \times {L}_{d}\times \dots \times {L}_{P}}$, it can be expressed as an outer product sum of N vectors under a special case:

## 3. Extraction Features for Hyperspectral Facial Image

#### 3.1. Local Feature Extraction

#### 3.1.1. Discriminative Patch Selection

^{2}patterns, which raised a large number of data processing problems. In order to overcome this weakness, a uniform pattern LBP was employed to solve this problem. The uniform pattern, as defined as the cyclic binary number corresponding to an LBP, has two hops from 0 to 1 or from 1 to 0. In the present research, we used the uniform pattern LBP method to extract features.

_{k}is presented by the Fisher ratio criterion, which is the simplest criterion [42]. A large β

_{k}value indicates better class separation capacity, where a multi-category case is explained as follows:

_{ij}is a two-category case between the ith and the jth class, that is

#### 3.1.2. Discriminant Local Feature Extraction

^{S}represents the spectral band number (l

^{W}< L

^{W}, l

^{H}< L

^{H}). The second step aims to integrate patches with the same features into a fourth-order tensor to extract discriminative features, as illustrated in Figure 5a. As the fourth-order tensors are redundant in both spatial and spectral modes, the third step is to establish a block-sparsity coefficient tensor and three dictionaries matrix to solve the problem of low SNR and high data dimensionality, as illustrated in Figure 5b.

_{1}norm. Note that matrix dictionary learning can be expanded into 3D space tensor by:

_{k}indicates the total number of kth patch features; and n is the total number of local features. In addition, ${D}^{W}\in {\mathbb{R}}^{{l}^{W}\times {d}^{W}}$, ${D}^{H}\in {\mathbb{R}}^{{l}^{H}\times {d}^{H}}$, and ${D}^{S}\in {\mathbb{R}}^{{l}^{S}\times {d}^{S}}$ (l

^{W}× d

^{W}, l

^{H}× d

^{H}, l

^{S}× d

^{S}), ${Z}^{k}\in {\mathbb{R}}^{{d}^{W}\times {d}^{H}\times {d}^{S}}$ are the redundant coefficient tensor of patch ${X}^{\left(k\right)}$.

#### 3.2. Global Feature Extraction

## 4. Ensemble Classification for Hyperspectral Facial Images

#### 4.1. Construction of Ensemble Classifier

_{k}is the kth LPC’s properly classified probability obtained by Native Bayes (NB) and w

_{k}represents the weight of kth LPC. Second, the $\xi $ is combined with the global classifier $\varphi $ to form an ELGC, denoted as C,

_{c}is the weight of LPC.

#### 4.2. Combination of Multiple Classifiers

## 5. Results and Discussion

#### 5.1. Hyperspectral Face Database

#### 5.2. Experiments on Global and Local Classifiers

#### 5.2.1. Experiments on the Global Classifier

#### 5.2.2. Experiments on the Local Classifier

#### Rank Parameters Learning and Patch Selection

#### Local Patch Classifiers and Their Ensemble

#### 5.3. Experiments on the Ensemble Classifier

#### 5.3.1. Comparison of Different Ensemble Methods

#### 5.3.2. Comparison of Different Classification Methods

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Lu, G.; Fei, B. Medical hyperspectral imaging: A review. J. Biomed. Opt.
**2014**, 19, 10901. [Google Scholar] [CrossRef] [PubMed] - Zonios, G.; Bykowski, J.; Kollias, N. Skin Melanin, Hemoglobin, and Light Scattering Properties Can Be Quantitatively Assessed In Vivo Using Diffuse Reflectance Spectroscopy. J. Investig. Dermatol.
**2001**, 117, 1452–1457. [Google Scholar] [CrossRef] [PubMed] - Pan, Z.; Healey, G.E.; Prasad, M.; Tromberg, B.J. Face recognition in hyperspectral images. IEEE Trans. Pattern Anal. Mach. Intell.
**2003**, 25, 1552–1560. [Google Scholar] [Green Version] - Ryer, D.M.; Bihl, T.J.; Bauer, K.W.; Rogers, S.K. QUEST hierarchy for hyperspectral face recognition. Adv. Artif. Intell.
**2012**, 2012, 1. [Google Scholar] [CrossRef] - Uzair, M.; Mahmood, A.; Shafait, F.; Nansen, C.; Mian, A. Is spectral reflectance of the face a reliable biometric? Opt. Express
**2015**, 23, 15160–15173. [Google Scholar] [CrossRef] [PubMed] - Di, W.; Zhang, L.; Zhang, D.; Pan, Q. Studies on Hyperspectral Face Recognition in Visible Spectrum with Feature Band Selection. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**2010**, 40, 1354–1361. [Google Scholar] [CrossRef] - Uzair, M.; Mahmood, A.; Mian, A. Hyperspectral Face Recognition using 3D-DCT and Partial Least Squares. In Proceedings of the BMVC, Bristol, UK, 9–13 September 2013. [Google Scholar]
- Uzair, M.; Mahmood, A.; Mian, A. Hyperspectral Face Recognition with Spatiospectral Information Fusion and PLS Regression. Image Process. IEEE Trans.
**2015**, 24, 1127–1137. [Google Scholar] [CrossRef] - Shen, L.; Zheng, S. Hyperspectral face recognition using 3D Gabor wavelets. In Proceedings of the International Conference on Pattern Recognition, Tsukuba, Japan, 11–15 November 2012; pp. 1574–1577. [Google Scholar]
- Zhu, Z.; Jia, S.; He, S.; Sun, Y.; Ji, Z.; Shen, L. Three-dimensional Gabor feature extraction for hyperspectral imagery classification using a memetic framework. Inf. Sci.
**2015**, 298, 274–287. [Google Scholar] [CrossRef] - Pan, Z.; Healey, G.; Tromberg, B. Comparison of Spectral-Only and Spectral/Spatial Face Recognition for Personal Identity Verification. Eurasip J. Adv. Signal Process.
**2009**, 2009, 8. [Google Scholar] [CrossRef] - Turk, M.A.; Pentland, A.P. Face recognition using eigenfaces. J. Cogn. Neurosci.
**2012**, 3, 71–86. [Google Scholar] [CrossRef] - Mairal, J.; Elad, M.; Sapiro, G. Sparse Representation for Color Image Restoration. IEEE Trans. Image Process.
**2007**, 17, 53–69. [Google Scholar] [CrossRef] - Wright, J.; Ma, Y.; Mairal, J.; Sapiro, G.; Huang, T.S.; Yan, S. Sparse Representation for Computer Vision and Pattern Recognition. Proc. IEEE
**2010**, 98, 1031–1044. [Google Scholar] [CrossRef] [Green Version] - Yang, M.; Zhang, L.; Yang, J.; Zhang, D. Robust sparse coding for face recognition. In Proceedings of the Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 20–25 June 2011; pp. 625–632. [Google Scholar]
- Vasilescu, M.A.O.; Terzopoulos, D. Multilinear Analysis of Image Ensembles: TensorFaces. In Proceedings of the European Conference on Computer Vision, Copenhagen, Denmark, 28–31 May 2002; pp. 447–460. [Google Scholar]
- Savas, B.; Eldén, L. Handwritten digit classification using higher order singular value decomposition. Pattern Recognit.
**2007**, 40, 993–1003. [Google Scholar] [CrossRef] - Li, C.; Ma, Y.; Huang, J.; Mei, X.; Ma, J. Hyperspectral image denoising using the robust low-rank tensor recovery. J. Opt. Soc. Am. A Opt. Image Sci. Vis.
**2015**, 32, 1604–1612. [Google Scholar] [CrossRef] [PubMed] - Lai, J.H.; Yuen, P.C.; Feng, G.C. Face recognition using holistic Fourier invariant features. Pattern Recognit.
**2001**, 34, 95–109. [Google Scholar] [CrossRef] - Sao, A.K.; Yegnanarayana, B. On the use of phase of the Fourier transform for face recognition under variations in illumination. Signal Image Video Process.
**2010**, 4, 353–358. [Google Scholar] [CrossRef] - Jing, X.Y.; Wong, H.S.; Zhang, D. Face recognition based on discriminant fractional Fourier feature extraction. Pattern Recognit. Lett.
**2006**, 27, 1465–1471. [Google Scholar] [CrossRef] - Zhang, D.; Ding, D.; Li, J.; Liu, Q. A novel way to improve facial expression recognition by applying Fast Fourier Transform. In Proceedings of the International MultiConference of Engineers and Computer Scientists, Hong Kong, China, 12–14 March 2014. [Google Scholar]
- Angadi, S.A.; Kagawade, V.C. A Robust Face Recognition Approach through Symbolic Modeling of Polar FFT features. Pattern Recognit.
**2017**, 71, 235–248. [Google Scholar] [CrossRef] - Zhao, R.; Li, X.; Sun, P. An improved windowed Fourier transform filter algorithm. Opt. Laser Technol.
**2015**, 74, 103–107. [Google Scholar] [CrossRef] - Yan, Y.; Wang, H.; Suter, D. Multi-subregion based correlation filter bank for robust face recognition. Pattern Recognit.
**2014**, 47, 3487–3501. [Google Scholar] [CrossRef] [Green Version] - Li, H.; Shen, F.; Shen, C.; Yang, Y.; Gao, Y. Face recognition using linear representation ensembles. Pattern Recognit.
**2016**, 59, 72–87. [Google Scholar] [CrossRef] - Zhang, L.; Zhou, W.D.; Li, F.Z. Kernel sparse representation-based classifier ensemble for face recognition. Multimed. Tools Appl.
**2015**, 74, 123–137. [Google Scholar] [CrossRef] - Zhu, P.; Zhang, L.; Hu, Q.; Shiu, S.C.K. Multi-scale patch based collaborative representation for face recognition with margin distribution optimization. In Proceedings of the European Conference on Computer Vision, Florence, Italy, 7–13 October 2012; pp. 822–835. [Google Scholar]
- Cheng, J.; Chen, L. A Weighted Regional Voting Based Ensemble of Multiple Classifiers for Face Recognition. In Proceedings of the International Symposium on Visual Computing, Las Vegas, NV, USA, 8–10 December 2014; pp. 482–491. [Google Scholar]
- Sidiropoulos, N.D.; Lathauwer, L.D.; Fu, X.; Huang, K.; Papalexakis, E.E.; Faloutsos, C. Tensor Decomposition for Signal Processing and Machine Learning. IEEE Trans. Signal Process.
**2017**, 65, 3551–3582. [Google Scholar] [CrossRef] - Phan, A.H.; Cichocki, A. Tensor decompositions for feature extraction and classification of high dimensional datasets. Nonlinear Theory Its Appl. Ieice
**2010**, 1, 37–68. [Google Scholar] [CrossRef] [Green Version] - Pfeifer, R.N.C.; Evenbly, G.; Singh, S.; Vidal, G. NCON: A tensor network contractor for MATLAB. arXiv, 2014; arXiv:1402.0939. [Google Scholar]
- Cichocki, A.; Mandic, D.; Lathauwer, L.D.; Zhou, G.; Zhao, Q.; Caiafa, C.; Phan, H.A. Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis. IEEE Signal Process. Mag.
**2014**, 32, 145–163. [Google Scholar] [CrossRef] - Russell, S.J.; Norvig, P.N. Artificial Intelligence: A Modern Approach. Prentice Hall. Appl. Mech. Mater.
**2003**, 263, 2829–2833. [Google Scholar] - Shi, Y.; Eberhart, R.C. Empirical study of particle swarm optimization. In Proceedings of the 1999 Congress on Evolutionary Computation-CEC99, Washington, DC, USA, 6–9 July 1999; Volume 321, pp. 320–324. [Google Scholar]
- Wang, X.; Han, T.X.; Yan, S. An HOG-LBP human detector with partial occlusion handling. In Proceedings of the 2009 IEEE International Conference on Computer Vision, Kyoto, Japan, 29 September–2 October 2009; pp. 32–39. [Google Scholar]
- Ameur, B.; Masmoudi, S.; Derbel, A.G.; Hamida, A.B. Fusing Gabor and LBP feature sets for KNN and SRC-based face recognition. In Proceedings of the 2016 International Conference on Advanced Technologies for Signal and Image Processing, Monastir, Tunisia, 21–23 March 2016; pp. 453–458. [Google Scholar]
- Lyons, M.; Akamatsu, S.; Kamachi, M.; Gyoba, J. Coding facial expressions with Gabor wavelets. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 14–16 April 1998; pp. 200–205. [Google Scholar] [Green Version]
- Yang, M.; Zhang, L.; Shiu, S.C.K.; Zhang, D. Gabor feature based robust representation and classification for face recognition with Gabor occlusion dictionary. Pattern Recognit.
**2013**, 46, 1865–1878. [Google Scholar] [CrossRef] [Green Version] - Ojala, T.; Harwood, I. A Comparative Study of Texture Measures with Classification Based on Feature Distributions. Pattern Recognit.
**1996**, 29, 51–59. [Google Scholar] [CrossRef] - Ojala, T.; Pietikainen, M.; Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell.
**2002**, 24, 971–987. [Google Scholar] [CrossRef] [Green Version] - Duda, R.O.; Hart, P.E. Pattern Classification and Scene Analysis; Wiley: Hoboken, NJ, USA, 1973; pp. 119–131. [Google Scholar]
- Du, B.; Zhang, M.; Zhang, L.; Hu, R.; Tao, D. PLTD: Patch-Based Low-Rank Tensor Decomposition for Hyperspectral Images. IEEE Trans. Multimed.
**2017**, 19, 67–79. [Google Scholar] [CrossRef] - Peng, Y.; Meng, D.; Xu, Z.; Gao, C.; Yang, Y.; Zhang, B. Decomposable Nonlocal Tensor Dictionary Learning for Multispectral Image Denoising. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 2949–2956. [Google Scholar]
- Saito, N. Simultaneous Noise Suppression and Signal Compression Using a Library of Orthonormal Bases and the Minimum-Description-Length Criterion; Academic Press: Cambridge, MA, USA, 1994; pp. 299–324. [Google Scholar]
- Averbuch, A.; Coifman, R.R.; Donoho, D.L.; Elad, M.; Israeli, M. Fast and accurate Polar Fourier transform. Appl. Comput. Harmon. Anal.
**2006**, 21, 145–167. [Google Scholar] [CrossRef] - Gonzalez, R.C.; Woods, R.E. Digital Image Processing; Prentice Hall International: Upper Saddle River, NJ, USA, 1992; Volume 28, pp. 484–486. [Google Scholar]
- Kittler, J.; Hatef, M.; Duin, R.P.W.; Matas, J. On Combining Classifiers. In Proceedings of the International Conference on Pattern Recognition, Vienna, Austria, 25–29 August 1996. [Google Scholar]
- Ellis, H.D.; Frse, M.A.J.; Newcombe, F.; Young, A. Aspects of Face Processing; Springer Science & Business Media: Berlin, Germany, 1986. [Google Scholar]
- Su, Y.; Shan, S.; Chen, X.; Gao, W. Hierarchical Ensemble of Global and Local Classifiers for Face Recognition. IEEE Trans. Image Process.
**2009**, 18, 1885–1896. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Eberhart, R.; Kennedy, J. A new optimizer using particle swarm theory. In Proceedings of the Sixth International Symposium on Micro Machine and Human Science, Nagoya, Japan, 4–6 October 1995; pp. 39–43. [Google Scholar]
- Mao, S.; Xiong, L.; Jiao, L.C.; Zhang, S.; Chen, B. Weighted ensemble based on 0–1 matrix decomposition. Electron. Lett.
**2013**, 49, 116–118. [Google Scholar] [CrossRef] - Robila, S.A. Toward hyperspectral face recognition. Proc. SPIE
**2008**, 6812, 68120. [Google Scholar] - Chang, C.C.; Lin, C.J. A Library for Support Vector Machines. ACM Trans. Intell. Syst. Technol.
**2011**, 2. [Google Scholar] [CrossRef] - Zhang, L.; Yang, M. Sparse representation or collaborative representation: Which helps face recognition? In Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 471–478. [Google Scholar]

**Figure 1.**Spectral reflectance values of a hyperspectral facial image at the check of the same area of interest in four different periods.

**Figure 3.**Band fusion process of a hyperspectral facial image. The images (

**left**) represent a 3D hyperspectral image, and the (

**right**) image is the fused 2D image.

**Figure 5.**(

**a**) Local samples of mouth with fourth-order tensor form. (

**b**) Sparse representation for a 4D tensor with a block-sparsity coefficient tensor and three dictionaries matrix.

Local Feature | Eyebrow | Nose | Mouth | Eyebrows and Eyes | Eyes and Nose |
---|---|---|---|---|---|

The number of atoms | [14, 31, 2] | [15, 15, 2] | [36, 15, 2] | [36, 36, 12] | [19, 19, 3] |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Wu, M.; Wei, D.; Zhang, L.; Zhao, Y.
Hyperspectral Face Recognition with Patch-Based Low Rank Tensor Decomposition and PFFT Algorithm. *Symmetry* **2018**, *10*, 714.
https://doi.org/10.3390/sym10120714

**AMA Style**

Wu M, Wei D, Zhang L, Zhao Y.
Hyperspectral Face Recognition with Patch-Based Low Rank Tensor Decomposition and PFFT Algorithm. *Symmetry*. 2018; 10(12):714.
https://doi.org/10.3390/sym10120714

**Chicago/Turabian Style**

Wu, Mengmeng, Dongmei Wei, Liren Zhang, and Yuefeng Zhao.
2018. "Hyperspectral Face Recognition with Patch-Based Low Rank Tensor Decomposition and PFFT Algorithm" *Symmetry* 10, no. 12: 714.
https://doi.org/10.3390/sym10120714