# A Lightweight Hyperspectral Image Anomaly Detector for Real-Time Mission

^{*}

## Abstract

**:**

## 1. Introduction

- To maximize the accuracy and throughput, an underlying relationship between AHCF and detection accuracy is built. It is described as a multi-objective optimization problem (MOP) to explore a probable structure and data width with less anomaly detection accuracy loss.
- By solving the above MOP with a non-dominated sorting genetic algorithm II (NSGA-II), a P-Q-AD is implemented. In P-Q-AD, to speed up the detection processing, the redundant neurons are removed to shrink the network and a mixed precision network is implemented with a delicate customized fixed-point data expression.

## 2. Materials and Methods

#### 2.1. Data Sets

#### 2.2. Deep Learning Based Online HSI AD

## 3. The Pruning-Quantization-Anomaly-Detector

#### 3.1. Basic Analysis for Real-Time Onboard HSI AD

#### 3.2. The Detection Accuracy Criterion for MOP

#### 3.3. The Quantified Detection Speed with Network Structure and Arithmetic Units in Hardware

#### 3.4. The Multiobjective Optimization with NSGA-II

## 4. Experiments and Results

#### 4.1. Experiment Environment and Evaluation Criteria

#### 4.2. Results and Discussion

## 5. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Yang, C.; Lee, W.S.; Gader, P. Hyperspectral band selection for detecting different blueberry fruit maturity stages. Comput. Electron. Agric.
**2014**, 109, 23–31. [Google Scholar] [CrossRef] - Weng, Q.; Hu, X.; Lu, D. Extracting Impervious Surfaces from Medium Spatial Resolution Multispectral and Hyperspectral Imagery: A Comparison. Int. J. Remote Sens.
**2008**, 29, 3209–3232. [Google Scholar] [CrossRef] - Transon, J.; Andrimont, R.; Maugnard, A. Survey of Hyperspectral Earth Observation Applications from Space in the Sentinel-2 Context. Remote Sens.
**2018**, 10, 157. [Google Scholar] [CrossRef] - Li, X.; Zhang, L.; You, J. Hyperspectral Image Classification Based on Two-Stage Subspace Projection. Remote Sens.
**2018**, 10, 1565. [Google Scholar] [CrossRef] - Nasrabadi, N.M. Hyperspectral Target Detection. IEEE Signal Process. Mag.
**2014**, 31, 34–44. [Google Scholar] [CrossRef] - Dong, Y.; Du, B.; Zhang, L.; Hu, X. Hyperspectral Target Detection via Adaptive Information-Theoretic Metric Learning with Local Constraints. Remote Sens.
**2018**, 10, 1415. [Google Scholar] [CrossRef] - Zhao, C.; Wang, Y.; Qi, B.; Wang, J. Global and Local Real-Time Anomaly Detectors for Hyperspectral Remote Sensing Imagery. Remote Sens.
**2015**, 7, 3966–3985. [Google Scholar] [CrossRef] [Green Version] - Zhu, L.; Wen, G.; Qiu, S. Low-Rank and Sparse Matrix Decomposition with Cluster Weighting for Hyperspectral Anomaly Detection. Remote Sens.
**2018**, 10, 707. [Google Scholar] [CrossRef] - Matteoli, S.; Diani, M.; Corsini, G. A Tutorial Overview of Anomaly Detection in Hyperspectral Images. IEEE Aerosp. Electron. Syst. Mag.
**2010**, 25, 5–27. [Google Scholar] [CrossRef] - Feng, J.; Fang, X.J.; Cao, X.; Ma, C.G.; Dai, Q.H.; Zhu, H.B.; Wang, Y.J. Advanced hyperspectral video imaging system using Amici prism. Opt. Express
**2014**, 22, 19348–19356. [Google Scholar] [CrossRef] [Green Version] - Reed, I.S.; Yu, X.L. Adaptive multiple-band CFAR detection of an optical pattern with unknown spectral distribution. IEEE Trans. Acoust. Speech Signal Process.
**1990**, 38, 1760–1770. [Google Scholar] [CrossRef] - Chang, C.I.; Chiang, S.S. Anomaly detection and classification for hyperspectral imagery. IEEE Trans. Geosci. Remote. Sens.
**2002**, 40, 1314–1325. [Google Scholar] [CrossRef] [Green Version] - Du, B.; Zhang, L. Random-Selection-Based Anomaly Detector for Hyperspectral Imagery. IEEE Trans. Geosci. Remote. Sens.
**2011**, 49, 1578–1589. [Google Scholar] [CrossRef] - Chang, C.-I.; Li, Y.; Hobbs, M.C.; Schultz, R.C.; Liu, W.-M. Progressive Band Processing of Anomaly Detection in Hyperspectral Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2015**, 8, 3558–3571. [Google Scholar] [CrossRef] - Chen, S.-Y.; Wang, Y.; Wu, C.-C.; Liu, C.; Chang, C.-I. Real-Time Causal Processing of Anomaly Detection for Hyperspectral Imagery. IEEE Trans. Aerosp. Electron. Syst.
**2014**, 50, 1510–1533. [Google Scholar] [CrossRef] - Li, W.; Du, Q. Collaborative Representation for Hyperspectral Anomaly Detection. IEEE Trans. Geosci. Remote. Sens.
**2015**, 53, 1463–1474. [Google Scholar] [CrossRef] - Li, J.; Zhang, H.; Zhang, L.; Ma, L. Hyperspectral Anomaly Detection by the Use of Background Joint Sparse Representation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2015**, 8, 2523–2533. [Google Scholar] [CrossRef] - Ma, D.; Yuan, Y.; Wang, Q. Hyperspectral Anomaly Detection via Discriminative Feature Learning with Multiple-Dictionary Sparse Representation. Remote Sens.
**2018**, 10, 745. [Google Scholar] [CrossRef] - Li, F.; Zhang, X.; Zhang, L. Exploiting Structured Sparsity for Hyperspectral Anomaly Detection. IEEE Trans. Geosci. Remote. Sens.
**2018**, 56, 4050–4064. [Google Scholar] [CrossRef] - Liang, H.; Li, Q. Hyperspectral Imagery Classification Using Sparse Representations of Convolutional Neural Network Features. Remote Sens.
**2016**, 8, 99. [Google Scholar] [CrossRef] - Zhao, C.; Li, X.; Zhu, H. Hyperspectral anomaly detection based on stacked denoising autoencoders. J. Appl. Remote Sens.
**2017**, 11, 1–19. [Google Scholar] [CrossRef] - Ma, N.; Wang, S.; Yu, J.; Peng, Y. A DBN based anomaly targets detector for HSI. In Proceedings of the Aopc 2017: 3D Measurement Technology for Intelligent Manufacturing, Beijing, China, 4–6 June 2017; Volume 10458, pp. 1–6. [Google Scholar] [CrossRef]
- Ma, N.; Peng, Y.; Wang, S.; Gao, W. A weight SAE based hyperspectral image anomaly targets detection. In Proceedings of the International Conference on Electronic Measurement & Instruments (ICEMI), Yangzhou, China, 20–22 October 2017; pp. 511–515. [Google Scholar] [CrossRef]
- Li, W.; Wu, G.; Du, Q. Transferred Deep Learning for Anomaly Detection in Hyperspectral Imagery. IEEE Geosci. Remote Sens. Lett.
**2017**, 5, 597–601. [Google Scholar] [CrossRef] - Lopez, S.; Vladimirova, T.; Gonzalez, C.; Resano, J.; Mozos, D.; Plaza, A. The Promise of Reconfigurable Computing for Hyperspectral Imaging Onboard Systems: A Review and Trends. Proc. IEEE
**2013**, 101, 698–722. [Google Scholar] [CrossRef] - Han, S.; Mao, H.; Dally, W.J. Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. arXiv
**2015**, arXiv:1510.00149. [Google Scholar] - Fujii, T.; Sato, S.; Nakahara, H. A Threshold Neuron Pruning for a Binarized Deep Neural Network on an FPGA. IEICE Trans. Inf. Syst.
**2018**, 101, 376–386. [Google Scholar] [CrossRef] - Cun, Y.L.; Denker, J.S.; Solla, S.A. Optimal brain damage. In Proceedings of the International Conference on Neural Information Processing Systems, Denver, CO, USA, 27–30 November 1989; pp. 598–605. [Google Scholar]
- Wen, W.; Wu, C.; Wang, Y.; Li, H. Learning Structured Sparsity in Deep Neural Networks. In Advances in Neural Information Processing Systems; NIPS: Grenada, Spain, 2016; pp. 2074–2082. [Google Scholar]
- Han, S.; Pool, J.; Tran, J.; Dally, W.J. Learning both Weights and Connections for Efficient Neural Networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS), Montreal, QC, Canada, 7–12 December 2015; pp. 1–9. [Google Scholar]
- Ullrich, K.; Welling, M. Soft weight-sharing for neural network compression. arXiv Preprint
**2017**, arXiv:1702.04008. [Google Scholar] - Park, E.; Ahn, J.; Yoo, S. Weighted-entropy-based quantization for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 7197–7205. [Google Scholar] [CrossRef]
- Tung, F.; Mori, G. Deep Neural Network Compression by In-Parallel Pruning-Quantization. IEEE Trans. Pattern Anal. Mach. Intell.
**2018**, 1–12. [Google Scholar] [CrossRef] [PubMed] - Hamlin, L.; Green, R.O.; Mouroulis, P.; Eastwood, M. Imaging Spectrometer Science Measurements for Terrestrial Ecology: AVIRIS and the Next Generation AVIRIS Characteristics and Development Status. IEEE Aerosp. Conf. Proc.
**2011**, 1–7. [Google Scholar] [CrossRef] - Curran, P.J.; Dungan, J.L. Estimation of signal-to-noise: a new procedure applied to AVIRIS data. IEEE Trans. Geosci. Remote. Sens.
**1989**, 27, 620–628. [Google Scholar] [CrossRef] - Gao, B.C.; Heidebrecht, K.B.; Goetz, A.F.H. Derivation of scaled surface reflectances from AVIRIS data. Remote. Sens. Environ.
**1993**, 44, 165–178. [Google Scholar] [CrossRef] - Rodger, A.; Lynch, J.M. Determining atmospheric column water vapour in the 0.4–2.5 μm spectral region. In Proceedings of the AVIRIS Workshop, Pasadena, CA, USA, 27 February–2 March 2001; Jet Propulsion Laboratory (JPL) Publication: Pasadena, CA, USA, 2001. [Google Scholar]
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016; pp. 499–522. [Google Scholar]
- Hinton, G.; Osindero, S.; Teh, Y. A fast learning algorithm for deep belief nets. Neural Comput.
**2006**, 7, 1527–1554. [Google Scholar] [CrossRef] [PubMed] - Soofbaf, S.R.; Sahebi, M.R.; Mojaradi, B. A Sliding Window-Based Joint Sparse Representation (SWJSR) Method for Hyperspectral Anomaly Detection. Remote Sens.
**2018**, 10, 434. [Google Scholar] [CrossRef] - Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett.
**2006**, 27, 861–874. [Google Scholar] [CrossRef] - Ramesh, S.; Kannan, S.; Baskar, S. Application of a fast and elitist multi-objective genetic algorithm to Reactive Power Dispatch. Serbian J. Electr. Eng.
**2009**, 6, 119–133. [Google Scholar] [CrossRef] - Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput.
**2002**, 6, 182–197. [Google Scholar] [CrossRef] [Green Version]

**Figure 1.**Lifetime of the main multispectral imagers sensors and the hyperspectral-image (HSI) sensor in past and plan [3].

**Figure 11.**The computation amount and the algorithm-hardware-cost-factor (AHCF) with different R. (

**a**) Computation amount of two different algorithms. (

**b**) A small algorithm implemented by the operations with less hardware resources. (

**c**) A big algorithm implemented by the operations with less hardware resources. (

**d**) A small algorithm implemented by the operations with huge hardware resources. (

**e**) A big algorithm implemented by the operations with huge hardware resources.

**Figure 12.**The analysis of throughput and the AHCF in device. (

**a**) The implementation of a big ${P}_{CR}$ algorithm in an abundant resources device. (

**b**) The implementation of a small ${P}_{CR}$ algorithm in an abundant resources device. (

**c**) The implementation of a big ${P}_{CR}$ algorithm in an insufficient resources device. (

**d**) The implementation of a small ${P}_{CR}$ algorithm in an insufficient resources device.

**Figure 14.**The flowchart of the proposed pruning–quantization–anomaly–detector (P-Q-AD) for onboard HSI AD.

**Figure 17.**The detection results for Louisiana data set. (

**a**) The anomaly score image of Louisiana by LRXD; (

**b**) The anomaly score image of Louisiana by collaborative representation based anomaly detector (CRD); (

**c**) The anomaly score image of Louisiana by floating point precision without pruning anomaly detectors (Floating-AD); (

**d**) The anomaly score image of Louisiana by floating point precision with pruning anomaly detectors (Floating-AD)P-Floating-AD; (

**e**) The anomaly score image of Louisiana by P-Fixed-AD; (

**f**) The anomaly score image of Louisiana by P-Q-AD.

**Figure 23.**The computational resources consumption by different detectors. To make those detectors reach its maximum parallel for high detection speed, the detectors are designed as high as possible until one of the resources consumption reaches the limitation of the field-programmable gate arrays (FPGA) chip for each detector.

Algorithm | C | ${\mathit{R}}_{\mathit{a}}$ | ${\mathit{P}}_{\mathbf{CR}}$ | Throughput | Resources Cost | Resources Utilization |
---|---|---|---|---|---|---|

A1 | 5000 | 16 | 80,000 | Floor (300,000/80,000) = 3 | 240,000 | 80% |

A2 | 9000 | 5 | 45,000 | Floor (300,000/45,000) = 6 | 270,000 | 90% |

Algorithm | C | ${\mathit{R}}_{\mathit{a}}$ | ${\mathit{P}}_{\mathbf{CR}}$ | Operation Cycles | Resources Cost | Resources Utilization |
---|---|---|---|---|---|---|

A1 | 5000 | 16 | 80,000 | Ceil (80,000/30,000) = 3 | - | - |

A2 | 9000 | 5 | 45,000 | Ceil (45,000/30,000) = 2 | - | - |

Gene Name | Leaky Value (k) | ${\mathit{n}}_{2}$ | ${\mathit{n}}_{\mathit{m}}$ | H | G | ${\mathit{b}}_{0}^{\mathit{I}}$ | ${\mathit{b}}_{0}^{\prime}$ | ${\mathit{b}}_{1}^{\mathit{I}}$ | ${\mathit{b}}_{1}^{\prime}$ | ${\mathit{b}}_{2}^{\mathit{I}}$ | ${\mathit{b}}_{2}^{\prime}$ |
---|---|---|---|---|---|---|---|---|---|---|---|

Maximum value | 10 | 100 | 80 | 30 | 29 | 5 | 16 | 6 | 10 | 7 | 9 |

Minimum value | 0 | 20 | 1 | 8 | 6 | 1 | 1 | 1 | 1 | 1 | 1 |

**Table 4.**The network structure and the operation precision for Local RXD(LRXD), collaborative representation based anomaly detector (CRD), floating point precision without pruning anomaly detectors (Floating-AD), P-Floating-AD, floating point precision with pruning anomaly detectors (P-Fixed-AD), and pruning–quantization–anomaly–detector (P-Q-AD).

Detector Name | Structure | Operation Precision | Data Bits |
---|---|---|---|

LRXD | - | Floating point | 32 |

CRD | - | Floating point | 32 |

Floating-AD | [166,80,20,80,166] ^{a} | Floating point | 32 |

P-Floating-AD | [166,41,14,41,166] ^{a} | Floating point | 32 |

P-Fixed-AD | [166,41,14,41,166] ^{a} | Fixed point | 32 |

${b}_{0}^{I}=4$, ${b}_{0}^{\prime}=12$ | |||

P-Q-AD | [166,41,14,41,166] ^{a} | Customized Fixed point | ${b}_{0}^{I}=4$, ${b}_{0}^{\prime}=8$ |

${b}_{0}^{I}=4$, ${b}_{0}^{\prime}=8$ |

^{a}The values are the neurons number of the first layer to the last layer, respectively.

**Table 5.**The area under the curve (AUC) value and the detection time consumption of Louisiana data set.

Detector Name | AUC Value | Detection Time (s) |
---|---|---|

LRXD | 0.9988 | 7734.00 |

CRD | 0.9977 | 2369.95 |

Floating-AD | 0.9976 | 284.43 |

P-Floating-AD | 0.9977 | 248.74 |

P-Fixed-AD | 0.9977 | 97.59 |

P-Q-AD | 0.9973 | 52.57 |

Detector Name | AUC Value | Detection Time (s) |
---|---|---|

LRXD | 0.7764 | 75.66 |

CRD | 0.9173 | 23.47 |

Floating-AD | 0.9489 | 3.31 |

P-Floating-AD | 0.9524 | 2.63 |

P-Fixed-AD | 0.9521 | 1.21 |

P-Q-AD | 0.9483 | 0.56 |

Detector Name | AUC Value | Detection Time (s) |
---|---|---|

LRXD | 0.9799 | 869.45 |

CRD | 0.9750 | 648.25 |

Floating-AD | 0.9894 | 40.69 |

P-Floating-AD | 0.9901 | 33.97 |

P-Fixed-AD | 0.9888 | 16.60 |

P-Q-AD | 0.9869 | 8.95 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ma, N.; Yu, X.; Peng, Y.; Wang, S.
A Lightweight Hyperspectral Image Anomaly Detector for Real-Time Mission. *Remote Sens.* **2019**, *11*, 1622.
https://doi.org/10.3390/rs11131622

**AMA Style**

Ma N, Yu X, Peng Y, Wang S.
A Lightweight Hyperspectral Image Anomaly Detector for Real-Time Mission. *Remote Sensing*. 2019; 11(13):1622.
https://doi.org/10.3390/rs11131622

**Chicago/Turabian Style**

Ma, Ning, Ximing Yu, Yu Peng, and Shaojun Wang.
2019. "A Lightweight Hyperspectral Image Anomaly Detector for Real-Time Mission" *Remote Sensing* 11, no. 13: 1622.
https://doi.org/10.3390/rs11131622