Analysis of Hyperspectral Data to Develop an Approach for Document Images
Abstract
:1. Introduction
2. Hyperspectral Images: Importance and Challenges
RGB Image Analysis and Hyperspectral Data
3. Spectral Unmixing in Hyperspectral Data Analysis
- Full Unmixing Problem (FUP): In the absence of any information regarding endmembers or abundances, the unmixing process implemented is known as FUP.
- Abundance Estimation Problem (AEP): Implementing end-member extraction and abundance estimation with prior information about the end members is known as AEP.
3.1. End-Member Extraction
3.1.1. Orthogonal Projection Methods
Pixel Purity Index (PPI)
Method | Nature Of Dataset | Sample | Studies |
---|---|---|---|
Orthogonal Projection Methods | |||
Pixel Purity Method (PPI) | AVIRIS Cuprite Dataset | Satellite Images | Guo et al. [25] |
AVIRIS Cuprite Dataset | Satellite Images | Gu et al. [26] | |
Real hyperspectral image scene collected by Hyperspectral Digital Imagery Collection Experiments (HYDICE) | Urban Images | Chang et al. [27] | |
AVIRIS Cuprite Dataset | Satellite Images | Sanchez et al. [28] | |
AVIRIS Cuprite Dataset | Satellite Images | Gonzalez et al. [29] | |
AVIRIS Cuprite Dataset | Satellite Images | Wu et al. [30] | |
AVIRIS Cuprite Dataset | Satellite Images | Valancia et al. [31] | |
Automatic Target Generation Process (ATGP) | HSI images captured via the HyMap airborne hyperspectral imaging sensor with coverage areas of 2.0 km2 in Cooke City town, MT, USA | Urban Images | Khoshboresh et al. [32] |
HYDICE 15-panel scene and HYDICE urban scene | Urban Images | Chang et al. [33] | |
HSI data collected by NASA’s AVIRIS instrument over the World Trade Center (WTC) in New York, five days after the terrorist attacks that collapsed the two main towers | Urban Images | Sierra-Pajuelo et al. [34] | |
HSI data collected by NASA’s AVIRIS instrument over the World Trade Center (WTC) in New York, five days after the terrorist attacks that collapsed the two main towers | Urban Images | Paz et al. [35] | |
HSI CD Dataset | Urban Images | Yadav et al. [36] | |
HYDICE Dataset | Urban Images | Chang et al. [37] | |
Custom dataset collected via the hyperion sensor over the mangrove habitats of Henry Island, West Bengal | Urban Images | Chakravortty et al. [38] | |
Convex Cone Analysis (CCA) | |||
Convex Cone Analysis (CCA) | WorldView-3 (WV3) Very High Resolution (VHR) satellite frame, HSI signatures of urban seismic rubble acquired through an ASD FieldSpec Pro hand-held radiometer, HSI signatures of uncolored typical cover and Pixel Digital Terrain Model (DTM) derived from LiDAR data about Amatrice | Urban Images | Pollino et al. [39] |
Landstat Dataset | Satellite Images | Milewski et al. [40] | |
Custom dataset of ASTER Level 1T (L1T) data acquired in August 2003 | Rock & Mineral Images | Esmaeili et al. [41] | |
Custom dataset collected through HSI microscope setup using Zeiss Axiovert 100 inverted microscope | Blood Images | Lee et al. [42] | |
MODIS NDVI Data. | Knight et al. [43] | ||
cloud-free satellite data which was derived from the United State Geological Survey (USGS) | Satellite Images | Singh et al. [44] | |
Custom dataset collected through HSI sensor on UAS platform by the Carinthia University of Applied Sciences (CUAS), Austria | Time Series & Urban Images | Milewski et al. [40] | |
Simplex Volume Analysis (SVA) | |||
NFINDR | Geostationary Ocean Color Imager (GOCI) dataset and HJ-1B dataset | Geostationary Ocean Satellite Images | Tao et al. [45] |
AVIRIS Cuprite Dataset | Satellite Images | Xiong et al. [46] | |
AVIRIS Cuprite Dataset | Satellite Images | Ji et al. [47] | |
Airborne Imaging Spectrometer for Application (AISA) and Compact Airborne Spectrographic Imager (CASI) dataset | Airborne Images | Song et al. [48] | |
AVIRIS Cuprite Dataset | Satellite Images | Quirita et al. [49] |
Automatic Target Generation Process (ATGP)
Vertex Component Analysis (VCA)
3.1.2. Convex Cone Analysis (CCA)
3.1.3. Simplex Volume Analysis (SVA)
NFINDR
3.2. Abundance Estimation and Abundance Mapping
3.2.1. Unconstrained Least Squared Methods (ULSs)
3.2.2. Non-Negative Least Squares (NNLS)
3.2.3. Unsupervised Fully Constrained Least Squared Method (UFCLS)
3.2.4. Image Space Reconstruction Algorithm (ISRA)
3.2.5. HSI Abundance Estimator Toolbox (HABET)
4. Deep Learning in HSI
- With or without manually constructed feature extraction methods, deep learning networks may extract linear and non-linear characteristics from raw data.
- Deep learning architectures can handle various forms of data; for example, in the case of HSI datasets, they can handle spectral and spatial data separately and simultaneously as well.
- Depending upon the nature of the problem and the type of available dataset, the choice for the architecture and implementation of the learning strategy varies.
- Due to their propensity to overfit if the training set only contains a few training samples, DNNs are inefficient at generalizing the distribution of HSI data. The DNN architecture being implemented is more prone to overfitting, necessitating changes during the training phase, limited generalization, and poor performance on the test set in the case of HSI datasets because of the high dimension and sparse training examples.
- Due to the curse of dimensionality, DNN architectures for HSI are computationally expensive and memory-intensive.
- Deeper networks with more parameters make training, optimization, and convergence more challenging and could result in several local minima.
- With the training process being a black box and the number of parameters for HSI, although various visualization processes can be implemented to visualize output at every layer, implementing optimization decisions and implementing more significant and interpretable filters is a tedious job.
4.1. HSI Data Handling
4.2. Deep Neural Network (DNN) Architecture
4.2.1. CNN Architectures for HSI Data
Architecture Details | Nature of Dataset | Studies |
---|---|---|
Convolutional Neural Network (CNN) | ||
Sixty-three images of the City of San Francisco from a custom dataset that was gathered using Google Earth | Satellite Images | Chen et al. [82] |
Sixty-three images of the City of San Francisco from a custom dataset that was gathered using Google Earth | Satellite Images | Chen et al. [83] |
Custom dataset of 25 hyperspectral images of the porcine eye cornea | Porcine eye cornea images | Noor et al. [84] |
Indian Pines & Salinas Valley Dataset | Satellite Images | Yang et al. [85] |
Houston & Trento Dataset | Satellite Images | Rasti et al. [86] |
Houston & Trento Dataset | Satellite Images | Li et al. [87] |
Houston Dataset | Satellite Images | Feng et al. [88] |
ICVL & CAVE Dataset | Street Scene Images | Chang et al. [90] |
Kennedy Space Center, Indian Pines, Pavia University, Salinas Scene datasets are used to evaluate the proposed DL architecture | Satellite & Urban Images | Luo et al. [91] |
Kennedy Space Center, Indian Pines, Pavia University, Salinas Scene datasets are used to evaluate the proposed DL architecture | Satellite & Urban Images | Chen et al. [92] |
Custom Diseased Leaves Dataset | Diseased Leaves Images | Liu et al. [93] |
Indian Pines, University of Pavia, WHU-Hi-HongHu dataset | Satellite Images | Dong et al. [94] |
Autoencoder-Decoder (AED) Architecture | ||
Kennedy Space Center & University of Pavia Datasets | Satellite & Urban Images | Lin et al. [95] |
Indian Pines, Pavia University, Salinas Scene datasets are used to evaluate the proposed DL architecture | Satellite & Urban Images | Shi et al. [96] |
Indian Pines & KSC datasets | Satellite & Urban Images | Zhao et al. [97] |
Indian Pines, Pavia University & Salinas Scene dataset | Satellite & Urban Images | Dou et al. [98] |
Pavia University, Indian Pines, Salinas Scenes dataset | Satellite & Urban images | Zhou et al. [75] |
Indian Pines, Salinas Scenes, Houston datasets | Satellite & Urban Images | Patel et al. [99] |
Generative Adversarial Networks (GANs) | ||
Indian Pines & Pavia University datasets | Satellite & Urban Images | Zhong et al. [100] |
Salinas Valley, Pavia University, KSC dataset | Satellite & Urban Images | Zhu et al. [101] |
Houston, Indian Pines, Xuzhou Dataset | Satellite Images | He et al. [102] |
Indian Pines, Houston2013, Houston2018 dataset | Satellite & Urban Images | Hang et al. [103] |
Recurrent Neural Networks (RNNs) | ||
Indian Pines, Pavia University, Salinas Scenes dataset | Satellite & Urban Images | Zhang et al. [104] |
Indian Pines & Pavia University dataset | Satellite Images | Hang et al. [105] |
Houston, Indian Pines & Pavia University dataset | Satellite & Urban Images | Mou et al. [106] |
Indian Pines, Pavia center scene & Pavia University dataset | Satellite & Urban Images | Shi et al. [107] |
Indian Pines, Big Indian Pines & Salinas Valley dataset | Satellite & Urban Images | Paoletti et al. [108] |
4.2.2. Autoencoder–Decoder Architectures for HSI Data
4.2.3. Generative Adversarial Neural Networks (GANs) for HSI Data
- Data dimensionality: Hyperspectral data typically have high-dimensional feature spaces, which can make training GANs more complex and computationally demanding. The increased dimensionality can lead to difficulties in capturing the intricate distributions and correlations present in hyperspectral data.
- Limited training data: GANs often require a large number of training data to effectively learn and generate high-quality samples. However, collecting and labeling large-scale hyperspectral datasets can be expensive and time-consuming, resulting in limited training data availability for GAN models.
- Mode collapse: Mode collapse refers to a situation where the generator network fails to capture the full diversity of the target hyperspectral data distribution and instead produces only a limited set of samples. This can result in generated hyperspectral images that lack variability and fail to represent the entire data distribution.
- Evaluation and validation: Assessing the quality and performance of GAN-generated hyperspectral data can be challenging. Metrics and evaluation methods specific to hyperspectral data need to be developed to ensure the generated samples are accurate representations of the original data and satisfy domain-specific requirements.
- Sensitivity to noise and artifacts: GANs can be sensitive to noise and artifacts present in hyperspectral data. This noise and artifacts can affect the training process and influence the quality of the generated samples, requiring additional preprocessing steps or regularization techniques to mitigate their impact.Addressing these challenges and developing robust GAN architectures tailored for hyperspectral data analysis can lead to improved generation and utilization of synthetic hyperspectral data for various applications.
4.2.4. Recurrent Neural Networks (RNN) for HSI Data
5. Evaluation Metrics for Deep Learning HSI Data Analysis
5.1. Structural Similarity Index Measurement (SSIM)
5.2. Peak Signal-to-Noise Ratio (PSNR)
5.3. Spectral Angle Mapping (SAM)
6. Discussion and Conclusions
- Data Volume and Complexity:Hyperspectral document images can contain hundreds or even thousands of spectral bands, leading to large data volumes and complexity. Processing and analyzing such large volumes of data can be computationally intensive and time-consuming.
- Preprocessing:Hyperspectral images require significant preprocessing to remove noise, normalize the data, and correct for any artifacts that may be present in the data.
- Spectral Variability:The spectral signature of a document can vary depending on factors such as ink type, paper type, and lighting conditions. This variability can make it difficult to develop robust algorithms for document analysis.
- Dimensionality Reduction:Given the huge number of spectral bands, dimensionality-reduction techniques are frequently required to simplify the calculation and boost the analysis’s precision.
- Spectral Mixing:When analyzing hyperspectral images, it is possible to encounter spectral mixing, where multiple spectral signatures are present in a single pixel or region of interest. This can make it challenging to accurately identify and classify different features in the image.
- Limited Availability of Data:The availability of hyperspectral document image datasets is limited, making it challenging to develop and test new algorithms and techniques.
- Interpretability:The many spectral bands included in hyperspectral photographs might make it challenging to understand the study’s findings, especially for non-experts.
Author Contributions
Funding
Institutional Review Board Statement
Conflicts of Interest
References
- Qureshi, R.; Uzair, M.; Khurshid, K.; Yan, H. Hyperspectral document image processing: Applications, challenges and future prospects. Pattern Recognit. 2019, 90, 12–22. [Google Scholar] [CrossRef]
- Amigo, J.M.; Babamoradi, H.; Elcoroaristizabal, S. Hyperspectral image analysis. A tutorial. Anal. Chim. Acta 2015, 896, 34–51. [Google Scholar] [CrossRef] [PubMed]
- Yan, L.; Zhao, M.; Wang, X.; Zhang, Y.; Chen, J. Object detection in hyperspectral images. IEEE Signal Process. Lett. 2021, 28, 508–512. [Google Scholar] [CrossRef]
- Mohan, B.K.; Porwal, A. Hyperspectral image processing and analysis. Curr. Sci. 2015, 108, 833–841. [Google Scholar]
- Sarris, A.; Papadopoulos, N.; Agapiou, A.; Salvi, M.C.; Hadjimitsis, D.G.; Parkinson, W.A.; Yerkes, R.W.; Gyucha, A.; Duffy, P.R. Integration of geophysical surveys, ground hyperspectral measurements, aerial and satellite imagery for archaeological prospection of prehistoric sites: The case study of Vésztő-Mágor Tell, Hungary. J. Archaeol. Sci. 2013, 40, 1454–1470. [Google Scholar] [CrossRef]
- Pereira, J.F.Q.; Silva, C.S.; Braz, A.; Pimentel, M.F.; Honorato, R.S.; Pasquini, C.; Wentzell, P.D. Projection pursuit and PCA associated with near and middle infrared hyperspectral images to investigate forensic cases of fraudulent documents. Microchem. J. 2017, 130, 412–419. [Google Scholar] [CrossRef]
- Edelman, G.J.; Gaston, E.; Van Leeuwen, T.G.; Cullen, P.; Aalders, M.C. Hyperspectral imaging for non-contact analysis of forensic traces. Forensic Sci. Int. 2012, 223, 28–39. [Google Scholar] [CrossRef] [Green Version]
- El-Hadidy, S.M.; Alshehri, F.; Sahour, H.; Abdelmalik, K.W. Detecting hydrocarbon micro-seepage and related contamination, probable prospect areas, deduced from a comparative analysis of multispectral and hyperspectral satellite images. J. King Saud Univ. Sci. 2022, 34, 102192. [Google Scholar] [CrossRef]
- Butt, U.M.; Ahmad, S.; Shafait, F.; Nansen, C.; Mian, A.S.; Malik, M.I. Automatic signature segmentation using hyper-spectral imaging. In Proceedings of the 2016 15th International Conference on Frontiers in Handwriting Recognition (ICFHR), Shenzhen, China, 23–26 October 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 19–24. [Google Scholar]
- Kim, S.J.; Deng, F.; Brown, M.S. Visual enhancement of old documents with hyperspectral imaging. Pattern Recognit. 2011, 44, 1461–1469. [Google Scholar] [CrossRef]
- Khan, M.J.; Yousaf, A.; Abbas, A.; Khurshid, K. Deep learning for automated forgery detection in hyperspectral document images. J. Electron. Imaging 2018, 27, 053001. [Google Scholar] [CrossRef]
- Lu, B.; Dao, P.D.; Liu, J.; He, Y.; Shang, J. Recent advances of hyperspectral imaging technology and applications in agriculture. Remote Sens. 2020, 12, 2659. [Google Scholar] [CrossRef]
- Wang, C.; Liu, B.; Liu, L.; Zhu, Y.; Hou, J.; Liu, P.; Li, X. A review of deep learning used in the hyperspectral image analysis for agriculture. Artif. Intell. Rev. 2021, 54, 5205–5253. [Google Scholar] [CrossRef]
- Petersson, H.; Gustafsson, D.; Bergstrom, D. Hyperspectral image analysis using deep learning—A review. In Proceedings of the 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), Oulu, Finland, 12–15 December 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 1–6. [Google Scholar]
- Yan, L.; Yamaguchi, M.; Noro, N.; Takara, Y.; Ando, F. A novel two-stage deep learning-based small-object detection using hyperspectral images. Opt. Rev. 2019, 26, 597–606. [Google Scholar] [CrossRef]
- Rosario-Torres, S.; Vélez-Reyes, M. An algorithm for fully constrained abundance estimation in hyperspectral unmixing. In Proceedings of the Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XI. SPIE, Orlando, FL, USA, 1 June 2005; Volume 5806, pp. 711–719. [Google Scholar]
- Torres, S.R. Iterative Algorithms for Abundance Estimation on Unmixing of Hyperspectral Imagery; University of Puerto Rico: Mayaguez, Puerto Rico, 2004. [Google Scholar]
- Veganzones, M.A.; Grana, M. Endmember extraction methods: A short review. In Proceedings of the International Conference on Knowledge-Based and Intelligent Information and Engineering Systems, Zagreb, Croatia, 3–5 September 2008; Springer: Berlin/Heidelberg, Germany, 2008; pp. 400–407. [Google Scholar]
- Dobigeon, N.; Altmann, Y.; Brun, N.; Moussaoui, S. Linear and nonlinear unmixing in hyperspectral imaging. In Data Handling in Science and Technology; Elsevier: Amsterdam, The Netherlands, 2016; Volume 30, pp. 185–224. [Google Scholar]
- Clevers, J.; Zurita-Milla, R. Multisensor and multiresolution image fusion using the linear mixing model. In Image Fusion: Algorithms and Applications; Elsevier: Amsterdam, The Netherlands, 2008; pp. 67–84. [Google Scholar]
- Keshava, N.; Mustard, J.F. Spectral unmixing. IEEE Signal Process. Mag. 2002, 19, 44–57. [Google Scholar] [CrossRef]
- Vélez-Reyes, M.; Rosario, S. Solving adundance estimation in hyperspectral unmixing as a least distance problem. In Proceedings of the IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium, Anchorage, AK, USA, 20–24 September 2004; IEEE: Piscataway, NJ, USA, 2004; Volume 5, pp. 3276–3278. [Google Scholar]
- Chang, C.I. Finding endmembers in hyperspectral imagery. In Real-Time Progressive Hyperspectral Image Processing; Springer: Berlin/Heidelberg, Germany, 2016; pp. 75–103. [Google Scholar]
- Tao, X.; Paoletti, M.E.; Han, L.; Haut, J.M.; Ren, P.; Plaza, J.; Plaza, A. Fast Orthogonal Projection for Hyperspectral Unmixing. IEEE Trans. Geosci. Remote Sens. 2022, 60, 1–13. [Google Scholar] [CrossRef]
- Guo, J.; Li, Y.; Liu, K.; Lei, J.; Wang, K. Fast FPGA implementation for computing the pixel purity index of hyperspectral images. J. Circuits, Syst. Comput. 2018, 27, 1850045. [Google Scholar] [CrossRef]
- Gu, J.; Wu, Z.; Li, Y.; Chen, Y.; Wei, Z.; Wang, W. Parallel optimization of pixel purity index algorithm for hyperspectral unmixing based on spark. In Proceedings of the 2015 Third International Conference on Advanced Cloud and Big Data, Yangzhou, China, 30 October–1 November 2015; IEEE: Piscataway, NJ, USA, 2015; pp. 159–166. [Google Scholar]
- Chang, C.I.; Li, Y.; Wang, Y. Progressive band processing of fast iterative pixel purity index for finding endmembers. IEEE Geosci. Remote Sens. Lett. 2017, 14, 1464–1468. [Google Scholar] [CrossRef]
- Sánchez, S.; Plaza, A. GPU implementation of the pixel purity index algorithm for hyperspectral image analysis. In Proceedings of the 2010 IEEE International Conference On Cluster Computing Workshops and Posters (CLUSTER WORKSHOPS), Heraklion, Greece, 20–24 September 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 1–7. [Google Scholar]
- González, C.; Resano, J.; Mozos, D.; Plaza, A.; Valencia, D. FPGA implementation of the pixel purity index algorithm for remotely sensed hyperspectral image analysis. Eurasip J. Adv. Signal Process. 2010, 2010, 969806. [Google Scholar] [CrossRef] [Green Version]
- Wu, X.; Huang, B.; Plaza, A.; Li, Y.; Wu, C. Real-time implementation of the pixel purity index algorithm for endmember identification on GPUs. IEEE Geosci. Remote Sens. Lett. 2013, 11, 955–959. [Google Scholar] [CrossRef]
- Valencia, D.; Plaza, A. FPGA-based hyperspectral data compression using spectral unmixing and the pixel purity index algorithm. In Proceedings of the Computational Science–ICCS 2006: 6th International Conference, Reading, UK, 28–31 May 2006; Proceedings, Part I 6. Springer: Berlin/Heidelberg, Germany, 2006; pp. 888–891. [Google Scholar]
- Khoshboresh-Masouleh, M.; Hasanlou, M. Improving hyperspectral sub-pixel target detection in multiple target signatures using a revised replacement signal model. Eur. J. Remote Sens. 2020, 53, 316–330. [Google Scholar] [CrossRef]
- Chang, C.I.; Cao, H.; Song, M. Orthogonal subspace projection target detector for hyperspectral anomaly detection. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2021, 14, 4915–4932. [Google Scholar] [CrossRef]
- Sierra-Pajuelo, F.; Paz-Gallardo, A.; Plaza, A. Perfomance optimizations for an automatic target generation process in hyperspectral analysis. In Proceedings of the ARCS 2015-The 28th International Conference on Architecture of Computing Systems, Porto, Portugal, 24–27 March 2015; Proceedings. VDE: Frankfurt am Main, Germany, 2015; pp. 1–6. [Google Scholar]
- Paz, A.; Plaza, A.; Blázquez, S. Parallel implementation of target and anomaly detection algorithms for hyperspectral imagery. In Proceedings of the IGARSS 2008-2008 IEEE International Geoscience and Remote Sensing Symposium, Boston, MA, USA, 7–11 July 2008; IEEE: Piscataway, NJ, USA, 2008; Volume 2, pp. 589–592. [Google Scholar]
- Yadav, P.P.; Bobate, N.; Shetty, A.; Raghavendra, B.; Narasimhadhan, A. ATGP based Change Detection in Hyperspectral Images. In Proceedings of the IECON 2022–48th Annual Conference of the IEEE Industrial Electronics Society, Brussels, Belgium, 17–20 October 2022; IEEE: Piscataway, NJ, USA, 2022; pp. 1–6. [Google Scholar]
- Chang, C.I.; Li, Y. Recursive band processing of automatic target generation process for finding unsupervised targets in hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 2016, 54, 5081–5094. [Google Scholar] [CrossRef]
- Chakravortty, S. Analysis of end member detection and subpixel classification algorithms on hyperspectral imagery for tropical mangrove species discrimination in the Sunderbans Delta, India. J. Appl. Remote Sens. 2013, 7, 073523. [Google Scholar] [CrossRef]
- Pollino, M.; Cappucci, S.; Giordano, L.; Iantosca, D.; De Cecco, L.; Bersan, D.; Rosato, V.; Borfecchia, F. Assessing earthquake-induced urban rubble by means of multiplatform remotely sensed data. Isprs Int. J. Geo Inf. 2020, 9, 262. [Google Scholar] [CrossRef] [Green Version]
- Milewski, R.; Chabrillat, S.; Bookhagen, B. Analyses of Namibian seasonal salt pan crust dynamics and climatic drivers using Landsat 8 time-series and ground data. Remote Sens. 2020, 12, 474. [Google Scholar] [CrossRef] [Green Version]
- Esmaeili, S.; Tangestani, M.H.; Tayebi, M.H. Sub-pixel mapping of copper-and iron-bearing metamorphic rocks using ASTER data: A case study of Toutak and Surian complexes, NE Shiraz, Iran. Nat. Resour. Res. 2020, 29, 2933–2948. [Google Scholar] [CrossRef]
- Lee, J.Y.; Clarke, M.L.; Tokumasu, F.; Lesoine, J.F.; Allen, D.W.; Chang, R.; Litorja, M.; Hwang, J. Absorption-based hyperspectral imaging and analysis of single erythrocytes. IEEE J. Sel. Top. Quantum Electron. 2011, 18, 1130–1139. [Google Scholar] [CrossRef]
- Knight, J.; Voth, M. Mapping impervious cover using multi-temporal MODIS NDVI data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2010, 4, 303–309. [Google Scholar] [CrossRef]
- Singh, R.; Kumar, V. Evaluating automated endmember extraction for classifying hyperspectral data and deriving spectral parameters for monitoring forest vegetation health. Environ. Monit. Assess. 2023, 195, 72. [Google Scholar] [CrossRef]
- Tao, X.; Cui, T.; Ren, P. Cofactor-based efficient endmember extraction for green algae area estimation. IEEE Geosci. Remote Sens. Lett. 2019, 16, 849–853. [Google Scholar] [CrossRef]
- Xiong, W.; Chang, C.I.; Wu, C.C.; Kalpakis, K.; Chen, H.M. Fast algorithms to implement N-FINDR for hyperspectral endmember extraction. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2011, 4, 545–564. [Google Scholar] [CrossRef]
- Ji, L.; Geng, X.; Sun, K.; Zhao, Y.; Gong, P. Modified N-FINDR endmember extraction algorithm for remote-sensing imagery. Int. J. Remote Sens. 2015, 36, 2148–2162. [Google Scholar] [CrossRef]
- Song, A.; Chang, A.; Choi, J.; Choi, S.; Kim, Y. Automatic extraction of optimal endmembers from airborne hyperspectral imagery using iterative error analysis (IEA) and spectral discrimination measurements. Sensors 2015, 15, 2593–2613. [Google Scholar] [CrossRef] [Green Version]
- Ayma Quirita, V.A.; da Costa, G.A.O.P.; Beltrán, C. A Distributed N-FINDR Cloud Computing-Based Solution for Endmembers Extraction on Large-Scale Hyperspectral Remote Sensing Data. Remote Sens. 2022, 14, 2153. [Google Scholar] [CrossRef]
- Boardman, J.W. Geometric mixture analysis of imaging spectrometry data. In Proceedings of the Proceedings of IGARSS’94-1994 IEEE International Geoscience and Remote Sensing Symposium, Pasadena, CA, USA, 8–12 August 1994; IEEE: Piscataway, NJ, USA, 1994; Vol. 4, pp. 2369–2371. [Google Scholar]
- Plaza, A.; Chang, C.I. Fast implementation of pixel purity index algorithm. In Proceedings of the Algorithms and Technologies for Multispectral, Hyperspectral, and Ultraspectral Imagery XI. SPIE, Orlando, FL, USA, 1 June 2005; Volume 5806, pp. 307–317. [Google Scholar]
- Ren, H.; Chang, C.I. Automatic spectral target recognition in hyperspectral imagery. IEEE Trans. Aerosp. Electron. Syst. 2003, 39, 1232–1249. [Google Scholar]
- Nascimento, J.M.; Dias, J.M. Vertex component analysis: A fast algorithm to unmix hyperspectral data. IEEE Trans. Geosci. Remote Sens. 2005, 43, 898–910. [Google Scholar] [CrossRef] [Green Version]
- Winter, M.E. N-FINDR: An algorithm for fast autonomous spectral end-member determination in hyperspectral data. In Proceedings of the Imaging Spectrometry V. SPIE, Denver, CO, USA, 27 October 1999; Volume 3753, pp. 266–275. [Google Scholar]
- Heinz, D.; Chang, C.I.; Althouse, M.L. Fully constrained least-squares based linear unmixing [hyperspectral image classification]. In Proceedings of the IEEE 1999 International Geoscience and Remote Sensing Symposium. IGARSS’99 (Cat. No. 99CH36293), Hamburg, Germany, 28 June–2 July 1999; IEEE: Piscataway, NJ, USA, 1999; Volume 2, pp. 1401–1403. [Google Scholar]
- Björck, Å. Least squares methods. Handb. Numer. Anal. 1990, 1, 465–652. [Google Scholar]
- Heinz, D.C. Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery. IEEE Trans. Geosci. Remote Sens. 2001, 39, 529–545. [Google Scholar] [CrossRef] [Green Version]
- Iqbal, K.; Khurshid, K. Automatic signature extraction from document images using hyperspectral unmixing: Automatic signature extraction using hyperspectral unmixing. Proc. Pak. Acad. Sci. Phys. Comput. Sci. 2017, 54, 269–276. [Google Scholar]
- Nascimento, S.M.; Ferreira, F.P.; Foster, D.H. Statistics of spatial cone-excitation ratios in natural scenes. JOSA A 2002, 19, 1484–1490. [Google Scholar] [CrossRef] [Green Version]
- Foster, D.H.; Nascimento, S.M.; Amano, K. Information limits on neural identification of colored surfaces in natural scenes. Vis. Neurosci. 2004, 21, 331–336. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chakrabarti, A.; Zickler, T. Statistics of real-world hyperspectral images. In Proceedings of the CVPR 2011, Colorado Springs, CO, USA, 20–25 June 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 193–200. [Google Scholar]
- Liang, J.; Zhou, J.; Bai, X.; Qian, Y. Salient object detection in hyperspectral imagery. In Proceedings of the 2013 IEEE International Conference on Image Processing, Melbourne, VIC, Australia, 15–18 September 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 2393–2397. [Google Scholar]
- Ke, C. Military object detection using multiple information extracted from hyperspectral imagery. In Proceedings of the 2017 International Conference on Progress in Informatics and Computing (PIC), Nanjing, China, 15–17 December 2017; IEEE: Piscataway, NJ, USA, 2017; pp. 124–128. [Google Scholar]
- Lee, H.; Kwon, H. Going deeper with contextual CNN for hyperspectral image classification. IEEE Trans. Image Process. 2017, 26, 4843–4855. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Li, S.; Song, W.; Fang, L.; Chen, Y.; Ghamisi, P.; Benediktsson, J.A. Deep learning for hyperspectral image classification: An overview. IEEE Trans. Geosci. Remote Sens. 2019, 57, 6690–6709. [Google Scholar] [CrossRef] [Green Version]
- Zhu, G.; Zheng, Y.; Doermann, D.; Jaeger, S. Multi-scale structural saliency for signature detection. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; IEEE: Piscataway, NJ, USA, 2007; pp. 1–8. [Google Scholar]
- Paoletti, M.; Haut, J.; Plaza, J.; Plaza, A. Deep learning classifiers for hyperspectral imaging: A review. ISPRS J. Photogramm. Remote Sens. 2019, 158, 279–317. [Google Scholar] [CrossRef]
- Doetsch, P.; Golik, P.; Ney, H. A comprehensive study of batch construction strategies for recurrent neural networks in mxnet. arXiv 2017, arXiv:1705.02414. [Google Scholar]
- Dowd, K.; Severance, C. High Performance Computing; OpenStax CNX: Houston, TX, USA, 2010. [Google Scholar]
- Quinn, M.J. Parallel Computing Theory and Practice; McGraw-Hill, Inc.: New York, NY, USA, 1994. [Google Scholar]
- Asanovic, K.; Bodik, R.; Demmel, J.; Keaveny, T.; Keutzer, K.; Kubiatowicz, J.; Morgan, N.; Patterson, D.; Sen, K.; Wawrzynek, J.; et al. A view of the parallel computing landscape. Commun. ACM 2009, 52, 56–67. [Google Scholar] [CrossRef] [Green Version]
- Signoroni, A.; Savardi, M.; Baronio, A.; Benini, S. Deep learning meets hyperspectral image analysis: A multidisciplinary review. J. Imaging 2019, 5, 52. [Google Scholar] [CrossRef] [Green Version]
- Abdi, H.; Williams, L.J. Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2010, 2, 433–459. [Google Scholar] [CrossRef]
- Stone, J.V. Independent component analysis: An introduction. Trends Cogn. Sci. 2002, 6, 59–64. [Google Scholar] [CrossRef]
- Zhou, P.; Han, J.; Cheng, G.; Zhang, B. Learning compact and discriminative stacked autoencoder for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2019, 57, 4823–4833. [Google Scholar] [CrossRef]
- Worrall, D.; Welling, M. Deep scale-spaces: Equivariance over scale. Adv. Neural Inf. Process. Syst. 2019, 32. [Google Scholar]
- Nusrat, I.; Jang, S.B. A comparison of regularization techniques in deep neural networks. Symmetry 2018, 10, 648. [Google Scholar] [CrossRef] [Green Version]
- Ajit, A.; Acharya, K.; Samanta, A. A review of convolutional neural networks. In Proceedings of the 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), Vellore, India, 24–25 February 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 1–5. [Google Scholar]
- Tschannen, M.; Bachem, O.; Lucic, M. Recent advances in autoencoder-based representation learning. arXiv 2018, arXiv:1812.05069. [Google Scholar]
- Creswell, A.; White, T.; Dumoulin, V.; Arulkumaran, K.; Sengupta, B.; Bharath, A.A. Generative adversarial networks: An overview. IEEE Signal Process. Mag. 2018, 35, 53–65. [Google Scholar] [CrossRef] [Green Version]
- Medsker, L.R.; Jain, L. Recurrent neural networks. Des. Appl. 2001, 5, 64–67. [Google Scholar]
- Chen, X.; Xiang, S.; Liu, C.L.; Pan, C.H. Vehicle detection in satellite images by parallel deep convolutional neural networks. In Proceedings of the 2013 2nd IAPR Asian Conference on Pattern Recognition, Naha, Japan, 5–8 November 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 181–185. [Google Scholar]
- Chen, X.; Xiang, S.; Liu, C.L.; Pan, C.H. Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett. 2014, 11, 1797–1801. [Google Scholar] [CrossRef]
- Md Noor, S.S.; Michael, K.; Marshall, S.; Ren, J. Hyperspectral image enhancement and mixture deep-learning classification of corneal epithelium injuries. Sensors 2017, 17, 2644. [Google Scholar] [CrossRef] [Green Version]
- Yang, J.; Zhao, Y.; Chan, J.C.W.; Yi, C. Hyperspectral image classification using two-channel deep convolutional neural network. In Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), Beijing, China, 10–15 July 2016; IEEE: Piscataway, NJ, USA, 2016; pp. 5079–5082. [Google Scholar]
- Rasti, B.; Ghamisi, P.; Gloaguen, R. Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis. IEEE Trans. Geosci. Remote Sens. 2017, 55, 3997–4007. [Google Scholar] [CrossRef]
- Li, H.; Ghamisi, P.; Soergel, U.; Zhu, X.X. Hyperspectral and LiDAR fusion using deep three-stream convolutional neural networks. Remote Sens. 2018, 10, 1649. [Google Scholar] [CrossRef] [Green Version]
- Feng, Q.; Zhu, D.; Yang, J.; Li, B. Multisource hyperspectral and lidar data fusion for urban land-use mapping based on a modified two-branch convolutional neural network. ISPRS Int. J. Geo Inf. 2019, 8, 28. [Google Scholar] [CrossRef] [Green Version]
- Jiang, B.; Karimi, H.R.; Kao, Y.; Gao, C. Adaptive Control of Nonlinear Semi-Markovian Jump TS Fuzzy Systems with Immeasurable Premise Variables via Sliding Mode Observer. IEEE Trans. Cybern. 2018, 50, 810–820. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Chang, Y.; Yan, L.; Fang, H.; Zhong, S.; Liao, W. HSI-DeNet: Hyperspectral image restoration via convolutional neural network. IEEE Trans. Geosci. Remote Sens. 2018, 57, 667–682. [Google Scholar] [CrossRef]
- Luo, Y.; Zou, J.; Yao, C.; Zhao, X.; Li, T.; Bai, G. HSI-CNN: A novel convolution neural network for hyperspectral image. In Proceedings of the 2018 International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, China, 16–17 July 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 464–469. [Google Scholar]
- Chen, Y.; Zhu, K.; Zhu, L.; He, X.; Ghamisi, P.; Benediktsson, J.A. Automatic design of convolutional neural network for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2019, 57, 7048–7066. [Google Scholar] [CrossRef]
- Liu, W.; Lee, J. A 3-D atrous convolution neural network for hyperspectral image denoising. IEEE Trans. Geosci. Remote Sens. 2019, 57, 5701–5715. [Google Scholar] [CrossRef]
- Dong, Y.; Liu, Q.; Du, B.; Zhang, L. Weighted feature fusion of convolutional neural network and graph attention network for hyperspectral image classification. IEEE Trans. Image Process. 2022, 31, 1559–1572. [Google Scholar] [CrossRef]
- Lin, Z.; Chen, Y.; Zhao, X.; Wang, G. Spectral-spatial classification of hyperspectral image using autoencoders. In Proceedings of the 2013 9th international conference on information, Communications & Signal Processing, Tainan, Taiwan, 10–13 December 2013; IEEE: Piscataway, NJ, USA, 2013; pp. 1–5. [Google Scholar]
- Shi, C.; Pun, C.M. Multiscale superpixel-based hyperspectral image classification using recurrent neural networks with stacked autoencoders. IEEE Trans. Multimed. 2019, 22, 487–501. [Google Scholar] [CrossRef]
- Zhao, C.; Wan, X.; Zhao, G.; Cui, B.; Liu, W.; Qi, B. Spectral-spatial classification of hyperspectral imagery based on stacked sparse autoencoder and random forest. Eur. J. Remote Sens. 2017, 50, 47–63. [Google Scholar] [CrossRef] [Green Version]
- Dou, Z.; Gao, K.; Zhang, X.; Wang, H.; Han, L. Band selection of hyperspectral images using attention-based autoencoders. IEEE Geosci. Remote Sens. Lett. 2020, 18, 147–151. [Google Scholar] [CrossRef]
- Patel, H.; Upla, K.P. A shallow network for hyperspectral image classification using an autoencoder with convolutional neural network. Multimed. Tools Appl. 2022, 81, 695–714. [Google Scholar] [CrossRef]
- Zhong, Z.; Li, J.; Clausi, D.A.; Wong, A. Generative adversarial networks and conditional random fields for hyperspectral image classification. IEEE Trans. Cybern. 2019, 50, 3318–3329. [Google Scholar] [CrossRef] [Green Version]
- Zhu, L.; Chen, Y.; Ghamisi, P.; Benediktsson, J.A. Generative adversarial networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2018, 56, 5046–5063. [Google Scholar] [CrossRef]
- He, Z.; Xia, K.; Ghamisi, P.; Hu, Y.; Fan, S.; Zu, B. HyperViTGAN: Semisupervised Generative Adversarial Network With Transformer for Hyperspectral Image Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2022, 15, 6053–6068. [Google Scholar] [CrossRef]
- Hang, R.; Zhou, F.; Liu, Q.; Ghamisi, P. Classification of hyperspectral images via multitask generative adversarial networks. IEEE Trans. Geosci. Remote Sens. 2020, 59, 1424–1436. [Google Scholar] [CrossRef]
- Zhang, X.; Sun, Y.; Jiang, K.; Li, C.; Jiao, L.; Zhou, H. Spatial sequential recurrent neural network for hyperspectral image classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 4141–4155. [Google Scholar] [CrossRef] [Green Version]
- Hang, R.; Liu, Q.; Hong, D.; Ghamisi, P. Cascaded recurrent neural networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2019, 57, 5384–5394. [Google Scholar] [CrossRef] [Green Version]
- Mou, L.; Ghamisi, P.; Zhu, X.X. Deep recurrent neural networks for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 2017, 55, 3639–3655. [Google Scholar] [CrossRef] [Green Version]
- Shi, C.; Pun, C.M. Multi-scale hierarchical recurrent neural networks for hyperspectral image classification. Neurocomputing 2018, 294, 82–93. [Google Scholar] [CrossRef]
- Paoletti, M.E.; Haut, J.M.; Plaza, J.; Plaza, A. Scalable recurrent neural network for hyperspectral image classification. J. Supercomput. 2020, 76, 8866–8882. [Google Scholar] [CrossRef]
- Hassan, M.; Bhagvati, C. Structural similarity measure for color images. Int. J. Comput. Appl. 2012, 43, 7–12. [Google Scholar] [CrossRef]
- Horé, A.; Ziou, D. Is there a relationship between peak-signal-to-noise ratio and structural similarity index measure? IET Image Process. 2013, 7, 12–24. [Google Scholar] [CrossRef]
- Shippert, P. Introduction to hyperspectral image analysis. Online J. Space Commun. 2003, 2, 8. [Google Scholar]
- Girouard, G.; Bannari, A.; El Harti, A.; Desrochers, A. Validated spectral angle mapper algorithm for geological mapping: Comparative study between QuickBird and Landsat-TM. In Proceedings of the XXth ISPRS Congress, Geo-Imagery Bridging Continents, Istanbul, Turkey, 12–23 July 2004; pp. 12–23. [Google Scholar]
- Plaza, J.; Hendrix, E.M.; García, I.; Martín, G.; Plaza, A. On endmember identification in hyperspectral images without pure pixels: A comparison of algorithms. J. Math. Imaging Vis. 2012, 42, 163–175. [Google Scholar] [CrossRef]
Method | Weblink | Access Date | Reference |
---|---|---|---|
Available Implementations of End-Member Extraction Algorithms | |||
Pixel Purity Index (PPI) | PPI Python Implementation WebLink | 13 March 2023 | Joseph W. Boardman [50] |
Fast Iterative Pixel Purity Index (FIPPI) | FIPPI Python Implementation WebLink | 13 March 2023 | Plaza et al. [51] |
Automatic Target Generation Process (ATGP) | ATGP Python Implementation WebLink | 13 March 2023 | Ren et al. [52] |
Vertex Component Analysis (VCA) | VCA Python Implementation WebLink | 14 March 2023 | Nascimento et al. [53] |
N-FINDR | NFINDR Python Implementation WebLink | 14 March 2023 | Winter et al. [54] |
Available Implementations of Abundance Estimation Methods Algorithms | |||
Unconstrained Least Squared Methods (ULS) | ULS Python Implementation WebLink | 15 March 2023 | Torres et al. [16] |
Non-Negative Least Squares (NNLS) | NNLS Python Implementation WebLink | 15 March 2023 | Torres et al. [16] |
Unsupervised Non-Negativity Constrained Least Squared Methods (UNCLS) | UNCLS Python Implementation WebLink | 16 March 2023 | Torres et al. [16] |
Fully Constrained Least Squared Method (UFCLS) | FCLS Python Implementation WebLink | 16 March 2023 | Heinz et al. [55] |
Image Space Reconstruction Algorithm (ISRA) | Python implementation is not available | Not Available | Samuel Rosario Torres [17] |
HSI Abundance Estimator Toolbox (HABET) | Python implementation is not available | Not Available | Samuel Rosario Torres [17] |
Methods and Nature of Dataset | Authors |
---|---|
Historical document enhancement on historical documents from the National Archief of the Netherlands NAN). | Kim et al. [10] |
Acquisition, pre-processing, and implementation review of the HSI data for signature segmentation, forgery detection, ink mismatch analysis, historical document analysis, and study of cultural artifacts | Qureshi et al. [1] |
Custom dataset of 300 hyperspectral document images captured at 2.1 nm resolution through a hyperspectral camera. | Butt et al. [9] |
A subset of 100 hyperspectral images from the dataset proposed by Malik et al. [9] is utilized for signature extraction. | Iqbal et al. [58] |
Review hyperspectral image data from Hyperion, CASI, and Headwall Micro-Hyperspec as well as multispectral images from Landsat, Sentinel 2, and SPOT for agricultural research | Lu et al. [12] |
Review of deep learning techniques for agricultural studies on the hyperspectral images from the Indian Pines, Salinas, and University of Pavia datasets. | Wang et al. [13] |
Custom HSI-MIR and HSI-NIR pictures with near- and middle-infrared hyperspectral images are connected with projection pursuit and PCA for the investigation of counterfeit documents in forensic situations | Pereira et al. [6] |
Review of principles, instrumentation, and analytical techniques for HSI analysis and processing for forensic science applications. | Edelman et al. [7] |
A state-of-the-art deep learning network for ink mismatch detection is proposed and tested on the UWA Writing Ink Hyperspectral Images (WIHSI) database for forgery detection. | Khan et al. [11] |
Method for soil mineralogical changes detection due to petroleum seepage through multispectral images from Landstat7 and Advanced Land Imager (Ali) and hyperspectral images from EO-1 and Hyperion. | El-Hadidy et al. [8] |
A bespoke collection of 1200 ground hyperspectral pictures acquired with the GER 1500 spectroradiometer allows for comparison with geographic surveys, ground hyperspectral data, aerial photography, and high-resolution satellite imaging for archaeological research | Sarris et al. [5] |
Hyperspectral object detection and classification network with custom HSI dataset of 400 hyperspectral images for object-level target detection. | Yan et al. [3] |
Visual saliency model for salient feature extraction and salient object detection on ground-based HSI datasets collected by Foster et al. [59,60] and Harvard University [61]. | Liang et al. [62] |
Military object detection using PCA, k-means clustering, and self-similarity was tested on San Diego HSI dataset. | Chen ke. [63] |
Comparison of existing performance accuracies of deep learning approaches on Indian Pines, Salinas, Pavia Center, and Kennedy Space Center (KSC) datasets. | Petersson et al. [14] |
HSI Classification through contextual CNNs with performance tested on the Indian Pines dataset, Salinas dataset, and Pavia University dataset and compared with bench-marked models | Lee et al. [64] |
A systematic review of deep learning-based HSI classification methods available in the literature and a comparison of available strategies. | Li et al. [65] |
A bespoke dataset of 60 pictures with 121 channels each collected by an NH series hyperspectral camera, serving as the testing and training ground for a two-stage deep learning hyperspectral neural network for person identification on sea surface. | Lu Yan et al. [15] |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zaman, Z.; Ahmed, S.B.; Malik, M.I. Analysis of Hyperspectral Data to Develop an Approach for Document Images. Sensors 2023, 23, 6845. https://doi.org/10.3390/s23156845
Zaman Z, Ahmed SB, Malik MI. Analysis of Hyperspectral Data to Develop an Approach for Document Images. Sensors. 2023; 23(15):6845. https://doi.org/10.3390/s23156845
Chicago/Turabian StyleZaman, Zainab, Saad Bin Ahmed, and Muhammad Imran Malik. 2023. "Analysis of Hyperspectral Data to Develop an Approach for Document Images" Sensors 23, no. 15: 6845. https://doi.org/10.3390/s23156845