Classification Endmember Selection with Multi-Temporal Hyperspectral Data

: In hyperspectral image classification, so-called spectral endmembers are used as reference data. These endmembers are either extracted from an image or taken from another source. Research has shown that endmembers extracted from an image usually perform best when classifying a single image. However, it is unclear if this also holds when classifying multi-temporal hyperspectral datasets. In this paper, we use spectral angle mapper, which is a frequently used classifier for hyperspectral datasets to classify multi-temporal airborne visible/infrared imaging spectrometer (AVIRIS) hyperspectral imagery. Three classifications are done on each of the images with endmembers being extracted from the corresponding image, and three more classifications are done on the three images while using averaged endmembers. We apply image-to-image registration and change detection to analyze the consistency of the classification results. We show that the consistency of classification accuracy using the averaged endmembers (around 65%) outperforms the classification results generated using endmembers that are extracted from each image separately (around 40%). We conclude that, for multi-temporal datasets, it is better to have an endmember collection that is not directly from the image, but is processed to a representative average.


Introduction
Hyperspectral remote sensing data have been used in many scientific fields for producing thematic maps through a diverse array of classification methods [1,2]. So-called spectral endmembers are used as reference for classifying a hyperspectral image [3][4][5]. Spectral endmembers are either extracted from an image or externally derived (e.g. from the field or an endmember library) [6,7]. The latter approach has been "mostly preferred and practiced" [3] and has "distinct advantages" [3]. These advantages include no influence from different characteristics between diverse sensors, closer to the mineral spectra in the study area than spectra from an existing spectral library and feasible for processing "large quantities of image data" [6][7][8]. Although these studies concluded that the classification of an image seems best done with endmembers derived from that image, it is unclear what is best for a multi-temporal image collection.
Multi-temporal remote sensing data are important for thematic analysis in diverse scientific fields, traditionally agriculture (e.g. [9][10][11]) and urban planning (e.g. [12][13][14]). It is also starting to be used in geological remote sensing (e.g. [15][16][17][18]). However, with the use of a multi-temporal dataset also, the complexity of a study grows [19]. Specifically, endmember extraction and selection is more complex, because extracted endmembers spectrally vary following the varying acquisition conditions one of each image [20]. There is neither a clear solution on how to extract endmembers from a multi-temporal dataset nor is it known what consequences there are of using extracted spectral endmembers for classifying a multi-temporal dataset.
This research evaluates the influence of spectral endmember extraction from a multi-temporal dataset on the consistency of classification results. We process three multi-temporal hyperspectral images with an identical atmospheric correction, use an automated endmember extraction method to classify the images, and evaluate the consistency. Specifically, we test how using a single set of spectral endmembers compares to using spectral endmembers extracted from each image separately.

Study Area
The study area is the Cuprite area in Nevada, USA ( Figure 1). Two lithological units are present in the area: tertiary volcanic and volcaniclastic rocks of mainly rhyolitic ash-flow tuffs with some air-fall tuff [21]. Hydrothermal alteration has widely metamorphosed these units into three alteration zones, which are argillaceous alteration zone, silicified alteration zone, and the opal alteration zone [22]. Swayze et al. [23] refer to the hydrothermal alteration zones in Cuprite as the "eastern alteration center" and the "western alteration center" (as shown in Figure 2).   [23]). The red box shows the study area of this research.
For three reasons, we chose the eastern alteration center as the study area in this research. First, the eastern alteration center hosts abundant hydrothermally altered minerals that can be identified by spectroscopy. Swayze et al. [23] map these minerals as Alunite, Kaolinite, Hydrated Silica, Montmorillonite, Buddingtonite, and a class as Kaolinite + Alunite, because of a high mixture of these two minerals. Second, only sparse vegetation is observed in the Cuprite area [25,26] and minerals should therefore be well exposed. Third, the Jet Propulsion Laboratory (JPL) uses the Cuprite area for calibrating and testing its spectrometers [27,28] and, as a result, several multitemporal hyperspectral scenes with similar spectral and spatial resolution are available.

Hyperspectral Data
Three hyperspectral images that were acquired by the airborne visible/infrared imaging spectrometer (AVIRIS) over three different years were chosen (Table 1). AVIRIS has 224 spectral bands covering a 400-2500 nm wavelength range. These images are downloaded at L1-B preprocessing level, which means that they were radiometrically calibrated to radiance-at-sensor values and orthorectified with a digital elevation model by JPL [29]. In our paper, we refer to the three images after the acquisition year: "2006", "2008", and "2010". .

Western center
Eastern center Table 1. Specifications of the three AVIRIS images used in this study and ancillary information of the images used in fast line-of-sight atmospheric analysis of hypercubes (FLAASH). The basic information was taken from JPL [24], the provider of the images. The image center location was automatically derived from three images by FLAASH. We derived the mean sensor altitude from JPL [24], while the ground elevation is an average value taken from Swayze et al. [23].

Atmospheric Correction
We applied the "fast line-of-sight atmospheric analysis of hypercubes" (FLAASH) software [30] in ENVI to atmospherically correct three images and convert the data from radiance-at-sensor to surface reflectance. FLAASH is based on MODTRAN4 and it was selected because it is frequently used to atmospherically correct AVIRIS images (e.g. [31][32][33]). Table 1 shows ancillary information used for FLAASH.

Data Subset
The atmospherically corrected images were subset spectrally and spatially for subsequent processing. For three reasons, the images were spectrally subset to a 2048-2308 nm wavelength range. First, we focus on minerals whose diagnostic absorption features are located in the shortwave infrared (SWIR). Second, bands that were shorter than 2048 nm were influenced by atmospheric absorption around 1900 nm. Third, the diagnostic absorption features of all six mineral types referred by Swayze et al. [23] are all in a 2048-2308 nm wavelength range (as shown in Figure  3 and Figure 4). Apart from the spectral subset, all three images were spatially limited to the eastern alteration center (the red box in Figure 1). We used the normalized difference vegetation index (NDVI) [34] to detect the presence of any vegetation. Additionally, we analyzed a possible change in vegetation cover over time and, as such, judged whether it could influence our results.

Endmember Libraries
Classification endmember spectra were extracted from the three images with the spatialspectral endmember extraction (SSEE) method [35], applied separately to each image. We compared all of the extracted endmembers with the mineral spectral library of the United States Geologic Survey (USGS) [36] and manually selected endmembers for subsequent classification. The criteria of the endmember selection mainly relate with the wavelength position and the absorption depth of diagnostic absorption features. The only exception is the endmember of montmorillonite; SSEE extracted a montmorillonite spectrum only from the 2008 image, while no montmorillonite spectra could be extracted from the 2006 and 2010 images. Therefore, we manually extracted the spectra from the 2006 image and 2010 image, at the same pixel location where the SSEE-extracted montmorillonite endmember in the 2008 image came from. Based on these criteria, we selected endmembers for each of the six alteration minerals referred to by Swayze et al. [23]. We assembled three spectral endmember libraries that were named after the method used to create them: extracted endmember libraries.
In addition, we created a fourth "averaged" spectral endmember library, based on the arithmetic average of the three extracted endmember libraries. The purpose of the averaged library is to classify all three AVIRIS images with a single endmember set as well. Subsequently, the consistency of the classification done with the averaged endmember library is compared with the consistency of the classification done with the three extracted endmember libraries.
We used linear correlation [37] to compare all endmembers representing the same mineral for understanding the linear similarity among endmembers in different libraries. The linear correlation calculation was done with Microsoft Excel software [38].

SAM Classification
The spectral angle mapper (SAM) [39] classifier was selected to produce mineral maps. SAM was selected, because it is a "popular" [40] and "the most famous" [41] classification method. We performed the classification on the images before subsequent resampling for change detection analysis, as Zhou et al. [42] found that resampling leads to mixing of spectra and thus loss of information.
The main challenge for applying SAM classification is the selection of appropriate classification thresholds [3]. In this research, six mineral maps were generated with SAM, by using the same class membership thresholds for each image (Table 2). These thresholds were manually determined by "trial and error" [43] while using the mineral map from Swayze et al. [23] as reference; an approach that is "widely used" [44]. Each image was once classified with the "extracted endmembers" associated with that image, and once classified with the "averaged endmembers", leading to six classified images in total.

Consistency Evaluation
For the evaluation of classification consistency on a pixel-by-pixel basis, the classification result of the 2010 image was resampled to the 3.3 m spatial resolution of the 2006 and 2008 images before collecting ground control points (GCPs) to undertake co-registration.
We used an image-to-image registration method [45] in IDL-ENVI software to spatially register all classified images. The image-to-image registration completes through resampling [45]. As indicated by Zhou et al. [42], resampling leads to a loss of information and leads to mixed spectra of pixels in the resampled images. Therefore, we registered classification results instead of the original images. The classification result of the 2006 image was set as the base image. GCPs were automatically collected from three images as ties to register the six classification results. The accuracy of automatically collected GCPs was tested through the root mean square error (RMSE) for each GCP and an overall RMSE for all GCPs [45,46]. For reducing the overall RMSE, we manually removed GCPs with an RMSE higher than 1 by "trial and error" and manually chose recognizable spots (e.g. street crossings) as GCPs.
We applied masking [47] to create maps with a single mineral class for comparing the classification results on a mineral-by-mineral basis (single-mineral map). With six mineral classes in six classified images, 36 single-mineral maps were produced in total (shown in Supplementary Materials: Figure S1-S36).
For evaluating the consistency of the 36 single-mineral maps, we used a change detection method [48] in ENVI. In change detection, we set a pair of single-mineral maps, as demonstrated in Figure 5. The change detection subtracts value of pixels in the "reference image" from value of pixels in the "test image". Therefore, by revaluation, the classification differences between two single-mineral maps can be presented as four statuses: omission, reproduced, unidentified, and commission ( Figure 5 (c)).
(a) (b) (c) Figure 5. (a) demonstrates for single-mineral map in "test image" the identified pixels were revalued as "2" and unidentified pixels were revalued as "0"; (b) shows an example for singlemineral map in "reference image", of which the identified pixels were set as "1" and unidentified pixels were set as "0"; (c) demonstrates the result of change detection, "-2" indicates a pixel identified in graph (b) but not in graph (a) (omission); "-1" indicates a pixel identified in both graph (a) and (b) (reproduced); "0" indicates a pixel unidentified in both of graph (a) and (b) (unidentified); "1" indicate a pixel identified in graph (a) but not in graph (b) (commission).
We summarized the number of pixels in each of the four statuses between each pair of the compared single-mineral maps as shown below: In this equation, R represents the consistency of certain mineral type, Re is the number of reproduced pixels, S is the amount of pixels in each of the compared single-mineral maps, and U is the number of unclassified pixels. Figure 6 shows the mean spectrum of each image before and after atmospheric correction. The atmospheric absorption around 1400 nm and 1900 nm were apparently overcorrected while the other atmospheric absorption features were corrected, as shown in Figure 6(a). Figure 6(b) shows the part of the spectra used for classification, which falls outside the overcorrected atmospheric windows.  ± Four spectral libraries were created with spectra that were extracted from the three images: one library for each image separately (three in total) and one library with averages of all endmember sets together (Figure 9). The extracted libraries consist of endmembers that were selected from all spectra that were extracted by SSEE from the three images. The similarity of all endmembers is shown in Table 3 and Table 4.  Figure 10 were generated with the SAM while using the extracted libraries and manually set classification thresholds (  Figure 10(b) were identified as hydrated silica, while only 45,704 pixels in Figure 10(a) and 123,656 pixels in Figure  10(c) were identified as hydrated silica. Similarly, 194,416 pixels in Figure 10(c) were identified as alunite, while 121,578 pixels in Figure 10(a) and only 20,300 pixels in Figure 10(b) were identified as alunite.

Results
(a) (b) (c) The SAM classification results of the three images using the averaged library are presented in Figure 11. Overall, the spatial patterns are similar, except for montmorillonite, which has 5791 pixels in the 2006 image, 1462 pixels in the 2008 image, and 1277 pixels in the 2010 image.

Discussion
We calculated NDVI to check whether vegetation changes over time and if masking would be necessary. The 2006 image was acquired on 2nd May, the 2008 image was acquired on 20th September, and the 2010 image was acquired on 14th October, covering spring to autumn. Results show that over 99.5% pixels of the three images present an NDVI value lower than 0.2: only approximately 0.5% pixels in the 2006 image, 0.03% pixels in the 2008 image, and 0.1% pixels of the 2010 image show a value higher than 0.2. This low NDVI value indicates that there is only sparse vegetation [34,49,50]. We can assume that vegetation cover in the Cuprite area remains sparse across different seasons.
FLAASH was used to atmospherically correct the three images, using the same atmospheric settings (Table 1) following Harris Geospatial Solution Inc. [51]. Nevertheless, the ATM correction remains an uncertainty in this research, as atmospheric conditions keep changing. However, the classification results that were derived from the ATM corrected images using the averaged endmembers ( Figure 11) show a high reproducibility (60-80%, except for montmorillonite). On the other hand, the comparison between the overall spectra derived from images between and after ATM correction ( Figure 6) illustrates that the atmospheric absorption curves have been corrected. Therefore, we conclude that the ATM correction does not play a major role in the reproducibility of classification.
As shown in Figure 3 and Table 3, endmembers of one mineral extracted from the three images have a similar shape but endmembers of certain minerals (e.g. hydrated silica) show different brightness (as shown in Figure 4). The brightness differences might be related to factors (e.g. illumination differences, different soil moisture conditions, etc.) that typically change with the acquisition of a multi-temporal dataset. It is necessary to properly select endmembers for classifying the multi-temporal dataset, given that the influence of the different brightness between endmembers to multi-temporal classification is hardly known.
In all classification results, Montmorillonite has a low abundance and also a low consistency. This could be a result of endmember selection: the SSEE only extracted a montmorillonite spectrum from the 2008 image and the montmorillonite endmembers in the other two extracted libraries were manually extracted from the same spot where the SSEE-extracted montmorillonite endmember came from.
We applied image-to-image registration to align the classification results pixel-by-pixel. An overall RMSE that indicates the average spatial shift between a pair of registered images was calculated after GCP collection [45]. Barazzetti et al. [46] indicated that image-to-image registration is inaccurate when the RMSE is higher than the ground sampling distance. Here, the RMSE of GCPs We decided to use classification consistency to observe the consequences of classifying multitemporal images while using extracted endmembers. This decision was based on two assumptions. First, the consequences of endmember selection will be highlighted when only endmembers are derived from different sources, while the same methods and parameter settings are used in every other link of multi-temporal images classification chain. We therefore did not change any parameter setting (including classification thresholds) or processing method for classifying the three images with a purpose for only focusing on endmembers over time. Second, the distribution of wellexposed minerals is rather stable over times. Thus, multi-temporal classification results should present similar mineral distribution, unless the endmember selection was influenced.
There will be an impending demand on a strategy to classify multi-temporal images consistently as both availability of hyperspectral datasets and the attention of multi-temporal analysis are growing. In this research, we find differences between the three classifications while using extracted endmembers (Figure 10), while the three classifications ( Figure 11) with the averaged endmember library show a better consistency. Therefore, we state that it is better to have an endmember collection as a representative average instead of using directly extract endmembers for a consistent classification of multi-temporal hyperspectral images.

Conclusions
With growing availability of hyperspectral datasets and attention to multi-temporal analysis, there is impending demand on a strategy to process multi-temporal hyperspectral datasets. The use of classification endmembers is a significant part of hyperspectral image processing chain, and endmember selection when dealing with multi-temporal hyperspectral analysis is therefore important for remote sensing studies.
The reproducibility of classification results that were created using the averaged endmember library outperforms the reproducibility of classification results generated by endmembers extracted from each image separately. This conclusion is contrary to the findings that were obtained on single images. The classification results produced using the extracted endmember libraries are different, although the endmembers of the extracted libraries are statistically highly correlated.
Although the advantage of using extracted endmembers has been shown for hyperspectral image classification, we conclude that an external library, which could be made by averaging multiple endmember libraries, leads to a set of more consistent and/or reproducible classification results when dealing with multi-temporal hyperspectral data.

Supplementary Materials:
The following are available online at www.mdpi.com/2072-4292/12/10/1575/s1, Figure S1: Change between alunite in 2006 and 2008 images separately produced using the extracted endmembers; Figure S2. Change between alunite in 2006 and 2008 images separately produced using the averaged endmembers; Figure S3. Change between alunite in 2006 and 2010 images separately produced using the extracted endmembers; Figure S4. Change between alunite in 2006 and 2010 images separately produced using the averaged endmembers; Figure S5. Change between alunite in 2008 and 2010 images separately produced using the extracted endmembers; Figure S6. Change between alunite in 2008 and 2010 images separately produced using the averaged endmembers; Figure S7. Change between buddingtonite in 2006 and 2008 images separately produced using the extracted endmembers; Figure S8. Change between buddingtonite in 2006 and 2008 images separately produced using the averaged endmembers; Figure S9. Change between buddingtonite in 2006 and 2010 images separately produced using the extracted endmembers; Figure S10. Change between buddingtonite in 2006 and 2010 images separately produced using the averaged endmembers; Figure S11. Change between buddingtonite in 2008 and 2010 images separately produced using the extracted endmembers; Figure S12. Change between buddingtonite in 2008 and 2010 images separately produced using the averaged endmembers; Figure S13. Change between kaolinite in 2006 and 2008 images separately produced using the extracted endmembers; Figure S14. Change between kaolinite in 2006 and 2008 images separately produced using the averaged endmembers; Figure S15. Change between kaolinite in 2006 and 2010 images separately produced using the extracted endmembers; Figure S16. Change between kaolinite in 2006 and 2010 images separately produced using the averaged endmembers; Figure S17. Change between kaolinite in 2008 and 2010 images separately produced using the extracted endmembers; Figure S18. Change between kaolinite in 2008 and 2010 images separately produced using the averaged endmembers; Figure S19. Change between kaolinite + alunite in 2006 and 2008 images separately produced using the extracted endmembers; Figure S20. Change between kaolinite + alunite in 2006 and 2008 images separately produced using the averaged endmembers; Figure S21. Change between kaolinite + alunite in 2006 and 2010 images separately produced using the extracted endmembers; Figure S22. Change between kaolinite + alunite in 2006 and 2010 images separately produced using the averaged endmembers; Figure S23. Change between kaolinite + alunite in 2008 and 2010 images separately produced using the extracted endmembers; Figure S24. Change between kaolinite + alunite in 2008 and 2010 images separately produced using the averaged endmembers; Figure S25. Change between hydrated silica in 2006 and 2008 images separately produced using the extracted endmembers; Figure S26. Change between hydrated silica in 2006 and 2008 images separately produced using the averaged endmembers; Figure S27. Change between hydrated silica in 2006 and 2010 images separately produced using the extracted endmembers; Figure S28. Change between hydrated silica in 2006 and 2010 images separately produced using the averaged endmembers; Figure S29. Change between hydrated silica in 2008 and 2010 images separately produced using the extracted endmembers; Figure S30. Change between hydrated silica in 2008 and 2010 images separately produced using the averaged endmembers; Figure S31. Change between montmorillonite in 2006 and 2008 images separately produced using the extracted endmembers; Figure S32. Change between montmorillonite in 2006 and 2008 images separately produced using the averaged endmembers; Figure S33. Change between montmorillonite in 2006 and 2010 images separately produced using the extracted endmembers; Figure S34. Change between montmorillonite in 2006 and 2010 images separately produced using the averaged endmembers; Figure S35. Change between montmorillonite in 2008 and 2010 images separately produced using the extracted endmembers; Figure S36.