^{1}

^{*}

^{2}

^{2}

^{3}

This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution license (

This paper considers an experimental approach for assessing algorithms used to exploit remotely sensed data. The approach employs synthetic images that are generated using physical models to make them more realistic while still providing ground truth data for quantitative evaluation. This approach complements the common approach of using real data and/or simple model-generated data. To demonstrate the value of such an approach, the behavior of the FastICA algorithm as a hyperspectral unmixing technique is evaluated using such data. This exploration leads to a number of useful insights such as: (1) the need to retain more dimensions than indicated by eigenvalue analysis to obtain near-optimal results; (2) conditions in which orthogonalization of unmixing vectors is detrimental to the exploitation results; and (3) a means for improving FastICA unmixing results by recognizing and compensating for materials that have been split into multiple abundance maps.

Hyperspectral imaging is a remote sensing approach that simultaneously collects both spatial and spectral data. Spectral data are collected in hundreds of narrow contiguous bands that may cover the visible, near-infrared, and short-wave infrared (0.4–2.5

There are a number of methods for exploiting hyperspectral image data to generate useful products. One common method is spectral unmixing. This process refers to one or both of two fundamental operations. The first is the identification of spectra that are representative of the distinct materials in the scene. These spectra are referred to as endmembers and the problem of identifying them as endmember extraction. It is possible that an endmember spectrum may not be found in an image pixel, even though the associated material is present in the scene. This occurs when the material associated with that endmember does not completely fill any single pixel in the image. In that case, which is not uncommon in real data, the endmember spectrum will only be present in an image pixel in combination with other endmember spectra. Because an endmember is uniquely associated with a specific material, the terms endmember and material are used interchangeably throughout the remainder or this paper.

The second spectral unmixing operation is abundance quantification, which entails determining the proportion of each endmember in each pixel of the image. Abundance maps provide useful visualizations of hyperspectral data, showing where each endmember is located in an image and how completely each pixel is filled by that endmember. Depending on the algorithm and the application, endmembers may be determined first and subsequently utilized for abundance quantification, the endmembers and abundances may be found simultaneously, or abundances may be computed without any prior endmember information [

There are a wide variety of algorithms that have been developed to unmix hyperspectral data. A recent survey article classified algorithms into one of four categories: (1) geometric; (2) statistical; (3) sparse regression based; and (4) spatial-spectral contextual [

Whenever spectral unmixing algorithms are assessed, two types of experiments are typically performed. In the first, synthetic images are created according to a simple generative model—usually the linear mixing model. The complexity of these images varies, but they are typically composed of 2–10 endmembers whose spectra are obtained from a real hyperspectral image or from a spectral reference library. In many cases spatial contiguity is incorporated using abundance maps consisting of simple square or circular regions. These kinds of test images are fairly common in the spectral unmixing literature [

The second type of experiment tests an algorithm by unmixing a real hyperspectral data set. The results of the unmixing are often assessed visually by recognizing landmarks in the original image and in the unmixed data [

Both of the experimental approaches described above are useful and even essential in assessing the effectiveness and behavior of a hyperspectral algorithm. There is, however, a third approach that can be viewed as something of a middle ground between the two. This approach utilizes synthetic images that more closely approximate real data by modeling scene geometry, material properties, sensor behavior, atmospheric contributions, and so forth. Complex scene geometry is desirable because it produces images that have regions of spatial contiguity, topographic variation, and endmember spectral variability. This approach also leads to broad variations in the spatial coverage of individual materials. Because the images are synthetic, complete ground truth data are still available. Such an approach is not intended to be a replacement for the existing methods described above. Instead, it should be treated as a complementary approach, allowing for exploration of unique insights and observations.

This complementary approach could be employed to explore a variety of hyperspectral unmixing algorithms. However, throughout the remainder of this paper, it is used to assess the behavior of FastICA. This exploration is warranted to confirm existing assertions regarding FastICA and also to provide further insight into the behavior of the algorithm.

The remainder of the paper proceeds as follows. Section 2 provides a basic overview of ICA and the FastICA algorithm. It also outlines the ICA data model and compares it with the linear mixing model used to describe hyperspectral data. Section 3 explains the approach taken to generate synthetic—but realistic—hyperspectral data cubes. Examples of both image data and abundance maps are shown. Section 4 describes the experiments performed, presents the results of those experiments, and provides insight into those results. Finally, Section 5 contains a few concluding observations and remarks.

Independent component analysis (ICA) is an approach for performing blind source separation (BSS). The generalized BSS problem is modeled as
_{1}(_{2}(_{K}^{T}_{1}(_{2}(_{L}^{T}

ICA is an approach that attempts to perform BSS by exploiting the statistical independence of the original sources. While this can be accomplished in a number of ways, many ICA algorithms invoke the central limit theorem [

Although nonlinear ICA methods exist [

The mixed data must satisfy two important conditions for ICA to be a valid unmixing approach. First, since ICA attempts to unmix the data by exploiting the independence of the sources, the sources must be independent. Second, because the methods of separation utilized by ICA algorithms attempt to maximize non-Gaussianity (based on the central limit theorem), no more than one source may be Gaussian distributed [

FastICA is an ICA algorithm that assumes the linear mixing model in

Prior to performing any source separation the observed data are whitened where ^{T}

As part of the whitening process the dimension of the observed data is reduced via principal component analysis (PCA). Unless specified by the user, the number of dimensions is determined automatically from the relative magnitudes of the eigenvalues of the covariance matrix of the observed data. This dimension reduction step is an attempt to estimate the number of sources and make the mixing matrix square, as required by the FastICA model.

After whitening and dimension reduction, the source separation is achieved by using a simple fixed-point algorithm to maximize a cost function. Thus, the source separation problem becomes

Because the whitening step effectively orthogonalizes the observed data, the unmixing matrix, ^{T}^{T}

One approach to modeling the radiance of a single pixel in a hyperspectral image is the linear mixing model [_{l}_{l}

The pixels in the observed cube can be indexed in row-scanned order so that each spectral band is represented as a one-dimensional vector, rather than a two-dimensional image. Then, the terms on both sides of

The hyperspectral mixing model in

In order to perform the kind of complementary experiments described earlier, a means of producing realistic images and the associated ground truth data is needed. This section describes the tool employed to produce the synthetic data that were incorporated into the experiments described in subsequent sections of this paper.

The Digital Image and Remote Sensing Image Generation (DIRSIG) software is a physics-based image simulation tool developed at the Rochester Institute of Technology (RIT) [

For our experiments, two test images were generated using DIRSIG. Both images incorporate the “MegaScene” geometric scene description, which models a 0.6 square mile area of Rochester, New York. A pushbroom spectrometer model that incorporates a spectral response between 0.4

The first radiance cube generated is referred to as “Mega1” because of its location within the first tile of the MegaScene. The scene is dominated by two large buildings surrounded by a parking lot. At the top of the image is a residential road with homes on either side that are mostly obscured by trees. Three tennis courts are located at the bottom of the image. The remainder of the scene is grass. There are 43 unique materials in this scene. The second radiance cube comes from the fourth MegaScene tile and is aptly named “Mega4.” This scene contains ten large industrial tanks surrounded by some buildings and parking lots. Around the periphery of the scene are areas of trees and grass. This scene contains 21 unique materials. Examples of the synthetic data are shown in

A list of the materials contained in each scene is provided in Section 5. These materials are sorted by the number of pixels in which they appear and are loosely segregated into four categories based on their spatial coverage in the image. Super-sparse materials are those with a combined coverage of less than one pixel. Materials in the sparse category typically are present in 1% or less of the image pixels and cover less than 0.5% of the image. They may or may not appear in the image as pure pixels. Dense materials appear in over half of the pixels in the image and consequently also constitute a large number of pure pixels. Materials falling between the sparse and dense categories are classified as intermediate materials. This categorization is used to analyze how materials of varying spatial distribution are affected in the spectral unmixing process. This is an example of the type of assessment that is not usually made in the two most common experimental scenarios described in Section 1.

Three sets of experiments were performed to characterize the utility of FastICA as a hyperspectral unmixing approach. The first set of experiments examined the impact of dimension reduction on the best-case unmixing scenario. Second, the effects of orthogonalization were explored, again considering a best-case unmixing scenario. Because dimension reduction and orthogonalization are not unique to FastICA, these two experiments are of interest beyond the scope of FastICA. In the final set of experiments, unmixing was performed using FastICA. The results of these experiments are quantified by comparing estimated material abundances with corresponding abundance ground truth. The quality of endmember extraction was not considered in these experiments. Some observations are made in the following narrative on the effects of adding noise to the synthetic images, but complete characterization of the impact of noise on the unmixing process is beyond the scope of this paper.

For the remainder of this paper, whenever performance is plotted versus material,

Because complete ground truth abundance maps are available, the optimal, linear unmixing vector and corresponding abundance estimate can be calculated for each material. This was done prior to performing any experiments. These results constitute a best-case unmixing scenario,

The unmixing formula _{i}_{i}

Because it is typically used as a preprocessing step in a variety of spectral unmixing approaches, including FastICA, an experiment was performed to examine the effect of dimension reduction on the best-case unmixing scenario. To do this, the maximum correlation abundance estimates were calculated using dimension-reduced data obtained from PCA. The same maximum correlation formulas _{N}_{N}

The results of this experiment are shown in

One approach to determining the number of dimensions that should be retained when performing PCA is to keep as many dimensions as are needed to retain some percentage of the total variance in the image. Retaining 99.9% of the total variance in the Mega1 and Mega4 images requires only six and five dimensions, respectively. Based on the results in

A second experiment examined the effect of constraining the unmixing vectors to be orthogonal. Because the PCA and whitening step decorrelates the observed data, it is expected that the unmixing vectors for the whitened data should be orthogonal. In the FastICA implementation, this constraint is enforced on the unmixing vectors at the end of each iteration of the cost function optimization.

To apply the orthogonality constraint to the optimal unmixing vectors requires a minor modification to the orthogonalization formula, since the optimal vectors were not calculated using whitened data. When the data are not whitened, the formulas for deflationary orthogonalization _{x}_{x}B^{T}

The optimal unmixing vectors calculated by

The results show that, in most cases, orthogonalization does not cause significant degradation of the estimates. This is true even in the presence of additive noise. There are a few exceptions, however, where the degradation is noticeable. Obvious examples of this are materials 2 and 6 in the Mega1 results. When symmetric orthogonalization is used, both show an appreciable decrease from the optimal correlation. When the ascending deflationary approach is used, material 2 is unaffected, but material 6 shows significant loss. Both are affected when the deflation is performed in descending material order. This behavior implies that there must be some information shared between the two materials. Thus, if material 2 is extracted first, it leads to a degradation when extracting material 6 and vice versa. Both experience degradation when the symmetric approach is used. This pattern can be explained by looking at an image representation of the matrix _{x}B^{T}

As a final experiment, FastICA was used to generate abundance maps for the Mega1 and Mega4 data. Each of the three cost functions in

Because the number of components was not specified, more independent components were generated than there are materials in the scene. For this experiment the normalized correlation coefficient of every independent component with every material ground truth was calculated, and the maximum was retained for each material. These results are shown in

Three ground truth images as well as the independent components most strongly correlated with them are shown in

The images in the first row of

The second observation is that these two components are both strongly correlated to the same material. The correlation coefficient of the first with the truth map is |

In this paper, the utility of realistic but synthetic data to assess spectral unmixing approaches was demonstrated using two hyperspectral images generated by DIRSIG. Three distinct but related experiments were performed to demonstrate this utility. The first experiment quantified the effect of dimension reduction using PCA and demonstrated that to achieve near-optimal results, more dimensions need to be retained than would be expected based on an analysis of eigenvalues. The number of additional dimensions that are necessary depends on the spatial distribution of the materials of interest, but is approximately an order of magnitude greater for sparse materials. The second experiment considered the impact of orthogonalization, which was found to reduce the correlation coefficient by less than 10% except in the case where sparsely distributed materials were found to be consistently co-located. The method of orthogonalization as well as the order of material extraction determines the severity of the effect for those materials. The final experiment showed that FastICA is effective at unmixing some, but not all, materials. This complementary experimental approach allowed for the identification of a splitting behavior in which FastICA produces multiple outputs containing distinct pieces of a common material. It was shown that these outputs can be merged in a way that produces improved results, increasing the correlation coefficient by 30%–60%. An approach to automatically identify and merge these outputs is an area of future research.

MegaScene 1, Tile 4 Test Image Materials.

Super-Sparse Materials (indicated by ○ in plots) | ||||

| ||||

1 | Sheet Metal, Maroon, Shiny, Fair | 1 | 0 | 0.016 |

2 | Tree, Maple, Trunk | 1 | 0 | 0.016 |

3 | Tree, Red Maple, Leaf | 1 | 0 | 0.016 |

4 | Tree, Dogwood, Trunk | 2 | 0 | 0.031 |

5 | Brick, Old Carolina Brick Company, Charlestowne | 2 | 0 | 0.266 |

Sparse Materials (×) | ||||

| ||||

6 | Sheet Metal, Black, Shiny, Dirty | 21 | 0 | 1.969 |

7 | Brick, Hampton Brick, Sandmist | 36 | 0 | 2.875 |

8 | Concrete, Cinder Blocks, Textured | 68 | 17 | 40.344 |

9 | Brick, Mixed Tan and Caramel Colors | 82 | 0 | 10.234 |

10 | Brick, Old Carolina Brick Co., Savannah Gray | 102 | 10 | 45.578 |

11 | Sheet Metal, White, Fair | 183 | 0 | 55.734 |

| ||||

Intermediate Materials (⋄) | ||||

| ||||

12 | Tree, Silver Maple, Leaf | 206 | 117 | 165.953 |

13 | Sheet Metal, Tan, Shiny, Fair | 276 | 187 | 229.172 |

14 | Building Roof, Painted Metal, Gray, Weathered | 333 | 129 | 224.063 |

15 | Tree, Dogwood, Leaf | 370 | 27 | 175.938 |

16 | Sheet Metal, Gray, Shiny, Dusty | 660 | 9 | 101.656 |

17 | Tree, Norway Maple, Leaf | 667 | 182 | 401.734 |

18 | Siding, Vinyl, Tan, Fair | 1,115 | 771 | 938.859 |

19 | Roof, Gravel, Gray | 1,194 | 767 | 998.188 |

| ||||

Dense Materials (□) | ||||

| ||||

20 | Grass, Brown and Green w/ Dirt | 8,718 | 2,534 | 5,864.234 |

21 | Asphalt, Black, New | 10,880 | 4,726 | 7,127.125 |

MegaScene 1, Tile 1 Test Image Materials.

Super-Sparse Materials (indicated by ○ in plots) | ||||

| ||||

1 | Siding, Mineral, Painted, Dark Green | 1 | 0 | 0.016 |

2 | Siding, Wood, Painted Off White, Fair | 1 | 0 | 0.078 |

3 | Tree, Black Oak, Bark | 2 | 0 | 0.031 |

4 | Siding, Cedar, Stained Dark Brown, Fair | 2 | 0 | 0.078 |

5 | Siding, Wood, Painted White, New, Rough | 2 | 0 | 0.094 |

6 | Brick, Old Carolina Brick Company, Charlestowne | 2 | 0 | 0.453 |

7 | Glass | 3 | 0 | 0.047 |

8 | Brick, Brampton Brick, Old School, Red | 4 | 0 | 0.313 |

9 | Siding, Vinyl, Off White, Fair | 4 | 0 | 0.594 |

10 | Roadway Surfaces, Sidewalk, Brick, Sealed, Mixed Color | 4 | 0 | 0.813 |

11 | Vinyl, Vision Pro Sample Board, Blue D-4 | 7 | 0 | 0.719 |

12 | Roof Shingle, Asphalt, Mix Brown, Good | 7 | 0 | 0.781 |

| ||||

Sparse Materials (×) | ||||

| ||||

13 | Stone Siding, Apple Ridge, Buckingham Fieldstone | 9 | 0 | 1.156 |

14 | Sheet Metal, Gray, Shiny, Dusty | 11 | 0 | 1.078 |

15 | Swimming Pool (Lining and Water) | 12 | 0 | 5.375 |

16 | Siding, Wood, Planks, Brown | 13 | 0 | 2.359 |

17 | Siding, Wood, Painted Tan, Fair | 15 | 0 | 2.313 |

18 | Roof Shingle, Asphalt, Harmony Sample Board, Cove Gray | 26 | 5 | 16.281 |

19 | Roof Shingle, Asphalt, Eclipse Sample Board, Twilight Gray | 27 | 0 | 1.859 |

20 | Roof Shingle, Asphalt, Black, Weathered | 29 | 16 | 20.516 |

21 | Roof Shingle, Asphalt, Black, Fair | 30 | 6 | 17.453 |

22 | Roof Shingle, Asphalt, Eclipse Sample Board, Shadow Black | 30 | 12 | 20.328 |

23 | Roof Shingle, Asphalt, Dark Light, Fair | 30 | 12 | 20.813 |

24 | Roof Shingle, Asphalt, Brown and Red Blend, Fair | 31 | 0 | 9.984 |

25 | Roof Shingle, Asphalt, Eclipse Sample Board, Forest Green | 35 | 15 | 24.281 |

26 | Roof Shingle, Asphalt, Brown, Black, New | 35 | 10 | 24.719 |

27 | Brick, Siding, Mix Brown, Fair | 44 | 13 | 33.750 |

28 | Roof Shingle, Asphalt, Harmony Sample Board, Sequoia Tile | 64 | 16 | 40.953 |

29 | Brick, Brampton Brick, Old School, Brown | 76 | 0 | 13.672 |

30 | Tree, Dogwood, Leaf | 77 | 3 | 30.797 |

31 | Brick, KF Plymouth Blend, Red Brick | 84 | 0 | 14.563 |

32 | Tree, Maple, Trunk | 140 | 0 | 5.406 |

33 | Tennis Court, Playing Surface, White Line | 194 | 0 | 37.625 |

| ||||

Intermediate Materials (⋄) | ||||

| ||||

34 | Tree, Black Oak, Leaf | 212 | 14 | 77.469 |

35 | Sheet Metal, White, Fair | 222 | 0 | 58.188 |

36 | Tennis Court, Playing Surface, Red | 250 | 67 | 155.688 |

37 | Tennis Court, Playing Surface, Green | 262 | 59 | 175.625 |

38 | Tree, Norway Maple, Leaf | 1,005 | 196 | 632.422 |

39 | Tree, Silver Maple, Leaf | 1,299 | 717 | 1,013.297 |

40 | Tree, Red Maple, Leaf | 1,360 | 7 | 611.625 |

41 | Roof, Gravel, Gray | 2,373 | 1,845 | 2,176.047 |

| ||||

Dense Materials (□) | ||||

| ||||

42 | Asphalt, Black, New | 8,198 | 2,928 | 4,975.422 |

43 | Grass, Brown and Green w/ Dirt | 9,275 | 3,124 | 6,158.922 |

Histograms of synthetically-generated abundance maps for

Examples of the test images generated in DIRSIG.

Correlation coefficient between optimal abundance estimates and corresponding ground truth abundances.

A comparison of material truth maps (first row

Normalized correlation coefficient of the maximum correlation estimates obtained using dimension reduced data. The first row shows the Mega1 results and the second shows the results for Mega4.

Normalized correlation coefficient of estimates obtained by orthogonalizing the optimal unmixing vectors for Mega1 (first row) and Mega4 (second row).

An image representation of the correlation coefficient of the optimal unmixing vectors for Mega1. Off-diagonal bright spots indicate correlation between the vectors, despite whitening. Notice the dark area in the bottom-right of the image due to the negative correlation between the dense materials.

Normalized correlation coefficient of estimates obtained using FastICA for Mega1 (first row) and Mega4 (second row). The deflationary orthogonalization results are shown with a solid line, symmetric orthogonalization with a dotted line.

Material truth maps from Mega1 (first row) and the independent components most correlated with them (second row).

Two independent components,

Number of dimensions necessary to obtain 95% and 75% levels of optimal correlation, by material classification.

95% | 75% | 95% | 75% | |
---|---|---|---|---|

super-sparse | 150 | 110 | 138 | 102 |

sparse | 116 | 53 | 46 | 29 |

intermediate | 36 | 10 | 23 | 8 |

dense | 16 | 6 | 12 | 4 |

Average normalized correlation coefficient FastICA based on material classification.

| ||||||
---|---|---|---|---|---|---|

Mega1, super-sparse | 0.8027 | 0.8095 | 0.7916 | 0.8023 | 0.6808 | 0.3745 |

Mega4, super-sparse | 0.7229 | 0.7146 | 0.7106 | 0.6943 | 0.6893 | 0.3547 |

Mega1, sparse | 0.7303 | 0.7191 | 0.7871 | 0.7435 | 0.7966 | 0.5463 |

Mega4, sparse | 0.7882 | 0.7765 | 0.8222 | 0.8174 | 0.8551 | 0.7446 |

Mega1, intermediate | 0.6055 | 0.5945 | 0.5841 | 0.5837 | 0.5876 | 0.5745 |

Mega4, intermediate | 0.6140 | 0.5909 | 0.5915 | 0.5787 | 0.6912 | 0.6012 |

Mega1, dense | 0.5566 | 0.4660 | 0.6390 | 0.4060 | 0.6467 | 0.3910 |

Mega4, dense | 0.5729 | 0.4881 | 0.6996 | 0.4120 | 0.7081 | 0.7384 |

Mega1, all | 0.7192 | 0.7084 | 0.7437 | 0.7145 | 0.7184 | 0.4964 |

Mega4, all | 0.6858 | 0.6636 | 0.6960 | 0.6586 | 0.7392 | 0.5965 |