The Sentinel 2 MSI Spectral Mixing Space

Abstract

A composite spectral feature space is used to characterize the spectral mixing properties of Sentinel 2 Multispectral Instrument (MSI) spectra over a wide diversity of landscapes. Characterizing the linearity of spectral mixing and identifying bounding spectral endmembers allows the Substrate Vegetation Dark (SVD) spectral mixture model previously developed for the Landsat and MODIS sensors to be extended to the Sentinel 2 MSI sensors. The utility of the SVD model is its ability to represent a wide variety of landscapes in terms of the areal abundance of their most spectrally and physically distinct components. Combining the benefits of location-specific spectral mixture models with standardized spectral indices, the physically based SVD model offers simplicity, consistency, inclusivity and applicability for a wide variety of land cover mapping applications. In this study, a set of 110 image tiles compiled from spectral diversity hotspots worldwide provide a basis for this characterization, and for identification of spectral endmembers that span the feature space. The resulting spectral mixing space of these 13,000,000,000 spectra is effectively 3D, with 99% of variance in 3 low order principal component dimensions. Four physically distinct spectral mixing continua are identified: Snow:Firn:Ice, Reef:Water, Evaporite:Water and Substrate:Vegetation:Dark (water or shadow). The first 3 continua exhibit complex nonlinearities, but the geographically dominant Substrate:Vegetation:Dark (SVD) continuum is conspicuous in the linearity of its spectral mixing. Bounding endmember spectra are identified for the SVD continuum. In a subset of 80 landscapes, excluding the 3 nonlinear mixing continua (reefs, evaporites, cryosphere), a 3 endmember (SVD) linear mixture model produces endmember fraction estimates that represent 99% of modeled spectra with <6% RMS misfit. Two sets of SVD endmembers are identified for the Sentinel 2 MSI sensors, allowing Sentinel 2 spectra to be unmixed globally and compared across time and space. In light of the apparent disparity between the 11D spectral feature space and the statistically 3D spectral mixing space, the relative contribution of 11 Sentinel 2 MSI spectral bands to the information content of this space is quantified using both parametric (Pearson Correlation) and nonparametric (Mutual Information) metrics. Comparison of linear (principal component) and nonlinear (Uniform Manifold Approximation and Projection) projections of the SVD mixing space reveal both physically interpretable spectral mixing continua and geographically distinct spectral properties not resolved in the linear projection.

1. Introduction

The Sentinel 2 constellation [1] extends the 50 year Landsat legacy of multispectral Earth imaging [2] with higher spatial, spectral and temporal resolution. The combined spatial and spectral resolution of a sensor is manifest in the spectral dimensionality and topology of the spectral feature space defined by dimensions corresponding to the imaging sensor’s different spectral band measurements [3,4]. In the case of decameter-resolution broadband sensors we refer to this feature space as a spectral mixing space to explicitly acknowledge the optical processes which occur when upwelling radiance from more than one spectrally distinct material is aggregated within the Instantaneous Field of View (IFOV) of a single pixel.

Previous analyses of spectrally diverse mixing spaces of both multispectral and hyperspectral sensors with decameter spatial resolution reveal a consistent topology with 97% to 99% of spectral variance embedded within a 3D linear mixing subspace bounded by distinct spectral endmembers [5,6,7,8,9]. These studies indicate that the vast majority of ice-free landscapes on Earth can be modeled accurately as linear mixtures of rock, alluvia and soil substrates (S), photosynthetic vegetation (V) and optically transmissive/absorptive materials like deep clear water, wet soils and low albedo substrates (e.g., ferromagnesian rocks). In addition, both micro and macro-scale shadow is often indistinguishable from absorptive/transmissive materials. Together, these components of the landscape are referred to as Dark targets (D) and comprise the third spectral endmember of the SVD linear spectral mixture model.

Within spectrally diverse mixing spaces, these materials are associated with endmember (EM) reflectance spectra that reside at the apexes of a triangular point cloud bounded by binary mixing lines between the Dark EM and each of the Substrate and Vegetation EMs. While both of these binary mixing lines are conspicuous in their linearity, the mixing trend between the Substrate and Vegetation EMs is generally concave inward as a result of the ubiquitous presence of multiple scales of shadow on most landscapes. In some landscapes the diversity of substrates is sufficient to extend the planar triangular mixing space into a 3D tetrahedral mixing space also bounded by a plane of substrates of varying reflectance, sometimes referred to as the “soil line” [10,11]—although this third dimension rarely accounts for >~3% of total spectral variance. The apparent linearity of spectral mixing within this Substrate, Vegetation, and Dark bounded mixing space renders it generally amenable to representation with linear spectral mixture models. The minimal linear model used to represent this mixing space is referred to here (and elsewhere) as the SVD model [5,6,7,8,9,12]. The utility of the SVD model is its ability to represent a wide variety of landscapes in terms of the areal abundance of their most spectrally and physically distinct components. Combining the benefits of location-specific spectral mixture models with standardized spectral indices, the SVD model offers simplicity, consistency, inclusivity and applicability. In contrast to spectral indices which map specific materials using only 2 or 3 spectral bands, spectral mixture models use all available spectral bands to map the areal abundance all spectrally distinct materials present within a pixel’s Instantaneous Field of View simultaneously.

The objective of this study is to quantify the information content, spectral dimensionality and topology of the spectral mixing space of a spectrally diverse compilation of Sentinel 2 MSI imagery collected over a wide range of terrestrial biomes. In anticipation of similar topology and linearity to the mixing spaces of other multispectral and hyperspectral sensors, we identify spectral endmembers and evaluate the validity and stability of the SVD model for the Sentinel 2 MSI sensors. In addition to the 3D spectral mixing space rendered by the (linear) principal components of the Sentinel 2 compilation, we also employ manifold learning to render nonlinear 2D embeddings of the mixing space to identify additional, more subtle spectral distinctions (clusters and mixing continua) among land cover subcategories that are not apparent in the principal component projections of the mixing space. This comparison of linear and nonlinear embeddings is extended to land cover-specific compilations of Sentinel 2 imagery in the companion analysis to that presented here [13].

2. Materials and Methods

2.1. Data

We construct a composite mixing space from a set of 110 Sentinel 2 MSI tiles collected from spectral diversity hotspots worldwide. The geographic locations and biome designations of these hotspots are shown in Figure 1. Unlike previous studies, we include numerous examples of cryospheric landscapes, evaporite basins and shallow marine environments, in addition to a range of anthropogenically modified landscapes (e.g., urban & agriculture). From this collection of (110 tiles × 10,980² pixels =) 13.3 × 10⁹ Sentinel 2 spectra we identify a second set of (12 categories × 10 examples =) 120 subtiles from specific land cover subcategories. Diverse sets of either 5 or 10 subtiles, each 10 × 10 km in area, are chosen from agricultural (10), evaporite (10), cryospheric (10), volcanic (5), urban (5), shallow marine (10), sand dune (10), closed (10) and open (10) canopy forest, scrub/shrub (5), tundra (5), wetland (10), and igneous/sedimentary/metamorphic rock + alluvium (10 + 10) landscapes for a total 120,000,000 subcategory-specific spectra. False color composites of both of these aggregate mosaics are shown in Figure 2. Tile IDs and subcategory subtile locations are given in Appendix B. All data were downloaded free-of-charge as Level 1C exoatmospheric reflectance from the USGS EarthExplorer data portal (https://earthexplorer.usgs.gov/, accessed on 1 October 2022).

Figure 1. Geographic and climatic distributions of 110 Sentinel 2 tiles from spectral diversity hotspots. Geographic distribution of sample sites is guided by climatic and geologic diversity as well as overall species biodiversity. Individual tile selection criteria favor spectral diversity arising from land cover diversity within and across biomes. Tile geographic coverage corresponds well to global land area distribution within the climatic parameter space (lower left) from [14]. All biomes are well represented. Biome classification (lower right) adapted from [15].

Figure 2. Sentinel 2 composites for 110 spectral diversity hotspot tiles (110 × 110 km) and specific land cover subcategories (10 × 10 km) selected from individual hotspot tiles. Identical 1% linear stretch applied to both mosaics.

The spectral diversity hotspots were chosen on the basis of climatic gradients, biome diversity, soil diversity, plant diversity and geologic diversity—both within and among individual tiles. Sources used include geologic (https://mrdata.usgs.gov/geology/world/map-us.html, accessed on 11 September 2021) and climatic (https://storymaps.arcgis.com/stories/61a5d4e9494f46c2b520a984b2398f3b, accessed on 11 September 2021) basemaps in conjunction with maps of soils (https://atlas-for-the-end-of-the-world.com/world_maps/world_maps_soils.html, accessed on 11 September 2021), biomes (https://www.worldwildlife.org/publications/terrestrial-ecoregions-of-the-world, accessed on 11 September 2021), plant biodiversity (https://databasin.org/datasets/43478f840ac84173979b22631c2ed672/, accessed on 11 September 2021) and crop wild relatives (https://colostate.pressbooks.pub/cropwildrelatives/chapter/introduction-to-crop-wild-relatives/, accessed on 11 September 2021).

2.2. Methods

All analyses described in this study use Sentinel 2 MSI bands 1, 2, 3, 4, 5, 6, 7, 8, 8a, 11, and 12. The 20 m and 60 m bands (1) are coregistered with and upsampled to the 10 m bands (2, 3, 4, 8) by bilinear interpolation.

The compilation of 13.3 × 10⁹ Sentinel 2 spectra from 110 spectral diversity hotspots provides a basis for statistical assessment of the information content of the MSI spectral bands. The parametric Pearson correlation coefficient and the non-parametric Mutual Information metric were computed for all band pairs. The Pearson correlation coefficient, r_xy, is given as:

$$r_{xy}=\frac{{{\displaystyle \sum }}_{i=1}^n\left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)}{\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(x_i-\overline{x}\right)}^2}\sqrt{{{\displaystyle \sum }}_{i=1}^n{\left(y_i-\overline{y}\right)}^2}}$$

(1)

where x and y are band-specific reflectances for each spectrum in the mosaic. Correlations are computed for all 55 band pairs.

The mutual information metric, as originally conceived by [16] and further formalized [17], is defined for two random variables X and Y as the Kullback–Leibler divergence from the product of the joint distribution from the product of the marginal distributions. That is,

$$MI=D_{KL}\left(p_{X,Y}\parallel p_X{p}_Y\right),$$

(2)

where MI is mutual information, D_KL is the Kullback–Leibler divergence, p_X,Y is the joint distribution of X and Y, and p_X and p_Y are the marginal distributions of X and Y. The MI of a variable with itself is defined as its self-information. Both mutual and self-information are bounded by [0, +$\infty$], and MI $\le$ SI. Conceptually, both self-information and MI can be understood as measures of “surprise”—the less probable are more surprising than probable events, and events with 100% probability are totally “unsurprising” (information = 0). Computation of MI was performed in Python using scikit-learn (package sklearn.featureselection.mutual_info_regression, with the implementation of [18,19]). As with the correlations, MI is computed for all 55 band pairs.

Spectral dimensionality is estimated from the variance partition of the eigenvalues of the spectra in the compilation mosaics described above. Variance partition by principal component dimension, given by the sum-normalized eigenvalues of the Singular Value Decomposition of the mosaics, and land cover subcategories, are shown in Figure 3. Because the tiles in the 120 tile spectral diversity hotspot mosaic are arranged in alphabetical order of the tile IDs (effectively random geographically), and most tiles are internally spectrally diverse, the mosaic can be subdivided into 5 subsets to assess the scaling of the spectral dimensionality of the compilation with number of spectra. Spectral dimensionality is estimated from variance partition for both the 110 tile Spectral Diversity Hotspot mosaic and the 120 subset land cover subcategory mosaic.

Figure 3. Spectral dimensionality from variance partition. Five 20 tile subsets (left) each have very similar variance partition to the full 110 tile aggregate (black) with 99% variance in the first 3 dimensions. The aggregate of 120 land cover subcategory subsets (right-black) also has very similar variance partition with 98% in the first 3 dimensions. The individual land cover subcategories vary somewhat, with sand and ice + snow having lower dimensionality than more heterogeneous categories. All have <1% variance in all dimensions >4. Both mosaics can be considered 3D in the sense that the 3 low order dimensions represent >98% of total variance.

The topology of the Sentinel 2 MSI spectral mixing space is characterized with the low order principal components identified by the variance partition. As described by [20], apexes of the mixing space indicate spectral endmembers and straight edges between apexes indicate binary mixing lines between the corresponding endmembers. Three pairwise combinations of the three low order PCs provide orthogonal planar projections of the 3D mixing space. Because the PC transform maximizes variance partition into the smallest number of uncorrelated dimensions, it reveals the “global” structure of the mixing space corresponding to differences in the shape of the spectral continuum of different endmembers. Therefore, different categories of land cover with physical properties form distinct limbs on the mixing space.

An alternative approach can be provided by manifold learning. Here, “local” structure is revealed by a nonlinear mapping of high-dimensional spectra into lower-dimensional embedding space. This mapping is constructed in such a way to optimally preserve local (statistical “nearest neighbor”) distance and/or connectivity structures. In this analysis we use the Uniform Manifold Approximation and Projection (UMAP) algorithm [21] to provide a complementary projection for comparison to the PC-derived mixing space. The UMAP algorithm assumes that the Sentinel-2 spectra are uniformly distributed on a locally connected Riemannian manifold with an (approximately) locally constant Riemannian metric. UMAP models this manifold using a fuzzy topological structure, then seeks a low-dimensional (2- or 3-D) embedding with an optimally similar fuzzy topological structure. In general, the resulting embedding is nonlinear and not invertible. For excellent background information on UMAP, see: https://umap-learn.readthedocs.io/en/latest/ (accessed on 1 October 2022).

All UMAP computations were performed using the open source umap-learn Python package on a commercially available laptop computer with 32 GB RAM, 2GHz Quad-Core Intel Core i5 CPU, and a 1536 MB Intel Iris Plus Graphics GPU. Hyperparameter sensitivity was investigated by sweeps which looped through various choices of n_components, n_neighbors, and min_dist. Results were found to be relatively insensitive to all three hyperparameters, within at least 2 orders of magnitude of variability. All UMAP shown in this analysis used n_components = 2, n_neighbors = 5 or 500, and min_dist = 0.1.

Spectral EMs derived from the composite spectral mixing space provide the basis for the standardized SVD model, which is inverted to provide endmember fraction estimates for all spectra in the mosaic. The 3 EM linear spectral mixture model is given explicitly as a set of 11 band-specific mixing equations. Because the number of mixing equations exceeds the number of unknown fractions, the system is overdetermined, allowing for inversion by minimization of model misfit.

FSE1 + FVE1 + FDE1     O1

.    .     .

.    .   =   .     In matrix notation:  O = FE + ε

.    .     .

FSE11 + FVE11 + FDE11   O11

(3)

where E is the 3 column matrix of 11-band endmember vectors, O is the observed spectral vector to be modeled, F_S|V|D is the vector of endmember fractions to be estimated, and ε is the model misfit to be minimized by the inversion. In addition, a unit sum constraint equation is included. The least squares solution, F = (E^TE)⁻¹E^T O [22] for the S,V,D endmember fraction estimates gives fractions well-bounded [0, 1]. Model validity is assessed by the Root Mean Square (RMS) of the difference between observed and modeled spectra using the S, V, D estimates and endmember spectra in the forward model (L₂ norm).

3. Results

Information content of the Sentinel 2 MSI bands is quantified using both Pearson’s r parametric correlation and the mutual information (MI) metric for all 55 band pairs for the 8 × 10 land cover subcategory mosaic. Table 1 and Table 2 give the r and MI matrices for each band pair for the 80 tile subset. In both matrices, values ≥ 0.8 are shown in bold italic. As in previous studies, the highest correlations occur between adjacent bands within the visible, near infrared (NIR) and shortwave infrared (SWIR) wavebands. However, unlike the study of [12] which focused only on soils and agriculture, this analysis also finds high correlations between visible and SWIR bands as a result of the high albedo sands included in the mosaic. The mutual information matrix shows a similar pattern of higher MI between adjacent band pairs within wavebands, but more pronounced than for r. In comparing correlations and MI estimates for all 55 band pairs, the two metrics show a decidedly nonlinear relationship, with a correlation of 0.88 and MI of 1.195 (r on MI) and 1.16 (MI on r) (Appendix A).

Table 1. Sentinel 2 MSI Band Pearson Correlation (r).

1	2	3	4	5	6	7	8	8a	11	12
1	0.62	0.57	0.5	0.47	0.31	0.21	0.21	0.18	0.39	0.42
0.62	1	0.96	0.85	0.8	0.53	0.35	0.37	0.3	0.67	0.71
0.57	0.96	1	0.95	0.93	0.7	0.52	0.55	0.48	0.82	0.85
0.5	0.85	0.95	1	0.99	0.76	0.57	0.6	0.53	0.92	0.95
0.47	0.8	0.93	0.99	1	0.84	0.65	0.69	0.63	0.95	0.96
0.31	0.53	0.7	0.76	0.84	1	0.9	0.96	0.94	0.87	0.8
0.21	0.35	0.52	0.57	0.65	0.9	1	0.91	0.92	0.71	0.62
0.21	0.37	0.55	0.6	0.69	0.96	0.91	1	0.98	0.76	0.66
0.18	0.3	0.48	0.53	0.63	0.94	0.92	0.98	1	0.71	0.6
0.39	0.67	0.82	0.92	0.95	0.87	0.71	0.76	0.71	1	0.98
0.42	0.71	0.85	0.95	0.96	0.8	0.62	0.66	0.6	0.98	1

Table 2. Sentinel 2 MSI Band Mutual Information (MI, 3 nearest neighbors).

1	2	3	4	5	6	7	8	8a	11	12
1.06	0.86	0.56	0.42	0.41	0.19	0.17	0.15	0.15	0.29	0.33
0.68	1.39	0.85	0.61	0.58	0.25	0.22	0.21	0.2	0.39	0.44
0.49	0.96	1.28	0.83	0.8	0.38	0.31	0.32	0.28	0.54	0.57
0.39	0.83	0.9	1.17	1.03	0.59	0.52	0.53	0.49	0.68	0.76
0.34	0.66	0.83	0.99	1.29	0.69	0.57	0.54	0.52	0.81	0.79
0.14	0.21	0.36	0.5	0.59	1.37	1.03	0.96	0.93	0.55	0.43
0.1	0.16	0.28	0.44	0.5	1.04	1.34	1.11	1.19	0.46	0.39
0.09	0.16	0.28	0.44	0.46	0.95	1.08	1.4	1.08	0.45	0.37
0.08	0.14	0.24	0.39	0.44	0.9	1.15	1.08	1.42	0.44	0.37
0.24	0.41	0.53	0.67	0.81	0.64	0.58	0.57	0.58	1.28	0.88
0.27	0.51	0.6	0.78	0.88	0.55	0.51	0.5	0.5	1.01	1.1

The spectral dimensionality of both the 110 tile spectral diversity hotspot mosaic and the 120 subset land cover subcategory mosaic are nearly identical (Figure 3), suggesting that the subcategories chosen encompass the salient structure of the larger spectral diversity hotspot mosaic. The variance partition in Figure 3 indicates that both mosaics are effectively 3D, containing 99% and 98% of spectral variance in the three low order dimensions and <<1% variance in all higher order dimensions. Figure 3 also shows the variance partition of 5 subsets of 20 tiles each, compared to the variance partition of the full 110 tile spectral diversity hotspot mosaic. The fact that all 5 subsets have nearly identical variance partition to the full mosaic suggests that each is sufficiently spectrally diverse to encompass the diversity of the full spectral mixing space. The implication of this convergence is that the full 110 tile mosaic is more than sufficient to encompass full global spectral diversity. Figure 3 also shows the variance partition of each of the 12 land cover subcategories compared to the variance partition of the full mosaic. As expected, there is significant variation among the subcategories with more homogeneous land cover like snow and sand showing lower relative dimensionality and others like evaporites showing higher relative dimensionality.

The topology of the spectral mixing spaces of both mosaics are very similar and consistent with their spectral dimensionality. Figure 4 shows the 3D mixing spaces of the 110 tile Spectral Diversity Hotspot mosaic and the 120 subset land cover subcategory mosaic as orthogonal projections of bivariate PC distributions, along with the most conspicuous spectral endmembers from each. Both mixing spaces show complete mixing continua spanning the Substrate, Vegetation and Dark endmembers (PC 3 vs. 2 projections)—although the greatest spectral variance is associated with the PC 1 vs. 2 projection, driven by the strong contrast of the two highest albedo endmembers, sand and snow. Snow/ice and reefs each form distinct mixing continua with the Dark endmember, while evaporites form more distinct clusters without a single dominant mixing continuum. This suggests a more complex spectral continuum that may not be as linear as the others. It is noteworthy that neither reefs nor evaporites generally represent linear spectral mixing among distinct land cover types. Whereas aggregate albedo of most landscapes is modulated by a combination of reflectance, illumination flux density and shadow, the albedo of reefs is also strongly influenced by water depth while the albedo of evaporite basins is most strongly modulated by moisture content and the presence of standing water. Purely cryospheric environments are distinguished from other environments by their more homogeneous gradients spanning the snow-firn-ice continuum. Partial snow cover in non-cryospheric environments (e.g., boreal forests) may exhibit linear or nonlinear spectral mixing, but is sufficiently complex to warrant a more focused investigation separately.

Figure 4. Spectral mixing spaces for Sentinel 2 mosaics. Orthogonal projections show the 3D topology of the PC spaces as continuous with clearly defined apexes corresponding to physically distinct spectral endmembers. Vegetation and Sand + Substrate endmembers show strongly linear mixing with the Dark (shadow or water) endmember. Both mosaics have very similar topology and endmembers in the 3D PC space, indicating that the 120 land cover subcategories capture the salient features of the 110 spectral diversity hotspots.

Because reefs, evaporite basins and cryospheric environments are geographically and spectrally distinct from the SVD continuum that encompasses the majority of Earth’s biomes, the focus of the rest of the analysis is on the 8 × 10 column subset of the land cover subcategories spanning the SVD continuum. This mixing space is effectively 2D with (81% + 14% =) 95% of variance in the two low order PC dimensions. This triangular mixing space is dominated by the Substrate-Dark and Vegetation-Dark mixing continua as seen in Figure 5. Unlike previous studies, the Sentinel 2 MSI SVD space shows a clear distinction (kink & discontinuity) between high albedo sands and the lower albedo substrates that bound one side of the SVD continuum. Most non-forest biomes fall within this continuum, with varying amounts of nonphotosynthetic vegetation (NPV) and exposed substrate interspersed with herbaceous and woody photosynthetic vegetation.

Figure 5. Sentinel 2 SVD spectral mixing space, spectral endmembers, and the corresponding SVD fraction space. An eight column (80,000,000 spectra) subset of the Land Cover Subcategory mosaic encompassing the SVD-bounded plane of the full mixing space is effectively 2D with PC dimensions 1 (81%) and 2 (14%) accounting for 95% of total variance, compared to PC 3 (2%). Maximum amplitude (Outer) and lower amplitude mean (Inner) endmember spectra for Substrate and Vegetation define bases for maximal and minimal SVD models (left). Inversion of the minimal model provides liberal estimates of SVD fractions (right), but excludes pure sand landscapes. Because sands lie outside the minimal SVD model, their Substrate fractions exceed 1.0 with Dark fractions < 0. The resulting planar SVD fraction distribution can be projected onto a 2D ternary diagram (lower right) with no loss of information.

The Substrate limb discontinuity, combined with the diffuse apex of the Vegetation limb suggest two different, but related, SVD models. Because pure sands (e.g., dunefields) rarely support vegetation communities, an SVD model using an inner Substrate endmember (S_i) is more physically plausible than an SVD model using an outer Substrate endmember (S_o) composed of pure sand. However, the outer Substrate endmember could be used for modeling landscapes where bright sands are prominent. Similarly, an outer Vegetation endmember (V_o) composed of a single pixel spectrum is less representative than an inner Vegetation endmember (V_i) composed of an average of several individual spectra at the more densely occupied inner Vegetation apex of the mixing space. Comparisons of inner and outer S and V endmembers are shown in Figure 5. All 5 endmember spectra are given in Table 3.

Table 3. Sentinel 2 MSI Spectral Endmembers (Exoatmospheric reflectance × 10,000).

λ (nm)	Si	Vi	D	So	Vo
443	1754	1084	1198	1536	1194
490	1799	827	946	1556	909
560	2154	892	739	2291	969
665	3028	410	280	5485	447
705	3303	1070	208	6236	1126
740	3472	4206	180	6889	4762
783	3656	5646	167	7323	6323
842	3566	5495	135	7176	6193
865	3686	6236	129	7530	6629
1610	5097	2101	26	10,252	1731
2190	4736	775	14	8745	712

Inversion of the SVD linear mixture model using the inner S_i and V_i endmembers yields the SVD fraction space shown in Figure 5. As expected, S fractions for the high albedo sands outside the triangular model exceed 1.0 with Dark fractions <0, but all other fraction estimates are well-bounded [0, 1]. Relatively small percentages of the binary S-D and V-D mixtures have V and S (respectively) fractions are slightly negative, but almost all are within 5% of 0. As shown in Figure 6, the spectra with these slightly negative near-zero fractions are limited to a few spatially contiguous geographies (e.g., mangroves, dunes or volcanic ash deposits). The distribution of RMS misfit between the observed and modeled spectra for the 80 subcategory composite has <6% misfit for >99% of 80,000,000 Sentinel 2 spectra, with the upper tail of higher misfits also limited to a few specific land covers not represented in the SVD model (e.g., turbid water, evaporites and light snow).

Figure 6. SVD model misfit and negative fraction distributions. RMS misfit map (linear stretch [0, 10%]) shows largest misfits associated with snow, evaporites and turbid water. RMS distribution shows 99% of spectra with <5% misfit (92% < 3%). Negative S (red) and V (green) fractions are well within 0.1 of zero.

The SVD fraction composite for the 8 × 10 land cover subcategories (Figure 7) is skewed toward RGB primaries, consistent with the S-D and V-D binary mixing continua seen in the SVD mixing space. The larger, more spatially heterogeneous collection of 110 spectral diversity hotspot tiles shows a wider range of intermediate spectral mixtures, as would be expected.

Figure 7. Minimal model SVD fraction mosaic for the primary land cover subset. Linear stretch [0, 1] for all fractions. Sands are saturated red because they are outside the minimal model with S_I fractions > 1.

The spectral mixing space rendered by the 2D UMAP embedding preserves the binary S-D and V-D mixing trends that dominate the PC and fraction spaces. Figure 8 shows a broader mixing continuum for the V-D limb extending to NPV-dominant biomes near the distinct S-D continuum. However, both UMAP embeddings also show a number of distinct clusters located outside the S-D and V-D continua. Most obvious are the several distinct sand clusters associated with different dunefields with distinct sand mineralogies. Additionally, noteworthy are the single distinct clusters associated with two tundra sites in the Canadian and Alaskan arctic and the single cluster associated with two mangrove sites in the Bangladesh Sundarban. It is noteworthy that all of the distinct clusters are associated with specific, spatially contiguous, areas within individual (sometimes multiple) subsets that are evident when labeled clusters in the UMAP space are back-projected to geographic space.

Figure 8. UMAP manifolds with distinct clusters labeled in mixing space then back-projected into geographic space (on RMS misfit map). Distinct clusters of spectra in the mixing space correspond to geographically distinct and spatially contiguous land cover.

4. Discussion

4.1. Spectral Information Content

The increased NIR spectral resolution of the Sentinel 2 MSI sensors contributes substantially to the spectral information content. While the correlations of adjacent spectral bands within the visible, NIR and SWIR are >0.8, scatterplots of band pairs reveal conspicuous departures from linearity in all but the highest correlations (>~0.95) which are reflected in correspondingly lower MI scores. A comparison of all bands with MSI band 8 in Appendix A illustrates a variety of such features, suggesting a diversity of spectra with significantly different reflectance (e.g., absorptions) between adjacent spectral bands. As in previous studies using Landsat multispectral and AVIRIS hyperspectral, the primary correlation structure clearly distinguishes visible, NIR and SWIR wavebands [12], but both r and MI suggest only moderate redundancy between adjacent spectral bands within each waveband. This may be partially responsible for the two clearly diverging lower limbs of the V-D continuum (F_v < ~0.5) trending toward the Dark endmember and toward the ternary mixing NPV + soil + shadow region near the S-D mixing continuum. Empirically, these mixing trends appear to correspond to closed and open canopy forest. Figure 5 also shows two distinct subparallel clusters on a single mixing trend on the V-D mixing continuum in the PC 3 vs. 2 space. These features are not seen in either the Landsat or MODIS mixing spaces [7,8].

Among the new VNIR bands provided by Sentinel 2 MSI, the greatest redundancy appears to be among bands 7, 8 and 8a, as indicated by some of the highest MI values. Whereas bands 4 and 5 have a correlation of 0.99, their MI is a comparatively lower 1.03, suggesting that the MI metric can resolve nonredundancies that correlation does not. The curvature of the log-linear relationship between the upper tails of the r and MI relationship (Appendix A) suggests that the two metrics are sensitive to different disparities between band reflectance distributions. Given the wide diversity of reflectances included in these mixing spaces, the similarities and differences in spectral continuum shape may overshadow meaningful differences in VNIR band information content. The fact that the UMAP embeddings identify a number of distinct clusters and apexes not apparent in the PC-derived feature space suggests that subtle differences in continuum curvature captured by the narrow NIR bands may indeed provide potentially useful resolving power to discriminate between otherwise similar reflectances within land cover subcategories.

4.2. Spectral Dimensionality and Mixing Space Topology

While the full mixing spaces shown in Figure 4 are both 3D, neither is amenable to a single mixture model containing 6+ endmembers (SVD + evaporite, reef, snow/ice). Such a mixture model would be both physically and mathematically implausible. Physically, as discussed above, reefs, evaporite basins and cryospheric landscapes are generally distinct from the continuum of biomes (Figure 1) in which a substrate continuum is interspersed with multiple scales of photosynthetic and nonphotosynthetic vegetation and structural shadow. Mathematically, the similarity of spectral shape of many evaporites with ice and snow spectra, and the inverse similarity of both to vegetation spectra will destabilize any inversion containing either two or all three of these endmembers because they are far from orthogonal, and often nearly colinear. The result is significant percentages of strongly negative (<−0.5) fractions in one or more of the fraction distributions. These are the primary reasons we focus on the 8 × 10 subset mosaic and the SVD model for the latter part of this analysis.

4.3. The SVD Model

The new, lower amplitude, Substrate endmember identified from the break in the S-D continuum supports a more physically plausible mixture model than earlier SVD models which used sand spectra for the Substrate EM. Using high albedo sand as an endmember has the undesirable consequence of likely underestimating the true substrate fraction in most situations where sand is not actually present. While we have advocated the use of local Substrate endmembers in previous studies, we find the use of a more realistic standardized Substrate reflectance as a reasonable substitute in situations where a single local Substrate endmember cannot be identified. In many landscapes, a more distinct plane of substrates may be more apparent than in the mixing spaces of these unusually diverse collections of spectra. If so, local soil and NPV endmembers might be distinguishable, as we have found in similar analyses of compilations of hyperspectral data in agricultural environments [13]. Because we did not identify a distinct apex associated with NPV in either of the composite mixing spaces, and observe that NPV occupies an intermediate region on the Substrate-Dark mixing continuum, we find no reason to extend the SVD model to include a NPV endmember.

The use of a lower amplitude inner Substrate endmember also provides more dynamic range to the Substrate fraction distribution by eliminating the much brighter sands which compress the dynamic range of the Substrate fraction. Allowing greater dynamic range may also facilitate distinction between different dry soil reflectances and different moisture content within a single soil type. At present, variations in dry reflectance and variations in moisture content are accommodated by varying fractions of Dark endmember—representing a fundamental ambiguity in the SVD model specifically and broadband reflectance generally. While this distinction may be impossible in single date imagery, spatiotemporal variations in soil moisture (and hence the S-D fraction continuum) of a single location may facilitate distinguishing these two effects on soil reflectance.

4.4. Manifold Topology and Spectral Resolution

The use of nonlinear, nonparametric dimensionality reduction provides a new and potentially very useful approach to spectral feature/mixing space characterization [23]. The presence of numerous distinct clusters in the UMAP projections contrasts strongly to the much more continuous PC-derived feature spaces. While the latter are essential to identifying spectral endmembers and verifying linearity of spectral mixing, the variance maximization on which the PC transform is based is much more sensitive to the shape of the spectral continuum than the presence of more subtle (lower variance) absorption features. In contrast, manifold learning algorithms that preserve local structure in the form of nearest neighbor proximity (e.g., UMAP, t-SNE) make it possible to identify more subtle differences in spectral curvature and absorption features (often aliased in multispectral imagery) that convey real physical meaning and may considerably extend the usable information content of narrowband multispectral sensors like Sentinel 2 MSI. The companion study to this [24] carries this duality a step further by combining the physically interpretable structure of the SVD mixing space with the more subtle features of local scale manifolds in the form of a joint characterization of the spectral mixing space.

In summary, the construction of a diverse collection of Sentinel 2 MSI tiles from 110 spectral diversity hotspots worldwide provides a basis for a globally representative spectral feature space. Because spectral mixing is pervasive in most biomes, even at 10 m resolution, we refer to this aliased feature space as a spectral mixing space. Identification of more spectrally homogeneous examples of a variety of specific land cover subcategories makes it possible to separate landscapes dominated by nonlinear spectral mixing (reefs, evaporite basins and cryospheric landscapes) from the more linear SVD continuum that spans most terrestrial biomes. Global standardized SVD endmembers chosen from the inner apexes of the SVD mixing space provide the basis for a general model of fractional subpixel land cover that is applicable to most biomes. While the PC-based mixing space allows for unambiguous identification of spectral endmembers, and determination of linearity of mixing (and inversion of linear mixture models), the use of nonlinear manifold learning to project proximity-preserving embeddings of the higher dimensional mixing space allows for identification of both mixing continua and isolated clusters of spectrally distinct land covers that are not generally apparent in the PC-based mixing space.