Hyperspectral Remote Sensing of Phytoplankton Species Composition Based on Transfer Learning

Abstract

Phytoplankton species composition research is key to understanding phytoplankton ecological and biogeochemical functions. Hyperspectral optical sensor technology allows us to obtain detailed information about phytoplankton species composition. In the present study, a transfer learning method to inverse phytoplankton species composition using in situ hyperspectral remote sensing reflectance and hyperspectral satellite imagery was presented. By transferring the general knowledge learned from the first few layers of a deep neural network (DNN) trained by a general simulation dataset, and updating the last few layers with an in situ dataset, the requirement for large numbers of in situ samples for training the DNN to predict phytoplankton species composition in natural waters was lowered. This method was established from in situ datasets and validated with datasets collected in different ocean regions in China with considerable accuracy (R² = 0.88, mean absolute percentage error (MAPE) = 26.08%). Application of the method to Hyperspectral Imager for the Coastal Ocean (HICO) imagery showed that spatial distributions of dominant phytoplankton species and associated compositions could be derived. These results indicated the feasibility of species composition inversion from hyperspectral remote sensing, highlighting the advantages of transfer learning algorithms, which can bring broader application prospects for phytoplankton species composition and phytoplankton functional type research.

1. Introduction

Marine phytoplankton plays a crucial role in aquatic ecosystems [1]. It contributes to primary production, affects the abundance and diversity of marine organisms [2], and exerts influence on climate processes [3]. Therefore, a better understanding of phytoplankton greatly enhances our understanding of the carbon and nitrogen cycles in marine ecosystems [4,5]. Phytoplankton functional types are species groups with specific roles in biogeochemical cycles [6]. Because the physiological processes of different phytoplankton species and associated compositions are distinct from each other, each individual phytoplankton species and its associated composition information are fundamental to its functional type [7]. In addition, they are also indicative of variability in phytoplankton species diversity in the ocean [8]. With the development of satellite-based ocean color remote sensing, which provides the advantages of wide-range, long-term coverage, high efficiency, and low cost [9], phytoplankton species, composition, and related research from space have been carried out [10,11].

The inherent optical properties of different phytoplankton species or groups have been studied [12,13], and numerous efforts have been made to understand phytoplankton communities in aquatic systems with the advancement of ocean color remote sensing. Much research has been focused on the identification of specific phytoplankton species or groups under blooming conditions using ocean color satellites with a moderate spectral resolution [14,15,16]. Furthermore, discrimination of phytoplankton phyla in blooming conditions has been achieved based on differences in optical signals among different phytoplankton populations [17]. However, these endeavors are constrained to a limited type of phytoplankton species or groups because of the limitations in band settings for multi-spectral remote sensing. Moreover, some studies have been focused on Case 1 water with relatively simple optical properties where optically active constituents co-vary with chlorophyll concentration [18,19]; however, it is more challenging to invert phytoplankton species composition from current ocean color remote sensing data with moderate spectral resolution in optically complex waters. Comparatively, hyperspectral remote sensing presents a promising tool to resolve the spectral variability of phytoplankton species [20].

Several ocean color algorithms have also been proposed for phytoplankton information inversion in natural waters [21,22,23,24]. Among these efforts, a derivative spectroscopy/similarity index (SI) approach is the most common method for identifying dominant phytoplankton species or groups [25,26,27]. However, because SI-based approaches assign unknown spectra to the reference spectra that have the largest SI, only dominant species or groups can be identified, so it is difficult to determine phytoplankton species composition using this method, and the algorithm degradation for coastal waters with high suspended particulate matter (SPM) and colored dissolved organic matter (CDOM) is inevitable [28,29].

Regarding data acquisition, most studies on phytoplankton community structures are based on the high-performance liquid chromatography–chemical taxonomy (HPLC-Chemtax) method [30]. On this foundation, Pan et al. established third-order polynomial functions for individual pigment concentration inversion [31,32,33], further estimating phytoplankton community composition along the northeastern coast of the United States and northern South China Sea. Zhang et al. constructed linear unmixing models for estimating phytoplankton taxonomic groups from absorption spectra of phytoplankton (a_ph(λ)) signals collected in the Chukchi and Bering Seas [34]. Related research based on in situ fluorescence measurements has been carried out [35,36]. For instance, Ling et al. used several band combinations of fluorescence signals to test their correlations with six phytoplankton groups in ocean regions in China [37]. Furthermore, neural network methods have also been explored [38,39]. However, most of these algorithms do not determine phytoplankton species relative composition from remote sensing reflectance (R_rs) spectra. Recently, with the development of learning algorithms and increasing computing power, machine learning has been applied to the field of earth science [40], including related successful applications in ocean color remote sensing [41,42,43]. Most machine learning methods function well under a common assumption—training and validation datasets are in the same feature space and follow the same distribution rule [44]. However, learning and prediction are often in different scenarios; sometimes we only have sufficient training data in one domain, whereas for the other related but different domain of interest, it is difficult to re-collect enough training data to rebuild the model. Thus, in such cases, transfer learning between task domains would be desirable [45].

In the present study, we measured the absorption spectra of 11 phytoplankton species frequently observed in ocean regions in China, and then built a general R_rs simulation dataset based on different species compositions, along with different chlorophyll a, SPM, and CDOM content. A machine learning-based method using hyperspectral remote sensing data was presented; by transferring part of the knowledge learned from the simulation dataset, with the preprocessed hyperspectral remote sensing reflectance as input, the composition of dominant phytoplankton species in natural waters could be predicted. We tested this method with in situ measurements in different ocean regions in China, and the results were adequate based on related statistical indicators. Subsequently, we applied it to Hyperspectral Imager for the Coastal Ocean (HICO) imagery to derive the spatial distributions of dominant species and compositions. This method could easily be applied to discrimination of phytoplankton groups and compositions.

The primary focus of this research was to determine phytoplankton species composition from ocean color R_rs in different ocean regions in China using transfer learning methods. Specifically, we focused on the following four aspects in this research: (1) developing a prediction model of phytoplankton species composition through a deep neural network (DNN) with a simulated R_rs dataset for 11 phytoplankton species cultures and other optical components combined, so-called NN_sim; (2) reforming the NN_sim through the introduction of an in situ R_rs dataset using a transfer learning approach, so-called NN_TL; (3) applying the NN_TL to HICO imagery to predict the dominant phytoplankton species composition in the Changjiang Estuary and adjacent waters; and (4) analyzing the sensitivity of our method under conditions of various concentrations of SPM, chlorophyll a and CDOM, spectral resolutions, and signal-to-noise ratios (SNRs).

2. Materials

2.1. In Situ Data

A total of 183 stations were surveyed during five cruise campaigns, namely, 201507cruise (i.e., cruise survey carried out in July 2015), 201605cruise, 201705cruise, 201805cruise, and 201807cruise, in different ocean regions of China. Water samples were collected simultaneously with the optical data at each station, as shown in Figure 1. There were 163 stations used for the NN_TL construction, and 20 stations (red stars in Figure 1) were randomly selected as the validation dataset from the uniformly spatially distributed stations in different cruise campaigns (Section 4.2).

2.1.1. Hyperspectral Radiometric Measurements

In situ radiometry measurements including the downwelling spectral irradiance (E_d), sky incoming spectral radiance (L_s), and total radiance of the water (L_tot) were carried out using the Hyperspectral Surface Acquisition System (HyperSAS, Satlantic corporation, Bellevue, WA, USA) by closely following NASA protocols [46]. The L_tot and L_s sensors were pointed to the sea and sky, respectively, at the same nadir and zenith angles between 30° and 50°, with an optimum of 40°. To minimize the sun glint effect, the azimuthal angle of the sensors was set to be within 90°–180° away from the sun, with an optimum of 135° [47,48]. R_rs(λ) was then calculated according to Equation (1):

$$R_{\mathrm{rs}}\left(\mathsf{\lambda }\right)=\frac{L_{\mathrm{tot}}\left(\mathsf{\lambda }\right)-{\rho }_{\mathrm{sky}}\left(\mathsf{\lambda }\right){\mathrm{L}}_{\mathrm{s}}\left(\mathsf{\lambda }\right)}{E_{\mathrm{d}}\left(\mathsf{\lambda }\right)}$$

(1)

where λ is the wavelength and ρ_sky(λ) is the sky radiance spectral reflectance at wavelength λ. The sun glint correction was performed to R_rs according to Busch et al. [49], and the corrected R_rs spectra were interpolated into 1 nm intervals between a wavelength of 370–858 nm.

2.1.2. Taxonomic Species Identification

Samples were fixed with formaldehyde (5%) immediately after collection and transferred back for the laboratory analyses. In the laboratory, phytoplankton cells were first concentrated with 100 mL settlement columns for 24 to 48 h, then identified and counted using an inverted microscope (Olympus corporation, Tokyo, Japan) [50,51]; for the method, we referred mainly to Utermöhl [52]. A total of 242 phytoplankton species were identified, including 129 diatoms, 97 dinoflagellates, 5 chlorophytas, 5 chrysophytas, 4 cyanophytas, 1 xanthophyta, and 1 euglenophyta (the classification was mainly based on http://www.algaebase.org/); after species identification and cell counting, the composition of each species in each station was calculated.

2.1.3. Validation Dataset

For this study, data at 20 stations were selected as the validation dataset (Figure 1), including hyperspectral R_rs data and concurrent data of phytoplankton species composition. In the algorithm validation stage (Section 4.2), by taking the preprocessed spectral data into the NN_TL, we then acquired the 26 most abundant phytoplankton species, which together accounted for more than 90% of the cell abundance of all species. A detailed description of the 26 phytoplankton species is listed in

Table 1. The 26 most abundant phytoplankton species acquired in the validation dataset used in this study.

Species Index	Species Name	Taxonomic Group	Species Index	Species Name	Taxonomic Group
1	Amphidinium carterae	Dino	14	Heterosigma akashiwo	Xant
2	Chaetoceros affinis	Diat	15	Noctiluca scintillans	Dino
3	Chaetoceros coarctatus	Diat	16	Prorocentrum dentatum	Dino
4	Chaetoceros lorenzianus	Diat	17	Prorocentrum minimum	Dino
5	Chaetoceros sp.	Diat	18	Pseudo-nitzschia delicatissima	Diat
6	Dictyocha fibula	Chry	19	Rhizosolenia hyalina	Diat
7	Gonyaulax spinifera	Dino	20	Rhizosolenia stolterforthii	Diat
8	Guinardia delicatula	Diat	21	Scrippsiella trochoidea	Dino
9	Gymnodinium lohmanni	Dino	22	Skeletonema costatum	Diat
10	Gymnodinium sp.	Dino	23	Thalassionema nitzschioides	Diat
11	Gymnodinium sp1	Dino	24	Thalassiosira angulata	Diat
12	Gymnodinium sp2	Dino	25	Thalassiosira sp.	Diat
13	Heterocapsa circularisquama	Dino	26	Trichodesmium thiebaultii	Cyan

Abbreviations: Dino = Dinoflagellate, Diat = Diatom, Chry = Chrysophyta, Cyan = Cyanophyta, Xant = Xanthophyta.

, and the species names are sorted alphabetically.

2.2. Simulation Dataset

2.2.1. Laboratory Data

Eleven phytoplankton species, namely, three dinoflagellates (Prorocentrum dentatum, zooxanthella, and Karenia mikimotoi), six diatoms (Skeletonema costatum, Thalassiosira weissflogii, Chaetoceros debilis Cleve, Phaeodactylum tricornutum, Chaetoceros curvisetus, and Cyclotella cryptica), one cryptophyta (Heterosigma akashiwo), and one chlorophyte (Nannochloris sp.), which were frequently observed in Chinese ocean regions [7,8,15], were cultured in a laboratory incubator. The temperature was set at 18–20 °C, light intensity was 2500 lx, 12 h light and 12 h dark. The a_ph(λ) spectra of phytoplankton species were measured using a Lambda-1050 UV/Vis Spectrophotometer (PerkinElmer corporation, Boston, MA, USA). The chlorophyll a concentration (C_ph) was measured using a F-2500 Fluorescence Spectrophotometer (Hitachi corporation, Tokyo, Japan). The representative mass-specific absorption spectra (normalized at 440 nm) of the 11 species are shown in Figure 2.

2.2.2. R_rs Simulation Dataset

Semi-analytical models were used to generate R_rs simulation dataset (Equations (2)–(10) in

Table 2. R_rs simulated formulas based on semi-analytical models in this study.

Eq.	Math Formula	References
(2)	$a_{\mathrm{ph}}\left(\mathsf{\lambda }\right){=a}_{\mathrm{ph}\text{\_}\mathrm{mix}}^{*}\left(\mathsf{\lambda }\right)C_{\mathrm{ph}}{,a}_{\mathrm{ph}\text{\_}\mathrm{mix}}^{*}\left(\mathsf{\lambda }\right)={\displaystyle {\displaystyle \sum }_{\mathrm{i}}^{\mathrm{n}1}}{\mathrm{w}}_{\mathrm{i}}{a}_{\mathrm{ph}\text{\_}\mathrm{i}}^{*}\left(\mathsf{\lambda }\right)$	[27,53]
(3)	$R_{\mathrm{rs}}\left(\mathsf{\lambda }\right)=\frac{0.52r_{\mathrm{rs}}\left(\mathsf{\lambda }\right)}{1-1.7r_{\mathrm{rs}}\left(\mathsf{\lambda }\right)}$	[54]
(4)	$r_{\mathrm{rs}}\left(\mathsf{\lambda }\right)=0.0895\mathrm{u}\left(\mathsf{\lambda }\right)+0.1247\mathrm{u}{\left(\mathsf{\lambda }\right)}^2,\mathrm{u}\left(\mathsf{\lambda }\right)=\frac{b_{\mathrm{b}}\left(\mathsf{\lambda }\right)}{a\left(\mathsf{\lambda }\right)+b_{\mathrm{b}}\left(\mathsf{\lambda }\right)}$	[54,55]
(5)	${a\left(\mathsf{\lambda }\right)=a}_{\mathrm{ph}}{\left(\mathsf{\lambda }\right)+a}_{\mathrm{spm}}{\left(\mathsf{\lambda }\right)+a}_{\mathrm{g}}{\left(\mathsf{\lambda }\right)+a}_{\mathrm{w}}\left(\mathsf{\lambda }\right)$	[56]
(6)	$b_{\mathrm{b}}\left(\mathsf{\lambda }\right){=b}_{\mathrm{bspm}}\left(\mathsf{\lambda }\right){+b}_{\mathrm{bph}}\left(\mathsf{\lambda }\right){+b}_{\mathrm{bw}}\left(\mathsf{\lambda }\right)$	[56]
(7)	$b_{\mathrm{bspm}}\left(\mathsf{\lambda }\right){=b}_{\mathrm{bspm}}\left(532\right){\left(\frac{532}{\mathsf{\lambda }}\right)}^{\mathrm{n}2},\mathrm{n}2=0.4114b_{\mathrm{bspm}}{\left(532\right)}^{-0.3},b_{\mathrm{bspm}}\left(532\right)=0.0068C_{\mathrm{spm}}$	[57]
(8)	$b_{\mathrm{bph}}\left(\mathsf{\lambda }\right)=0.005b_{\mathrm{bph}}(660)C_{\mathrm{ph}}^{0.795}\left(\frac{\mathsf{\lambda }}{660}\right),b_{\mathrm{bph}}\left(660\right)=0.407$	[58]
(9)	$a_{\mathrm{g}}\left(\mathsf{\lambda }\right){=a}_{\mathrm{g}}\left(440\right)\exp \left(-0.015\left(\mathsf{\lambda }-440\right)\right)$	[59]
(10)	$a_{\mathrm{spm}}\left(\mathsf{\lambda }\right){=a}_{\mathrm{spm}}\left(440\right)\exp \left(-0.009\left(\mathsf{\lambda }-440\right)\right){,a}_{\mathrm{spm}}\left(440\right)=0.0267C_{\mathrm{spm}}+0.2916$	[60]

where: λ = wavelength, a_ph(λ) = absorption coefficient of phytoplankton, a^*_{ph_mix}(λ) = mixed algae specific absorption, C_ph = concentration of chlorophyll a, n1 = total number of phytoplankton species, a^*_{ph_i}(λ) = ith species’ mass-specific absorption coefficient, w_i = composition of ith phytoplankton species, R_rs = remote sensing reflectance, r_rs(λ) = sub-surface remote sensing reflectance, b_b(λ) = backscattering coefficient, a(λ) = absorption coefficient, a_spm(λ) = absorption coefficient of suspended particulate matter, a_g(λ) = absorption coefficient of colored dissolved organic matter, a_w(λ) = absorption coefficient of water, b_bspm(λ) = backscattering coefficient of suspended particulate matter, b_bph(λ) = backscattering coefficient of phytoplankton, b_bw(λ) = backscattering coefficient of water, C_spm = concentration of suspended particulate matter, n2 = a coefficient, which can be calculated by b_bspm(532).

). The mass-specific absorption spectra of mixed algae a^*_{ph_mix}(λ) were calculated using Equation (2), where n1 is the total number of phytoplankton species, a^*_{ph_i}(λ) is the ith species’ mass-specific absorption spectra (Section 2.2.1), w_i is the composition of ith phytoplankton species, the sum of w_i is 1 and can be interpreted as the contribution of each species to the total a_ph(λ), the range of C_spm was set to 0.1–200 g/m³, the range of C_ph was set to 0.1–50 μg/L, and the range of a_g(440) was set to 0.01–0.5 m⁻¹. After the variables were set, the R_rs spectra of mixed algae were calculated using Equations (2)–(10). In this study, a total of 200,000 R_rs spectra of mixture algae were generated, and the band range of simulated R_rs was 370–858 nm, with 1 nm intervals, which was consistent with the in situ measurements.

2.3. Satellite Data

One cloud-free HICO image [61] over the Changjiang Estuary was acquired on 28 March 2012 (H2012088004724.L1B_ISS) via the ocean color website (http://oceancolor.gsfc.nasa.gov/) (Figure 1). L1B (Level 1 B: Top of atmosphere radiance) data were processed using SeaDAS software (https://seadas.gsfc.nasa.gov/; version 7.4, NASA, Washington D.C., WA, USA) to first perform a geometric correction, and the atmospheric correction was made using the approach referred to in [62] to generate ocean color R_rs. Then, the R_rs spectra of each pixel was interpolated into 1 nm intervals between 370 and 858 nm. Thereafter, we smoothed the R_rs spectra with a locally weighted scatterplot smoothing (LOWESS) filter [63], and the 2nd derivative spectra of smoothed R_rs spectra were then calculated with a band separation of 27 nm [64] and normalized by the L2-norm (Section 3.2.1).

3. Methods

3.1. Introduction of Transfer Learning

Transfer learning is often used in the scenario, where training and validation data do not feature in the same space or follow the same distribution [65]. Taking a remote sensing application as an example, Jean et al. [66] used high-resolution satellite images and convolutional neural networks to predict poverty in five African countries; with limited training data about socioeconomic indicators, nighttime light intensities were used as a data-rich proxy by the transfer learning method. The results proved that the method is feasible. In the present study, we built a large and general R_rs dataset (200,000 simulated spectra) with 11 common species, and the data volume was adequate to train a reasonable neural network; meanwhile, the phytoplankton species diversity was more abundant in the field measurement R_rs (e.g., there were 242 phytoplankton species during the five cruise investigations from 2015 to 2018). The parameter-transfer approach was used by freezing the first few layers of the trained neural network for the simulation dataset, and updating the last few layers with in situ dataset. This can be accomplished because, for the vast majority of DNNs, the first few layers learn features that are more general and appear not to be specific to a particular dataset or task, and eventually transitions from general to specific in the last few layers [67].

3.2. Transfer Learning for Deep Neural Network Construction

As described in Section 2.2.2, a large general simulation dataset was first generated, and a DNN was subsequently trained using this simulation dataset. However, the trained neural network was not directly applicable for the in situ dataset because of the differences in the numbers, as well as the types of phytoplankton species between the simulation and the in situ datasets. To cope with this challenge, we used the transfer learning technique to transfer part of the knowledge learned from the simulation dataset to the prediction model for the in situ dataset.

3.2.1. Preprocessing for Input Data

Several preprocessing procedures were taken for input data. First, a LOWESS filter (fraction: 0.1; weight function: quadratic function; iterations: 2) was applied to the raw R_rs spectra to minimize random noises [63]. Second, to enhance detailed information about small spectral variations, the 2nd derivative spectra of smoothed R_rs spectra (${\overline{R}}_{\mathrm{rs}}$) were calculated according to Equation (11) [68]:

$$\frac{{\mathrm{d}}^2{\overline{R}}_{\mathrm{rs}}}{\mathrm{d}{\mathsf{\lambda }}^2}{|}_{\mathrm{j}}=\frac{{\overline{R}}_{\mathrm{rs}}\left({\mathsf{\lambda }}_{\mathrm{i}}\right)-2{\overline{R}}_{\mathrm{rs}}\left({\mathsf{\lambda }}_{\mathrm{j}}\right)+{\overline{R}}_{\mathrm{rs}}\left({\mathsf{\lambda }}_{\mathrm{k}}\right)}{{\left(\mathsf{\Delta }\mathsf{\lambda }\right)}^2}$$

(11)

where Δλ = λ_k−λ_j = λ_j−λ_i, is the band separation, which was set to 27 nm in this study according to Torrecilla et al. [64]. Additionally, to emphasize the shape of the spectra rather than its magnitude, each derivative spectrum was normalized by its L2-norm.

3.2.2. Architecture

A five-layer neural network architecture for the simulation dataset including an input, output, and three hidden layers, i.e., NN_sim, was developed. The activation functions of the first two hidden layers were set to be ReLU [69,70], and for the last hidden and output layers, the activation function was set to be Sigmoid and Softmax [71,72], respectively. The dimension of the input layer was 435, the same length as the normalized derivative spectra, and the dimension of the output layer was 11, corresponding to the number of phytoplankton species in the simulation dataset. The dimensions for the hidden layers were 256, 64, and 32, respectively, and the simulation dataset was split into training (90%) and validation (10%) sets. When fitting the model, 90% of the training set was randomly chosen for training and the remaining 10% was used for testing; the Adam optimizer was used [73] and the loss function was set to MAE (mean absolute error); and the training procedure stopped after 800 epochs with a batch size of 512.

The NN_TL architecture was defined as the same as that of NN_sim, except that the dimension of the output layer was the species number of the in situ dataset, and the batch size was 5. The weights of the first three layers were copied from NN_sim and were frozen during the following training procedure, whereas the weights of the last two layers were set to be trainable. Among the 183 in situ samples, 163 were used for training and 20 samples were used for validation (Section 4.2), and at each epoch, NN_TL was trained on 130 random choices of samples (80% of the 163 samples) and tested on the remaining 33 samples (20% of the 163 samples). The training procedure stopped after 250 epochs. The flowchart of the method is shown in Figure 3.

The parameter settings in the model training (data distribution ratios for the simulation dataset and the in situ dataset, the number of dimensions for three hidden layers, the number of the batch size, the number of epochs, the activation functions) in the NN_sim and NN_TL are the parameter adjustment results. The standards of adjustments are the convergence performance of the loss function in the model training. We conducted tests and found the optimal parameters in this study, as shown in

Table 3. Model parameter adjustment test and optimal options.

Model Parameter	NNsim		NNTL
	Test	Optimal Option	Test	Optimal Option
Data distribution ratios	9:1, 8:2, 7:3, 6:4	9:1	9:1, 8:2, 7:3, 6:4	8:2
Number of dimensions	256:128:64, 256:64:32, 256:64:16, 256:32:4	256:64:32	256:128:64, 256:64:32, 256:64:16, 256:32:4	256:64:32
Number of epochs	800/1600	800	250/500	250
Number of the batch size	512, 256, 128	512	5, 20, 80	5
Activation functions	ReLU, Sigmoid, Softmax	Hidden layer 1: ReLU	ReLU, Sigmoid, Softmax	Hidden layer 1: ReLU
		Hidden layer 2: ReLU		Hidden layer 2: ReLU
		Hidden layer 3: Sigmoid		Hidden layer 3: Sigmoid
		Output layer: Softmax		Output layer: Softmax

.

The program was coding with keras 2.2.4 (using TensorFlow backend) and Python 3.6.3, and running on a personal computer with CoreTM i7 processor (Intel Corporation, City of Santa Clara, CA, USA) and 20 GB Random Access Memory (RAM). Training the NN_sim took about 2 h and 12 min, and training the NN_TL took about 3 min. The layers were fully connected. In NN_sim, there were 139,723 weights in total and all weights were learnable; in NN_TL, there were 147,346 weights in total and 10,066 were learnable and 137,280 were non-learnable. The neural networks were stored as a JavaScript Object Notation (.json) file, and can be called by Python conveniently.

3.3. Accuracy Evaluation

The model performance was evaluated in terms of mean absolute error (MAE), root mean squared error (RMSE), and mean absolute percentage error (MAPE) according to Equations (12)–(14), respectively:

$$\mathrm{MAE}={\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j=1}^m\left|P_{\mathrm{prd}}^{i,j}-P_{\mathrm{true}}^{i,j}\right|/\left(m·n\right)$$

(12)

$$\mathrm{RMSE}=\sqrt{{\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j=1}^m{\left(P_{\mathrm{prd}}^{i,j}-P_{\mathrm{true}}^{i,j}\right)}^2/\left(m·n\right)}$$

(13)

$$\mathrm{MAPE}(\%)=\frac{100}{\left(m·n\right)}{\displaystyle \sum }_{i=1}^n{\displaystyle \sum }_{j=1}^m|\left(P_{\mathrm{prd}}^{i,j}-P_{\mathrm{true}}^{i,j}\right)/\left(P_{\mathrm{true}}^{i,j}\right)|$$

(14)

where m is the number of species and n is the number of samples in the training and validation process, $P_{\mathrm{prd}}^{i,j}$ is the predicted composition of species j in sample i, and $P_{\mathrm{true}}^{i,j}$ is the true composition of species j in sample i.

4. Results

4.1. Transfer Learning and Neural Network Test

The DNN with transfer learning was tested; in addition, the conventional DNN was considered for comparative analysis. As shown in Figure 4, this includes mainly three parts: NN_sim (simulation dataset), NN_TL (combined with simulation dataset and in situ dataset), and conventional DNN (combined with simulation dataset and in situ dataset). The convergence process of NN_sim is shown in Figure 4a, where MAE decreased rapidly in the first 20 epochs, the rate of descent flattened out with increasing epoch numbers, and we found that, after 800 epochs, MAE was less than 1%, and NN_sim tended to be stable. By the transfer learning method, the trained parameters of first few layers in NN_sim were preserved, after updating the last few layers with the in situ dataset, NN_TL was well constructed (Figure 4b); the convergence process of NN_TL was similar to that of NN_sim and, after 250 epochs, the MAE of the test set was stable around 4%. The convergence process of the conventional DNN is shown in Figure 4c; the convergence process of the conventional DNN failed due to the enlargement of the MAE with the increasing epoch numbers.

Further tests of NN_sim were conducted. For the randomly selected 10% of the simulation dataset (there are 20,000 spectra), the predicted compositions versus the true compositions are shown in Figure 5. The statistical indicators are as follows: R² = 0.97, MAE = 0.52%, MAPE = 43.62%, and RMSE = 0.90% on average. The predicted accuracy is acceptable.

4.2. Phytoplankton Species Composition Prediction and Validation

Through comparing the in situ and NN_TL-predicted compositions (%) in the validation dataset, 26 species were acquired (Section 2.1.3). All species compositions and the sum of the rest of the species compositions (“others”) in each station are presented in the form of stacked bars (Figure 6). Different color columns represent different types of phytoplankton and the height of one column represents the composition of that phytoplankton. At some stations (e.g., stations 7, 8, 9, 13 and 20), specific species (S. costatum, P. delicatissima, P. dentatum, T. thiebaultii) were predominant, whereas at other stations, multiple species co-existed at similar ratios. Generally, NN_TL-predicted species compositions were highly consistent with the in situ measurements.

Specifically, the statistical indicators of 26 species predicted were calculated (Figure 7): there are 139 points in total (phytoplankton species composition greater than 0), and the in situ measurements and NN_TL-predicted results presented a good correlation: R² = 0.88, MAE = 3.38%, RMSE = 4.4%, and MAPE = 26.08%. In conclusion, the predicted results are acceptable.

4.3. Phytoplankton Species Composition Prediction from HICO

Through performing HICO data preprocessing (Section 2.3), the normalized 2nd derivative spectra data in each pixel of HICO were obtained, and the dimension of the spectra were kept consistent with the input layer in the NN_TL. In the model output phase, we acquired the 12 most abundant species, which together accounted for more than 99% of the cell abundance of all species in each pixel on average. The predicted phytoplankton compositions of these 12 species and the sum of the remaining species (“others”) are shown in Figure 8. The composition of each species varied greatly from each other and, for the same species, the composition varied spatially: among the 12 species, P. dentatum occupied the largest composition from the whole image, whereas S. costatum and P. delicatissima accounted for relatively large proportions near the Changjiang Estuary.

There were no coincident matchups between satellite and in situ observations to validate the predicted species composition. However, Song et al. [74] reported phytoplankton species distributed in this region through a cruise survey carried out from 22 to 28 May 2012, and the sampling range (30.5–32° N, 121.5–123.5° E) basically overlaps with the coverage of HICO. There are five of ten dominant species identified by light microscope in the survey of Song et al. [74], which are consistent with our HICO-predicted results (P. dentatum, Paralia sulcata, P. delicatissima, S. costatum, and S. trochoidea). This investigation validated our predicted results to some extent.

The phytoplankton species compositions are complex and variable, and which are affected by various environmental factors (e.g., temperature, transparency and nutrients) [74]. For example, the ratio of nitrogen to phosphorus (N/P) is considered an important factor influencing community structure; as the N/P increases, the composition of dinoflagellate increases and the composition of diatom decreases [75]. These factors linked to the remote sensing of phytoplankton species composition will be explored in the future.

5. Discussion

5.1. Transfer Learning for Phytoplankton Community

In Section 4.2, the phytoplankton composition inversion at the species level by transfer learning-based DNN was validated, and the predicted results are acceptable (R² = 0.88, MAE = 3.38%, RMSE = 4.4%, and MAPE = 26.08%). Whether the method is applicable to inversion of phytoplankton composition at the community level is discussed below with respect to related tests and comparative analysis.

5.1.1. DNN Tests for Phytoplankton Community Composition

As mentioned in Section 2.2.1, four communities (11 species) were cultured in the laboratory, and the absorption spectral data were reorganized at the community level. Similar to the research process at the species level, a NNC_sim (DNN based on the simulation dataset at the community level) was constructed. Applying the NNC_sim to randomly selected 10% of the simulation dataset at the community level, the predicted results are shown in Figure 9, and the statistical indicators are as follows: R² = 0.99, MAE = 0.43%, RMSE = 0.84%, and MAPE = 17.03% on average.

Following the DNN tests at the species level, the results are shown in Figure 5, and the statistical indicators are as follows: R² = 0.97, MAE = 0.52%, MAPE = 43.62%, and RMSE = 0.90% on average. Compared with the predicted accuracy at the species level (MAPE = 43.62%), the accuracy at the community level (MAPE = 17.03%) is improved significantly.

5.1.2. Validation for Phytoplankton Community Composition

A transfer learning-based DNN, combined the simulation dataset with in situ dataset at the community level, short as NNC_TL, was also established. The in situ and NNC_TL-predicted compositions (%) in the validation dataset (Section 2.1) are shown in Figure 10. The NNC_TL-predicted seven community compositions (dinoflagellate, chrysophyta, chlorophyta, xanthophyta, cyanophyta, euglenophyta, and diatom) are highly consistent with the in situ measurements. In addition, the statistical indicators of seven communities predicted were calculated (Figure 11): there are 63 points in total (phytoplankton community composition greater than 0), indicating R² = 0.99, MAE = 0.33%, MAPE = 1.74%, and RMSE = 1.28%.

Through comparing results shown in Figure 6 and Figure 10, the predicted results at the community level are more consistent with the in situ measurement than those at the species level, which is also proved by comparing results shown in Figure 7 and Figure 11. Compared with the predicted accuracy at the species level (MAPE = 26.08%), the predicted accuracy improved by an order of magnitude at the community level (MAPE = 1.74%). Because the types and concentrations of pigments in different species are different, this results in the subtle absorption differences. The pigment difference of different communities is greater than different species. Thus, the DNN may have fewer errors for the prediction at the community level.

5.2. Sensitivity Analysis

Sensitivity analyses of the transfer learning-based DNN were performed under different conditions at the species and community levels. Initially, the optical active components’ effects were considered (Figure 12). It can be seen that, among the three optical active components (C_spm, C_ph, a_g(440)) considered, the predicted MAPE (%) increased as C_spm increased (Figure 12a), and the variation range was reasonable (maximum of less than 35% at the species level, maximum of less than 17% at the community level) [34], even under relatively high SPM concentrations (180–200 g/m³), which revealed the strong robustness to the SPM of our method. This is very important for phytoplankton species/community composition inversion in optical complex waters, such as in the Changjiang Estuary, where the SPM was dominant in optical active components [76]. Thereafter, MAPE (%) decreased as C_ph increased (Figure 12b); because C_ph can be regarded as an indicator of phytoplankton biomass [77], our results indicated that the model performance was better in high biomass conditions. However, it was even more remarkable that, under lower C_ph conditions, our model still worked well (MAPE less than 35% at the species level, MAPE less than 15% at the community level); therefore, the model feasibility in Case 1 waters is predictable. The CDOM (in terms of a_g(440)) had small effects on the model performance (Figure 12c) and the range of variation was less than 4%, possibly because the optical contribution of the CDOM to R_rs was relatively small. All these conclusions prove that our method has reliable prediction results in different water environments.

The effects of the spectral resolution and signal-to-noise ratio (SNR) to the transfer learning-based DNN at the species and community levels were analyzed synchronously. Different proportions of random noise were added to the simulated and in situ spectral data (at the species and community levels) simultaneously, and the SNR was set as 100, 200, 500, 1000 and +∞ separately. The simulated hyperspectral R_rs and in situ R_rs with 1 nm bandwidth was resampled at different bandwidths synchronously, and the bandwidth was set from 1 to 20 nm and increased at 1 nm intervals. Then, we conducted data preprocessing as described in Section 3.2.1, and the transfer learning-based DNN for specific spectral resolution and SNR at the species and community levels was retrained. Afterward, the transfer learning-based DNN was used to predict phytoplankton species/community compositions under the corresponding spectral resolution and SNR conditions. The results of the accuracy evaluation are shown in Figure 13 (a at the species level and b at the community level). It was found that the MAPE increased with increasing bandwidth and decreased with the increasing SNRs. For the bandwidth greater than 5 nm (species level) and greater than 14 nm (community level), the MAPE increased significantly. The MAPE was 29.61%, 31.46%, 34.56%, and 37.69% (bandwidth equal to 5 nm at the species level), and it was 16.24%, 22.59%, 31.19%, and 34.34% (bandwidth equal to 14 nm at the community level), corresponding to 1000, 500, 200, and 100 of the SNR, respectively. Therefore, for ocean color satellite sensors in the future [78], sensors with bandwidths less than 5 nm at the species level, sensors with bandwidths less than 14 nm at the community level, and concurrently with higher SNR should be recommended.

5.3. Potentials and Limitations

Original domain (simulation dataset) and target domain (in situ dataset) exist in terms of our work, and there are similarities between these domains but they are not identical, which satisfies the precondition for knowledge transfer [65]. We used a mixture of the mass-specific absorption coefficients of 11 phytoplankton species cultures as the input for the simulation dataset, and the predicted results in natural waters were not limited to these 11 species. In addition, the associated composition percentages were also different; in fact, during the learning process for the general simulation dataset, there were about 140,000 nonlinear parameters to solve. Therefore, the general knowledge of phytoplankton species and associated composition was learned well. The optical properties of different species cultivated in the laboratory and natural waters were different, which was related to the associated abundance and growth conditions [79], but they shared similar spectral shapes. This is a disadvantage for traditional methods for distinguishing between them, but is beneficial for general knowledge transfer, as in our study.

Following the transfer learning method, we first used the in situ dataset to update the last layers in the corresponding DNN, which transitions the species information from general to specific as the number of layers increases, and then obtained predictions for phytoplankton species and associated compositions in natural waters. The output of our method is dynamic, and the dominant species and numbers depend on the R_rs spectra and total percentages we set up. For example, in the neural network model for the field measurements (NN_TL; Section 4.2), there were 26 species, which together accounted for more than 90% of the cell abundance of all species, and when applied to the HICO imagery (Section 4.3), 12 species were acquired, which totaled 99% of the total cell abundance.

We demonstrated the feasibility of the transfer learning approach in predicting phytoplankton species composition in natural waters. Further improvements could be made to this work. First, as the environment between laboratory measurements and in situ measurements is different [77], more effort should be placed on establishing a more complete simulation dataset, following these steps: 1. More species must be cultivated in the laboratory for optical properties research; 2. The optical properties of mixed species must be taken into consideration; 3. Light, nutrition, and related growth parameters should be connected with optical properties and finally incorporated into the simulation dataset [80]. Second, many factors affect the signal transmission from natural waters to the satellite sensor and much work needs to be done; for example, spectral classification is an effective method for deriving useful information [81], accurate atmospheric correction is also an important research direction [82], and if the atmospheric correction can be done well, combined with other advantages (sensor hardware situations, inversion algorithms, etc.), an accurate inversion of phytoplankton information from space is feasible. In addition, the inversion accuracy still needs to be improved, although transfer learning methods can make up for the deficiency of an in situ dataset to some extent; inversion accuracy could be optimized by increasing the amount of data, especially in situ-measured data. We used semi-analytical models [54,55] for building the simulation R_rs dataset, instead of a more accurate radiative transfer numerical model (e.g., HydroLight) [83]. However, HydroLight requires some time. For instance, we would need more than two months to build the simulation dataset, if 30 s was necessary for 1 spectrum on average, whereas the semi-analytical method only takes a few hours. As the previous study undertaken by Chen [58] has shown, compared with in situ-measured data, semi-analytical models show an underestimation in comparison with HydroLight. Thus, inversion accuracy would possibly decrease more than in a numerical model.

6. Conclusions

In the present study, we presented a machine-learning based phytoplankton species composition inversion method using hyperspectral optical data. First, we trained a deep neural network (NN_sim) based on learning from the simulated R_rs dataset; second, we reformed the NN_sim into an in situ neural network (NN_TL) through the introduction of an in situ R_rs dataset using transfer learning. Using the NN_TL, the validation of the predicted results of phytoplankton species composition shows acceptable results in different ocean regions and different cruise campaigns, indicating R² = 0.88, MAPE = 26.08%, MAE = 3.38%, and RMSE = 4.4%.

Assuming that the types and optical properties of phytoplankton species in one region do not vary considerably, the NN_TL can be applied to other hyperspectral measurements, such as the Hyperspectral Imager for the Coastal Ocean (HICO). The HICO-predicted results indicated that the compositions of different phytoplankton species varied considerably between and within species, with uneven spatial distributions. For the Changjiang Estuary, P. dentatum accounted for the largest average composition.

The performances of the transfer learning-based DNN at the species and community levels were analyzed comparatively, and the inversion accuracy at the community level was better than that at the species level (17.03% versus 43.62% in the randomly selected 10% of the simulation dataset, 1.74% versus 26.08% in the validation dataset). Sensitivity analyses indicated that the MAPE increased as SPM concentration increased and chlorophyll a concentration decreased. The signal-to-noise (SNR) and bandwidth also affected the predicted accuracy. The bandwidth smaller than 5 nm at the species level, smaller than 14 nm at the community level, and higher SNR should be suggested for the acceptable accuracy.

As the physics involved in all three methods is different (simulation dataset, in situ measured dataset, and HICO data), we analyzed the effective methods for improving differences between three datasets used in this article, by building a more accurate and complete simulation dataset, accurate atmospheric correction, etc. Moreover, as we only used an in situ dataset collected from five cruises in spring and summer, the types and optical properties of phytoplankton species may change in different seasons. To make the method more applicable, collecting more in situ data should be included in future work. Although the HICO ended operation in 2014, successive hyperspectral satellite missions (e.g., Chinese GF-5 (launched in 2018), scheduled NASA PACE and German EnMAP, etc.) may allow us to have more opportunities of matchups between satellite and in situ data for validation in the future.