Decision Fusion of Deep Learning and Shallow Learning for Marine Oil Spill Detection

Abstract

Marine oil spills are an emergency of great harm and have become a hot topic in marine environmental monitoring research. Optical remote sensing is an important means to monitor marine oil spills. Clouds, weather, and light control the amount of available data, which often limit feature characterization using a single classifier and therefore difficult to accurate monitoring of marine oil spills. In this paper, we develop a decision fusion algorithm to integrate deep learning methods and shallow learning methods based on multi-scale features for improving oil spill detection accuracy in the case of limited samples. Based on the multi-scale features after wavelet transform, two deep learning methods and two classical shallow learning algorithms are used to extract oil slick information from hyperspectral oil spill images. The decision fusion algorithm based on fuzzy membership degree is introduced to fuse multi-source oil spill information. The research shows that oil spill detection accuracy using the decision fusion algorithm is higher than that of the single detection algorithms. It is worth noting that oil spill detection accuracy is affected by different scale features. The decision fusion algorithm under the first-level scale features can further improve the accuracy of oil spill detection. The overall classification accuracy of the proposed method is 91.93%, which is 2.03%, 2.15%, 1.32%, and 0.43% higher than that of SVM, DBN, 1D-CNN, and MRF-CNN algorithms, respectively.

1. Introduction

Marine oil spill accidents have occurred frequently in recent years, causing serious harm to the marine environment. The explosion of the Deepwater Horizon platform in 2010 [1] led to crude oil leakage for nearly three months. Total oil spill volume reached about 4.9 million barrels, and the polluted seawater area was at least 10,000 square kilometers. The accident had a devastating impact on the marine ecological environment and biological resources in the Gulf of Mexico, which was the most serious oil spill accident in the history of the United States. More than 7000 tons of crude oil leaked from the blowout accident of Penglai 19-3C platform in 2011 [2], polluting 6200 square kilometers of seawater, resulting in serious pollution damage to the marine ecological environment of the Bohai Sea. The cost of compensation for marine ecological loss caused by the oil spill accident reached 250 million dollars. The explosion of coastal oil pipeline in the Jiaozhou Bay in 2013 [3] resulted in 62 deaths and 136 injuries. The oil spill area on the sea surface was about 3000 square meters, and the direct economic loss was about 120 million dollars. In 2018, the “SANCHI” tanker sank after collision with the cargo ship outside the Yangtze River Estuary in the East China Sea [4]. About 136,000 tons of condensate oil loaded on the tanker leaked, polluting 100 square kilometers of sea area, which threatened the marine ecological environment in the East China Sea and caused serious air pollution. Damage to marine ecological environments caused by large-scale marine oil spills is difficult to repair even over long periods of time. Worldwide attention has therefore been paid to the problem of marine oil spills, which have been listed by the American Academy of Science as one of 32 scientific problems to be solved by 2030.

Affected by wind and waves, the oil film on the sea surface is dynamic. Rapid and effective monitoring of the location and range of the oil spill is very important for rapid response. Remote sensing technology is the main means of emergency monitoring of oil spill on the sea surface, and has the outstanding advantages of a large area, synchronous and rapid detection. Using remote sensing data can not only monitor oil spill on sea in a large area, guide marine surveillance ships and aircraft to carry out law enforcement monitoring, but also provide the basis for law enforcement claims. At the same time, the range of oil pollution and the direction of oil spill diffusion can be tracked continuously according to remote sensing data, which are helpful to determine the best oil removal scheme. In addition, the establishment of an effective marine oil spill detection model based on remote sensing data can provide important technical support for the oil spill monitoring business of relevant departments, and save a lot of resources needed for field research, such as time, manpower, material resources, and financial resources.

Optical remote sensing and microwave remote sensing are the prevailing techniques for oil spill monitoring [5,6]. Among them, hyperspectral remote sensing has rich spectral information, which shows the different spectral response between oil film and seawater obviously. Its technical advantages in oil spill detection [7,8,9,10,11], oil spill pollution type identification [12,13,14,15] and oil film thickness estimation [16,17,18] are approved. The oil film on the sea surface is dynamic, and it is difficult to recognize. At the same time, it is affected by the complex marine environment and light conditions, such as waves and sun glints, which bring uncertainty to oil spill optical remote sensing detection [19]. Even though several oil spill incidents have occurred in recent years, there are fewer available optical remote sensing data due to the influence of clouds and weather, as well as satellite revisit periods. In the case of limited samples, the ability of features extraction using a single classifier is limited, and it cannot meet the need for oil spill accurate monitoring. In order to meet the need, some scholars have proposed several oil spill detection algorithm using hyperspectral data, mainly focusing on spectral unmixing, dimension reduction, image segmentation, and feature fusion algorithms [20,21,22,23,24].

Deep learning is a new field in machine learning research that is widely used in image processing, has become a popular method of remote sensing image classification, and is gradually being applied to hyperspectral image classification [25,26,27,28,29,30]. Among them, the convolutional neural network (CNN) and deep belief network (DBN) are the most representative algorithms and have achieved good results in hyperspectral image classification with accuracies higher than other classification methods. Many scholars have made great contributions in this research, and these novel deep learning network framework were proposed, such as a Stacked Network with Dilated CNN Features [31], two-stream deep convolutional neural network [32], etc., and have achieved remarkable classification results. CNN and DBN have been applied in coastal wetland classification [33,34,35,36], sea surface targets detection (oil spill, red tide, floating raft aquaculture) [37,38,39,40,41], and the detection accuracy is higher than those of other typical classification methods. However, all of the above research were conducted by a single algorithm or on the same scale. Different classifiers offer different generalization abilities in sample learning. Oil spill images show different features in different scales, and decision fusion algorithm based on fuzzy membership degree makes use of the complementarity among different classifiers and inherits advantages of different single classifiers and different scale feature results through different fusion strategies. Previous studies have shown that decision fusion can improve the accuracy of remote sensing classification [42,43,44,45,46,47].

At present, the challenges of marine oil spill detection using optical remote sensing can be summarized into two aspects: One is that the sea surface oil film boundary is not clear under the influence of marine environment such as wind, wave, current, and the interference of sun glint; second, in the case of limited oil film samples, the feature extraction ability of a single classifier is limited. Therefore, this paper develops a decision fusion algorithm for marine oil spill detection to integrate deep learning methods and shallow learning methods based on multi-scale features. On the basis of multi-scale features after wavelet transform, two classical shallow learning algorithms and two deep learning methods are used to detect oil spill information. From the perspective of target recognition information fusion, the decision fusion algorithm based on fuzzy membership degree is introduced to fuse the oil spill information of deep learning and shallow learning methods under different scale features.

The main contributions of our study are mentioned as follows:

(1) The developed multi-scale features extraction algorithm based on Daubechies wavelet is conducive to extract multi-scale feature information of irregular and unclear oil film boundary.

(2) Considering the ability of feature extraction of different classifiers is different, the decision fusion algorithm based on fuzzy membership degree is introduced to integrate multi-scale shallow and deep feature information for marine oil spill detection.

(3) To evaluate the effect of the algorithm proposed in this study, we compared the oil spill detection results of the proposed method with other four mainstream methods such as SVM, DBN, 1D-CNN, and MRF-CNN. The experimental outcomes of the proposed algorithm are inspiring and are suitable for oil spill detection due to higher accuracy as comparing to state-of-the-art.

2. Data and Preprocessing

2.1. Accident Summary and AISA+ Hyperspectral Image

The formation pressure of platform B in the Penglai 19-3 oil field was excessively high owing to water injection on 4 June 2011, which led to crude oil leakage. A well kick accident occurred on 17 June because of improper drilling operations on the platform C, resulting in the overflow of crude oil and oil-based mud from the well into the sea. The accident polluted about 6200 km² of sea area (exceeding the class I seawater quality standard), of which 870 km² were seriously polluted (exceeding the class IV seawater quality standard) [2,48].

The AISA+ airborne imaging spectrometer of Finland’s Specim company was installed on the sea surveillance aircraft of the State Oceanic Administration for trial use in 2005, showing extensive application prospects in marine resource protection, environmental monitoring, and marine law enforcement. The hyperspectral image used in this study is an AISA+ airborne image covering the oil spill area of Penglai 19-3 platform C (Figure 1a) acquired by the China Marine Surveillance North Sea aviation detachment on 23 August 2011. Detailed parameters of the AISA+ hyperspectral sensor are listed in

Table 1. AISA+ hyperspectral sensor parameters.

Parameter	Index
number of bands	258
spectral rang	400–1000 nm
spectral resolution	5 nm
spatial resolution	1.41 m@1 km
field of view	39.7°

. Figure 1b is AISA+ radiance data without preprocessing such as atmospheric correction, band selection or dimensionality reduction, with the size of 3904 × 512 pixels, which is marked with a blue rectangle in Figure 1a. A study area of 444 × 364 pixels (Figure 1d) was clipped from the overall image, which is marked with a red rectangle in Figure 1b.

2.2. Sample Selection

Based on the field data, 9442 samples (pure pixels) were selected, including 7048 training samples for model establishment, and 2394 verification samples for adjusting the model parameters and determining the network structure. An interpretation map (Figure 2c) is made by the China Marine Surveillance North Sea aviation detachment, the marine oil spill operational monitoring department, according to field aerial photos (Figure 1c) combined with human-computer interactive methods. The interpretation map was used to test the performance of the optimal model by evaluating its generalization ability. The sample selection is shown in

Table 2. The number of four types of training samples and test samples.

Accident	Data	Feature Types	Pixels Number of Training Samples	Pixels Number of Test Samples
well kick accident of platform C in Penglai 19-3 Oilfield	AISA+ hyperspectral image	oil slick	2073	704
		sea water	4381	1448
		platform and ships	322	157
		shadow	272	85

and the spatial distribution is shown in Figure 2.

3. Method

3.1. Multi-Scale Features Extraction Algorithm Based on Daubechies Wavelet

Due to the influence of sunlight and waves, there are fine ripples similar to oil slick in the sea water area, and fine ripples similar to seawater in the oil spill area, which will bring great error to oil spill detection. Discrete wavelet analysis method can not only separate the noise from the signal, but also highlight the characteristics of the original signal by decomposing the signal, whose essence is to extract image details by gradually thinning the sampling step in spatial domain, to separate the spatial feature images of different scales and to reflect them on the detailed images of different resolutions. Thus, wavelet transform has better scale characteristics.

In this paper, Daubechies (db) wavelet basis is selected as wavelet decomposition function to decompose hyperspectral data. In the first decomposition level, the original image is decomposed by wavelet transform into detail coefficients representing the high-frequency component and approximate coefficient representing the low-frequency component in the first scale space. Low-frequency component can save most of the low-frequency information of the image, smooth the spectral image, and eliminate the image noise. Then the approximate coefficient is decomposed into high-frequency coefficient and approximate coefficient in the second scale space. Finally, low-frequency component images of different scales are reconstructed through inverse wavelet transform. The 1-level and 2-level low-frequency component image are shown in Figure 3b,c, respectively.

3.2. Deep Learning Oil Spill Detection Algorithms Based on Multi-Scale Features

Hyperspectral images are characterized by the union of imagery and spectrum and contain rich spectral information. Deep learning has strong data mining ability and feature extraction ability and can automatically learn to obtain deep level feature information. Learned features are essential for data description to make classification more conducive.

3.2.1. Convolutional Neural Network (CNN)

CNN is the most prominent model in deep learning and has recently become a research hotspot for hyperspectral remote sensing classification. CNN has two main characteristics. One is local receptive fields, i.e., hidden layer neurons that only connect with the local image, and global image information can be synthesized by each neuron’s local perception. The other is weight sharing, i.e., each neuron in the hidden layer uses the same convolution kernel to convolve the image, and all neurons in the same feature plane have the same weight, which effectively reduces the number of parameters in the network and makes CNN have displacement invariance. If the number of feature maps is too large, an over-fitting phenomenon occurs in the CNN model algorithm. The maximum pooling method is used to cluster features in different locations. The CNN model structure used in this paper is shown in Figure 4, which consists of seven information layers, including one input layer, two convolutional layers, two pooling layers, one full connection layer, and one output layer.

• Convolutional Layer

Different convolution kernels are used to perform convolution operations to extract different features from the input feature map. Each convolution kernel detects the specific features at all locations and achieves weight sharing on the input feature map. The forward propagation of the convolutional layer is formulated as [34]

$$Q_s^t=f\left({\displaystyle \sum _{r\in V}{Q}_r^{t-1}\ast k_{rs}^t+b_s^t}\right),$$

(1)

where $Q_s^t$ represents the activation value of the output feature map $s$ in layer $t$, $V$ denotes a selection of input feature maps, $k_{rs}^t$ is the kernel linking the input feature map $r$ in layer $t-1$ to the output feature map $s$ in layer $t$, $b_s^t$ represents the bias associated with output feature map $s$ in layer $t$, “$\ast$” denotes convolutional multiplication, and $f(\cdot )$ is the sigmoid function.

• Pooling Layer

The pooling layers is also known as the subsampling layers, which are periodically introduced between convolutional layers, whose main purpose is to reduce the parameters of the output feature map from the convolutional layer and vaguely increase the rotation invariance of the features. The forward propagation of the convolutional layer is described as

$$Q_s^t=f\left(m_s^t\times down\left(Q_s^{t-1}\right)\right)+b_s^t,$$

(2)

where $m_s^t$ denotes the multiplicative bias of the output feature map $s$ in layer $t$, and $b_s^t$ represents the additive bias of the output feature map $s$ in layer $t$. Each output map is given its own multiplicative bias and additive bias. $down(\cdot )$ denotes a subsampling function. After the pooling operation, the resolution of the output feature map decreases but the features described by the high-resolution feature map are well maintained.

3.2.2. Deep Belief Network (DBN)

DBN combines forward unsupervised learning with reverse supervised learning, which can effectively restrain the over-fitting phenomenon of a neural network from occurring during classification. This improves the classification accuracy of the DBN model. The model is composed of a multilayer unsupervised Restricted Boltzmann Machine (RBM) and a layer of supervised back propagation network [39]. The classification process of the DBN model includes two stages: pre-training and fine-tuning. The DBN model structure used in this paper is shown in Figure 5, which consists of five information layers, including one input layer, one visible layers, two hidden layers, and one output layer.

The standard Boltzmann Machine is a full-connect graph and the training network is very complex. The vector $s\in {\{0,1\}}^n$ represents the state of $n$ neurons, ${\omega }_{ij}$ denotes the weight of the connection between visible-layer neuron $i$ and hidden-layer neuron $j$, and ${\theta }_i$ represents the threshold value of neuron $i$. The energy function $E\left(s\right)$ [49] of the Boltzmann Machine corresponding to the state vector $s$ is as follows. The occurrence probability $P\left(s\right)$ of the state vector $s$ is determined by its energy $E\left(s\right)$ and energy of all possible state vectors $E\left(t\right)$:

$$E\left(s\right)=-{\displaystyle \sum _{i=1}^{n-1}{\displaystyle \sum _{j=i+1}^n{\omega }_{ij}{s}_i}}{s}_j-{\displaystyle \sum _{i=1}^n{\theta }_i}{s}_i,$$

(3)

$$P\left(s\right)=\frac{e^{-E\left(s\right)}}{{\displaystyle {\sum }_t{e}^{-E\left(t\right)}}},$$

(4)

RBM only keeps the connection between the visible layer and hidden layer, and there is no self-feedback phenomenon in the layer. Thus, the structure of the Boltzmann Machine is simplified from a complete graph to a two-part graph [50]. Contrastive divergence (CD) is often used to train RBM [51]. Suppose there are $d$ visible-layer neurons and $q$ hidden-layer neurons in the network, and $\upsilon$ and $h$ represent state vector of visible layer and hidden layer, respectively. Because there is no connection in the same layer, then

$$P\left(\nu |h\right)={\displaystyle \prod _{i=1}^{d}P\left({\nu }_i|h\right)},$$

(5)

$$P\left(h|\nu \right)={\displaystyle \prod _{j=1}^{q}P\left(h_i|\nu \right)},$$

(6)

For each training sample $\nu$, the probability distribution of the neuron state in the hidden layer can be calculated by the CD algorithm according to Equation (6), and then $h$ can be obtained by sampling according to probability distribution. Similarly, ${\nu }^{\prime }$ is generated from $h$ according to Equation (5), and then $h^{\prime }$ is generated from ${\nu }^{\prime }$. The updated formula of connection weight is

$$\Delta \omega =\eta \left(\nu h^{\mathrm{T}}–{\nu }^{\prime }{h^{\prime }}^{\mathrm{T}}\right),$$

(7)

The forward learning process of the DBN model is the process of feature extraction. When RBM maps feature information of neurons in the visible layer, each neuron in the hidden layer of the RBM has the same probability of being activated. Features of the neurons in the visible layer can be accurately expressed after several training times. In this case, RBM can be regarded as a self-encoder to extract feature information of neurons in the visible layer.

In the pre-training process, the DBN model carries out forward training through layer by layer initialization, and maps and transmits the characteristic information of the input layer data by stacking RBM layers. In this paper, a softmax classifier is set at the top of the top-level RBM, which receives the output information of the top-level RBM as the input information. The softmax classifier outputs the classification results of the forward learning process by comparing the probability distribution. In the process of fine-tuning, on the basis of pre-training, each layer of RBM network can only ensure that the weight of this layer can achieve the optimal expression of the characteristic information of this layer, and cannot achieve the optimal mapping of the whole DBN model to the input information. Therefore, BP algorithm needs to be used to combine with forward unsupervised classification results and label data, and fine tune the connection weight and bias between neurons in each layer of the whole DBN model layer by layer from the top to the bottom according to the law of error back propagation.

3.3. Classical Shallow Learning Algorithms

Shallow learning usually refers to shallow neural network, but here refers to methods other than deep learning algorithms. The occurrence time of oil spill events is often uncertain, which makes oil spill data difficult to obtain. Classical shallow learning methods have better performance for smaller datasets and are more easily understood.

3.3.1. Support Vector Machine (SVM)

SVM is a shallow learning method based on statistical learning theory. It can automatically find the support vector that has a greater ability to distinguish classification and then construct the classifier, which can maximize the interval between classes to achieve good statistics when the number of samples is small. This method has high convergence efficiency, training speed, and classification accuracy, and has been widely used in many fields of research in recent years [52,53]. The kernel function is the radial basis function (RBF) and the decision function is

$$f(x)=\mathrm{sgn}({\displaystyle \sum _{i=1}^n{\omega }_i{e}^{\left(\gamma {\Vert x_i–x\Vert }^2\right)}}+b),$$

(8)

where ${\omega }_i$ represents the coefficient of support vector, $\gamma$ is the parameter in the kernel function, the value here is 0.004, $x_i$ is the support vector, $x$ are samples of labels to be predicted, and $b$ is the offset coefficient.

3.3.2. Mahalanobis Distance (MD)

Mahalanobis Distance represents the distance between a point and a distribution, which is an effective method to calculate the similarity of two unknown sample sets. Its calculation is based on the overall sample. Different from Euclidean Distance, Mahalanobis distance takes into account the relationship between various characteristics, and can eliminate the interference of correlation between variables. Its disadvantage is that it exaggerates the effect of small variable.

3.4. Decision Fusion Method Based on Fuzzy Membership Degree

In this paper, the decision fusion algorithm based on fuzzy membership degree is used to realize the fusion of multi-source oil spill information obtained by deep learning model and shallow learning method [35]. The basic idea (Figure 6) is: (a) the detection results $C_m$ and $C_{m+1}$ of two classifiers are compared and if the pixel types in the same position are the same, (i.e., $C_m\left[i,j\right]=C_{m+1}\left[i,j\right]$), then the fusion image pixel types remain unchanged. (b) If the types are different, then the class $\max C_{m+1}$ with the largest number of object types in the window with a pixel size of 3 × 3 centered on the pixel is counted to determine whether the class is the same as that of the pixel in the corresponding position in another image, (i.e., $C_m\left[i,j\right]=\max C_{m+1}$), then the fusion pixel is $F\left[i,j\right]=C_m\left[i,j\right]$. Similarly, if $\max C_{m+1}$ is different from $C_m\left[i,j\right]$, the relationship between $\max C_m$ and $C_{m+1}\left[i,j\right]$ is calculated. If $C_{m+1}\left[i,j\right]=\max C_m$, then the fusion pixel is $F\left[i,j\right]=C_{m+1}\left[i,j\right]$. (c) When the pixel category cannot be discriminated, the membership degree $P_r^m$ that the pixel $m$ belonging to category $r$ is calculated according to Equation (9). If the maximum membership degree meets certain conditions, then the category with the largest membership degree is selected as the final category of the pixel.

$$P_r=0.5+{\left\{{\displaystyle \sum _{j=1}^n{\left[{\omega }_j\left(P_r^m-0.5\right)\right]}^{\alpha }}\right\}}^{\frac{1}{\alpha }},$$

(9)

where $P_r$ denotes the degree of belonging of type $r$, the range of category $r$ is [1,4] in the experiment, that is, four categories, and $n$ represents the number of classification images, ${\omega }_j$ denotes the weight of each single objective feature, and satisfies the equation of ${\displaystyle \sum _{j=1}^n{\omega }_j=1}$. $\alpha$ is odd, and the classification accuracy of the two classifiers is $a$ and $b$, respectively. Then, ${\omega }_1=\frac{a}{a+b}$ and ${\omega }_2=\frac{b}{a+b}$.

4. Results

4.1. Oil Spill Detection Results of Single Classifier under Different Scales

Aiming at three scale features, namely the original image (original scale), the low-frequency component image after 1-level wavelet transform combined with the original image (first-level scale), and the low-frequency component image after 1-level and 2-level wavelet transform combined with the original image (second-level scale), two deep learning methods and two shallow learning methods are used to extract the oil spill information based on the same training samples (Table 2), detection results are shown in Figure 7. Two deep learning models and two shallow learning classifiers have different abilities in mining data feature, and the performance of oil spill detection is also different. Intuitively, there is no obvious boundary between oil film and seawater in the detection results with different scales of MD algorithm, and there is more speckle noise, which is quite different from the interpretation map (Figure 2c) obtained by human–computer interaction. The oil film patches in the detection results of SVM based on different scale features are discontinuous, and some seawater pixels are mistakenly divided into platform and ship pixels. The detection results with different scales of CNN and DBN algorithms can keep the continuity of the oil film on the sea surface well. The detection results of CNN and DBN based on the first-level scale feature are more consistent with the interpretation map obtained by human–computer interaction, and effect is the best (Figure 7b). Some seawater pixels are mistakenly divided into oil film pixels in CNN and DBN detection results based on second-level scale characteristics (Figure 7c). These differences are the basis of data complementarity using decision fusion method, and also the significance of decision fusion of deep learning method and shallow learning method. The specific analysis will be described in combination with the accuracy evaluation later.

4.2. Experimental Results of Decision Fusion

Based on the oil spill detection results of two deep learning algorithms CNN and DBN, and two shallow learning algorithms SVM and MD, the decision fusion algorithm is introduced to fuse the oil spill information from the perspective of target recognition information fusion. The fusion results are shown in Figure 8. Compared with the single classifier, the oil film in the decision fusion results of deep learning and shallow learning at different scales is more continuous, and there are fewer broken patches, and the detection effect of the oil film is improved to varying degrees. Among them, decision fusion results based on the first-level scale features of two deep learning algorithms and SVM match the interpretation graph best (Figure 8b). Decision fusion detection results of MD, CNN and DBN algorithm at different scale feature can overcome the problem that there is no obvious boundary between oil film and seawater in MD detection results. At the same time, they inherit the characteristics of single classifier detection results and still have more speckle noise.

5. Discussion

5.1. Accuracy Evaluation of Oil Spill Detection

Precision and Recall are two measures widely used in statistical classification to evaluate the quality of classification results. Precision represents the ability of the classification model to return only relevant instances. Recall represents the ability of the classification model to identify all relevant instances, but sometimes there is a contradiction between Precision and Recall. To effectively evaluate the advantages and disadvantages of the different algorithms, the F1 score, also known as the balanced F score, is introduced to harmonize Precision and Recall. The F1 score has a better evaluation ability for a binary classification problem. Here, the detection performance of different algorithms is evaluated for the single target of the oil film.

Table 3. Oil spill detection accuracy of four methods based on different scale features.

	Evaluation Criterion	Pixels Number Correctly Detected of Oil Spill	Pixels Number of Oil Spill in Interpretation Map	Pixels Number of Oil Spill Detected by the Classifier	Recall (%)	Precision (%)	F1 1
Methods
M D	original scale	41,890	53,822	66,752	77.83	62.75	0.6948
	first-level scale	42,646	53,822	65,397	79.24	65.21	0.7154
	second-level scale	41,263	53,822	68,561	76.67	60.18	0.6743
S V M	original scale	42,030	53,822	45,637	78.09	92.10	0.8452
	first-level scale	42,785	53,822	46,694	79.49	91.63	0.8513
	second-level scale	42,772	53,822	46,533	79.47	91.92	0.8524
C N N	original scale	45,872	53,822	54,252	85.23	84.55	0.8489
	first-level scale	46,223	53,822	52,260	85.88	88.45	0.8715
	second-level scale	45,208	53,822	51,707	84.00	87.43	0.8568
D B N	original scale	46,324	53,822	55,445	86.07	83.55	0.8479
	first-level scale	45,103	53,822	50,638	83.80	89.07	0.8635
	second-level scale	43,675	53,822	49,111	81.15	88.93	0.8486

¹ F1 = Precision ∗ Recall ∗ 2/(Precision + Recall).

shows that on the basis of different scale feature images, compared with the shallow learning method, two deep learning methods have higher detection accuracy, among which CNN algorithm has the highest detection accuracy, followed by the DBN algorithm, and MD has the lowest detection accuracy. The detection accuracy of CNN algorithm based on the first-level scale features is the highest, with F1 value of 0.8715. This proves that the deep-seated feature information extracted by deep learning model is more conducive to oil spill detection. At the same time, we can find that for MD, CNN, and DBN, detection accuracies based on the first-level scale features are the highest, with F1 values of 0.7154, 0.8715, and 0.8635, respectively. The detection accuracy of SVM algorithm increases with the increase of scale, and optimal detection accuracy is 0.8524. For SVM, CNN, and DBN, detection accuracies based on the first-level scale and the second-level scale features are better than that based on the original scale feature. The existence of strong solar flares will interfere the accurate detection of oil spill on the sea surface. The low-frequency components of different scales generated by wavelet transform can eliminate image noise and improve the accuracy of oil spill detection to a certain extent.

It can be seen from

Table 4. Oil spill detection accuracy at different scale for decision fusion based on fuzzy membership degree.

	EvaluationCriterion	Pixels Number Correctly Detected of Oil Spill	Pixels Number of Oil Spill in Interpretation Map	Pixels Number of Oil Spill Detected by the Classifier	Recall (%)	Precision (%)	F1 1
Methods
CNN-SVM	original scale	46,370	53,822	52,529	86.15	88.28	0.8720
	first-level scale	47,030	53,822	53,049	87.38	88.65	0.8801
	second-level scale	46,917	53,822	54,083	87.17	86.75	0.8696
DBN-SVM	original scale	45,949	53,822	52,111	85.37	88.18	0.8675
	first-level scale	46,193	53,822	51,904	85.83	89.00	0.8738
	second-level scale	45,470	53,822	51,573	84.48	88.17	0.8628
CNN-MD	original scale	49,304	53,822	69,164	91.61	71.29	0.8018
	first-level scale	49,331	53,822	67,863	91.66	72.69	0.8108
	second-level scale	48,817	53,822	72,353	90.70	67.47	0.7738
DBN-MD	original scale	48,588	53,822	67,100	90.28	72.41	0.8036
	first-level scale	48,968	53,822	66,297	90.98	73.86	0.8153
	second-level scale	48,482	53,822	71,330	90.08	67.97	0.7748

¹ F1 = Precision ∗ Recall ∗ 2/(Precision + Recall).

that under the same scale feature, detection accuracies of decision fusion results of deep learning and shallow learning methods are better than those of single classifiers. For example, the F1 value of CNN and SVM decision fusion results based on original scale features is 0.8720, while the oil spill detection accuracy of CNN and SVM is 0.8489 and 0.8452, respectively; the F1 value of decision fusion result of CNN and SVM based on the first-level scale feature is 0 8801, while the accuracy of CNN and SVM based on the first-level scale feature is 0.8715 and 0.8513, respectively. Oil spill detection accuracies for decision fusion of SVM and two deep learning methods based on different scale features are higher than those of MD and two deep learning methods. Decision fusion results of CNN and SVM (CNN-SVM) have the highest detection accuracy, decision fusion results of DBN and SVM (DBN-SVM) take the second place, and decision fusion results of CNN and MD (CNN-MD) has the lowest detection accuracy, which shows that two single classifiers with better oil spill detection accuracy will have higher detection accuracy of decision fusion. At the same time, we can find that in the decision fusion results of deep learning and shallow learning methods, detection accuracies based on the first-level scale feature are the highest, followed by the original scale feature, and detection accuracies based on the second scale feature are the lowest, which is related to the better oil spill detection effect of single classifiers under the first-level scale feature. The decision fusion algorithm uses the complementarity of single deep learning model and shallow learning classifier, and uses the fusion strategy to give full play to the advantages of different classifiers, and further improves the accuracy of oil spill detection on the sea surface.

5.2. AVIRIS Hyperspectral Application of the Proposed Method

In order to verify the effectiveness and applicability of the proposed decision fusion method, in this section, we apply the algorithm to the 2010 AVIRIS oil spill hyperspectral data of the Gulf of Mexico. Figure 9a shows the location of the oil spill image. The AVIRIS image (Figure 9b) has 224 bands, with spectral resolution of 10 nm and spatial resolution of 0.89 m@1 km. The spectral range is 350–2500 nm, covering visible, near-infrared and shortwave infrared spectra. The field of view of the sensor is 34°. The imaging time of the AVIRIS oil spill hyperspectral imagery is 18 May 2010, and the scene size is 400 × 400 (Figure 9c). The AVIRIS image input into the model is radiance data without preprocessing steps such as atmospheric correction, band selection or dimensionality reduction.

There are three types of ground truth samples in this image. The number of training samples and test samples for each class is shown in

Table 5. The number of three types of training samples and test samples.

No.	Feature Types	Pixels Number of Training Samples	Pixels Number of Test Samples
1	oil slick	1008	54,598
2	seawater	1282	103,598
3	cloud	246	1804

.

We carry out oil spill detection experiment by our proposed decision fusion algorithm. Oil spill detection results of MD, SVM, DBN, and CNN based on different scale features using the same training samples are shown in Figure 10, and their decision fusion results at different scales are shown in the Figure 11. The F1 score is used as an indicator for accuracy evaluation. The accuracies for oil spill detection of four methods based on different scale features are listed in

Table 6. Oil spill detection accuracy at different scale for four methods.

	Evaluation Criterion	Pixels Number Correctly Detected of Oil Spill	Pixels Number of Oil Spill in Interpretation Map	Pixels Number of Oil Spill Detected by the Classifier	Recall (%)	Precision (%)	F1
Methods
M D	original scale	34,439	54,598	36,755	63.08	93.70	0.7540
	first-level scale	36,638	54,598	39,222	67.11	93.41	0.7810
	second-level scale	37,702	54,598	41,744	69.05	90.32	0.7827
S V M	original scale	44,102	54,598	46,732	80.78	94.37	0.8705
	first-level scale	44,001	54,598	46,493	80.59	94.64	0.8706
	second-level scale	43,058	54,598	45,579	78.86	94.47	0.8596
C N N	original scale	47,247	54,598	52,086	86.54	90.71	0.8857
	first-level scale	47,242	54,598	51,520	86.53	91.70	0.8904
	second-level scale	47,361	54,598	52,699	86.74	89.87	0.8828
D B N	original scale	45,725	54,598	49,873	83.75	91.68	0.8754
	first-level scale	47,498	54,598	52,260	87.00	90.89	0.8890
	second-level scale	45,168	54,598	48,086	82.73	93.93	0.8797

, and the detection accuracies of their decision fusion results are listed in

Table 7. Oil spill detection accuracy at different scale for decision fusion algorithm based on fuzzy membership degree.

	Evaluation Criterion	Pixels Number Correctly Detected of Oil Spill	Pixels Number of oil Spill in Interpretation Map	Pixels Number of Oil Spill Detected by the Classifier	Recall (%)	Precision (%)	F1
Methods
CNN-SVM	original scale	47,209	54,598	51,315	86.47	92.00	0.8915
	first-level scale	47,093	54,598	50,912	86.25	92.50	0.8927
	second-level scale	47,323	54,598	52,428	86.68	90.26	0.8843
DBN-SVM	original scale	46,245	54,598	49,772	84.70	92.91	0.8862
	first-level scale	47,346	54,598	52,031	86.72	91.00	0.8881
	second-level scale	45,650	54,598	48,612	83.61	93.91	0.8846
CNN-MD	original scale	46,823	54,598	50,933	85.76	91.93	0.8874
	first-level scale	47,376	54,598	51,465	86.77	92.05	0.8934
	second-level scale	47,989	54,598	54,107	87.90	88.69	0.8829
DBN-MD	original scale	45,236	54,598	48,566	82.85	93.14	0.8770
	first-level scale	47,903	54,598	53,527	87.74	89.49	0.8861
	second-level scale	45,439	54,598	48,398	83.22	93.89	0.8823

.

Table 6 shows that CNN and DBN have higher detection accuracy than the shallow learning methods, among which CNN algorithm has the highest detection accuracy, followed by the DBN algorithm, and MD has the lowest detection accuracy. The detection accuracy of CNN algorithm based on the first-level scale features is the highest, with F1 value of 0.8904. At the same time, we can find that no matter which method is used, the detection accuracy based on the first-level scale feature image is better than that based on the original scale feature.

It can be seen from Table 7 that under the same scale feature, detection accuracies of decision fusion results of deep learning and shallow learning methods are better than those of single classifiers. For example, the F1 value of CNN and SVM decision fusion results based on the original scale features is 0.8915, while the oil spill detection accuracy of CNN and SVM is 0.8857 and 0.8705, respectively. At the same time, we can find that in the decision fusion results of deep learning and shallow learning methods, detection accuracies based on the first-level scale feature are the highest, followed by the original scale feature, and detection accuracies based on the second scale feature are the lowest.

Through the above analysis of oil spill AVIRIS detection experiments, we can draw the same conclusion with the previous ones using AISA+ data, which show that the developed decision fusion method has an applicability in different oil spill scenarios, and can detect the oil spill on the sea surface.

5.3. Satellite Hyperspectral Application of the Proposed Method

Up to now, hyperspectral remote sensing technology has been widely used in many fields, such as environmental monitoring, atmospheric exploration, earth resources survey and natural disasters monitoring. Marine oil spill is a kind of emergent incident, which requires the operational departments to respond quickly. Airborne hyperspectral remote sensing has the characteristics of flexibility, fast acquisition, high spatial and spectral resolution. Therefore, airborne hyperspectral sensors have an advantage in obtaining oil spill image in time. However, due to the limitation of weather conditions, it is difficult to acquire aerial data, especially during oil spill accidents. The advantages of spaceborne hyperspectral remote sensing are: (a) continuity, (b) consistency, and (c) global coverage. Although spaceborne hyperspectral remote sensing have some advantages, and many satellite hyperspectral sensors (such as EO-1 Hyperion, ISS HICO, and GF-5 AHSI, etc.,) are still in service, but they also face some challenges, such as cloud, low spatial resolution, narrow swath, long revisit period, and low signal-to-noise ratio. It is exciting that a new generation of hyperspectral satellites (PRISMA and EnMAP) may provide better data. The main parameters of several airborne and spaceborne hyperspectral imagers are listed in

Table 8. Main parameters of eleven hyperspectral sensors.

Sensor		Spectral Ange (nm)	Number of Bands	Spectral Resolution (nm)	Spatial Resolution (m)	Swath (km)	Platform
Airborne	AVIRIS	350~2500	224	10	0.89 m@1 km	-	-
	AISA	400~1000	258	5	1.41 m@1 km	-	-
	CASI	380~1050	288	3.5	1.42 m@1 km	-	-
	HyMap	400~2500	128	VNIR:15 SWIR:20	2.25 m@1 km	-	-
Spaceborne	Hyperion	400~2500	242	10	30	7.7	EO-1
	CHRIS	400~1050	18/62	5~12	17/34	14	PROBA
	AHSI	400~2500	330	VNIR:5 SWIR:10	30	60	GF-5
	HSI	450~950	220	5	100	50	HJ-1A
	HICO	360~1080	128	5.7	90	42	ISS
	PRISMA	400~2500	250	<12	30	30	PRISMA
	HIS	420~2450	262	VNIR:6.5 SWIR:10	30	30	EnMAP

.

In order to verify the portability of this method on hyperspectral satellite data, we apply the developed decision fusion algorithm to the oil spill hyperspectral data of Liaodong Bay in 2007, which is obtained by EO-1 Hyperion (Figure 12). The Hyperion hyperspectral image has 242 bands in total, of which 198 bands are radiometric calibrated, while bands 1–7, 58–76, and 225–242 are 0, which must be removed. At the same time, due to the influence of signal-to-noise ratio and water vapor, 19 bands also need to be eliminated. In the experiment, Hyperion image containing 179 bands are used. The spectral range is 350–2500 nm, spectral resolution of 10 nm, and spatial resolution of 30 m. However, the image contains a lot of stripe noise and bad lines, which seriously affect the oil spill detection. It is indicated from Figure 13, the oil spill detection effects of the four algorithms are poor due to the stripe noise and bad lines of the image. However, this does not mean that our method cannot be applied to hyperspectral satellite data of oil spill. We plan to apply this method to other hyperspectral satellite data with high imaging quality to prove the feasibility of this method on satellite records.

5.4. Comparison with Other Algorithms

To further evaluate the effect of the algorithm proposed in this study, four mainstream algorithms such as SVM, DBN [39], 1D-CNN [38], and MRF-CNN [54] were chosen for comparative analysis and evaluation. The MRF-CNN (Markov Random Field-Convolutional Neural Networks) regional fusion decision strategy exploited the complementary characteristics of the two classifiers, which can overcome the problem of losing effective resolution and uncertain prediction at object boundaries, which is especially pertinent for complex fine spatial resolution image.

During the experiment, the parameters of the four comparison algorithms were set as the default values corresponding to the parameter settings in the study. Oil spill detection results for each algorithm based on AISA+ and AVIRIS hyperspectral images are shown in Figure 14 and Figure 15, respectively. Compared with other four algorithms, the oil film in the experimental results obtained by the algorithm proposed in this paper is more continuous, and there are fewer broken patches, and the detection effect of the oil film is improved to varying degrees. Oil spill detection accuracy of each algorithm is shown in

Table 9. The accuracy evaluation table of various oil spill detection algorithms.

	Methods	SVM	DBN	1D-CNN	MRF-CNN	The Proposed Algorithm
Evaluation Criterion
AISA+ hyperspectral image	F1	0.8524	0.8635	0.8715	0.8672	0.8801
	OA(%)	89.90	89.78	90.61	91.5	91.93
AVIRIS hyperspectral image	F1	0.8706	0.8890	0.8904	0.8668	0.8927
	OA(%)	91.51	92.23	92.47	90.76	92.86

. The overall classification accuracy (OA) and F1 score of the algorithm proposed in this study were higher than those of the SVM, DBN, 1D-CNN, and MRF-CNN algorithms. The improvement of F1 score was 0.0277, 0.0166, 0.0086, and 0.0129, respectively. The overall classification accuracy of the proposed method is 2.03%, 2.15%, 1.32%, and 0.43% higher than that of the other four algorithms, respectively.

5.5. Other Considerations

Whether for single-target detection or multi-target classification, the decision fusion algorithm based on fuzzy membership degree integrates the advantages of multiple single classification algorithms from the perspective of target recognition information fusion, and the classification accuracy of fusion results is improved, which shows that the algorithm is practical and effective. On the premise of existing single classifier resources, it is an important way to improve the classification accuracy of hyperspectral images. When an oil spill accident in marine occurs, remote sensing is an important means to detect the oil spill on the sea surface in a large scale, especially the aerial optical remote sensing, which can provide oil spill information timely and effectively. However, due to the limitation of observation geometry and the influence of sea waves, sun glints will inevitably appear in airborne images, which will interfere the oil slick detection. Although the discrete wavelet analysis method can produce different scale features by decomposing the signal, and eliminate part of the noise, it cannot completely suppress the influence of sun glints. This experiment is an attempt to use decision fusion algorithm of deep learning models and shallow learning methods based on different scale feature for oil spill detection. In the near future, we plan to use other solar flare suppression methods combined with decision fusion to carry out experiments, and compare with other decision fusion methods to further highlight the advantages of the decision fusion algorithm.

6. Conclusions

Aiming at the oil spill event of Penglai 19-3 platform in 2011, based on the airborne AISA + hyperspectral image and the multi-scale features after wavelet transform, this paper uses two deep learning methods and two shallow learning methods to extract the oil spill information at three different scales. Based on oil spill detection results of single algorithms, the decision fusion algorithm based on fuzzy membership degree is used to fuse multi-source oil spill information under the same scale. The main conclusions are as follows: (1) oil spill detection accuracies of deep learning methods based on different scale features are higher than those of shallow learning methods with corresponding scale, which proves that the deep-seated feature information extracted by deep learning model is more suitable for oil spill detection on sea surface. (2) At the same scale, the decision fusion algorithm based on fuzzy membership has better oil spill detection performance than those of single classifiers. For instance, oil spill detection accuracy (F1 value) using decision fusion algorithm based on the original scale feature is 0.8720, which is improved by 0.025 on average than that of the state-of-art single algorithms. (3) For single detection algorithms, oil spill detection accuracies based on the first-level scale feature and the second-level scale feature are better than those based on the original scale feature. For decision fusion results of deep learning and shallow learning methods, oil spill detection accuracies based on the first-level scale feature are the highest, followed by the original scale, and detection accuracies based on the second-level scale feature are the lowest. (4) The overall classification accuracy of the proposed method is 91.93%, which is 2.03%, 2.15%, 1.32%, and 0.43% higher than that of SVM, DBN, 1D-CNN, and MRF-CNN algorithms, respectively. The improvement of F1 score is 0.0277, 0.0166, 0.0086, and 0.0129, respectively.

The algorithm developed in this paper is an oil spill detection model based on the decision fusion of shallow learning and deep learning. The detection results of the developed model depend on the classification results of the basic classifiers to a certain extent, that is, the detection results of shallow learning algorithm and deep learning algorithm. Therefore, in a practical application, the selection of basic classifiers is particularly important. It is necessary to select basic classifiers with strong feature extraction ability in order to make the oil spill detection accuracy of decision fusion based on fuzzy membership better. Rapid and effective monitoring of the location and range of the oil spill is very important for rapid response. With the development of unmanned aerial vehicle (UAV) technology, it is a trend to apply oil spill detection algorithm to UAV system to realize real-time oil spill detection. At the same time, the coordination of multi-source remote sensing for marine oil spill detection is also the research direction in the future.