Next Article in Journal
Sheet-like Skeleton Carbon Derived from Shaddock Peels with Hierarchically Porous Structures for Ultra-Fast Removal of Methylene Blue
Previous Article in Journal
Assessment of Pollutants from Diffuse Pollution through the Correlation between Rainfall and Runoff Characteristics Using EMC and First Flush Analysis
Previous Article in Special Issue
Current Progress on Marine Microplastics Pollution Research: A Review on Pollution Occurrence, Detection, and Environmental Effects
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Cloud-Based Framework for Large-Scale Monitoring of Ocean Plastics Using Multi-Spectral Satellite Imagery and Generative Adversarial Network

1
Civil Engineering Department, Faculty of Engineering, University of Karabük, Karabük 78050, Turkey
2
Department of Electrical and Computer Engineering, Memorial University of Newfoundland, St. John’s, NL A1B 3X5, Canada
*
Author to whom correspondence should be addressed.
Water 2021, 13(18), 2553; https://doi.org/10.3390/w13182553
Received: 24 August 2021 / Revised: 15 September 2021 / Accepted: 16 September 2021 / Published: 17 September 2021
(This article belongs to the Special Issue Microplastics Pollution in Marine Environment)

Abstract

:
Marine debris is considered a threat to the inhabitants, as well as the marine environments. Accumulation of marine debris, besides climate change factors, including warming water, sea-level rise, and changes in oceans’ chemistry, are causing the potential collapse of the marine environment’s health. Due to the increase of marine debris, including plastics in coastlines, ocean and sea surfaces, and even in deep ocean layers, there is a need for developing new advanced technology for the detection of large-sized marine pollution (with sizes larger than 1 m) using state-of-the-art remote sensing and machine learning tools. Therefore, we developed a cloud-based framework for large-scale marine pollution detection with the integration of Sentinel-2 satellite imagery and advanced machine learning tools on the Sentinel Hub cloud application programming interface (API). Moreover, we evaluated the performance of two shallow machine learning algorithms of random forest (RF) and support vector machine (SVM), as well as the deep learning method of the generative adversarial network-random forest (GAN-RF) for the detection of ocean plastics in the pilot site of Mytilene Island, Greece. Based on the obtained results, the shallow algorithms of RF and SVM achieved an overall accuracy of 88% and 84%, respectively, with available training data of plastic debris. The GAN-RF classifier improved the detection of ocean plastics of the RF method by 8%, achieving an overall accuracy of 96% by generating several synthetic ocean plastic samples.

1. Introduction

Detection of anthropogenic marine debris (AMD) accumulated in coastlines, ocean surfaces, marshes, estuaries, and even in marine depths is among the most challenging environmental issues worldwide [1,2,3]. AMD is considered one of the current growing threats to the Earth’s environment’s health, particularly the oceans [4,5,6]. Marine litter can be defined as “any persistent, processed, manufactured solid material discarded, disposed, or abandoned in the coastal or marine environments” [7]. Plastics are about 50% to 90% of the total AMD, according to several studies on the composition of AMD globally [8,9]. It should be noted that the production of plastics raised from 5 to 311 million tons from 1960 to 2014 [10]. Marine debris, specifically the long-lasting plastics that are ubiquitous in the marine environment, has become a global threat [11,12,13]. Marine litter, plastics, in particular, can accumulate in high densities on the seafloor. As a result, it has harmful effects on the livelihood of marine inhabitants. Entanglement and ingestion, which leads to the injury and death of fishes, birds, marine mammals (i.e., seals), and turtles, is considered one of AMD’s negative impacts on the marine environments [14,15,16,17,18,19]. In addition, the marine debris moved to the ground surface (e.g., beaches), in particular the plastic fishing nets, can physically damage the reef and beach environments and continue to entangle and cause the death of inhabitant species located in reefs, ocean, and sea surfaces, as well as the beaches.
Moreover, non-organic and organic pollutants can be transferred by the AMD [20]. Additionally, AMDs are used as shelters for small animals [21]. Consequently, gas exchange between sediments and water columns can be reduced, displacing several benthonic species [21]. On the other hand, with the reduction of aesthetic value that leads to the decrease of economic activities of infrastructures in beaches and other public places, the AMD will negatively affect the livelihood of human beings [1]. Marine pollutants, such as buoys, derelict fishing nets, and ropes that become entangled in reefs have placed a significant threat on the inhabitants and marine species [22]. Although the long-term effects of marine pollutants on the ocean’s health are not apparent, it is believed that AMDs will play a significant role in the potential collapse of aquatic health, particularly in the oceans. The AMD, with other global major factors, including warming waters, rising sea levels, and changes in the chemistry of oceans, have added significant global stress on the livelihood of fauna and fungi globally. As a result, developing and proposing new tools to monitor and remove marine pollutants at sea and ocean, as they have become a danger to critical habitats, is of great importance at national and global levels. Similar to any environmental monitoring problem, it is vital to recognize the limitation and difficulties of detecting marine pollutants at sea/ocean surfaces. Moreover, understanding the behavior and characteristics of AMD is of great importance [22].
Beach surveys that are conducted mainly through volunteering groups are the most common method of marine debris detection [23], which is spatially limited for small-scale marine debris monitoring. The other possible technique that is a low-cost and efficient method of AMD monitoring is the use of webcams [24,25,26], mostly used for specific beaches and remote area marine pollution monitoring. It is worth highlighting that although AMD detection/monitoring with the utilization of webcams is an efficient and low-cost technique, it cannot be used as a large-scale AMD monitoring technique. Light detection and ranging (LiDAR) is another technology proposed for marine debris monitoring [27]. Due to the high cost of data acquisition and processing of LiDAR data, it cannot be used as a large-scale AMD monitoring. Aerial survey with the utilization of fixed-wing planes is another potential tool for the detection/monitoring of abundant AMD accumulated in coastlines and beaches [28]. The utilization of aerial surveys, which require several ground control points, is valuable and efficient for large-scale AMD monitoring. However, it requires a higher cost than conventional volunteering beach surveys and automated webcam beach monitoring. It should be noted the utilization of aerial photographic surveys with high-resolution digital cameras has played a significant role in the efficient large-scale AMD monitoring/detection in recent years [29,30,31].
It is worth highlighting that abundant marine pollutions, specifically plastics, have two possible fates when entering the ocean—floating on the surface of the sea or sinking [32]. In addition, inevitably, macroplastics will degrade into microplastics. Therefore, detection of large marine pollutions would be significantly useful for the protection of marine environments. Several studies conducted on the detection of macroplastics (>5 mm) have shown the possibilities of the use of optical satellite data [33,34,35,36,37,38]. Currently, remote sensing is regarded as the leading technology for the collection of high-quality images of the Earth’s environment. Factors such as limited and coarse spatial, temporal, and spectral satellite data were the reason for the few previous studies conducted on the utilization of satellite data for the detection/monitoring of AMD. For example, although satellites, such as RapidEye and SkySat, have high spatial resolution images, they provide few spectral bands. The launch of Sentinel-2A and 2B by the European Space Agency (ESA) in 2015 and 2017, respectively, substantially improved the spectral and spatial resolution of the available Earth observation data that can be efficiently used for the detection of marine pollutants, specifically the large-size plastics [39,40,41,42].
There has been no research on large-scale mapping/monitoring of marine pollution using advanced technologies, such as state-of-the-art deep learning models and cloud API such as Sentinel Hub. Therefore, we proposed the use of the generative adversarial network (GAN) model on the Sentinel dataset for the large-scale detection and monitoring of marine pollution at sea/ocean surfaces. Our main objective was to assess the potential utilization of the leading technologies, such as Sentinel Hub and Google Colaboratory, to solve the large-scale detection of large-size marine pollutants faced by the current stressed global marine environment. Moreover, we evaluated the performance of two well-known shallow algorithms, namely random forest (RF) and support vector machine (SVM), as well as the deep learning method of the generative adversarial network–random forest (GAN-RF) for the detection of marine pollution (i.e., plastics). It should also be noted that the study was carried out on material floating on the surface of the sea and not on submerged material.

2. Methods

2.1. Study Area and Satellite Data

The study area is located in Mytilene, Greece (Figure 1). Mytilene was built on the southeast of the capital of the Greek Lesbos Island, which is also the capital and administrative center of the region of North Aegean. The land in Mytilene Island is approximately equal to 107.46 km2, with a population of 36,196 inhabitants. The average maximum temperature in Mytilene Island is around 30.4 Celsius in July, and the average minimum temperature reaches about 6.7 Celsius in January. It is worth highlighting that Greece has been a subject of extensive research for marine pollution detection, as there have been numerous reports on accumulated pollutions on its various beaches [43,44,45,46].
Training data used in this study are part of the PLP2019 dataset [47]. The data were created for a project to evaluate the possibilities of the utilization of Sentinel-2 optical images for AMD detection (Figure 2) [47]. As the data were collected and prepared to assess the use of optical Sentinel-2 images for the detection of large-size marine debris, 5 Sentinel-2 satellite images can be used and evaluated for the AMD monitoring/detection (Table 1) in the pilot site of Mytilene Island. On 18 April 2019, data were created as 45 m by 5 m plastic objects made of plastic bottles (50 percent volume) and plastic bags (50 percent volume). The University of Aegean team positioned 45 m by 5 m plastic targets at a long distance from one another on 3 May 2019. On that day, in addition to the four primary objects, 21 m by 10 m targets were also utilized. One of the added objects was built out of plastic bottles, while the other was made of a mix of plastic bags and plastic mesh. On 7 June 2019, the team performed another test with 45 m by 5 m plastic objects that covered 12 Sentinel-2 grids (for more information, refer to Topouzelis [47]). In our study, we used the Sentinel-2 Level-2 A image collected on 18 April 2019 that is atmospherically corrected. The reason is that there is no proper atmospheric correction library for the Sentinel-2 images in the Google Colaboratory.
The number of training and testing data for the shallow classifiers of the RF, SVM, and the deep learning algorithm of GAN-RF is presented in Table 2. It is worth highlighting that as the GAN network generated several synthetic ocean plastic samples, its training data are higher than the RF and SVM classifiers. In other words, the shallow algorithms of the SVM and RF are trained with just actual data (i.e., ocean plastic), while the GAN-RF algorithm is trained by both real and synthetic ocean plastic samples.

2.2. Methods

2.2.1. Sentinel Hub and Google Colaboratory

For the large-scale marine pollution monitoring/detection, we propose the utilization of the integration of Sentinel Hub and Google Colaboratory. Sentinel Hub provides a python package (https://sentinelhub-py.readthedocs.io/en/latest/ (accessed on 14 September 2021)) that can be efficiently imported and implemented in Google Colaboratory (https://colab.research.google.com/notebooks/ (accessed on 14 September 2021)). This Python package provides tools and functionalities required for the downloading and processing of several satellite data, including the Sentinel-2 images. For implementing the machine learning techniques for the detection of marine pollution, the leading notebook of Google Colaboratory was used in this project.

2.2.2. Algorithms

For the detection of marine debris, we evaluated the performance of two well-known shallow algorithms of the SVM, RF, and the deep learning method of GAN-RF implemented in Google Colaboratory. RF is an ensemble tree-based classifier, wherein several decision trees (DTs) are used to reach an optimal global solution [48]. RF is computationally efficient and resistant to the noise in the sample data. In the RF classifier, each DT is built by random vectors sampled independently using the input data. Additionally, a voting procedure is used to find the most popular prediction among various DTs for a given vector data (Figure 3). For instance, majority voting or average values are used to integrate the results of several different DTs to reach an optimal global model [49,50].
Boosting and bagging are two popular ensemble learning algorithms. Boosting can be described as a sequence model where each classifier tries to increase the accuracy of its previous model/classifier in that sequence [51,52]. Bagging (bootstrap aggregation) decreases the error of integrated models while the variance is attempted to be reduced (Figure 4). The bagging technique is considered to be more robust against overfitting over the boosting methods [53]. It is worth noting that the RF classifier integrates the bagging and the random selection methods.
The classification/regression accuracy is improved using RF with the use of bootstrap aggregation [54]. Gini index [55] and information gain ratio criterion [56] are two popular attributes selection techniques. For the RF classifier, the Gini index is usually used. It is worth mentioning that the full-grown trees in the RF classifier are not pruned, which is an advantage compared to other DT algorithms [55].
The SVM is a kernel-based algorithm [57] that is highly used in remote sensing image classification due to its capabilities (e.g., good results with a low number of training data) [58]. The SVM is a non-parametric technique that is insensitive to the input data distribution that is considered one of its most significant advantages. The main goal of SVM is to solve a convex quadratic optimization issue to achieve a globally optimal solution [59,60]. This is noteworthy since we had to address the challenge of a limited number of training data for ocean plastics detection. Therefore, the SVM classifier would be a suitable shallow algorithm in this study.
Currently, deep learning methods, specifically the convolutional neural networks (CNNs), have shown great success in remote sensing image classification, as they learn from representation rather than an empirical feature [61,62,63,64]. Conventional methods, such as RF and SVM, have two separate components of feature extraction and classification [65]. First, the feature extraction procedure transforms the spectral, spatial, and temporal Earth observation (EO) data into feature vectors. Afterward, those extracted features are utilized for training the shallow classifiers (e.g., RF). Therefore, the accuracy of the conventional classifiers depends on the feature engineering part (feature extraction procedure), while internal features are automatically learned in the deep learning methods. As a result, the deep learning algorithms are highly efficient for remote sensing image classification. The disadvantage of the deep learning methods is their dependency on a large number of training samples. This issue can be solved by two approaches of transfer learning and generating new synthetic samples to increase the training data. The deep learning field was revolutionized by the introduction of the new architecture of GAN proposed by Goodfellow et al. [66]. GAN includes two networks of generator and discriminator, as seen in Figure 5.
Using a random noise vector, the generator network generates new synthetic samples, and the discriminator network differentiates between the synthetic samples from the real ones. In other words, the GAN network is trained while the generator creates more realistic samples to fool the discriminator. In contrast, the discriminator tries to recognize the synthetic samples from the real ones [67,68,69]. As we have the challenge of the low number of training data for marine pollution detection, such advanced machine learning technology can be efficiently used to improve marine debris detection. We will evaluate the possibility of using a GAN network to generate synthetic ocean plastics to be used by an RF classifier to improve the marine debris classification accuracy in the next section. To implement the GAN network, we used the PyTorch framework and Sklearn library to implement the RF and SVM classifiers. In our GAN model’s generator, discriminator, and classifier networks, we used a simple structure of six consecutive linear functions and LeakyRelu [70] (Equations (1) and (2)).
y = xA T + b
LeakyRelu ( x ) = { x ,     if   x 0 negative   slope ,     otherwise
We set the number of batch size, noise dimension, hidden layer size, and learning rate to 64, 50, 16, and 0.0005, respectively. Figure 6 presents the architecture of the developed GAN model. First, with the use of a GAN model, synthetic marine pollution samples are generated. Then, the synthetic and real data of plastic marine pollutants are combined and fed to an RF classifier.

2.2.3. Features

As suggested by several previous research for the marine pollution detection, we utilized Sentinel-2 bands of the blue, green, red, red-edge-2, near infrared ( NIR ), and short wave infrared-1 ( SWIR 1 ), as well as the normalized difference vegetation index ( NDVI ) and the floating debris index ( FDI ) (Equations (3)–(5)) [71].
NDVI = NIR Red NIR + Red
FDI = NIR NIR
NIR = RE 2 + ( SWIR 1 RE 2 ) × ( λ NIR λ Red λ SWIR 1 λ Red ) × 10
where λ NIR , λ Red , and λ SWIR 1 are the central wavelength of the bands NIR , Red , and SWIR 1 , respectively.

2.2.4. Accuracy Assessment

The marine pollution (i.e., ocean plastics) detection results were evaluated in terms of overall accuracy, precision, recall, and F1-score statistical indices (Equations (6)–(9)).
Accuracy = ( True   positive + True   negative ) Total   number   of   pixels × 100
Precision = True   positive ( True   positive + False   positive )
Recall = True   positive ( True   positive + False   negative )
F 1 score = 2 × Precision   × Recall Precision +   Recall
It is worth mentioning that true positive and true negative values are those pixels that are correctly predicted as positive (debris) and negative (non-debris) objects, respectively. The false-positive values are those non-debris objects that are incorrectly classified as debris pixels, while the false negative are those debris objects that are incorrectly classified as the non-debris pixels in the Sentinel-2 imagery.

3. Results

Maps of marine pollution detection based on the RF, SVM, and GAN-RF algorithms with the integration of GAN and RF models in Sentinel Hub are shown in Figure 7.
It should be noted that there were few available training data on ocean plastics for the study area of Mytilene Island, Greece. Based on the results, the algorithms of the RF and SVM achieved an overall accuracy of 88% and 84%, respectively (Table 3). For the ocean plastics detection, the RF classifier obtained precision, F-1 score, and recall values of 0.67, 0.73, and 0.80, respectively. The shallow algorithm of the SVM achieved the recall, F-1 score, and precision values of 0.62, 0.71, and 0.83, respectively, for the ocean debris detection. The GAN-RF classifier obtained the best performance, reaching an overall accuracy of 96%, outperforming both shallow algorithms of the SVM and RF. The GAN-RF classifier’s precision, recall, and F-1 score was about 0.83, 0.91, and 1, respectively. All machine learning algorithms had better performance for the detection of non-plastic pixels. The reason is that there were much higher training data for the non-plastic pixels compared to the marine debris pixels.
The GAN-RF classifier had better performance over the RF and SVM methods, as we generated several synthetic ocean plastic samples with the GAN network and fed the generated synthetic data and the real ocean plastics data to the RF algorithm for the ocean plastic detection.

4. Discussion

Based on our study, one of the most challenging parts of marine debris detection is the lack of a sufficient number of training data that can be used for marine pollution detection with the utilization of satellite images (i.e., Sentinel-2 imagery). Moreover, the spatial resolution of the Sentinel-2 image is equal to 10 m (with resampling methods). As a result, the Sentinel-2 data can be used for the detection/monitoring of large-size AMD (e.g., ocean plastics). It is worth highlighting that with the use of pan-sharpening algorithms, all Sentinel-2 bands (i.e., 20- and 60 m bands) can be resampled to 10 m spatial resolution. Although satellites, such as Planet and RapidEye, have a higher spatial resolution, their spectral bands are limited. The other challenge is the lack of a proper library for the atmospheric correction of the Sentinel-2 images in Python. It should be noted that based on the previous research, the sen2cor Python package is not useful for the atmospheric correction of marine environments. It is suggested to use the sen2cor python package for the urban areas [57], not for the research on the marine environment. It should be noted that atmospheric correction is important, specifically when we are working on time-series satellite data. Errors may arise due to the lack of a proper method of atmospheric correction while working on different temporal resolution satellite images.
It is worth mentioning that in our study, we used Sentinel-2 Level-2 A that is atmospherically corrected and radiometrically calibrated. The reason is that there were no packages for the atmospheric correction available in the Google Colaboratory. Moreover, we only focused on mapping marine debris floating on the surface of the sea. The possibility of the recognition of drowned plastics through optical satellite data can be investigated in future research. On the other hand, based on the achieved results, the GAN-RF with an accuracy of 96% model was superior to both well-known shallow algorithms of RF (88%) and SVM (84%); specifically, the GAN-RF improved the results of marine debris detection of the implemented RF by 16% and 11% in terms of precision and recall statistical indices (Table 3). Therefore, generated synthetic marine pollutions did not increase the error, contradictory they increased obtained accuracy. Results indicate the importance and usefulness of advanced deep learning models such as GAN to solve environmental issues, specifically in study locations with a limited number of ground truth data (e.g., plastic marine pollutions).

5. Conclusions

Marine pollution and climate change factors, such as the sea-level rise and warming water, have become a global threat to the marine environment’s health. There are two fates for the abundant marine pollution, specifically the plastics—floating on the surface of the ocean/sea or sinking. Inevitably, macroplastics will degrade into microplastics, as a result of which the detection of large-size plastics would be significantly valuable for the protection of the marine environment. Therefore, we developed a framework for the detection of marine pollution with the integration of GAN and RF models in Sentinel Hub based on the Sentinel-2 images. Sentinel-2 satellite provides 10-, 20-, and 60 m spatial resolution and 12 spectral bands that can be efficiently utilized to detect large-size marine debris.
To detect ocean plastics in the pilot site of Mytilene Island, Greece, we evaluated the performance of two shallow classifiers of SVM and RF and the deep learning technique of the GAN-RF. The shallow algorithms of RF and SVM achieved F-1 scores of 73% and 71%, respectively, for the detection of plastic marine debris. Both algorithms of the RF and SVM with F-1 scores of 92% and 89%, respectively, had better performance for detecting non-debris pixels than the marine debris detection. The lower F-1 scores for the detection of plastics can be explained by their lower number of training data available in Mytilene Island. By generating several synthetic debris and non-debris samples, the GAN-RF algorithm improved the F-1 score of the RF algorithm by 27% and 3% for the detection of ocean plastic debris and non-plastic debris, respectively. Based on our results, the available tools, and functionality provided by advanced machine learning models, the new technologies of Sentinel Hub and Google Colaboratory can be efficiently used for large-scale marine debris detection at a regional, national, and even global scale.

Author Contributions

Conceptualization, A.J. and M.M.; methodology, A.J. and M.M.; formal analysis: A.J.; writing—original draft preparation, A.J. and M.M.; writing—review and editing, A.J., M.M.; supervision, M.M. Both authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Acuña-Ruz, T.; Uribe, D.; Taylor, R.; Amézquita, L.; Guzmán, M.C.; Merrill, J.; Martínez, P.; Voisin, L.; Mattar, C. Anthropogenic Marine Debris over Beaches: Spectral Characterization for Remote Sensing Applications. Remote Sens. Environ. 2018, 217, 309–322. [Google Scholar] [CrossRef]
  2. Thompson, R.C.; Moore, C.J.; vom Saal, F.S.; Swan, S.H. Plastics, the Environment and Human Health: Current Consensus and Future Trends. Philos. Trans. R. Soc. B: Biol. Sci. 2009, 364, 2153–2166. [Google Scholar] [CrossRef]
  3. Woodall, L.C.; Sanchez-Vidal, A.; Canals, M.; Paterson, G.L.J.; Coppock, R.; Sleight, V.; Calafat, A.; Rogers, A.D.; Narayanaswamy, B.E.; Thompson, R.C. The Deep Sea Is a Major Sink for Microplastic Debris. R. Soc. Open Sci. 2014, 1, 140317. [Google Scholar] [CrossRef] [PubMed][Green Version]
  4. Goddijn-Murphy, L.; Peters, S.; van Sebille, E.; James, N.A.; Gibb, S. Concept for a Hyperspectral Remote Sensing Algorithm for Floating Marine Macro Plastics. Mar. Pollut. Bull. 2018, 126, 255–262. [Google Scholar] [CrossRef][Green Version]
  5. Cózar, A.; Echevarría, F.; González-Gordillo, J.I.; Irigoien, X.; Úbeda, B.; Hernández-León, S.; Palma, Á.T.; Navarro, S.; García-de-Lomas, J.; Ruiz, A.; et al. Plastic Debris in the Open Ocean. Proc. Natl. Acad. Sci. USA 2014, 111, 10239. [Google Scholar] [CrossRef] [PubMed][Green Version]
  6. Ostle, C.; Thompson, R.C.; Broughton, D.; Gregory, L.; Wootton, M.; Johns, D.G. The Rise in Ocean Plastics Evidenced from a 60-Year Time Series. Nat. Commun. 2019, 10, 1622. [Google Scholar] [CrossRef] [PubMed]
  7. UNEP (United Nations Environment Programme). Marine Litter: A Global Challenge 2009. Available online: file:///C:/Users/MDPI/AppData/Local/Temp/151908.pdf (accessed on 17 September 2021).
  8. Derraik, J.G.B. The Pollution of the Marine Environment by Plastic Debris: A Review. Mar. Pollut. Bull. 2002, 44, 842–852. [Google Scholar] [CrossRef]
  9. Pham, C.K.; Ramirez-Llodra, E.; Alt, C.H.S.; Amaro, T.; Bergmann, M.; Canals, M.; Company, J.B.; Davies, J.; Duineveld, G.; Galgani, F.; et al. Marine Litter Distribution and Density in European Seas, from the Shelves to Deep Basins. PLOS ONE 2014, 9, e95839. [Google Scholar] [CrossRef][Green Version]
  10. Europe, P. Plastics—The Facts 2016. In An Analysis of European Plastics Production, Demand and Waste Data; Plastics Europe: Brussels, Belgium, 2016. [Google Scholar]
  11. Pichel, W.G.; Veenstra, T.S.; Churnside, J.H.; Arabini, E.; Friedman, K.S.; Foley, D.G.; Brainard, R.E.; Kiefer, D.; Ogle, S.; Clemente-Colón, P.; et al. GhostNet Marine Debris Survey in the Gulf of Alaska – Satellite Guidance and Aircraft Observations. Mar. Pollut. Bull. 2012, 65, 28–41. [Google Scholar] [CrossRef]
  12. Barnes, D.K.A.; Walters, A.; Gonçalves, L. Macroplastics at Sea around Antarctica. Mar. Environ. Res. 2010, 70, 250–252. [Google Scholar] [CrossRef] [PubMed]
  13. Morishige, C.; Donohue, M.J.; Flint, E.; Swenson, C.; Woolaway, C. Factors Affecting Marine Debris Deposition at French Frigate Shoals, Northwestern Hawaiian Islands Marine National Monument, 1990–2006. Mar. Pollut. Bull. 2007, 54, 1162–1169. [Google Scholar] [CrossRef] [PubMed]
  14. Gall, S.C.; Thompson, R.C. The Impact of Debris on Marine Life. Mar. Pollut. Bull. 2015, 92, 170–179. [Google Scholar] [CrossRef] [PubMed]
  15. Hardesty, B.D.; Good, T.P.; Wilcox, C. Novel Methods, New Results and Science-Based Solutions to Tackle Marine Debris Impacts on Wildlife. Ocean. Coast. Manag. 2015, 115, 4–9. [Google Scholar] [CrossRef][Green Version]
  16. Newman, S.; Watkins, E.; Farmer, A.; Ten Brink, P.; Schweitzer, J.-P. The Economics of Marine Litter. In Marine Anthropogenic Litter; Bergmann, M., Gutow, L., Klages, M., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 367–394. ISBN 978-3-319-16510-3. [Google Scholar]
  17. Law, K.L. Plastics in the Marine Environment. Annu. Rev. Mar. Sci. 2017, 9, 205–229. [Google Scholar] [CrossRef][Green Version]
  18. Nelms, S.; Coombes, C.; Foster, L.; Galloway, T.; Godley, B.; Lindeque, P.; Witt, M. Marine Anthropogenic Litter on British Beaches: A 10-Year Nationwide Assessment Using Citizen Science Data. Sci. Total. Environ. 2017, 579, 1399–1409. [Google Scholar] [CrossRef] [PubMed][Green Version]
  19. Chubarenko, I.; Stepanova, N. Microplastics in Sea Coastal Zone: Lessons Learned from the Baltic Amber. Environ. Pollut. 2017, 224, 243–254. [Google Scholar] [CrossRef] [PubMed]
  20. Romera-Castillo, C.; Pinto, M.; Langer, T.M.; Álvarez-Salgado, X.A.; Herndl, G.J. Dissolved Organic Carbon Leaching from Plastics Stimulates Microbial Activity in the Ocean. Nat. Commun. 2018, 9, 1430. [Google Scholar] [CrossRef]
  21. Watters, D.L.; Yoklavich, M.M.; Love, M.S.; Schroeder, D.M. Assessing Marine Debris in Deep Seafloor Habitats off California. Mar. Pollut. Bull. 2010, 60, 131–138. [Google Scholar] [CrossRef]
  22. Mace, T.H. At-Sea Detection of Marine Debris: Overview of Technologies, Processes, Issues, and Options. Mar. Pollut. Bull. 2012, 65, 23–27. [Google Scholar] [CrossRef]
  23. Opfer, S.; Arthur, C.; Lippiatt, S. NOAA Marine Debris Shoreline Survey Field Guide; NOAA Marine Debris Program: Silver Spring, MD, USA, 2012; p. 14. [Google Scholar]
  24. Kataoka, T.; Hinata, H.; Kato, S. Analysis of a Beach as a Time-Invariant Linear Input/Output System of Marine Litter. Mar. Pollut. Bull. 2013, 77, 266–273. [Google Scholar] [CrossRef]
  25. Kataoka, T.; Hinata, H.; Kato, S. Backwash Process of Marine Macroplastics from a Beach by Nearshore Currents around a Submerged Breakwater. Mar. Pollut. Bull. 2015, 101, 539–548. [Google Scholar] [CrossRef] [PubMed]
  26. Kako, S.; Isobe, A.; Kataoka, T.; Yufu, K.; Sugizono, S.; Plybon, C.; Murphy, T.A. Sequential Webcam Monitoring and Modeling of Marine Debris Abundance. Mar. Pollut. Bull. 2018, 132, 33–43. [Google Scholar] [CrossRef] [PubMed]
  27. Ge, Z.; Shi, H.; Mei, X.; Dai, Z.; Li, D. Semi-Automatic Recognition of Marine Debris on Beaches. Sci. Rep. 2016, 6, 25759. [Google Scholar] [CrossRef] [PubMed][Green Version]
  28. Veenstra, T.S.; Churnside, J.H. Airborne Sensors for Detecting Large Marine Debris at Sea. Mar. Pollut. Bull. 2012, 65, 63–68. [Google Scholar] [CrossRef]
  29. Kataoka, T.; Murray, C.C.; Isobe, A. Quantification of Marine Macro-Debris Abundance around Vancouver Island, Canada, Based on Archived Aerial Photographs Processed by Projective Transformation. Mar. Pollut. Bull. 2018, 132, 44–51. [Google Scholar] [CrossRef]
  30. Fallati, L.; Polidori, A.; Salvatore, C.; Saponari, L.; Savini, A.; Galli, P. Anthropogenic Marine Debris Assessment with Unmanned Aerial Vehicle Imagery and Deep Learning: A Case Study along the Beaches of the Republic of Maldives. Sci. Total. Environ. 2019, 693, 133581. [Google Scholar] [CrossRef] [PubMed]
  31. Garcia-Garin, O.; Monleón-Getino, T.; López-Brosa, P.; Borrell, A.; Aguilar, A.; Borja-Robalino, R.; Cardona, L.; Vighi, M. Automatic Detection and Quantification of Floating Marine Macro-Litter in Aerial Images: Introducing a Novel Deep Learning Approach Connected to a Web Application in R. Environ. Pollut. 2021, 273, 116490. [Google Scholar] [CrossRef]
  32. Lebreton, L.; Egger, M.; Slat, B. A Global Mass Budget for Positively Buoyant Macroplastic Debris in the Ocean. Sci. Rep. 2019, 9, 12922. [Google Scholar] [CrossRef] [PubMed][Green Version]
  33. LI, W.C.; TSE, H.F.; FOK, L. Plastic Waste in the Marine Environment: A Review of Sources, Occurrence and Effects. Sci. Total. Environ. 2016, 566–567, 333–349. [Google Scholar] [CrossRef]
  34. Carlson, D.F.; Özgökmen, T.; Novelli, G.; Guigand, C.; Chang, H.; Fox-Kemper, B.; Mensa, J.; Mehta, S.; Fredj, E.; Huntley, H.; et al. Surface Ocean Dispersion Observations From the Ship-Tethered Aerostat Remote Sensing System. Front. Mar. Sci. 2018, 5, 479. [Google Scholar] [CrossRef]
  35. Hu, C. Remote Detection of Marine Debris Using Satellite Observations in the Visible and near Infrared Spectral Range: Challenges and Potentials. Remote Sens. Environ. 2021, 259, 112414. [Google Scholar] [CrossRef]
  36. Kikaki, A.; Karantzalos, K.; Power, C.A.; Raitsos, D.E. Remotely Sensing the Source and Transport of Marine Plastic Debris in Bay Islands of Honduras (Caribbean Sea). Remote Sens. 2020, 12, 1727. [Google Scholar] [CrossRef]
  37. Kremezi, M.; Kristollari, V.; Karathanassi, V.; Topouzelis, K.; Kolokoussis, P.; Taggio, N.; Aiello, A.; Ceriola, G.; Barbone, E.; Corradi, P. Pansharpening PRISMA Data for Marine Plastic Litter Detection Using Plastic Indexes. IEEE Access 2021, 9, 61955–61971. [Google Scholar] [CrossRef]
  38. Balsi, M.; Esposito, S.; Moroni, M. Moroni Hyperspectral Characterization of Marine Plastic Litters. In Proceedings of the 2018 IEEE International Workshop on Metrology for the Sea; Learning to Measure Sea Health Parameters (MetroSea), Bari, Italy, 8–10 October 2018; pp. 28–32. [Google Scholar]
  39. Biermann, L.; Clewley, D.; Martinez-Vicente, V.; Topouzelis, K. Finding Plastic Patches in Coastal Waters Using Optical Satellite Data. Sci. Rep. 2020, 10, 5364. [Google Scholar] [CrossRef][Green Version]
  40. Themistocleous, K.; Papoutsa, C.; Michaelides, S.; Hadjimitsis, D. Investigating Detection of Floating Plastic Litter from Space Using Sentinel-2 Imagery. Remote Sens. 2020, 12, 2648. [Google Scholar] [CrossRef]
  41. Topouzelis, K.; Papageorgiou, D.; Karagaitanakis, A.; Papakonstantinou, A.; Ballesteros, M.A. Ballesteros Plastic Litter Project 2019: Exploring the Detection of Floating Plastic Litter Using Drones and Sentinel 2 Satellite Images. In Proceedings of the IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA, 26 Septemper–2 October 2020; pp. 6329–6332. [Google Scholar]
  42. Topouzelis, K.; Papageorgiou, D.; Karagaitanakis, A.; Papakonstantinou, A.; Arias Ballesteros, M. Remote Sensing of Sea Surface Artificial Floating Plastic Targets with Sentinel-2 and Unmanned Aerial Systems (Plastic Litter Project 2019). Remote Sens. 2020, 12, 2013. [Google Scholar] [CrossRef]
  43. Angelidis, M.O.; Aloupi, M. Geochemical Study of Coastal Sediments Influenced by River-Transported Pollution: Southern Evoikos Gulf, Greece. Mar. Pollut. Bull. 2000, 40, 77–82. [Google Scholar] [CrossRef]
  44. Stefatos, A.; Charalampakis, M.; Papatheodorou, G.; Ferentinos, G. Marine Debris on the Seafloor of the Mediterranean Sea: Examples from Two Enclosed Gulfs in Western Greece. Mar. Pollut. Bull. 1999, 38, 389–393. [Google Scholar] [CrossRef]
  45. Kapsalis, K.; Kavvalou, M.; Damikouka, I.; Cavoura, O. Investigation of Petroleum Hydrocarbon Pollution along the Coastline of South Attica, Greece, after the Sinking of the Agia Zoni ΙΙ Oil Tanker. SN Appl. Sci. 2021, 3, 48. [Google Scholar] [CrossRef]
  46. Charitou, A.; Naasan Aga-Spyridopoulou, R.; Mylona, Z.; Beck, R.; McLellan, F.; Addamo, A.M. Investigating the Knowledge and Attitude of the Greek Public towards Marine Plastic Pollution and the EU Single-Use Plastics Directive. Mar. Pollut. Bull. 2021, 166, 112182. [Google Scholar] [CrossRef]
  47. Topouzelis, K. PLP2019 Dataset. 2020. Available online: https://zenodo.org/record/3752719#.YURAO7gzZPZ (accessed on 14 September 2021).
  48. Breiman, L. Random Forests. Mach. Learn. 2001, 54, 5–32. [Google Scholar] [CrossRef][Green Version]
  49. Jamali, A.; Mahdianpari, M.; Brisco, B.; Granger, J.; Mohammadimanesh, F.; Salehi, B. Comparing Solo Versus Ensemble Convolutional Neural Networks for Wetland Classification Using Multi-Spectral Satellite Imagery. Remote Sens. 2021, 13, 2046. [Google Scholar] [CrossRef]
  50. DeLancey, E.R.; Kariyeva, J.; Cranston, J.; Brisco, B. Monitoring Hydro Temporal Variability in Alberta, Canada with Multi-Temporal Sentinel-1 SAR Data. Can. J. Remote Sens. 2018, 44, 1–10. [Google Scholar] [CrossRef]
  51. Mahdianpari, M.; Salehi, B.; Mohammadimanesh, F.; Motagh, M. Random Forest Wetland Classification Using ALOS-2 L-Band, RADARSAT-2 C-Band, and TerraSAR-X Imagery. ISPRS J. Photogramm. Remote Sens. 2017, 130, 13–31. [Google Scholar] [CrossRef]
  52. Sheykhmousa, M.; Mahdianpari, M.; Ghanbari, H.; Mohammadimanesh, F.; Ghamisi, P.; Homayouni, S. Support Vector Machine vs. Random Forest for Remote Sensing Image Classification: A Meta-Analysis and Systematic Review. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 13, 6308–6325. [Google Scholar] [CrossRef]
  53. DeLancey, E.R.; Simms, J.F.; Mahdianpari, M.; Brisco, B.; Mahoney, C.; Kariyeva, J. Comparing Deep Learning and Shallow Learning for Large-Scale Wetland Classification in Alberta, Canada. Remote Sens. 2019, 12, 2. [Google Scholar] [CrossRef][Green Version]
  54. Izquierdo-Verdiguier, E.; Zurita-Milla, R. An Evaluation of Guided Regularized Random Forest for Classification and Regression Tasks in Remote Sensing. Int. J. Appl. Earth Obs. Geoinf. 2020, 88, 102051. [Google Scholar] [CrossRef]
  55. Quinlan, J.R. C4.5: Programs Mach. Learn; Morgan Kaufmann: San Mateo, CA, USA, 1993. [Google Scholar]
  56. Leo, B.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification and Regression Trees; Chapman and Hall/CRC: Boca Raton, FL, USA, 1984. [Google Scholar]
  57. Cortes, C.; Vapnik, V. Support-Vector Networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
  58. Foody, G.M.; Mathur, A. Toward Intelligent Training of Supervised Image Classifications: Directing Training Data Acquisition for SVM Classification. Remote Sens. Environ 2004, 93, 107–117. [Google Scholar] [CrossRef]
  59. Jamali, A.; Mahdianpari, M.; Brisco, B.; Granger, J.; Mohammadimanesh, F.; Salehi, B. Wetland Mapping Using Multi-Spectral Satellite Imagery and Deep Convolutional Neural Networks: A Case Study in Newfoundland and Labrador, Canada. Can. J. Remote Sens. 2021, 1–18. [Google Scholar] [CrossRef]
  60. Jamali, A. Improving Land Use Land Cover Mapping of a Neural Network with Three Optimizers of Multi-Verse Optimizer, Genetic Algorithm, and Derivative-Free Function. Egypt. J. Remote Sens. Space Sci. 2020. [Google Scholar] [CrossRef]
  61. Zhang, C.; Pan, X.; Li, H.; Gardiner, A.; Sargent, I.; Hare, J.; Atkinson, P.M. A Hybrid MLP-CNN Classifier for Very Fine Resolution Remotely Sensed Image Classification. ISPRS J. Photogramm. Remote Sens. 2018, 140, 133–144. [Google Scholar] [CrossRef][Green Version]
  62. Mahdianpari, M.; Salehi, B.; Rezaee, M.; Mohammadimanesh, F.; Zhang, Y. Very Deep Convolutional Neural Networks for Complex Land Cover Mapping Using Multispectral Remote Sensing Imagery. Remote Sens. 2018, 10, 1119. [Google Scholar] [CrossRef][Green Version]
  63. Rezaee, M.; Mahdianpari, M.; Zhang, Y.; Salehi, B. Deep Convolutional Neural Network for Complex Wetland Classification Using Optical Remote Sensing Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 3030–3039. [Google Scholar] [CrossRef]
  64. Mohammadimanesh, F.; Salehi, B.; Mahdianpari, M.; Gill, E.; Molinier, M. A New Fully Convolutional Neural Network for Semantic Segmentation of Polarimetric SAR Imagery in Complex Land Cover Ecosystem. ISPRS J. Photogramm. Remote Sens. 2019, 151, 223–236. [Google Scholar] [CrossRef]
  65. Jamali, A. Land Use Land Cover Mapping Using Advanced Machine Learning Classifiers: A Case Study of Shiraz City, Iran. Earth Sci. Inform. 2020, 13, 1015–1030. [Google Scholar] [CrossRef]
  66. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Nets. In Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; Volume 27, pp. 2672–2680. [Google Scholar]
  67. Roy, S.K.; Haut, J.M.; Paoletti, M.E.; Dubey, S.R.; Plaza, A. Generative Adversarial Minority Oversampling for Spectral-Spatial Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2021, 1–15. [Google Scholar] [CrossRef]
  68. Audebert, N.; Le Saux, B.; Lefèvre, S. Lefevre Generative Adversarial Networks for Realistic Synthesis of Hyperspectral Samples. In Proceedings of the IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 4359–4362. [Google Scholar]
  69. Zhu, L.; Chen, Y.; Ghamisi, P.; Benediktsson, J.A. Generative Adversarial Networks for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens. 2018, 56, 5046–5063. [Google Scholar] [CrossRef]
  70. Xu, J.; Li, Z.; Du, B.; Zhang, M.; Liu, J. Reluplex Made More Practical: Leaky ReLU. In Proceedings of the 2020 IEEE Symposium on Computers and Communications (ISCC), Rennes, France, 7–10 July 2020; pp. 1–7. [Google Scholar]
  71. Basu, B.; Sannigrahi, S.; Sarkar Basu, A.; Pilla, F. Development of Novel Classification Algorithms for Detection of Floating Plastic Debris in Coastal Waterbodies Using Multispectral Sentinel-2 Remote Sensing Imagery. Remote Sens. 2021, 13, 1598. [Google Scholar] [CrossRef]
Figure 1. Location of the study area of Mytilene Island, Greece.
Figure 1. Location of the study area of Mytilene Island, Greece.
Water 13 02553 g001
Figure 2. An example of the available training data of marine pollution for the study area of Mytilene Island, Greece (pixel sizes of A1 to A6 are 10 m by 10 m) [40].
Figure 2. An example of the available training data of marine pollution for the study area of Mytilene Island, Greece (pixel sizes of A1 to A6 are 10 m by 10 m) [40].
Water 13 02553 g002
Figure 3. An example structure of the random forest algorithm.
Figure 3. An example structure of the random forest algorithm.
Water 13 02553 g003
Figure 4. The boosting and bagging structures (red color dots present errors).
Figure 4. The boosting and bagging structures (red color dots present errors).
Water 13 02553 g004
Figure 5. Generator and discriminator networks of the generative adversarial network.
Figure 5. Generator and discriminator networks of the generative adversarial network.
Water 13 02553 g005
Figure 6. The architecture of the GAN model.
Figure 6. The architecture of the GAN model.
Water 13 02553 g006
Figure 7. Marine pollution mapping in Mytilene based on the RF (top right image), SVM (bottom left image), and GAN-RF (bottom right image) algorithms.
Figure 7. Marine pollution mapping in Mytilene based on the RF (top right image), SVM (bottom left image), and GAN-RF (bottom right image) algorithms.
Water 13 02553 g007
Table 1. Description of the Sentinel-2 level-1 C images that can be used for debris mapping.
Table 1. Description of the Sentinel-2 level-1 C images that can be used for debris mapping.
ImageDate of
Acquisition
S2B_MSIL1C_20190418T085559_N0207_R007_T35SMD_20190418T11044118 April 2019
S2A_MSIL1C_20190503T085601_N0207_R007_T35SMD_20190503T1032213 May 2019
S2B_MSIL1C_20190518T085609_N0207_R007_T35SMD_20190518T11303218 May 2019
S2B_MSIL1C_20190528T085609_N0207_R007_T35SMD_20190528T11544028 May 2019
S2B_MSIL1C_20190607T085609_N0207_R007_T35SMD_20190607T1103357 June 2019
Table 2. The number of training and testing data of plastic debris in the pilot site of Mytilene Island, Greece.
Table 2. The number of training and testing data of plastic debris in the pilot site of Mytilene Island, Greece.
AlgorithmTraining (Debris/Non-Debris)Test (Debris/Non-Debris)
RF15/416/19
SVM15/416/19
GANs29/866/19
Table 3. Performance/confusion matrix based on the validation set data using RF and SVM algorithms.
Table 3. Performance/confusion matrix based on the validation set data using RF and SVM algorithms.
Algorithm PlasticNon-PlasticOA (%)F1-Score (%)PrecisionRecall
RFPlastic42880.730.670.80
Non-plastic1180.920.950.90
SVMPlastic51840.710.830.62
Non-plastic3160.890.840.94
GAN-RFPlastic519610.830.91
Non-plastic0190.9510.97
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Jamali, A.; Mahdianpari, M. A Cloud-Based Framework for Large-Scale Monitoring of Ocean Plastics Using Multi-Spectral Satellite Imagery and Generative Adversarial Network. Water 2021, 13, 2553. https://doi.org/10.3390/w13182553

AMA Style

Jamali A, Mahdianpari M. A Cloud-Based Framework for Large-Scale Monitoring of Ocean Plastics Using Multi-Spectral Satellite Imagery and Generative Adversarial Network. Water. 2021; 13(18):2553. https://doi.org/10.3390/w13182553

Chicago/Turabian Style

Jamali, Ali, and Masoud Mahdianpari. 2021. "A Cloud-Based Framework for Large-Scale Monitoring of Ocean Plastics Using Multi-Spectral Satellite Imagery and Generative Adversarial Network" Water 13, no. 18: 2553. https://doi.org/10.3390/w13182553

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop