A Synergic Integration of AIS Data and SAR Imagery to Monitor Fisheries and Detect Suspicious Activities †
Abstract
:1. Introduction
2. Materials and Method
2.1. AIS Data and Processing
2.1.1. AIS Data Pre-Processing
2.1.2. Vessel Classification
2.2. SAR Images and Processing
2.2.1. SAR Images Download
2.2.2. Detection of Ships and Platforms
2.3. Supplementary Information
2.4. AIS-SAR Matching Algorithm
- Case #1 Point-to-point: the SAR target is within a buffer area centered on AIS data. Retrieving all the AIS pings temporally closest to the SAR image, buffer sizes are set proportional to the distance d each vessel can travel in the time interval from the AIS transmission to the image acquisition. Taking into account that vessels can increase their speed during , the buffer radius is increased by 20%:In case of SAR targets falling within multiple buffers, the choice was made based on the hypothetical positions that vessel could reach at SAR epoch and the target was associated with the nearest buffer edge. This occurs in areas of high traffic density where vessels are close to each other.
- Case #2 Point-to-line: A SAR target is detected but no AIS reports are available in the vicinity. This could be caused by a transmission problem or a voluntary switching off.An attempt was done to associate the SAR target to the nearest blackout, by creating a variable size buffer whose radius r increases moving away from the SAR epoch.The buffer was generated dynamically by moving a pointer along the blackout line (i) according to N regular intervals di that were defined as:Again, in case of SAR targets falling within multiple buffers, the closest buffer is selected for the Point-to-Line association.
- Case #3 No match: The SAR target is unmatched because the AIS data is not available. It is due to a remaining false alarm, a blackout longer than 2 h or a vessel that is not adopting AIS.
- Case #4 No match: A SAR target is missed even if AIS data is available, due to the failure of the vessel detection. This may occur for several reasons, including small boats that were not detected by SUMO or bugs/shortfalls of the algorithm itself.
Algorithm 1: AIS-SAR Matching Algorithm. |
|
|
3. Results
4. Discussion and Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Roney, J.B. [Mis-]managing Fisheries on the West Coast of Ireland in the Nineteenth Century. Humanities 2019, 8, 4. [Google Scholar] [CrossRef] [Green Version]
- Sims, D.; Southward, A. Dwindling fish numbers already of concern in 1883. Nature 2006, 439, 660. [Google Scholar] [CrossRef] [PubMed]
- FAO. Review of the state of world marine fishery resources. In Global Overview of Marine Fishery Resources; FAO Fisheries Division: Rome, Italy, 2011. [Google Scholar]
- FAO. The State of Mediterranean and Black Sea Fisheries 2020. In General Fisheries Commission for the Mediterranean; FAO: Rome, Italy, 2020. [Google Scholar] [CrossRef]
- United Nation. Transforming Our World: The 2030 Agenda for Sustainable Development; Resolution Adopted by the General Assembly; Technical Report; United Nation: New York, NY, USA, 2015. [Google Scholar]
- Perretti, C.T.; Deroba, J.J.; Legault, C.M. Simulation testing methods for estimating misreported catch in a state-space stock assessment model. ICES J. Mar. Sci. 2020, 77, 911–920. [Google Scholar] [CrossRef]
- Hilborn, R. The state of the art in stock assessment: Where we are and where we are going. Sci. Mar. 2003, 67, 15–20. [Google Scholar] [CrossRef]
- Guldbrandsen Frøysa, K.; Bogstad, B.; Skagen, D.W. Fleksibest-an age-length structured fish stock assessment model. Fish. Res. 2002, 55, 87–101. [Google Scholar] [CrossRef]
- Kanjir, U.; Greidanus, H.; Oštir, K. Vessel detection and classification from spaceborne optical images: A literature survey. Remote Sens. Environ. 2018, 207, 1–26. [Google Scholar] [CrossRef]
- Back, M.; Kim, D.; Kim, S.; Won, J. Two-Dimensional Ship Velocity Estimation Based on KOMPSAT-5 Synthetic Aperture Radar Data. Remote Sens. 2019, 11, 1474. [Google Scholar] [CrossRef] [Green Version]
- Kourti, N.; Shepherd, I.; Greidanus, H.; Alvarez, M.; Aresu, E.; Bauna, T.; Chesworth, J.; Lemoine, G.; Schwartz, G. Integrating remote sensing in fisheries control. Fish. Manag. Ecol. 2005, 12, 295–307. [Google Scholar] [CrossRef]
- Dechesne, C.; Lefèvre, S.; Vadaine, R.; Hajduch, G.; Fablet, R. Ship identification and characterization in Sentinel-1 SAR images with multi-task deep learning. Remote Sens. 2019, 11, 2997. [Google Scholar] [CrossRef] [Green Version]
- Duan, J.; Wu, Y.; Luo, J. Ship Classification Methods for Sentinel-1 SAR Images. In International Conference in Communications, Signal Processing, and Systems; Lecture Notes in Electrical Engineering; Springer: Singapore, 2020; Volume 571, pp. 2259–2269. [Google Scholar] [CrossRef]
- Greidanus, H. Satellite Imaging for Maritime Surveillance of the European Seas. In Remote Sensing of the European Seas; Barale, V., Gade, M., Eds.; Springer: Dordrecht, The Netherlands, 2008; pp. 343–358. [Google Scholar] [CrossRef]
- Galdelli, A.; Mancini, A.; Tassetti, A.; Ferrà Vega, C.; Armelloni, E.; Scarcella, G.; Fabi, G.; Zingaretti, P. A cloud computing architecture to map trawling activities using positioning data. In Proceedings of the ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Anaheim, CA, USA, 18–21 August 2019; Volume 9. [Google Scholar] [CrossRef]
- Kroodsma, D.A.; Mayorga, J.; Hochberg, T.; Miller, N.A.; Boerder, K.; Ferretti, F.; Wilson, A.; Bergman, B.; White, T.D.; Block, B.A.; et al. Tracking the global footprint of fisheries. Science 2018, 359, 904–908. [Google Scholar] [CrossRef] [Green Version]
- Zhou, Y.; Daamen, W.; Vellinga, T.; Hoogendoorn, S.P. Ship classification based on ship behavior clustering from AIS data. Ocean Eng. 2019, 175, 176–187. [Google Scholar] [CrossRef]
- Riveiro, M.; Pallotta, G.; Vespe, M. Maritime anomaly detection: A review. WIREs Data Min. Knowl. Discov. 2018, 8, e1266. [Google Scholar] [CrossRef] [Green Version]
- Singh, S.; Heymann, F. Machine Learning-Assisted Anomaly Detection in Maritime Navigation using AIS Data. In Proceedings of the 2020 IEEE/ION Position, Location and Navigation Symposium (PLANS), Portland, OR, USA, 20–23 April 2020; pp. 832–838. [Google Scholar]
- Tassetti, A.; Ferrà, C.; Fabi, G. Rating the effectiveness of fishery-regulated areas with AIS data. Ocean Coast. Manag. 2019, 175, 90–97. [Google Scholar] [CrossRef]
- Ferrà, C.; Tassetti, A.N.; Armelloni, E.N.; Galdelli, A.; Scarcella, G.; Fabi, G. Using AIS to Attempt a Quantitative Evaluation of Unobserved Trawling Activity in the Mediterranean Sea. Front. Mar. Sci. 2020, 7, 1036. [Google Scholar] [CrossRef]
- Grover, A.; Kumar, S.; Kumar, A. Ship Detection Using Sentinel-1 SAR Data. ISPRS Ann. Photogramm. Remote Sens. Spat. Inf. Sci. 2018, IV-5, 317–324. [Google Scholar] [CrossRef] [Green Version]
- Santamaria, C.; Stasolla, M.; Fernandez Arguedas, V.; Argentieri, P.; Alvarez, M.; Greidanus, H. Sentinel-1 Maritime Surveillance; Technical Report; European Union: Luxembourg, 2015. [Google Scholar]
- Santamaria, C.; Alvarez, M.; Greidanus, H.; Syrris, V.; Soille, P.; Argentieri, P. Mass Processing of Sentinel-1 Images for Maritime Surveillance. Remote Sens. 2017, 9, 678. [Google Scholar] [CrossRef] [Green Version]
- Tanveer, H.; Balz, T.; Mohamdi, B. Using convolutional neural network (CNN) approach for ship detection in Sentinel-1 SAR imagery. In Proceedings of the 2019 6th Asia-Pacific Conference on Synthetic Aperture Radar (APSAR), Xiamen, China, 26–29 November 2019. [Google Scholar] [CrossRef]
- Lemoine, G.; Chesworth, J.; Schwartz-Juste, G.; Kourti, N.; Shepherd, I. Near real time vessel detection using spaceborne SAR imagery in support of fisheries monitoring and control operations. In Proceedings of the IGARSS 2004—IEEE International Geoscience and Remote Sensing Symposium, Anchorage, AK, USA, 20–24 September 2004; Volume 7, pp. 4825–4828. [Google Scholar]
- Graziano, M.D.; Renga, A.; Moccia, A. Integration of Automatic Identification System (AIS) Data and Single-Channel Synthetic Aperture Radar (SAR) Images by SAR-Based Ship Velocity Estimation for Maritime Situational Awareness. Remote Sens. 2019, 11, 2196. [Google Scholar] [CrossRef] [Green Version]
- Snapir, B.; Waine, T.; Biermann, L. Maritime vessel classification to monitor fisheries with SAR: Demonstration in the North Sea. Remote Sens. 2019, 11, 353. [Google Scholar] [CrossRef] [Green Version]
- Kurekin, A.; Loveday, B.; Clements, O.; Quartly, G.; Miller, P.; Wiafe, G.; Agyekum, K. Use of Sentinel-l and Sentinel-2 for Monitoring Illegal Fishing Off Ghana. In Proceedings of the IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain, 22–27 July 2018; pp. 6875–6878. [Google Scholar] [CrossRef]
- Lang, H.; Wu, S.; Xu, Y. Ship Classification in SAR Images Improved by AIS Knowledge Transfer. IEEE Geosci. Remote Sens. Lett. 2018, 15, 439–443. [Google Scholar] [CrossRef]
- Chaturvedi, S.; Yang, C.S.; Ouchi, K.; Shanmugam, P. Ship recognition by integration of SAR and AIS. J. Navig. 2012, 65. [Google Scholar] [CrossRef]
- Rodger, M.; Guida, R. Classification-Aided SAR and AIS Data Fusion for Space-Based Maritime Surveillance. Remote Sens. 2021, 13, 104. [Google Scholar] [CrossRef]
- Mazzarella, F.; Vespe, M.; Santamaria, C. SAR Ship Detection and Self-Reporting Data Fusion Based on Traffic Knowledge. IEEE Geosci. Remote Sens. Lett. 2015, 12, 1685–1689. [Google Scholar] [CrossRef]
- Young, D.L. Deep Nets Spotlight Illegal, Unreported, Unregulated (IUU) Fishing. In Proceedings of the 2019 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), Washington, DC, USA, 15–17 October 2019; pp. 1–7. [Google Scholar]
- Ferlansyah, N.; Suharjito, S. A Systematic Literature Review of Vessel Anomaly Behavior Detection Methods Based on Automatic Identification System (AIS) and another Sensor Fusion. Adv. Sci. Technol. Eng. Syst. J. 2020, 5, 287–292. [Google Scholar] [CrossRef]
- Zhao, Z.; Ji, K.F.; Xing, X.W.; Zou, H.X. Effective Association of SAR and AIS Data Using Non-Rigid Point Pattern Matching. IOP Conf. Ser. Earth Environ. Sci. 2014, 17, 012258. [Google Scholar] [CrossRef]
- Pelich, R.; Longépé, N.; Mercier, G.; Hajduch, G.; Garello, R. AIS-Based Evaluation of Target Detectors and SAR Sensors Characteristics for Maritime Surveillance. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2015, 8, 3892–3901. [Google Scholar] [CrossRef] [Green Version]
- Park, J.; Lee, J.; Seto, K.; Hochberg, T.; Wong, B.A.; Miller, N.A.; Takasaki, K.; Kubota, H.; Oozeki, Y.; Doshi, S.; et al. Illuminating dark fishing fleets in North Korea. Sci. Adv. 2020, 6. [Google Scholar] [CrossRef]
- Galdelli, A.; Mancini, A.; Ferrà, C.; Tassetti, A.N. Integrating AIS and SAR to monitor fisheries: A pilot study in the Adriatic Sea. In Proceedings of the 2020 IMEKO TC-19 International Workshop on Metrology for the Sea, Naples, Italy, 5–7 October 2020; pp. 39–44. [Google Scholar]
- Greidanus, H.; Alvarez, M.; Santamaria, C.; Thoorens, F.X.; Kourti, N.; Argentieri, P. The SUMO Ship Detector Algorithm for Satellite Radar Images. Remote Sens. 2017, 9, 246. [Google Scholar] [CrossRef] [Green Version]
- Kourti, N.; Shepherd, I.; Schwartz, G.; Pavlakis, P. Integrating Spaceborne SAR Imagery into Operational Systems for Fisheries Monitoring. Can. J. Remote Sens. 2001, 27, 291–305. [Google Scholar] [CrossRef]
- Mancini, A.; Tassetti, A.N.; Cinnirella, A.; Frontoni, E.; Zingaretti, P. A novel method for fast processing of large remote sensed image. In International Conference on Image Analysis and Processing; Springer: Berlin/Heidelberg, Germany, 2013; pp. 409–418. [Google Scholar]
- Bastardie, F.; Angelini, S.; Bolognini, L.; Fuga, F.; Manfredi, C.; Martinelli, M.; Nielsen, J.R.; Santojanni, A.; Scarcella, G.; Grati, F. Spatial planning for fisheries in the Northern Adriatic: Working toward viable and sustainable fishing. Ecosphere 2017, 8, e01696. [Google Scholar] [CrossRef] [Green Version]
- Lambert, G.I.; Jennings, S.; Hiddink, J.G.; Hintzen, N.T.; Hinz, H.; Kaiser, M.J.; Murray, L.G. Implications of using alternative methods of vessel monitoring system (VMS) data analysis to describe fishing activities and impacts. ICES J. Mar. Sci. 2012, 69, 682–693. [Google Scholar] [CrossRef] [Green Version]
- Freund, Y.; Schapire, R.E. A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting. J. Comput. Syst. Sci. 1997, 55, 119–139. [Google Scholar] [CrossRef] [Green Version]
- MATLAB. Analyze Signals in the Frequency and Time-Frequency Domains. Available online: https://www.mathworks.com/help/signal/ref/pspectrum.html (accessed on 12 April 2021).
- ESA. Sentinel-1 Product Definition. Document S1-RS-MDA-52-7440, 2016. European Space Agency (ESA). Available online: https://dragon3.esa.int/documents/247904/1877131/Sentinel-1-Product-Definition (accessed on 12 April 2021).
- Wessel, P.; Smith, W. A global, self-consistent, hierarchical, high-resolution shoreline database. J. Geophys. Res. 1996, 101, 8741–8743. [Google Scholar] [CrossRef] [Green Version]
- EMODnet. Available online: https://www.emodnet-humanactivities.eu (accessed on 12 April 2021).
- Zhong, H.; Song, X.; Yang, L. Vessel Classification from Space-based AIS Data Using Random Forest. In Proceedings of the 2019 5th International Conference on Big Data and Information Analytics (BigDIA), Kunming, China, 8–10 July 2019; pp. 9–12. [Google Scholar] [CrossRef]
- Tello, M.; Lopez-Martinez, C.; Mallorqui, J.; Bonastre, R. Automatic Detection of Spots and Extraction of Frontiers in SAR Images by Means of the Wavelet Transform: Application to Ship and Coastline Detection. In Proceedings of the 2006 IEEE International Symposium on Geoscience and Remote Sensing, Denver, CO, USA, 31 July–4 August 2006; pp. 383–386. [Google Scholar]
Datetime | AIS (in Port *) | SUMO (45 nm **) | BL 1 | FP 2 | SP 3 | PP 4 | PL 5 | SB 6 |
---|---|---|---|---|---|---|---|---|
20 March 2017 16:56:52 | 85 (44) | 290 (102) | 2 | 1 | 18 | 28 | 2 | 1 |
19 July 2019 05:10:31 | 46 (20) | 253 (117) | 8 | 1 | 28 | 21 | 8 | 3 |
20 July 2019 16:57:09 | 48 (28) | 128 (37) | 8 | 0 | 14 | 7 | 8 | 1 |
24 July 2019 05:18:43 | 83 (52) | 222 (57) | 11 | 2 | 10 | 18 | 11 | 0 |
25 July 2019 05:11:14 | 88 (65) | 195 (70) | 14 | 2 | 28 | 20 | 14 | 0 |
26 July 2019 16:57:52 | 47 (21) | 144 (39) | 3 | 1 | 17 | 11 | 3 | 0 |
9 March 2018 16:57:40 | 49 (23) | 147 (39) | 3 | 0 | 17 | 8 | 3 | 0 |
9 March 2019 05:10:32 | 48 (20) | 228 (116) | 1 | 2 | 36 | 20 | 1 | 0 |
9 March 2020 05:11:14 | 79 (48) | 321 (187) | 5 | 2 | 28 | 28 | 5 | 0 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Galdelli, A.; Mancini, A.; Ferrà, C.; Tassetti, A.N. A Synergic Integration of AIS Data and SAR Imagery to Monitor Fisheries and Detect Suspicious Activities. Sensors 2021, 21, 2756. https://doi.org/10.3390/s21082756
Galdelli A, Mancini A, Ferrà C, Tassetti AN. A Synergic Integration of AIS Data and SAR Imagery to Monitor Fisheries and Detect Suspicious Activities. Sensors. 2021; 21(8):2756. https://doi.org/10.3390/s21082756
Chicago/Turabian StyleGaldelli, Alessandro, Adriano Mancini, Carmen Ferrà, and Anna Nora Tassetti. 2021. "A Synergic Integration of AIS Data and SAR Imagery to Monitor Fisheries and Detect Suspicious Activities" Sensors 21, no. 8: 2756. https://doi.org/10.3390/s21082756
APA StyleGaldelli, A., Mancini, A., Ferrà, C., & Tassetti, A. N. (2021). A Synergic Integration of AIS Data and SAR Imagery to Monitor Fisheries and Detect Suspicious Activities. Sensors, 21(8), 2756. https://doi.org/10.3390/s21082756