Remote Sensing for Maritime Monitoring and Vessel Identification

1. Maritime Monitoring

According to the statistics published by the United Nations Conference on Trade and Development [1], the total fleet worldwide consisted of more than $2.2\times {10}^9$ deadweight tons in 2023 with about 105,000 vessels against less than $1.5\times {10}^9$ in 2011 with about 83,000 vessels. Even recognizing a slightly decreasing trend in recent years [2], the European Maritime Security Agency certifies a growing number of marine accidents of EU ships in domestic waters, from about 1300 in 2011 to about 2500 in 2022. This situation makes marine traffic surveillance essential for border control, monitoring of illegal activities as well as general security and emergency management. During the last decades, many systems to permit a ship to locate her position at sea have been developed, e.g., the Global Positioning System (GPS). A useful review on the evolution of the related technical standards, by Zalewski et al., can be found in this issue: https://www.mdpi.com/2072-4292/14/21/5291 (accessed on 2 February 2024). A further step towards maritime safety consists in systems that use automatic positioning and other information to enable individual vessels and the maritime administrative, emergency and law-enforcing authorities to form a clear picture of the current situation in an area of interest and to foresee its most likely evolution. Standards and rules to be followed for deployment and use of such systems are established by the International Maritime Organization (IMO [3]) and mostly depend on the collaboration of the vessels by either human intervention or through automatic transponders. A relevant example is represented by the Automatic Identification System (AIS).

Wherever and whenever the collaborative vessel traffic services are not operational or not reliable for system failures, spoofing or other malicious actions, or some vessel is suspected of sending falsified messages, remote sensing is the only possibility to properly ensure safety and security and take the appropriate reactions/countermeasures for any targeted event. Several problems are still open to cope with the wide range of applications related to maritime surveillance without the help of collaborative systems [4], even though many technologies and platforms are operational for detecting and locating even the faintest objects on the sea surface, ranging from optics in various bands to radio/acoustic waves and from satellite to underwater platforms. Marine traffic understanding is not only ship detection and tracking: a remote-sensed target recognized as a vessel should at least be assigned to its specific ship type and possibly identified as an individual ship, even without relying on its active collaboration. Moreover, having information about the current course and velocity would be necessary to estimate the most likely target positions in a specified time frame [5]. Finally, a complete situational awareness would be achieved if the transported passengers and cargo could be identified and the complete ship behavior could be evaluated. To this end, it would be indispensable to integrate data from different specialized sensors and other available information, such as meteorological, geographical and historical data. Data science, including deep learning and artificial intelligence as well as pattern recognition, image analysis and statistical signal processing, is thus essential to provide a complete assessment of the maritime traffic status, for safety, security and general maritime traffic management purposes. The need for advances in information technology is also recognized by the IMO [6] up to possible “negative consequences” (see also [7]). It is apparent that despite the open problems, the mandatory use of specific technologies has produced a marked decrease in the effects of marine accidents: the number of injuries produced by contact or collision events in EU waters was 136 in 2014 against 43 in 2022. Also, the number of ships lost or damaged and the related pollution events is steadily decreasing [2].

The multi-faceted interests backing maritime activities [8] all need efficient awareness and communication systems, so all the goals mentioned above are pursued by several international and governmental agencies, as well as many commercial companies for security, scheduling and fleet management purposes, and are the subject of active research from academic, government and private institutions [7,9]. As remote sensing is the ultimate resource for situational awareness in the absence of collaborative actions, we tried to gather a number of colleagues in this field and collect their contributions, either theoretical or applicative, in a systematic way. Below, we present the content of this Special Issue.

2. In This Issue

Table 1. Overview of published papers.

Paper	Platform	Sensor	Data Processing	Application
Zalewski et al.	Space			Positioning (Review)
Lang et al.	Space	SAR	Deep learning	Classification
Chen et al.	Space	SAR	Deep learning	Detection
Bezerra et al.	Space	SAR	Statistics	Detection
Reggiannini et al.	Space	SAR Optical–Infrared	Filtering– Machine learning– Spectral analysis	Detection- Classification– Behavior analysis
Jian et al.	Space	Optical	Deep learning	Detection- Classification
Meng et al.	Space	Optical– Multispectral	Filtering	Detection-Tracking
Troupiotis et al.	Space– Surface	Optical–SAR	Interpolation– Data fusion	Traffic density mapping
Zhou et al.	Surface	Optical–Infrared	Filtering–Statistics	Detection
Cope et al.	Surface	Optical– Marine Radar	Data fusion	Tracking– Behavior analysis
Liu et al.	Underwater	Acoustic	Deep learning	Classification- Identification

outlines the contents of the published papers. Among the variety of topics we suggested when launching this Special Issue, we received a good response from our colleagues dealing with visible-infrared and radar sensors, used alone or in combination. The most considered sensing platforms are located in space, thus confirming the advantages derived from the wide swaths obtainable over the more limited views offered by airborne and terrestrial sensors. Among the spaceborne sensors, SAR seems to be preferred by virtue of its all-weather, all-lighting properties, despite its limited spatial resolution and attitude/incidence-dependent distortions. Figure 1 shows the number of papers included in this Special Issue, organized by platform (a), sensor (b), data processing (c), and application (d). Detection and classification dominate. Deep learning approaches play a major role in data processing. As anticipated, the data processing strategies are the characterizing features of all the papers, allowing a maximum of interpretable information from heterogeneous sensor and auxiliary data. Besides the already cited review by Zalewski et al. on the evolution of maritime GNSS and RNSS standards, this Special Issue features ten papers, summarized here by application.

Three papers are entirely devoted to detection. Chen et al. (https://www.mdpi.com/2072-4292/15/10/2589 (accessed on 2 February 2024)) propose a modified feature pyramid convolutional neural network and an attention mechanism to improve ship detection from particularly crowded satellite SAR scenes possibly including terrestrial structures. Bezerra et al. (https://www.mdpi.com/2072-4292/15/13/3441 (accessed on 2 February 2024)) also deal with SAR data, and modify the thresholds for the classical CFAR detection on the basis of wind and wave information provided by the ERA5 meteorological data. Zhou et al. (https://www.mdpi.com/2072-4292/15/9/2354 (accessed on 2 February 2024)) consider infrared sensors located on the ground and improve the performance of ship detection by applying two different models (violent changes and stable changes) for the sea surface.

Meng et al. (https://www.mdpi.com/2072-4292/15/8/2069 (accessed on 2 February 2024)) propose a method for both detection and association tracking of small targets from satellite multispectral image sequences possibly integrated with AIS data. Cope et al. (https://www.mdpi.com/2072-4292/15/13/3216 (accessed on 2 February 2024)) face the problem of traffic monitoring in marine protected areas by detecting and tracking vessel activities through fusion of data from an X-band marine radar, optical cameras and AIS. Aspects of detection and tracking are also considered by Reggiannini et al. (https://www.mdpi.com/2072-4292/16/3/557 (accessed on 2 February 2024)) and Troupiotis-Kapeliaris et al. (https://www.mdpi.com/2072-4292/15/21/5080 (accessed on 2 February 2024)). Three papers essentially deal with classification, all relying on deep learning for data processing. Lang et al. (https://www.mdpi.com/2072-4292/14/23/5986 (accessed on 2 February 2024)) adopt satellite SAR images for feature refinement in a shallow convolutional neural network pretrained through maritime-specific optical data. Jian et al. (https://www.mdpi.com/2072-4292/15/17/4319 (accessed on 2 February 2024)) propose a YOLOv5 deep learning model modified by a block attention module, a cross-layer connection channel and an improved loss function for detection and fine-grained ship classification. Liu et al. (https://www.mdpi.com/2072-4292/15/8/2068 (accessed on 2 February 2024)) leverage signals from underwater acoustic sensors to identify different individual ships from their radiated noise using a multi-scale feature-adaptive generalized network. Reggiannini et al. also deal with ship classification, among other problems, proposing some refinements to a maritime traffic monitoring system already presented in the literature for ship detection, classification, velocity estimation, behavior analysis and route prediction, especially as far as SAR image processing and thermal infrared data are concerned.

Finally, the paper by Troupiotis-Kapeliaris et al. relies on collaborative data (terrestrial or satellite AIS) and satellite imagery to estimate ship trajectories and gathering them to form maritime traffic density maps on very large areas. The AIS data are first cleaned by removing unreliable or erroneous messages, thus obtaining space-time sequences with more or less large gaps, then filled by a specific algorithm, possibly with the help of optical or SAR satellite imagery.