Towards Automatic Recognition of Wakes Generated by Dark Vessels in Sentinel-1 Images

Abstract

The recognition of wakes generated by dark vessels is a tremendous and interesting challenge in the field of maritime surveillance by Synthetic Aperture Radar (SAR) images. The paper aims at assessing the detection performance in different scenarios by processing Sentinel-1 SAR images along with ground truth data. Results confirm that the Radon-based approach is an effective technique for wake-based detection of dark vessels, and they lead to a deeper understanding of the effects of different sea and wind conditions. In general, the best applicative scenario is a marine image characterized by homogeneous sea clutter; the presence of natural surface film or strong transition from low wind speed areas to more windy zones worsen the detection performance. Nonetheless, the proposed approach features dark vessel detection capabilities by identifying their wakes, without any a priori knowledge of their positions.

1. Introduction

The maritime domain is constituted by all areas and things of, on, under, relating to, adjacent to, or bordering on a sea, ocean, or other navigable waterway, including all maritime-related activities, infrastructure, people, cargo, and vessels and other conveyances. Thus, no one country, department, or agency holds all the authorities and capabilities to achieve effective Maritime Domain Awareness (MDA) on its own. However, by combining separate pieces of information from agencies at the federal, state, local, and even tribal level around the world with information from the maritime industry and other non-governmental organizations, it is possible to keep track of the status of every ocean-bound and sea-bound vessel [1].

The maritime domain, hence, develops on a global scale, and satellite-based systems naturally answer to the specific needs of remote access, global reach, and regular update. The interest in the space-based images for maritime surveillance has strongly increased, owing to the increased number of the optical and radar satellites, especially in the last decade. Concerning this, several techniques, tools, and projects have been developed to improve the maritime domain awareness [2,3,4,5,6,7,8,9].

The common thread of all past and on-going projects is to detect vessels in Synthetic Aperture Radar (SAR) images and characterize them in terms of class and motion-related parameters when they are visible in images. The most exploited technique for ship detection is based on Constant False Alarm Rate (CFAR) algorithms [2,10,11,12,13], in which the target is detected as pixels brighter than the background ones, keeping constant the false alarm rate over the image. A different concept is exploited by the multi-look/sub-aperture detection algorithm [14,15]. It implies that the ships are discriminated as targets showing different behaviors in different sub-looks of the image. This technique has been analyzed under different polarization, frequency and resolution conditions in [15] for detection purposes, and, as a discrimination step after the prescreening carried out by CFAR technique in [16]. Finally, a different approach exploiting the combination of the two cross-polarized channels has been proposed to improve the detection performance. However, the application of polarimetric technique is strongly limited by the availability and coverage of the relevant data [17]. Recent studies have detected ships using deep learning techniques, such as Convolutional Neural Networks (CNNs) for image-based feature extraction [18,19,20,21,22]. In the ambit of MDA, most studies using deep learning techniques focus on methods for the detection of artificial targets, such as ships, characterized by a strong backscattering.

The above-mentioned techniques enable the detection of ships appearing as a cluster of bright pixels in SAR images. Then, if also the ship wakes are distinguishable, information about their route, i.e., heading and velocity, can be reached by wake detection and analysis [23,24,25,26]. Currently, wake detection in SAR images has been faced following two main approaches. The first, more assessed, one exploits the typical appearance of wake in SAR images: since they appear as dark or bright lines, they can be detected as linear features in a noisy background, e.g., by Radon Transform [25,26]. Recent studies [24] have proposed a different method for wake detection by using CNNs, using Inception v3 by Google as a representative model for detecting, classifying, and learning targets from images. Even if the algorithm has been applied only over 12 SAR images gathered by the TanDEM-X mission (including 13 wakes), the results suggests that the CNN is a useful tool for detecting the presence of wake on the wide SAR image. However, even though the automatic detection rate is quite high in general, data collected with higher wind condition (speed of about 7.9 m/s) shows an accuracy of wake detection of 60%.

Both Radon-based and CNN-based techniques for wake detection strongly rely on a priori knowledge of vessel position; in other words, they have been developed to look for a wake in small image patch, e.g., for supporting the route estimation of previously detected vessels.

In recent years, the operations of high-speed planing vessels for military, commercial and leisure use have increased. US navy has conducted a survey of the operators of high-speed craft [27], showing a high probability of serious injury. In fact, the use of this class of vessels has been signaled by pirates crossing from the mother ship to a passenger ship in the Somali area, or for trafficking goods in the Adriatic Sea, or for transporting drugs in the Alboran sea, or as a fishing boat plundering marine protected areas [28].

Since they have to face high-speed operations, these activities are typically carried out by using vessels, called GoFast [29,30] with hulls made of fiberglass, a sharp, vertically rounded bow, and a transom stern, in order to reduce resistance. They are also characterized by high values of the deadrise angle (“deep-V hull”) to improve seakeeping and maneuverability performances, even if observations in very calm and narrow water, such as the area between St. Lawrence in the Cornwall are of Ontario, between the United States and Canada, have found that the deep-V design is replaced by a flat-bottomed skiff version. Moreover, a boat with a broader beam and no chine, but with two engines to reach high speed, has been used by Somali pirates.

The detection of GoFast ships in satellite images is a tremendous challenge. Due to their building materials, size and sea-wind conditions, their observation in radar images is almost unfeasible, and, operating typically during the nighttime, the optical sensors cannot be used. Nevertheless, thanks to their high velocity and narrow beam, the GoFast ships are likely to produce long visible wakes. Such wakes are very persistent, lasting for distances of several tens of kilometers [28,29,30], and can be observed by SAR imagery [31,32].

Since during the unreported transits, unauthorized commodity transshipment, smuggling, illegal fishing, and other unlawful activities, the GoFast ships operate with their automatic identification system (AIS) and other identification systems switched off–against IMO regulations, they can be also called dark vessels.

Preliminary results on the GoFast ships detection by wake pattern recognition are shown in [33], and no additional literature results are available, since in both [24,25], no wakes generated by GoFast vessels have been detected. The algorithm in [33] has been applied over an X-band image gathered by TerraSAR-X over the Gulf of Naples, and the results confirm its potentialities, in terms of wake pattern recognition in SAR images and rejection of false detections.

The present paper moves from those results and aims to:

• Extend the technique to a different radar band (C-band) and lower resolution images (Sentinel-1);
• Get further insight into the capabilities and the limitations of the approach for wake pattern recognition in difference scenarios.

The latter, and more important, aim paves the way towards an operative application of the technique to achieve an effective maritime surveillance. In fact, thanks to the availability of ground truth data for the analyzed dataset, the manuscript provides an understanding of wake detection performance in case of marine image characterized by (i) homogeneous sea clutter, (ii) the presence of natural surface film, and (iii) strong transition from low-speed areas to more windy zones.

2. Method

Wake pattern recognition is used to support ship detection, as detailed in [33]. Firstly, the input image is analyzed by a scanning window (Step 1 of Figure 1). In details, at each position of scanning window, the wake presence in the window is analyzed by: (i) applying Radon Transform, (ii) detecting the peak and tough representative of narrow-V and turbulent wake, respectively, as bright and dark dots in the Radon domain within 4° of angular distance, and (iii) estimating merit indexes, which measure how distinguishable is the wake component appearance with respect to the background. Indeed, the wake is classified as distinguishable if it is composed by at least two components: a dark linear feature, representative of the turbulent wake, surrounded by a bright component, related to the narrow-V wake.

Hence, at each step of the scanning window, the merit indexes of the bright (F_v) and dark (F_I) components are estimated as follows:

$$F_v=\frac{CD{F}^{-1}(0.95)}{0.95n\overline{I}}-1$$

(1)

$$F_I=\frac{{\overline{I}}_t}{\overline{I}}-1$$

(2)

where CDF(x_i), the cumulative distribution function of the image intensity (x_i) along the identified half-line. In addition, n is the number of pixels of the wake component, and $\overline{I}$ is the intensity mean value over the whole tile and ${\overline{I}}_t$ is the intensity mean value over the turbulent wake.

The outputs of the Step 1 are: confirmation or denial of wake presence at each position of scanning window, and if the wake is confirmed, the estimation of relative merit indexes and (θ,s) pair, representing the coordinates of peak and tough in the Radon domain, from which the wake direction in the image domain is obtained.

Then, in Step 2 of Figure 1, a list of candidate wake positions are identified as the pixels satisfying Equation (3), in which $\mu$ and $\sigma$ are the mean value and the standard deviation of the $F_w$ distribution, respectively, $F_v$ and $F_I$ are obtained by the previous step, k is assumed to be equal to 2 as in [33].

$$F_w=F_v\cdot \left|F_I\right|>\mu +k\sigma$$

(3)

From the knowledge of (θ,s) pair of each pixel satisfying Equation (3), it is possible to reconstruct the wake. In details, assuming a (x,y) reference frame fixed in the center of the scanning window with x-axis oriented as ground range direction and y-axis oriented as the anti-azimuth direction, θ defines the angle between the normal to the wake and the x-axis, whereas s defines the orthogonal distance between the wake and the center of the reference frame. From this information, it is possible to build the line representative of the wake, but it is not possible to set the wake vertex. To this end, the intensity distribution along the line is analyzed, and the darker half-line is selected as the turbulent wake component.

Finally, since the turbulent component extends for kilometers, the same wake component is detected in neighbor pixels, i.e., in different positions of the scanning window (Step 3 in Figure 1) and pixels most likely affected by wakes are the ones crossed by the largest number of reconstructed wakes.

3. Results

3.1. Dataset

In this study, VV-polarized Interferometric Wide (IW) swath mode Sentinel-1 C-band images are obtained by Copernicus Open Access Hub. The images correspond to Level-1 Ground Range Detected (GRD) products [34]. All the images are acquired in ascending phase over the Gulf of Genoa, Italy, with a ground range and an azimuth spacing of 10 m.

Table 1. Dataset details.

Date	6 July 2020
Image ID	S1A_IW_GRDH_1SDV_20200706T171436_20200706T171501_033337_03DCC6_1122
Image ID	S1A_IW_GRDH_1SDV_20200706T171411_20200706T171436_033337_03DCC6_3302
Wind@Genoa	Speed: 2 m/s	Direction: 204°N
Wind@LaSpezia	Speed: 5 m/s	Direction: 134°N
Date	18 July 2020
Image ID	S1A_IW_GRDH_1SDV_20200718T171436_20200718T171501_033512_03E220_DEFC
Image ID	S1A_IW_GRDH_1SDV_20200718T171411_20200718T171436_033512_03E220_1765
Wind@Genoa	Speed: 2.2 m/s	Direction: 259°N
Wind@LaSpezia	Speed: 2.9 m/s	Direction: 149°N
Date	23 August 2020
Image ID	S1A_IW_GRDH_1SDV_20200823T171414_20200823T171439_034037_03F36B_5DA2
Image ID	S1A_IW_GRDH_1SDV_20200823T171439_20200823T171504_034037_03F36B_447E
Wind@Genoa	Speed: 2 m/s	Direction: 188°N
Wind@LaSpezia	Speed: 2.5 m/s	Direction: 150°N
Date	29 August 2020
Image ID	S1B_IW_GRDH_1SDV_20200829T171409_20200829T171434_023141_02BF02_2FD4
Image ID	S1B_IW_GRDH_1SDV_20200829T171344_20200829T171409_023141_02BF02_36A0
Wind@Genoa	Speed: 1.1 m/s	Direction: 194°N
Wind@LaSpezia	Speed: 9.4 m/s	Direction: 210°N

lists the image IDs along with wind conditions, derived from [35,36,37] as in situ measurements at Genoa and at La Spezia stations, respectively.

The satellite images are completed by the maritime traffic data, provided by the Automatic Identification System (AIS) and used as ground truth. The data are acquired by the AIS ground station of La Spezia-Castellana (Station #890) at latitude and longitude of 44.07°N and 9.82°E, respectively, and from an altitude of 200 m. The AIS data were stored with a time step of 1 s between successive messages of the same ship, so they are particularly suited for SAR matching in a coastal area.

The AIS data are exploited to select ships with characteristics of length-to-beam ratio (LBR) and velocity similar to GoFast class. In detail, hereinafter, all the ships with LBR larger than 3.75 [27] and velocity larger than 15 knots are classified as GoFast-like ships.

Three different test scenarios are processed by the implementation in MATLAB^® environment of the proposed method. The scenarios can be described as follows:

• Scenario 1: wakes generated by GoFast-like ships in a homogeneous wind speed area. Figure 2a is a tile of the image acquired on 6 July 2020. Four ships are included in the tile: ship 247056100 has a velocity of 22.6 knots and LBR of 7.5, so it is classified as GoFast-like ship, whereas the other three ships moves slower than 15 knots (ship 636014645 has a velocity of 13.1 knots and LBR of 3.5, ship 232003476 and ship 2473622320 have LBR of 3.5 and velocity of about 7 knots). Figure 2b shows the wind field, whose speed values over the tile range from 1.6 to 4.5 m/s, whereas Figure 2c portrays the location map of the scenario.
• Scenario 2: wakes generated by GoFast-like ships in a sea area affected by natural surface film [38]. The tile (Figure 3a) of the image gathered on 18 July 2020 is characterized by a sea area with natural surface film and three distinguishable wakes. The leftmost ship (2470036100) has a velocity of 25 knots and LBR of 7.04, and the rightmost ship has a velocity of 18.4 knots and LBR of 5.6, so they can be both considered as GoFast-like ships. In addition, a non-collaborative ship is moving between the previous two ships. Figure 3b shows the estimation of wind field over the tile, whereas Figure 3c portrays the location map of the scenario.
• Scenario 3: wakes generated by GoFast-like ships close to a low wind speed area. In Figure 4a, ship 255805983 has a velocity of 21 knots and LBR of 7.9. It moves along an almost azimuthal direction (162°N) close to an area with low wind speed. In details, the wind speed (Figure 4b) reduces from about 2 m/s around the ship until 0.3 m/s in the dark area of the tile. With specific reference to this scenario an additional case is analyzed in Section 3 for which the transition between dark and bright areas is even clearer.

3.2. Detection Performance Estimation

The proposed approach is applied to assess the detection performance in all the scenarios introduced in Section 3.1.

The tile of Figure 2a is exported from SNAP as a tiff image and processed in the MATLAB^® environment. The white dots in Figure 5a are the points in which the merit indexes are estimated during the Step 1, i.e., all the positions of the scanning window over the tile. The red circles, instead, identify the pixels satisfying Equation (3), thus providing the candidate wake positions.

It is possible to note a concentration of red circles around the true position of the wakes but also a large number of sparse candidate positions within the tile. As foreseen in Step 3, in each red circle, the wake is reconstructed thanks to the knowledge of (θ,s) pair and the blue pixels in Figure 5b are the pixels crossed by the largest number of reconstructed wakes.

The locations of blue pixels enable the local wake detection analysis as in [25], whose outcomes are shown in Figure 5c–d for ship 247056100 and 636014645, respectively.

Scenario 1 is characterized by homogenous wind speed condition and sea clutter and results confirm that this is the best case for the wake pattern recognition. Indeed, no wake is lost, and no false detections are identified. Moreover, the blue pixels are almost distributed along a line.

The tile under analysis for assessing the detection performance in case of sea affected by natural surface film is shown in Figure 6a. Similar to the previous case, the candidate wake positions (red circle) are concentrated around the real wakes, but the presence of dark strips due to the surface film generates additional and false candidate wake positions. The blue pixels in Figure 6b suggest that the validation of the wake presence among neighbor pixels is an effective mean to discard such false wake positions. However, as a result of the trade-off between false and missed detections, the wake of ship 672269000 is not clearly distinguishable, so that it is not recognized. A local wake analysis around blue pixels in Figure 6b enables the wake reconstruction for ship 247036100, as shown in Figure 6c.

The tile selected for testing the scenario 2 (Figure 6a) is particularly interesting since it includes a very small dark vessel. Since during the step 1 the ship is masked to avoid that its returns affect the Radon-domain processing, the correct detection of such wake is an example of the operative application of the proposed approach. In details, assuming that the AIS data of the ships are exploited to classify the ships 247036100 and 672269000 as collaborative, the application of the proposed technique has enabled the detection of a non-collaborative ship by means of its wake. The local wake analysis shown in Figure 6d confirms the presence of the wake related to the dark vessel.

Finally, it is worth noting that the blue pixels represent pixels crossed by the largest number of reconstructed wakes, i.e., pixel with the high level of confidence that they are crossed by wakes. If a lower level of confidence is acceptable, the wake of ship 672269000 is partially detected, but at the time, false alarms also result from portions of dark features on the sea surface.

As far as the influence of dark sea area on the detection performance is concerned, Figure 7a shows that no concentration of candidate wake positions is due to bright spot over a dark background in the low wind speed area, which could have led to false detections. The detection results (Figure 7b) enable the identification of the wake, and no false detections are related to the dark sea.

However, it is worth noting that in this case there is a gradual increase in wind speed from the dark area to the bright one. Instead, Figure 8a shows a tile gathered on 29 August 2020, showing another portion of low-speed area. A GoFast-like ship is included (mmsi: 67224800) with a velocity of 17.6 knots and LBR of 6.3. The tile includes also two dark vessels (ship A and ship B). The appearance of ship A is very similar to GoFast ship, since its echoes affects a very low number of pixels. Ship B, instead, shows a non-linear wake, so it is interesting to assess the detection performance with respect to not straight wake components. In this tile, the transition between dark and bright area is clearer and developing along a curvilinear direction. This determines some false alarms (Figure 8b) in the border strip. Their number could be theoretically reduced by increasing the size of scanning windows, which is however not a viable solution in the considered case due to a very close coastline.

4. Discussion and Conclusions

Different reference scenarios were analyzed to characterize the problem of recognition of wakes generated by dark vessels in SAR images to achieve an effective maritime surveillance. In this concern, it is clear that limitations in terms of SAR images availability, revisit time, and delivery time play an important role when the images are to be exploited for surveillance applications. However, the upcoming SAR satellite constellations based on inclined orbits with different right ascension of the ascending node will enable nighttime observations and very short revisit time, e.g., 1 h by the complete ICEYE constellation. This is expected to overcome the limitations of the current SAR satellites and constellations, which operate in dusk/dawn orbit and, practically, share the same line-of-nodes. In addition, with the NRT (near real-time) mode, the information on suspect vessels is made available to the user within 30 min from the acquisition, thus enabling reliable and operational detection of dark vessels.

The first analyzed scenario focused on an area in which the wind field is homogeneous and ships with characteristics similar to GoFast ships in terms of velocity and length/beam ratio are identified. In Section 3, an example of such scenario is shown and discussed. The results show that thanks to homogeneity of the sea clutter, ship detection by wake recognition is very effective. No false detections are observed in the tested cases, even if the application to a wider dataset is required to confirm this evidence.

The second scenario implies a case in which the sea is affected by natural surface film. In this case, the wake recognition also shows very good performance when the wake appearance is more distinguishable than the dark features over the sea surface. In details, the most visible wakes are identified with the highest level of confidence, that is the maximum number of reconstructed lines crossing a single image pixel. On the contrary, a lower level of confidence, i.e., less than the maximum number of reconstructed lines crossing a pixel, is estimated for the wakes whose visibility are similar to the dark features over the surface. It is evident that a trade-off is required between the probabilities of false and missed detection. In the present case, the method was set to decrease the former, thus also increasing the latter.

The third scenario has been revealed as the most challenging for the proposed approach. In fact, it investigated the case in which both high and low wind speed areas are imaged. This situation is prone to generate both false and missed wake detections. The first ones have been clearly individuated in the processed dataset, and they are generated when the transition is so sharp and almost linear in shape. As far as missed wakes are concerned, even if they were not observed in the analyzed images, they can occur across the transition depending on the motion directions of the vessels.

For each scenario, three trials have been analyzed, and the results show that:

• The best applicative scenario is the one characterized by homogeneous sea clutter, in which a 100% of success rate of wake identification is observed, with no missed wakes.
• When the scenarios include prominent linear features (i.e., natural surface film almost linear or sharp linear transitions from bright to dark image portions), the detection rate is affected. In details, we have observed that in one out of three scenarios with natural surface film, the wake detection shows false alarms, because the dark features related to natural surface film is recognized as wake due to almost linear shape in the tile. Instead, the area with low wind speed is well managed by the proposed techniques, which does not provide any false detections related to brighter spots in the dark low speed areas. The attention, in this scenario, shall be paid to the transition edge from dark to bright sea: if the border is almost linear in the tile, it can lead to false wake identification, as the one shown in Figure 8. This has occurred in one out of three tested trials.

The analysis of scenarios increases the understanding of wake recognition performance in different conditions, showing the potentialities of a Radon-based approach for wake detection also in the cases of dark vessels.

This analysis paves the way to future investigations in which the benefits, and the limitations, of Radon-based approaches must be recast in a more complete framework, i.e., the data fusion with other techniques. This includes the analysis of wind field, especially in the case of low wind speed areas, and the on-going studies on wake recognition by CNN. Indeed, it is well-known that the processing of very wide scene is a challenge for the Radon-based technique. Similarly, letting a deep network be aware of the quality of its prediction is yet an important problem [37]. Hence, combining the two approaches, e.g., in a cascade prescreening/discrimination fashion, can offer indisputable capabilities of either fast processing of wide scene (by CNN) and of keeping the quality of the classification of wake component (by Radon-based approaches), both required for an effective maritime surveillance.