Multi-Spectral Ship Detection using Optical, Hyperspectral, and Microwave SAR Remote Sensing for Sustainability of the Coastal Region

As human activities of the countries in the East Asia have been remarkably expanding over recent decades, various problems in relation to ships, such as oil spill and many other coastal marine pollution, are continuously occurring in the coastal region. In order to conserve marine resources and prepare for possible ship accidents in advance, the need for efficient ship management is increasing over time. Multi-satellite, multi-sensor, multi-wavelength or multifrequency observations make it possible to monitor a variety of vessels in the coastal region. This study presents the results of ship detection methodology applied to multi-spectral satellite images in the seas around Korean Peninsula based on optical, hyperspectral, and microwave remote sensing. To detect ships from hyperspectral images with a few hundreds of spectral channels, spectral matching algorithms are used to investigate similarity between the spectra and in-situ measurements. In the case of SAR (Synthetic Aperture Radar) images, the Constant False Alarm Rate (CFAR) algorithm is used to discriminate the vessels from backscattering coefficients of Sentinel-1 SAR and ALOS-2 PALSAR2 images. The present ship detection methods can be extensively utilized for optical, hyperspectral, and SAR images for comprehensive coastal management purposes toward perpetual sustainability in the future.


Introduction
The seas around the Korean Peninsula are some of the marginal seas of the Northwest Pacificconsisting of the East Sea (called Sea of Japan), Yellow Sea, and East China Sea-and they border East Asian countries with some of the largest populations in the world [1]. Over the past few decades, the economic activities of the countries in East Asia have expanded the most significantly in the world [2][3][4]. As a result, port facilities have been continuously constructed along the coasts of each of these countries, and maritime trades have been rapidly increasing as well. The seas around the Korean Peninsula possess great strategic value owing to the increasing economic activities and volume of marine trade. Therefore, an efficient marine monitoring system is crucial for achieving the sustainable development of coastal and marine areas as well as to protect coastal resources for ensuring public safety against frequent marine accidents.
Incessant monitoring of time-varying oceanographic features at diverse temporal and spatial scales is required for diagnosing and forecasting the impacts of global warming and rapid climate change on the ocean. In recent years, with the increasingly rapid development of science and technology, new satellite instruments have been developed for marine and earth observation and have been used extensively for varied purposes. Satellite observations are more efficient than traditional marine in situ observations in terms of data sampling capability in space and time. These synoptic, simultaneous, repetitive measurements based on satellite remote sensing play an important role in understanding the spatial and temporal variations of oceanic phenomena. Satellite sensors have long been used to observe invaluable oceanic variables such as sea-surface temperature, sea level, sea-surface wave height, sea ice, and chlorophyll-a concentration in phytoplankton. In addition, the satellite observations also facilitate the detection of vessels, objects on the sea surface, marine pollution, and oil spills owing to ship collisions. Such frequent coastal observations allow us to monitor a large number of ships at sea continuously. In particular, microwave sensors, as an allweather sensor, have outstanding advantages in detecting target objects regardless of precipitation and clouds in the atmosphere.
As the number of vessels at the coast of the Korean Peninsula increases, various types of accidents, such as collisions between diverse types of ships, occur frequently [5]. The oil spill caused by the accident between a crane barge and the crude oil carrier Hebei Spirit caused unprecedented marine environment pollution over a wide area of the western coast of the Korean Peninsula [6,7]. Moreover, another crude oil spill accident occurred in Gwangyang Bay on the southern coast of the Korean Peninsula in 2014, which caused massive damage to the marine environment in the coastal region [8].
The coastline of the east coast of the Korean Peninsula is close to a straight line, while its western coast is composed of several small islands and a ria coast with a shallow and complicated sea floor topography ( Figure 1). As shown in Figure 1, there is a well-developed warm current system called the Tsushima Warm Current at the eastern coast while the western coastal region is characterized by ocean currents having opposite directions with dominant seasonality. These complicated environmental factors and a variety of waters have led to an abundant diversity of species and aquaculture in the coastal areas. These elevate the necessity and validity of coastal management for monitoring the locations of ships and retrieving ship-related information. Therefore, the development of an advanced remote sensing methodology based on satellite or aircraft observations is essential for the real-time monitoring of vessels contributing to heavy maritime traffic [9][10][11]. The objectives of this study are as follows: 1) to detect pixels corresponding to vessels based on the statistical spectral similarity between the hyperspectral data and in situ spectral measurements, 2) to develop a ship-detection algorithm for an optical image and apply it to high-resolution satellite data, 3) to develop algorithms for detecting ships in SAR images by using the statistical characteristics of the backscattering cross section within multiple moving windows, and 4) to compare and discuss the obtained results for the purpose of realizing sustainability at the coastal region.

Hyperspectral Data
To develop and verify ship-detection algorithms for hyperspectral images, an AVIRIS ultraspectral sensor image, obtained by NASA/JPL in the USA, was used in this study [12]. It has 224 channels at a wavelength range of 400-2500 nm with a bandwidth of approximately 10 nm. The spatial resolution of the hyperspectral image depends on the altitude of the aircraft containing AVIRIS and is approximately 20 m (4 m) for an altitude of 11 km (1.9 km) [13]. The hyperspectral image data used herein has a spatial resolution of 16.7 m. Figure 2b shows an RGB composite image using three bands, of 29 (644.86 nm), 20 (557.77 nm), and 12 (480.38 nm) out of 224 channels, obtained at the coast of the USA on April 14, 2014, as indicated in the black box off the western coast. An enlarged image of the seawater, marked using a gray box, presents a specific ship of green color. Several algorithms are applied to identify the ship in the hyperspectral image with hundreds of wavelength channels.

High-Resolution Optical Image
To apply a method for detecting vessels from an optical image, a high-resolution image obtained from KOMPSAT-2 (Korea Multi-Purpose SATellite-2), launched by the Korea Aerospace Research Institute (KARI) on July 28, 2006, was used in this study. The satellite observes the sea surface with a coverage of 15 km and has a total of four multi-spectral sensors and one panchromatic sensor. The sensor has a blue band of 450-520 nm, a green band of 520-600 nm, a red band of 630-690 nm, and a near-infrared band of 760-900 nm with a spatial resolution of 4 m. The Kompsat-2 satellite image used in this study was acquired at Gwangyang Bay at the southern coast of the Korean Peninsula (black box in Figure 3a

SAR Image
To verify a ship detection algorithm using SAR images, data obtained from the Sentinel-1A/ B satellite, launched by the European Space Agency, with C-Band (5.405 GHz) SAR are used. These satellites are advantageous for monitoring the global environment owing to their wide observation width as compared with those of the previous SAR satellites that had a relatively small coverage. The Sentinel-1A and Sentinel-1B satellites were launched on April 3, 2014 and April 25, 2016, respectively, and have been observing the ocean surface at an altitude of approximately 700 km with a revisit time of 12 days. Of the four modes of the Sentinel-1A / B observations, called stripmap mode, interferometric wide-swath mode (IW), extra-wide-swath mode, and wave mode, the IW mode with dual polarization was selected for this study while considering its spatial resolution and spatial coverage with a relatively wide swath. Figure 4 shows a series of spatial distributions of backscattering coefficients (in decibel units) of Sentinel-1B IW-mode SAR images observed in the seas off the southern and eastern coasts of the Korean Peninsula. The five eastern SAR images were obtained from the northeast to the southwest following a descending orbit for a temporal period

In-situ Hyperspectral Measurement
To apply the ship-detection algorithm to the hyperspectral images, in situ hyperspectral measurements covering a wide range of wavelengths are required as reference data. This study used the FieldSpec 4 Wide-Res Field Spectroradiometer to measure the radiance of ships with 2,151 spectral wavelengths in the 350-2500-nm wavelength range. The sampling interval is approximately 1.4 nm at a wavelength of less than 1,000 nm and 1.1 nm at wavelengths greater than 1,000 nm. Figure 5a presents a schematic representation of the field measurement method using the spectroradiometer. It was mounted on a ship during the daytime when sunlight was present, and the radiance was measured by directing the beam at a 30° direction based on the target surface of the object (Figures 5b and 5c). A total of five vessels in green color were selected to compare the hyperspectral values corresponding to the green ship with the in situ spectral measurements. The spectral radiance observation of the surface of the ship deck was performed from August 11 to 29, 2017. For convenience, each ship was marked as S1, S2, S3, S4, and S5, respectively. The radiance measurements were acquired three times at each spot of the ships and an average of the three values was calculated to be used in the hyperspectral data classification method. Figure 5d shows an example of the spectral radiance measured at the surface of the five vessels during the observation period. Figure 5. (a) Schematic of spectral radiances, in-situ measurements of spectral radiance (b) near the sea surface and (c) near the ship deck using an ASD Inc. spectroradiometer, and (d) an example of observed radiance values for five vessels (S1 to S5) as a function of wavelength.

Normalized Irradiance
Hyperspectral measurements of the radiance emitted from the ship are varied as a function of several wavelengths as well as sunlight conditions or the measured incident angle. Therefore, it should be normalized in order to facilitate a comparison with the observed quantity under diverse conditions. One of the simplest methods used for the normalization of the measured radiance was estimating the ratio of the radiance of each wavelength divided by the sum of the square of the radiance of all the channels as follows [14]: where is normalized radiance of -th wavelength band, is the radiance based on in-situ measurements, and is the total number of hyperspectral bands -224 herein.

Spectral Similarity Derivation
As hyperspectral measurements contain radiance values at a few hundreds of bands, it is possible to calculate the similarity based on the total spectral shape without using individual band values. As candidates of spectral matching methods, the following five representative methods are adopted in this study-spectral distance similarity (SDS), spectral correlation similarity (SCS), spectral similarity value (SSV), spectral angle mapper (SAM), and spectral information divergence (SID)-as indicated in Equations. (2) to (9).
In the SDS method, the similarity is measured by estimating the spectral distance between a target spectrum and a reference spectrum [15]. In the SCS method, the spectral correlation coefficient, ranging from 0 to 1, is used between the target spectrum and the reference spectrum as a measure of similarity [16].In the SSV method, both the spectral distance between the target spectrum and the Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 29 August 2018 doi:10.20944/preprints201808.0504.v1 reference spectrum and the correlation coefficient are used as a measure of similarity [17]as shown in (4).
The fourth method comprises the use of SAM as a measure of similarity between the target spectrum and the reference spectrum based on the spectral angle difference between the reference spectrum and the target spectrum in the two-dimensional coordinate system, as described in [18] As the angle approaches 0°, the similarity of the two spectra becomes greater. If the angle approaches 90°, the similarity of the two spectra is determined to be low.
In the fifth method, SID, the probability distribution distance between the target spectrum and the reference spectrum is used as another measure of similarity. In this method, it is assumed that each pixel is an arbitrary random variable and the separation between two spectra is measured [19]. The object and reference spectra are divided by the total sum to obtain the probability vector, and the relative entropy is then summed. For the object and reference spectrum in Equation (6), Equation (7) represents the probability vector with values between 0 and 1, and Equations. (8) and (9) are the relative entropy and sum of each other of (7). As the relative entropy is small, the probability of ship detection is evaluated to be high.

Ship Detection from High-Resolution Optical Image
For the detection of ships in a high-resolution optical satellite image, the maximum likelihood classifier (MLC) method is used under the assumption that the statistics for each class in each band are normally distributed. The probability that a given pixel belongs to a specific class is calculated by considering dispersion and covariance between the mean value of the pre-classified class and a pixel to be classified in the multispectral space [20]. In this study, each pixel of the optical image is assigned to one of the classes that has the highest probability over the predetermined threshold.

Ship Detection on SAR Image
The SAR transmits microwaves and receives energy reflected from the sea surface. In the case of a calm sea surface without features, single scattering is dominant, and the reflected energy is reduced. In contrast, if there is a ship in the ocean, both double scattering and volume scattering are predominantly caused by the scatterers such as the upper and side surfaces of the ship, or the inside of the ship. Owing to these scatterings, the energy reflected to the satellite sensor tends to increase. Therefore, the backscattering coefficients of the pixels corresponding to the ship are much higher than those of the surrounding sea surface in the SAR image. Based on these characteristics, three representative methods such as the global threshold method, adaptive threshold method, and neural network method have been applied for detecting vessels [21][22][23]. Owing to a limitation of the global threshold method in terms of the incidence angle, the adaptive threshold method was applied to the ship detection in this study [24].
The adaptive threshold method called as a constant false alarm rate (CFAR) algorithm uses three windows (the target window, guard window, and background window) surrounding a central pixel to be identified as a ship or a non-ship pixel based on local statistics of backscattering coefficients within each window (Figure 8a). The average and standard deviation are calculated from the background window and not the guard window, and the average in the target window is calculated beforehand. Using the calculated mean and standard deviation, the detection parameter d is calculated according to the following equation: (a) (b) Where ̅̅̅ is the mean of the target window, ̅̅̅ is the mean of the background window, and B is the standard deviation of the background window. If is greater than the threshold value, the central pixel within the target window is determined to be a ship; otherwise, it is determined to be an ocean pixel (Figure 8b). The target window accompanied by the two windows were moved oneby-one in the azimuth direction and the range direction to find the pixels with statistical characteristics similar to the ship.

Hyperspectral-based Ship Monitoring
Using the five methods of the spectral characteristic matching algorithm described earlier, the similarity between in situ spectral measurements of green ships and hyperspectral radiance data was estimated at every pixel to classify the ship pixels and other oceanic pixels corresponding to non-ship pixels. Figure 9a shows an RGB image composited from three bands of AVIRIS, which clearly shows the green ship with a white deck at the lower end of the ship. The spatial distribution of the SDS results, based on the Euclidean distance difference, presents relatively small values of less than 0.008 as compared with the edge of the vessel of approximately 0.01. The pixels corresponding to ocean pixels contain high values that are greater than 0.014, which means the pixels have vastly different statistical characteristics from those of the ship. Figure 9c exhibits the results of the SCS method with spectral correlation coefficients in a range of the whole wavelengths of the hyperspectral data. The pixels corresponding to the ship tend to contain high coefficients of greater than 0.8, which contrasts with relatively smaller correlation coefficients of approximately 0.65 in the neighboring pixels that are concluded to be sea pixels. The coefficients of SSV in Figure 9d are inversely proportional to those of the SCS result as it is a combination of the SDS and SCS methods. As the coefficient of the SSV method is small with a value of less than 0.2, the pixel is regarded to be a ship pixel as shown in Figure 9d. The ocean pixel has relatively high coefficients of approximately 0.4.  Figure 9e shows the result of the SAM method obtained using the angle of each band between two spectra as a measure of the spectral similarity. It is of less than 0.3 at the pixels corresponding to the surface of a ship while the ocean pixels have values greater than 0.5 (Figure 9e). The last method, SID, reveals somewhat complicated features at the ocean pixels in contrast to the relatively uniform coefficients obtained with the other methods. However, the ship pixels can be easily detected because of large differences between the ocean pixels (>0.8) and ship pixels (<0. 4). An investigation of the coefficients of each method proves that the SID method shows the highest differences (>0.6) between the ocean and ship pixels. One of conspicuous features of the SID result is the distribution of random noises without any relation to the specific pattern of the sea surface surrounding the ship. Figure 10 presents the results of the ship detection based on each of the five methods used. The ocean pixels and ship pixels have digital numbers of 1 and 0. The majority of the methods have a good ability to detect ship pixels with negligible differences particularly in the fore or aft parts of the vessel.

Optical Ship Monitoring
If the high-resolution satellite optical image is enlarged, as shown in Figure 4c, the position of the ship can be visually confirmed. However, in order to automate the ship detection procedure in near-real time along with data acquisition, a variety of technologies that can be applied objectively and in real time are required. In this study, two classes for ship and seawater are defined in advance, and all the pixels are classified into two classes of ship and ocean by adapting the MLC method using  Figure 11 shows the image of the ship detection of six vessels in the Kompsat-2 optical image. The locations of the ships have a value of zero, as marked in gray, and a value of 1, as marked in white, for the oceanic pixels. When compared with the RGB image of Figure 4, the existence of ships at the same positions can be confirmed. As only the two classes of ship and sea are used as the reference classes, the differences induced by the green and white structures inside the ship appearing in the RGB image tended to be ignored in the classification result because the surface of the vessel was assumed to be a uniform single body. Figure 11. Spatial distribution of detected vessels based on maximum likelihood classifier (MLC) method, where gray color with zero value stands for the locations of detected vessels and the white color with the number 1 corresponds to the non-vessel pixels representing ocean pixel.

SAR-based Ship Monitoring
As the optical images described earlier cannot be used observe ships or the sea surface features under poor weather conditions or on cloudy days, optical-based ship detection methods are significantly influenced by local weather conditions such as fog, clouds, and rainfall. In contrast, allweather satellite SAR images can overcome the limitation in relation to these atmospheric problems. The all-weather SAR sensor has a disadvantage in terms of the number of observations required, but once the image is acquired, the position of the ship can be detected very precisely irrespective of the existence of cloudy situations.
The ALOS-2 PALSAR-2 image was used to verify if the SAR-based ship-detection algorithm used in this study accurately detected the pixels corresponding to diverse ships. Figure 12a Figure 12a, indicates the locations of the ships presented in a white pixel with a value greater than −5dB. The ships tend to have significantly higher backscattering coefficients than the surrounding oceanic pixels in the SAR image. When a single threshold is applied as a threshold over the entire area without applying any land masking procedure, the vessel detection can fail owing to the high backscattering values of the land area, which is likely to be misidentified as a vessel. Therefore, only after the land-masking procedure is performed using the Shuttle Radar Topography Mission (SRTM) Digital Elevation Model (DEM) data, the adaptive threshold algorithm should be applied for detecting the ship. As indicated in red in Figure 12c, after performing the masking procedure of the island, a total of 22 vessels are detected near the southeastern part of the island called Sungdado (Figure 12c). One of the other differences found in the distribution of vessels is that several vessels are concentrated at the coast in the region to the south of the national border line. In the sea of North Korea, however, the vessels are rather evenly located along the coast.
To determine the maritime activities based on ship detection, we estimated the density of ships as a function of the distance from the coast. Ships tend to be within 30 km of the coast and this value decreases exponentially in the direction away from the coast (Figure 14), which implies that several human activities requiring ships have been being actively performed near the coast. The increasing trend of human economic activities in the coastal region, such as maritime trade and fishing activities, can increase the risk of potential accidents. This supports the importance of ship monitoring using all resources available from multi-satellite measurements. Therefore, the satellite SAR can play a very important role in monitoring as it has outstanding advantages in terms of high spatial resolution as well as a representative all-weather sensor. Thus, ship detection and coastal utilization determined based on SAR images can be confirmed to have numerous uses in the management of coastal regions in the future.

Conclusions
A variety of human activities, particularly over the last few decades, have resulted in increased ship traffic, which may have an influence on the sea in relation to several oceanic environmental problems such as vessel collisions, oil spills, ship groundings, anchor damage, and oily wastage. Such issues have threatened several marine species and invaluable ecosystems as well as the prosperous quality of human life, human health, and even security in coastal regions. One of the methods of reducing the significance of the impacts of human activities and contributing to sustainable growth and development in coastal regions is the effective and efficient management of coastal environments. The number of vessels in the ocean has been increasing in proportion to human requirements with respect to fisheries, shipping, international trades, etc.
This study proposes methods to detect ships using optical, hyperspectral, and SAR-based remote sensing techniques efficiently using the advantages of satellites. As a result of applying these methods to the coastal waters of the Korean Peninsula, various types of ships were accurately detected. High-resolution optical images and hyperspectral observations are significantly useful for ship management on cloud-free days. It is also expected that all the ship-related information, such as the length and width of the ship as well as its position, structure, type, and wake, can be operationally obtained for real-time purposes if optical and hyperspectral satellite images have been regularly acquired.
In addition to these, all-weather SAR images can also extensively used on cloudy days. However, SAR satellites have been used to observe the sea surface very intermittently at a significantly lower observation rate than the frequency at which land observations have been made. Even the SAR data itself has long been distributed very limitedly, which makes it difficult to utilize them in near-real time. However, recently, coastal observations such as those obtained from Sentinel-1 A/B have been increasing, and it has become easier to acquire high-quality data owing to the policies for the free distribution of data. This positive environment enables us to construct an efficient method of ship detection and devise comprehensive coastal management systems using multi-satellite data. It is expected that the multi-spectral and high-resolution ship-detection algorithms presented in this study can be useful for the detection of even small vessels along the coast. The results can be prepared in advance for coping with ship accidents and missing-vessel detection.
The majority of people on Earth have long lived close to the coast while enjoying the benefits of life near the ocean and coast. In contrast, the oceans are suffering from the diverse problems of marine litter, frequent ship accidents, oil pollution, ecosystem changes, large fluctuations in fish stocks, rising sea levels and wave heights, and devastating typhoons caused by various types of human activities. Since oceanic changes due to global warming and climate change are rapidly evolving beyond expectations, the development of scientific technologies should be continued for realizing sustainable development. In this context, it is anticipated that this this study represents a small step toward exploring the possibility of satellite data applications in coastal management around the Korean Peninsula with the objective of long-term sustainability in coastal regions.