Assessment of Marine Debris on Hard-to-Reach Places Using Unmanned Aerial Vehicles and Segmentation Models Based on a Deep Learning Approach

: It is difﬁcult to assess the characteristics of marine debris, especially on hard-to-reach places such as uninhabited islands, rocky coasts, and seashore cliffs. In this study, to overcome the difﬁculties, we developed a method for marine debris assessment using a segmentation model and images obtained by UAVs. The method was tested and veriﬁed on an uninhabited island in Korea with a rocky coast and a seashore cliff. Most of the debris was stacked on beaches with low slopes and/or concave shapes. The number of debris items on the whole coast estimated by the mapping was 1295, which was considered to be the actual number of coastal debris items. However, the number of coastal debris items estimated by conventional monitoring method-based statistical estimation was 6741 ( ± 1960.0), which was severely overestimated compared with the mapping method. The segmentation model shows a relatively high F1-score of ~0.74 when estimating a covered area of ~177.4 m 2 . The developed method could provide reliable estimates of the class of debris density and the covered area, which is crucial information for coastal pollution assessment and management on hard-to-reach places in Korea.


Introduction
Marine debris is defined as persistent solid material disposed of in marine and coastal environments [1][2][3][4]. Coastal pollution by marine debris has a devastating impact on ecosystems and on human and marine lives [5][6][7]. The solution to marine litter is likely to be found in a transition towards more sustainable ways of production and consumption, which are also promoted via the Sustainable Development Goals (SDGs) [8]. Recently, the UN sustainable development agenda presented a plan of action involving 17 SDGs, including targets to prevent and significantly reduce marine pollution of all kinds, including marine litter [8,9]. Marine debris monitoring has played a crucial role in assessing pollution characteristics for a suitable response to marine pollution.
Several studies have tried to assess coastal pollution and to estimate marine debris standing stock with conventional monitoring methods [10][11][12][13][14][15][16][17][18][19][20][21][22][23][24][25]. The approaches based on conventional monitoring methods help us to understand the pollution characteristics of marine debris. However, the conventional monitoring methods are inaccurate and expensive because they manually investigate marine debris in randomly selected narrow areas [26][27][28][29][30]. According to the conventional monitoring method of Korea since 2018, a transect of a 100 m width is divided by a sub-transect of a 5 m width, and then, four sub-transects are randomly selected among them [30]. The number and weight of the debris items distributed in the four sub-transects are manually investigated by local private organizations. Although there are differences in the width of transect for each country, the conventional methods have manually investigated the number of debris items in selected narrow areas [31]. In addition, the conventional monitoring methods are difficult to use for the investigation of hard-to-reach places such as uninhabited islands, rocky coasts, and seashore cliffs.
To overcome the disadvantages of the conventional monitoring methods, many studies have recently tried to assess the debris quantity, type, and distribution with unmanned aerial vehicles (UAVs) and machine/deep learning technologies [32][33][34][35][36][37][38][39][40]. The aerial survey using UAVs has been recognized as a suitable and effective low-cost image acquisition platform for high-resolution monitoring [41]. Moreover, machine/deep learning technologies in marine science have great potential for the assessment of marine pollution. For assessing the pollution level of marine debris, it is important to estimate its amount, distribution, and area coverage. Some studies have assessed the areas polluted by marine debris using aerial survey and image processing techniques [42,43]. However, there is still a lack of crucial information to characterize the marine debris on hard-to-reach places.
In this study, we assessed and characterized the coastal pollution of debris distributed on the whole coast of an uninhabited island using a deep learning-based segmentation model with the images obtained by aerial surveys. The images were merged without overlapping, and the number density of the debris items distributed on the whole coast was visualized by heatmap. Then, the number of debris items estimated by the mapping method was compared with that estimated by conventional monitoring method-based statistical estimation. Moreover, a deep learning-based image segmentation model was used to estimate the area covered by coastal debris. The image segmentation model classified debris items into six classes and quantitatively estimated the area covered by them. The methods developed in this study could provide essential information for the assessment and management of the marine debris, especially on the hard-to-reach places in the Republic of Korea.

Study Area
Seokdae-do (-do means an island) is an uninhabited island with a rocky coast and a seashore cliff. The island is located in the region west of the Korean peninsula and has a coastline composed of sands, pebbles, and rocks, as shown in Figure 1. The island has a coastline~2.1 km long and a total area of~128,036.6 m 2 . The island is one of the national beach-debris monitoring sites of Korea. Most of the marine debris on the beach of Seokdae-do must be from the ocean because the island is an uninhabited island. In addition, aquaculture farms are distributed around the island, and Muchangpo Beach, where there is a leisure beach crowded with tourists in the summer, is nearby, as shown in Figure 1. Ninety-two images extracted from the video were used for the datasets of the image segmentation model. Each image had a dimension of 4096 × 2160 pixels.  Figure 2 shows the workflows for mapping the distribution of coastal debris ( Figure  2a) and for estimating the class of the covered area (Figure 2b). Using the images by the

Aerial Surveys Using Unmanned Aerial Vehicle (UAV)
Aerial surveys using a UAV were conducted twice with different height and photographing angles. A drone (Phantom 4 Pro V2.0, DJI, Shenzhen, China) with 20 megapixels (MP) camera took the pictures and the video. During the first survey, the drone took pictures at a 60 m altitude with a camera angle of 90 • (perpendicular to the water surface). The 203 images collected by the first aerial survey were used to map the number (i.e., the number of debris items per unit grid (10 m × 10 m)). However, the images taken by the first survey were not good enough to estimate the area covered by coastal debris with high accuracy. So, a second aerial survey was conducted to obtain high-resolution image data. Because the resolution of an image taken at a lower altitude is higher than that taken at a higher altitude under the same camera conditions, we therefore took a video along the coastline at a 40 m altitude with a camera angle of 45 • to obtain high-resolution images. Ninety-two images extracted from the video were used for the datasets of the image segmentation model. Each image had a dimension of 4096 × 2160 pixels. Petersburg, Russia) to align the images acquired by aerial survey. To prevent the duplication of a debris item, the overlapped area between each image was removed in the merging process of the software. The debris items were labelled, counted, and classified into six categories, including Plastic, Styrofoam, Metal, Rubber, Fishing gear, and Unspecified. Then, the characterized debris items were visualized and mapped as a heatmap.   In our previous study, we carried out an investigation to characterize beach debris using an object detection model (i.e., YOLOv5) [44]. The object detection model showed a good performance in detecting and classifying beach debris; however, that model could not estimate the debris classes of the covered area [44]. To tackle the problem, we used the image segmentation model to estimate the debris classes of the area covered in this study. The red dot box of Figure 2b shows the image analyzing process in which the segmentation model classified the items distributed on the whole coast into six classes and estimated their coverage area. The covered area of debris items means the projected area of the items. Ninety-two images extracted from a video by the second aerial survey with a 40 m altitude and a 45 • photographing angle were divided into 4600 patch images with 512 × 512 pixels, and the patch images were used to train the segmentation model. For labeling the debris items, a commercial geographic information system (GIS) program (i.e., ArcMap 10.7, Esri, CA, USA) was used. In each patch image, a polygon was created by connected vertices along the outer boundary of the debris item. Each polygon was marked as a debris item to be distinguished from the background in the images. For the training of the segmentation model, the labeled items were classified into six classes, including Plastic, Styrofoam, Metal, Rubber, Fishing gear, and Unspecified. Moreover, a pair of images consisting of the original patch image and a labeled image (i.e., ground truth) was configured as one dataset. Then, the trained segmentation model could classify the debris items and estimate their pixels with each class. The covered areas of each item were finally calculated by multiplying the number of pixels and the average area per unit pixel (2.58 × 10 −5 m 2 ), as shown in Table 1.

Model Performance
We used the U-Net model based on a fully convolutional network (FCN) as an image segmentation model to classify the debris items and to estimate the debris classes of the covered area. The U-Net is a highly symmetric U-shaped architecture where skip connections are used to directly link the output of each level from the encoder (contracting path) to the corresponding level of the decoder (expansive path) [45]. The model transfers the entire feature map to the corresponding decoders and concatenates them with the upsampled decoder feature maps. The U-Net model showed a high performance even when the amount of training data was insufficient.
The model performance was evaluated on the test set using the F1-score (the harmonic mean of precision and recall), which is a quantitative metric useful for imbalanced training data [46]. The F1-score shows the harmonic mean of precision and recall, where the score reaches its best value at 1 and its worst value at 0. In addition, class-balanced (CB) loss function was used to reduce the effect of the data imbalance on the model, which is a method of weighting samples to fit a given data distribution [47]. The CB loss function gives a high weight to the minor class and a low weight to the major class. Moreover, we used a data augmentation method to compensate for the limited data. There are various data augmentation techniques, such as geometric transformations, color space augmentations, kernel filters, mixing images, random erasing, feature space augmentation, adversarial training, generative adversarial networks, neural style transfer, and meta-learning [48]. For the segmentation model used in this study, we applied position augmentation (such as flipping, rotation, and translation) and color augmentation (such as brightness, contrast, and saturation adjustment) to our model to compensate for the limited data.
Using the R algorithm, a transect with a 100 m width and its 20 sub-transects of a 5 m width were automatically selected, and the number of debris items in the subtransects was also counted automatically. Then, the number of debris items that was statistically expanded with the length of the whole shoreline was compared with that by the mapping method. Figure 3 shows the number of classified/counted debris items in the merged aerial survey image. The debris items distributed on the coast were classified into six classes; Plastic, Styrofoam, Metal, Rubber, Fishing gear, and Unspecified. The total number of debris items was 1295. The number of Styrofoam items was found to be the highest, with 989 items (~76.4%), followed by Plastic, Unspecified, and Fishing gear, with 130 (~10.0%), 128 (~9.9%), and 34 (~2.6%) items, respectively. Rubber and Metal had 8 (~0.6%) and 6 (~0.5%) items, respectively. data augmentation techniques, such as geometric transformations, color space augm tions, kernel filters, mixing images, random erasing, feature space augmentation, a sarial training, generative adversarial networks, neural style transfer, and meta-lea [48]. For the segmentation model used in this study, we applied position augmen (such as flipping, rotation, and translation) and color augmentation (such as brigh contrast, and saturation adjustment) to our model to compensate for the limited dat Using the R algorithm, a transect with a 100 m width and its 20 sub-transects m width were automatically selected, and the number of debris items in the sub-tran was also counted automatically. Then, the number of debris items that was statist expanded with the length of the whole shoreline was compared with that by the map method. Figure 3 shows the number of classified/counted debris items in the merged survey image. The debris items distributed on the coast were classified into six cl Plastic, Styrofoam, Metal, Rubber, Fishing gear, and Unspecified. The total number bris items was 1295. The number of Styrofoam items was found to be the highest, wit items (~76.4%), followed by Plastic, Unspecified, and Fishing gear, with 130 (~10.0% (~9.9%), and 34 (~2.6%) items, respectively. Rubber and Metal had 8 (~0.6%) and 6 (~ items, respectively.   Figure 4a. The average dens debris items was 0.07 counts·100 m −2 . The proportions of the low-density (0.01 counts·100 m −2 ), medium-density (0.21-0.40 counts·100 m −2 ), and high-density (0.41 counts·100 m −2 ) items were 90.0%, 8.5%, 0.5%, respectively. The areas of low-density widely distributed over the whole coast, but the areas of the highest density were lo at Site 1 (0.53 counts·100 m −2 ) and at Site 2 (0.52 counts·100 m −2 ), respectively, as indi in Figure 4. The number density of the debris items was visualized as a heatmap, as s   Figure 4a. The average density of debris items was 0.07 counts·100 m −2 . The proportions of the low-density (0.01-0.20 counts·100 m −2 ), medium-density (0.21-0.40 counts·100 m −2 ), and high-density (0.41-0.60 counts·100 m −2 ) items were 90.0%, 8.5%, 0.5%, respectively. The areas of low-density were widely distributed over the whole coast, but the areas of the highest density were located at Site 1 (0.53 counts·100 m −2 ) and at Site 2 (0.52 counts·100 m −2 ), respectively, as indicated in Figure 4. The number density of the debris items was visualized as a heatmap, as shown in Figure 4b. The heatmap shows simultaneously the number density of the debris of the whole coast with the location and the morphological characteristics of the island.

Covered Area of Debris Items with Class
Four thousand six hundred patch images were generated from the ninety-two images obtained using the second aerial survey, and the patch images were used as a dataset of the segmentation model, as shown in Figure 2b (the red dot line). The datasets were split into three parts: the training set (70%), the validation set (20%), and the test set (10%). Figure 5 shows the F1-score of the segmentation model developed in this study. The average F1-score is 0.74 (red dotted line in Figure 5). The F1-scores of Plastic (0.93), Styrofoam (0.97), and Metal (0.86) were higher than the average F1-score, while those of Rubber (0.26), Fishing gear (0.72), and Unspecified (0.69) were lower than the average F1score. Figure 6 shows examples of the image data, including patch image, ground truth, and classification result. The patch image is the image segmented by 512 × 512 pixels from the original image acquired by aerial survey. Ground truth means a "true" image labeled with the six classes. Moreover, the classification result evaluated by the segmentation model was labeled as Classification. The class of debris items was identified by color. The trained model estimated the number of pixels with the six classes. Then, the covered area was calculated by multiplying the number of pixels and the average area per unit pixel, as shown in Table 1. Figure 7 shows the number of pixels and the covered area with the debris class, as estimated by the segmentation model. The total area covered by the debris items was estimated to be ~177.40 m 2 . The area covered by Styrofoam was the largest area, with ~84.28 m 2 , which accounted for ~47.5% of the total area covered by marine debris. Plastic was the second largest area, with ~60.21 m 2 , followed by Unspecified, Fishing gear, Metal, and Rubber, with ~16.64 m 2 , ~7.60 m 2 , ~4.67 m 2 , and ~4.00 m 2 , respectively.

Covered Area of Debris Items with Class
Four thousand six hundred patch images were generated from the ninety-two images obtained using the second aerial survey, and the patch images were used as a dataset of the segmentation model, as shown in Figure 2b (the red dot line). The datasets were split into three parts: the training set (70%), the validation set (20%), and the test set (10%). Figure 5 shows the F1-score of the segmentation model developed in this study. The average F1-score is 0.74 (red dotted line in Figure 5). The F1-scores of Plastic (0.93), Styrofoam (0.97), and Metal (0.86) were higher than the average F1-score, while those of Rubber (0.26), Fishing gear (0.72), and Unspecified (0.69) were lower than the average F1-score.     Table 1. Figure 7 shows the number of pixels and the covered area with the debris class, as estimated by the segmentation model. The total area covered by the debris items was estimated to be~177.40 m 2 . The area covered by Styrofoam was the largest area, with 84.28 m 2 , which accounted for~47.5% of the total area covered by marine debris. Plastic was the second largest area, with~60.21 m 2 , followed by Unspecified, Fishing gear, Metal, and Rubber, with~16.64 m 2 ,~7.60 m 2 ,~4.67 m 2 , and~4.00 m 2 , respectively.    Figure 8 shows the average area covered by a single debris item of each class distributed on Seokdae-do Island. The area covered per unit item of Metal was the highest value, at 0.78 m 2 ·count −1 , but the number was the lowest, as shown in Figure 3. This means that it has a remarkably larger size than the other classes. Interestingly, one of the six metal   Figure 8 shows the average area covered by a single debris item of each class distributed on Seokdae-do Island. The area covered per unit item of Metal was the highest value, at 0.78 m 2 ·count −1 , but the number was the lowest, as shown in Figure 3. This means that it has a remarkably larger size than the other classes. Interestingly, one of the six metal items was the metal frame of the aquaculture farm, which took up most of the total covered area of Metal. The area covered per unit item of Styrofoam was significantly lower, at 0.09 m 2 ·count −1 , although the number of items, 989, was the highest, as shown in Figure 3.
Most of the Styrofoam items on the coast were Styrofoam buoys, which were almost broken, so the covered area per unit item was relatively small. In addition, the Styrofoam, Metal, and Fishing gear found on the island had to be from the aquaculture farms. Their total number and area covered were 1029 (~79.5%) and 411.7m 2 (~80.7%), respectively. This means that the marine-based sources contributed more than the land-based sources to the pollution by marine debris on Seokdae-do Island.  Figure 8 shows the average area covered by a single debris item of each class distributed on Seokdae-do Island. The area covered per unit item of Metal was the highest value, at 0.78 m 2 ·count −1 , but the number was the lowest, as shown in Figure 3. This means that it has a remarkably larger size than the other classes. Interestingly, one of the six metal items was the metal frame of the aquaculture farm, which took up most of the total covered area of Metal. The area covered per unit item of Styrofoam was significantly lower, at ~0.09 m 2 ·count −1 , although the number of items, 989, was the highest, as shown in Figure  3. Most of the Styrofoam items on the coast were Styrofoam buoys, which were almost broken, so the covered area per unit item was relatively small. In addition, the Styrofoam, Metal, and Fishing gear found on the island had to be from the aquaculture farms. Their total number and area covered were 1029 (~79.5%) and 411.7m 2 (~80.7%), respectively. This means that the marine-based sources contributed more than the land-based sources to the pollution by marine debris on Seokdae-do Island.  Figure 9 shows the conventional method-based statistical estimation of the debris on the whole coast of Seokdae-do Island. The average number of debris items in four randomly selected sub-transects was 64.2, and the standard deviation was the unreasonable value of ±18.7. The total number of debris items on the whole coast estimated by the conventional monitoring method-based statistical estimation was 6741(±1960.0). However,  Figure 9 shows the conventional method-based statistical estimation of the debris on the whole coast of Seokdae-do Island. The average number of debris items in four randomly selected sub-transects was 64.2, and the standard deviation was the unreasonable value of ±18.7. The total number of debris items on the whole coast estimated by the conventional monitoring method-based statistical estimation was 6741(±1960.0). However, the number of debris items surveyed by the mapping method was 1295. The results showed that the conventional monitoring method-based statistical estimation gave an overestimation that was~5.2 times higher than that of the mapping method.

Comparison of Mapping Method Using UAV and Conventional Monitoring Method
Sustainability 2022, 14, x FOR PEER REVIEW 9 of 14 the number of debris items surveyed by the mapping method was 1295. The results showed that the conventional monitoring method-based statistical estimation gave an overestimation that was ~5.2 times higher than that of the mapping method.

Comparison of Mapping Method Using UAV and Conventional Monitoring Method
There are various conventional monitoring methods for beach debris [26][27][28][29][30][31]49]. The conventional monitoring methods differ slightly from each other, but they have a common feature which is that the survey area is not the whole beach area. The conventional moni-

Comparison of Mapping Method Using UAV and Conventional Monitoring Method
There are various conventional monitoring methods for beach debris [26][27][28][29][30][31]49]. The conventional monitoring methods differ slightly from each other, but they have a common feature which is that the survey area is not the whole beach area. The conventional monitoring method-based statistical estimation could highly overestimate, or also highly underestimate, the amount of debris. Specifically, when the number of coastal debris items investigated in the four sub-transects was statistically expanded along the length of the whole coastline, the number of debris items increased from 64.2 (±18.7) to 6741 (±1960.0), as shown in Figure 9. This means that the number of debris items estimated by the conventional monitoring method-based statistical estimation highly depends on the randomly selected transects. Therefore, it is hard to insist that the characteristics estimated by the conventional monitoring method represent the characteristics of the debris of the whole beach.

Pollution Assessment
The topographical characteristics of the coastline could significantly affect the distribution of the marine debris. The coasts with a low slope and a wide, concave shape have high litter abundance. It is particularly well known that the slope of a coast is one of the factors that significantly influences the abundance of the coastal debris [50]. Site 2 in Figure 4a, with a low slope and a wide, concave shape, had a debris density of 52 counts·100 m −2 . Moreover, Site 1 in Figure 4a, with a concave shape composed of gravel, had the highest density with 52 counts·100 m −2 . Most of the marine debris distributed on the Seokdae-do coast was located and stacked on the coastal regions with a low slope and a concave shape, as shown in Figure 4. The technique developed in this study can be applied to assess the floating debris on the sea surface. However, a model should be established and trained using new datasets of floating debris.

Accuracy
The accuracy of the segmentation model was significantly affected by the number of datasets and the resolution of the acquired images. The lack of datasets reduces the model performance and prediction accuracy. However, the low accuracy due to the data imbalance caused by the lack of datasets could be improved by the CB loss function [46]. To compensate for the lack of datasets, we applied the CB loss function in the network model, as mentioned in Section 2.4. Table 2 shows the pixels of the labeled items and their class weight determined by the CB loss function. Among the six classes, Styrofoam had the highest value of 4,911,259 pixels, and its class weight had the lowest value of 9.93, while Rubber had the lowest value of 25,431 pixels, and its class weight had the highest value of 1917.29. The CB loss could improve the performance of the image segmentation model, but it is difficult to completely overcome the lack of datasets. Interestingly, the number of Metal items found in Seokdae-do Island was six, which was similar to that of Rubber, as shown in Figure 3, but the F1-score of Metal was~0.86, which was significantly higher than that of Rubber (~0.26), as shown in Figure 5. The average area covered by Metal was relatively higher than that of Rubber, as shown in Figure 8; moreover, the pixel count of labeled Metal was~12 times higher than that of Rubber, as shown in Table 2. It is notable that one of the six metal items found on the coast of Seokdae-do was a very large steel frame from the aquaculture farm, which accounted for most of the total metal pixels. This means that the accuracy of the image segmentation model can be affected more by an item's size and its area of coverage than when the number of items is used as the dataset.
The prediction accuracy of the deep learning-based image analysis model is also significantly affected by the resolution of the image data. Previous studies advise acquiring data from a lower altitude to improve resolution [51,52], but there was a limitation of altitude due to seashore cliffs and trees on the island. Therefore, we tried to improve the resolution of the image data by an aerial survey conducted at a 40 m altitude with a camera angle of 45 • . However, the covered area still includes errors due to the camera angle and the image resolution. Figure 10 shows the images taken under the same conditions of the aerial survey when using samples of known size. The number of pixels of each sample was counted, and the area covered per unit pixel of each sample was calculated, as shown in Table 1. The mean and standard deviation of the area per unit pixel of the tested samples were~2.58 × 10 −5 m 2 and~0.79 × 10 −5 m 2 , respectively. The mean value of area per unit pixel of the test samples was used to calculate the area coverage of the debris items on Seokdae-do Island. The standard deviation accounted for~31% of the mean, which includes the errors in the projected area by camera angle and the number of pixels by image resolution.

Conclusions
It is difficult to assess the amount, distribution, and type of marine debris, especially on hard-to-reach places such as uninhabited islands, rocky coasts, and seashore cliffs. In this study, to overcome the difficulties, we investigated the mapping technology using the

Conclusions
It is difficult to assess the amount, distribution, and type of marine debris, especially on hard-to-reach places such as uninhabited islands, rocky coasts, and seashore cliffs. In this study, to overcome the difficulties, we investigated the mapping technology using the combination of images obtained by UAV (i.e., a drone) and a segmentation model based on a deep learning approach. A mapping method was developed and verified on Seokdae-do Island, which is an uninhabited island with rocky coasts and seashore cliffs. The number of debris items distributed on the coast was visualized as a heatmap of the merged aerial image. The total number of debris items on the whole coast was discovered to be 1295, and the average debris density was~0.07 counts·100 m −2 . Most of the marine debris was concentrated on beach areas with a low slope and a concave shape. The amount of whole-coastal debris estimated by the mapping method was compared with that by the conventional monitoring method-based statistical estimation. The results show that the statistical estimation based on the conventional monitoring method gave an overestimatioñ 5.2 times higher than that of the mapping method. Moreover, the area covered was estimated by a segmentation model in conjunction with images obtained by a drone. The result of the segmentation model shows a relatively high F1-score of~0.74. In particular, the F1-scores for Styrofoam and Plastic, which are the most serious pollutants as they are a cause of microplastic, had the relatively high values of~0.97 and~0.93, respectively. The assessment method developed in this study could provide reliable estimates of the debris density and the area covered according to the class of item, which is crucial information for coastal pollution assessment and the management of hard-to-reach places.