Automatic Semantic Segmentation of Benthic Habitats Using Images from Towed Underwater Camera in a Complex Shallow Water Environment

: Underwater image segmentation is useful for benthic habitat mapping and monitoring; however, manual annotation is time-consuming and tedious. We propose automated segmentation of benthic habitats using unsupervised semantic algorithms. Four such algorithms—-Fast and Robust Fuzzy C-Means (FR), Superpixel-Based Fast Fuzzy C-Means (FF), Otsu clustering (OS), and K-means segmentation (KM)—-were tested for accuracy for segmentation. Further, YCbCr and the Commission Internationale de l’ É clairage (CIE) LAB color spaces were evaluated to correct variations in image illumination and shadow effects. Benthic habitat ﬁeld data from a geo-located high-resolution towed camera were used to evaluate proposed algorithms. The Shiraho study area, located off Ishigaki Island, Japan, was used, and six benthic habitats were classiﬁed. These categories were corals ( Acropora and Porites ), blue corals ( Heliopora coerulea ), brown algae, other algae, sediments, and seagrass ( Thalassia hemprichii ). Analysis showed that the K-means clustering algorithm yielded the highest overall accuracy. However, the differences between the KM and OS overall accuracies were statistically insigniﬁcant at the 5% level. Findings showed the importance of eliminating underwater illumination variations and outperformance of the red difference chrominance values (Cr) in the YCbCr color space for habitat segmentation. The proposed framework enhanced the automation of benthic habitat classiﬁcation processes.


Introduction
Benthic habitats and seagrass meadows are biologically complex and diverse ecosystems with enormous ecological and economic value [1]. These ecosystems contribute substantially to nutrient cycling and nitrogen and carbon sequestration, create a natural barrier for coastal protection, and provide income to millions of people [2,3]. These ecosystems have suffered worldwide decline over the past three decades [4]. For instance, about 80% and 50% of coral cover were lost in the Caribbean and the Indo-Pacific, respectively, during this time [5]. Our understanding of this rapid degradation is limited by the global lack of benthic habitat mapping and seagrass meadow data [6]. Such data are vital for accurate assessment, monitoring, and management of aquatic ecosystems, especially in light of current global change, in which numerous stressors act simultaneously [7].
Underwater video and image surveys using scuba diving, towed cameras, and ROVs are valuable remote-sensing techniques for monitoring changes and distribution of benthic habitats with high spatial and temporal resolution [8][9][10]. Despite the merits of these platforms, i.e., low cost, non-destructive sampling, and fast data collection [11], they suffer from major drawbacks for underwater image analysis. First, inadequate illumination and variable water turbidity cause poor image quality [12]. Second, the variable physical studies were performed to overcome the main challenges of towed underwater imagesillumination variation, blurring, and light attenuation [36]. Improving the accuracy of towed underwater images will allow the processing of spatial and temporal scales for benthic habitat research to close the gap between the collected and analyzed. Consequently, the development of automatic benthic habitat semantic segmentation frameworks that can be applied to complex environments with reliable speed, cost, and accuracy is still needed [37].
We present an automated framework for such segmentation. The main contributions described can be summarized: (i) we tested several automatic segmentation algorithms for unsupervised semantic benthic habitat segmentation; (ii) we demonstrated that the Kmeans clustering method outperforms other segmentation algorithms using a heterogeneous coastal underwater image dataset; (iii) we evaluated various image color spaces to overcome towed underwater image drawbacks; (iv) we show that using YCbCr color space for underwater images in shallow coastal areas accomplished superior segmentation accuracy; (iv) we demonstrate that the proposed automatic segmentation methods can be used to create fast, efficient, and accurate classified images.

Study Area
The Shiraho coast subtropical territory, Ishigaki Island, positioned south of Japan in the Pacific Ocean, was the study area chosen for the proposed framework assessment (Figure 1). This Island is rich in marine biodiversity, with shallow, turbid water, and a maximum water depth of 3.5 m [38]. Moreover, it has a heterogeneous ecosystem with various reefscapes, including hard corals, such as Acropora spp., Montipora spp., and Porites cylindrica, and blue corals, such as Heliopora coerulea, considered the largest blue ridge coral colony in the northern hemisphere. It also has a wide range of brown and other algae, as well as a variety of sediments (mud, sand, and boulders). Further, a dense Thalassia hemprichii seagrass meadow spreads across the same seafloor.

Benthic Cover Field Data Collection
Field data collection in the study area was performed during the typhoon season on 21 August 2016. Outflows from the Todoroki River tributary to the Shiraho reef were boosted by rainstorms before data acquisition. These outflows increased turbidity in underwater images obtained using a low-cost, high-resolution towed video camera (GoPro HERO3 Black Edition) [39]. The video camera was attached beneath the motorboat side just under the water surface to record shallow seabed images. Four hours of recordings were collected and video to JPG converter software was used to extract images from video files. The extracted images had two-second time intervals synchronized with GNSS surveys.

Methodology
The proposed framework for benthic habitat automatic segmentation over the Shiraho area was performed as: 1.
All images from the video to JPG converter program were divided into five benthic cover datasets with dominant habitats-brown algae, other algae, corals (Acropora and Porites), blue corals (H. coerulea), seagrass, and the sediments (mud, sand, pebbles, and boulders)-included in all images.

2.
A total of 125 converted images were selected individually by an expert and divided equally to represent the above benthic cover categories.

3.
These images included all challenging variations in the underwater images, including poor illumination, blurring, shadows, and differences in brightness.

4.
A manual digitizing was applied carefully for these images. Each image displayed two or three categories, converted to raster form using ARC GIS software.

5.
Manually categorized images were reviewed by two other experts. These experts compared manually categorized images to original images to guarantee correctness before evaluating proposed methods. 6.
A color invariant (shadow ratio) detection equation [40] using the ratio between blue and green bands was used to separate images automatically. 7.
RGB color space images that showed positive and negative values, indicating high illumination, low brightness variation, and no shadow effects, were used for segmentation. 8.
Otherwise, RGB color space images with only negative values, indicating low illumination, high brightness variation, and shadow effects, were converted to (CIE) LAB and YCbCr color spaces before segmentation. 9.
The Cr band from YCbCr color spaces represents the difference between the red component and a reference value, and the Ac band from (CIE) LAB color spaces represents the magnitude of red and green tones. Converted images were used for segmentation. 10. Proposed unsupervised algorithms were assessed for segmentation and compared to manually categorized images.

K-Means Algorithm
KM clustering is a classical unsupervised segmentation algorithm. This algorithm is simple, fast, easy to implement, and efficient and can be applied to large datasets [41]. The algorithm works by partitioning a dataset into k clusters, pre-defined by users [42]. Euclidean distance is used to compare distances between features and cluster centers and assigns each feature to the nearest center [43]. Then, variance is used to remove outlier pixels and regions with an object remover technique after segmentation. Finally, a median filter is applied to remove noise from segmented images [44]. KM algorithm steps were previously described [45].

Otsu Algorithm
The OS method is a straightforward, stable, and effective threshold-based segmentation algorithm widely used for image processing applications [46]. Initially, a greyscale his- togram is used to divide the image into two classes-background and target objects [47,48]. Next, an exhaustive search selects an optimal threshold that maximizes separability and minimizes variance within classes. This threshold is the weighted sum of variances of background and target classes [49]. Processing steps for the OS method were also previously described [48].

Fast and Robust Fuzzy C-Means Algorithm
Fuzzy C-means (FCM) [45] is a soft clustering method that depends on the fuzziness value concept and removing conditions. Each pixel may be included in two or more clusters with various degrees of membership ranging between 0 and 1 [50]. Each objective function can be described by distance aggregates between patterns and cluster centers. However, this method yielded poor results if analyzed images included outliers, noise, and imaging artifacts [51]. Thus, an FR algorithm [52] was developed to overcome the limitations of the FCM method. FR applied a morphological reconstruction operation to integrate local spatial information into images. Membership separation was also modified using local membership filtering based on spatial neighbors of the membership divide [53]. These amendments improved algorithm celerity, robustness, and efficiency.

Superpixel-Based Fast Fuzzy C-Means Algorithm
The FF algorithm [54] was proposed to improve the FCM method, particularly its computational complexity and time consumption. FF uses a morphological gradient reconstruction process to generate superpixel images. This approach is more helpful for color image segmentation. The image histogram is then computed for the produced superpixel image. Finally, the FCM method was applied using histogram parameters to produce the final segmentation [55]. The proposed algorithm is more robust and faster than the conventional FCM and yields better results for color image segmentation.
Numerous studies are available for comparing KM, OS, and FCM methods. These studies illustrate the drawbacks of OS and FCM methods and support a preference for the KM approach [56][57][58]. The OS method is a global thresholding algorithm that depends on pixel grey values, while KM is a local thresholding method. Moreover, the OS method must compute a greyscale histogram before running, and KM does not require this step. Furthermore, the OS method produced good results only if the image histogram was not affected by image noise and had a bimodal distribution [59]. Consequently, KM is faster, more efficient, avoids image noise, and can be enhanced to multilevel thresholding [57].
FCM requires various fuzzy calculations and iterations that increase its complexity and computation time compared to KM. Further, FCM is sensitive to noise, unlike KM clustering. Hassan et al. [60] discussed the differences between KM and FCM.
All benthic cover unsupervised segmentation algorithms were applied in the MATLAB environment with pre-defined numbers of clusters. The assessment of all algorithms depended on two matrices normally used for segmentation evaluation-Intersection Over Union (IOU) and F1 − score (F1) [18]: The procedures for automatic benthic habitat segmentation methods are shown in Figure 2.

Results of Automatic Segmentation of Algal Images
The comparison of segmentation methods for both highly illuminated and low contrast algal images and poorly illuminated and high contrast algal images are presented in The results of evaluating the tested segmentation algorithms using highly illuminated and low contrast algae images. Images (1-6) had two categories, and (7-11) had three categories: (a) F1score; (b) IOU.

Results of Automatic Segmentation of Algal Images
The comparison of segmentation methods for both highly illuminated and low contrast algal images and poorly illuminated and high contrast algal images are presented in Figures 3 and 4

Results of Automatic Segmentation of Algal Images
The comparison of segmentation methods for both highly illuminated and low contrast algal images and poorly illuminated and high contrast algal images are presented in

Results of Automatic Segmentation of Brown Algae Images
The comparisons of segmentation methods for both highly illuminated and low contrast brown algae images and poorly illuminated and high contrast brown algae images are presented in Figures

Results of Automatic Segmentation of Brown Algae Images
The comparisons of segmentation methods for both highly illuminated and low contrast brown algae images and poorly illuminated and high contrast brown algae images are presented in Figures 9 and 10

Results of Automatic Segmentation of Brown Algae Images
The comparisons of segmentation methods for both highly illuminated and low contrast brown algae images and poorly illuminated and high contrast brown algae images are presented in Figures  The results of evaluating the tested segmentation algorithms using highly illuminated and low contrast brown algae images. Images (1-10) had two categories, and (11)(12)(13)(14)(15)(16)(17)(18)(19) had three categories: (a) F1-score; (b) IOU.
The results of evaluating the tested segmentation algorithms using low illuminated and high contrast brown algae images. Images (1-3) had two categories, and (4-6) had three categories: (a) F1-score; (b) IOU.

Results of Automatic Segmentation of Brown Algae Images
The comparisons of segmentation methods for both highly illuminated and low contrast brown algae images and poorly illuminated and high contrast brown algae images are presented in Figures

Results of Automatic Segmentation of Blue Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast blue coral images and poorly illuminated and high contrast blue coral images are presented in Figures 15 and 16

Results of Automatic Segmentation of Blue Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast blue coral images and poorly illuminated and high contrast blue coral images are presented in Figures 15 and 16

Results of Automatic Segmentation of Blue Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast blue coral images and poorly illuminated and high contrast blue coral images are presented in Figures 15 and 16, respectively. Examples of classified images with two and three categories are presented in Figures 17-20.

Results of Automatic Segmentation of Blue Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast blue coral images and poorly illuminated and high contrast blue coral images are presented in Figures 15 and 16, respectively. Examples of classified images with two and three categories are presented in Figures 17-20.

Results of Automatic Segmentation of Blue Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast blue coral images and poorly illuminated and high contrast blue coral images are presented in Figures 15 and 16

Results of Automatic Segmentation of Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast coral images and poorly illuminated and high contrast coral images are presented in Figures 21 and 22, respectively. Examples of classified images with two and three categories are presented in Figures 23-26.

Results of Automatic Segmentation of Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast coral images and poorly illuminated and high contrast coral images are presented in Figures 21 and 22, respectively. Examples of classified images with two and three categories are presented in Figures 23-26.

Results of Automatic Segmentation of Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast coral images and poorly illuminated and high contrast coral images are presented in Figures 21 and 22, respectively. Examples of classified images with two and three categories are presented in Figures 23-26.

Results of Automatic Segmentation of Coral Images
The comparisons of segmentation methods for both highly illuminated and low contrast coral images and poorly illuminated and high contrast coral images are presented in Figures 21 and 22

Results of Automatic Segmentation of Seagrass Images
The comparisons of segmentation methods for both highly illuminated and low contrast seagrass images and poorly illuminated and high contrast seagrass images are presented in Figures 27 and 28, respectively. Examples of classified images with two and three categories are presented in Figures 29-32.

Results of Automatic Segmentation of Seagrass Images
The comparisons of segmentation methods for both highly illuminated and low contrast seagrass images and poorly illuminated and high contrast seagrass images are presented in Figures 27 and 28, respectively. Examples of classified images with two and three categories are presented in Figures 29-32.

Results of Automatic Segmentation of Seagrass Images
The comparisons of segmentation methods for both highly illuminated and low contrast seagrass images and poorly illuminated and high contrast seagrass images are presented in Figures 27 and 28, respectively. Examples of classified images with two and three categories are presented in Figures 29-32.

Results of Automatic Segmentation of Seagrass Images
The comparisons of segmentation methods for both highly illuminated and low contrast seagrass images and poorly illuminated and high contrast seagrass images are presented in Figures 27 and 28   Results presented in Figure 33 and Table 1 correspond to results in Figures 3, 9, 15, 21, and 27. Figure 33 illustrates overall accuracy for highly illuminated and low contrast images, and Table 1 shows F1-Scorse and IOU results. Results in Figure 34 and Table 2 correspond to results in Figures 4,10,16,22,and 28. Figure 34 shows overall accuracy for poorly illuminated and high contrast images, and Table 2 presents F1-Scores and IOU values. Table 3 illustrates the evaluation of the statistical significance of differences between the tested algorithms.
(a)  Results presented in Figure 33 and Table 1 correspond to results in Figures 3, 9, 15, 21, and 27. Figure 33 illustrates overall accuracy for highly illuminated and low contrast images, and Table 1 shows F1-Scorse and IOU results. Results in Figure 34 and Table 2 correspond to results in Figures 4,10,16,22,and 28. Figure 34 shows overall accuracy for poorly illuminated and high contrast images, and Table 2 presents F1-Scores and IOU values. Table 3 illustrates the evaluation of the statistical significance of differences between the tested algorithms.
(a) Results presented in Figure 33 and Table 1 correspond to results in Figures 3, 9, 15, 21 and 27. Figure 33 illustrates overall accuracy for highly illuminated and low contrast images, and Table 1 shows F1-Scorse and IOU results. Results in Figure 34 and Table 2 correspond to results in Figures 4, 10, 16, 22 and 28. Figure 34 shows overall accuracy for poorly illuminated and high contrast images, and Table 2 presents F1-Scores and IOU values. Table 3 illustrates the evaluation of the statistical significance of differences between the tested algorithms.
Results presented in Figure 33 and Table 1 correspond to results in Figures 3, 9, 15, 21, and 27. Figure 33 illustrates overall accuracy for highly illuminated and low contrast images, and Table 1 shows F1-Scorse and IOU results. Results in Figure 34 and Table 2 correspond to results in Figures 4,10,16,22,and 28. Figure 34 shows overall accuracy for poorly illuminated and high contrast images, and Table 2 presents F1-Scores and IOU values. Table 3 illustrates the evaluation of the statistical significance of differences between the tested algorithms.

Discussion
Benthic habitat surveys using towed cameras attached beneath motorboats have several advantages. First, this approach can cover large habitat areas without environmental damage. Second, the system requires only simple logistics and is economical, which increases its utility for marine applications. Third, it can be used to monitor changes in heterogeneous ecosystems annually. However, such surveys are performed under inadequate natural lighting conditions that affect image quality. Thus, segmentation algorithms need to overcome the influence of varying light and shadow.
We assessed several color spaces to eliminate the influence of varying illumination and shadows on survey images. Images were converted to color spaces, and color information was separated from luminance or brightness of images. This process decreased the impact of brightness variation on segmentation. Both Hue Saturation Value (HSV) and (CIE) XYZ color spaces were assessed for removing illumination effects. However, these color spaces yielded significantly lower F1-scores and IOU segmentation accuracies. However, the Cr band from YCbCr and the (Ac) band from (CIE) LAB color space outperformed other tested color spaces. The Cr band achieved the highest F1-scores and IOU segmentation accuracy. Segmentation results of the KM method RGB color space were added to the assessment to demonstrate the importance of using color spaces for images with variable lighting.
The reason that red-green bands outperform in processing underwater images from coastal areas is often due to agricultural runoff or impurities discharged from rivers. Therefore, both short and long wavelengths are attenuated to some extent. In these areas, blue and green wavelengths are absorbed more efficiently, leaving more red light that causes a brown hue [5]. Moreover, the common notion that red color is attenuated by water more rapidly with depth than blue and green colors only holds for oceanic waters [61].
Pixel, color, thresholding, and region-based segmentation techniques with different architectures were tested in the present study for unsupervised benthic habitat segmentation. Mean shift [62], Normalized Cut [63], Efficient graph-based [64], Ratio-contour [65], Expectation-Maximization [66], and Slope Difference Distribution [67] methods have also been evaluated. However, these latter methods yield relatively low F1-scores and IOU segmentation accuracy values.
Complete comparisons among the evaluated algorithms show maximum, minimum, and median values for tested algorithms in box plots (Figures 33 and 34). The KM method produced the highest segmentation accuracies for all categories. Lower accuracy was found for OS, FR, and FF methods, in that order. Additionally, we tried to integrate KM, OS, and FR segmentation algorithms, but this ensemble produced an accuracy that was similar to the k-means segmentation algorithm alone. The ensemble only increased segmentation accuracy slightly for a few images. Therefore, results from integrated algorithms are not presented.
Moreover, to evaluate the statistically significant differences between the proposed algorithms, the parametric paired t-test and its non-parametric alternative Wilcoxon test [68] were performed. We assessed whether the statistical differences in F1-score and IOU values between the tested approaches were significant or not. Both tests had two output values, the p-value and the h value. The p-value indicated the significance of the statistical null hypothesis test. Small p-values mean strong proof against the null hypothesis. The h value is the hypothesis test result returned as 1 or 0. If the resulted h value was 1, the null hypothesis test was rejected. This indicates a statistically significant difference between the compared algorithms and 0 otherwise. Overall, non-parametric tests are stronger and safer than parametric tests, particularly when comparing a pair of algorithms [69].
As the KM method yielded the highest F1-score and IOU accuracies, we compared the other proposed algorithms against the KM method in order to test whether the F1-score and IOU results were significantly different between the compared methods (Table 3). Based on the F1-score results, the paired t-test indicated a significant difference between the KM and OS methods. Conversely, the Wilcoxon test p-value result was slightly greater than the 0.05 significance level. As a result, the Wilcoxon test failed to reject the null hypothesis test and indicated no statistically significant difference between these methods. Moreover, based on the paired t-test and Wilcoxon test, there was no significant difference between the IOU results of KM and OS methods at the 5% significance level. On the other hand, the KM algorithm was significantly different from both FR and FF algorithms.
We assessed various benthic images with different light conditions, species diversity, and turbidity levels to avoid statistical bias. In the majority of images, the brown algae and the other algae were mixed in the same locations as were blue corals and other corals, which confused all algorithms. The most challenging segmentation task was discriminating among brown algae, other algae, and sediments. All of these habitat types display smallsized objects and similar speckled shapes (see Figure 14). Thus, brown algae and other algae, especially in poorly illuminated images, yielded the lowest IOU segmentation accuracy (see Figure 34). Discriminating blue coral segments from other coral species was also challenging because corals exhibit similar colors and shapes (see Figure 20). Conversely, the seagrass habitat was categorized with high accuracy. Considering species heterogenicity and the poor quality of tested images, accuracies obtained in the present study can be considered reliable for automatic benthic habitat segmentation. Note that comparing our segmentation accuracies with previous studies is difficult due to differences in the quality of images used, water turbidity levels in various locations, and the variety of substrates.
These automatic segmentation results encourage more research in this field. Such studies might assess the same methods with remotely operated vehicle (ROV) systems with auxiliary light sources for monitoring benthic habitats in deeper seafloor areas. The additional light source will increase image quality and segmentation accuracy. Additionally, segmentation algorithms that can decrease light variation and shadow effects [70,71] might be evaluated with the same poor-quality images. Finally, segmentation techniques can be evaluated with more complex habitat images for the same targets.
Conversely, assessing the performance of deep learning algorithms trained by the segmented images might lead to more robust benthic habitat monitoring systems. Integrating automatically segmented images with known segmented locations might allow classifying satellite images over coral reef areas.

Conclusions
This study used images from a towed underwater camera to evaluate the performance of unsupervised segmentation algorithms for automatic benthic habitat classification. Moreover, we assessed various color spaces to remove illumination variation and shadow effects from poor-quality images. The Shiraho coastal area, which includes heterogeneous blue corals, corals, algae, brown algae, seagrass, and sediment, was used as a validation site. Our results demonstrate superior performance for the Cr band from the YCbCr color space to remove light variation and shadow effects. Further, we found that the KM segmentation algorithm achieved the highest F1-scores and IOU accuracies for habitat segmentation. The significance testing of the F1-score and IOU statistics revealed that the KM model performed significantly better (at the 5% level) than FR and FF methods. Moreover, a similar performance was found between the KM and OS methods. The proposed automatic segmentation framework is fast, simple, and inexpensive for seafloor habitat categorization. Thus, a large number of benthic habitat images can be accurately categorized with minimal effort.

Conflicts of Interest:
The authors declare no conflict of interest.