Integration of Superpixel Segmentation, Convolutional Neural Networks and Vision Transformers for Automatic Benthic Habitats Classification

Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have achieved significant success in various computer vision applications, including the classification of high-resolution imagery. However, a notable limitation of these deep learning approaches is their tendency to inadequately preserve the precise edges and shapes of target objects. In contrast, Object-Based Image Analysis (OBIA) offers a methodology that emphasizes the preservation of object boundaries by segmenting images into meaningful objects. Combining CNNs and ViTs with OBIA leverages the feature extraction capabilities of these deep learning algorithms and the boundary-preserving advantages of OBIA, leading to enhanced classification accuracy and improved delineation of object boundaries in high-resolution images. Still, the main challenge for combining these methods lies in effectively aligning the irregularly shaped image objects produced by OBIA with the regular image patches required by CNNs and ViT architectures. In this study, we propose a novel approach that integrates superpixel segmentation with CNNs and ViTs for the automatic classification of benthic habitats using high-resolution orthomosaic images. Initially, the Simple Linear Iterative Clustering (SLIC) algorithm was applied to segment the high-resolution orthomosaic images into superpixels. Subsequently, the central points of the resulting superpixels were utilized to generate square image patches. These patches performed as inputs for ConvNeXt-Base and EfficientNet-B0 pre-trained CNNs to extract fine-grained features and Dinov2 ViTs to extract high-level features. Then, a Support Vector Machine (SVM) classifier was trained using these attributes to classify benthic habitats. Eventually, the classification label derived from the SVM defined the class of each superpixel segment. This method achieved an average overall accuracy of 0.96 in classifying benthic habitats. Overall, we demonstrate that combining CNNs, ViTs, and superpixel segmentation is an effective approach to benthic habitats classification, providing accurate high-resolution maps of heterogeneous reef environments.