In this study we used still imagery from the VENUS cabled observatory C-MAP Cyclops underwater camera, modified with a Olympus^® C8080 wide zoom (8 Megapixels, f2.4, 5x optical zoom) in order to acquire digital still photos of the surrounding benthic area at a high resolution (3,264 × 2,448 pixels; 72 dpi). The camera was mounted on a small, ROV-deployable tripod together with a Sidus SS209 pan & tilt unit (±/− 90° tilt and ±/− 180° pan). An Ikelite 200 Ws flash was used in all image acquisition sessions. To acquire imagery users logged in remotely to the shore-station computer that controlled all camera and accessory functions.

For the development of the automated image analysis protocol for Munida counting, we acquired 100 digital photos at hourly intervals from 2 to 6 December 2009, beginning at 8:00 am and ending at 10:00 am. An oblique angle of 45° was chosen for photo acquisition (i.e., at fixed pan/tilt camera coordinates) in order to encompass a seabed area of approximately 90 × 1,200 cm.

For bacterial mat estimation, image acquisition occurred with the camera oriented vertically down, picturing a seabed area of approximately 30 × 40 cm. The white bacterial mats were highly reflective and a vertically down camera angle provided a more uniform lighting field than did oblique photos.

An automated image analysis protocol was developed in MatLab 7.1. A flow-chart depicting the consecutive steps used for the tracking and classification of is presented in Figure 1. Principal problems in relation to the automation of animal detection were: (i) uneven seabed illumination by the platform lights; (ii) temporally variable background texture; (iii) water column turbidity; and finally, (iv) the presence of other, non-target species.

Figure 2 shows an example of the digital outputs for the processing steps indicated in the flowchart of Figure 1. Filtering and background corrections were preformed on original Red, Green, and Blue (RGB) digital images [Figure 2(A)]. In order to reduce the overall noise, images were filtered three times by means of a Median Filter [7 × 7]. Then, the uneven illumination conditions were adjusted by applying a Top-Hat Filter to the background (i.e., dimension 25) [Figure 2(B)]. The name “Top-Hat” originates from the shape of the filter, which is a rectangle function, when viewed in the domain of the frequencies in which the filter is constructed [29]. This filter is highly efficient for the enhancement of small objects in busy backgrounds, when a background image with no animals as reference is not available for frame subtraction. It uses a morphological method to extract the background “grain” associated with an image resulting from the subtraction between the original and filtered images. The morphological filter dimension is important. At small scales, the filter enhances particularities, while as scale increases, the filtering efficacy is reduced and becomes more general. Therefore, we selected its relative dimension by considering the size relationship between the object (i.e., Munida) and ROI size.

Segmentation and object identification (see Figure 1) were carried out on the Top-Hat filtered images. The Euclidean Distance (ED) between R and G channels was calculated for each pixel [Figure 2(C)]. This image was segmented into binary form, by applying a threshold value corresponding to the 95th percentile of the ED distribution [Figure 2(D)] and by using a Scale-Invariant Feature Transform (SIFT) (see below). This final digital product was used to identify and to extract the mean RGB value (RGBv) of each object from the original image and the Fourier Descriptors (FD).

Object identification was based on two different approaches (see Figure 1). Animal bodies were identified according to FD analysis or alternatively with SIFT. The outputs of both methods were compared in terms of efficiency.

FD were obtained from the Fourier transformation of a complex function representing Munida outlines in pixel coordinates [30]. FD analysis is based on the scalability of a curve as a closed contour describing the shape of an organism: by varying the number of Fourier coefficients used, different levels of approximation of the Fourier function to the animal profile can be obtained [31]. FD values for Munida were normalized and transformed to be size, orientation, and translation independent. Sixty descriptors were used, corresponding to 128 coefficients. Only objects having a pixel area comprising between 200 and 150,000 pixels were considered for this analysis in order to eliminate random contingent noise (i.e., turbidity, debris and non target benthic species).

SIFT is an algorithm developed in the computer vision domain for detecting and describing local features in digital images [32]. The convolved images are grouped by octave (i.e., an octave corresponds to doubling the value of scale space). After a preliminary and technical overview, we established the following states for the SIFT parameters: (i) the number of octaves was equal to 9; (ii) the number of levels per octave of the Difference of Gaussian scale space was equal to 3; (iii) Peak selection thresholds also equalled 3; and finally, (iv) The Non-edge Selection threshold was set at 7. The technical processing for Munida identification is detailed in Figure 3. For each extracted feature, 128 descriptors were used as variables in the analyses from that moment on. In our case, we considered tracked displacing objects to belong to the Munida category (i.e., hence counting for one animal), if these had a number of features greater than 6, with a pixel distance closer than 500.

Object Classification was the last step of the analysis (see Figure 1). This was carried out by using the tools of morphometric multivariate statistics. Two different methods were applied in order to classify each new tracked animal within the Munida species as an a priori category: Partial Least Square Discriminant Analysis (PLSDA) and ED.

PLSDA is a soft-modelling multivariate statistical approach that allows identification of correlated pairs of linear combinations (i.e., singular vectors) between two blocks of variables [33]. The result of this technique is the construction of a predictive model made by many and highly collinear factors [34].

PLSDA analysis was conducted on both the SIFT features plus the RGBv for each object and Fourier Descriptors (RGBv+FD) matrixes. For both datasets a preliminary training procedure was applied, in order to build the model of reference. This training procedure was accomplished by using: (i) 38 images of Munida (i.e., 2219 features within and 880 outside the animals’ body; see Figure 3) for SIFT matrix; (ii) 4800 objects (i.e., 57 Munida and 4,743 other extraneous objects) for the RGBv+FD. PLSDA looked for correlations among the matrices values (x-block; 128 matrix for SIFT and 3+128 matrix variables for RGBv+FD) and the y-block, which was composed by two dummy (i.e., categorized) variables. Dummy variables were categorized as (1) and (2) if “belonging” or “not belonging” to an extracted Munida body, respectively. The X-block was pre-processed with the ‘mean centre’ procedure.

After tracking animals, PLSDA allows an evaluation of its classification performance, indicating the modelling efficiency in terms of sensitivity and specificity of the chosen parameters. The sensitivity is the percentage of the samples of a category accepted by the class model. The specificity is the percentage of the samples of the categories different from the modelled one, which are rejected by the class model. Generally, the residual errors show a decreasing trend in the calibration phase (i.e., Root Mean Square Error of Calibration, RMSEC) and an increasing trend in the validation phase (i.e., Root Mean Square Error of Cross-Validation, RMSECV; [35]).

For FD and SIFT analyses, each group was subdivided into two sub-groups: 70% of objects for the class modelling and validation, and 30% of objects for the independent test, optimally chosen with the ED based on the algorithm of Kennard and Stone [36]. This algorithm selects objects without an a priori knowledge of a regression model (i.e., the hypothesis is that a flat distribution of the data is preferable for a regression model; [37]).

The RGBv+FD matrix (i.e., 3+128 variables) was also classified using the ED approach, being these distances equal to the square root of the sum of the squared difference for each dimension (i.e., variable). ED’s are extremely sensitive to the scales of the variables involved. In geometric situations, all variables are measured in the same units of length. For this reason, we computed two different ED thresholds for RGBv values and FD.

In accordance with current standards [30,38,39], an efficiency test was carried out in order to evaluate the performance of the different methodologies employed for the automatic object classification (see Figure 1). Error estimation was calculated on all the 100 images in comparison with visual results provided by a trained operator (i.e., manual counting). Error typologies were categorized into object identification and object classification. The occurrence of the two different errors was hence determined: misclassification when a Munida was present within the frame but not detected (i.e., Error Type-1); and wrong classification when a Munida was detected when not present in the picture (i.e., Error Type-2). Also, a time series of visual count outputs from our automated protocol for the different combinations of analytic methods (i.e., RGBv+FD and PLSDA; SIFT and PLSDA; RGBv+FD and ED) were graphed together, in order to obtain a visual estimation of their efficiency.

We developed an automated image analysis protocol for bacterial mat coverage estimations in MatLab 7.1. Our aim was to compute the Percentage of Coverage and Fractal Dimension. A flow chart detailing the consecutive steps of image treatment is presented in Figure 4.

A total of 52 images were considered as suitable for the analysis. Benthic species commonly present in the area caused a disturbance effect, either covering the ROI or moving sediment around. These species were chiefly the slender sole (Lyopsetta exilis), the Pacific herring (Clupea pallasii), and Munida itself [23].

Matabos et al. [23] describe the estimation of bacterial mat coverage by measuring how the mats filled space in images of the seafloor. This type estimation can be carried out by computing the Percentage of Coverage (i.e., the surface covered by bacterial mats) and the Fractal Dimension, the latter being a measure of how completely the bacterial mats fill the space at increasingly smaller scales (i.e., the covering geometrical complexity). Fractal analysis can be used to describe the occupation of space by biological forms such as the branching patterns in tree roots or spatial structure in mussel beds (reviewed by reference [40]). Both parameters are usually estimated in a semi-automated fashion, using software that requires the manual identification of areas to be analyzed in digital images.

The main operative problems in establishing a successful protocol were related to the uneven background illumination. Additionally, the analysis was complicated by: (i) variable and fragmented background, and (ii) fish-eye image distortion due to the camera lens.

The different outputs of automated image processing are reported as an example in Figure 5. As a first step in processing, a ROI was preliminarily selected in each original RGB image [2,448 × 3,264; Figure 5(A)]. For each image the background was extracted applying a Gaussian Blurring Filter procedure [41] [Figure 5(B)]. The ROI was then enhanced and preserved as common for all processed images [Figure 5(C)]. This low-pass filter attenuates high frequency signals by applying a Gaussian function to a squared pixel kernel (hsize). The standard deviation of this function (sigma) was also defined. Both values were empirically determined in a ten images training set, as follows: (1) hsize = 10 % Area   img (2) sigma = 10 % Area   img

During the second step of automated processing (i.e., Image Subtraction; see Figure 4), the resulting background image was subtracted to the original one within the ROI [Figure 5(D)]. The resulting RGB values were then summed [Figure 5(E)].

In order to enhance the images which varied in the distribution of their intensity, 10% smaller and 1% larger values were eliminated, by setting them as equal to the lower and higher nearest values, respectively. These values were chosen after empirical evaluation of ten images in the training set. The resulting gray scale image was then rescaled from 0 to 255 values [Figure 5(F)]. Then, a fixed threshold was set up on the resulting digital product and objects smaller than 1000 pixels were eliminated (i.e., the Area Factor), resulting in the final black and white image [Figure 5(G)].

The Percentage of Coverage by the bacterial mats in this image was calculated, together with the Fractal Dimension. This last parameter was estimated using the ‘box-counting method’ [42]. In order to create “square” boxes, the image was automatically resized to a square dimension such that the length, measured in number of pixels, was of a power of 2. This allowed for the square image to be equally divided into four quadrants and each subsequent quadrant could be then re-divided into four quadrants, and so on. The number of boxes containing “black” pixels was noted as a function of the box-size (i.e., the length of box). The natural log of all these points were calculated and plotted. A linear interpolation was calculated and the slope of the line was estimated as the Fractal Dimension value.

The efficiencies in the automated computing of both Percentage of Coverage and Fractal Dimension were also compared with results generated using the software Image J (National Institutes of Health, USA; http://rsbweb.nih.gov/ij/). The latter requires the manual processing of images for: (i) binarization, in order to enhance white bacterial patches against the grey sediment background; and (ii) the estimation of parameters of interest by the Fractal Box Counter (i.e., a variable number of boxes of pixels sizes equals to 2, 3, 4, 6, 8, 12, 16, 32, and 64 are manually overlaid on the image [23]).

We identified a total number of 103 squat lobsters from all images by manual counting. By comparison, the automated image analysis protocol showed different efficiencies, depending on the terminal processing steps used (see Figure 1 for reference), which were applied in parallel on all images. RGBv+FD and PLSDA overestimated animals up to 172 positive identifications. Conversely, both SIFT and PLSDA as well as RGB+FD and ED, subestimated Munida counts number to a different extent (65 and 12, respectively). The results of PLSDA models on SIFT and RGBv+FD matrices are reported in Table 1. For both methods high percentages of correct classification were possible for the model and the test set, as well as high values of efficiency parameters (i.e., specificity and sensitivity). We also observed low percentages of RMSEC (Table 1) and classification error (Table 2).

The SIFT approach returned better results in term of animal identifications (Table 2). The presence of animals was correctly classified in 51% of images (i.e., an animal is detected instead of being not present (Error Type-2). Also, lower values (51) of misclassification (i.e., an animal is present but not detected; Error Type-1) were obtained. Differently, RGBv+FD and ED returned no wrong classifications but most of the objects were misclassified (91 out of 103). Figure 6 presents examples of the different typologies of error we found by processing the same image with the three different analytical methods.

Time series of counted Munida using the different automated protocols (i.e., RGB+FD and PLSDA; SIFT and PLSDA; RGB+FD and ED; see Figure 1 for processing steps details) are presented in Figure 7 as an indication of their variable performance. The automated protocols showed little difference when compared together, while this difference was always larger in relation to the manual counting. This was chiefly due to the presence of Type-2 Errors in the automated counting. Moreover, no diel (i.e., 24-h based) variations in counted animals were apparent.

The outputs from single digital images of bacterial mat identification are presented in Figure 8 as an example of the manual and automated processing outputs. This provides a general overview of achieved efficiency with the automation procedure proposed in Figure 4. By considering an originally acquired digital image [Figure 8(A)] and delimiting only a fraction of it [Figure 8(B)], it is possible to observe the different results of manual and automated estimation [Figure 8(C)]. Generally, the automated method extracted a smaller area with respect to the manual one due to the background correction approach.

The Pearson correlation coefficients for the Percentage of Coverage and the Fractal Dimension for the manual and automated processing were high for both, being 0.67 and 0.76, respectively, hence confirming the good accordance between the methods.

The manual evaluation generally gave higher estimations than the automated processing for both indicators. Time series comparing manual and automated outputs of bacterial mat assessment in terms of Percentage of Coverage and Fractal Dimension (Figure 9) confirmed the differences in both estimation methods as well as over the time, as detailed in the example of Figure 8. The overestimation tendency of the manual method is particularly evident for Fractal Dimension values, while for the Percentage of Coverage, the same seems to occur in a variable fashion with the time of the day. Out of 52 images the manual analysis overestimated the Percentage of Coverage 27 images (52.9%) in comparison to the automated protocol. There were 42 overestimation events in the case of manual counting for Fractal Dimension (80.7%). The temporal plot also provides additional insights of a more biological nature. The bacterial mat started to disappear with increasing oxygen levels (data not shown). This was observed in the area until the end of December when bacteria totally disappeared [23]. One explanation for this disappearance is a downward migration into the sediment to reach the interface between anoxic and oxic conditions, which is their required habitat [23].