Automated Image Analysis of Offshore Infrastructure Marine Biofouling

In the UK, some of the oldest oil and gas installations have been in the water for over 40 years and have considerable colonisation by marine organisms, which may lead to both industry challenges and/or potential biodiversity benefits (e.g., artificial reefs). The project objective was to test the use of an automated image analysis software (CoralNet) on images of marine biofouling from offshore platforms on the UK continental shelf, with the aim of (i) training the software to identify the main marine biofouling organisms on UK platforms; (ii) testing the software performance on 3 platforms under 3 different analysis criteria (methods A–C); (iii) calculating the percentage cover of marine biofouling organisms and (iv) providing recommendations to industry. Following software training with 857 images, and testing of three platforms, results showed that diversity of the three platforms ranged from low (in the central North Sea) to moderate (in the northern North Sea). The two central North Sea platforms were dominated by the plumose anemone Metridium dianthus; and the northern North Sea platform showed less obvious species domination. Three different analysis criteria were created, where the method of selection of points, number of points assessed and confidence level thresholds (CT) varied: (method A) random selection of 20 points with CT 80%, (method B) stratified random of 50 points with CT of 90% and (method C) a grid approach of 100 points with CT of 90%. Performed across the three platforms, the results showed that there were no significant differences across the majority of species and comparison pairs. No significant difference (across all species) was noted between confirmed annotations methods (A, B and C). It was considered that the software performed well for the classification of the main fouling species in the North Sea. Overall, the study showed that the use of automated image analysis software may enable a more efficient and consistent approach to marine biofouling analysis on offshore structures; enabling the collection of environmental data for decommissioning and other operational industries.


Introduction
Permanent offshore structures may form artificial reefs, which provide attachment and settlement sites for marine organisms (defined herein as marine biofouling).In the UK, some of the oldest oil and gas installations have been in the water for over 40 years and have undergone considerable colonisation by marine biofouling organisms.Marine biofouling organisms in the UK generally include algae, soft corals and mussels in the photic zone as well as anemones, hydroids, tubeworms, barnacles and cold-water corals on the deeper sections of the platforms [1].The location (e.g., distance to coast, proximity to other platforms), sediment type, prevailing water current, depth, water temperature and material of the structure all have an influence on the type, density and zonation pattern of marine fouling.Generally, the same major groups of organisms are responsible for platform biofouling worldwide, but the individual species involved tend to vary [2].The climax stage in both coastal and offshore areas is represented by communities in which the dominant forms are anemones, mussels, barnacles, sea squirts, sponges and algae [3,4].
In terms of the oil and gas industry, the North Sea is usually referred to by region: southern, central, and northern North Sea, (and West of Shetland; Figure 1), principally.In these regions, the vertical zonation of marine fouling varies.A series of studies for Oil and Gas UK were undertaken to collate knowledge and experience on the management of marine biofouling during decommissioning [5,6].BMT Cordah [5] and Sell [7] report on the difference in marine fouling zonation in the North Sea.The southern North Sea is shallow (approximately 30 m depth) and generally has a higher abundance of mussels near the surface (compared to the other regions) and a lower abundance of anemones.In the central North Sea, the water is slightly deeper (approximately 90 m depth) and soft corals tend to be present throughout this depth, and there is a dominance by anemone.The northern North Sea has the highest species diversity compared to other regions of the North Sea, although percentage cover of individual species may decrease.The cold-water coral Lophelia pertusa is only present on northern structures from circa 60 m depth to 140 m depth.Although not included in the report to Oil and Gas UK, anecdotal evidence from industry ROV footage, may suggest that zonation might be less pronounced West of Shetland, where water depth exceeds 200 m; and anemones are not the dominant fouling species as seen in the North Sea.
Challenges and issues caused by marine biofouling for the oil and gas industry may include: corrosion of structures, impairment of visual inspection, obstruction of equipment and survey access, disruption of anodes and alteration of hydrodynamic loading [8][9][10][11].The addition of substantial biomass due to marine biofouling (e.g., weighing from a few hundred to a few thousand tonnes) also means that its subsequent disposal needs to be carefully considered if brought onshore (for disposal to landfill or composting at licensed facilities [11]).Furthermore, if marine biofouling includes species of conservation importance (SpCI), the Convention on International Trade in Endangered Species (CITES) regulations will need to be considered during platform decommissioning.
In areas of the Gulf of Mexico, a "rigs-to-reef programme", (the conversion of offshore platforms into designated artificial reefs [12]) is in progress.However, in Europe, particularly in the North East Atlantic OSPAR 1 region, there is a requirement to remove all offshore oil and gas infrastructure, excluding pipelines, from the seabed (although derogations may be granted; under OSPAR Decision 98/3).As part of a decommissioning plan, an operator may be required to assess the extent of marine biofouling on a platform in order to estimate its potential weight for removal and disposal; and to determine the extent of any SpCI (e.g., Lophelia pertusa and Sabellaria spinulosa).
Other areas of related, potentially important, research on marine biofouling include: the potential for the spread of marine invasive species [13]; especially when considering the movement and storage of decommissioned structures between marine and coastal areas; potential "stepping stone" habitats between natural ecosystems [14]; artificial reefs for conservation (e.g., de-facto MPAs [15]); platforms as fish aggregation devices [16,17]; or the use of pipelines by fish [18] and fishermen (S.Rouse, pers.comms.). 1 The Convention for the Protection of the Marine Environment of the North-East Atlantic (The OSPAR Convention).The OSPAR region refers to the North-East Atlantic and associated contracting countries.Quantification of marine organisms from underwater photography is more challenging than in situ inspection.This is due to limited image resolution, variable lighting conditions, water turbidity and the inability to interact with the organisms [19].This is particularly true when considering monitoring near-surface offshore infrastructure, due to swell, sea state and light penetration.Analysis of marine growth on offshore platforms is conducted at greater depths and with limited light conditions [20,21], compared to similar studies on shallower coral reefs [19].Inferior conditions may have an influence on the quality of the images collected.
However, underwater photography or videography methodologies do provide a number of advantages.These include, the creation of a permanent record of species presence at a site; the ability to record more data with limited field deployments; the possibility of recording data where in situ observations are challenging or dangerous (e.g., where technical diving or submarine access is required) (e.g., [4]).Photographic surveys are now considered the standard within marine ecological field studies, due to the ability to collect substantial datasets efficiently, consistently and safely.Ongoing advancements and improvements in underwater digital photography quality, digital storage and computer vision methods will rapidly accelerate the analysis of underwater photography [19].
Analysis of marine biofouling organisms on offshore oil and gas infrastructure is conducted either by Remotely Operated Vehicle (ROV) operators, that generally identify the main fouling assemblages and percentage cover of "hard" and "soft" growth; or by marine growth analysts (MGAs) if a more scientifically accurate analysis is required.MGAs review ROV survey footage Quantification of marine organisms from underwater photography is more challenging than in situ inspection.This is due to limited image resolution, variable lighting conditions, water turbidity and the inability to interact with the organisms [19].This is particularly true when considering monitoring near-surface offshore infrastructure, due to swell, sea state and light penetration.Analysis of marine growth on offshore platforms is conducted at greater depths and with limited light conditions [20,21], compared to similar studies on shallower coral reefs [19].Inferior conditions may have an influence on the quality of the images collected.
However, underwater photography or videography methodologies do provide a number of advantages.These include, the creation of a permanent record of species presence at a site; the ability to record more data with limited field deployments; the possibility of recording data where in situ observations are challenging or dangerous (e.g., where technical diving or submarine access is required) (e.g., [4]).Photographic surveys are now considered the standard within marine ecological field studies, due to the ability to collect substantial datasets efficiently, consistently and safely.Ongoing advancements and improvements in underwater digital photography quality, digital storage and computer vision methods will rapidly accelerate the analysis of underwater photography [19].
Analysis of marine biofouling organisms on offshore oil and gas infrastructure is conducted either by Remotely Operated Vehicle (ROV) operators, that generally identify the main fouling assemblages and percentage cover of "hard" and "soft" growth; or by marine growth analysts (MGAs) if a more scientifically accurate analysis is required.MGAs review ROV survey footage manually, recording the percentage cover and thickness of different species at various points and depth zones on the platform jacket.These results are then extrapolated to estimate the percentage cover, thickness and the mass of total marine growth over the entire platform.As such, only a limited number of images may be analysed within the project timescale.
Beijbom et al. [19] discussed the ongoing advances in underwater image collection, image analysis and the advancement of underwater robotics (e.g., Autonomous Underwater Vehicles (AUVs) and ROV technologies).As such, Beijbom et al. [19,22] developed an automated annotation system (CoralNet, https://coralnet.ucsd.edu/)for coral reef survey images, with a publically available, user-friendly interface.The system assesses the texture and colour of a local image patch around randomly allocated annotation points, assigning the point to a predefined list of species or labels, using recent advances in computer vision science [19,22].Full details of the model, including system algorithms are available in Beijbom et al. [19,22].
Although designed for the purpose of coral reef analysis, it is proposed that the automated annotation method described above, could be used to provide overview analysis of species types, levels of biodiversity and depth and degree of the zonation of organisms on offshore structures.If the automated annotation system works well for North Sea species then taking up this approach would enable a far greater number of images to be assessed more efficiently and the automated nature will allow a much higher degree of data and analysis consistency across the oil and gas industry.
This project is a scoping study, with the objective to test the use of automated image analysis software on images of marine biofouling collected from offshore platforms on the UK continental shelf (UKCS).The aims of the project are to: (i) train the software to identify the main marine biofouling organisms on UKCS platforms; (ii) test and compare different analysis criteria (methods A, B and C) to determine suitability of this type of analysis methodology on platform ROV footage; (iii) calculate the percentage cover of marine biofouling organisms on three test platforms to assess software performance; and to determine the platform species diversity and zonation patterns; and (iv) provide the reasoning and outline methodologies to industry for the use of this type of analysis software.

Image Collection and Initial Training of CoralNet Software for North Sea Species
The project uses the image annotation method developed by the CoralNet project (part of a National Science Foundation (NSF) funded project, Computer Vision Coral Ecology by University of California, San Diego, in 2012 [19,22]).ROV survey video footage (general visual inspection; GVI; defined as a regular routine structural survey undertaken for the purpose of assessing structure and component integrity) was obtained from a number of North Sea operators for a selection of North Sea platforms across the northern, central and southern North Sea regions (Figure 1), for software training.Platform location, names and operator names are anonymous.Footage was viewed and images were randomly collected via screen grab, where possible, across the full depth range of the platform/footage.Images were selected based on image quality, as judged by the analyst (e.g., Figure 2 showing an example of "good" quality) and containing a scale bar, where possible.This allowed for distance from camera to be estimated (allowing for known image pixel size to be entered into CoralNet).
Training of the CoralNet software was required, in order to "train" the software to identify the species that are present on North Sea platforms.In order to train the software, a number of criteria needed to be set within CoralNet for this project: 1.
Image Boundary

•
A boundary was set for each image, to prevent annotation points being placed too close to the image edge.Image X and Y boundaries were set at 10% and 95% respectively (meaning bottom and left 10% of image; and upper and right 5% of image will not contain annotation points), defining the rectangle within which annotation points would be generated (annotation area; the suggested default boundary within CoralNet; Figure 2).

Annotation point generation (number and pattern)
• Annotation point generation was set to "simple random (random within the defined annotation area)" for 20 points (Figure 2).

•
The aim was to train the software on an estimated 500-1000 annotation points per species (across the 21 species listed in Table 1).This was not possible for all species as some were too infrequent.Therefore, on average, 800 annotation points per species were allocated (Table 1).

•
Where possible, image distance was estimated (in cm), based on scale bar presence (e.g., Figure 2, estimated distance 10 cm) or based on approximation.All images used were taken within 100 cm of the infrastructure surface.

•
The confidence threshold for the automated annotation was set to 100% (e.g., all points required confirmation by a human analyst, following the computer classification/analysis).
Each time the software analyses or classifies a group of images, this is subsequently called a classifier.

Identified species list
• A set of species/labels was determined from CoralNet's species list (defined within CoralNet as the labelset) (Table 1).These labels were used by the computer and the human analyst to classify the annotation points.Following initial project training, three North Sea platforms PlatMX (central North Sea), PlatEP (central North Sea) and PlatMS (northern North Sea; see Figure 1) were designated as test platforms, with the images excluded from initial software training (no southern North Sea platforms were included in the software testing as the project partner does not own any southern North Sea assets).Images were collected from each platform and collated by 10 m depth ranges from 10 m depth to the seabed.Image quality from 0 m to 10 m depth were not suitable for use due to swell; and these were subsequently disregarded.In addition, images were not available for all depth ranges on PlatMS.The selected training images were uploaded to CoralNet for processing.These training images were "anonymously" uploaded (i.e., no image/platform metadata was included) to the defined CoralNet project.The computer analysed the images, and made "best guess" identifications of the annotation points.The analyst then confirmed any correctly identified annotation points, and corrected any incorrectly identified annotation points (classifier trained and confirmed/corrected).The remaining images were uploaded in groups, classifiers trained and confirmed; with a total of 857 images used for software training.Images used for training, will form the basis from which the software will analyse any subsequent "test" images (in this study for the 3 "test" platforms).Analysis results do not include any training images.Images used in testing, once uploaded will help improve the "training" of the software.
Within the project, the accuracy of the automated classifier increases as the number of training images for each species/label increases, as expected [18].The training of a new classifier (e.g., computer learning) is triggered when the number of images within the project increases by 10%.In addition, within the CoralNet project a "classifier-validation-step" is also used.A new classifier is only accepted into the system if it increases the accuracy of the classifier by at least 1% over the previous classifier.An example of training annotation point distribution is shown in Figure 2.
Following initial project training, three North Sea platforms PlatMX (central North Sea), PlatEP (central North Sea) and PlatMS (northern North Sea; see Figure 1) were designated as test platforms, with the images excluded from initial software training (no southern North Sea platforms were included in the software testing as the project partner does not own any southern North Sea assets).Images were collected from each platform and collated by 10 m depth ranges from 10 m depth to the seabed.Image quality from 0 m to 10 m depth were not suitable for use due to swell; and these were subsequently disregarded.In addition, images were not available for all depth ranges on PlatMS.

Testing Accuracy of Different Methods of Image Analysis
Three criteria/methods of image analysis were carried out in the CoralNet software, as per Table 2. Method A was set at 20 random points and a confidence threshold of 80% (the software would automatically confirm annotation points it was 80% or more certain where identified correctly).Twenty random points were selected to mirror training criteria.Method B was set at 50 points, stratified random (as defined by CoralNet) within a grid of 5 rows by 5 columns (2 points per cell), confidence threshold was set at 90%.The confidence threshold was increased to test software improvement.Finally, for method C, 100 annotation points were set as a uniform grid of 10 rows by 10 columns (1 point per cell) with a confidence threshold of 90%.This method was applied to PlatMS only following initial analysis of methods A and B. * image classifier ran before completion of full confirmation/correction as per CoralNet classifier-validation-step. 1 As defined as "stratified random" within CoralNet.
For each platform and testing method, images were uploaded (by platform) and the images analysed by CoralNet.The automated annotation (classifier results) image percentage covers were exported and the average percentage cover of each species was calculated for all depth ranges (unconfirmed % cover).
The automated annotation points for each image were then confirmed or corrected where necessary by the analyst.Where one or more annotation points had been automatically confirmed by the software (i.e., the software was confident (80% or 90% depending on version) that it was identifying the species/label correctly) the number and species was recorded, along with any annotation errors (e.g., the occurrence of errors in the classifier's confidence threshold, i.e., how often did the classifier incorrectly identify a species with confidence (annotation points confirmed by the classifier above the confidence threshold) was noted).Following full confirmation or correction, the image percentage covers were exported and average percentage cover calculations were repeated (confirmed % cover).Finally, the percentage cover of species at each depth range for all three platforms was normalised to allow for the removal of the "no data" dataset; therefore the percentage cover was expanded to remove any areas of "no data".This was done in order to confirm the similarity between the manual analysts with the confirming analyst.
In parallel to the annotation classification, a separate manual assessment of the percentage cover of marine organisms was carried out on the same three test platforms (using the same set of images and depth ranges), by an independent analyst with no prior image bias.The average percentage cover was calculated (manual % cover).The manual assessment was undertaken on images without annotation points, and represents the current methodology applied by one of the leading industry consultants undertaking oil and gas marine growth assessments (e.g., [23]).

Biodiversity Analysis and Comparison of Methods
A Shannon-Wiener Diversity index (H) was calculated in Excel to determine species diversity at each depth range for each platform.Finally, a comparison of each testing methods (A, B, C) and type (unconfirmed, confirmed and manual) for each platform was undertaken; and a two-way paired, 2-tailed distribution student t-test (p < 0.05; PlatMX n = 9; PlatEP n = 8; PlatMS n = 10) was calculated in Excel for each comparison pair (unconfirmed, confirmed, manual and normalised; per species across each platform), followed by Bonferroni correction applied to the t-test results.

Initial Training and Testing of CoralNet Software for North Sea Species
Following software training, the range of species that were identified confidently by the classifier, that is, above the confidence threshold, and subsequently confirmed correct, was limited across all three test platforms (Figure 3).The classifier identified six species/labels correctly/confidently across all three platforms: Metridium dianthus, Alcyonium digitatum, Mytilus edulis, Lophelia pertusa, "no data" and brittlestars (Figure 3).The species correctly identified most often on PlatEP and PlatMX was M. dianthus; and the label "no data" on PlatMS.

Initial Training and Testing of CoralNet Software for North Sea Species
Following software training, the range of species that were identified confidently by the classifier, that is, above the confidence threshold, and subsequently confirmed correct, was limited across all three test platforms (Figure 3).The classifier identified six species/labels correctly/confidently across all three platforms: Metridium dianthus, Alcyonium digitatum, Mytilus edulis, Lophelia pertusa, "no data" and brittlestars (Figure 3).The species correctly identified most often on PlatEP and PlatMX was M. dianthus; and the label "no data" on PlatMS.
The PlatMX platform was the most confidently classified platform (A and B).The results showed 625 (33%) and 1750 (37%) correctly identified species/labels from a total of 1900 and 4750 annotation points respectively (Figure 4).The error rate (number of errors observed in the annotation points the software identified, i.e., above the confidence threshold) for PlatMX was 3.1% (method A) and 0.6% (method B).All other remaining annotation points for PlatMX (A = 1255, 66%; B = 2989, 63%) were not confidentially identified by the computer (i.e., were below the confidence threshold).The error rate (percentage of the total number of annotation points, correct and incorrect, above the confidence threshold that were incorrect) was below 4% for all platforms.The percentage of correctly identified species/labels (of the total number of annotation points) ranged from 3% (PlatEP) to 37% (PlatMX).
Across all three platforms, the most common error was the identification of M. dianthus as Lophelia pertusa, representing 16 erroneous annotation points (Table 3).The PlatMX platform was the most confidently classified platform (A and B).The results showed 625 (33%) and 1750 (37%) correctly identified species/labels from a total of 1900 and 4750 annotation points respectively (Figure 4).The error rate (number of errors observed in the annotation points the software identified, i.e., above the confidence threshold) for PlatMX was 3.1% (method A) and 0.6% (method B).All other remaining annotation points for PlatMX (A = 1255, 66%; B = 2989, 63%) were not confidentially identified by the computer (i.e., were below the confidence threshold).The error rate (percentage of the total number of annotation points, correct and incorrect, above the confidence threshold that were incorrect) was below 4% for all platforms.The percentage of correctly identified species/labels (of the total number of annotation points) ranged from 3% (PlatEP) to 37% (PlatMX).Across all three platforms, the most common error was the identification of M. dianthus as Lophelia pertusa, representing 16 erroneous annotation points (Table 3).
No significant difference (across all species) was noted between confirmed annotation methods (A, B and C) on all three platforms.On PlatEP, there was only a significant difference between unconfirmed methods A and B, for M. dianthus only (Table 4).
Finally, no significant difference was noted on PlatMX and PlatEP for the confirmed-normalised vs. manual comparison (for both A and B methods; PlatMx p = 0.18-0.80;PlatEP p = 0.07-0.91).For PlatMS, a significant difference (p < 0.003) was noted between the confirmed-normalised vs. manual comparisons (A and C) for M. dianthus and other anemones (p = 0.00).If other anemones and M. dianthus are grouped to form "all anemones", this reduces the significant difference to none (Table 4).

CoralNet Software Ability to Assess Percentage Cover
Tables and graphs presenting the percentage cover of species (species/labels) for each platform and analysis type (unconfirmed, confirmed and manual) are presented in the supplementary material (Tables S1-S17; Figures S1-S17).
The plumose anemone Metridium dianthus was the most dominant organism by percentage cover on the PlatMX platform, again followed by the label "no data".Also present on the PlatMX platform were the soft coral Alcyonium digitatum, the barnacle Chirona hameri, the blue mussel Mytilus edulis, the hydroids Obelia sp. and Tubularia sp., sponge spp., tubeworms and the label infrastructure surface (Tables S1-S5; Figures S1-S5).
The most dominant organism by percentage cover on the PlatEP platform across all depth ranges was M. dianthus, followed by the label "no data" (Tables S6-S10; Figures S6-S10).Also present on the PlatEP platform were A. digitatum, C. hameri, M. edulis, Obelia sp., sponge spp.and the label infrastructure surface.

Identifying Biodiversity Differences between Platforms
Shannon-Wiener (H) index values (Table 5) showed that PlatMX has the lowest species diversity (H = 0.28 and 0.36) of the three test platforms, representing overall low species diversity (H < 1; for this study the H index has been interpreted as moderate diversity = H > 1; high diversity = H > 3), with diversity at its highest at 30 to 40 m depth (H = 0.59 and 0.83).PlatEP also shows overall low diversity (H = 0.79 and 0.80), with highest diversity recorded (moderate diversity; H > 1) at 50 to 60 m depth (H = 1.00 and 1.03).PlatMS has the highest diversity (H = 1.89-2.06)representing overall moderate species diversity (H > 1), with highest diversity recorded at 150 to 160 m depth (1.43 and 1.59).Lowest diversity (low diversity; H < 1) was recorded at 10 to 20 m depth (H = 0.81 and 0.95).Evenness (EH) was greatest on PlatMS (EH = 0.78 and 0.79) showing that individuals on this platform are distributed more equitably among the recorded species.Lowest evenness was recorded on PlatMX (EH = 0.14 and 0.16) which is represented by the dominance of the anemone M. dianthus on this platform.

Discussion
This study presents a method for the automated analysis of marine biofouling on offshore platforms; and is, as far as the authors are aware, the first attempt at using such analysis methods on these types of offshore structures and using industry collected data.
Overall, this study has shown that the use of automated image analysis software may enable a more efficient and consistent approach to marine biofouling analysis on offshore structures; enabling the collection of environmental data for decommissioning and for other operational industries.It was considered that the software performed well for the classification of the main fouling species in the North Sea.
In relation to time-saving and efficiency, it was estimated that the manual analysis of the selected images (excluding time to collect the images), was approximately five to six hours per platform.In comparison, the annotation of images within CoralNet, took roughly two to three hours per platforms.These timing were estimated as it was not a specific aim of the study, and time was spent recording errors of the annotations, which would not ordinarily be recorded when analysing images on a regular basis.

Training and Testing Software Performance
For some species/labels, the computer will not perform well, as there are not enough examples of the species/label in the training set.In this study, ten species/labels were below the targeted number of annotation points (at least 500): other anemone, Balanus balanus, Chirona hameri, sponge, tubeworms, bryozoan, brittlestars, unknown invertebrates, algae and "unknown".The labels for Scalebar and water were also below the target number, however, these were subsequently recorded as "no data" during testing.It is acknowledged that there are limitations to the use of the software, particularly for less frequently occurring species, which does ultimately lower the accuracy of the results presented here.However, for this study, the accuracy of the software was not our main focus, and are not looking to rely solely on this particular software for biofouling analysis, but instead use it as a tool to assist in the analysis of marine biofouling.The purpose of the study was to test the suitability of the software for an industry application, and make recommendations for its future industry use.
The testing data within this study will now be incorporated into the "training" dataset, for subsequent platform analysis following the completion of this project.Therefore, the training of the software will improve as more analysis is undertaken; as well as amendment to the original species/label list as required (e.g., by making changes to the species and label list in the project within CoralNet).
As the software relies on size and texture for the identification of the species/labels, the quality of the image is important, and a limitation of this study; a "good" quality image of known pixel size is essential.However, even on images of "good" quality, errors are possible.The most errors identified were the misidentification of M. dianthus as L. pertusa.On North Sea platforms, M. dianthus and L. pertusa form both orange and white colonies [24] and have a similar texture, particularly when L. pertusa polyps form young colonies and their tentacles are extended.In addition, A. digitatum appeared pale orange in most of the images collected, and when their tentacles are extended, they have a "fluffy" appearance.Showing a likeness to M. dianthus this may confuse the software; which is also the case for Tubularia sp.
The second highest error noted was between "no data" and M. edulis.On the images used, "no data" (training annotation points n = 3112) was primarily attributed to the survey text on the images, or to areas off the structure, that in most images appeared black (or pale blue/green in areas of higher light intensity, representing open water).The M. edulis (training annotation points n = 1077) shells in the images were also predominantly black, which created a challenge for the software.Subtle texture on the M. edulis shells may therefore have had an influence, as "no data" is associated with areas of very "flat" black colour.

"No Data"
Ideally, an image used in the automation software would contain no "no data", therefore showing only the surface of the structure at a set distance from camera, with no areas off-structure.Given that this is not easy to achieve from archived ROV footage, "no data" needed to be taken into account (including areas of on-screen text).Normalising the annotation data (percentage cover) following removal of the "no data" allowed for a more comparable dataset, when comparing manual and annotation analysis.
The results from this study showed that there was no significant difference between the "confirmed-normalised" analysis and the manual analysis for PlatMX and PlatEP platforms, and the significant difference on PlatMS was limited to M. dianthus and other anemones, but when these two labels were combined into "all anemones" was reduced to no significant difference.This is possibly due to the experience of the manual analyst.This suggests that, with the removal of the "no data" from the analysis, the manual analyst and the "confirming" analyst performed consistently, when restricted to a defined species/label.The unconfirmed methods showed some differences between species, however, this would likely improve as more images are trained.
"No data" may also be represented by other criteria, not just "off-structure", such as a scale bar, probe, ROV arm or any other object not attached to the platform.It is recommended that where possible, images are collected showing "on-structure" areas only, with no additional text on screen, or other image limiting objects.It is suggested that non-inhibiting scale measures be used (e.g., laser scales).Additionally, when other interventions are not possible, it is recommended that the annotation point layout is tailored to minimise areas of "no data", while maximising species coverage.

Annotation Points
Three different testing criteria were used in order to determine the optimal number and layout for the annotation points.It was determined that there was no significant difference between all confirmed methods (A, B and C) for all three platforms following correction.However, analysis prior to correction should that there was a difference between PlatMS A and B (20 to 50 points); and A and C (20 to 100 points).
From the visual inspections and the analysis results, PlatMS is a more diverse platform than PlatMX or PlatEP.The results of the comparative assessment suggest that the number of annotations points needs to be greater on more diverse platforms.This is due to the necessity of getting enough training examples for "rarer" species/labels.The threshold for the minimum number of points required is the number of total examples of the rarest categories.If these rarer categories are important then more points per image and more images will need to be used in the manual image training annotation set.In this study, however, there would appear to be a cut-off point, as no difference was noted between 50 and 100 points on PlatMS.
It was assumed that the more annotation points that are analysed, the better the percentage cover predictions; however, this did not appear to be the case.This is likely to be due to the lack of diversity observed on the test platforms.It is likely that where less common species are noted, a larger number of annotation points may increase the chance of the points being placed on these more sporadically occurring species.If an image is being analysed manually, these sporadic species may well be recorded, whereas they may be missed using an automated system.Due to time and resource constraints, it was not possible to repeat the manual analysis using the annotated images as well as the industry standard method (as undertaken here).However, this wasn't considered an issue, as it was important to understand how this type of software compares to the industry standard method, and how it could improve or support industry analysis in the future.
Hence, a balance is needed between analyst effort and the overall percentage cover accuracy when selecting the annotation criteria.From the results presented here, it is recommended that 50 annotation points per image would suffice, applied to at least 10 images per depth range, however, this will be dependent on the image quality and number, and overall diversity of species within the collected images.If more images were available, e.g., 500 images per depth range, then statistically, you would be able to undertake analysis with less annotation points per image (e.g., [10]), to achieve the same outcome.This should therefore be considered on a platform by platform basis.

Limitations
Although the use of automated image analysis software presents a significant opportunity for data collection, there are a number of limitations with the current software that should be taken into account.
The quality of the images uploaded to CoralNet should be carefully considered.The footage available from offshore operators tends to be focused on structural survey requirements, fulfilling the operator's regulatory requirements.Images may not be of sufficient resolution, or contain suitable scales for the assessment of marine growth organisms.Images presented in Figure 5 represent the variation in "good quality" images collected for use in this study.Figure 5a you would be able to undertake analysis with less annotation points per image e.g.[10], to achieve the same outcome.This should therefore be considered on a platform by platform basis.

Limitations
Although the use of automated image analysis software presents a significant opportunity for data collection, there are a number of limitations with the current software that should be taken into account.
The quality of the images uploaded to CoralNet should be carefully considered.The footage available from offshore operators tends to be focused on structural survey requirements, fulfilling the operator's regulatory requirements.Images may not be of sufficient resolution, or contain suitable scales for the assessment of marine growth organisms.Images presented in Figure 5 represent the variation in "good quality" images collected for use in this study.Figure 5a  The diversity of species within the images also presents a number of challenges.The testing of the software is reliant on the quality and quantity of the training images, with the classifier improving with increasing numbers of training/confirmed images.Collection of images of certain marine growth organisms such as M. dianthus, Obelia sp. and A. digitatum is relatively easy, given that they are dominant species on North Sea structures.The collection of images of other typical, but rarer fouling species or groups, such as sponges, bryozoans, tubeworms, other anemone species and some corals are harder to locate, as they have patchy distribution; or are concealed by larger, more dominant species.Collecting images from video footage also presents challenges, particularly with the movement of the ROV.If surveys do not settle to take stationary measurements on the infrastructure surface, the resulting images are blurred.
The diversity of species within the images also presents a number of challenges.The testing of the software is reliant on the quality and quantity of the training images, with the classifier improving with increasing numbers of training/confirmed images.Collection of images of certain marine growth organisms such as M. dianthus, Obelia sp. and A. digitatum is relatively easy, given that they are dominant species on North Sea structures.The collection of images of other typical, but rarer fouling species or groups, such as sponges, bryozoans, tubeworms, other anemone species and some corals are harder to locate, as they have patchy distribution; or are concealed by larger, more dominant species.
One particular challenge of using automated software, versus traditional marine growth assessments is the analysis of multi layers of species.For example, it may be expected that over 100% marine growth may be recorded from a particular image.As presented in Figure 5d, the infrastructure surface is 100% covered with marine growth, but there is a layer of Obelia sp. over the M. edulis.At present, there was no way to correct this within the software used herein.Consequently, the total percentage cover of species may be under-estimated.One way to address this within the software, may be to create species/label lists that were applicable to overlaying species, for example, mussels and seaweed; mussels and hydroids; tubeworms and hydroids etc.The analyst(s) would need to create a set of rules as to what to label overlaying species.This is not a fault of the software, more an ecological challenge that would need to be adapted to.
The CoralNet software is based only on the annotation points, and does not extrapolate up over the entire image.Therefore, if the annotation points are not assigned to an individual species (in particular less common species), it will not be recorded within the percentage cover plot.It is, therefore, important to consider the layout and number of annotation points.A high enough number of points and images are needed to statistically represent the percentage cover.
Finally, at present the CoralNet software does not allow for the transfer of training images (privately) between projects, therefore, images will be need to be trained if other users wished to use this methodology to analyse their own images.It is hoped that will be addressed by the CoralNet project team in due course.

Percentage Cover and Zonation
In the North Sea, zonation of fouling organisms is dependent on the location of the platform as previously outlined.This study has analysed three North Sea platforms from the central and northern regions (Figure 1) and has corroborated earlier studies [5,7] on marine fouling zonation.Of note, the extent of the dominance of anemones on the assigned central North Sea platforms (PlatMS and PlatEP) was perhaps unexpected and potentially to the exclusion of some other expected species (e.g., soft corals or mussels); which may explain the low diversity score reported.On the northern North Sea platform (PlatMS), higher diversity was reported, which was to be expected and species dominance was not as obvious.A southern North Sea platform was not included in this study due to the lack of assets in this region operated by our project partner.
It should be noted that the diversity of the platforms analysed in this study, may be underestimated given that there is a chance that not all species (particularly those less seen e.g., those <500 annotation points) have been recorded.
A challenge of assessing percentage cover of biofouling is that the judgement of percentage cover by different analysts varies.One person's interpretation of a percentage may be different to another's.The use of automated software helps address this challenge in part, as only the annotation points are analysed.The use of experienced North Sea MGAs is important.The use of this software does not remove the need for a MGA entirely, but allows for more images to be analysed more consistently, even if multiple MGAs are utilised.The results of this study showed that there was some difference between the MGA confirming the images within the software, and the MGA undertaking the analysis manually.However, this issue was improved following the normalisation of the data to remove "no data".
One aspect of the software, which is not replicable by eye, is the ability to export the annotation points for use on another image.For example, if it were possible to collect the same image from a platform over a defined time-series, the MGA would be able to overlay the initial annotation points within the software.This would enable comparative analysis to examine how marine biofouling may change over time.

Recommendations
The final objective of the study was to make recommendations on the use of this type of automated image analysis software for industry.The results were presented at small industry engagement sessions and a discussion was had with industry ROV operators and survey managers about how they envisaged incorporating the collection of suitable data for this type of analysis.
In summary, the following recommendations are made for the use of the CoralNet software for the analysis of percentage cover of marine biofouling organisms on offshore structures: 1.
When collecting new survey footage, the use of a high definition (HD) video or camera is preferred.2.
If using video only, allow time for the ROV to settle at various points on the platform jacket.

3.
Settle at different locations within 10 m depth ranges, at different orientations and perpendicular to the structure.

4.
Stay within 1 m of the structure and try to fill the frame with the structure in order to limit "off-structure" areas within images.

5.
Allow for a minimum of 10 images to be collected from each 10 m depth range.6.
Use scale bars or scale lasers as accurate pixel size estimation is critical to the accuracy of the automated system.Ensure that the scale bar is not intrusive to the footage/image and ensure ROV arms or cathodic protection (CP) probes are not within the shot.7.
Remove overlay text from survey footage, except for depth; or provide depth details in metadata or image title.8.
Where text overlay is removed, the image boundary within CoralNet can be set to X: 10-95%/ Y: 10-95%.Where the text overlay is present, it may be necessary to test the boundary to minimise the chance of points landing on the text.9.
It is recommended that 50 annotation points per image should be used, however, this will be dependent on the image quality, the number of rare species of interest and the total number of images taken per depth.This should be considered on a platform by platform basis.10.The annotation point distribution should be set within a grid-either uniform or stratified random (as defined by CoralNet) to ensure no overlap of points and equal coverage of the image.11.Where it is not possible to use images with no "no data", following analysis, normalise the dataset to remove "no data".

Conclusions
Marine biofouling on offshore structures is an important topic due to the extensive interest in the potential for turning offshore platforms into artificial reefs.However, the presence of marine biofouling on offshore structures, does not necessarily equate to these structures being classed as a "reef".Artificial reefs are defined by OSPAR as " . . .a submerged structure placed on the seabed deliberately, to mimic some characteristics of a natural reef . . .." [25] and natural reefs (e.g., Annex I reefs, stony or biogenic, under the EC Habitats Directive) are defined as " . . .a habitat that is colonised by many different marine animals and plants . . .and provides a home to many species . . .as well as giving shelter to fish and crustaceans such as lobsters and crabs" [26].Reefs are globally considered to be diverse ecosystems capable of supporting a variety of marine life, throughout the food chain.
Therefore, without the knowledge of what grows on the offshore platforms, how this varies over a geographic region or an understanding of what additional species the structures support and how, it is not possible to establish the true benefits of these potential artificial reefs.There are research gaps on how these structures are used by fish or marine mammals for example (such as a food source or shelter from fishing); and understanding how these structures may contribute to carbon sequestration or to productivity are essential to informing the debate and policy.
One of the challenges facing industry and the research sector is the access to industry data.In some circumstances the industry is reluctant to share data (in this case ROV survey footage) too widely.This project has demonstrated the use of automated software for analysing marine biofouling on offshore structures and has identified potential areas of future study.This study provides the initial evidence to show that is it very possible now to undertake further analysis of offshore structures on the UKCS using automated image analysis, and to process and present/map the results in a way that is satisfactory to the industry, ensuring commercial sensitivity is addressed and results are available for research and/or policy use.

Supplementary Materials:
The following are available online at www.mdpi.com/2077-1312/6/1/2/s1,Table S1: Average percentage cover of species per depth range PlatMX A Unconfirmed, Table S2: Average percentage cover of species per depth range PlatMX A Confirmed, Table S3: Average percentage cover of species per depth range PlatMX B Unconfirmed, Table S4: Average percentage cover of species per depth range PlatMX B Confirmed, Table S5: Average percentage cover of species per depth range PlatMX Manual, Table S6: Average percentage cover of species per depth range PlatEP A Unconfirmed, Table S7: Average percentage cover of species per depth range PlatEP A Confirmed, Table S8: Average percentage cover of species per depth range PlatEP B Unconfirmed, Table S9: Average percentage cover of species per depth range PlatEP B Confirmed, Table S10: Average percentage cover of species per depth range PlatEP Manual, Table S11: Average percentage cover of species per depth range PlatMS A Unconfirmed, Table S12: Average percentage cover of species per depth range PlatMS A Confirmed, Table S13: Average percentage cover of species per depth range PlatMS B Unconfirmed, Table S14: Average percentage cover of species per depth range PlatMS B Confirmed, Table S15: Average percentage cover of species per depth range PlatMS C Unconfirmed, Table S16: Average percentage cover of species per depth range PlatMS C Confirmed, Table S17: Average percentage cover of species per depth range PlatMS Manual, Figure S1

Figure 1 .
Figure 1.Map of "generally referred" to oil and gas regions in the UK sector of the North Sea (boundaries, dotted line, are approximate and not administratively imposed).

Figure 1 .
Figure 1.Map of "generally referred" to oil and gas regions in the UK sector of the North Sea (boundaries, dotted line, are approximate and not administratively imposed).

Figure 2 .
Figure 2. Screen grab representing training annotation layout 20 simple random points, annotation area: X: 10-95%/Y: 10-95%; and scale bar protrusion used for determining distance from ROV to surface.Platform and location identifies have been removed from all images.Purple crosses and numbers indicate annotation points.

Figure 2 .
Figure 2. Screen grab representing training annotation layout 20 simple random points, annotation area: X: 10-95%/Y: 10-95%; and scale bar protrusion used for determining distance from ROV to surface.Platform and location identifies have been removed from all images.Purple crosses and numbers indicate annotation points.

Figure 3 .
Figure 3. Percentage of correctly identified (by software) annotation points per platform and method, separated by species/labels.Graph only shows proportion of correctly identified species and is not calculated in relation to the total number of points.

Figure 3 .
Figure 3. Percentage of correctly identified (by software) annotation points per platform and method, separated by species/labels.Graph only shows proportion of correctly identified species and is not calculated in relation to the total number of points.

Figure 4 .
Figure 4. Percentage of the total number of annotation points per platform correctly identified (pale grey) by the software; percentage of the number of annotation points identified by the software (all points above the confidence threshold) that were incorrect/errors (black); and % of the total number of annotations points below the confidence threshold (dark grey).

Figure 4 .
Figure 4. Percentage of the total number of annotation points per platform correctly identified (pale grey) by the software; percentage of the number of annotation points identified by the software (all points above the confidence threshold) that were incorrect/errors (black); and % of the total number of annotations points below the confidence threshold (dark grey).
,b are fairly typical images collected from ROV footage, but may be considered to lack clarity and scale and are poorly exposed.In comparison, Figure 5c,d represent the best examples of ROV images, demonstrating high definition, clear, and steady, with a scale.Better quality images are usually only collected during a specialist marine growth survey.J. Mar.Sci.Eng.2018, 6, 2 16 of 20

Figure 5 .
Figure 5. Representative images collected from offshore structure ROV surveys.(a,b) represent typical quality footage which may not be suitable for use in CoralNet; (c,d) represent the best quality images that can be gathered from offshore ROV surveys.

Figure 5 .
Figure 5. Representative images collected from offshore structure ROV surveys.(a,b) represent typical quality footage which may not be suitable for use in CoralNet; (c,d) represent the best quality images that can be gathered from offshore ROV surveys.
: Percentage cover per species by depth range for PlatMX A Unconfirmed, Figure S2: Percentage cover per species by depth range for PlatMX A Confirmed, Figure S3: Percentage cover per species by depth range for PlatMX B Unconfirmed, Figure S4: Percentage cover per species by depth range for PlatMX B Confirmed, Figure S5: Percentage cover per species by depth range for PlatMX Manual, Figure S6: Percentage cover per species by depth range for PlatEP A Unconfirmed, Figure S7: Percentage cover per species by depth range for PlatEP A Confirmed, Figure S8: Percentage cover per species by depth range for PlatEP B Unconfirmed, Figure S9: Percentage cover per species by depth range for PlatEP B Confirmed, Figure S10: Percentage cover per species by depth range for PlatEP Manual, Figure S11: Percentage cover per species by depth range for PlatMS A Unconfirmed, Figure S12: Percentage cover per species by depth range for PlatMS A Confirmed, Figure S13: Percentage cover per species by depth range for PlatMS B Unconfirmed, Figure S14: Percentage cover per species by depth range for PlatMS B Confirmed, Figure S15: Percentage cover per species by depth range for PlatMS C Unconfirmed, Figure S15: Percentage cover per species by depth range for PlatMS C Confirmed, Figure S17: Percentage cover per species by depth range for PlatMS Manual.

Table 2 .
Outline of the three testing methods: confidence threshold, annotation area, number and layout of annotation points, number of images from platform and the number of images trained in the classifier.[#: number of].

Table 3 .
Number of annotation for species/labels identified incorrectly by classifier alongside the correct species/label.[#: number of].

Table 3 .
Number of annotation for species/labels identified incorrectly by classifier alongside the correct species/label.[#: number of].

Table 4 .
Difference in means between comparison pairs, two-way paired Student t-Test comparison, with Bonferroni correction per species for all testing methods (A, B and C; Con = confirmed, Ucon = unconfirmed, Eye = manual, Con-N = confirmed-normalised) for each platform (PlatMX, PlatEP and PlatMS).

Table 4 .
Cont.Bold ** = significant following Bonferroni correction (corrected p value in final row), PlatMS A Con-N vs. Eye and C Con-N vs. Eye = no significant difference across platform if combining anemone labels, Grey = no comparison available as "no data" not recorded during Manual analysis, Blank = no data recorded for label/species.Con = confirmed, UCon = unconfirmed, Eye = manual, Con-N = confirmed-normalised.

Table 5 .
Shannon-Wiener species diversity index (H), total number of species (S) and Evenness (EH) per platform and testing methods (A, B and C) at 10 m depth intervals.
H = Shannon-Wiener Index, S = total number of species in the community (richness) (Note: non-species labels have been removed: "no data", scale bar, water and infrastructure surface), EH = equitability (evenness).Blank = no data recorded.