1. Introduction
Biological processes are a focal point of many mapping and monitoring efforts in the ocean. Whether the focus is oil and gas production, minerals and mining, aquaculture, habitat mapping, or fisheries, all ocean exploration activities have a connection with biological activity as they seek understanding while minimizing detrimental impacts to marine ecosystems. However, assessing biological processes and/or impacts is a challenge for many reasons. Biological systems are dynamic in time and space, operate at horizontal scales smaller that ship board sampling allows, and the organisms being studied are often directly disturbed by the sampling platforms themselves. Sensors that measure of biological processes have also generally not kept pace with the development of physical sensors. On this last point, biological information is not obtained from a single, simple proxy, rather often the synthesis of information from a suite of sensors and some degree of sensor data fusion is required.
There are a number of examples of single data stream sensors and multi-sensor systems on autonomous packages. Alkire
et al. [
1], for example, integrated oxygen sensors, optical backscatter, beam attenuation, a chemical nitrate sensor, and a standard conductivity and temperature sensor on an autonomous profiling float. These data were used to estimate net community production and export and the mechanisms for bloom development in the North Atlantic [
2], processes that are critical for understanding carbon cycling and climate variability. The data synthesis or data fusion in this example was a post-processing endeavor after the field effort. While this kind of integrated information is the outcome of the majority of projects/programs, post-processing of the data to reveal results after a collection effort critically misses the opportunity to optimize sampling.
This leads to the primary question of what is require to optimally assess biological processes in the marine environment? Here, we advocate four elements when aggregated provide a powerful means of future investigations in the marine environment. First, and demonstrated frequently, is the need for multi-sensor systems. Multiple sourced information provides additional layers of information which often are at the core of understanding a process. Williams [
3] provides a review of early accounts of these efforts in assessing biological activity in large ecosystems using optics and acoustical approaches. There are also recent examples of this in remote sensing where multiple satellite sensors have been combined for an enhanced understanding of biological production [
4]. Second is a platform that addresses the spatial and temporal scales of the process [
5]. Traditional sampling approaches (
i.e., moorings, ships) do not capture many of these scales [
6], have reduced capacity in severe weather [
7], and in the case of ships can often influence the biology targeted for study [
8,
9,
10]; all problems that can be overcome to various degrees with autonomous platforms. There is an additional benefit with multiple networked platforms or vehicles [
11], which has been demonstrated to significantly decrease the uncertainties in predictive oceanography [
12]. Given the diversity of vehicle types, speeds, and durations, blending these overlapping capabilities further captures the scale of ocean processes [
13,
14]. Third is the need for real-time information synthesis. As highlighted above, it is combination of variables that provide new knowledge and developing real-time capabilities allow for reaction to the environment. Kirkpatrick
et al. [
15] developed a system whereby full spectral characteristics of water samples are taken, an algorithm is applied and compared with a spectral library to determine an index for identifying plankton species in real time, targeting harmful algal species, important for aquaculture, fisheries, and tourism. This sensor was then incorporated into AUVs [
16] and gliders [
17] for monitoring blooms. In another example, Baumgartner and Fratantoni [
7] integrated digital acoustic recorders in three gliders. An automated detection algorithms was used onboard to simultaneously process sound to detect and localize Sei whales. In neither of these examples, however, was the processed data product coupled with the vehicle behavior, leading to the fourth and last requirement. In combination with the previous three requirements, is the need for real-time decision making, either by the operator, or ideally, autonomously by the vehicle based on the real-time synthesized data, again to address issues of scale and relevant sampling. It is this last requirement which is largely absent when reviewing the use of autonomous platforms with integrated biological sensors. There are a number of examples of single or multiple sensor feedback into vehicle decision-making in biological applications such as plume tracking [
18] and targeted sampling [
19], but here data are used to provide improved probability of a target being present, not used to identify the target itself. One example of where this has previously been implemented is in the use of visual image processing to identify specific fish species, which in turn informed path planning [
20].
Taking all four of these requirements together, the assessment of biological processes in the marine environment is a grand challenge for a number of disciplines. For biologists, development of the synthesized products is critical in addition to defining features to target for sampling. For computer scientists and engineers, the onboard data analysis, decision-making structure and navigation interface is key. While tremendous progress has been made in the development and proliferation of autonomous underwater vehicle, there is also the ongoing engineering challenges of platform development, navigation enhancements, power sources, endurance, and sensor integration. Finally, the dynamic marine environment and specifically addressing biological problems provides for countless challenging applications for the robotics field. There are examples for these, such as localization [
21,
22], simultaneous localization and mapping (SLAM) [
23], autonomy [
24], and artificial intelligence [
25] using underwater vehicles, but the end goal or real-world application has often been missing.
Here, we will focus on one example system with a biological sensor payload that results in complex data that must be interpreted and synthesized to inform control systems and navigation for optimized sampling. While there are numerous autonomous underwater vehicle (AUV) studies demonstrating data feedbacks to inform biological sampling, this study is uncommon in that a processed data product is used to identify a specific target real time and improve sampling density of that target by a simple autonomous response. In this case example, dual frequency echosounders are integrated into a commercial AUV to target a suite of organisms in the ocean’s water column between 0 and 1200 m. The system is briefly described followed by field results and concludes with a discussion of implications and additional applications for this approach.
3. Results
Application of the complete integrated system first took place off the coast of California in 2013 with 28 mission traveling over 650 km at depths to 550 m with the longest mission of ~17 h. Flight metrics showed robustness in maintaining depth target (±0.05 m), heading (±0.1°), pitch (±0.3°) and roll (±0.3°). Level flight is especially important for these acoustic sensors making measurements 600 m away from the vehicle [
29]. In these same initial tests we determined the noise floor of the integrated system and robustness in delineating single targets. While the vehicle autonomy was not implemented during the field efforts off of California as there were prescribed transects needed to complete the intended experimental design, the data processing and threshold determination performed as intended.
A second field effort was recently completed within the Tongue of the Ocean (TOTO), Bahamas in July 2015 (
Figure 6). Here we completed 22 missions over 11 d with over 700 km underwater time (
Figure 7). These missions like the previous ones off California were designed to quantify distributions of whale prey (fish, squid, and crustaceans) in the mesopelagic (200–1000 m). Because of the need to broadly cover the area and duplicate the experimental design, as was set up off the California coast, most of the missions were long (20–30 km) transects. For one mission however, we implemented a response mission whereby real-time data processing and the detection of aggregated squid triggered the vehicle to initiate a secondary mission to resolve finer scale distributions and increase the data density of the same target organism (
Figure 8). On the positive trigger, the vehicle started a pre-programed fine grid mission and completed the entire mission prior to additional commands or the continuation of the larger mission. In order to best identify the distribution of squid, it was important for the vehicle not to react or respond to an isolated squid signal (a single “1” from the data analysis software) but be in aggregation or cluster. Therefore, the threshold for initiating the fine scale grid mission was the first instance where greater than half of the most recent 10 bins were positive (
Figure 8). The vehicle approached from the east and began the secondary mission. The grid was designed as an expanding box starting in the center with 100 m separation between boxes, growing to 200 m, and finally one at 500 m separation with dimensions of the largest box of 750 m (
Figure 8). On completing the mission, the vehicle surfaced, acquired a GPS position and corrected for the offset by ocean currents ending the secondary mission in the northeast quadrant (intended box–pink) and continuing with the original mission in the center and then follow the trackline to the west. As the vehicle does not have intertial navigation and only relies on compass heading, ocean currents pushed the vehicle off course to the south. Even though the mission data was not symmetrical, the mission goal of sampling over smaller scales was achieved and demonstrated in challenging field conditions. Data collected here showed a number of spatial features in the organism distributions that would have otherwise not been possible by other means, both in responsiveness and of a depth beyond the ships echosounder range. A ‘hotspot’ of squid was detected over the eastern edges of the 200 m spaced boxes. This scale was within a general transition to higher squid abundance to the southeast. While the contour resolution chosen for
Figure 8 is to illustrate these two scales, the binary output from a section of the southern line of the 500 m box also illustrates that there are additional higher spatial resolutions to consider for this target organism.
Figure 6.
The Tongue of the Ocean area where the AUV was tested in the Bahamas in July, 2015. Highlighted is the AUV transect and the location where the high resolution grid was run.
Figure 6.
The Tongue of the Ocean area where the AUV was tested in the Bahamas in July, 2015. Highlighted is the AUV transect and the location where the high resolution grid was run.
Figure 7.
REMUS AUV (A) being deployed and (B) underway beginning a mission in July 2015 in the Bahamas.
Figure 7.
REMUS AUV (A) being deployed and (B) underway beginning a mission in July 2015 in the Bahamas.
Figure 8.
Mission designed to resolve smaller scale distributions of organisms that was run on 10 July 2015 in the Bahamas. The intended track is in pink with the actual track in black. Offset is due to currents (see text). The rate of positive detections for squid targets per 100 m is shown in red. In the yellow box is a binary stream from the processed data at the beginning of the mission and the sequence in italics that met the threshold for initiating the finer grid mission. In contrast, the gray box is the real-time onboard binary output from a portion of the southerly leg showing higher squid densities throughout the water column. Binary data for the mission are shown as smoothed surface color plot. Apparent are two features that would not have been identified or appropriately scaled from the linear transect: a strong local gradient and a small ‘hotspot’ of squid detection across several of the nested sampling boxes.
Figure 8.
Mission designed to resolve smaller scale distributions of organisms that was run on 10 July 2015 in the Bahamas. The intended track is in pink with the actual track in black. Offset is due to currents (see text). The rate of positive detections for squid targets per 100 m is shown in red. In the yellow box is a binary stream from the processed data at the beginning of the mission and the sequence in italics that met the threshold for initiating the finer grid mission. In contrast, the gray box is the real-time onboard binary output from a portion of the southerly leg showing higher squid densities throughout the water column. Binary data for the mission are shown as smoothed surface color plot. Apparent are two features that would not have been identified or appropriately scaled from the linear transect: a strong local gradient and a small ‘hotspot’ of squid detection across several of the nested sampling boxes.
4. Discussion
Results here present a new tool for quantifying the abundance and distribution of nektonic organisms in an area of the bathypelagic previously unattainable by ship board measurements. Of equal importance to the sensor integration and the depth range are the onboard data processing and synthesis, and the ability of the vehicle to independently respond to varied products; different organisms, size, number, depth distribution and biomass. While this example is one of the simplest possible, it demonstrates the powerful combination of multiple data sources, a platform that can improve time/space sampling, real-time data synthesis, and autonomous decision-making. The mission was able to address the science objective of providing information on the horizontal distribution of squid of a particular size and the scales of their organization. Additionally, observations indicated that the squid target distribution was not equally displaced by the currents as the vehicle (550 m), suggesting these deep-dwelling organisms were is a more quiescent environment when sampled. Given the number of product variables output from processing on this vehicle, there are a host of scenarios that one could employ to improve understanding of the biology in the system. Not only could the data combination triggering the behavior be more complex, but the autonomy could also increase in complexity. While only one ‘decision’ cycle has being demonstrated here, it does not preclude the possibility for much faster cycle times for decision-making. These options for autonomy will be explored and tested in future deployments. While the possible applications of this system are numerous, it is also important to place these possibilities in the context of the scientific goals. Even the basic understanding of the horizontal distributions of these organisms in this depth region are unknown and is what motivated the first step presented here.
There are many applications for closed loop decision-making. For example in oil and gas production, there is a need for constant monitoring of the marine environment for both chronic problems and acute effects such as drill cuttings, seeps and leaks. Vehicles could be configured to measure these plumes with fluorometric sensors and internally process data to build a real-time map of a given plume, constantly updating its navigational goals. In aquaculture, low oxygen, harmful algae and disease are constant threats and integrated monitoring with long endurance vehicles, such as gliders outfitted with the appropriate sensors suite is possible [
17,
32]. Fisheries assessment is difficult and a vehicle similar to the one presented here could allow for improved quantification of biomass, school size, and migration patterns of commercial species (
i.e., herring, pollock, salmon, krill). Vehicles configured to image the seafloor could be used similarly to track populations of bottom fish and habitat [
33]. If fact, this approach has been used by Clarke and colleagues [
20] to identify and track rockfish off the coast of California. By image identification of bottom fish species and coincidently imaging the seafloor, they were able to assess associations and identify usable habitat. While the example presented here focused on layers and schools of organisms, a similar system could be used to track individual organisms, such as marine mammals and top other predators without the use of tagging as has been demonstrated [
21]. More challenging than sampling larger organisms is applying these closed-loop approaches to smaller organisms as these targets are much more difficult to identify in real time. Novel and innovative AUV autonomy work has used proxy measures of the environment to improve the likelihood of sampling larval organisms and harmful algae in vertical layers and fronts [
14,
19,
34,
35,
36]. While clearly effective, this approach may produce false positives and must be locally tuned based on target species (
i.e., Pseudo-nitzschia spp.). New imaging technologies for phytoplankton, larval invertebrates and larval fishes are currently available [
37,
38] and if integrated into autonomous platforms could directly identify these targets and influence both sampling and navigation. While the sensors, platforms, and data fusion need to vary to match the diversity of marine organisms, we advocate a strategy towards on-board product generation to drive navigation and sampling as demonstrated here could be similar.