Development of Modular Bio-Inspired Autonomous Underwater Vehicle for Close Subsea Asset Inspection

Abstract

To reduce human risk and maintenance costs, Autonomous Underwater Vehicles (AUVs) are involved in subsea inspections and measurements for a wide range of marine industries such as offshore wind farms and other underwater infrastructure. Most of these inspections may require levels of manoeuvrability similar to what can be achieved by tethered vehicles, called Remotely Operated Vehicles (ROVs). To extend AUV intervention time and perform closer inspection in constrained spaces, AUVs need to be more efficient and flexible by being able to undulate around physical constraints. A biomimetic fish-like AUV known as RoboFish has been designed to mimic propulsion techniques observed in nature to provide high thrust efficiency and agility to navigate its way autonomously around complex underwater structures. Building upon advances in acoustic communications, computer vision, electronics and autonomy technologies, RoboFish aims to provide a solution to such critical inspections. This paper introduces the first RoboFish prototype that comprises cost-effective 3D printed modules joined together with innovative magnetic coupling joints and a modular software framework. Initial testing shows that the preliminary working prototype is functional in terms of water-tightness, propulsion, body control and communication using acoustics, with visual localisation and mapping capability.

1. Introduction

The use of offshore wind power will play an essential role in our future electricity generation. It is forecast that by 2050, 12 percent of the world’s primary energy supply will come from wind energy, and 20 percent of this will come from offshore wind [1,2]. However, ongoing wear and corrosion from the harsh sea environment drives up cost and introduces downtime to this renewable and clean energy source [3]. To ensure reliable production, regular inspection tasks during high seas up to 100 m depth need to be performed in a cost-effective and safe manner [4]. These tasks are currently being conducted largely using Remotely Operated Vehicles (ROVs), which generally need tethers and a human operator, or using Autonomous Underwater Vehicles (AUVs), which are limited in their accessibility and maneuverability [5,6]. To extend AUV intervention ability and perform critical inspection tasks, they need to be efficient and flexible in operation. A fish-like AUV with a bending body of a spinal column design that is able to mimic propulsion techniques of living fish can provide efficient thrust at minimum swimming velocities, and higher maneuverability in limited spaces during sensor data acquisition.

RoboFish was created by the project “Autonomous Biomimetic Robot-fish for Offshore Wind Farm Inspection” supported by the EPSRC Supergen Renewable Energy Hub and “Innovating the Future of Bio-Inspired Autonomous, Robots for Offshore Renewable Energy Inspection” supported by the White Rose University Consortium. It was specifically aimed at investigating and exploiting bio-inspired mobility features to facilitate autonomous inspection of offshore infrastructure and is an agile and efficient biomimetic AUV that will in the near future be able to continuously inspect the foundations of offshore wind turbines and drastically reduce potential risks to divers, maintenance costs, and operational constraints. RoboFish replicates the full-body movement of an eel, allowing agility and energy efficiency in close proximity to structures.

The understanding of fish swimming behaviours and the exploration of their benefits and application in engineering designs is an interdisciplinary research field of significant and ongoing interest [7,8,9,10]. Swimming robots that mimic the techniques of natural swimmers promise to provide an increase in overall swimming performance over conventional thruster propelled systems. Reference [11] shows that thrusters waste energy by generating a vortex perpendicular to the desired thrust direction. On the other hand, aquatic animals are able to efficiently produce a jet in the desired direction through actively and passively controlled body motion. Based on the modular assembly of identical body modules and the resulting equal mass distribution, a swimming gait resembling that of an eel is anticipated. Research into eel locomotion in Reference [12] predicts swimming efficiencies of 0.5 to 0.87, depending on choice of calculation, compared to thruster efficiencies of up to 0.4 in Reference [11]. Among the two main categories of fish swimming, propulsion employing displacement of the centre line of the fish, the so-called Body Caudal Fin (BCF), is suggested to have advantages in speed and long-distance travel over flapping fin propulsion of Median Paired Fin [13]. Given that the target application of RoboFish is wind farm inspection, the slender body design of a BCF swimmer is beneficial for the anticipated long-distance travel between wind turbines, maintaining a high level of maneuverability through its body flexibility. This also makes more complex routes available that can potentially reduce travel distance. Low noise and mitigated risk of entanglement of continuously rotating parts suggests lower environmental disturbance. Furthermore, the multi-actuated system allows flexibility and adaptability in entering tight spaces and maneuvering in complex environments. The long body shape is also appropriate for a modular design, enabling extendibility and flexibility for mission setup of different intervention tasks and increased robustness and survivability in case of isolated module failures.

2. Motivation and Background

Traditionally, offshore infrastructure such as wind turbines has been inspected in person by humans, with the associated risks to safety in inclement weather and changing underwater conditions. More recently, automated inspection systems such as drones above the water and underwater vehicles have been developed, but with limited autonomy and loitering time. Human intervention to control an underwater vehicle can be quite beneficial, especially during complex inspection tasks, which require human judgement and intuition. ROVs have been in existence since 60 s [14], and received international attention following the aftermath of the Deepwater Horizon disaster in the Gulf of Mexico in 2010 [15]. In this disaster, human operators sent ROVs fitted out with a saw and manipulators to cut and cap an oil well head at a depth of one mile. The precise control, flexibility and ability to have dangerous jobs done at great depths make ROVs an ideal solution for such inspection tasks in open water. ROVs enable unique access to the underwater world and can also have robotic arms for object manipulation to provide a safe alternative to perform otherwise costly and dangerous tasks. Since they are tethered, their advantage over AUVs will, however, be restricted by the complexity of the underwater infrastructure.

Unlike ROVs, AUVs have no human intervention in their control loop, and they run more independently. AUVs are traditionally used to gather oceanographic data using cameras, SONAR and other sensing instruments. Using advanced control algorithms, AUVs can run in an autopilot mode for hours and even days without receiving constant operator guidance. REX II [16] from MIT is a unique AUV that can run autonomously and through a remote operator. While loitering autonomously, Rex II can transmit video images over a wireless channel using a tethered buoy equipped with a radio modem, which is also used in the manual operating mode to enable remote control by an operator. Odyssey IV is an AUV with a pioneered concept known as hovering [17]. It is capable of remaining stationary anywhere up to 6000 m depth. After AUVs became able to reach great depths and hover around in the oceans, the ability to operate over a longer period of time and cover an extended range were the next features to improve. AUVs can, otherwise, catch only brief glimpses in time and space of the underwater world. Thus, a newer class of more recent AUVs such as Autosub-Long-Range [18] and HUGIN-AUV [19] were developed to push beyond their powers of endurance for longer ranges and larger sensor payloads. This class of AUVs is particularly useful in offshore surveying applications.

Although the aforementioned sophisticated AUVs are extremely capable, they are not the optimal platform to operate in shallow water and inspect assets closely in critical locations due to their relatively large size, unbending bodies. Because of the limitations of AUVs and constraints of ROVs in certain applications, a new, low-cost, bendable vehicle was needed to efficiently perform research missions in shallow water and inspect subsea assets. This requirement is what initiated the design for RoboFish, a low-cost, modular, hovering AUV or wireless ROV. The concept of a flexible subsea vehicle comprising a chain of joints that are collectively able to change shape was previously successfully implemented by Eelume-AS [20]. Eelume demonstrated dexterity and hyper-redundancy that has not been commercially available before in the inspection, maintenance and repair (IMR) applications. During IMR, the vehicle is able to transit over distances and hover around using ducted lateral and vertical thrusters attached along its flexible body. Unlike Eelume, RoboFish does not use any thrusters and has the ability to run both autonomously anor remotely controlled by means of an acoustic communication system.

Fish-like robots have been an active research area due to the remarkable physical mobility of fish in nature. A review of biomimetic robotic fish, their gaits and actuators is provided in [21]. The eel gait (Anguilliform) is most suitable for the current eel-like body of RoboFish, and the trout gait (Subcarangieform) is more likely to show instability in this kind of robot than robotic fish with a trout-like body [22]. The eel gait is used in many similar robot fish and is well known in the literature. Reference [23] shows an underwater snake robot named Mamba created in 2016. These long and slender robots can maneuvre through narrow openings and confined areas. Other related fish-like robot projects include Envirobot by EPFL [24] and ACM R5 by Hirose Fukushima Robotics lab in Japan [25]. The Envirobot platform has improved energy use and efficiency than this lab’s previous segmented anguilliform swimming robots and uses an ARM microcontroller in the head unit and additional microcontrollers in each body segment. ACM R5 was developed in 2005 and to be an amphibious snake-like robot that undulates its body to move both on land and underwater. ACM R5 uses paddles for water locomotion and passive wheels on land and uses an advanced control system that includes a CPU, a battery and motors in each independently operating segment. Segments communicate to coordinate and identify automatically how many segments are joined, providing the ability to remove, add and exchange segments freely.

In this paper, we show some new features that RoboFish includes that extend the state of the art. This paper is intended as a high-level overview of the modular RoboFish architecture that uses magnetically coupled joints to form an eel-type body. We consider the way they are applied in RoboFish to be essential for fulfilling several fundamental requirements that are common to many modular autonomous underwater systems. These include a single universal end-to-end communications system; a modular control and software architecture using off the shelf parts for cost effectiveness; and a physical embodiment that is 3D-printable yet fully enclosed and watertight without the need for rotary seals. This paper describes the first working prototype of RoboFish that is equipped with an acoustic modem, a SONAR rangefinder, a camera, and uses computer vision for close range navigation and inspection of structures with the ability to build complete visual models of the structure using 3D reconstruction methods. This prototype is a cost-effective underwater platform and could be spun out to a successful commercial product.

The paper is organised as follows: Section 1 is an introduction; Section 2 provides the motivation and background; Section 3 discusses the system design; Section 4 describes the vision system; Section 5 describes the acoustic communication system; Section 6 is the locomotion control design of the RoboFish.

3. Robofish Design

Development of a modular bio-inspired autonomous underwater vehicle for close subsea asset inspection is a task of extraordinary hardware and software challenges (shown in Figure 1). Splitting a protective, watertight 3D printed enclosure into jointed segments, collectively mimicking the motion of a fish, is an example of these challenges. To overcome this, innovative mechanical and electronic modular designs were created, as this section introduces.

3.1. Vehicle Requirements

The current RoboFish design was created within the scope of offshore wind farm inspection. While the mission of RoboFish is clear, there were a number of other requirements that had to be involved into the design such as affordability, underwater docking, manoeuvrability and acoustic remote control. To meet all the requirements, the academic and industrial project partners were involved in early design meetings. The following list outlines the partners that were involved in defining the current RoboFish prototype’s requirements.

• University of York (Intelligent Systems and Nanoscience Group and Underwater Communication Group)
• University of Strathclyde (Computational Fluid Dynamics and Fluid Structure Interaction Research Group)
• Supergen ORE Hub
• PicSea Ltd. (Edinburgh, Scotland)
• EC-OG Ltd. (Bridge of Don, Scotland)
• Offshore Renewable Energy Catapult

Consulting with the aforementioned partners, the budget boundaries were defined in order to avoid involving materials, features and characteristics that were beyond the budget. Next, through collective research and engineering discussions, the minimum requirements to operate RoboFish in the ocean environment around wind farms was defined. Finally, the type of data required in inspection missions was decided. The primary RoboFish requirements defined in the early design stage are:

• Manoeuvrability
• Affordability
• Portability
• Modularity
• Self-sufficiency

3.2. Key Performance Attributes

Ideally, all design requirements were defined at the top level to ensure that the mission of RoboFish was comprehensively covered. In the design process of RoboFish, the attributes that ensure meeting the minimum design requirements were further defined. This was achieved by creating Key Performance Attributes (KPAs) as depicted in Figure 2. KPAs were linked to the top-level design requirements in order to determine how RoboFish would meet the overall requirements of a subsea asset inspection mission. The current RoboFish KPAs are determined based on the mission of offshore wind farm inspection and are measurable design characteristics that control the overall effectiveness of the RoboFish design. The KPAs for the current prototype are listed in

Table 1. RoboFish Key Performance Attributes (KPAs).

Attribute	Objective
Depth [m]	100
Mission Duration [hrs]	3
Weight [kg]	30
Length [m]	1.9
Duty Cycle [%]	75
Modular	Yes
Speed [knot]	0.5

. Based on the top-level design requirements, a decision matrix was created to determine the best off-the-shelf options with regards to batteries, cameras, servos and micro-controllers. Using KPAs, associated weights are used to evaluate each decision matrix. In general, the authors were guided by a design philosophy that can be quoted as:

Design a low-cost, modular AUV to perform underwater inspection around complex structures. To keep costs at minimum, off-the-shelf parts and accessible additive manufacturing technologies will be used. The vehicle will be easy to launch, capture videos, recharge and return to a home location with minimum or no human intervention.

3.3. Mechanical Design

RoboFish is composed of several separate body segments with a head at one end and a caudal fin at the other end. The segments are joined together using an innovative magnetically coupled joint. This allows it to have the required multiple degrees of freedom in its agility in order to move very precisely by aiming its head and undulating its body. With this type of locomotion, RoboFish features greater agility in close proximity to structures compared to conventional underwater vehicles. The current RoboFish prototype was developed using off-the-shelf parts and a common 3D printing technology, i.e., Fused Deposition Modelling (FDM). The prototype currently consists of three sections due to space constraints of laboratory testing. Being modular, it is scaleable and expandable. Five sections were created and can be assembled easily during field testing to produce longer operation time, more efficient movement and higher agility. Buoyancy control is necessary for long-term loitering capability of biomimetic vehicles, and the buoyancy control of RoboFish is currently still being refined in design as the miniaturization and pressure capability of such a buoyancy unit is a considerable challenge. To allow pitch control, one buoyancy unit is ultimately installed in each segment of RoboFish, and the segments operate independently to trim the attitude of the vehicle. The buoyancy units draw a small amount of water from a port outside the body segment and compress the air inside to increase the mass of the segment a small amount, enough to offset the buoyancy of the vehicle for rising and diving. Roll is statically limited by placing the batteries low in the body.

3.3.1. Body Segment

This is a 3D printed enclosure using Acrylonitrile Styrene Acrylate (ASA) material. The primary part of the enclosure takes the form of cylinder of 9.3 cm internal diameter and 23.3 cm length, as shown in Figure 3. The total length of a segment can be variable with any modifications that are needed, but the length of the current configuration is 43 cm due to the size of the servomotors used. To reach the inside of the enclosure, O-ringed stainless steel rings with a male-to-female fit are used to hold the two parts of the enclosure together. This allows convenient disassembly while keeping the system watertight under high pressure. The enclosure is designed with a fork at one end to interlock with the rotor of the following segment, whereas the other end of the enclosure is fused to a magnetic coupling joint containing a rotor. The top of the enclosure allows wire entry via M10 penetrators, making a waterproof, high-pressure seal to pass Ethernet cable into the segment. The bottom of the segment is fitted with a M10 plugged vent, allowing trapped pressure to escape from the segment while it is being closed. This is also used for testing water tightness on the segment using a vacuum pump inserting into the enclosure vent. Segments are joined together using a magnetic-coupling joint that allows a servo in each joint to rotate an external rotor that in turn rotates an internal rotor to move the next joint connected to the fork. Four guides with holes are built in on the outside circumference to allow the attachment of fins, ballast, or other accessories as required. Internally, components are mounted on a 3D-printed mounting plate. The servo fits into a 3D-printed frame moving on linear rails, working as a tilting drawer to provide the required tension for the timing belt by adjusting the sliding servo on the rails and locking it in place with two screws.

3.3.2. Head

This is a modified segment with the same 9.3 cm diameter cylindrical enclosure, but with a front end that appears like a cockpit, allowing the attachment of clear acrylic dome end cap. The dome shape allows for extra room within the head for additional two or more cameras or sensors. It gives the camera a wider view than that of a flat end cap. It is very transparent and does not warp or distort camera images. The dome is fit into the head using a flange that has a double O-ring seal. Like the other segments, the head enclosure is fit with a pressure-releasing vent and two cable penetrators. It is also provided with an additional M10 penetrator at the nose of the head, allowing a waterproof high-pressure seal to pass a 4–8 mm tether into the head (should it be required). To mount the acoustic modem and rangefinder on the head without being obstructed, the head has an external hollow at the bottom, in which both devices are placed. Internally, like in the segment, components are mounted on a 3D-printed mounting plate, and a servo is fitted into a pull-on 3D printed frame (shown in Figure 4).

3.3.3. Tail

This is modelled after a caudal fin directly connected to a magnetic joint that enables active control of the fin motion, manoeuvrability and thrust generation for the overall body. An appropriate fin design can contribute to the overall device stability and manoeuvrability. Many species use their caudal fin as the main propulsive and manoeuvring appendage in addition to the body. For example, almost all of the thrust comes from the caudal fin for Thunnus albacares and Acanthocybium solanderi as suggested by Fierstine and Walters (1968) in [26]. Moreover, the tail may also help produce lift force to balance gravity and buoyancy [27]. In the current design, the caudal fin is directly attached to an actuated joint (shown in Figure 5). This makes it possible to optimise the interaction between the body and tail to enhance propulsion performance and achieve manoeuvrability, e.g., braking, when necessary. The caudal fin in this work has another function to provide additional buoyancy by using a hollow design. In this way, the mass of the caudal fin itself is decreased, and it also reduces the energy consumption when the joint servo actuates the rotation of the tail.

Using Computational Fluid Dynamics (CFD) techniques and Fluid Structure Interaction (FSI) numerical solvers, it was possible to numerically study the propulsion performance ahead of the manufacturing stage. This provides insights into the structural design and material selection. Using a fully coupled FSI numerical solver consisting of a finite-volume-method-based fluid solver and finite-element-method based structural solver [28], a preliminary analysis was performed on the motion control of the simplified system [29]. The caudal fin was simplified as a 2D cross-section in rotation locomotion. The yaw angle was a result of PID control with feedback, and the control objective is to find the yaw angle matching with the specified steady swimming speed. Initial results showed that the medium stiffness is the most favourable in terms of thrust production, which provides insights into our material selection of the caudal fin and locomotion parameters in the design of the AUV.

The current fin is printed with ASA materials, which are rigid, to manufacture a fish-inspired tail. Subsequently, the project consortium is curious as to whether flexibility can enhance thrust production, and if so, how flexible the fin needs to be to achieve the most thrust improvement. For a real fish, the conformation of flexible fins would be changed as the fin rays and membrane deform under hydrodynamic forces and inertial force. In return, the fin deformation changes the surrounding flow field, and thus, the resultant force conditions of the fin. During the dynamic interplay between the flexible caudal fin and immersed fluid, the propulsive capabilities may be improved significantly compared with cases when a rigid fin is adopted.

3.3.4. Magnetic Coupling Joint

This is a mechanism that mechanically joins two watertight enclosures together and transmits the torque of a rotary actuator between an outer driving shaft and an inner driven shaft without physical contact. This enables a servomotor in one of the enclosures to actuate the other enclosure and achieve a precise control of angular position, velocity and acceleration of the body. The contactless bond is created by the magnetic attraction of a number of magnetic blocks evenly distributed on the side surface of the two shafts with opposite polarity. This allows the two enclosures to function like a robotic arm with rotational joint motion. To keep costs to a minimum, off-the-shelf small magnetic bricks were used. Figure 6 illustrates the magnetic joint’s internal parts. The recent paper [30] provides additional details about the implementation of RoboFish magnetic coupling joints and how to maximise the transmittable torque with different numbers, types and arrangement of magnetic blocks.

3.4. Electronic Design

A simplified design schematic of the RoboFish electronic systems is shown in Figure 7. RoboFish uses a modular software and hardware architecture. Each segment is self-contained and includes self-managed battery power, internal and external sensor data, and actuator control using a low-cost microcontroller. Communications and power transfer between segments are performed through a customised 100 Mbit Ethernet bus, and it can charge autonomously underwater by docking with a source such as EC-OG’s Subsea Power Hub. The head segment contains a powerful Xilinx Zynq SoC that serves as a master control node, communications router, and FPGA-accelerated vision platform with an acoustic rangefinder for position detection. While Wi-Fi communication is only available on the surface, RoboFish can also communicate at low rates underwater by an acoustic modem. It currently uses vision for close-range navigation and inspection of structures, with the ability to build complete visual models of the structure by using 3D reconstruction methods.

3.4.1. Requirements

As the RoboFish project aims to produce an autonomous agent, significant processing capabilities are required. On board real-time vision processing is required for navigation. Acoustic communication is required for feedback and issuing control commands during operation. Pressure sensing is required for water depth acquisition. A SONAR sensor is used for range-finding. Each of these sensory inputs are to be used as inputs to the control system of the robot. Actuation is produced using servo motors. The system of inputs and outputs is summarised in Figure 8.

3.4.2. Hardware Choices

To fulfil the requirements stated in the previous section, while also making the platform upgradable in the future, the Xilinx Zynq 7000 SoC platform was chosen for the main processor of the system. The Zynq 7000 SoC is built around a hybrid processor and FPGA architecture. It consists of two ARM Cortex-A9 processor cores and Artix-7 FPGA-programmable logic, with a high-bandwidth AMBA–AXI interface between them. This platform enables rapid development of software systems using a Linux operating system on the processor cores, with the ability to offload processor-intensive tasks to the FPGA fabric. Offloading demanding tasks to the FPGA speeds up execution time for tasks like vision processing with potential power-saving benefits too, which is important for a battery-powered autonomous vehicle such as this. The FPGA fabric can also be used to create an inter-segment communications controller for communicating between the head and other segments without sacrificing processor time, resulting in higher-reliability communication. For the other segments in the robot, the STM32 platform was chosen. Each segment is a modular element of the system, which accelerates development and upgradability.

3.4.3. Hardware Implementation

Head board: The head board is based around a Trenz electronic TE0720 system on Module. This module incorporates the Zynq 7020 SoC, a 1 GB DDR3 RAM, 32 MB QSPI flash for configuration, an 8 Gbyte E.MMC flash for non-volatile storage, along with the power supply and configuration electronics for the SoC. This module was chosen over creating a custom board to accelerate development and ease upgradability (shown in Figure 9). If additional processing power and FPGA fabric is required in the future, this module can be swapped for a more powerful one without affecting the carrier board.

The carrier PCB, shown in Figure 10, contains all of the necessary hardware for interfacing the Trenz SoM with the rest of Robofish, programming the SoM and regulating the battery power. Camera interfacing can be accomplished using either a MIPI CSI-2 connector or USB. An SD card slot is provided to increase onboard non-volatile storage. For communication with other modules in the system, CAN was used for initial testing, and Ethernet was chosen as the final solution. Power is transferred between modules by using a modified power-over-ethernet (PoE) methodology with the DP83825 PHY chip and HX1198FNLT transformer IC. It also contains an LSM9DS1 IMU to provide orientation awareness of the head segment. The head also interfaces with the acoustic modem and SONAR rangefinder via RS-485 bus and breaks out GPIO pins used to drive LEDs, one PWM signal that controls the servo that drives the movement of the segment, and another PWM signal to be used for a buoyancy control unit that is still in development as of this writing. A general SPI and power pin header is provided for future expansion also.

Segment board: The segment board is built around an STM32F417 Microcontroller. This serves as a networked extension to the robot’s capabilities in a segment. It communicates with the head board using CAN bus (initial testing) or Ethernet with PoE and contains all of the necessary IO for any servos or sensors that may be required. It also contains an LSM9DS1 IMU for orientation awareness (shown in Figure 11) and breaks out control pins for driving LEDs and the servo and a buoyancy control unit with PWM and the general SPI and power pin header.

4. Underwater Vision

While visual simultaneous localisation and mapping (SLAM) has seen impressive development for autonomous ground vehicles (AGVs) [31], unmanned aerial vehicles (UAVs) [32] and unmanned underwater vehicles [33], the technical challenges presented by underwater environments have hindered progress for AUVs, particularly in real-time applications. Many unique visual phenomena affect underwater images such as wavelength-dependent attenuation, floating particles and bubbles, underwater caustics in shallow water, varying lights and shadows, moving flora and fauna and refractions through thick glass housing needed for waterproofing camera systems [34,35], some examples of which are shown in Figure 12.

In the RoboFish project, the research aimed to test current state-of-the-art SLAM algorithms on underwater visual datasets and to quantify performance and suitability of those algorithms for use with low-cost Raspberry Pi cameras. To achieve this, a graphical user interface (GUI) was developed in Python and OpenCV [36] to enable the real-time modification of popular feature-matching algorithm parameters whilst providing visual feedback on performance and an estimation of the camera’s 3D trajectory using visual odometry (VO). The most suitable parameters and image-processing algorithms were then determined and implemented in a modified version of ORB SLAM 2 [31].

The GUI was built in Python using the Matplotlib library. It was decided that only ORB [37] and BRISK [38] feature-matching algorithms would be tested; however, the design enables the addition of SIFT [39] and SURF [40] feature detectors with only minor modifications. Figure 13 shows the GUI. It enables the adjustment of either ORB or BRISK parameters in real-time via sliders and buttons, with the effects of these changes visible both qualitatively in the overlaid video feeds and quantitatively in the graphs. Parameters can also be set prior to a test, and it enables a previous test’s data to be displayed simultaneously on the graphs, allowing comparisons of performance for each test. The camera’s position is estimated using VO, the implementation of which was based closely on PySLAM [41].

5. Acoustic Communication

The RoboFish-specific powerful Xilinx Zynq SoC acts as a minicomputer on-board, processing a number of operations, one of which is communication. A half-duplex 64 bps acoustic modem, called Water Linked M64 Acoustic Modem [43], is used to provide low-rate communications at medium range (i.e., 200 m) for remote control, telemetry, and inter-vehicle coordination. This self-contained modem supports omnidirectional operation, which keeps the data link stable even when the RoboFish is in motion. It is programmed with a packet-based protocol with extensive use of error detection to enable a highly robust transmission at very low power consumption. It communicates via a serial 115,200 baud UART 3.3V interface with the SoC board. Its small size enables easy integration in the RoboFish head. The Xilinx Zynq SoC includes an FPGA, which will be used for acceleration of inter-vehicle communication architectures, protocols and applications for efficient RoboFish swarm communication networks in the future.

An interactive Python GUI, shown in Figure 14, was developed to run the RoboFish manually from a distance using the acoustic modem. The modem has a configurable data link and is interfaced using a lightweight API, on which the GUI design is based. The default serial protocol is documented in Reference [44]. This document describes the modem’s Data Link Layer protocol. With this protocol, packets are sent to and received from the modem with serial communication commands taking this format 115,200 8-N-1 (payload size is 8 bytes). A Python script was put together to enable sending and receiving of these commands to and from the modems through the serial port. The commands can be sent as a string represented by descriptive variable names or the GUY. By configuring the modem that is installed in RoboFish as a receiver and the topside modem as a transmitter, an operator can send these predefined commands to control RoboFish manually over the acoustic channel if required. Through this GUI, the operator can primarily control the degree of freedom for each joint by sending over acoustically the required angle from the topside computer to RoboFish. In addiition, the GUI enables remote ON/OFF control, steering, selection of communication channel and displays notifications received from RoboFish in humanly readable format for the operator.

In addition the acoustic modem, RoboFish uses Ping SONAR Altimeter and Echo-sounder [45] that is a single-beam echo-sounder with a maximum range of 30 m, a beam width of 30 deg and a maximum depth rating of 300 m. It is connected to the RoboFish’s SoC through a serial connection using one of its Serial/UART ports. Distances read by this Rangefinder can be read from a user interface running on the operator’s computer.

6. Locomotion Control

Biological fish in nature repeat the same locomotion pattern for swimming to move forward straight over a given period. It is possible to construct a precise mathematical model through analytical approaches because its locomotion involves hydrodynamics and kinematics [46]. However, for real-time control with microcontroller hardware, a simpler parametric control method is sought. Using hydrodynamic analysis, control parameters that produce stable locomotion are produced for two approaches to locomotion that are currently being tested, as follows.

6.1. Conventional Control

The first step of most conventional control design procedures is to establish the mathematical model of the dynamic system, which is a set of ordinary differential equations [47]. The RoboFish has multiple joints and strong influences from the operational environment. The control problem for stabilising the attitude and maximising the forward velocity using the causal fin is high-dimensional and underactuated. Designing a controller taking into account the full nonlinear dynamics is challenging. The second step is obtaining an approximate model for each operation scenario, i.e., the forward swimming or the turning manoeuvre. This step is frequently performed using the feedback linearization procedures [48]. Recently, reinforcement learning provides a promising performance to deal with nonlinearity directly with less conservative design problems [49]. The third step is to design a controller for the linearized system using linear control design procedures, e.g., LQR (Linear Quadratic Regulator) and PID (Proportional Integral Derivative) [50]. There are several attempts to combine reinforcement learning with conventional control [51,52]. The combined methods would provide the capabilities to exploit the nonlinearity in the nonlinear region and provide stability assurance in the linear domain. Internal uncertainties and external disturbances would deteriorate the stability and the performance. An external disturbance observer is combined in the last step of the control design [53], and finally, the robustness analysis is performed [54]. In summary, the first control method implemented on RoboFish will be a conventional controller combining linearization with reinforcement learning.

6.2. CPG-Control

Traditional model-based control via numerical techniques, kinematic approaches and geometric approaches is not always very well suited to dynamic and changing conditions [55]. Biological systems produce rhythmic patterns using a functional unit called a central pattern generator. A CPG can be considered as a dedicated neural mechanism involving a group of neurons that generate coordinated rhythmic signals without sensory feedback [56]. While sensory feedback is needed to shape the CPG signals, the CPG can run independently without input. This method is widely used for the locomotion of robots such as crawling, flying, swimming, hopping, walking and running. The general design of CPG-based control has been focused on three aspects: CPG modelling and analysis, CPG modulation (parameter tuning and gait transition) and CPG implementation [57].

6.3. RoboFish Locomotion Control Architecture

In Figure 15, the RoboFish prototype is shown with its main control components. A monocular camera in the head is used for visual odometry and for detecting and tracking obstacles in the environment, with image processing running on the Zynq Z-7020 SoC in the head module. The inertial measurement units in each module of the body provide dynamic feedback from the body position. These are the main sources of sensory input for the locomotion control system. Currently, in the absence of sensory data (for example, if no visual odometry information is available), the system runs in open-loop mode, and control parameters for forward velocity and angular velocity are read directly from the desired movement commands. The output of the CPG-based controller is transmitted to the servo motors in each joint via PWM signalling. The feature parameters of CPG will change the speed of the robotic fish while swimming. The power consumption of the servo motors will be recorded to compare the energy consumption corresponding to specific sets of CPG feature parameters. The modulation of the CPG will be restricted by each module’s battery life. A comparison of swimming performance resulting from the conventional control methods cited and the CPG design will be done after both control methods are implemented on RoboFish.

7. Initial Testing and Lessons Learned

The work described in this paper led to the initial testing of the first RoboFish prototype shown in Figure 16. This prototype is mechanically quite mature and had a minimum number of completed modules in the initial testing to test water-tightness in the first place. Although full autonomy has yet to be integrated into this prototype, adequate electronic parts and processing capabilities were included in the initial testing to fully program the vehicle with a basic operating system to primarily test propulsion. The computer vision system and acoustic communication system have been completed, and next trials will be fully integrated into the prototype. As a proof of concept, both systems were tested separately in the initial testing, and they were fully operational.

7.1. Testing Propulsion

This prototype is printed in ASA, with print parameters listed in

Table 2. List of the 3D Printer Parameters.

Parameter	Value	Comment
Layer height	0.254 mm	Standard
extrusion width	0.5 mm	Standard
Wall thickness	2.032 mm	To print more perimeters per layer
Solid infill	Enabled	To help preventing water ingress
Variable width fill	Enabled	To fill any small gaps
Room temperature	${25}^{\circ }$	Enclosure

and KPAs listed in Table 1. The prototype underwent its first test outdoors in December 2020. The test went well and answered a number of questions. In this test, the prototype undertook some important tasks, but the test was not a very long test that examined all the RoboFish features. This test was the foundation of more task-oriented trials to come. The objectives of the test can be summarised as follows:

• Testing water-tightness
• Testing the functionality of magnetic-coupling joints
• Testing propulsion

These initial trials were conducted in the University of York Campus West lake. The depths were around 1–2 m, with temperature of around 8 °C, 10 mph wind speed and poor water visibility. The prototype was put together and tested briefly on the shoreline (the lake’s edge platform) just before it was let go into the water as shown in Figure 17. In one testing scenario, the RoboFish was dropped slowly into the water from the platform using two ropes. To test swimming on the surface, two side plastic buoys were included to maintain positive buoyancy and good balance with the right position by preventing RoboFish from going below surface or turning upside down. With it being directed toward the centre of the lake, the Go button was pressed and the RoboFish swam as expected. It was tethered to be brought back to the home point in the case of failure or untimely need for battery recharge. In another testing scenario, the RoboFish was released to operate underwater. This was the first outdoor trial for RoboFish. The shallow lake seems to be an ideal place to carry out more tests to examine the functionality of control, electronic and communication. As for computer vision, the location needs to be investigated further.

Given that it was the first real outdoor trial, the performance of RoboFish was as good as was predicted. Initial testing of the propulsion mechanism revealed problems with electrical connections and power cable wiring associated with batteries. To overcome this, a new battery mounting plate was designed and is currently being 3D-printed to enclose all of the power network connections. The prototype is fitted out with cable penetrators, ensuring watertight connections for the discrete cable that is used for both power distribution and control signal communications between modules. In future design, plug and bulkhead socket connectors would be a better option. Furthermore, if the modules are equipped with wireless chargers as an option, it will save time, especially during testing. Improvements in its buoyancy, thrust and swimming gait can be achieved via further hydrodynamic analysis. This could involve making the head undulate less and the tail oscillate more. Adding more segments will also improve the swimming gait.

7.2. Testing Computer Vision

In order to quantify the performance of the computer vision system, a dataset with ground truth was required. To the best of our knowledge, one of the only underwater datasets to provide a trajectory estimate is the AQUALOC dataset. This dataset provides an structure-from-motion trajectory calculated offline [42]. The assumption was then made that improvements in the accuracy of the PySLAM-based VO calculated using ORB features would result in improvements to ORB SLAM 2. A Python script was written to cycle through various OpenCV image processing techniques (e.g., histogram equalisation and image filtering) and multiple ORB and BRISK parameters to determine which combination produced the most accurate estimate of the camera’s trajectory. This was determined using the mean squared error between the VO estimate and the structure-from-motion ground truth trajectory obtained from the AQUALOC dataset. A graph of the result of these tests with the most accurate configuration selected is shown in Figure 18.

It was determined that the highest accuracy was achieved when using Contrast-Limited Adaptive Histogram Equalization (CLAHE) and an ORB feature matcher with the following parameters: Edge Threshold and Patch Size of 30; Minimum FAST Threshold of 30; First Level of 4; Maximum ORB Features of 1500 and all others at default OpenCV values. The ORB SLAM 2 code was then modified to include CLAHE image processing and the calculated ORB feature-matching parameters. This was then compared against a version of ORB SLAM 2 without CLAHE image processing and using ORB-SLAM 2’s default ORB feature-matching parameters. Tests were conducted on both the AQUALOC and Marine Autonomous Robotics for InterventionS MARIS [58] underwater datasets. The modified ORB SLAM 2 appeared to yield improved SLAM accuracy, losing tracking a reduced number of times on each dataset. ORB SLAM 2 ran at usable framerates on a Raspberry Pi 4 of around 15–20 fps, suitable for slow-moving AUVs. It is recommended that ORB SLAM 2 with the provided settings be used as an initial platform on which to develop further underwater visual SLAM robotic applications.

7.3. Testing Acoustic Communication and Rangefinding

The RoboFish prototype uses an M64 Acoustic Modem [43]. Because this modem was still a Beta version during the initial testing, a number of in-water trials were conducted to establish whether the two pairs of RoboFish uses were working. Both modems were functional, and a point-to-point acoustic link was established and packets transmitted over it successfully. Apart from minor issues in the beginning, mainly with wiring and serial port configurations, the modem’s Channel 3, which is between 93.75 khz and 125.00 khz, offered a very reliable acoustic link over 50–80 m range in open water, as well as inside a compact water tank of 302 L. Channel 1 had a lower signal strength causing a shorter range. Channel 4 was more unpredictable, as it worked but with a shorter range and was slightly unstable. Channels 6 and 7 were not tested, as they would give a shorter range and were not required at this stage. These parallel channels can be used by RoboFish for networking in the future, as it is possible to switch between channels to enable communication between more than two modems without packet collisions (but not at the same time).

The minor wiring and interface issues were related to the 3.3V UART to USB serial converter. A pair of Blue Robotics’ BLUART USB-to-serial converters [59] were used. To avoid such issues, the converter and the modem need to be common-grounded. The UART TX from the modem needs to be connected to the UART RX on the converter board and similarly for the RX pins. The modems need to work in water to avoid unwanted overheat. A blinking light about every 2 s on the modem will indicate it is powered, but no link is established. The head of the RoboFish is designed so that it has the modem fitted outside.

The range finder was also tested and is currently fully operational in RoboFish. Its readings will be integrated in the final mission-oriented control system. Distances read by this Rangefinder can be read from a displaying interface running on the topside computer. This window consists of four important components as shown in Figure 19:

• Distance Readout: The Distance Readout presents the distance to the target in the latest measurement. The reading that is shown in Figure 19 was the distance to the floor in a testing tank during RoboFish’s initial trials. The confidence measurement for the newest range reading is presented below the distance reading and is colour-coded based on strength as follows: green = 100%, yellow = 50% and red = 0%.
• Distance Axis: This vertical axis represents the distance from the transducer built in the Echo-sounder. It starts from the top of the window, which represents zero distance from the face of the transducer and runs down vertically with the distance to the farthest object being at the bottom. Its scale automatically adjusts to indicate a live scanning range of the rangefinder.
• Return Plot: The Return Plot presents the echo strength against the distance of the newest profile sample. The stronger an echo is, the wider its trace appears.
• Waterfall: The Waterfall is a 3D trace presenting consecutive profile samples. The X axis is time; and Y axis is new distance reading shifting from right to left as a new echo arrives.

8. Future Work

The RoboFish prototype is under continuing development. Future versions of a smaller-size RoboFish, with particular focus on the modularity of the body design and easy connect/disconnect magnetic joints, will provide a flexible and dynamic platform for numerical data validation and experimental investigation in hydrodynamic laboratory testing. This will be highlighted in future projects as this work could not be done under the pandemic restrictions. Anticipated investigations include the analysis of the flow field influenced by different fin and body geometries and kinematic locomotion parameters, smart soft materials for passively deformed body parts as well as analysis of different actively controlled body kinematics using linear, nonlinear and CPG-based control. This will provide further insight to disseminate the hydrodynamic performance under different flow conditions to prepare for application within complex chaotic and harsh ocean environments. In a practical sense, this will especially support the targeted underwater docking, which requires accuracy and reliability of the swimming motion. Another direction of future work is to investigate the use of networks or swarms of RoboFish carrying out large-scale subsea monitoring or exploration missions, e.g., seafloor mapping, marine archaeology. This will involve a significant challenge in implementing underwater network protocols for cooperative acoustic localisation and navigation, real-time remote control and data gathering from multiple RoboFish.

9. Conclusions

The work described in this paper led to the development of a fish-like AUV, namely RoboFish, with a bending body that works as a spinal column and is able to mimic propulsion techniques of living fish. The first RoboFish prototype was built successfully and was able to complete minimum lake trials. A substantial amount of knowledge was gained from the construction of RoboFish about the technologies that a robotic fish requires to be able to loiter with a camera around complex structures autonomously or remotely controlled over an acoustic link. The use of modular electronics and actuator control algorithms, the networking architecture, the 3D-printing approach and the magnetic joint design are novel contributions to the state of the art that will enable new opportunities. This represents opportunities for additional research arising from further field tests of RoboFish and increases the likelihood of more advanced RoboFish versions.