1.1. Motivation
Nowadays, the increase in the world’s population represents a challenge for the food industry. Environmental considerations in food production are one of the main challenges. Livestock activities generate a significant quantity of greenhouse gas emissions, not to mention excessive water consumption and land use. To address these environmental challenges, exploring alternatives outside of traditional land-based solutions becomes crucial. The ocean, with its abundant resources, can provide one promising avenue. Seafood, mainly fish, is a high-quality protein source with high nutritional, excellent amino acid scores, and great digestibility characteristics [
1]. Current fishing activities need to be applied sustainably, otherwise this will lead to environmental degradation and will generate a negative impact on the wild fish population. This is where fish farms can help to reduce the impact of fishing by offering an alternative source of seafood.
Currently, fish farms play a significant role in the food industry. According to the Food and Agriculture Organization (FAO), aquaculture represented 49.2% of global fish production in 2020, nearly half of the quantity of wild-caught fish [
2]. The European Commission [
3] states that fish-farming activities directly employ 70,000 people. The sector consists of 15,000 enterprises, primarily small businesses or microenterprises in coastal and rural areas. Currently, the prevailing trend is to expand fish farms and promote sustainable fishing methods, as described in [
4].
Fish farms are predominantly sea-based and primarily consist of floating net cages. These net cages are typically built using circular plastic structures with diameters greater than 20 m and depths varying from 15 m to 48 m [
5]. The working environment of a fish farm is hostile, it involves physically demanding tasks, exposure to low-water-quality environments, working in confined spaces, and continuous exposure to various hazards and safety risks. Certain operations and tasks, such as cleaning the net mooring, inspecting the nets, repairing the structures supporting the nets, or collecting fish carcasses are some of the most performed on a daily basis. According to a study conducted in Norway, there have been 34 recorded fatalities and multiple accidents from 1982 to 2015 [
6].
Therefore, it is necessary to continue bringing more comfort and safety to fish-farm activities so that people are not exposed to hazards, and the work can be done in a more efficient and secure way.
1.2. Related Work
The demand for robots in aquaculture environments is growing because of the needs of the industry. Robots can offer accuracy in repetitive tasks, real-time inspection, monitoring, and they can also reduce the manual labor of workers [
7].
Activities such as cleaning the net mooring, inspecting the nets, repairing the structures supporting the nets, or collecting fish carcasses represent a significant expense for fish farms, primarily because of the specialized training, certifications, and equipment required for scuba divers working in this environment. Additionally, the limited time divers can spend underwater and the need for multiple divers to perform a mission due to safety reasons (as the Aquaculture Safety Code of Practice establishes [
8]) increase the complexity and costs of the operations. However, advances in technology have introduced potential solutions, such as remotely operated vehicles (ROVs), which can minimize the exposure of scuba divers to hazards. Solutions like [
9], where the authors developed an analysis of a novel autonomous underwater robot for biofouling prevention and inspection can help to reduce the use of scuba divers in complex and dangerous missions.
Some problems that have been addressed using autonomous underwater vehicles (AUVs) are net inspection, net cleaning, removal of objects from the net, and gathering fish carcasses from the net cages. Examples of solutions and tools for each task are described next. In [
10], an inexpensive underwater robotic arm was created to collect objects and fish carcasses from the nets. For the net cleaning task, AUVs have been used to reduce and clean biofouling in the cage structure. Furthermore, a modeling analysis of a spherical underwater robot for aquaculture biofouling cleaning was developed by Amran et al. [
11]. That paper focused on the mechanical design and the finite element analysis, showing that the designed shape produced a low friction coefficient. Finally, the net inspection problem and the other problems mentioned are significant challenges due to different factors such as the size and location of the nets. Additionally, unexpected incidents can occur that may damage the nets, complicating the task. Moreover, the safety and risks for human operators cannot be overlooked.
The net inspection problem at fish farms is a captivating issue that experts have addressed in the literature. The net inspection task can be divided into different subtopics, starting with mechatronics and the robotic platform or focusing on the underwater robot trajectory and computer vision for detecting holes or any other anomalies. The most commonly used robotic platform for the net inspection is a ROV or an AUV equipped with a high-definition camera, embedded computer vision algorithms, localization techniques using proper sensors (such as Doppler velocity log (DVL), ultrashort baseline (USBL), and an inertial measurement unit (IMU)) and a trajectory-tracking algorithm. Some works go further in the solution and include a surface vehicle or a floating platform to increase communications and facilitate the ROV launch, control, and recovery. For instance, Osen et al. [
12] designed a floating station. That floating station could serve as a robotic platform capable of omnidirectional movement as well as provide a ROV docking and locking mechanism, along with a winch used to lift and control the umbilical cable of the ROV.
Communication plays a significant role in this application. The restrictions of underwater communications has led to the development of a floating platform or a unmanned surface vehicle (USV) that can serve as a link between the underwater robot and the ground station. Examples such as in [
13], where a platform composed of USV swarms was used for real-time monitoring. That platform proposed a simultaneous multicommunication technology in a synchronized approach using LoRa, IEEE 802.11n (WiFi), and Bluetooth as a way of creating a multirange communication channel. It is important to note that it was designed to work in aquaculture environments. In this case, as in most of the works in the literature, the connection between the ROV and the surface vehicle was achieved with a tether cable, but some works such as [
14] proved that optical and acoustic communications could be used to establish a wireless communication link between the underwater robot and the surface vehicle. We have performed several experiments to cope with underwater wireless communications (UWC). Acoustic and optical modems have been used to deal with this issue.
After introducing the robotic platform concept and going further into the specific inspection trajectory, some underwater localization concepts are presented. It is well known that robot localization in underwater environments is not as simple as in air, mainly because of the nonfunctioning of the Global Position System (GPS). Su et al. [
15] conducted a review of the state of the art for underwater localization techniques, algorithms, and challenges. The authors pointed out that the most popular sensors used in small environments like a fish farm that allow the positioning of the robot are acoustic-based, including sonars, DVL, and USBL. Evidence of this can be found in the research conducted by Karlsen et al. [
16], where an implementation of an autonomous mission control system for unmanned underwater vehicle operations in aquaculture showed the performance of the localization and trajectory tracking of an AUV using a DVL, an IMU, and an USBL as sensors. The last referenced work discussed the use of a finite state machine to ensure the complete inspection of the net and proved that even when the data from the DVL were not available for different parts of the trajectory, the AUV could complete the trajectory. Furthermore, Amundsen et al. [
17] conducted research where an autonomous ROV inspection of aquaculture net cages was conducted using a DVL. In that application, the DVL was mounted on the bow of the ROV and the desired distance of the net was fixed to 2 m. The real experiments proved that the system followed a circular trajectory inside the net cage with some unknown ocean currents. However, the authors reported that velocity gains had to be low because of the noise measurements of the DVL.
Computer vision can be an alternative for positioning in confined environments. Evidence of this can be found in [
18], where an intelligent navigation and control of an AUV for automated inspection of aquaculture net pen cages was developed using an optical camera. In that case, some markers were attached to the net. By using computer vision, an IMU, and a depth sensor, the AUV could develop an inspection trajectory. The pose of the AUV in a net fish cage was also estimated with a monocular camera in the work of C. Schellewald [
19], where the input image was processed, and a squared region of interest was analyzed to detect regular peaks in the Fourier transform indicating the presence of a fish net. Once the fish net was detected and knowing the camera parameters, the position and orientation from the ROV to the net could be computed. The algorithm was tested with real fish-farm images containing salmons, and it could detect the region of interest and estimate the pose.
A. Duda et al. [
20] proposed an algorithm for pose estimation, “the X junction”, which could detect the fish-net knots and their topology from camera images. After detecting the knots, the pose of the camera was estimated. The authors estimated the distance between the fish net, roll, pitch, and yaw with high accuracy at low distances (15 cm to 65 cm). A visual servoing scheme for autonomous aquaculture net pens’ inspection using a ROV was developed by Akram et al. [
21]. The referenced work proposed the use of ropes attached to the net to estimate the relative position of the ROV between the net using computer vision. In a related context, ref. [
22] introduced another monocular visual odometry work where the authors developed a real-time monocular visual odometry algorithm for turbid and dynamic underwater environments; the work compared the drift against the level of noise, depending on the turbidity of the water.
Most of these works that just depend on a monocular camera are sensible to low-light or high-turbidity water environments. To overcome that, Hoosang L. et al. [
23] proposed an autonomous underwater vehicle control for fish-net inspection in turbid water environments. The authors trained a convolutional neural network (CNN) to predict the ideal net distance and the yaw set point to control the AUV.
Research involving net hole detection was conducted by Lin et al. [
24], where an omnidirectional surface vehicle (OSV) for fish-net inspection was designed. The work also incorporated AI (artificial intelligence) planning methods for inspecting the whole net surface or focusing on an interesting area. The omnidirectional surface vehicle presented in [
24] has an onboard camera with adjustable depth. That camera is used to take images from the net. In fish farms, nets can be larger than 20 m, and having only a surface vehicle limits the area of inspection; this means that having an underwater vehicle plays a significant role when inspecting the total net area.
Underwater imaging involves big challenges principally due to the environmental properties. Light scattering, light absorption, or water turbidity are some examples of phenomena that affect underwater images. To overcome this problem, traditional computer vision and deep learning techniques are applied. In [
25], an underwater object detection review is presented. The most used architectures and detection algorithms are exposed in that work, such as CNNs, recurrent convolutional neural networks (RCNNs) and the You Look Only Once (YOLO) algorithm versions.
There are few works that use traditional computer vision methods like the Otsu threshold, the Hough transform, and convex hull algorithms that are used in [
26] to reconstruct and detect holes in fish nets from a fish farm cage. The authors report a 79% of accuracy when examining net damages.
Deep learning techniques are the most used algorithms when detecting underwater objects, and these algorithms do not work the same in air as in underwater environments due to the environmental properties mentioned before. Works like [
27] proposed a method for improving object detection in underwater environments based on an improved EfficientDet. That method modified the structure of the neural network and the results showed that the mean average precision (mAP) reached 92.82%. Also, the processing speed showed an increase reaching 37.5 FPS. This was important due to the onboard robot hardware characteristics.
As mentioned before, YOLO is one of the most used detection algorithms. In [
28], the fifth version of that algorithm was tested and compared versus other algorithms such as faster RCNN or a fully convolutional one-stage (FCOS) method, and the results showed that the small version of YOLOv5 reached an mAP of 62.7% and around 50 FPS showing the best-combined results of the YOLOv5 versions.