An Automated Fish-Feeding System Based on CNN and GRU Neural Networks

Abstract

AI plays a pivotal role in predicting plant growth in agricultural contexts and in creating optimized environments for cultivation. However, unlike agriculture, the application of AI in aquaculture is predominantly focused on diagnosing animal conditions and monitoring them for users. This paper introduces an Automated Fish-feeding System (AFS) based on Convolutional Neural Networks (CNNs) and Gated Recurrent Units (GRUs), aiming to establish an automated system akin to smart farming in the aquaculture sector. The AFS operates by precisely calculating feed rations through two main modules. The Fish Growth Measurement Module (FGMM) utilizes fish data to assess the current growth status of the fish and transmits this information to the Feed Ration Prediction Module (FRPM). The FRPM integrates sensor data from the fish farm, fish growth data, and current feed ration status as time-series data, calculating the increase or decrease rate of ration based on the present fish conditions. This paper automates feed distribution within fish farms through these two modules and verifies the efficiency of automated feed distribution. Simulation results indicate that the FGMM neural network model effectively identifies fish body length with a minor deviation of less than 0.1%, while the FRPM neural network model demonstrates proficiency in predicting ration using a GRU cell with a structured layout of 64 × 48.

1. Introduction

The concept of Artificial Intelligence (AI), initially proposed in the 1950s, has seen explosive growth since the onset of the Fourth Industrial Revolution. Just a few years ago, AI struggled to effectively perform tasks such as static image recognition or natural language processing. However, it is now utilized in autonomous driving vehicles based on static images (like road lines) and LiDAR sensors, as well as in agriculture through the use of drones and soil sensors. AI has progressed to the point where it can diagnose the health of animals in livestock and aquaculture industries, providing monitoring capabilities for users. Jahmy Hindman, the Chief Technology Officer (CTO) of John Deere, an American agricultural machinery company, emphasized that AI should be prioritized in addressing food-related challenges such as agriculture, livestock, and aquaculture [1].

Precision aquaculture tries to achieve its goal, i.e., ensuring profitability, sustainability, and protection of the environment, by incorporating different innovative technologies such as Artificial Intelligence (AI) and the Internet of Things (IoT) in aquaculture [2]. Furthermore, the majority of fish farms employing such programs utilize computer vision (51%) and artificial intelligence (70%). Despite the increasing adoption of programs in fish farming, the level of AI implementation in this sector remains relatively low compared to applications like autonomous vehicles and plant cultivation. Unlike other domains, most current AI technologies related to fish farming primarily focus on monitoring tasks. Even the most recent studies, including those conducted in 2022, utilize AI for tasks such as estimating fish populations or assessing water quality to determine feeding schedules [3,4]. While these approaches effectively leverage AI in fish farming, they are unable to fully replace human intervention. For instance, when it comes to food rationing, the quantity of food to be dispensed varies based on the fish’s growth rate, and environmental factors within the aquarium significantly influence food distribution. Therefore, precise analysis is essential for adjusting ration amounts accurately.

Therefore, this paper proposes an Automation Fish-feeding System (AFS) based on Convolutional Neural Networks (CNNs) and Gated Recurrent Units (GRUs) Neural Networks to automate feed distribution in fish farms. The AFS gathers environmental data regarding the fish’s growth from sensors within the tank, employs artificial intelligence to assess the fish’s growth status, and calculates the appropriate food ration based on the overall conditions of the fish farm. This study conducts training using publicly available data to validate the effectiveness of the AFS and utilizes test datasets to confirm its efficacy in managing fish growth and rationing. The structure of this paper is as follows:

• Section 2 discusses the significance of rations in fish farms and reviews existing research on AI applications in this domain.
• Section 3 elucidates the architecture and logical design of the AFS proposed in this paper.
• Section 4 employs publicly available data to train the neural networks within each module of the AFS and utilizes test data to verify its capability to accurately assess fish growth rates and adjust food distribution accordingly.
• Section 5 provides a brief summary of the proposed AFS and outlines future directions based on simulation results.

2. Related Works

In 2020, aquaculture production contributed to 49% of the total fisheries and aquaculture production, indicating its significant role in the industry. As the aquaculture sector continues to expand, its importance is expected to grow steadily [5]. Consequently, the introduction of smart farming practices becomes essential to enhance convenience and efficiency within fish farming operations [6].

To further the development of fish farming, studies have been conducted to ascertain the direction, species, and growth stage of fish. In 2006, research utilized a moment-invariant method to determine fish direction, identify species (flounder or round fish), and measure fish length [7]. Subsequently, with the advancement of deep learning techniques, investigations have explored methods to segment and measure fish body characteristics by integrating technologies such as Mask R-CNN [8]. More recently, a refined approach for accurately measuring fish characteristics has been proposed. This method incorporates an open fish dataset along with data augmentation techniques and an enhanced Mask R-CNN network. Additionally, studies have focused on evaluating fish feeding intensity by employing Convolutional Neural Networks (CNNs) to classify fish appetite into four scales: none, weak, medium, and strong. These advancements signify ongoing efforts to improve fish farming practices through innovative technological applications and methodologies.

A study to detect fish in an aquarium showed that fish detection and species classification were possible with YOLOv3 and CNN-SENet [9]. It can be difficult to distinguish between fish and fish species on the seafloor due to various environmental problems such as brightness, light refraction, and noise. YOLO-Fish-1 improves the performance of YOLOv3 by correcting the upsampling step size problem to reduce false detection of small fish. YOLO-Fish-2 further improved the model by adding spatial pyramid pooling to the first model, adding the ability to detect fish shapes in a dynamic environment [10]. Images acquired with the Deep Vision imaging system of commercial fishing gear are used to localize and segment each individual fish in the image using the Mask R-CNN architecture to estimate the boundaries of the fish. This allows complex images containing overlapping fish to be successfully processed [11]. YOLOv8 is a version released in 2023 and was built as an integrated framework to train instance segmentation and image classification models [12].

Fish are detected by acquiring the area of the fish in the optical image and distinguishing it from the background and other organisms. Afterwards, the length of the fish is calculated according to the position of the fish [13]. A computer vision system (CVS) was designed for fast, accurate, and indirect measurement of characteristics such as body weight (BW) and weight (CW) of fish, and a linear model for prediction was developed. Predictions were made using a dataset created by photographing 1653 fish, achieving R2 of 0.96 and 0.95 for fish BW and CW, respectively [14]. We built a system to predict fish length in the 20–40 cm range using Mask R-CNN. The root mean square deviation for the average length was 1.9 cm, and the maximum deviation between the estimated and measured average body length was successfully predicted to be 4.0 cm [15]. We developed a deep learning/statistical approach to obtain data on fish length in fisheries. We present an operational system using a deep convolutional network (Mask R-CNN) combined with a statistical model to automatically estimate the number and average fork length of dolphin fish (Coryphaena hippurus) caught in Mediterranean fisheries. The proposed system was operated during the Coryphaena hippurus fishing season in Mallorca and showed excellent accuracy and precision [16]. Two approaches based on image processing and machine learning techniques are combined in an environment integrated with new techniques (e.g., edge or corner detection and pattern stretching) to estimate the relative length, height, and area occupied by the fish in the image [17].

In studies such as object recognition, images converted to grayscale showed higher accuracy than those using RGB images [18].

Various studies have been conducted on CNN, a deep learning structure suitable for image processing, such as fish detection and growth measurement. In 2012, one of the largest CNNs was trained using five conv layers and three fc layers, achieving unprecedented results in ILSVRC-2010 and ILSVRC-2012, reducing model performance and learning time [19]. We succeeded in learning a model with a depth of 16 to 19 layers using only 3 × 3 filters in all layers [20]. Afterwards, it was designed to increase the depth and width of the network while maintaining the computing budget in 2015 [21]. Next, by using Batch Normalization, we can see the effect of fast learning and suppressing overfitting without relying on the initial value, and solve the problem of vanishing gradients [22]. To solve the feature reuse problem that occurs in structures using existing deep layers, Dense Net prevents information loss by connecting the feature map of the first layer to the feature map of the last layer [23]. Inception-v4 was designed to make the Inception neural network wider and deeper more effectively, resulting in a faster learning speed [24]. As CNNs developed, very deep CNNs became mainstream, but there were limitations due to model size and computational efficiency. To solve this problem, a structure was used to reduce the amount of computation by using several small filters [25]. When increasing model accuracy, a compound scaling method was proposed that can efficiently adjust the depth, width, and input image size of the model, which are commonly adjusted [26].

In fish farming, the food level of fish has the greatest impact on production efficiency and reproduction costs [27]. Insufficient feed ration reduces the production efficiency of fish, and excessive feed ration increases feed costs and causes pollution of the aquarium environment [28]. However, too many factors influence food ration to optimize it [29]. Food ration is most influenced by the size, quantity, and water quality of the fish. Fish that do not consume food properly may develop inappropriate competition for food due to stress. This directly affects food consumption [30]. Therefore, optimization of feed distribution has great economic significance in aquaculture. Additionally, in aquaculture, it is important to know the moment to stop feeding to maximize efficiency. While the aquaculture industry has grown over the past 30 years, the fish food market has also grown in importance. Therefore, reducing feed costs has become an important issue in aquaculture [31]. Food ration prediction optimizes food distribution and further reduces feeding costs. Ultimately, it becomes the basis for food distribution automation, and food distribution automation is one of the important technologies in smart aquaculture or aquaponics systems.

Hu et al. present an automatic food distribution system that regulates fish food distribution based on deep learning computer vision technology. It recognizes the size of the waves generated when the fish eats food and determines whether to stop or continue feeding. As a result, an accuracy of up to 93.2% was achieved [32]. The application of machine vision technology in aquaculture is an important recent research topic. In particular, the application of machine vision in the field of feed distribution contributes to the advancement of increasing production efficiency, reducing feed costs, and automating feed distribution [33]. Predicting food rations using traditional mapping methods is a very difficult problem. Traditional mapping methods artificially link food rations to environmental factors. Therefore, there is a strong tendency to rely on subjective experience, and there is a nonlinear relationship between food ration and environmental factors. This is a very complex relationship and resolving it is very difficult [34]. Therefore, in this paper, we design a deep learning model that imitates the food distribution pattern of aquaculture experts, rather than a traditional mapping method. By constructing the model in a way that mimics the expert’s food distribution pattern, low-cost learning is possible using only fish growth information, environmental information, and the expert’s food distribution record.

GRU is a unit optimized for time-series data analysis while solving the long-term dependency problem, which is a drawback of RNN (Recurrent Neural Network) [35]. Mao et al. built a prediction model for pregnant sow’s feed intake using a GRU-based deep learning model. Performance was evaluated by comparing LSTM, RNN, DNN, and GRU. As a result of the experiment, the GRU model had a faster training speed and higher prediction accuracy than other models [36]. Yang et al. compared LSTM and GRU models, which are models that complement the long-term dependency problem of RNN. GRU has a structure with one fewer gate than LSTM. This has the advantage of reducing the computational process and increasing the efficiency of training time [37]. In this paper, a GRU-based deep learning model is used to predict food distribution. This enables learning of time-series data and training of deep learning models with fewer operations than LSTM.

3. The Design of an AFS

This section delineates the design of the AFS. Section 3.1 provides an overview of the AFS, encompassing its logical structure, external data sources, and the conclusive outcomes obtained from the test bed. Section 3.2 delves into the operation of the Fish Growth Measurement Module (FGMM), elucidating how it refines input data, the construction and training of a CNN-based growth detection neural network model, as well as the outcomes and utilization of the neural network model. Section 3.3 details the functionality of the Feed Ration Prediction Module (FRPM), outlining the generation of time-series data, the construction and training of a GRU-based ration prediction neural network model, and the results achieved by the AFS along with their practical implications.

3.1. Overview

In conventional feed distribution systems, determining the type and quantity of feed dispensed relies heavily on human intervention, considering factors such as the fish’s growth environment, its current growth status, and species-specific requirements. Consequently, the Automation Fish-feeding System (AFS) proposed in this paper aims to enhance the efficiency of fish farming by automating the food distribution process, traditionally performed manually in fish farms. The AFS leverages sensor data from the tank to gather information about the fish’s growth environment, utilizes artificial intelligence to assess the fish’s growth status, and computes the appropriate food ration based on the overall conditions of the fish farm. This paper presents a detailed examination of the AFS, emphasizing its ability to accurately determine the optimal ration amount through the utilization of two distinct modules. Figure 1 represents the overall structure of AFS.

Firstly, the Fish Growth Measurement Module (FGMM) captures images of fish using an underwater camera and processes these images to generate a single fish image through cropping, resizing, and perspective correction techniques. The module then creates training data by associating the modified image with the corresponding fish growth level at the time of capture, forming a dataset for training a CNN-based neural network model. Once trained, this neural network model utilizes fish data to determine the degree of fish growth, transmitting this information to the feed ration prediction module.

Secondly, the Feed Ration Prediction Module (FRPM) utilizes sensor data from the fish farm, along with fish growth data and the current feed ration status represented as time-series data, to calculate the rate of increase or decrease in ration based on the prevailing fish conditions. In this paper, we introduce PredFeedNet (PFN), which predicts feed ration by learning from data. To achieve this, we construct training data using feed rations adjusted by fish farmers according to the specific circumstances of the fish farm and apply it to FRPM. Through the integration of these two modules, this paper automates the feed distribution process in fish farms and validates the efficacy of automated feed distribution.

3.2. Fish Growth Measurement Module

The Fish Growth Measurement Module (FGMM) undertakes two primary functions. Firstly, it performs image preprocessing, which involves retrieving images from JSON-based public datasets collected externally. It corrects the perspective of the acquired images, crops and resizes them, and refines the images to ensure they are suitable inputs for the neural network. Secondly, FGMM measures the growth stage of fish by employing the Growth-rate Prediction Neural Network (GPNN), a CNN-based neural network model. Upon completion of the learning process and confirmation of GPNN’s ability to accurately measure fish growth, the measurement data is transmitted to the Feed Ration Prediction Module (FRPM). Figure 2 illustrates the data flow within FGMM, encompassing the training and utilization of the artificial intelligence model.

3.2.1. Input Data Cleaning Process

In this paper, we obtain fish growth data and images in JSON format from AI Hub to facilitate the learning of the AI model [38]. FGMM exclusively collects image data from the JSON format data, standardizing it into a uniform format suitable for input into the AI model. Additionally, FGMM assigns labels to each image to construct a training dataset. The process through which FGMM generates a training dataset is outlined as follows:

Firstly, FGMM processes images to ensure that only one fish is recognized within an existing image containing multiple fish. It achieves this by isolating and cropping the image to include only one fish object. Subsequently, the image is resized to dimensions of 224 × 224 to maintain uniform input values. FGMM enhances recognition performance by converting the resized image to grayscale. Lastly, a training dataset is generated by correlating the body length of the fish at the time of photography with the resized image. The body length data is sampled between 0 and 1 based on the degree of growth, and labels are assigned to the image accordingly.

$$BL{\left(z_k\right)}_{nom}=\frac{BL\left(z_k\right)-\min \left(BL\right(z_i\left)\right)}{\max \left(BL\left(z_i\right)\right)-\min \left(BL\right(z_i\left)\right)},$$

(1)

Equation (1) represents the sampling method for generating body length data.

Table 1. The meaning of the symbols used in Equation (1).

Valuable	Description
$BL{\left(z_k\right)}_{nom}$	Normalized fish ($z_k$) length
$BL\left(z_k\right)$	Length of the currently measured fish ($z_k$)
$\min \left(BL\right(z_i\left)\right)$	Minimum length of the entire fish
$\max \left(BL\left(z_i\right)\right)$	Maximum length of the entire fish

provides a detailed explanation of the variables utilized in Equation (1). Essentially, this equation samples the current length of the fish between 0 and 1 based on the length of the fish recorded in the training data.

FGMM not only generates training data, but also enhances input images captured in a test environment. Unlike the training dataset, where the fish’s body length is already documented, the images captured in the test environment depict fish whose growth rates need to be assessed. Depending on the positioning of the photographed fish, there is a possibility that the fish’s size may be inaccurately measured relative to its actual size. To address this issue, this paper introduces a measuring plate capable of determining the distance between the camera lens and the fish, which is installed on the test bed to mitigate measurement errors stemming from the camera-to-fish distance. Images captured using this method serve as input datasets by applying the same image processing methodology employed during the creation of the training dataset.

3.2.2. The Training Process of the Growth-Rate Prediction Neural Network

Following this, FGMM proceeds to train Growth-rate Prediction Neural Network (GPNN) using the refined training data. GPNN is a neural network designed to predict the growth rate of the fish currently being photographed. GPNN is a CNN-based neural network model comprising five convolution layers and three fully connected layers. It is a neural network that alters the configuration of fully connected nodes conducted after the convolution operation using images. Figure 3 illustrates the simplified structure of GPNN.

The GPNN conducts a convolution operation using a 3 × 3 filter with a stride of 1 and sets Padding to 1, padding the outskirts of the image with 0-valued data. Additionally, GPNN employs anchor boxes of various sizes in the multi-scale feature map output to effectively detect objects of different sizes. Figure 3 depicts the addition of an independent node, such as bias, to the fully connected layer. This node represents the fish species in the current image. In this study, only two fish species (pollock and salmon) were used, with the corresponding node value being 0 for pollock and 1 for salmon.

Furthermore, GPNN utilizes dropout, a technique that randomly removes neurons with a probability between 0 and 1 from the interconnected network, to alleviate computational burden and prevent overfitting. Dropout is applied to only 20% of the three fully connected layers.

The composition of the neural network model for GPNN is as follows: The input data image size is set to 224 × 224, and the first convolution layer operates on it. Subsequently, max pooling is performed in the first pooling layer to reduce the feature map size to 112 × 112. The second convolution layer conducts a convolution operation to generate a 112 × 112 feature map, followed by max pooling in the second pooling layer to further reduce the feature map size to 56 × 56. This pattern continues through additional convolutional and pooling layers, gradually reducing the feature map size to 7 × 7 in the fifth pooling layer.

The data from the 7 × 7 feature map is then inputted as a row into the FC1 layer, which consists of 1024 nodes connected using the Fully Connect method. Additionally, a node capable of identifying fish species is added to the FC1 layer. The additional nodes are indicated in Figure 3 by red lines for distinction. The FC2 layer forms a fully connected layer with 512 nodes. The activation function of the FC3 layer is set to sigmoid to ensure the output value falls within the range of 0 and 1.

The output nodes of GPNN, ranging from 0 to 1, predict the growth level of the fish. This value represents the current growth status of the fish based on its body length, allowing for automatic adjustment of feed distribution in accordance with its condition. For example, if a fish that needs to grow to 60 cm is photographed during rearing and GPNN predicts it to be 0.654, this means that the fish is currently about 39.2 cm and the food distribution can be adjusted depending on its condition. In essence, GPNN can assess the growth status of fish by autonomously determining the growth status based on body length, a method commonly employed in existing fish farms.

3.3. Feed Ration Prediction Module

In a fish farm, the feed ration is determined by both the tank environment and fish growth. The FRPM generates training data for a neural network by configuring expert feed rations based on the aquarium environment and fish growth data collected over time. This generated training data is then utilized to train a Pred-feed Neural Network (PNN), capable of predicting the appropriate food ration. The aquarium environmental data includes parameters such as water temperature (°C), dissolved oxygen levels (mg/L), ammonium concentration (mg/L), nitrite concentration (mg/L), nitrate concentration (mg/L), and suspended solids (mg/L). On the other hand, growth data comprises the average length (m) and average weight (kg) of the fish. As each parameter is recorded over time, it can be structured as time-series data. Figure 4 represents the overall data flow of FRPM.

Given that each parameter of the aquarium environment and fish growth data holds varying degrees of influence on the food ration, the FRPM normalizes the data and assigns weights according to the significance of each parameter. The PNN is trained using expert feed rations as label data to learn the feeding patterns of experts. Since fish growth data evolves meaningfully over time, the PNN is designed as a Gated Recurrent Unit (GRU)-based model. GRU is a type of Recurrent Neural Network (RNN) model in deep learning, where the previous state influences the subsequent state through a hidden state during learning, making it suitable for learning time-series data. The performance of the PNN is assessed based on its accuracy in predicting test data after being trained on the training dataset.

3.3.1. Configuring Input Data

The input data for the Pred-feed Neural Network (PNN) comprises three main categories: time, aquarium environment, and fish growth information. Here is a breakdown of the data included in the input:

• Time:
  Hours: Date (year/month/day) and time (hour: minute)
  Date (year/month/day) and time (hour: minute).
  Aquarium environment:
  Water temperature (°C), Dissolved oxygen amount (mg/L),
  Ammonium concentration (mg/L), Nitrite concentration (mg/L),
  Nitrate concentration (mg/L), Suspended solids (mg/L).
• Fish growth information:
  Average length (m), average weight (kg).

The PNN utilizes nine input nodes, and since it is a GRU-based neural network, the dataset to be inputted at the input layer consists of a time-series dataset. As the PNN is a supervised learning neural network, a table of target answer data is required. Therefore, the neural network is trained using the expert’s feeding log as the target answer table.

3.3.2. Construction and Learning of the PNN

The Input Layer of the Pred-feed Neural Network (PNN) accepts the real values of the input dataset outlined in Section 3.3.1 and forwards them to the nodes. Subsequently, the input data undergoes batch normalization at the BN Layer before being transmitted to the Hidden Layer. Batch normalization plays a pivotal role in mitigating the challenges associated with weight initialization, and augments the learning speed of the network. Equations (2) and (3) elucidate the procedure through which the BN Layer computes the average and variance of the input data. These calculations are essential for the normalization process, ensuring the stability and effectiveness of the neural network during training.

$${\mu }_{Batch}=\frac{1}{m}\sum _{i=1}^{m}I{N}_i,$$

(2)

$${\sigma }_{\left(Batch\right)}^2=\frac{1}{m}\sum _{i=1}^m{(I{N}_i-{\mu }_{Batch})}^2,$$

(3)

BN Layer normalizes data through standard distribution of input data. In Equation (2), $I{N}_i$ is the real number data currently stored in the input node, $m$ means the batch size, and ${\mu }_{Batch}$ means the average of $I{N}_i$. In Equation (3), ${\sigma }_{\left(Batch\right)}^2$ means the variance of the input data calculated through ${\mu }_{Batch}$. Equation (4) shows how to calculate normalized data $x_i$ through the results of Equations (2) and (3).

$$x_i=\frac{I{N}_i-{\mu }_{Batch}}{\sqrt{{\sigma }_{\left(Batch\right)}^2+ϵ}},$$

(4)

In Equation (4), $ϵ$ is a small constant that prevents division by 0 during the normalization process. Lastly, the BN Layer determines the value to be input to the BN Layer using the learning variables $\gamma$ and $\beta$ that are used only for that layer. Equation (5) shows how to calculate the value input to the BN node.

$$B{N}_i=\gamma x_i+\beta ,$$

(5)

In this paper, $\gamma$ uses the variance of $I{N}_i$, and $\beta$ uses the mean of $I{N}_i$. $\gamma$ and $\beta$ are learned using back propagation, and this paper sets the batch size to 12. Data entered the BN Layer is transmitted to the Hidden Layer. The Hidden Layer is composed of two layers of GRU cells. GRU uses Sigmoid and tanh because the problem of using ReLU (Rectified Linear Unit) as an activation function causes the value to become too large. The GRU-based deep learning model was designed with a two-layer shallow structure because when the Hidden Layer is designed deeply, the performance difference is not large, and the possibility of overfitting increases. The configuration of the GRU cell used in PNN is shown in Figure 5.

GRU cells use a reset gate, update gate, and input gate. The reset gate calculates how much data from the previous point is to be removed, the update gate calculates how much data from the previous point to retain, and the input gate calculates the data to be delivered to the next node, including the information of the reset gate and update gate. Equation (6) represents the result of the reset gate, Equation (7) represents the result of the update gate, and Equations (8) and (9) represent the process of calculating the result of the input gate.

$$r_t=\sigma \left(W_{BNr}B{N}_t+W_{hr}{h}_{t-1}+b_r\right),$$

(6)

$$z_t=\sigma \left(W_{BNz}B{N}_t+W_{hz}{h}_{t-1}+b_z\right),$$

(7)

$$g_t=\tanh (W_{BNg}B{N}_t+W_{hg}(r_t \odot h_{t-1})+b_g),$$

(8)

$$h_t=\left(1-z_t\right) \odot g_t+z_t \odot h_{t-1},$$

(9)

Table 2. The meaning of the symbols used in Equations (6)–(9).

Valuable	Description
$r_t$	The output of Reset gate. A value between 0 and 1
$z_t$	The output of update gate. A value between 0 and 1
$g_t$	Data to use currently
$h_t$	Final data to be passed to the next node
$W$	Weight
$h_{t-1}$	Node value of hidden layer at previous time
$b$	Bias

shows the meaning of the symbols used in Equations (6)–(9), and the $\odot$ operation used in Equations (8) and (9) refers to Hadamard product, which multiplies each component of two matrices, not general matrix multiplication.

The computations are carried out within the two hidden layers through the GRU cell, and the Pred-feed Neural Network (PNN) employs the hyperbolic tangent (tanh) function between the hidden layer and the output layer. This function scales the output values to fall within the range of −1 to 1 at the output node. There exists a single output node in the output layer, which directly outputs the result of the tanh function without applying any threshold. The output value obtained from the output node represents the increase or decrease rate of the amount of feed currently being distributed. To ascertain this rate, the neural network calculates the difference between the current ration and compares it with the feeding log used as learning data.

For the learning process, the PNN utilizes Mean Squared Error (MSE) as the loss function. This function quantifies the disparity between the predicted output and the actual output, thereby guiding the network’s learning process towards minimizing this discrepancy.

4. System Experiment

In this paper, a simulation is carried out to demonstrate the effectiveness and suitability of the Automation Fish-feeding System (AFS) for automation. For this simulation, salmon and pollock aquaculture data from the “Gangwon-do annual fish (salmon, pollock) intelligent farming comprehensive data” provided by AI Hub was utilized [38]. A total of 2500 images were included in the dataset, with 1750 images allocated for training data and 750 images for validation data. These images were used to train the Growth-rate Prediction Neural Network (GPNN) and verify its ability to accurately measure the body length of the fish. Subsequently, the Pred-feed Neural Network (PNN) was trained using sensor measurement data (dissolved oxygen, water temperature), water quality analysis data (ammonium, nitrite, nitrate, suspended solids), salmon and pollock growth stage data (average weight, average length), and feed management data (feed ration) recorded from 21 July 2021 to 21 September 2021. The trained PNN was then tested to assess its accuracy in predicting changes in food ration. Furthermore, this paper explores the optimization of the GRU Shape for the PNN by experimenting with different GRU Shape configurations.

The neural network training was conducted using Google Colab, with the model constructed using Keras based on the Python3 language. The hardware utilized was a T4 GPU with a VRAM capacity of 12 GB. Additionally, 51 GB of RAM and approximately 27 GB of disk capacity were utilized in the training process.

4.1. The Learning Results of the GPNN

This paper aimed to verify the effectiveness of the Growth-rate Prediction Neural Network (GPNN) by utilizing Mean Squared Error (MSE) and Mean Absolute Error (MAE) as loss functions during the learning process. Figure 6 illustrates the outcomes of GPNN learning based on the analysis of 2500 image data points.

According to the findings depicted in Figure 6, it is observed that as the learning process advances, both Mean Absolute Error (MAE) and Mean Squared Error (MSE) decrease gradually, indicating effective learning of the Growth-rate Prediction Neural Network (GPNN). In the case of learning using MAE, the error rate started at 0.1765% at Epoch 1 and steadily decreased to 0.0011%. Similarly, when validating the data, the MAE commenced at 0.1342% and decreased to 0.0398%. It is noteworthy that at approximately 10 epochs, the training loss and validation loss graphs intersect, indicating the onset of overfitting. Subsequently, the difference between the two losses gradually widens in subsequent epochs. Moreover, the error rate of the validation data reaches a minimum of 0.0398% and does not decrease any further. This implies that GPNN exhibits the capability to accurately identify the body length of fish with a minimal discrepancy of less than 0.1%.

4.2. The Learning Results of the PNN

To train the PNN, this paper utilizes a dataset comprising the following variables: dissolved oxygen amount, sensor measurement time, water temperature, ammonium concentration, nitrite concentration, nitrate concentration, suspended matter, average length of fish, and average weight of fish. This dataset serves to determine the appropriate food ration corresponding to the environmental conditions. The dataset is labeled to indicate whether the food ration was adjusted and is employed as the training dataset. Specifically, 60% of the dataset is allocated for training, 20% for validation, and the remaining 20% for testing purposes. This partitioning ensures that the model is adequately trained, validated, and tested on distinct subsets of data, thus facilitating a comprehensive evaluation of its performance.

The simulation involved comparing the Mean Absolute Error (MAE) throughout the learning and testing phases of three distinct models with varying GRU connection structures. A larger MAE implies greater deviation between the model’s predictions and the actual values. The three models were configured by adjusting the GRU shape to 16 × 10, 32 × 16, and 64 × 48, respectively. Figure 7 depicts the results of the training and validation process for each GRU cell. The simulation was executed with identical specifications for the number of hidden layers, input layer, and output layer across all models. Key parameters of the simulation include:

• Batch size: 12
• Epochs: 100
• Optimization algorithm: Root Mean Square Propagation (RMSProp)
• Loss function: Mean Squared Error (MSE)
• The neural network architecture follows the structure: Input Layer -> Batch Normalization (BN) Layer -> Hidden Layer (GRU) -> Output Layer.

Table 3. The structure of the GRU cells for each model.

	Model A	Model B	Model C
GRU Shape	16 × 10	32 × 16	64 × 48
Test MAE	5.14	4.53	2.96

presents the MAE values corresponding to different GRU shapes, offering insights into the performance of each model.

Model A was trained in a 16 × 10 structure. Model A recorded a satisfactory MAE during the learning process, with a Training MAE of 1.5 and a Validation MAE of approximately 3.4. Model B, like Model A, showed very good MAE results during the learning process, with a Training MAE of 1.09 and a Validation MAE of 1.46. However, looking at Table 3, during the testing process, both models show MAEs that are about four times larger than during the learning process, and the actual predictions are performed incorrectly. Model C is a model with twice as many GRU Units connected to the Hidden Layer compared to Model B. As a result, good performance was confirmed during the learning process with Training MAE of 0.85 and Validation MAE of 1.46 and showed a small error of 2.96 in the actual test environment. Therefore, among the three models, the 64 × 48 GRU neural network connection structure shows the highest accuracy when predicting increases or decreases in food ration. Therefore, it is appropriate for the PNN to use a GRU cell with a 64 × 48 structure.

4.3. The Efficiency Verification of GRU Cell

Figure 8 shows the learning outcomes of PNN when utilizing GRU cells and LSTM cells. The results of the simulation indicated that for a cell shape of 16 × 10, LSTM achieved a Test Mean Absolute Error (MAE) of 6.17 g, outperforming GRU, which yielded a Test MAE of 6.81 g. However, as the cell size increased, GRU demonstrated superior performance compared to LSTM. Specifically, when the cell size was set to 32 × 16, the Test MAE for GRU was 3.47 g, whereas LSTM exhibited a Test MAE of 4.53 g. Similarly, for a cell size of 64 × 48, GRU attained a Test MAE of 3.35 g, while LSTM yielded a Test MAE of 4.55 g. In summary, the results indicate that as the cell size increases, the accuracy of the PNN utilizing GRU cells improves, and it requires fewer epochs for learning compared to LSTM. Consequently, learning the PNN using GRU cells is deemed the most effective approach based on the observed outcomes.

5. Discussion

Sustainable aquaculture development remains critical to supply the growing demand for aquatic foods [5]. However, currently, AI in livestock and aquaculture only supports biological or environmental monitoring through images. Aquaculture requires humans to measure the condition of the fish and manually adjust the growing environment or food ration accordingly. Therefore, this paper proposed an Automation Fish-feeding System (AFS) Based on CNN and GRU Neural Networks that can accurately measure the condition of fish in a fish farm, and automatically distribute food according to the current tank environment and the growth status of the fish.

AFS consists of two modules. The first module, Fish Growth Measurement Module (FGMM), has two functions. The first function collects images from JSON-based public datasets collected externally, corrects the perspective of the collected images, crops them, and resizes them to refine the images so that the correct images can be input to the neural network. The second process measures the growth stage of the fish by learning the Growth-rate Prediction Neural Network (GPNN), a CNN-based neural network model. The second module, Feed Ration Prediction Module (FRPM), generates learning data for a neural network by organizing expert feed rations for fish growth data in time-series units. Next, FRPM predicts the food ration of fish by learning a Pred-feed Neural Network (PNN) that can predict food ration through the generated learning data.

Three simulations were conducted to verify the efficiency of the two modules. First, we wanted to check whether the GPNN was properly trained. As a result of the simulation, GPNN was able to well identify the body length of the fish with a slight difference of less than 0.1. Second, we wanted to check whether the PNN was properly trained and find the size of the GRU cell most suitable for the PNN. As a result of the simulation, when using cells of sizes 16 × 10, 32 × 16, and 64 × 48, among the three models, the 64 × 48 GRU neural network connection structure showed the highest accuracy when predicting the increase or decrease in food ration. Lastly, to prove the efficiency of the GRU cell, the PNN’s GRU cell was replaced with an LSTM cell and trained. As the cell size grew, the accuracy of the PNN using GRU cells increased, and the epochs required for learning also decreased.

However, AFS has not been tested in an actual water tank, and went through the process of converting JSON data to use data from an existing AI Hub. We are preparing a testbed to apply AFS to an actual aquarium. We will construct a testbed through the preparatory steps outlined in

Table 4. Items included in the testbed to be built.

Item	Description
Small fish tank	Individual observation and control of salmon egg hatching and trout growth will be achieved by introducing them into test tubes.
Large fish tank for fish farming	Sensor data from the tanks will be collected to monitor the overall aquaculture conditions, which will be integrated with the hydroponic cultivation environment.
Dead fish freezer	Management and tracking of mortalities.
Small hydroponic model	Measurement of plant growth status and conditions.

to measure the overall environment of the aquaculture farm. Furthermore, we plan to establish a basic hydroponic cultivation environment to diagnose the soil condition and plant status. In the future, research is needed to determine whether AFS can be applied to testbeds to replace tasks that previously required humans.