Accurate Wound and Lice Detection in Atlantic Salmon Fish Using a Convolutional Neural Network

Abstract

The population living in the coastal region relies heavily on fish as a food source due to their vast availability and low cost. This need has given rise to fish farming. Fish farmers and the fishing industry face serious challenges such as lice in the aquaculture ecosystem, wounds due to injuries, early fish maturity, etc. causing millions of fish deaths in the fish aquaculture ecosystem. Several measures, such as cleaner fish and anti-parasite drugs, are utilized to reduce sea lice, but getting rid of them entirely is challenging. This study proposed an image-based machine-learning technique to detect wounds and the presence of lice in the live salmon fish farm ecosystem. A new equally distributed dataset contains fish affected by lice and wounds and healthy fish collected from the fish tanks installed at the Institute of Marine Research, Bergen, Norway. A convolutional neural network is proposed for fish lice and wound detection consisting of 15 convolutional and 5 dense layers. The proposed methodology has a test accuracy of 96.7% compared with established VGG-19 and VGG-16 models, with accuracies of 91.2% and 92.8%, respectively. The model has a low false and true positive rate of 0.011 and 0.956, and 0.0307 and 0.965 for fish having lice and wounds, respectively.

1. Introduction

Nordic countries such as Norway are affected by harsh cold weather conditions and have only 3% of land available for farming [1]. Hence, people traditionally rely heavily on salmon fish to fulfill their daily food needs. According to the Food and Agriculture Organization of the United Nations, 7% of fish worldwide come from Norway [2]. Norway is the largest salmon fish producer in the world, but the aquaculture ecosystem is affected by diseases such as bacterial infections and fish wounds as well as parasites such as lice, which affect most aquaculture industries. As a result, salmon lice cause losses of up to 9% of farmers’ revenue [3].

The presence of sea lice in salmon fish aquaculture affects fish health by stopping or slowing down several stages of their development [4]. Antiphrasis drugs are used to avoid sea lice, but eventually, this causes drug resistance in sea lice after a long time. Hence, the fishing industry is searching for non-pathological solutions for controlling fish lice [5]. A recent study shows that feeding salmon fish with immunostimulants or essential oils reduces sea lice by 20% [6]. Various research has concluded that lice cannot be removed entirely but can be reduced to a certain amount [7]. Newer intelligent fish farm techniques are proposed by different researchers in which the use of automated underwater robots is proposed for counting sea lice but has a high implementation cost [8]. Current techniques rely more on manual visual methods. Therefore, a cost-efficient approach is needed to detect lice in fish.

Salmon fishes also suffer from wounds that can cause fish mortality as well. The presence of lice in the aquaculture fish leaves wounds, with maturity also causing fish wounds, especially in the mouth area [9]. Hence, wounds can give a hint about the irregularities rising inside the ecosystem. Automatic detection of fish containing injuries in a real aquaculture environment can help save the ecosystem. Fish behavior, tracking, counting, and species identification are possible using machine learning techniques utilizing underwater videos and images. Some of the recent valuable contributions have come from Li, Daoliang [10], Goodwin, et al. [11], Aditya et al. [12], and Knausgård et al. [13]. The availability of labeled datasets and powerful computational tools are among the highlighted reasons for this recent development [14]. An image-based fish disease detection is proposed by performing wound detection [15]. Image segmentation and feature extractions are performed using K-Means clustering. They utilize those extracted features and an SVM classifier to detect the disease and achieve an accuracy of 94%. However, the small dataset of only 210 fish can be seen as a limitation of this study. Biologists can identify epizootic ulcerative syndrome (EUS) diseases by analyzing the wounds in fish. Malik et al. use canny edge detection and the nearest neighbor (K-NN) algorithm to segment wounds in dead fish images [16]. The proposed technique is able to classify diseases with an accuracy of 86%.

Carrión proposes a mouse image-based wound assessment technique using YOLO-3 [17]. Wound images of the mouse are acquired at regular intervals to monitor the healing process. YOLO-3 is an efficient model for salmon fish and species detection by Olsvik et al. [18] but it has not been much utilized for fish wound detection. Earlier chemical analysis tests are also utilized to find fish diseases. 3-D imaging for salmon fish assessment is proposed by Sture et al. [19]. Irregularities such as fish wounds and body deformation are identified by utilizing shape and color. For wound detection, the red color channel of the image is used. A total of 45 dead fish are utilized for this study, in which, 16 fishes are affected by wounds. The proposed study shows an accuracy of 90%. Balaban et al. [20] use adaptive thresholding techniques to identify the blood spots in fish. These blood spots are considered wounds. The study was performed only on ten fish images; further validation on a larger dataset should be considered for future work. Pate [21] uses AccuProbe Mycobacterium gordonae and restriction enzyme analysis of PCR products (PCR-RFLP) test to identify fish diseases. The study was performed on 35 fish.

The underwater scenario is affected by challenges such as low illumination, dust particles, and oxygen bubbles, which make identifying lice and wounds difficult in a live tank [8]. Hence, there are chances of misclassifying wounds and lice with severe implications for the fish.

Limitations

Researchers have performed wound detection on dead fish using 2-D images; thus, using such a technique will not benefit the farmers working to treat live fish wounds. At the same time, many used smaller datasets, less than 100 fish, to test the algorithm. Many researchers also utilized a red color channel for wound detection, which may not be able to identify superficial wound damage or non-red-small wounds. Some used complex 3-D cameras on the dead fish, whereas some of the earlier research used chemical analysis for wound detection. Lice in freshwater lakes attack the fish and eat fish skin, leading to fish wounds. After the fish die, the lice detach themselves from their bodies. Hence, there is a lesser probability of detecting fish lice in dead fish and it is essential to identify lice and wounds on live fish rather than on dead fish.

New, simple, practical, and cost-effective techniques for 2-D live fish images need to be developed. This will allow aquacultural framers to treat wounds, identify lice in a fish tank, and save other fish by performing the necessary treatment. No specified datasets are present that contain wounds, non-wounds, and lice images of salmon fish. Therefore, there is a need to create such datasets, which other researchers can then use.

The following are the objectives of this study:

• Prepare an image dataset of underwater live salmon fish containing lice and wounds under various lighting conditions.
• Develop a convolutional neural network (CNN) based on fish lice and wound detection techniques using 2-D images.
• Design the convolutional neural network (CNN) to work efficiently in underwater scenarios and can able to classify fish as having wounds, lice, or an absence of wounds (nonwounded and non-lice) which performs better than standard models.

The paper is arranged as follows: Section 2 describes dataset preparation, whereas Section 3 thoroughly discusses the proposed machine-learning technique. Section 4, i.e., the results, describes the implementation of the algorithm and its outcomes with the help of Figures and Tables. The conclusions, including limitations and future work, are discussed in Section 5. The CNN models are built in python 3 taking the help of the Tensoflow library. Models are simulated on well know GPU known as Tesla v100 GPU.

2. Dataset

The first objective is to prepare an image dataset of healthy, wounded, and fish containing lice from real fish tanks utilized in aquaculture. We should also ensure that we capture these images under different illumination—we used a 2-D welfare camera from the Norwegian industry known as Createview¹ and worked with artificial intelligence for aquaculture. The output image has a dimension of 4112 × 3008 (B × H). The cameras are installed by the Norwegian Institute of Marine Research (IMR) situated in Bergen, Norway.

Datasets Creation

For dataset creation, the images of live fish are captured from fish tanks day and night to get variations in illumination conditions to match the datasets up with the practical scenario. The dataset is collected from October 2021 to January 2022. The acquired images contain dust particles, water bubbles, fish with lice, and fish affected by small and large wounds, as shown in Figure 1.

As the camera captures live images from the fish tank under different light conditions, hence a single image contains a lot of healthy, wounded, and fish with lice as shown in Figure 1. Wounded, nonwounded, and lice fish are cropped from the image to prepare the dataset. We can observe that some fish have tiny wounds, whereas others have more extensive wounds, i.e., considerable diversity among the wounds, representing the real world. Similarly, small fish containing lice are also captured for datasets. Changes in illumination especially at night make it harder to identify minor wounds and lice.

A total of 68, 71, and 70 images of healthy, wounded, and lice-infected fish were captured. Even though healthy fish images are abundant, to avoid data imbalance only limited healthy fish images are considered. A small dataset cannot fetch good results as neural networks are data-hungry. Thus, data augmentation strategies such as rotation zooming, shifting, horizontal and vertical flipping, contrast variation, etc., are used to increase the dataset. Thus, 15 images are generated only utilizing single images. An example of data generation is shown in Figure 2.

After data augmentation, the dataset size is increased to 3289 images containing wounded, nonwounded, and lice fish images. The dataset is split into 80:20 ratios, i.e., 2589 and 700 fish images are used for training and validating the model. Among these 700 fish images, 230 are lice fish images, and 230 and 240 represent nonwounded and wounded fish, respectively.

The objective is to build a CNN model which can detect different kinds of salmon fish wounds under multiple lighting conditions, which can include small skin damage (Figure 3c), lice attack (Figure 3a), or large wounds (Figure 3c,d). Figure 3 shows some of the sample images from the dataset with variable lighting conditions and different fish sizes.

3. Results

As a proof of concept of our data, A convolutional neural network (CNN) model is trained to perform the classification task. Images from the dataset are fed to train the CNN model. The flow chart of the proposed system can be identified in Figure 4.

3.1. Preprocessing

As mentioned earlier, captured images are affected by different lighting conditions; hence, eliminating this contrast variation is required to make an efficient classification. Unlike traditional histogram equalization, contrast-limited adaptive histogram equalization (CLAHE) prevents over-amplification of noise [22]. The CLAHE technique is utilized for this study to perform contrast stretching. Directly applying CLAHE on underwater RGB images corrupts the human color sense. Hence, CLAHE is applied on an HSV plane to spread the color intensities uniformly, leaving the colors themselves [23]. This study uses CLAHE over the HSV plane obtained after RGB to HSV transform HSV. After applying CLAHE, HSV to RGB color plane transform is applied to get back the original color space.

3.2. Convolutional Neural Network Model

The developed CNN model used for this study is based on the VGG-19 model but this model is specifically tailored to perform wound and fish lice detection. Unlike VGG 16 and VGG 19 [24], which contain 16 and 19 layers, respectively, the developed model consists of 15 convolutional layers, one flattening layer followed by five dense layers. Figure 5 clearly explains the proposed CNN model: A total of 15 convolutional layers are divided into 5 convolutional blocks. The first two convolutional blocks contain two convolutional layers, each containing 64 and 128 filters. The third convolutional block contains three convolutional layers having 256 filters. Each of the fourth and fifth convolutional blocks contains four convolutional layers, each containing 512 filters. The Max-Polling layer used after every convolutional block is 2 × 2. The first four dense layers are 2048, 1024, 256, and 64, respectively. The fifth dense layer contains the number of decision outcomes, i.e., three. Every convolutional filter is 3 × 3. (See Figure 5). Leaky-Relu is used as an activation function in the convolutional and first four dense layers., whereas in the last dense layer i.e., the fifth, the ‘Softmax’ function is utilized as an activation. Leaky-Relu does not suffer from dying Relu problems. The previous study also shows that Leaky-Relu performs better than the Relu function. The total trainable parameters for the proposed model is 35.5 million compared with 72.4 million and 79.8 million for VGG19 and VGG16. Hence, the suggested system is computationally less complex than existing standard models in the literature.

A dropout of 20% is utilized in every dense layer to help in reducing overfitting. The dropout layer after every convolutional block is avoided as these will decrease the model accuracy. A dropout layer of 10% is introduced after the second and third convolutional blocks. Overfitting problems and regularization can be resolved by introducing Gaussian noise during CNN model training [25]. A zero mean Gaussian noise with a standard deviation of 0.01 and 0.03 is added before the input of the second and third convolutional blocks, respectively (refer to Figure 5). Adding Gaussian noise will also help reduce overfitting and improve the system’s accuracy [25]. To further overcome overfitting, L1 regularization is introduced in the convolutional and dense layers.

4. Discussion

The proposed machine learning technique was applied to 3289 equally distributed datasets of healthy fish, fish affected by lice, and wounds. In total, 20% of the dataset, i.e., 700 images, were used for validation purposes. The test images contained 230 wounded and lice fish and 240 healthy fish. Due to fish image cropping from the captured image, resizing the image was required. Every image was resized to 180 × 100 × 3 (W × H × no. of channels). As discussed earlier in the methodology section, input images were converted to the HSV color space. Then, the hue plane was taken, CLAHE was applied, and images were again converted to the RGB color space. This operation helps to tackle the light variation effect.

The proposed CNN model was trained for 400 epochs utilizing the Adam optimizer [26] with a learning rate of 0.0001 or 1 × 10⁻⁴. The total number of parameters was 35.5 million, among which, 6784 are non-trainable parameters. Validation accuracy was monitored as an early stopping parameter for 30 epochs, to stop the training of the CNN model automatically if the model cannot improve on validation accuracy. A checkpoint was also used. It stores the best model among all the iterations based on the validation accuracy of each epoch.

It took 30 min to train the CNN model on the above-mentioned GPU, compared to 12 h. on the 10th generation laptop with 8 GB RAM and an Intel i5 processor. The training and validation accuracy and loss vs. epoch graphs were plotted, as shown in Figure 6. The accuracy of the training dataset is more than 99% before 50 epochs and the validation accuracy reaches a maximum accuracy of 96.7%.

The confusion matrix (True label and Predicted label) is plotted based on the obtained output for the proposed CNN model. The confusion matrix can be found in Figure 7. Here, normal fish are referred to as healthy fish.

The false-positive rate (FPR) and true-positive rate (TPR) are calculated utilizing Equations (1) and (2), respectively [27].

$$\mathit{FPR}=\frac{FP}{FP+TN}$$

(1)

$$\mathit{TPR}=\frac{TP}{FN+TP}$$

(2)

where TN and FP are true negatives, and false positives, respectively, and FN and TP are false negatives and true positives.

FPR and TPR were calculated for individual classification using Equations (1) and (2). For wound detection, the system had a false negative (not a wound but counted as a wound) and a true negative value of 14 and 456. The true positive and false negative were 222 and 8. Similarly, for lice detection, the system had a false negative (no lice present but classified as lice) and a true negative value of 5 and 465. The true positive and false negative were 220 and 10. The proposed model achieves a FPR of 0.0109 and 0.030 for lice and wound detection, respectively, whereas the TPR was 0.956 and 0.965 for lice and wound detection. During the overall calculation, the false positive was 5 (falsely predicted as a wound or lice when it is a healthy fish) and the true negative was 235, which were healthy fish. The CNN model had an overall accuracy of 96.5% on the testing dataset. The FPR and TPR were 0.0212 and 0.97, respectively. The FPR and TPR for overall wound and, lice detection are described in

Table 1. The FPR and TPR for wound and lice detection.

Parameter	FPR	TPR
Wound Detection	0.0307	0.965
Lice Detection	0.0109	0.956
Overall	0.0212	0.97

.

The proposed CNN model was also compared with VGG-16 and VGG-19 in

Table 2. Comparison of different models.

Model	Validation Accuracy
Proposed Model	96.7%
VGG-16	91.2%
VGG-19	92.81%

. The proposed system had better accuracy of 96.7% compared to 91.2% and 92.85% with VGG-16 and VGG-19.

5. Conclusions

The aquacultural fish industry is highly affected by lice in the fish tank. It is often hard to recognize these lice at the start. Similarly, wounds affect fish due to early maturity and sea lice, disease, and fish fighting among themselves, causing economic losses due to fish mortality. Identifying lice and wounds on salmon fish is challenging but could save the aquaculture ecosystem and prevent money losses for farmers. The objective of this study is to propose a salmon fish wound and sea lice detection technique using an image-based convolutional neural network-based method.

No specific datasets were available containing fish with lice, wounds, and healthy fish. The datasets contain images of healthy salmon, fish affected by sea lice, and wounds in equal distribution captured from live fish tanks. A total of 20% of the dataset was used for the validation of the system.

A CNN model consisting of 5 dense and 15 convolutional layers is trained and later tested for the detection of fish with wounds or lice as a proof-of-concept implementation to check the validity of the dataset. The model is trained and tested using a GPU called Tesla V100. Leaky-Relu was used as an activation. To avoid overfitting Gaussian noise, dropout layers are also utilized in the CNN model. Validation accuracy was used as deciding parameter for an early stop, which helped to avoid overfitting. The proposed CNN model gives a test accuracy of 96.7%, which is better than the accuracy of 91.2% and 92.81%, achieved by the VGG-16 and VGG-19 models, respectively. The overall true-positive and false-positive rates are 0.97 and 0.0212, respectively.

Currently, the proposed methodology can only classify wounds, lice, and healthy fish, but the system cannot locate the wound or lice on the fish. Hence, locating wounds and lice on fish using object detection techniques can be seen as future work. In addition, several types of wounds were present on fish, from small skin removal to large deep wounds. The researchers can therefore take the classification of these wounds as future work.