Predicting the Body Weight of Tilapia Fingerlings from Images Using Computer Vision

Abstract

The aim of this study was to develop a mathematical model to predict the body weight of tilapia fingerlings using variables obtained through computer vision. A total of 2092 tilapia fingerlings and juveniles, weighing between 10 and 100 g, were fasted for 12 h, anesthetized, weighed, and photographed using an iPhone 12 Pro Max at 33 cm height in a closed container with different bottom colors. Images were segmented using Roboflow’s instance segmentation model, achieving 99.5% mean average precision, 99.9% precision, and 100% recall. From the segmented images, area, perimeter, major axis (MA), minor axis (SA), X and Y centroids, compactness, eccentricity, and the MA/SA ratio were extracted. Seventy percent of the data was used to build the model, and 30% for validation. Stepwise multiple regression (backward selection) was performed, using body weight as the dependent variable. The prediction model was: −17.7677 + 0.0007539(area) – 0.0848303 (MA) – 0.108338(SA) + 0.0034496(CX). The validation model showed similar coefficients and R² = 0.99. The second validation, using observed versus predicted values, also yielded an R² of 0.99 and a mean absolute error of 1.57 g. Correlation and principal component analyses revealed strong positive associations among body weight, area, axes, and predicted values. Computer vision proved effective for predicting tilapia fingerlings’ weight.

1. Introduction

Evaluating animal weight is fundamental throughout all stages of fish production, as it is essential for growth monitoring, feed adjustment, and determining the optimal harvest time [1]. Fish weight changes rapidly during the initial growth phase when animals consume the highest proportion of feed. These changes are influenced by genetic, environmental, nutritional, and husbandry factors [2]. Such weight variations have direct economic implications since heavier and larger specimens command higher market prices [3].

In most fish farms, weight monitoring is performed manually [4]. Manual weighing presents several challenges, including stress induced in fish during handling, which frequently occurs when farmers capture and release fish without anesthesia [5]. As a manual process, it is also prone to human measurement errors [6]. In aquaculture operations, fish weighing typically occurs near tanks or ponds. This practice is not only labor-intensive but may also lead to physical and psychological strain for the operators involved. Furthermore, it represents a costly activity due to its high labor requirements [7].

The automation of fish fingerlings’ weight biometrics presents a significant challenge for enhancing aquaculture productivity [8]. Computer vision techniques offer a solution by enabling rapid, precise, and non-invasive image analysis, thereby reducing animal stress while improving data collection efficiency [3]. Image-based approaches–utilizing either photographs or videos–have been successfully implemented across multiple aspects of fish production, including disease assessment, behavior monitoring, biomass estimation, population counting, and size classification [9,10].

Image segmentation is one of the applied computer vision techniques that allows the identification and delimitation of areas of interest in captured images [11]. Previous research has already been carried out with some fish species at or near the commercial weight stage using segmentation to estimate body mass and carcass yields [12]. There are few scientific studies evaluating this technique during the fry stage, and little effectiveness has been found for most of the species studied [13,14].

Furthermore, existing studies face several limitations, such as: (i) low accuracy for smaller fry due to image resolution, (ii) difficulty in generalizing to different, complex, and underwater farming environments, (iii) variability in specimen morphology, (iv) dataset sample size, and (v) uncertainty in model selection and technical issues with image-based methods. These factors can reduce the accuracy of the evaluated models and hinder their implementation in real aquaculture environments [11,13].

Therefore, it is relevant to computer vision techniques that are effective for estimating weight during the fingerling and juvenile stages, and can contribute to the development of technological products that automate weight estimation using less invasive and less stressful methods. Since tilapia is the main species produced in Brazil [15], the aim of this study was to develop a model for predicting the body weight of tilapia fry using variables obtained through computer vision.

This study stands out by applying an instance segmentation model specifically for estimating the body weight of Nile tilapia fingerlings and juveniles. The fingerlings stage is particularly sensitive, as physical measurement is delicate and prone to error. By also including juvenile specimens, a phase characterized by rapid growth and high morphological variability, the predictive models become more robust to the range of sizes and weights present during early development. The use of advanced segmentation techniques allows for the extraction of morphometric features with greater precision, even in small-sized and variably shaped individuals, enhancing the model’s accuracy and applicability in real-world production settings.

2. Materials and Methods

2.1. Data Collection

This project used 2092 tilapia fingerlings and juveniles (Oreochromis niloticus) between 10 and 100 g. The experimental procedures related to the animals were carried out following the standard guidelines issued by the Ethics Committee on the Use of Animals (CEUA–6002300124).

To obtain the images and the animals’ body weight, they were fasted for 12 h, then anesthetized by immersion in a solution of 60 mg of benzocaine. L-1, weighted, and photographed. The digital photos were taken using an iPhone 12 Pro Max. The device has three cameras, one with a 12 megapixel main sensor (f/1.6), a 12 megapixel telephoto lens (f/2.2), and a 12 megapixel ultrawide (f/2.4), with a resolution of 4000 × 3000 pixels. The main camera has a focal length of 26 mm, the telephoto lens 52 mm, and the ultrawide camera 13 mm. The Nestter JBA14008 electronic scale was used to weigh the fish, and the LCD display made it possible to view the commands and weighing values.

The images were collected in a closed container (sides and roof) with varying background colors, simulating different scenarios to ensure a consistent database. The photos were collected at a standard height of 33 cm (Figure 1). After collecting the images and weighing the fish, a database was created under the name FISHIJPG–2092SEGI.

The collected digital photos were converted to JPG and resized to 640 × 640 pixels. The database was expanded using data augmentation techniques (Figure 2) from the Roboflow platform (Roboflow website), with the following techniques to increase the generalization power of the model and the database:

(A) Rotation: to create a mirrored version of the original image, resulting in a horizontal or vertical inversion;

(B) Shear: to obtain images tilted along the horizontal or vertical axis;

(C) Grayscale: to obtain grayscale images, so that the colors are less emphasized in the model;

(D) Hue: to obtain different colored images without affecting brightness or saturation;

(E) Saturation: to obtain images with different color intensities without altering hue or brightness.

The images were also labeled and pre-processed on the Roboflow platform. A neural network was then trained to segment the images in order to automate the segmentation process. To do this, 2092 images were used, 70% for training, 20% for validation, and 10% for testing. The following metrics were used to assess the quality of the segmentation: Mean Average Precision (mAP), precision, and recall.

With the model trained, masks were generated for the images to identify the regions of interest. Using the masks generated, a new data set was created with the segmented images and the information on area, perimeter, major axis (MA), minor axis (SA), X centroid (CX) and Y centroid (CY) extracted using the Python program (version 3.12.0) using the OpenCV library (Open Source Computer Vision Library). The Major-Minor Axis Ratio (RMM) [16], compactness [17] and eccentricity [18] were calculated:

$$RMM={\displaystyle \frac{4\pi xArea}{Perimete{r}^2}}$$

$$Compactness={\displaystyle \frac{Area}{Perimete{r}^2}}$$

$$Eccentricity=\sqrt{1-\left({\displaystyle \frac{smallestaxi{s}^2}{largestaxi{s}^2}}\right)}$$

Compactness is a measure that expresses how closely the shape of a segmented object resembles a circle. Values closer to 1 indicate more rounded shapes, while higher values suggest elongated or irregular forms. This metric is useful for assessing the regularity of object boundaries. Eccentricity represents the degree of elongation of an object, based on the ellipse that best fits its shape. It ranges from 0 to 1, where 0 corresponds to a perfectly circular shape and values close to 1 indicate more stretched or elliptical forms. Major-Minor Axis Ratio (RMM) is the ratio between the length of the major axis and the minor axis of the best-fitting ellipse around the object. This ratio indicates shape elongation: values close to 1 suggest rounded shapes, while higher values indicate greater elongation.

2.2. Statistical Analysis

An exploratory analysis of the data was carried out to check for consistency, and only observations within ±3.0 standard deviations of the mean were considered for each variable. To analyze the data, 70% of the data was used to obtain the model, and 30% for validation. The model for predicting body weight, based on the information extracted from the segmentation, was obtained using multiple linear regression methods using the Stepwise procedure with the Backward option, considering body weight as the dependent variable and the independent variables obtained from the segmentation.

The model used for the multiple linear regression [19] was Equation:

Yi = β₀ + β₁Xi₁ + β₂Xi₂ + …+ β_pXi_n + e

where Y represents the dependent variable, which is the explanatory variable (constant); β_p is the coefficient of each independent variable, and represents the random (residual) experimental error.

To do this, all possible combinations between the independent variables previously selected by the Stepwise procedure were tested, and the VIF values and coefficients of determination (R²) were assessed in the models containing two and three variables. Based on these criteria, the model with the absence of multicollinearity (VIF ≤ 1) and the highest R² was selected, ensuring greater explanatory capacity and statistical robustness. The fit of the validation data to the model obtained was checked, the residual analysis was also carried out, and the predicted values were obtained and compared to the observed values using the regression model [19]:

Y = β₀ +β₁X,

H₀: β₀ = 0 and β₁ = 1

H_a: not H₀

where Y represents observed values, X represents predicted values, β₀: intercept of the equation, and β₁: angular coefficient of the equation. The regression was evaluated according to the following statistical hypotheses [16].

To better understand the relationship and importance of the variables, principal component analysis and correlation analysis were conducted on the validation database and the predicted value. The analysis was carried out using the computer packages corrplot, factoextra, FactoMineR, MVar.pt, stats, and tidyverse from the R computer program (version 4.4.1; R Core Team, 2024).

3. Results

Training the neural network for segmentation showed that the trained model obtained mAP 99.5%, precision 99.9%, and recall 100%. The model obtained using the Stepwise procedure with the Backward option was Y = 17.7677 + 0.00075-area − 0.0848 − MA–0.1083-SA + 0.00345-CX, showing a high coefficient of determination (R² = 0.99), which indicates excellent explanatory power (

Table 1. Parameters, estimates, standard error, lower limit (LI), upper limit (LS), and coefficient of determination (R²) of the models obtained and validated.

Models	Parameter	Estimate	Standard Error	LI	LS	R2
Model Stepwise
Obtained	Intercepto	17.760	1.702	14.432	21.103	0.99
	Area	0.00075	0.000011	0.000732928	0.000774872
	MA	−0.0848	0.002712	−0.090146016	−0.079514584
	SA	−0.1083	0.007221	−0.122491848	−0.094184352
	CX	0.00345	0.000248	0.002962736	0.003936464
Validation	Intercepto	23.567	4.200	15.335	31.799	0.99
	Area	0.0008	0.000026	0.000749658	0.000853342
	MA	−0.0974	0.006705	−0.1105229	−0.0842393
	SA	−0.1281	0.01735	−0.16209	−0.094078
	CX	0.00436	0.0006	0.003184896	0.005536504
Final Model
Obtained	Intercepto	−28.83	0.5742	−29.53	−27.7	0.99
	Area	0.000462	0.00001243	0.00045937	0.00046427
	CX	0.004234	0.000387	0.0036287	0.004839
Validation	Intercepto	−28.86	0.6946	−29.81	−27.09	0.99
	Area	0.000462	0.00001514	0.00045976	0.0004657
	CX	0.003917	0.0003739	0.003183755	0.004650813

LI: lower limit; LS: upper limit; R²: coefficient of determination; MA: major axis; SA: minor axis; CX: X centroid.

). However, analysis of the variance inflation factors (VIF) revealed excessively high values for the variables area (VIF = 116.85), MA (VIF = 39.25), and SA (VIF = 47.75), showing strong multicollinearity. This compromises the stability of the estimated coefficients and may affect the reliability of the individual inferences for these variables, despite the overall good fit of the model.

After evaluating all the possible combinations between the independent variables based on the VIF values, and starting from the initial model generated by the Stepwise procedure, a more parsimonious model was obtained, with less collinearity between the predictors: Y = −28.83 + 0.0004618-area + 0.004234-CX. In this model, the VIF values for both the area and CX variables were equal to 1.0057, indicating no multicollinearity. This reformulation preserves the explanatory power of the model while improving the stability of the estimates, making it more robust for predictive and inferential purposes.

Table 1 shows the adjusted model for the test set, considering the same variables, with the respective coefficients, standard errors, confidence intervals, and the coefficient of determination. The models generated by the Stepwise procedure and the final model showed no significant differences between the estimated parameters and the validated models. All the parameters were significant, and both models had coefficients of determination of 0.99.

The fit test Y = β₀ + β₁X was performed only for the final model. showed that there was no difference between the predicted value and the actual value (H₀: β₀ = 0 and β₁ = 1), with a coefficient of determination of 0.99 (R²), a root mean square error (RMSE) of 2.52 and a mean absolute error (MAE) of 1.42 g.

Principal component analysis showed that the first and second components explained 79.9% of the total variation in the data (Figure 3). The first component explained 56.9%, with a high and positive correlation with the variables area, perimeter, major axis, minor axis, observed body weight, and predicted body weight. The second component explained 23% of the total variation in the data, correlating positively and highly with compaction and RMM. The second component showed a high and negative correlation with eccentricity. The importance of the variables in each principal component is shown in Figure 4.

It was observed that SA, MA, weight, area, observed, and predicted body weight showed high and positive correlations with each other (Figure 5). Compaction showed a high and positive correlation with RMM, and these variables showed high and negative correlations with Eccentricity.

4. Discussion

It was found that the neural network used to segment the images was effective, as the metrics (mAP, precision, and Recall) obtained were above 99% mAP, precision, and Recall. To the author’s knowledge, there is no research that defines the minimum value for these metrics to classify/qualify the neural network; however, this value usually varies between 0 and 100 (personal observation). The higher these metrics are, the more effective the neural network is in machine learning.

Neural networks have also been used to segment images in aquatic camera environments with various species of fish, with results of over 90% with 2000 images [20]. In images collected all with an in-water camera, accuracy is expected to be lower due to various factors such as the complexity of the marine environment, water turbidity, color contrasts, lack of focus, and fish movement.

Image segmentation has also been carried out on Pintado Real^® catfish fingerling (Pseudoplatystoma corruscans) without anesthesia, resulting in mAP below 50% [21]. This low accuracy is due to the quality of the images, as the fish were constantly moving during collection, causing different body shapes and orientations in the images, as well as a reduction in focus.

The automation of image segmentation using neural networks in this study proved to be effective due to the high mAP. This high mAP is due to the way the images were collected, since the fish were anaesthetized (immobile), out of the water, and considering only the body of the fish, not the fins (body appendage).

Fernandes et al. reported in their work with tilapia that segmenting the fins is a challenge, due to the smaller number of pixels, the difficulty in differentiating the edges of the fins from the body and the background of the image [12]. The authors reported that image segmentation can indirectly influence fish weight. This was not observed in this study with tilapia fry to juveniles. Another difficulty in considering the fins is the way they are arranged in the image, especially the dorsal and caudal fins (tilted, lowered, or turned).

Tilapia fins have a low yield, ranging from 1 to 3% [22]. This variation is due to the genetic variation of the trait itself. The fact that the fins are not considered when segmenting is justified because the fins can be lost or damaged by conflicts between the fish or even by aquaculture management activities. In addition, since fins are positioned in the extremities, they can have lesions due to common aquatic pathogens, such as Flavobacterium spp.

Automating segmentation using neural networks offered several advantages in this work, including: (a) Increased accuracy by reducing human errors that can occur due to fatigue or subjectivity; (b) Improved consistency, as the trained algorithm provides more consistent results, where uniformity is essential; (c) Faster processing and scalability, as automated systems can process a lot of data in less time than a human, improving productivity; (d) Reduced labor costs.

Through multiple regression analysis using the Stepwise procedure with the Backward option, an initial model with high explanatory power (R² = 0.99) was obtained, composed of the variables area, MA, SA, and CX. All these variables showed statistically significant coefficients, indicating a relevant contribution to the prediction of body weight. However, the variance inflation factor (VIF) analysis revealed high levels of multicollinearity among the predictors area (VIF = 116.85), MA (VIF = 39.25), and SA (VIF = 47.75), which compromises the stability and interpretation of the estimated coefficients, despite the overall quality of the fit.

Given this limitation, new combinations of variables were evaluated based on the VIF values in order to obtain a more parsimonious model. Based on the initial model generated by the Stepwise procedure, a final version was created consisting only of the area and CX variables, both statistically significant and with VIF values equal to 1.0057, indicating no multicollinearity.

This reformulated model maintained its high coefficient of determination (R² = 0.99), demonstrating that the exclusion of collinear variables did not compromise the model’s predictive capacity. In addition, simplification favors data collection, reduces computational effort, and improves statistical robustness, making it more suitable for practical applications in production contexts, such as estimating the body weight of tilapia fry.

Maintaining the statistical significance of the variables in the final model reinforces their relevance in explaining body weight variation, while eliminating multicollinearity increases the reliability of inferences. Therefore, the final model represents a balanced solution between complexity, performance, and interpretability [23,24,25].

The high coefficients of determination found in the adjustment and validation models, using the variables extracted through segmentation, showed the importance of these variables in predicting body weight. This was also observed when modeling weight as a function of the predicted value. According to [2], it is recommended that the coefficient of determination should be greater than 0.70; in other words, that it should be able to explain at least 70% of the total variation of the data.

The equality of the parameters obtained in the adjusted and validation models indicates that the adjusted model does not differ from the model obtained from the validation database. This indicates that the model can be used to estimate the weight of tilapia fry. This was also confirmed when adjusting body weight as a function of the predicted value.

Model validation is essential to ensure the reliability of predictions and the generalizability of results [23]. The equality of the parameters between the fitted model and the validation model indicates that the fitted model is capable of providing accurate estimates when estimating the weight of tilapia fry. In addition, the high association between the actual value and the predicted value of body weight, together with the significant correlations with other morphological variables, reinforces the validity and usefulness of the model for further analysis and decision-making in aquaculture.

The correlation and principal component analysis showed a high positive association between the actual value and the predicted value of body weight, which were high and significant with area, perimeter, MA, and SA. These positive correlations were to be expected, as weight increases, so do body shape characteristics such as area, perimeter, major axis, and minor axis.

Although the use of high-resolution images taken under controlled conditions (with anaesthetized fish, a fixed distance from the camera, and a standardized background) provided excellent quality data for training the model, it is recognized that this approach may limit its direct applicability in real aquaculture production systems. In commercial environments, operating conditions tend to be more variable, with challenges such as irregular lighting, spontaneous fish movement, different depths, and greater visual complexity in the background of the images.

However, the methodology adopted in this study was carefully structured to automate a classic fish farming operation: manual weighing or biometrics, often carried out on trays in sorting and grading processes. The proposed system, with the camera fixed to the top of the containment structure, was designed to simulate this operating environment, offering an automated, accurate, and easy-to-implement alternative with the potential to replace the conventional method without the need for significant changes to the production routine.

In addition, the conditions under which the images were taken were deliberately kept simple and adaptable to the reality of the field, in order to make it easier to replicate the system in different production units. It should be noted that the only source of light used was the cell phone’s own flashlight, without the need for reflectors, auxiliary equipment, or strict lighting control. This aspect demonstrates the viability of the system even in environments with basic infrastructure, maintaining the quality required for effective segmentation and reliable prediction of body weight.

The main benefits associated with adopting this automated system include: a reduction in human errors when weighing and recording data, standardized measurements, faster information processing, and the possibility of integration with digital aquaculture management platforms, contributing to a more efficient, traceable, and technically sustainable production process.

To enhance model applicability, future studies should consider image acquisition under actual farming conditions, without requiring anesthesia, aiming to train more variation-resistant models. Furthermore, data augmentation techniques can simulate environmental variations during training to improve generalization—an approach implemented in the current study. Other promising strategies include object detection or semantic segmentation algorithms capable of operating in noisier environments, along with deployment of real-time multisensor systems for automated monitoring in commercial tanks.

Model validation using only an internal dataset, without testing on independent data or under real operational conditions, represents a limitation for the generalizability of the results. Techniques such as k-fold cross-validation could be incorporated in future studies, as this method may reduce overfitting and provide reliable estimates of model performance. However, it does not fully replace the need for evaluation using external datasets [24,26].

Thus, the lack of validation in diverse environments (e.g., different aquaculture farms, variable lighting conditions, or distinct growth stages) may limit the model’s practical applicability, especially in commercial production systems where variability is significantly higher [25,27]. To address this limitation, future studies should prioritize collecting and evaluating independent datasets obtained under real farming conditions. Collaboration with different production units would allow testing the model’s robustness against uncontrolled variations, such as changes in water quality, stocking density, and fish behavior.

Furthermore, implementing techniques such as transfer learning or federated models could improve adaptation to new environments, ensuring the tool maintains high accuracy even when applied outside its initial development context. These advancements could further contribute to transforming the tested model into a reliable solution for precision aquaculture [26,27,28,29,30].

Another point is the weight of the juvenile tilapia evaluated in this study, which ranged from 10 g to 100 g. The reason for choosing this range was due to the critical importance of this stage for aquaculture productivity. At this stage, small variations in weight have a direct impact on feed efficiency and batch uniformity, which are relevant economic factors in production [30,31]. In addition, models for juveniles may require different parameters, such as metabolic rate and feed conversion, compared to adults, given the non-linearity of growth [32,33,34].

We recognize that the models evaluated were not tested on adult tilapia, so future studies will evaluate the weight range above 100 g, as well as validating other production species, such as specimens that include the Serrasalmidae family, such as pacu and tambaqui, which are also economically important fish, especially in Latin America.

5. Conclusions

It can be concluded that the body weight of tilapia fingerlings and juveniles can be effectively predicted using a multiple linear regression model with vision data obtained by segmenting images using computer vision. Future work evaluating different technologies for implementing the model in real time should be conducted to better understand the feasibility of such a technique in an aquaculture setting.