Machine Learning-Based Live Weight Estimation for Hanwoo Cow

: Live weight monitoring is an important step in Hanwoo (Korean cow) livestock farming. Direct and indirect methods are two available approaches for measuring live weight of cows in husbandry. Recently, thanks to the advances of sensor technology, data processing, and Machine Learning algorithms, the indirect weight measurement has been become more popular. This study was conducted to explore and evaluate the feasibility of machine learning algorithms in estimating the body live weight of Hanwoo cow using ten body measurements as input features. Various supervised Machine Learning algorithms, including Multilayer Perceptron, k-Nearest Neighbor, Light Gradient Boosting Machine, TabNet, and FT-Transformer, are employed to develop the models that estimate the body live weight using body measurement data. Data analysis is exploited to explore the correlation between the body size measurements (the features) and the weights (target values that need to be estimated) of cows. Data analysis results show that ten body measurements have a high correlation with the body live weight. High performance of all applied Machine Learning models was obtained. It can be concluded that estimating the body live weight of Hanwoo cow is feasible by utilizing Machine Learning algorithms. Among all of the tested algorithms, LightGBM regression demonstrates not only the best model in terms of performance, model complexity and development time.


Introduction
In South Korea, among all types of beef in the market, people prefer native beef despite their prices being much higher than that of imported products. Among four types of native cattle breeds being raised for beef demand, Hanwoo is the most popular one [1]. With highly marbled fat, thin muscle fibers, and minimal content of connective tissues, Hanwoo beef is well-known for its distinctive flavor [2]. To maintain the valuable characteristics of beef, the livestock management procedure takes an important role. In that procedure, livestock live weight monitoring is critical since it is considered one of the most important traits affecting animal condition [3]. In the management procedure, accurately estimating or measuring live weight is of fundamental importance to any livestock research and development.
Currently, there are two main approaches available to measure the live weight of livestock, including direct and indirect methods. The direct measurement method using scales can get very high accuracy. However, it still has some limitations. First of all, the measurement process in this approach requires removing bulls from cages or paddocks, and guiding them one by one to the weighting station or the site of scale. This process is highly time-consuming and cumbersome. Sometimes workers who stay close to the bulls might get hurt as some cattle individuals are very stubborn. Secondly, this process is believed to be able to cause stress and potentially harmful to bulls, even leading to weight loss or death [4]. Because of those disadvantages of the direct measurement methods, various ways of indirect measurement have been proposed as the alternative approach [4][5][6][7][8][9][10][11][12]. The indirect approach is considered an estimation of the true value of live weight since it indirectly computes that value using sensor data and computational techniques. Weight estimation using body size measurements has been used extensively in the livestock industry, for both carcass weight and live weight. A typical indirect live weight measurement method consists of three steps. In the first step, different body characteristics and size of cattle are collected by sensors such as 2D camera [4], thermal camera [13], 3D camera [14,15] and ultrasonic sensor [16,17]. In the second step, body features are extracted by data processing techniques. Finally, body features are fed into a regression model to estimate the body weight.
The task of estimating body weight using body measurements can be considered a regression problem where the body measurements are input features and body weight is the target value that the regression model needs to predict. The estimation of dairy Holstein cattle live weight was reported by Tedde et al. with Root Mean Square Error (RMSE) ranging from 52 to 56 kg [12]. A study on estimating the live weight of pigs was conducted by Sungirai et al. [11,18]. Regarding sheep live weight estimation, a study was conducted by Sabbioni et al. [10]. It should be noted that one estimation model, when applied to different cattle breeds could have different prediction performances. For example, regression analysis was exploited to predict body weight from body measurements in Holstein, Brown Swiss, and crossbred cattle with R2 scores of 92%, 95%, and 68%, respectively [8]. In the case of the Hanwoo cattle, the study of live weight estimation was carried out by Jang et al. with the performance demonstrated by RMSE and MAPE errors of 51.4% and 17.1%, respectively, using body size measurements including body length, withers height, chest width, and body width [7].
Machine Learning (ML) has a long history dating back to the year 1959, and the term was coined by Arthur Samuel [19]. ML algorithms can be categorized into three types of learning: reinforcement learning, unsupervised learning, and supervised learning. Among three categories of learning algorithms, supervised learning is employed for the task of estimating live body weight of cattle in livestock. Supervised learning algorithms try to learn from the labeled datasets to approximate the mapping function between inputs (features) and outputs (target values). There are a huge number of supervised learning algorithms, such as Linear Regression, k-Nearest Neighbor (kNN), Support Vector Machine (SVM), Decision Tree, and Artificial Neural Network (ANN) or Neural Network (NN). Among existing supervised learning algorithms, ANN is considered an arbitrary accuracy function approximation [20]. ANN is a ML algorithm that utilizes data computational structure inspired by the nervous system of the superior organisms [21]. A typical NN consisted of an input layer, hidden layers, and an output layer. Deep Learning (DL) or Deep Neural Network (DNN) are special neural networks that consist of many layers of data processing units. The main advantage of DNN over NN is the ability to automatically learn features from raw data, without the hand-craft features. Nowadays, DL and DNNs are dominant in almost every kind of unstructured data: serial data, 2D data, and 3D data. Various types of DNN have been proposed for different types of data. For example, Convolutional Neural Network (CNN) models are suitable for image data [22] while attention-based neural network models are dominant in natural language processing applications [23]. In tabular data, tree-based ensemble learning is still believed to outperform other types of learning algorithms [24,25]. However, some DNN models proposed recently can have comparable performance in tabular data tasks [24].
In this paper, a dataset including 33,536 samples of Hanwoo cows is used to develop live weight predictive models. Each sample consists of 10 body measurements along with the age and body weight. Although previously, live weight body estimation of Hanwoo was studied by Jang et al. [7] with only 4 body measurements, the predictive performance is still low (RMSE 51.4 and MAPE 17.1%). In this paper, a dataset with more features and much more samples, besides conventional ML algorithms, more advanced ML-based sophisticated predictive models are employed to develop live weight predictive models. The main contributions of this paper are as follows: • Analyze ten body measurements of Hanwoo and their impact on the prediction of body weight. • Investigate ML algorithms in estimating live body weight. • Improve predictive performance over previous studies.

Hanwoo Body Measurement Data
The Hanwoo data used in this research was provided by the National Institute of Livestock Science, Korean Rural Development Administration. The data consist of 33,546 records of male individuals with ages of 6, 12, 18, and 24 months. The total numbers of individuals in four age groups are 4088, 16,574, 7185, and 5699 respectively. The dataset is split into training, validation, and test datasets with a ratio of 70%-15%-15%. The training and validation datasets are used for developing predictive models. During the training process, the validation dataset helps avoid the over-fitting phenomenon of the training process. The test dataset is used to evaluate the performance of trained models on unseen data.

Machine Learning-Based Predictive Models
In this work, one of the major goals is to investigate the performance of supervised ML algorithms in estimating Hanwoo cow live weight. Currently, there are a huge number of supervised ML algorithms. Therefore, an exhaustive investigation considering all of the algorithms is not feasible in the scope of this research. As a result, three representative algorithms are taken into consideration, including Light Gradient Boosting Machine (LightGBM) [27], TabNet [28], and FT-Transformer [29]. In tabular data, tree-based ensemble learning is still believed to outperform other types of learning algorithms [24,25]. Nowadays, among various types of tree-based machine learning algorithms that have been proposed, LightGBM is considered one of the most efficient algorithms [27]. Whiles, DNN models are extensively employed for unstructured data. Recently researchers have been attempting to use DNN models, the most prominent are TabNet and FT-Transformer, for solving tabular data tasks.
Besides three modern ML models, kNN and MLP are two traditional ML algorithms taken into consideration to make a comparison. Among the five models using in this work, TabNet and FT-Transformer are DNN models, whiles kNN, MLP, and LightGBM are shallow ML models. In order to evaluate the performance of weight estimation, two metrics are exploited including Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE).

Machine Learning Models
LightGBM is a Gradient Boosting Decision Tree (GBDT) algorithm invented by Ke et al. [27]. LightGBM incorporated two novel techniques: Gradient-based One-Side Sampling (GOSS) and Exclusive Feature Bundling (EFB). GOSS help to exclude a significant proportion of data samples with small gradients and keep the remaining data samples for estimating information gain. EFB helps to bundle mutually exclusive features to reduce the number of features. In this work, the LightGBM Python Package (version 3.3.2) was used to build the model with 100 base learners (decision tree) and the maximum tree depth of base learners is 32.
MLP is a conventional NN whose parameters are updated by the back-propagation training process [30]. MLPs are universal function approximators as shown by Cybenko's theorem [31]. Model kNN is a non-parametric ML algorithm since it does not make any assumptions on the data [32]. kNN algorithm uses feature similarity to predict the target values of new samples. This means that the target value of a new sample is computed by its distances to the data samples in the training dataset. Model MLP composes of two hidden layers with 30 and 20 neurons, respectively; and model kNN with k = 11 were built with the the Scikit-Learn Python Package (version 1.0.2).

Deep Neural Network Models
TabNet is a DNN model employing the attention mechanism for tabular data invented by Sercan O Ark and Tomas Pfister in 2021 [28].
FT-Transformer was invented by Gorishniy Y. et al. [29] in 2021 is a simple and efficient adaptation of transformer architecture-based for the tabular data. The synonym FT-Transformer stands for Feature Tokenizer Transformer. The architecture of FT-Transformer consists of Feature Tokenizer module and Transformer module as shown in Figure 3. Feature Tokenizer module transforms the input features x into embedding T. After tokenizing, the stacked embedding T 0 is obtained by stacking the embedding T token [CLS]. Transformer layers F 1 , . . . , F L are applied to obtain T i , where T i = F i (T i−1 ). In this work, model FT-Transformer was built with RTDL Python Package (version 0.0.13) [29]. To train the model with back-propagation training, AdamW optimization algorithm [33] was adopted with parameters learning rate = 0.001, betas = (0.9, 0.999), epsilon = 10 −8 , and weight decay = 0.01.

Correlation Analysis
Pearson correlation [34] is employed to measure the linear correlation between two body size measurements. The correlation results are shown in Table 6. All the corresponding p-values are also computed and showed very small values which indicate that all measurements have significant correlations with the live weight of the cattle body. Of all body measurements, Hungwe has the highest correlation with body weight under all ages of cows. The analysis also shows that all body size measurements highly correlate with each other, as in Figure 4.   It can be observed that Hungpok, Yogak, Gonpok, and Jagol change their correlation with live weight a lot among four categories of months. For example, in 6-month data, the correlation value between Jagol feature and weight is only 0.295, while in 18-month age data the correlation value is 0.65. Based on that observation, body size measurements are organized into three groups: group A is of stable correlation variables (Hungwe, Chego, Sibza, Chejang, Hungsim, and Gojang); group B includes unstable correlation variables (Hungpok, Yogak, Gonpok, and Jagol), and group C consists of all variables.

Live Weight Prediction Results
Each considered ML model will be developed with three different groups of variables: A, B, and C. Moreover, the age values of individuals are also considered an independent variable and used as input features to train predictive models. The training and validation datasets are used during the training process. The training dataset is used to update the model parameters while the validation dataset is used to stop the training process early to prevent overfitting. The test dataset is reserved for evaluating trained models. All final evaluation results computed on the test dataset are shown in and Figure 5.   Among three groups of features A, B, and C, it can be observed that group B which consists of unstable features (Hugok, Yogak, Gonpok, Jagol) has the poorest performance. Group A which consists of stable features (Chego, Sibza, Chejang, Hungsim, Hungpok, Yogak, Gojang, Gonpok, Jagol, Hungwe) has better performance. Group C which includes all features gives the best performance of all groups.
Compared between the two cases of using and not using age as an input feature, it can be observed that when a model uses age as an input feature, it often has lower RMSE and MAPE values. In the cases of feature groups A and C, the differences between using age and not using age are very small. However, in the case of feature group B, the difference is quite obvious. For example, in the FT-Transformer case, using age can have 36.818% and 7.179% for RMSE and MAPE, respectively. But if not using age, the performance decreases dramatically to 52.009% and 11.236% for RMSE and MAPE, respectively.
Among the five compared models, LightGBM has the best performance. After considering all combinations of cases, it can be noted that the best performance is in the case of using 10 features, which means 10 body size measurements without age, with model LightGBM (RMSE = 24.754, MAPE = 4.721%). The worst case is the case with the FT-Transformer model, using only four features Hungpok, Yogak, Gonpok, and Jagol; in this case, RMSE = 52.009, MAPE = 11.236%.
In order to attenuate the effect of the randomness in analyzing the results of predictions, experiences with each model are conducted 6 times with the random initialization of parameters and split of training set-validation set. Dispersion analysis of RMSE and MAPE errors are shown in Table 7 and Figure 6. In all the cases, it can be observed that the kNN model has the smallest dispersion and the MLP model has the biggest dispersion. The reason explaining for that small dispersion value is that kNN is a non-parametric model. kNN is not affected by the random initialization of parameters. In general, all models but MLP have small dispersion which indicates that the prediction results of these models are reliable.

Feature Importance
The feature importance of body measurements according to the LightGBM model in the case of using all features is shown in Figure 7. It can be observed that Hungwe is the most important feature while Jagol is the least important feature. The second most important feature is Chejang. All other features have lower and similar importance but still have a large contribution to the prediction result of the LightGBM model. This result indicates that all features should be included in the predictive models.

Conclusions
In this work, ten body measurements of Hanwoo cow were used as the input features for the estimation of the body live weight. Pearson correlation analysis showed that all of the body measurement has the high correlation with the body weight. Among all features, the girth of chest girth (Hungwe) has the highest correlation with the body weight, while the width of hip bone (Jagol) has lowest correlation with the body weight. Experiment results showed that using different sets of features affects the performance of weight estimation. Using all features together provided the best performance in all cases of the estimation models. Age value was used as another feature to estimate body weight, and that often give a slightly better results in most case. Five different ML models have been investigated and evaluated. The tree-based model LightGBM regression demonstrated the highest performance. The results of this work will be used to develop an indirect live weight estimation for Hanwoo, in which machine vision technology is utilized to automatically measure ten body features of cows.