Analyzing Internal and External Factors in Livestock Supply Forecasting Using Machine Learning: Sustainable Insights from South Korea

Abstract

The supply of livestock products depends on many internal and external factors. Omitting any one factor can make it difficult to describe the market patterns. So, forecasting livestock indexes such as prices and supplies is challenging due to the effect of unknown factors. This paper proposes a Stacking Forest Ensemble method (SFE-NET) to forecast pork supply by considering both internal and external factors, thereby contributing to sustainable pork production. We first analyze the internal factors to explore features related to pork supply. External factors such as weather conditions, gas prices, and disease information are also collected from different sources. The combined dataset is from 2016 to 2022. Our SFE-NET method utilizes Random Forest, Gradient Boosting, and XGBoost as members and a neural network as the meta-method. We conducted seven experiments for daily, weekly, and monthly pork supply using different sets of factors, such as internal, internal and external, and selected. The results showed the following findings: (a) The proposed method achieved Coefficient of Determination scores between 84% and 91% in short and long periods, (b) the external factors increased the performance of forecasting methods by about 2% to 12%, and (c) the proposed stacking ensemble method outperformed other comparative methods by 1% to 18%. These improvements in forecasting accuracy can help promote more sustainable pork production by enhancing market stability and resilience.

1. Introduction

The sustainability of the pork market is essential for the government, producers, and consumers in South Korea. According to the Organization for Economic Co-operation and Development (OECD) [1], since 2014, pork consumption has been about 2.7 times higher than beef, 1.7 times higher than poultry, and 108 times higher than mutton. As a result, the demand, supply, and price of pork have become key economic indicators for South Korea [2,3]. The South Korean government is essential in regulating and monitoring pork production and trade to maintain stable supply and prices [2,4,5,6]. A stable supply and demand for pork ensure fair market prices, benefiting the government, farmers, and consumers.

However, environmental challenges, human and livestock infectious diseases, geopolitical tensions, consumer preferences, and other factors can impact production and distribution, challenging the sustainability of the pork market. Forecasting pork supply in advance can help mitigate the effects of these sudden events. This paper aims to utilize machine learning algorithms to forecast future livestock values based on various associated factors, thereby contributing to the sustainability of the pork market.

Sustainability in the pork market includes economic, environmental, and social aspects. The economic aspect ensures stable market conditions, helping farmers earn a living and providing consumers with affordable products. Environmentally, sustainable pork production involves managing resources efficiently to minimize waste and reduce greenhouse gas emissions. Socially, it addresses food security and public health concerns related to pork consumption. By improving the forecast of pork supply, we can enhance the market’s resilience to disruptions, contributing to overall sustainability.

In recent years, machine learning algorithms have been used to forecast future livestock values based on various associated factors. These factors are classified as internal and external. Internal factors directly related to pork supply include feed costs, shipment volumes, breeding counts, and slaughter sizes. Several studies have focused on internal factors in forecasting livestock values. For instance, Zhang et al. [7] developed a forecasting method for the pig population based on factors related to pig breeding and slaughter. Gauthier et al. [8] forecasted daily feed intake using offline clustering and online forecasting methods based on lactating sows’ index. Zhang et al. [9] also forecasted pork supply based on pig population indexes such as new left gilts and breeding sows.

In contrast, external factors include a broader range of features like disease outbreaks, exchange rates, gasoline prices, and weather conditions. Studies have shown that weather [10], government policy [11], social trends [12,13], market indexes [14,15], and infectious diseases [16,17] greatly affect livestock forecasts. Previous livestock forecasting approaches have made significant improvements by considering either internal or external factors. However, focusing solely on one type of factor can lead to less reliable predictions due to the exclusion of other influential elements. Some approaches [18,19,20] integrate both internal and external factors to improve forecast accuracy. For instance, Song et al. [18] utilized a backpropagation neural network method to forecast the number of breeding sows based on pigs’ age transfer. The authors improved the reliability of forecasting methods by considering the influence of external factors such as epidemic diseases and relevant policies. Yu et al. [19] analyzed several internal and external factors, including piglet cost, consumer demand, and diseases, to forecast monthly pork prices. Despite these advancements, many studies focus only on a single factor like weather, disease, or oil, failing to fully show what affects the livestock market. Therefore, further research is necessary to understand the combined impact of internal and external factors on livestock analysis.

Many statistical and machine learning methods have been employed to forecast livestock indexes, including population [7], breeding cost [8], supply [9,12,21], and prices [11,22]. Despite their diversity, they have difficulties in accurate and robust forecasting, especially when many factors are involved. This makes it challenging for forecasting methods to recognize the pattern of each factor. To address the challenge, Yuan et al. [23] proposed a dynamic ensemble learning method for beef and lamb price forecasting. They combined the advantages of different member methods. Ensemble methods can achieve better performance and robustness in livestock forecasting [24,25], as they can better manage the complexity of the diverse factors influencing the livestock market. Furthermore, it is important to consider different time ranges in forecasting livestock indexes. Different time ranges, such as daily, weekly, and monthly, can show different trends and influences on livestock supplies.

This study investigates the effects of internal and external factors on forecasting pork supply using an ensemble machine learning method. We first analyze internal livestock data, examine data distribution, and explore features related to pork supply. We also collect external data, such as weather, gasoline prices, disease information, and exchange rates. These datasets are combined and preprocessed using various techniques. To address the complexity of forecasting, we develop a Stacking Forest Ensemble method (SFE-NET) that consists of three member methods and one meta-method. The three member methods are Random Forest (RFR), Gradient Boosting (GBR), and XGBoost (XGBR). The forecasting results of these member methods are combined with the meta-method neural network. Each member method has advantages: RFR reduces the risk of overfitting, GBR has robust predictive accuracy, and XGBoost handles large-scale data efficiently. As a meta-method, the NN learns from the combined outputs of the member methods because it can handle complex, non-linear relationships in the data. SFE-NET forecasts daily, weekly, and monthly pork supply. It uses data from 2016 to 2021 for training and data from 2022 for testing. Our experimental results show that the stacking ensemble method outperforms other state-of-the-art methods by 1% to 18% in terms of R² score. This improvement validates our method choice and highlights the importance of advanced machine learning techniques in solving real-world problems in livestock sustainability. More specifically, our contributions are as follows:

• We propose a methodology to forecast pork supply using both internal and external factors. Internal factors include breeding count, feeding cost, shipment cost, and production size, covering the entire process from breeding to market delivery. External factors include market price, weather, exchange rates, gasoline prices, and disease outbreaks. Unlike previous methods, we consider a broader range of factors to create a more comprehensive method for forecasting livestock indexes.
• We propose SFE-NET, a stacked forest ensemble neural network, for more accurate and robust pork supply forecasting. SFE-NET uses a two-layer training process. The first layer trains member methods, specifically Random Forest (RFR), Gradient Boosting (GBR), and XGBoost (XGBR), on the same data. The second layer trains a neural network on the output of these methods. SFE-NET combines the strengths of different methods and reduces their weaknesses, resulting in more reliable forecasts for both short and long periods.
• We conducted experiments to compare the effectiveness of the proposed method with various state-of-the-art and well-established methods. These experiments evaluated the impact of using only internal factors, both internal and external factors, and selected factors on forecasting accuracy. SFE-NET was tested on daily, weekly, and monthly forecasting scenarios, showing accuracies of 91%, 84%, and 88%, respectively. External factors improved forecasting accuracy by 2% to 12% compared to using only internal factors in terms of R². SFE-NET outperformed the comparative methods by 1% to 18% in R².

The rest of the paper is organized into the following sections: (2) Related Studies, (3) Materials and Methods, (4) Results, and (5) Conclusions.

2. Related Studies

This section describes the previous studies on livestock indexes (i.e., pork supply, pig population, prices, and import/export) and forecasting studies mentioned in the previous section. We categorize the studies into three subsections: (1) Methods based on Internal Factors, (2) Methods based on External Factors, and (3) Methods based on Internal and External Factors.

2.1. Methods Based on Internal Factors

Zhang et al. [7] employed a statistical method to calculate the monthly pig population using internal factors such as newborn piglets, sows, boars, and slaughtered pigs. Their proposed forecasting method includes four methods: (a) a recursive method for pig population, (b) an estimation method for new piglets and breeding sows, (c) an estimation method of the initial state for pig population, and (d) a related index method to calculate the pig population. The dataset used in the study spans from 2000 to 2015 in the Sichuan Province of China. The experimental results showed that the relative error of the pork supply prediction was approximately 0.06. Gauthier et al. [8] combined offline time-series clustering and online forecasting methods for daily feed intake. They first clustered the existing feed intake data to define groups of sows with the same feed intake. Subsequently, the online forecasting method learned from each group to provide more reliable forecasting results. Here, the dataset was collected from an automated feeder that recorded the feed intake of sows in six different farms. The experiments achieved high performance accuracy, with the RMSE at 1.06 kg/d. Zhang and Wang [9] predicted monthly pork supply using pig population indexes, such as slaughtered pigs, breeding sows, and left piglets. The authors developed two methods: a recursive method for pig population and an estimation method for pork supply. The pig population method considers new left gilts and breeding sows monthly. The pork supply method forecasts the pig population based on the results of the pig population method. This dataset was provided by the Heilongjiang Province, China, from 2016 to 2018. The experimental results showed high accuracies ranging from 94% to 98%.

2.2. Methods Based on External Factors

Kim and Choi [11] studied the economic impact of government policy on pork prices in South Korea. The authors implemented a Quasi-Experimental Hedonic Price approach to analyze pork prices before and after the implementation of government policy aimed at increasing the consumption of low-fat pork cuts. The authors collected pork cut data from the Rural Development Administration from January 2012 to November 2014 in Seoul, Gyeonggi-do, and Incheon, South Korea. Their experiments showed that, while high-fat pork prices were stable during the policy, low-fat pork prices increased. This result demonstrates that government policy can be important in maintaining sustainability in the livestock area. Ryu et al. [13] forecasted the purchase amount of pork using external factors such as broadcast news, TV programs/shows, and blogs. The factors consist of structured data (i.e., prices, sales, import/export, and others) and unstructured data (i.e., news frequency, emotions, comments, video frequency, and others). The dataset from 2010 to 2016 was trained on five machine learning methods, and the dataset from 2017 was tested. The authors compared popular statistical and machine learning methods such as ARIMA, Random Forest, Gradient Boosting, and Long Short-term Memory (LSTM). The experimental results found that external factors like social trends can express livestock production patterns and trends. Vu et al. [14] conducted an empirical study that examined the relationship between oil and agricultural prices. The authors proposed an interacted panel vector autoregressive framework to investigate the relationship between agricultural and oil prices. The dataset from 2000 to 2019 includes prices for ten agricultural commodities, oil prices, biofuel production, and exchange rates. The authors found that oil prices can influence agricultural prices through biofuel and exchange rates.

2.3. Methods Based on Internal and External Factors

Song et al. [18] used internal and external factors to develop a forecasting method for the number of breeding sows. Here, internal factors included the numbers of gilts, boars, and hogs, while external factors comprised the influence of epidemics and policies. The proposed method consists of three parts: (1) a recurrence method that learns internal factors, (2) the random disturbance term that considers external factors, and (3) an Improved Flower Pollination Algorithm-Back Propagation Neural Network (IFPA-BPNN) that trains and fits the number of breeding sows. For this study, Heilongjiang Province and Anhui Province of China provided the dataset from 2009 to 2021. The experiments with the proposed method showed superior accuracy compared to other forecasting methods. Yu et al. [19] used the Projection Pursuit Regression method for monthly pork price prediction using internal and external factors, such as consumer demand, diseases, logistics, and international environments. The dataset was collected in China from 2000 to 2023. The experimental results showed that the proposed method outperformed other comparative methods, such as support vector regression, error backpropagation neural network, and dynamic method average. The inclusion of internal and external factors enhanced the suitability, applicability, and reliability of the prediction methods. Li et al. [20] forecasted livestock product prices through internal and external factors, including feed costs, temperature, rainfall, and policies, collected from the Hebei Province, China. The dataset spanned from 2000 to 2021. The authors developed a novel forecasting method based on GRU neural networks and energy decomposition. The experimental results showed R² scores of more than 0.95 and proved that combining internal and external factors is helpful in livestock product price forecasting.

Unlike previous studies, we consider a more comprehensive range of factors from internal and external sources. Internal factors relate to pig breeding and the pork preparation process. External factors include weather, gas prices, exchange rates, and diseases. We also propose SFE-NET, a stacked ensemble method that can effectively forecast short and long periods of pork supply. Combining comprehensive factors and ensemble methods makes the forecasting results more accurate and reliable.

3. Materials and Methods

3.1. Overview

Figure 1 illustrates the overall flow of the study. This methodology includes data integration, data preprocessing, correlation analysis, method training, and evaluation steps. The datasets include two main factors: internal (i.e., pork-related features) and external (i.e., weather, exchange, and others). The datasets are combined based on the DateTime feature. Subsequently, the data preprocessing steps, including mitigating outliers, filling missing values, normalization, and feature selection, are applied to clean the combined dataset. Next, we analyze the correlations between pork supply and other features to identify the relationship between them. After preparing the experimental dataset, we train the forecasting methods using the data from 2016 to 2021. Finally, we evaluate the forecasting methods using the test dataset from 2022 based on well-known evaluation metrics such as Coefficient of Determination (R²), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Symmetric Mean Absolute Percentage Error (sMAPE). We describe each part in detail in the following subsections.

3.2. Dataset

The dataset used in the study comprises features of two main factors: internal and external. The internal factors include features directly from pork production processes such as supply, pig breeding, preparation of pork, market conditions, and pricing. We collected external factors from various open data sources, including weather conditions, exchange rates, gasoline prices, and livestock disease information.

Table 1. Details of the dataset.

Factor	Source	Name	Features	Samples	Date
Internal	EKAPE [26]	Supply	2	2060	1 January 2016–10 August 2022
		Breeding	11	1887
		Process	10	2414
		Price	5	1686
External	KMA [27]	Weather	5	2354
	INVESTING [28]	Exchange rate	4	1782
	OPINET [29]	Gasoline	5	2414
	DATAGO [30]	Disease	2	568
Total (Excluding DateTime)			35	2414

shows detailed information about the datasets. The dataset spans from January 2016 to August 2022, consisting of 35 (i.e., excluding the DateTime) features and 2414 samples. Data related to the internal factors were collected from the Korea Institute for Animal Products Quality Evaluation (EKAPE). Data related to the external factors were obtained from several open data sources: Korea Meteorological Administration (KMA), Korea National Oil Corporation (OPINET), Ministry of the Interior and Safety (DATAGO), and Investing.com. Based on these factors, we conducted three types of experiments: using only internal factors, using both internal and external factors, and using selected factors.

3.3. Correlation Analysis

Correlation analysis is a method used to discover the relationship between two features. We use the Pearson correlation coefficient, which measures the linear correlation between two features. The result of the Pearson correlation coefficient has a value between −1 to 1. Here, a value close to −1 represents a stronger negative correlation, a value close to 1 represents a stronger positive correlation, and 0 indicates no relation between features. Figure 2 demonstrates the correlation scores of internal features (Figure 2a,b) and external features (Figure 2c). Among the internal features, the preparing pork and market features have the highest correlation scores, ranging from 0.40 to 0.51. For the external features, holidays (−0.21), temperature (−0.20), gasoline prices (0.19), and exchange rate of CNY (0.11) show notable correlations with pork supply.

3.4. Data Preprocessing

3.4.1. Mitigating Outliers

The first data preprocessing step is mitigating outliers. The top row of Figure 3 shows examples of outliers in the dataset. More specifically:

• The mean daily pork supply is about 3500 tons. However, there are days when supplies drop to nearly 3 tons.
• The mean feeding cost is approximately KRW 368 million. However, there are a few cases where the cost is around KRW 110 billion.
• The mean market sales weight is about 940,000 kg. However, there are a few days when it exceeds 18 million kg.

These values are considered outliers as they deviate significantly from other values and negatively impact the forecasting results. To mitigate these outliers, we used the Winsorized Mean method, which replaces the outlier data with the closest normal data. The steps of Winsorized Mean are as follows:

• Sort the data points.
• Set the minimum ($i$) and maximum ($j$) percentages to replace.
• Replace the smallest $i$ values with the ($i+1$)th smallest value.
• Replace the largest $j$ values with the ($j+1$)th largest value.

We did not remove or mitigate all outliers, as some data may represent sudden changes rather than outliers. Therefore, we first checked all features using boxplot visualization and determined minimum and maximum thresholds for mitigation. The bottom row of Figure 3 shows the adjusted values after applying the Winsorized Mean method to mitigate outliers.

3.4.2. Filling Missing Values

The target feature (pork supply) has 2060 samples. However, other features have different numbers of samples, as shown in Table 1. This difference may result in null values when combining the datasets based on the DateTime feature. Figure 4 shows the number of null values per feature. Specifically, the dataset has 387 null values in pig breeding features and about 400 to 600 null values in the price features.

Figure 5 describes three simple ways to fill missing values: backward fill (BFILL), forward fill (FFILL), and interpolation. BFILL fills a null value using the next point, while FFILL fills the null value using the last point. On the other hand, interpolation finds a particular value between two given points on a line or curve. We use the interpolation method to fill in missing values in the dataset because it creates new points within the range of known data, providing a more accurate representation of the missing values.

3.4.3. Data Normalization

Table 2. Data description after mitigating outliers and filling in missing values.

Feature	Count	Zeros	Mean	Min	25%	75%	Max
Packaged Purchase Weight	2060	0	6,621,952.2	514.5	3,450,553.65	9,200,377.25	27,269,761.02
Packaged Purchase Count	2060	1	66,655.46	0.0	34,445.75	92,929.5	122,017.0
Packaged Shipment Weight	2060	0	4,526,624.91	13,185.46	2,503,470.91	6,231,381.2	11,654,205.66
Packaged Shipment Count	2060	0	126,730.31	1050.0	69,540.5	169,443.5	238,523.0
Sales Purchase Weight	2060	0	1,498,249.56	14,710.62	922,222.99	2,007,284.41	3,522,359.17
Sales Purchase Count	2060	0	17,158.64	620.0	11,853.75	22,043.25	35,995.0
Sales Shipment Weight	2060	0	906,030.35	1057.34	487,615.53	1,235,821.39	2,065,810.64
Sales Shipment Count	2060	0	19,950.11	59.0	13,119.75	26,520.75	46,251.0
Wholesale Price	2060	0	5322.05	2707.0	4350.5	6157.5	7528.0
Large Mart Price	2060	0	20,256.35	12,033.0	17,645.17	22,819.25	29,830.0
Supermarket Price	2060	0	21,432.71	18,100.0	19,806.0	23,230.0	26,140.0
Retail Price	2060	0	20,899.57	14,476.0	18,581.75	23,022.0	29,590.0

provides general information about the dataset after mitigating outliers and filling in missing values. After mitigating outliers and filling in missing values, the next issue is different ranges of features. For example, the retail price ranges from about 14,000 to 30,000, while the market sales weight ranges from 1057 to 20 million. The features with larger ranges may significantly impact the results of forecasting methods due to their larger values.

The top row of Figure 6 shows the original values of the retail price and market sales weight features. Due to the significant difference in data ranges, it is difficult to observe the fluctuation in the retail price. As mentioned, this difference can negatively impact the performance of machine learning methods, especially neural networks. To address this issue, we normalized the dataset using the Min–Max normalization method, which transforms all data points into the range [0, 1]. The bottom row of Figure 6 shows the values of the retail price and market sales weight features after data normalization.

3.5. SFE-NET: Stacked Forest Ensemble Neural Network

Figure 7 demonstrates an overview of the SFE-NET methodology for pork supply forecasting. Here, we divide the dataset based on years: 2016 to 2021 are used for training, and 2022 is used for testing. The training dataset contains 1895 samples, while the testing dataset contains 165 samples. Following this, a stacking ensemble method is designed to forecast the pork supply, consisting of two-level forecasting methods called Level-0 and Level-1. Level-0 trains several member methods, such as RFR, GBR, and XGBR. Level-1 trains a neural network using combined outputs from Level-0 methods. This two-level approach improves the performance and robustness using the combined outputs of two or more member methods. It is important to note that the methods used in SFE-NET are selected based on their strengths in handling the complex and noisy structure of data in the livestock market and their proven performance in similar forecasting tasks.

4. Results

4.1. Experimental Setup

This subsection describes the experimental setup, including the experiment environment, dataset, and evaluation metrics. We implemented the proposed method on a Windows 11 computer with the following specifications: an Intel(R) Core(TM) i9-9900K 3.60 GHz CPU, an NVIDIA GeForce RTX 2080 GPU, and 32 GB of memory. All methods and experiments were conducted using Python (version 3.8.10) and the following libraries:

• scikit-learn (version 1.1.3) for implementing RFR, GBR, and SFE-NET;
• xgboost (version 1.7.1) for implementing XGBR;
• lightgbm (version 4.4.0) for implementing LGBMR;
• gplearn (version 0.4.2) for implementing SYMR;
• scikit-elm (version 0.21) for implementing ELMR;
• scipy (version 1.9.3) for implementing statistical tests.

4.1.1. Dataset

We conducted seven experiments, both short- and long-term, using different forecasting types, features, and parameters. The experiments are divided into daily (Experiments 1–4), weekly (Experiments 5 and 6), and monthly (Experiment 7) forecasts. We also trained the methods using internal (Experiments 1 and 5), internal and external (Experiments 2 and 6), and selected factors (Experiments 3, 4, and 7). Additionally, we tested the effect of parameter tuning in Experiment 4. For all experiments, data from 2016 to 2021 were used for method training, while data from 2022 were used for method testing.

Table 3. Description of experimental data.

Name	Type	Features	Training	Testing
Experiment 1	Daily	Internal (22)	1895 (2016–2021)	165 (2022)
Experiment 2		Internal + External (35)
Experiment 3		Feature Selection (13)
Experiment 4		Parameter tuning (13)
Experiment 5	Weekly	Internal (22)	313 (2016–2021)	33 (2022)
Experiment 6		Internal + External (35)
Experiment 7	Monthly	Feature Selection (7)	72 (2016–2021)	10 (2022)

provides a detailed description of the types of experiments conducted.

For Experiments 3, 4, and 7, we selected the features based on correlation analysis (Figure 4) and feature importance. Figure 2 and Figure 8 show that the pork processing and delivering features have high scores in correlation analysis and feature importance. Weather, gasoline prices, and holidays also have higher correlations and feature importance scores. Although the CNY exchange rate showed high correlation scores, it did not improve the performance of the proposed method. As the results of the analysis, the following features were selected: process in quantity, process weight, process out quantity, market in quantity, market sales weight, market out quantity, lowest temperature, highest temperature, precipitation, diesel gas price, LPG gas price, livestock disease infections, and holidays.

4.1.2. Comparative Methods

This study proposes SFE-NET, a stacking ensemble of the RFR, GBR, and XGBR methods. To evaluate our results, we compare our proposed method with other well-established methods. Specifically, we compare our method with the individual member methods used in the proposed ensemble. This comparison shows that aggregating these member methods improves the performance of pork supply forecasting. Additionally, we compare our method with other well-established machine learning methods like LightGBM (LGBMR), Symbolic Regressor (SYMR) [31,32] and Extreme Learning Machine (ELMR) [33]. The comparative methods are described as follows:

• RFR combines multiple decision trees to perform robust and accurate predictions. It can handle high-dimensional data and provide feature importance insights.
• GBR is considered highly accurate for non-linear data with various characteristics. It is also effective for handling noisy data, including outliers.
• XGBR includes several regularization techniques to avoid overfitting, which makes it efficient for complex datasets.
• LGBMR is widely used in classification and regression due to its high efficiency and scalability in handling large-scale and complex datasets.
• SYMR searches the fixed forms of mathematical expressions to find the method for a given dataset. It tries to discover the method’s structure and parameters.
• ELMR is a neural network in which only the weights between the hidden and output layers are learned. It offers a fast and efficient training process compared to traditional neural networks.

4.1.3. Evaluation Metrics

This study uses four main evaluation metrics to assess the proposed method.

• The Coefficient of Determination (R²) measures the proportion of actual values that forecasted values can explain. We use the percentage scale of R² by multiplying the original scale (0 to 1) by 100. A value of 0 means the method explains nothing, while 100 means the method explains perfectly.
• Root Mean Squared Error (RMSE) is the average magnitude of prediction error using the same units as prediction. Lower RMSE values indicate better method performance.
• MAE stands for Mean Absolute Error, measures the average magnitude of the errors and is less sensitive to outliers compared to RMSE. Lower MAE values indicate better method performance.
• Symmetric Mean Absolute Percentage Error (sMAPE) measures the accuracy of time-series forecasting. Lower sMAPE values indicate better method performance.

These metrics were chosen to provide both absolute and relative error measures, allowing us to present a balanced view of method performance. All evaluation metrics were calculated using the following Equations (1)–(4). Here, $n$ is the number of samples, $i$ is the index of a sample, $y_i$ is an actual value, $\widehat{y}$ is a predicted value, and $\overline{y}$ is the mean value of the actual values.

$$R^2=\left(1-{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-\widehat{y_i}\right)}^2}{{{\sum }_{i=1}^n\left(y_i-\overline{y}\right)}^2}}\right)\times 100$$

(1)

$$RMSE=\sqrt{{\displaystyle \frac{{\sum }_{i=1}^n{\left(y_i-\widehat{y_i}\right)}^2}{n}}}$$

(2)

$$MAE={\displaystyle \frac{\sum _{i=1}^n|y_i-\widehat{y_i}|}{n}}$$

(3)

$$sMAPE={\displaystyle \frac{1}{n}}\sum _{i=1}^n{\displaystyle \frac{|y_i-\widehat{y_i}|}{\left(\right|y_i|+|\widehat{y_i}\left|\right)/2}}$$

(4)

4.2. Experimental Results

4.2.1. Results of Hyperparameter Tuning

Choosing optimal hyperparameters in machine learning methods improves performance and avoids overfitting or underfitting. Experiment 4 is conducted with a hyperparameter tuning procedure. We first create a few possible hyperparameter values per hyperparameter of each method. Then, we use the grid search cross-validation method to find the optimal values for hyperparameters by evaluating the methods using all possible combinations of hyperparameter values.

Table 4. Parameter selection results of hyperparameter tuning.

Method	Parameter	Options	Selected	Note
RFR	n_estimators	40,50,60,100	50	Number of trees in the forest
	max_depth	10,15,20,25	15	Maximum depth in each tree
	min_samples_leaf	1,2,3,4	1	Minimum number of samples to be at a leaf node
GBR	n_estimators	50,60,70,100	60	Number of trees in the forest
	max_depth	12,14,16,18	14	Maximum depth in each tree
	min_samples_leaf	1,2,3,4	4	Minimum number of samples to be at a leaf node
XGBR	n_estimators	150,180,200	180	Number of trees in the forest
	max_depth	2,4,6,8	6	Maximum depth in each tree
	learning_rate	0.01,0.03,0.05	0.03	Tuning parameter of optimization
LGBMR	n_estimators	100,200,500	500	Number of trees in the forest
	learning_rate	0.1,0.01,0.001	0.01	Tuning parameter of optimization
	max_depth	−1,10,20,30	−1	Maximum depth in each tree
SYMR	generations	5,10,20	5	The number of generations to evolve
	population_size	100,1000,1500	1000	The number of programs in each generation
	init_depth	2,4,6,8,10	2,6	The range of tree depths
ELMR	n_neurons	10,13,16,20	13	The number of hidden layer neurons
	density	0.1,0.3,0.5,0.7	0.3	The proportion of connections NN layer
	ufunc	relu,tanh,sigm	relu	Transformation function of hidden layer neurons
NN	hidden_layer_size	20,30,40,100	30	Number of hidden layers
	learning_rate_init	1 × 10−3,1 × 10−4,5 × 10−4	5 × 10−4	Tuning parameter of optimization

shows the selected hyperparameters of Experiment 4.

4.2.2. Results of Daily Pork Supply Forecasting

Figure 9 shows the R² and RMSE results of daily pork forecasting methods. Here, we evaluate and compare methods, such as RFR, GBR, XGBR, LGBMR, SYMR, ELMR, and the proposed SFE-NET methods. From the figure, we can observe that the proposed method outperformed all base methods in all experimental cases by about 1% to 8% in terms of R² scores. In Experiment 1, which used only internal features, the highest performances achieved were 81% for R² and 457.7 for RMSE. In Experiment 2, which used both internal and external factors, the highest performances achieved were 84% for R² and 416.3 for RMSE. In Experiment 3, we used correlation analysis and feature importance to select key features and applied them to forecasting methods. The experiment results demonstrate that methods achieve better results with fewer features. Finally, Experiment 4 used a cross-validation technique to choose the most efficient hyperparameters, resulting in the best accuracy and outperforming other experiments by about 4% to 13% in the R² score. It is also important to note that our proposed SFE-NET method outperformed all comparative methods because it combines the strengths of different methods and effectively captures the patterns in the dataset, resulting in more reliable forecasts.

Figure 10 shows the daily pork supply forecasting results of the proposed SFE-NET. In the figure, blue, yellow, green, red, and violet represent the actual pork supply from Experiment 1, Experiment 2, Experiment 3, and Experiment 4, respectively. The forecasting results of all daily experiments closely match the actual values. However, there are cases where the results from Experiments 1 and 2 deviate significantly from the actual values. On the other hand, fewer errors are observed in Experiments 3 and 4, meaning that selecting appropriate factors and parameters reduces higher fluctuations in forecasting results.

4.2.3. Results of Weekly Pork Supply Forecasting

Figure 11 shows the R² and RMSE results of weekly pork forecasting methods. For this experiment, we evaluated and compared methods such as RFR, GBR, XGBR, LGBMR, SYMR, ELMR, and the proposed SFE-NET methods. The proposed method outperformed all base methods in all experimental cases by about 2% to 9% in R² scores. In Experiment 6, which used all features (i.e., internal and external), the highest performances achieved were 84% for R² and 1455.0 for RMSE. Including external features increased the prediction method’s performance by around 12% in terms of R², as the methods were able to capture more essential patterns and details of the pork market.

Figure 12 shows the results of the SFE-NET method’s weekly pork supply forecasting. In the graph, the blue line represents the actual pork supply, the yellow line represents the results from Experiment 5, and the green line represents the results from Experiment 6. The graph shows that the results from Experiment 6 match the actual pork supply more closely than the results from Experiment 5. This indicates that considering both internal and external factors can more accurately represent the pork market.

4.2.4. Results of Monthly Pork Supply Forecasting

Figure 13 shows the R² and RMSE results of monthly pork forecasting methods. For this experiment, we evaluated and compared methods such as RFR, GBR, XGBR, LGBMR, SYMR, ELMR, and the proposed SFE-NET methods. The proposed method outperformed all base methods by about 1% to 18% in terms of R² scores. The highest performances achieved were 88% for R² and 2289.3 for RMSE. In monthly forecasting, we considered only selected factors due to the impact of internal and external factors, which has been proven in previous experiments.

4.2.5. Ablation Study on SFE-NET

In this subsection, we conducted an ablation experiment to assess the contribution of each component to SFE-NET. In this experiment, SFE-NET was built with 11 possible combinations of four individual member methods: RFR, GBR, XGBR, and LGBMR. We trained all combination methods using the dataset described in Experiment 4 and measured their performance using the R² score. Figure 14 shows the results of SFE-NET with the different combinations of member methods. In the figure, the X-axis represents the combinations, and the Y-axis represents the R² scores. Each combination is defined by the colors corresponding to the selected methods. The experimental results show that SFE-NET with three methods, including RFR, GBR, and XGBR, showed the highest performance. Based on the experiment, we selected these methods as the members of our proposed SFE-NET.

4.2.6. Comparison of All Results and Discussion

Table 5. Total results comparison of all experiments.

Method	Evaluation	Exp 1	Exp 2	Exp 3	Exp 4	Exp 5	Exp 6	Exp 7
RFR	R2	78.0	83.0	86.0	87.0	68.0	77.0	84.0
	RMSE	494.8	429.3	398.6	381.9	2044.1	1738.9	2674.5
	MAE	347.4	288.2	282.5	246.2	1609.8	1317.2	1568.7
	sMAPE	10.3	8.1	7.8	6.6	8.6	7.1	1.8
GBR	R2	75.0	79.0	85.0	88.0	70.0	72.0	77.0
	RMSE	525.5	485.0	407.2	360.6	2001.1	1939.2	3266.3
	MAE	377.4	349.4	287.7	255.3	1556.4	1457.9	2174.0
	sMAPE	11.2	10.3	8.7	6.7	8.1	7.7	2.5
XGBR	R2	78.0	80.0	87.0	88.0	63.0	72.0	87.0
	RMSE	489.0	467.2	385.1	360.7	2211.4	1928.7	2444.4
	MAE	343.0	325.0	276.5	251.7	1448.0	1475.3	1461.3
	sMAPE	9.6	9.2	7.5	6.8	7.8	7.8	1.6
LGBMR	R2	79.0	83.0	86.0	86.0	66.0	68.0	71.0
	RMSE	480.0	431.3	394.0	388.6	2119.9	2071.6	3654.0
	MAE	333.8	341.3	256.1	244.4	1565.2	1640.7	3465.8
	sMAPE	9.6	9.1	7.4	6.9	8.4	8.6	3.8
SYMR	R2	76.0	79.0	80.0	83.0	63.0	76.0	78.0
	RMSE	512.2	482.6	468.9	430.5	2203.7	1795.6	3152.9
	MAE	397.4	357.0	332.8	326.0	1848.5	1508.1	2213.0
	sMAPE	10.9	10.0	9.36	8.1	9.43	7.8	2.51
ELMR	R2	71.0	75.0	77.0	78.0	66.0	78.0	84.0
	RMSE	566.36	523.2	510.5	497.8	2108.9	1711.3	2710.6
	MAE	441.16	399.1	355.6	330.1	1756.0	1404.1	2149.4
	sMAPE	13.31	12.0	12.5	10.6	8.8	7.2	2.38
SFE-NET	R2	81.0	84.0	87.0	91.0	72.0	84.0	88.0
	RMSE	457.7	416.3	384.4	323.5	1924.3	1455.0	2289.3
	MAE	318.5	274.4	269.8	235.9	1390.8	1144.0	1494.3
	sMAPE	8.8	7.8	7.2	6.4	7.5	6.1	1.7

compares the results of all seven experiments, demonstrating the potential of the proposed method against comparative methods. The experiments include daily, weekly, and monthly forecasting with different sets of factors, such as internal only, internal and external, and selected. Specifically, Table 5 provides the following insights:

• External features can significantly improve pork supply forecasting performance (i.e., Experiments 2, 6).
• Carefully selected relevant features can improve the performance of pork supply forecasting (i.e., Experiments 3, 4, 7).
• Selecting optimal hyperparameters is an essential step to achieving good forecasting performance (Experiment 4).
• The proposed SFE-NET method outperforms all base methods in terms of accuracy metrics, R², RMSE, MAE, and sMAPE.
• The SFE-NET method provides accurate and robust short and long-term forecasting performance.

The results displayed in Table 5 show that the proposed SFE-NET outperformed the other comparative methods based on evaluation metrics. However, it is essential to determine whether these performance differences are statistically significant or due to random variation. Therefore, we utilized the Wilcoxon signed-rank test, a non-parametric test, to compare two related samples to assess whether their population mean ranks would differ. Figure 15 shows the heatmap of the $p \mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}\mathrm{s}$ from Wilcoxon signed-rank tests comparing the performance of SFE-NET against other comparative methods. We compared the evaluation results of the seven experiments; the X-axis represents the comparative methods, and the Y-axis represents the evaluation metrics. Each cell in the heatmap displays a p-value, which indicates the statistical difference between the methods. The green cells ($p \mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e} \le 0.05$) suggest that the performance metrics between the methods are statistically significant. All results in R² and RMSE showed significant differences, suggesting that SFE-NET differs from other methods. On the other hand, red cells ($p \mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}>0.05$) indicate that the performance metrics between the methods were not statistically different, suggesting similar effectiveness in these cases. By employing the Wilcoxon signed-rank test, we demonstrated that the observed differences in method performances, particularly in R² and RMSE metrics, were statistically significant and not due to random variations.

5. Conclusions

This study introduces SFE-NET, a method to forecast pork supply using the stacking ensemble method over short and long periods. The stacking ensemble method consists of three member methods, RFR, GBR and XGBR, combined by a neural network meta-method. The experimental results showed that SFE-NET consistently outperforms other comparative methods by 1–18% in terms of R². Moreover, incorporating external factors can further enhance the performance of forecasting methods. The proposed method achieved between 84% and 91% accuracy for daily, weekly, and monthly pork supply forecasting.

Our study demonstrates that both internal factors (e.g., number of pigs, feed costs, market prices) and external factors (e.g., weather conditions, livestock diseases, currency exchange rates, gas prices, and public holidays) significantly impact pork supply forecasting. Understanding the influence of these factors improves the accuracy and reliability of pork supply forecasting, which in turn helps to promote more sustainable pork production by reducing environmental and social impacts while supporting farmers and consumers.

The proposed method can be adapted to forecast the supply, demand, and prices of other types of meat. In the future, we plan to implement the proposed method in a web application in which users can select factors to forecast livestock indexes such as population, supply, and price. This tool can be beneficial for governments, farmers, and consumers by providing a means to assess and plan markets effectively.