Optimization of Forecasting Performance in the Retail Sector Using Artificial Intelligence ^†

Abstract

In the retail industry, demand forecasting is absolutely crucial for guaranteeing efficient inventory and supply chain control. Different artificial intelligence (AI) techniques have been used lately to improve forecasting performance. Demand fluctuation, seasonal patterns, and outside influences continue to create difficulties, though. Using several machine-learning techniques Linear Regression, XGBoost, Random Forest, Decision Tree, Prophet, and LSTM this paper offers a comparative study to forecast product demand. A retail dataset obtained from Kaggle served as the basis for training and testing the forecasting models. The experimental results demonstrate that the LSTM model outperforms all others with accuracy, precision, recall, and F1-score of 92.31%, 92.31%, 100.00%, and 96.00%, respectively, followed by Prophet with 85.71%, 92.31%, 92.31%, and 92.31%, respectively, Decision Tree with 93.05%, 75.76%, 76.13%, and 75.94%, respectively, Random Forest with 91.99%, 66.86%, 88.08%, and 76.02%, respectively, XGBoost with 83.21%, 45.70%, 87.84%, and 60.12%, respectively, and Linear Regression with 60.67%, 25.46%, 89.75%, and 39.67%, respectively. These results verify that ensemble and deep learning models can greatly help retailers in raising operational efficiency and notably improve forecasting accuracy.

1. Introduction

In recent years, the retail sector has seen substantial changes as a result of the integration of modern technology, particularly artificial intelligence (AI) [1]. Demand forecasting, a vital component of inventory and supply chain management, is increasingly relying on advanced predictive models. Traditional statistical methods frequently fail when dealing with complex, nonlinear patterns in sales data. As a result, machine learning and deep learning approaches have developed as effective methods for enhancing forecast accuracy.

This study investigates and compares the performance of various AI algorithms for anticipating retail demand, including Linear Regression, XGBoost, Random Forest, Decision Tree, Prophet, and Long Short-Term Memory (LSTM) networks. Using a publicly available dataset from Kaggle [2], we present empirical evidence on the strengths and limits of each technique, providing significant insights for shops looking to enhance operations.

Machine learning (ML) [3] and artificial intelligence (AI) technologies have emerged as viable tools for improving demand forecasting in recent years. These methods provide excellent predictive accuracy, pattern recognition capabilities, flexibility, and the ability to learn from previous data sets. This research focuses on intelligent forecasting systems that aim to increase operational efficiency and support decision-making in retail environments. The literature investigates many demand forecasting methods, ranging from simple, interpretable models like Decision Tree and Linear Regression to complex ensemble methods such as Random Forest and XGBoost. Recent models, such LSTM and Prophet, efficiently capture temporal interdependence. This study compares the performance of six models with a Kaggle retail dataset, utilizing metrics such as accuracy, precision, recall, and F1-score.

A novel contribution is the systematic comparison of several models using a unified experimental setup that includes both regression and binary classification viewpoints. Furthermore, computational efficiency training and inference time, as well as memory usage is examined, addressing practical factors that are sometimes disregarded in retail demand forecasting.

This paper is organized as follows: Section 2 reviews the relevant literature on AI-based forecasting techniques. Section 3 describes the research methodology, including data preprocessing and model implementation. Section 4 presents and discusses experimental results. Section 5 concludes the study and outlines future research directions.

2. Related Literature

Recent research has demonstrated the efficacy of Machine Learning and Deep Leraning in retail forecasting. Thivakaran and Ramesh [4] discovered that Random Forest outperformed XGBoost in predicting BigMart sales. Falatouri et al. [5] demonstrated that LSTM outperforms SARIMA for some fresh items. Rahman Mahin et al. [6] developed a voting regression that outperformed RF and KNN in accuracy. Riachy et al. [7] utilized COVID-19 constraints to improve Deep Learning forecast robustness. Darshan et al. [8] used pattern mining and Prophet to optimize inventory planning. Ref. [9] utilized EDA and ML to demonstrate that Decision Trees accurately forecasted financial earnings. Seyedan et al. [10] used LSTM, CNN, and Bayesian averaging to enhance inventory forecasting. Asma ul Husna. [11] used forecasting and fuzzy optimization to make supplier decisions.

Table 1. Summary of forecasting approaches in the literature (Part 1).

Author(s)	Year	Studied Variables	Research Methods	Findings	Limitations
Thivakaran and Ramesh	2022	BigMart sales	ANN, Random Forest, XGBoost	Random Forest outperforms XGBoost	Data limited to BigMart
Falatouri	2022	Fresh product sales	SARIMA, LSTM	LSTM better for potatoes and tomatoes	SARIMA better for cucumbers and lettuce
Rahman Mahin	2025	Sales data	RF, KNN, Voting Regressor	R2 = 0.9997	Complex hybrid model
Riachy	2025	Sales with COVID-19 restrictions	Deep Learning	Improved forecasting accuracy	Pandemic-specific data

and

Table 2. Summary of forecasting approaches in the literature (Part 2).

Author(s)	Year	Studied Variables	Research Methods	Findings	Limitations
Sri M Darshan	2024	Sales data	Association Rules, Prophet	Pattern mining + time-series	Method complexity
Potta Chakri	2023	Financial ratios, revenue	EDA, LR, KNN, SVR, Decision Tree	Decision Tree (depth=9) best	No time-series or risk analysis
Asma ul Husna	2025	Demand, costs, customs, delivery	RF, GBR, LSTM, ARIMA, Fuzzy logic	Supplier optimization model	No real-time test
Seyedan Mahya	2023	Retail demand, inventory	LSTM, CNN, Bayesian Averaging	Better inventory forecasting	No real-time adaptability

provide an overview of recent studies, highlighting their forecasting methods, key contributions, and identified limitations.

3. Materials and Methods

3.1. Proposed Methodology

This research attempts to provide a dependable and effective forecasting system for retail goods demand. Accurate projections aid in inventory optimization, stockout prevention, and supply chain decision-making. Six machine learning and deep learning models were examined using a structured pipeline in Figure 1 to determine the optimal technique.

• Data preparation: involves cleaning and preparing the input features.
• Feature engineering: to extract relevant temporal and product-related characteristics.
• Model training and evaluation: models are trained and evaluated using consistent criteria across all methods.
• Performance comparison: using standard classification and regression metrics.
• Final model selection: based on experimental results.

To ensure a comprehensive evaluation, the following models were chosen:

• Linear Regression: a conventional regression method utilized as a reference point.
• Decision Tree: a rule-based model known for interpretability.
• Random Forest: an ensemble of decision trees that improves generalization.
• XGBoost: a gradient boosting model known for accuracy and performance.
• Prophet: a time-series forecasting model designed to capture seasonality and trend.

Figure 1. Overview of the proposed forecasting pipeline.

Figure 1. Overview of the proposed forecasting pipeline.

All models were developed utilizing an identical dataset and input factors, which encompassed past demand figures, parts of the date (like month, day, and day of the week), and product identifiers. To properly learn temporal patterns, the LSTM model requires reorganizing the input into a sequential shape. To convert the regression task into a binary classification problem, a fixed classification threshold was utilized, defined as the target variable’s mean (Order_Demand = 5248.1).

3.2. Dataset Description

The dataset used in this study was obtained from Kaggle and contains historical retail sales records. The primary target variable is Order_Demand, which represents the number of product units requested. The dataset includes the following:

• Over 100,000 records;
• Key variables: product code, warehouse ID, date, and order demand;
• CSV format, with missing and inconsistent values.

3.3. Data Preprocessing and Binarization

The data preprocessing stage involved the following:

• Removing null or inconsistent entries;
• Transforming categorical variables through label encoding or one-hot encoding;
• Extracting time components such as day, month, and weekday from the date;
• Exploratory Data Analysis (EDA):The data reveals a clear rising trend, with a 30 day rolling average of 3,069,188 units and weekly seasonality averaging 2,733,319 units. Outliers were deleted to better model training.

To convert the regression issue into a binary classification challenge, a specific limit was established using the average value of Order_Demand (5248.1). According to this limit:

• Low Demand: instances where Order_Demand ≤ 5248.1
• High Demand: instances where Order_Demand > 5248.1

This binary transformation mirrors practical business concerns, where choices are frequently based on whether demand reaches a key threshold rather than exact quantities. It also facilitates model evaluation by providing simple measures such as accuracy and recall, which are easier to grasp for operational decision-making and allowed us to evaluate classification performance in addition to regression metrics.

3.4. Model Implementation

Each forecasting model was implemented using Python 3.12 and trained on the same dataset to ensure a fair and consistent comparison. All models were provided with an identical set of input features, which included historical demand values, date based attributes (such as day, month, and day of the week), and product identifiers.

Model-specific preprocessing and tuning: LSTM inputs were organized to capture temporal patterns and long-term relationships needed for time series forecasting. To ensure consistency, all models employed the same relevant input features, such as time-based signals and product identification. Feature uniformity and hyperparameter tuning: To ensure consistency, all models used the same input features, regardless of their internal architecture or learning mechanisms. Demand forecasting features such as time-based signals and product identifiers were routinely used. Grid search was used to tune hyperparameters such as tree depth, learning rate, and regularization in tunable models (e.g., XGBoost, Random Forest).

This exhaustive search method systematically evaluated different parameter combinations to identify configurations that achieved the best performance on validation data.

3.5. Experimental Setup

The experiments were carried out utilizing Google Colab (Python 3.12) and Jupyter Notebook 7.5.0, which allowed for rapid development, code execution, and interactive visualization. These platforms also allowed seamless integration with machine learning libraries and cloud-based resources.

To manage and monitor training, MLflow was used to track experiments, log performance metrics, and store models, assuring repeatability and ease of comparison between runs. Models were built utilizing existing libraries.

• Scikit-learn for preprocessing, evaluation metrics, and baseline models;
• XGBoost for gradient boosting;
• Prophet for additive time series forecasting;
• Keras/TensorFlow for implementing the LSTM model.

Common Training Settings:

• Number of epochs (LSTM): 50
• Batch size: 32
• Train/test split: 80% 20%

Additional performance study was undertaken to analyze training and inference times, as well as the models’ memory consumption:

-

XGBoost

• training time: 4.12 s
• Training memory use: 6.71 MB
• Prediction time: 0.09 s
• Peak memory usage during prediction: 1.68 MB.

-

LSTM

• training time: 20.60 s
• Maximum memory use during training: 4.68 MB
• Prediction time: 0.39 s

Evaluation metrics: Accuracy, Precision, Recall, F1-score, R² (coefficient of determination), RMSE (Root Mean Squared Error). These unified settings ensured that all models were evaluated under consistent conditions, enabling a fair and reliable comparison of their performance.

4. Results

This section presents and analyzes the forecasting results obtained from the various models applied to predict product demand in the retail sector. To assess classification performance, the continuous target variable Order_Demand was binarized using a fixed threshold equal to its mean (5248.1), distinguishing between two classes: Low Demand and High Demand. The performance metrics accuracy, precision, recall, F1-score, RMSE, R² summarized in

Table 3. Comparative performance of forecasting models.

Metric	Linear Regression	XGBoost	Random Forest	Decision Tree	LSTM	Prophet
Accuracy (%)	60.67	83.21	91.99	93.05	92.31	85.71
Precision (%)	25.46	45.70	66.86	75.76	92.31	92.31
Recall (%)	89.75	87.84	88.08	76.13	100.00	92.31
F1-score (%)	39.67	60.12	76.02	75.94	96.00	92.31
RMSE	28,803.57	22,469.02	21,527.80	25,317.23	-	-
R2 Score	0.0226	0.4052	0.4540	0.2449	-	-
Classes	High Demand, Low Demand

.

• Accuracy: Measures the model’s overall accuracy by calculating the proportion of true findings (including true positives and true negatives) across all examples evaluated.

  $$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$
• Precision (Pre): Indicates the percentage of accurately predicted positive observations among all predicted positives.

  $$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$
• Recall (Rec): Measures the proportion of true positives that were successfully identified.

  $$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}$$
• F1-Score: The harmonic mean of Precision and Recall achieves a balance between the two.

  $$F1=2\times {\displaystyle \frac{\mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$
• RMSE (Root Mean Square Error): Determines the average magnitude of prediction mistakes in regression tasks.

  $$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{(y_i-{\widehat{y}}_i)}^2}$$
• R² Score (Coefficient of Determination): Indicates how closely the forecasts match the actual facts.

  $$R^2=1-{\displaystyle \frac{{\sum }_{i=1}^n{(y_i-{\widehat{y}}_i)}^2}{{\sum }_{i=1}^n{(y_i-\overline{y})}^2}}$$

To supplement the quantitative performance study, this section provides visual representations of each model. We include the confusion matrices (Figure 2) and, which provide detailed insight into the correct and incorrect predictions. These visuals provide a better grasp of each model’s strengths and limits in categorizing demand as ’High Demand’ or ’Low Demand’.

5. Discussion

The overview show that LSTM and Prophet outperform traditional regression models. LSTM excels at modeling temporal dependencies, whereas Prophet produces accurate and interpretable forecasts with high F1 and recall scores. Among classical models, Decision Tree [12] and Random Forest [13] perform well, with Random Forest having the lowest RMSE. XGBoost produces balanced measures with reduced precision, while Linear Regression performs poorly. These findings emphasize the relevance of time-aware models in demand forecasting, with LSTM being the preferable method for dynamic inventory management.

6. Conclusions

This study sought to improve demand forecasting in the retail sector by utilizing a variety of artificial intelligence (AI) technologies. Using historical sales data, we conducted a thorough comparison of the performance of various forecasting models, including Linear Regression, Random Forest, XGBoost, and LSTM. The findings showed that sophisticated machine learning algorithms, particularly XGBoost and LSTM, outperformed traditional methods for predicting future demand. XGBoost displayed great stability and efficiency in processing structured data, whereas LSTM excelled at identifying temporal dependencies, making it ideal for time-series forecasting. These findings highlight the significant potential of AI-based models to improve demand prediction accuracy and, as a result, optimize inventory management and supply chain operations in the retail business. By synchronizing supply and demand, AI-driven forecasting enables retailers to make data-driven decisions, save money, and enhance customer happiness.

Automating demand forecasts eliminates stockouts and overstocking while optimizing resources and reducing waste. Future work could include conducting more exploratory data analysis to find trends and better feature engineering, as well as including external elements such as weather and marketing. Hybrid models that combine sequential and non-sequential methods may improve forecast accuracy and robustness.