This chapter presents the results of the research and their details. In the previous chapter, under the section on objectives, we outlined the three main areas of focus in this research: implementing multiple EOQ models within the framework, performing multi-objective optimization, and applying an ML-based surrogate model to reduce simulation costs. In this chapter, we first present the preparatory modeling steps, such as generating the time-series environment, initializing the various policies, and creating the simulation environment. We then examine the defined research questions in separate subsections, presenting the modeling steps and their results using mathematical modeling and data visualization tools.
3.1. Modeling Preparation
The first contribution of this research – and its first departure from classical EOQ models—is that we apply a stochastic, changing demand environment. This approach is essential for the framework to be applicable in realistic environments, and it is an important precondition for multi-objective optimization, as this is where inventory problems such as service level achievement and stockouts arise. The first step in modeling stochastic demand is to define the time horizon, since every seasonal demand simulation is based on a single horizon:
Stochastic demand consists of a deterministic component and stochastic noise. The deterministic seasonal component of demand is represented by a sinus function, whose parameters are the baseline demand (
b), the amplitude (
A), and the periodicity (
P), since all demand exhibits some form of repetition (e.g., weekly or monthly repetition):
We supplement this formula with stochastic noise, as real demand is not deterministic, which can be described by the following formula:
This noise appears at every time step and follows a normal distribution with a mean of 0 and a standard deviation of
. Once the two components have been determined, it is important to maximize their value, since demand cannot be zero. The final formula is:
After determining the time index (
t) and demand (
), various ordering policies can be created. These strategies are derived from classical EOQ models, which we implement within the established framework. In addition to these parameters, it is important to define the inventory level (
), which indicates how much inventory is available at time
t. A particularly important parameter during ordering is the lead time (
L), which indicates when the ordered material will arrive after an order is placed. The total inventory position consists of the current inventory level, inventory that has been ordered but not yet received (on order), and unfulfilled demand (backorder):
These parameters can be used to describe the EOQ policies applied in the research. In the (
s,
Q) policy, a fixed quantity
Q is ordered at a given time if the inventory level falls below a predetermined level
s:
In the (
s,
S) policy, an order of
is placed when the inventory level falls below the predetermined level
s, as in the previous case; however, the order quantity is not fixed, but rather the order is placed so that the inventory is replenished to the specified level
S:
The (
R,
S) policy operates differently from the previous two cases, as it is time-based instead of status-based. In this system, ordering takes place within a fixed
R ordering cycle, such that inventory is reordered to reach a predetermined
S level:
After initializing demand and policies, we can define the inventory simulator. This is a crucial step in setting up the framework, as the simulation environment determines the scope of industrial problems to which the system can be extended and applied. The most important parameters in the simulator are the inventory level (
), unfulfilled customer demand, the backorders (
), and the set of orders that have been placed but not yet received (
), where each element is of the form (
r,
q), representing the quantity
q arriving on day
r. Initial values of these parameters:
The first step of the simulator is to process the arrival of previously placed orders. An order arrives if it is found in the set
and its parameter
r matches the given day
t. At that point, the following inventory arrives:
After delivery, the inventory level is updated, and the received orders are removed from the
set:
Following the processing of the received items, it is necessary to update the “on order” (
) stock currently in transit, as this is required to determine the total inventory position:
The system makes the daily ordering decision based on the applicable policy after the inventory position is determined. To define the selected policy, we use the policy function (
), which describes the decision-making logic, and the parameter vector (
), which contains the specific parameter values. The ordering decision can be described by the following formula:
The result of the ordering decision is the order quantity (
) and the lead time (
). This differs from the classic EOQ model and is one of the contributions of this research, since the classic model assumes a fixed lead time, whereas it is essential for the realistic framework defined as the research objective that the arrival time be treated as a variable in the model. A precondition for the order is that delivery must occur within the simulation time horizon:
The order is valid if
. In this case, after the order is placed, it is added to the set
of ordered but not yet received items:
One of the objectives of the framework’s objective function is to minimize costs, so it is necessary to determine the ordering cost:
After the orders and their costs have been defined, the simulator performs demand fulfillment. The required quantity on a given day is composed of the backorders (
) remaining from previous days that have not been fulfilled and the demand for that day (
):
However, the required quantity cannot necessarily be fulfilled, since demand can be met only up to the quantity in stock (
). Accordingly, the quantity actually fulfilled is:
The next day’s opening inventory is calculated according to the previous formulas, to which the quantity received on the following day is added:
During system operation, on any given day, backorder requests are always fulfilled first, followed by the fulfillment of that day’s demand; however, there is an upper limit to this, so new backorders may arise from that day’s demand. An important parameter in the optimization process is the quantity fulfilled from that day’s demand, which can be expressed by the following formula:
To update the backorder, we need to determine the remaining, unfulfilled demand:
This results in the following day’s backorder inventory:
At the end of the inventory simulation, it is also necessary to determine its cost parameters. In our research, the total cost is the sum of ordering costs, holding costs, and shortage costs. The ordering cost has already been presented; the inventory holding cost is composed of the next-day inventory and the unit inventory holding cost (
h), while the shortage cost is derived from the next-day backorder inventory and the unit shortage cost parameter (
p):
The total simulation cost must be calculated over the entire time horizon:
Another important parameter of the framework is the fulfillment rate; that is, the proportion of total demand that is fulfilled. Total demand during the simulation:
Total quantity actually supplied to fulfil daily demand:
The performance rate required for optimization is the quotient of the two preceding parameters:
These formulas describe the entire simulation process as well as the parameters required for optimization. To support interpretability, we present the stochastic demand described earlier, along with the operation of the policy and the simulator, using an example. The figure below shows a 120-day demand time series based on an (
s,
Q) policy. The blue line represents the daily demand, which has a 14-day periodicity. Its value was determined randomly, resulting in the base level (
b), the amplitude (
A), and the stochastic noise (
). The green line represents the total inventory position (
IP), which consists of existing stock and stock awaiting arrival; the orange line represents the currently received and usable stock level. The green and orange lines largely overlap; they diverge when an order is placed due to the nature of the policy, at which point the on-hand stock and total quantity differ until the lead time (
L) days when the order arrives.
Figure 5 shows that in certain cases, the on-hand stock is smaller than the demand, resulting in a backorder. Based on these parameters, the simulation can calculate the total inventory cost as well as the fill rate.
3.2. Integration of Multiple Inventory Policies
When applying multiple policies, their parameters differ because their meanings and ranges are different. This situation can cause problems during optimization, as it results in a fixed-length vector, and the surrogate model can only operate in a numerical feature space. One of the goals of the developed framework is to be able to handle multiple different policies, which significantly contributes to the realistic application of the system, as strategic decisions in industrial practice also require the application of different policies. We implement the different inventory policies within the framework by introducing a unified representation for the purpose of common optimization.
The framework describes every possible solution in an encoded representation called a candidate. This consists of two parts: a policy identifier (
m) and a fixed-length gene vector (
g):
where the following applies to every gene:
We generate the initial population from various random candidates, which must be decoded. The unified representation values are important for the optimization algorithm and the ML model, while the decoded, real parameters are important for the simulation. Decoding can be formally described as the following mapping:
where
m is the candidate identifier,
g is the gene vector, and
e is the vector of environmental statistics. The purpose of this parameter is to allow certain parameters of the inventory simulator to be predefined for population specification in order to ensure realistic modeling. In the framework, we define the average demand and average lead time values:
The result of the decoding is the selected policy (
) and its parameters (
). The various policies can be listed in the following set:
When generating the population, we map the candidate ID to a valid policy ID using a modulus based on the size of the policy set:
This ensures that each candidate can always be assigned to one of the defined policies, while random sampling guarantees an even distribution across the policies.
Figure 6 illustrates the distribution of inventory policies decoded from randomly generated candidate solutions. The results show that the three policy types appear in approximately equal proportions. This confirms that the encoding and decoding processes are unbiased, and that a balanced policy space is created during the optimization process.
The parameters of a policy can be represented by gene vectors. A precondition for this is the definition of the parameters’ bounds. In a stock-based modeling system, it is essential to define the ranges of the various parameters, as this significantly reduces modeling costs and aids in the creation of realistic policies. The set of parameters for a policy can be modeled as follows:
These intervals depend on the given environmental statistics (
e). Therefore, their bounds can also be interpreted in terms of those statistics:
For example, in a given environment:
When representing genes as real parameters, these limits must be taken into account. This is done by rescaling the gene value (whose range is
to fall within the parameter limits, and finally rounding it to the nearest integer:
Parameters are assigned based on their index. Since the policy contains k parameters and the gene vector is n-dimensional, if there are more parameters than dimensions, the gene vector can be used in a circular manner. Otherwise, certain dimensions of the gene vector will not be used.
Finally, policy-specific cases must be handled, as certain policies require the application of logical constraints. A (s, S) policy is only valid if , since the order level is not greater than the post-order level; therefore, if this condition is not met, the system corrects it: . In the case of the (R, S) policy, both parameters must be positive. Thus, if this condition is not met after decoding, the system must also correct it: , .
The output of the decoding process is therefore a specific policy and its parameter vector, which can be used directly in the simulation; the result of this is the
f objective vector:
3.3. Multi-Objective Optimization
The second objective of this research is the application of multi-objective optimization. This differs from the general EOQ model, which aims only to achieve minimum cost. In industrial practice, however, numerous other factors are important for a manufacturing company, and the order of priority among these factors depends on the specific strategic decision. When creating a realistic framework, it is essential to consider multiple objectives and apply multi-objective optimization. In the course of the research, we identified three objective parameters, which have already been described in the Research Objectives chapter. In addition to minimizing costs—a factor also present in the classic EOQ model – we defined the maximization of the service level and the minimization of the average inventory level.
As a preparatory step for optimization, it is important to note that the inventory simulator generates parameters suitable for optimization. The following parameters must be specified for the simulation:
- -
Time horizon (T);
- -
Initial inventory level;
- -
Unit ordering cost (K);
- -
Unit holding cost (h);
- -
Unit shortage cost (p).
The specific parameters generally depend on the industrial partner and can be extended to encompass supply chain-wide optimization. However, this is not the objective of the present study. In addition to initializing the initial parameters, the simulator requires the selected policy and its parameters to be specified. After the preparatory processes, the simulation generates the target values used by the optimization. The goal of the optimization algorithm is to determine a result that meets the various criteria.
After completing the preparatory steps, we perform multi-objective optimization. To achieve this, we use the previously mentioned NSGA-II algorithm, which is one of the most widespread and frequently used multi-objective algorithms. In the research problem, we optimize a nonlinear, simulation-based stochastic system that does not require an analytical model, as the system is not closed-form and has no definable gradient. NSGA-II operates on a population-based principle, so it does not require continuity [
31]. Thus, it can handle discrete policies and integer parameters well, and global search is also feasible, since the algorithm does not get stuck in local optima due to the population. The model yields a Pareto front, which is a set of trade-offs containing solutions from which no single best solution can be chosen; rather, each element can be considered the best according to some criterion [
32].
The research problem can be interpreted as a multi-objective optimization problem, where the solution is
, where x represents the policy parameters:
Each solution corresponds to multiple objective function values, and the research problem involves minimizing these values simultaneously:
In this research, therefore, the objective of maximizing inventory levels must be reversed, since the optimization algorithm can handle only one type of optimization; thus, instead of this parameter, the objective is to minimize the fill rate. Consequently, the objective function can be written as follows:
Given two solutions, solution
x dominates solution y if solution x is at least as good as solution
y on every objective and better than solution y on at least one objective. This logic helps determine the hierarchy among the solutions:
The algorithm then sorts all individuals in the population into fronts using the principle of non-dominated sorting. A solution is considered non-dominated if there is no other solution that is better than it in every respect. These can be regarded as compromise solutions, since none is clearly better than the others, but each excels in a particular parameter. These solutions are placed in the first front:
The non-dominated solutions can be regarded as the current Pareto front. The second front consists of those individuals that are dominated only by elements of the first front, and this process is repeated. Based on the fronts, each individual is assigned a rank, and the lower the rank, the better the individual is considered to be:
If two solutions lie on the same front, the algorithm makes a decision based on the crowding distance function. This measures how sparsely or densely a given individual is distributed within the target function space. The algorithm favors the higher value because such individuals better preserve the diversity of the target function set:
The selection step can be carried out based on the previously determined rank and crowding distance values. This process is one of the key steps of the algorithm and models natural selection. For two individuals, the one with the lower rank is considered better, or, in the case of equal ranks, the one with the larger crowding distance value. This ensures that the algorithm strives simultaneously for convergence and the preservation of diversity. During selection, the algorithm selects two individuals from the population, and the one with the better value becomes the parent. It repeats this process twice, selecting two better solutions from four. The algorithm then creates new individuals, or offspring, which can also be called child individuals:
There are three main steps to creating a new individual: first, the child is generated from the parents’ parameters, which serves to pass on the traits of both parents:
Next, in the mutation step, the resulting child individual is slightly modified by a random shift (
), which must be normalized to the specified range. The purpose of this is to ensure that the new individuals possess their own characteristics and are not merely combinations of their parents:
Finally, the algorithm repeats these steps until the number of offspring equals the number of parents. The process of generating offspring ensures the creation of new solutions, their combinations, and their fine-tuning:
After the child population is generated, the algorithm combines it with the original parent population (
) to create the combined population:
The algorithm applies the specified ranking functions to the new population and repeats this process iteratively until the specified number of generations is reached, resulting in the formation of the Pareto front, which contains compromise solutions between the various objective functions.
After initializing the algorithm, we integrate it into the complete framework and perform the optimization on a randomized population. We examine the framework’s performance over a 365-day time horizon, for which we randomly generate a demand time series based on the parameters presented. The selection of this time horizon is justified by the fact that the model’s stable and balanced performance can be clearly observed over a 365-day period, whereas a shorter period does not fully reflect this and may lead to a biased performance evaluation. Furthermore, annual seasonality is commonly observed in many real-world inventory systems, especially in the procurement of raw materials, where demand is influenced by yearly cycles such as market trends, production planning, and external factors.
The optimization always runs on the same demand, so the different simulations are easily comparable. When defining the simulation environment, we specify the initial inventory and the unit costs, which are constant values, and from these, we determine the environmental statistics required for decoding (average inventory level and lead time). We then set the boundaries of the unified search space presented in the previous subsection. Its elements are the candidate identifier, from which the policy can be derived, and the candidate’s gene vector. By applying this, we ensure that every policy appears in the same unified search space; that is, the optimizer does not treat different policies separately but is able to represent every instance using the same structure. The gene values must be normalized to the range [0, 1]:
The step following decoding is simulation, which allows us to define objective functions for the decoded poly and its parameter values, with the goal of minimizing each of them. After this, all prerequisites are in place for the framework to perform the optimization. In the presented example, the NSGA-II algorithm is configured with a population size of 80 and 40 generations, resulting in a total of 80 + 40 ∗ 80 = 3280 simulation evaluations. During the selection of parameters, the main consideration was to efficiently explore the search space while maintaining computational efficiency. Increasing the population or generation size would have resulted in significant computational costs, while we observed that increasing the parameters did not lead to substantial changes in the structure of the resulting Pareto front, which confirms that the selected configuration provides reliable results.
The framework operates successfully; the simulation of the objective functions and the resulting multi-objective optimization were supported by realistic, randomized demand data. During testing of the framework, we initialized numerous parameters with realistic values, and the optimization results are also realistic, making them well-suited for application to real-world industrial data. The result of running the framework is the Pareto front, which can be seen in the figure below. For visualization, we converted the fill rate parameter back to its original, maximized form.
Figure 7 shows that none of the possible solutions for the (R, S) policy made it onto the Pareto front.
Based on the examined scenario, the absence of the (R, S) policy can be explained by differences in demand characteristics, cost structures, and operating methods. It is particularly important to note that the (R, S) policy differs significantly from the other two cases, as it is based on periodic analysis, which limits its responsiveness to short-term demand fluctuations in the case of seasonal demand. Consequently, based on the target parameters examined, the following policy is less favorable than the (s, Q) and (s, S) models.
3.4. Application of Surrogate Model
Based on the results presented above, the framework is well-structured and capable of effectively applying multiple policy strategies to a realistic demand time series. The main limitations of the framework are runtime and computational costs, as the NSGA-II algorithm involves many generations of iterations, and simulating inventory levels over these generations is a computationally intensive task. To support this, we defined the third objective of the research: the application of a surrogate model. A surrogate model is a computationally cheap approximation model that emulates the behavior of an expensive simulation model [
33]. In this research, we investigate the extent to which simulation costs can be reduced by using a surrogate model and to what extent this degrades the quality of the evaluation.
A surrogate model does not refer to a specific algorithm but rather to a concept aimed at reducing simulation costs and runtime using machine learning-based solutions. The specific solution is generated by a regression model. In this research, we use the Random Forest Regressor model to generate predictions that replace simulations. This algorithm is capable of handling nonlinear relationships well, performs well in noisy environments, and requires little preprocessing [
34]. These properties are key to model selection, as the model must handle the nonlinear effects arising from various policy parameters as well as the noisy demand time series.
The selection of the Random Forest regressor is motivated by its robustness and stability in stochastic environments in contrast to more complex models (e.g., neural networks). When choosing the model, these advantages were particularly important, since it is only a small part of the overall framework. Our goal was therefore to select a model that is simple yet highly effective.
To analyze the contribution of the surrogate model, we will examine a specific case. The simulation environment covers a 365-day time horizon. The real simulation and the surrogate model operate on the same seasonal demand pattern, making the results of the two runs easily comparable. We use the same values for the population and generation parameters as in the previous subsection, so the real, simulation-based result is based on a total of 3280 simulations. In addition, we initialize the initial inventory level, the specific order cost (
K), holding cost (
h), and shortage cost (p). To simplify the comparison process, we apply only the (
s,
Q) policy, for which we define the value bounds to produce the realistic results mentioned in the previous sections. The simulation results in the real Pareto front, for each element of which (
) the objective function vector is stored. We organize these into the following matrix:
We compare these values with the results of simulations supported by the surrogate model. The key point here is that not every candidate solution needs to be evaluated through simulation. Instead, the model uses an approximation algorithm to estimate the target values of certain individuals.
The first steps of the method involve creating an initial training set (300 candidates). The goal is to generate combinations of decision variables. Here, the target value vectors are determined through real simulations. Choosing the right size for the dataset is crucial, as too few samples will result in an inaccurate model, while too many samples render the approach ineffective, since its primary goal is to reduce the need for real simulations.
Next, we fit the previously described regression model to the initial training set, which learns the relationships between the input parameters and the target value vectors. As a result, this model can later be used to approximate the actual simulation. During the execution of the optimization algorithm, the majority of the population is evaluated not by the actual simulation, but by this regression model. During execution, dominance determination, non-dominated sorting, and selection are based on these estimates, thereby achieving faster execution times.
In order to achieve realistic results, the model evaluates a smaller set (20 candidates) in each generation using real simulations; 15 of these candidates are close to the Pareto front, allowing us to assign real simulation values to them, while the purpose of the remaining 5 candidates is to cover new areas in the search space and thereby avoid local optima. The actual evaluated samples are added to the training set, causing this set to grow continuously. We retrain the model iteratively, making it increasingly accurate. The result of the surrogate model-assisted optimization is also the Pareto front, whose elements (
) are included in the following set:
The sizes of the two sets reflect the extent to which the surrogate model contributed to the optimization:
Thus, by using the surrogate model, the simulation cost – runtime of the framework—was significantly reduced to one-sixth of the original. This represents a significant speedup, and as a result, the industrial applicability of the framework has increased considerably. However, it is important to examine the extent to which this acceleration is achieved at the expense of optimization results. First, we examine the extent to which the best elements of the three objective values have changed. The results are shown in
Table 2.
An analysis of each parameter individually shows that surrogate-assisted optimization did not achieve the same best result for cost minimization alone as it did for the other two parameters. This can be considered a particularly good result; however, more advanced methods can also be used for a constructive comparison of multi-objective optimization models. Hypervolume (HV) measures the volume of solutions defined by the Pareto front relative to a predefined reference point. Its value reflects the model’s convergence, i.e., the proximity of the solutions to the optimal point; diversity, which indicates the degree to which solutions are scattered along the front; as well as the coverage of the front. Thus, HV is a complex metric that effectively reflects the accuracy and distribution of solutions. When comparing the results of different methods, a higher HV value is considered to indicate better performance [
35].
The HV value of the model based on full simulation is 1.168, while that of the surrogate-assisted optimization model is 1.123. In terms of the HV parameter, the original model performs 4% better. To summarize the results of the surrogate model, the runtime of the accelerated model is nearly one-sixth that of the original, full simulation model, while its performance is only 4% better. This improvement in runtime is a drastic and significant result for the operation of the entire framework, while its performance has deteriorated only minimally. Based on these values, it can be concluded that the implementation of the surrogate model into the framework is successful and useful.
To evaluate the accuracy of the surrogate model, we computed standard regression error metrics, including Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) normalized RMSE (NRMSE), and the coefficient of determination (
R2), for each objective function. The results indicate that the surrogate model provides sufficiently accurate approximation across all objectives. The
R2 are consistently high (above 0.97), which demonstrates and excellent fit between predicted and true values. Although the prediction error is higher for the inventory level, this can be attributed to the increased variability of this objective in stochastic inventory systems. The reported metrics are computed on candidate solutions that were re-evaluated by the true simulator during the refinement phase of the surrogate-assisted optimization (
Table 3). In summary, the total error remains within an acceptable range for surrogate-based optimization, while the runtime is reduced to approximately one-sixth of the original.
Following the initial implementation and experimental results, we tested the presented model using various parameters to examine the variability arising from stochastic demand modeling and the random behavior of the algorithm. We performed a total of 25 runs, based on five different demand scenarios and seed combinations. Based on the results, the surrogate-based approach resulted in an average 3.1% reduction in HV compared to the standard NSGA-II algorithm, while the number of simulation evaluations decreased by an average of 5.39 times. The results of the more robust evaluation differ minimally from those of the initially implemented model, but their scale is the same, thereby further validating the research findings.
To evaluate the unified approach we developed, we used the same 25 runs employed for the robustness evaluation to compare it with a single policy-based benchmark, in which we optimized each inventory strategy separately using the NSGA-II algorithm, and then subsequently unified the resulting Pareto fronts and selected the best solution set. The results show that the average HV value of the developed unified framework (without surrogate assistance) is 1.134, while the average HV value of the benchmark single-policy method is 1.144. There is a total difference of only 0.9% between the two values, which is practically negligible. It is important to note that the single-policy method requires three different optimizations, resulting in significantly longer runtime despite minimal improvement. Since examining implementation time was among the research objectives, the developed framework is a significantly more favorable solution due to automatic policy selection, as it provides a more flexible and efficient solution in practical applications (
Table 4).
We conducted our research using a randomized, noise-contaminated demand time series, in which the initialization parameters were also randomly selected; thus, a potential avenue for further research is the extension of the framework to real-world industrial problems. On the one hand, this can provide feedback on the level of applicability resulting from the research contributions; on the other hand, practical cases can define new simulation tasks.